Entwickler-Ecke
Dateizugriff - Automatische Buffer Anordnung
RamiroCruzo - Sa 16.07.16 08:24
Titel: Automatische Buffer Anordnung
Greetings Masters,
I was trying to Multithread an app which uses stdin & stdout. But, I'm constantly having problems with output writing as the streams need to be written in correct order in order to make the CRC proper. I tried "Critical Sections", they limit one thread thread at a time but not the order.
Without the MT part, normal code looks like this:
Delphi-Quelltext
1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20:
| for I := 1 to SL1.Count do begin if Length(SL1[I - 1]) > 12 then begin ms1 := TMemoryStream.Create; ms1.CopyFrom(mystream1, StrToInt64('$' + DecodeRefStr(SL1[(I - 1)])[1])); ms2 := TMemoryStream.Create; ms2.CopyFrom(mystream1, StrToInt64('$' + DecodeRefStr(SL1[(I - 1)])[2])); Output := TMemoryStream.Create; processstream(ms1,ms2,Output); mystream2.CopyFrom(Output, 0); ms1.Free; ms2.Free; Output.Free; end else mystream2.CopyFrom(mystream1, StrToInt64('$' + SL1[(I - 1)])); end; |
SO, is there anyway that we can define the position while copying like we do in arrays so that we can define the position of buffer in integers not offsets?
jaenicke - Sa 16.07.16 10:37
If the size of the data is identical, you could calculate the position and set it before you write to the output stream.
If this is not the case, I would give the threads an ascending number to know which result should be written next. Then I would simply give the results combined with the numbers of the results to an output class, which writes the next result to be written as it gets it and stores all other results until the missing results have arrived.
You did not write which version of Delphi you are using. In newer versions I would use a TDictionary to store the results until all parts between have been written.
So in pseudocode this would be:
Delphi-Quelltext
1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21: 22: 23: 24: 25: 26: 27: 28: 29: 30: 31: 32:
| Results: TObjectDictionary<Integer, TStream>;
procedure ExecuteThread(Number, Data); begin WorkWithData; HandleResult(Number, WorkResult); end;
procedure HandleResult(Number, WorkResult); var CurrentResult; begin TMonitor.Enter(Results); try if NextNumber = Number then begin WriteResult(WorkResult); Inc(NextNumber); while Results.TryGetValue(NextNumber, CurrentResult) do begin WriteResult(CurrentResult); Results.Remove(NextNumber); Inc(NextNumber); end; end else Results.Add(Number, WorkResult); finally TMonitor.Exit(Results); end; end; |
If this wasn't detailed enough, I can of course explain more detailed. :)
TMonitor is more lightweight than a full critical section. If you have a large number of chunks and threads, I would recommend to separate storing the results and writing them to the output. So you only lock Results, write the results to it and unlock again. Another thread could check periodically for results (where of course it has to enter the lock too) and write them to the output. This way the periods, in which one thread is inside the lock, are not this long as the actual writing to the output takes place outside.
RamiroCruzo - So 17.07.16 20:22
Thanks for the reply :D I've both XE8 & 10.1 Berlin...
Well, TObjectDictionary is totally new to me, can ya please explain how it works?
Also, I'm using a thread declared as TRestoreThread with parameters of I, InputStream, OutputStream. I use Semaphore to control the number of threads.
Entwickler-Ecke.de based on phpBB
Copyright 2002 - 2011 by Tino Teuber, Copyright 2011 - 2024 by Christian Stelzmann Alle Rechte vorbehalten.
Alle Beiträge stammen von dritten Personen und dürfen geltendes Recht nicht verletzen.
Entwickler-Ecke und die zugehörigen Webseiten distanzieren sich ausdrücklich von Fremdinhalten jeglicher Art!