I am going to read 771 lines from a file consisting of an index number followed by text, and store each number in an array of variables.
I have no need to write anything to an output file.
BUT I can read the input file NEARLY TWICE AS FAST if I write each line to an output file
and EVEN FASTER if I refrain from writing each line BUT write a null output to an output file.
I discovered this peculiarity when reading a file and writing an output of the same text preceded by an incrementing line count,
and I wondered how much time was consumed by each element of code within the loop,
and was surprised that it took NEARLY TWICE AS LONG to read the file when I removed the element that created the output file.
This is so counter-intuitive.
WHY does writing an output file speed up the reading of an Input File ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
This is the code in question.
Code: Select all
:X1
SETLOCAL
%N_1%
<%1 (
for /L %%j in (!LO!,1,!HI!) do (
SET "LN=" & SET /P "LN="
ECHO %%j !LN!
)
) > %2
SET "T_Str=:CALLED %0 ; ECHO %%j LN to %1"
%DIF#%
EXIT/B
That code as shown takes 260 mSec to :-
read 771 lines from a 25 kByte file; and
write the same with an incrementing prefix to a 33 kByte file
:X2 Is the same code after removing the line ECHO %%j !LN!
and this is 60 mSec faster because there is no output so File %2 is forced to size zero.
No surprise there.
:X3 suffers the extra removal of > %2
and that strangely INCREASES the execution time to 430 mSec
Having created "Input.txt" I know that there are no empty lines,
and therefore see no reason to include any SET "LN="
:X4 to :X6 are 60 mSec FASTER variants of :X1 to :X3 in which I have removed
Code: Select all
SET "LN=" &
As a "practical example" of when an output file is NOT needed,
:Y5 and :Y6 simply read the input file and increment a variable when the line says "Have a Nice Day"
By using " > %2 " and setting Output.txt to size zero, :Y5 spends 150 mSec
By omitting " > %2 " and doing nothing to Output.txt, :Y6 spends 390 mSec
The mathematics suggests that when only one file is used,
Windows Ultimate allocates only one core from my quad core processor to my needs.
But when two files are used a state of emergency is declared,
and Windows Ultimate allocates ALL four cores from my quad core processor to my needs.
Please rescue my sanity and tell me what is happening
This is the start of Input.txt
Code: Select all
100001 [ *]
100003 [.NET Framework Logs*]
100004 [.NET Reflector*]
100002 [.Thumbnails*]
100005 [32bit Web Browser*]
This is the start of Output.txt
Code: Select all
100001 100001 [ *]
100002 100003 [.NET Framework Logs*]
100003 100004 [.NET Reflector*]
100004 100002 [.Thumbnails*]
100005 100005 [32bit Web Browser*]
This is what I captured from a DOS window.
Press any key to continue . . .
E:\T\CCleaner\New\SRx2\#TRIMMER>#
FILE #.bat CREATED 19:06 22/02/2012 Launched 21:02:02.24
LO=100001 ; HI=100771
21:02:02.54 - 21:02:02.29 = 250 mSec :CALLED :X1 ; ECHO %j LN to Output.txt
21:02:02.80 - 21:02:02.55 = 250 mSec :CALLED :X1 ; ECHO %j LN to Output.txt
21:02:03.00 - 21:02:02.81 = 190 mSec :CALLED :X2 ; No output, zero size Output.txt
21:02:03.20 - 21:02:03.01 = 190 mSec :CALLED :X2 ; No output, zero size Output.txt
21:02:03.63 - 21:02:03.20 = 430 mSec :CALLED :X3 ; No Output, No Output File
21:02:04.06 - 21:02:03.64 = 420 mSec :CALLED :X3 ; No Output, No Output File
21:02:04.26 - 21:02:04.07 = 190 mSec :CALLED :X4 ; No "LN=", ECHO %j LN to Output.txt
21:02:04.46 - 21:02:04.27 = 190 mSec :CALLED :X4 ; No "LN=", ECHO %j LN to Output.txt
21:02:04.61 - 21:02:04.47 = 140 mSec :CALLED :X5 ; No "LN=", No output, zero size Output.txt
21:02:04.75 - 21:02:04.61 = 140 mSec :CALLED :X5 ; No "LN=", No output, zero size Output.txt
21:02:05.12 - 21:02:04.75 = 370 mSec :CALLED :X6 ; No "LN=", No Output, No Output File
21:02:05.49 - 21:02:05.13 = 360 mSec :CALLED :X6 ; No "LN=", No Output, No Output File
21:02:05.65 - 21:02:05.50 = 150 mSec :CALLED :Y5 ; "NICE = 0", zero size Output.txt
21:02:05.80 - 21:02:05.66 = 140 mSec :CALLED :Y5 ; "NICE = 0", zero size Output.txt
21:02:06.20 - 21:02:05.81 = 390 mSec :CALLED :Y6 ; "NICE = 0", No Output File
21:02:06.59 - 21:02:06.20 = 390 mSec :CALLED :Y6 ; "NICE = 0", No Output File
21:02:06.79 - 21:02:06.60 = 190 mSec :CALLED :X4 ; No "LN=", ECHO %j LN to Output.txt
21:02:06.80 - 21:02:02.24 = 4560 mSec Duration of Above Tests
Press any key to continue . . .
E:\T\CCleaner\New\SRx2\#TRIMMER>
This is my script under the name #.bat
Code: Select all
@echo off
SET "T_Str=In_Line Measurement"
SET "N_1=SET "T_1=!TIME!" & FOR /F "tokens=1-4 delims=:." %%d in ("!T_1!") do (SET /A V_1=%%d*360000+1%%e*6000+1%%f*100+1%%g-610100)"
SET "N_2=SET "T_2=!TIME!" & FOR /F "tokens=1-4 delims=:." %%d in ("!T_2!") do (SET /A V_2=%%d*360000+1%%e*6000+1%%f*100+1%%g-610100)"
SET "DIF=SET /A DELAY=!V_2!-!V_1! & ECHO !T_2! - !T_1! = !DELAY!0 mSec !T_Str!"
SET "DIF#=%N_2% & %DIF%"
setlocal EnableDelayedExpansion
ECHO FILE %~nx0 CREATED 19:06 22/02/2012 Launched %TIME%
%N_1%
SETLOCAL
SET "T_Str=Within SETLOCAL In_Line Measurement"
SET LO=100001
for /f %%a in ('find /c /v "" ^< Input.txt') do SET /A HI=%LO%-1+%%a
ECHO LO=!LO! ; HI=!HI!
CALL :X1 Input.txt Output.txt
CALL :X1 Input.txt Output.txt
CALL :X2 Input.txt Output.txt
CALL :X2 Input.txt Output.txt
CALL :X3 Input.txt
CALL :X3 Input.txt
CALL :X4 Input.txt Output.txt
CALL :X4 Input.txt Output.txt
CALL :X5 Input.txt Output.txt
CALL :X5 Input.txt Output.txt
CALL :X6 Input.txt
CALL :X6 Input.txt
SET NICE=0
CALL :Y5 Input.txt Output.txt
CALL :Y5 Input.txt Output.txt
CALL :Y6 Input.txt
CALL :Y6 Input.txt
CALL :X4 Input.txt Output.txt
SET "T_Str=Duration of Above Tests"
%DIF#%
PAUSE
EXIT /B
:X1
SETLOCAL
%N_1%
<%1 (
for /L %%j in (!LO!,1,!HI!) do (
SET "LN=" & SET /P "LN="
ECHO %%j !LN!
)
) > %2
SET "T_Str=:CALLED %0 ; ECHO %%j LN to %2"
%DIF#%
EXIT/B
:X2
SETLOCAL
%N_1%
<%1 (
for /L %%j in (!LO!,1,!HI!) do (
SET "LN=" & SET /P "LN="
)
) > %2
SET "T_Str=:CALLED %0 ; No output, zero size %2"
%DIF#%
EXIT/B
:X3
SETLOCAL
%N_1%
<%1 (
for /L %%j in (!LO!,1,!HI!) do (
SET "LN=" & SET /P "LN="
)
)
SET "T_Str=:CALLED %0 ; No Output, No Output File"
%DIF#%
EXIT/B
:X4
SETLOCAL
%N_1%
<%1 (
for /L %%j in (!LO!,1,!HI!) do (
SET /P "LN="
ECHO %%j !LN!
)
) > %2
SET "T_Str=:CALLED %0 ; No "LN=", ECHO %%j LN to %2"
%DIF#%
EXIT/B
:X5
SETLOCAL
%N_1%
<%1 (
for /L %%j in (!LO!,1,!HI!) do (
SET /P "LN="
)
) > %2
SET "T_Str=:CALLED %0 ; No "LN=", No output, zero size %2"
%DIF#%
EXIT/B
:X6
SETLOCAL
%N_1%
<%1 (
for /L %%j in (!LO!,1,!HI!) do (
SET /P "LN="
)
)
SET "T_Str=:CALLED %0 ; No "LN=", No Output, No Output File"
%DIF#%
EXIT/B
:Y5
SETLOCAL
%N_1%
<%1 (
for /L %%j in (!LO!,1,!HI!) do (
SET /P "LN="
IF "!LN!"=="Have a nice day" SET /A NICE+=1
)
) > %2
SET "T_Str=:CALLED %0 ; "NICE = !NICE!", zero size %2"
%DIF#%
EXIT/B
:Y6
SETLOCAL
%N_1%
<%1 (
for /L %%j in (!LO!,1,!HI!) do (
SET /P "LN="
IF "!LN!"=="Have a nice day" SET /A NICE+=1
)
)
SET "T_Str=:CALLED %0 ; "NICE = !NICE!", No Output File"
%DIF#%
EXIT/B
Regards
Alan
