Faster batch macros

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Message
Author
jeb
Expert
Posts: 1042
Joined: 30 Aug 2007 08:05
Location: Germany, Bochum

Re: Faster batch macros

#16 Post by jeb » 19 Jan 2024 07:09

aGerman wrote:
17 Jan 2024 11:17
Uhh that's something that can drive you mad. You need to glue it to the next command because it would be treated as a separate command otherwise. Even an empty string or a space is parsed like that. I can't make my brain remember the order and details of script parsing.
The parser splits in phase 2 the tokens and the first token is the command token, even if it evaluates to an empty string later.
Therefore the metavariable %%? has to be pasted with the command.

The command token is parsed independent of the argument tokens, this can be important when delayed expansion is enabled.

See the difference

Code: Select all

setlocal EnableDelayedExpansion
for /F "delims=" %%? in ("echo=") do (
  %%?^^caret1! ^^caret2 ^^caret3
  %%?^^caret1 ^^caret2 ^^caret3!
  %%? ^^caret1 ^^caret2 ^^caret3!
)
Output:

Code: Select all

caret1 ^caret2 ^caret3
^caret1 caret2 caret3
 caret1 caret2 caret3
jeb

jfl
Posts: 226
Joined: 26 Oct 2012 06:40
Location: Saint Hilaire du Touvet, France
Contact:

Re: Faster batch macros

#17 Post by jfl » 19 Jan 2024 11:42

Thanks @jeb, it's very clear now.

aGerman
Expert
Posts: 4654
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Faster batch macros

#18 Post by aGerman » 28 Jan 2024 08:39

I found some time to dig a little deeper ...
As much as I agree with Arthur Clarke, things are often not that obvious. Especially in highly abstracted languages where you don't know how much complexity is hidden in the implementation of a simple command such like REM, where we already know that it isn't a NOP. I remember jeb explaining that REM can't ever be a NOP as long as it must evaluate at least the next token (the reason why REM /? prints the help message), and with ECHO turned ON you'll even see variables being expanded etc. So, shorten the batch code may or may not lead to a better performance as it is heavily influenced by the performance of the implementation behind the scenes. We have no better choice than testing ...

Code: Select all

@echo off

for /f %%! in ("! ^! ^^^!") do ^
set timediff=for %%# in (1 2) do if %%#==2 for /f "tokens=2" %%$ in ("%%!%%! 1 0") do ((if 1==%%$ setlocal EnableDelayedExpansion)^&for /f "tokens=1-3" %%- in ("%%!_i_%%!") do (set "_t1_=%%!%%~-: =0%%!"^&set "_t2_=%%!%%~.: =0%%!"^&^
%=% set /a "_d_=(8640000+(((1%%!_t2_:~,2%%!*60+1%%!_t2_:~3,2%%!)*60+1%%!_t2_:~6,2%%!-366100)*100+1%%!_t2_:~-2%%!-100)-(((1%%!_t1_:~,2%%!*60+1%%!_t1_:~3,2%%!)*60+1%%!_t1_:~6,2%%!-366100)*100+1%%!_t1_:~-2%%!-100))%%8640000,_o_=100000000+(_d_%%100),_d_/=100,_o_+=(_d_%%60)*100,_d_/=60,_o_+=(_d_%%60)*10000+_d_/60*1000000"^&^
%=% set "_o_=%%!_o_:~1,2%%!:%%!_o_:~3,2%%!:%%!_o_:~5,2%%!.%%!_o_:~-2%%!"^&for /f %%' in ("%%!_o_%%!") do ((if 1==%%$ endlocal)^&if "%%~/"=="" (echo %%') else set "%%~/=%%'"))) else set _i_=

(set \n=^^^
%= these lines create an escaped line feed, do not alter =%
)

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

setlocal DisableDelayedExpansion
echo        delayed expansion
echo      disabled  ^|   enabled
echo    ------------+------------

:: uses FOR as condition to call SETLOCAL/ENDLOCAL, close to jeb's original approach
:: with SETLOCAL in the ELSE branch, FOR is apparently slower than IF for this purpose (see 4 and 5)
<nul set /p "=1. "
for /f %%! in ("! ^! ^^^!") do ^
set ThisMacro=for /f "tokens=2" %%E in ("%%!%%! D """) do for %%# in (1 2) do if %%#==2 (%\n%
  rem %%!Args_ThisMacro:~1%%!%\n%
  for %%E in (%%~E) do endlocal%\n%
) else (for %%E in (%%~E) do setlocal EnableDelayedExpansion)^&set Args_ThisMacro=
call :check

:: having SETLOCAL/ENDLOCAL expanded from the tokens is quick, but the REM commands
:: decelerate the execution if delayed expansion is already enabled
<nul set /p "=2. "
for /f %%! in ("! ^! ^^^!") do ^
set ThisMacro=for %%# in (1 2) do if %%#==2 for /f "tokens=2-4" %%1 in ("%%!%%! setlocal rem endlocal rem") do (%\n%
  %%1 EnableDelayedExpansion%\n%
  rem %%!Args_ThisMacro:~1%%!%\n%
  %%3%\n%
) else set Args_ThisMacro=
call :check

:: even empty tokens prefixing SETLOCAL and ENDLOCAL seem to reduce the performance,
:: REM does it anyway
:: you may still like this in terms of readability, however it isn't my first choice
<nul set /p "=3. "
for /f %%! in ("! ^! ^^^!") do ^
set ThisMacro=for %%# in (1 2) do if %%#==2 for /f "tokens=2" %%? in ("%%!%%! "" rem=") do (%\n%
  %%~?setlocal EnableDelayedExpansion%\n%
  rem %%!Args_ThisMacro:~1%%!%\n%
  %%~?endlocal%\n%
) else set Args_ThisMacro=
call :check

:: using IF is a little slower than the direct expansion of tokens (as in 2), however
:: the IF condition is the fastest option when delayed expansion is already enabled
<nul set /p "=4. "
for /f %%! in ("! ^! ^^^!") do ^
set ThisMacro=for %%# in (1 2) do if %%#==2 for /f "tokens=2" %%1 in ("%%!%%! 1 0") do (%\n%
  if 1==%%1 setlocal EnableDelayedExpansion%\n%
  rem %%!Args_ThisMacro:~1%%!%\n%
  if 1==%%1 endlocal%\n%
) else set Args_ThisMacro=
call :check

:: just like 4, but with SETLOCAL in the ELSE branch as in 1
:: 4 and 5 perform the same in terms of speed
<nul set /p "=5. "
for /f %%! in ("! ^! ^^^!") do ^
set ThisMacro=for /f "tokens=2" %%1 in ("%%!%%! 1 0") do for %%# in (1 2) do if %%#==2 (%\n%
  rem %%!Args_ThisMacro:~1%%!%\n%
  if 1==%%1 endlocal%\n%
) else (if 1==%%1 setlocal EnableDelayedExpansion)^&set Args_ThisMacro=
call :check

pause
goto :eof

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:check
set "t1=%time%"
for /l %%i in (0 1 20000) do %ThisMacro% X
set "t2=%time%"
if "!!"=="" (
  %timediff% t1 t2
  goto :eof
)
%timediff% t1 t2 diff
<nul set /p "=%diff% | "
setlocal EnableDelayedExpansion
call :check
endlocal
goto :eof
Typical result:

Code: Select all

       delayed expansion
     disabled  |   enabled
   ------------+------------
1. 00:00:11.73 | 00:00:02.74
2. 00:00:09.83 | 00:00:04.25
3. 00:00:10.51 | 00:00:05.09
4. 00:00:10.17 | 00:00:02.23
5. 00:00:10.14 | 00:00:02.24
See code comments for more information.

I'm inclined to prefer 5 even if 4 makes it a little clearer that we can no longer rely on pre-existing variables being guarded by SETLOCAL and ENDLOCAL in the macro code. However, 5 might be still better for people who are fully aware of this issue. They can use it carelessly in an environment where delayed expansion is disabled, or place the call in a subenvironment with delayed expansion enabled, where the macro code is executed repeatedly. Users need to be careful not to conflict variable names then, which applies to all of these design patterns anyway.

In every test macro I only have a REM line representing the code dedicated to a specific purpose. REM is used to strengthen the influence of the design pattern that wraps it. In the real world, though, the importance of the design could be greatly overshadowed by the time the dedicated code takes.

Steffen

Post Reply