Variable expansion inside :: comments?

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Post Reply
Message
Author
Sponge Belly
Posts: 216
Joined: 01 Oct 2012 13:32
Location: Ireland
Contact:

Variable expansion inside :: comments?

#1 Post by Sponge Belly » 25 Aug 2017 07:56

Hello All! :)

I was trying to catch errors from overflowing variables (is it possible, btw?) when I noticed the following:

Code: Select all

@echo off & setlocal enableextensions disabledelayedexpansion

:: create variable containing 4k hashes
set "hashFile=%tmp%\hashFile.tmp"
if not exist "%hashFile%" (
    <nul set /p "=#" >"%hashFile%"
    for /l %%I in (1 1 12) do type "%hashFile%" >>"%hashFile%"
)
for /f "usebackq delims=" %%A in ("%hashFile%") do set "hashStr=%%A"

:: next line parses ok
:: %hashStr:#=$%
echo(works ok

:: next line throws 2 errors
:: %hashStr:#=$$%
echo(never reached

endlocal & goto :EOF


It seems there is some form of variable expansion going on inside :: comments after all! :shock:

- SB

pieh-ejdsch
Posts: 239
Joined: 04 Mar 2014 11:14
Location: germany

Re: Variable expansion inside :: comments?

#2 Post by pieh-ejdsch » 25 Aug 2017 08:36

Hello,
the content of this variable has Parenthesis!?
Please note that these lines are read in completely. %Variable% and Parameter %1 ... %9 %* as well as special characters are treated as on a line to be executed (see Jebs declaration). The alone is already the reason to write these commentary lines carefully. These :: comment lines can be written as multilines, meaning that the (unmasked) caret at the end of the line, the following line is considered as continuation of the comment. Thus, several successive comment lines without a double point are possible.

https://stackoverflow.com/questions/43410288/batch-setting-rem-as-variable-to-make-code-look-different
use rem instead

edit
you may use

Code: Select all

if . == ^%var:~8192,1%. (echo var OK) else echo var bad

\edit
Phil

dbenham
Expert
Posts: 2461
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Variable expansion inside :: comments?

#3 Post by dbenham » 25 Aug 2017 09:43

@Sponge Belly - Absolutely :!: :twisted:

The same thing will happen if you put the expansion within a REM (instead of ::).

The REM fatal error is predicted by the cmd.exe batch parsing rules documented by jeb, because variable expansion in phase 1 occurs before REM detection in phase 2.

I believe label detection also occurs early in phase 2 (the bit that prevents labels from executing, not the CALL or GOTO search for the label). But as of yet, that is missing from jeb's post.

I've documented two other expansion fatal errors in my % expansion rules, both of which can occur within REM and label comments.

All of the following will result in a fatal error:

Code: Select all

:: %~ Missing digit following the tilde gives fatal error
REM %~ Missing digit following the tilde gives fatal error

set DefinedVar=1
:: %DefinedVar:=  Missing find string before = gives fatal error
:: %DefinedVar:*=  Missing find string before = gives fatal error
REM %DefinedVar:=  Missing find string before = gives fatal error
REM %DefinedVar:*=  Missing find string before = gives fatal error

The only mechanism for safely embedding an unrestricted comment without fear of a batch fatal error is to put the comment somewhere where it can never be executed. For example, after EXIT /B, or use GOTO to bypass the comment.


Dave Benham

aGerman
Expert
Posts: 4654
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Variable expansion inside :: comments?

#4 Post by aGerman » 25 Aug 2017 15:00

dbenham wrote:or use GOTO to bypass the comment.

Where that means you don't even need to bypass the whole line. It's sufficient to have it in the same line along with the label.

Code: Select all

goto :comment
:comment %hashStr:#=$$%
echo(works, too

Finding a label using GOTO and CALL only sees the first token.

Steffen

dbenham
Expert
Posts: 2461
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Variable expansion inside :: comments?

#5 Post by dbenham » 25 Aug 2017 15:28

Nice clarification. Thanks :)

Sponge Belly
Posts: 216
Joined: 01 Oct 2012 13:32
Location: Ireland
Contact:

Re: Variable expansion inside :: comments?

#6 Post by Sponge Belly » 05 Sep 2017 09:22

@Dave

Thanks for the refresher! :)

@Phil

Something like:

Code: Select all

set "var=%var:#=$$%"


could potentially double a variable in size. And if the variable is over 4095 characters to begin with, it will overflow and cause a fatal error.

Is there any way to catch such errors and take remedial action?

- SB

pieh-ejdsch
Posts: 239
Joined: 04 Mar 2014 11:14
Location: germany

Re: Variable expansion inside :: comments?

#7 Post by pieh-ejdsch » 12 Sep 2017 09:56

Hello,
I have a bit in my old things around and found something.
The FOR loop goes only up to ca. 8190 characters + "" and then goes out.
The census of the foreseeable length is also clear.

Code: Select all

@echo off
setlocal enableextensions disabledelayedexpansion
call :setAllMacros
:: create variable containing 4k hashes
set "hashFile=%tmp%\hashFile.tmp"
del %hashfile%
if not exist "%hashFile%" (
    <nul set /p "=#" >"%hashFile%"
    for /l %%I in (1 1 12) do type "%hashFile%" >>"%hashFile%"
)
for /f "usebackq delims=" %%A in ("%hashFile%") do set "hashStr=%%A"

:: next line parses ok
:: %hashStr:#=$%
echo(works ok

 rem one method
 %checkString(var):var=hashStr%
if NOT errorlevel 1 echo Lenght is OK

echo  # ^> $$
 %checkString(var):var=hashStr:#=$$%
if     errorlevel 1 echo can not be changed from # to $$

 rem other method
setlocal enabledelayedexpansion
%strLen(var):var=!hashStr!%
set /a "hashStrLength = len"

%strLen(var):var=!hashStr:#=!%
set /a reducedLength = len
set /a increasedLength = (hashStrLength -reducedLength) *2
set /a newLength = increasedLength +reducedLength

echo HashString Length = %hashStrLength%
echo    reduced Length = %reducedLength%
echo  increased Length = %increasedLength%
echo        new Length = %newLength%
if %newLength% gtr 8189 echo is to long from # to $$
pause

exit /b

:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:setAllMacros
:: define LF as a Line Feed (newline) character
set ^"LF=^

^" Above empty line is required - do not remove

:: define a newline with line continuation
set ^"\n=^^^%LF%%LF%^%LF%%LF%^^"

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:strLen.var
@for %%T in ("%temp%\%~n0.tmp.cmd") do @(
 @ >%%T (
  echo(@set strLen(var^)=(%%\n%%
  echo( set "str=Avar"%%\n%%
  echo( set "len=0"%%\n%%
  for /l %%i in (12 -1 0) do @(
   echo( set /a "len|=1<<%%i"%%\n%%
   echo( for %%%%# in (!len!^) do if .!str:~%%%%#^^^^^^,1!==. set /a "len&=~1<<%%i"%%\n%%
  )
  echo(^)
 )
 call %%T
 del %%T
)

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
@set checkString(var)=(%\n%
 setlocal enabledelayedexpansion%\n%
 ( for /f tokens^^^=*delims^^^=^^^ eol^^^= %%V in (%\n%
  "!var!") do @%\n%
 ) ^|^| call%\n%
 endlocal%\n%
 if     errorlevel 1 echo string is BAD!%\n%
 if NOT errorlevel 1 echo string is ok%\n%
)
set "LF="
set "\n="
exit /b

Phil
Last edited by pieh-ejdsch on 13 Sep 2017 14:07, edited 1 time in total.

jeb
Expert
Posts: 1041
Joined: 30 Aug 2007 08:05
Location: Germany, Bochum

Re: Variable expansion inside :: comments?

#8 Post by jeb » 13 Sep 2017 07:02

Sponge Belly wrote:Is there any way to catch such errors and take remedial action?


I see no way to catch the error with batch itself, as this kind of syntax errors stops the batch file immediately, even the setlocal stack remains unchanged.
This can enable delayed expansion for the command line!

To catch it you can use a subprocess with cmd /c, FOR /F seems not to work here, as it stops but I can't see how to fetch the error

Code: Select all

@echo off
setlocal EnableDelayedExpansion
rem if "%~1" NEQ "" exit /b
if "%~1" == ":longFail" (
    call %~1 %~2
    exit /b
)

call :test 12
call :test 1
exit /b

:test
echo(
echo test %1 %~f0
(
    cmd /c %~f0 :longFail %1
) || echo ###### CATCH FAILURE ######
echo ERRORLEVEL=!ERRORLEVEL!
echo END
exit /b

:longFail
set "long=-"
echo %~0 started %1

for /L %%n in (%1 1 13) do set "long=!long:~0,4000!!long:~0,4000!"

echo %long%%long%
 
echo %~0 END
exit /b

Aacini
Expert
Posts: 1885
Joined: 06 Dec 2011 22:15
Location: México City, México
Contact:

Re: Variable expansion inside :: comments?

#9 Post by Aacini » 13 Sep 2017 09:59

jeb wrote:
Sponge Belly wrote:Is there any way to catch such errors and take remedial action?


I see no way to catch the error with batch itself, as this kind of syntax errors stops the batch file immediately, even the setlocal stack remains unchanged.
This can enable delayed expansion for the command line!


I tried to make good use of this behavior and wrote this program, intended to leave the Delayed Expansion status Enabled at the command line:

Code: Select all

@echo off
setlocal EnableDelayedExpansion
rem %~ Missing digit...

However, it don't works... I also tested all possible errors described in this thread, but none of them leave Delayed Expansion on... :( Windows 8.1 here.

Antonio

jeb
Expert
Posts: 1041
Joined: 30 Aug 2007 08:05
Location: Germany, Bochum

Re: Variable expansion inside :: comments?

#10 Post by jeb » 13 Sep 2017 11:18

Aacini wrote:However, it don't works... I also tested all possible errors described in this thread, but none of them leave Delayed Expansion on... Windows 8.1 here.


I retested it with Win7 and found that this only works when a call is involved.

Code: Select all

@echo off
setlocal EnableDelayedExpansion
call :func

:func
rem %~ Missing digit...


And this enables directly the possibility to supress the error message

Code: Select all

call :func 2> nul

Aacini
Expert
Posts: 1885
Joined: 06 Dec 2011 22:15
Location: México City, México
Contact:

Re: Variable expansion inside :: comments?

#11 Post by Aacini » 13 Sep 2017 11:44

Ok. This works (I called it EDE.bat):

Code: Select all

@echo off
setlocal EnableDelayedExpansion
call :func 2> nul

:func
rem %~ Missing digit...

Thanks, jeb!

Antonio

Post Reply