Variable nesting, odd behavior maybe useful. Update: not new, already handled at DosTips

[Maylow]

I wanted a routine for expanding nested variables no matter how many levels deep.
I was fiddling with dynamically constructing FOR statements for each nested level to expand nested variables when i stumbled upon some interesting behavior, and an error but that was expected and easily resolved.
Below is the code to reproduce the results.

Code: Select all

REM Case study: "variable refences variables references variable..."
set "test1=a value 123"
set "test2=!test1! 456"
set "test3=!test2! 789"
set "test4=!test3! 10"
set "test5=!test4! 11"
set "test6=!test5! 12"
set "test7=!test6! 13"
set "test8=!test7! 14"
set "test9=!test8! 15"
echo(Normal echo: %test9%
setlocal enableDelayedExpansion
echo(Expanded echo: %test9%
call:ResolveNested "%test9%"
endlocal
echo(End
exit /b

:ResolveNested 
for /f "tokens=*" %%b in ("%1") do (
    echo(Label FOR echo: %%~b
)
exit /b

This produces:
Normal echo: !test8! 15
Expanded echo: !test7! 14 15
Label FOR echo: !test5! 12 13 14 15
End
All expected.

When adjusting routine ResolveNested:

Code: Select all

:ResolveNested 
for /f "tokens=*" %%b in ("%1") do (
    echo(Resolving variables...
    for /f "tokens=*" %%c in ("%b") do (
        
    )
    echo(Label FOR echo: %%~b
)
exit /b

Notice there is no code after the second FOR statement which obviously states an error:
Normal echo: !test8! 15
Expanded echo: !test7! 14 15
) was unexpected at this time.
Also, totally expected.

Now comes the odd behavior, just some code like a simple REM statement in the FOR block:

Code: Select all

:ResolveNested 
for /f "tokens=*" %%b in ("%1") do (
    echo(Resolving variables...
    for /f "tokens=*" %%c in ("%b") do (
        REM
    )
    echo(Label FOR echo: %%~b
)
exit /b

and running it again IN THE SAME CONSOLE WINDOW gives the following results:
Normal echo: a value 123 456 789 10 11 12 13 14 15
Expanded echo: a value 123 456 789 10 11 12 13 14 15
Resolving variables...
Label FOR echo: a value 123 456 789 10 11 12 13 14 15
End
All variables are completely resolved! Didn't expect that.
It seems, when stumbling upon such error, the command parser resolves all variables before exit.
Though the FOR statement without inner code must be placed inside a routine for this behavior to work as far as i know for now.

I wonder about the mechanism behind this and if this can be used somehow.

I haven't come accross this in the forum so i thought to mention it.
Currently investigating the caveats and possibilities of implementation.

Edit: just a note to emphasize that the error has to occur to reproduce the results as far as i know for now.
Edit: some typos.

[Maylow]

Smaller code sample to reproduce effect:

Code: Select all

@echo off
REM Case study: "variable refences variables references variable..."
set "test1=a value 123"
set "test2=!test1! 456"
set "test3=!test2! 789"
set "test4=!test3! 10"
set "test5=!test4! 11"
set "test6=!test5! 12"
set "test7=!test6! 13"
set "test8=!test7! 14"
set "test9=!test8! 15"
echo(%test9%
setlocal enableDelayedExpansion
call:ResolveNested "%test9%"
endlocal
echo(End
exit /b

:ResolveNested 
for /f "tokens=*" %%b in ("%1") do (
    
)
exit /b

Run this in console window, results:
!test8! 15
) was unexpected at this time.

Leave console window open and adjust code:

Code: Select all

:ResolveNested 
for /f "tokens=*" %%b in ("%1") do (
    REM
)
exit /b

Run this again in same console window, results:
a value 123 456 789 10 11 12 13 14 15
End

Edit: some typos

[Maylow]

Even simpler code to reproduce it:

Code: Select all

@echo off
REM Case study: "variable refences variable references variable..."
set "test1=a value 123"
set "test2=!test1! 456"
set "test3=!test2! 789"
set "test4=!test3! 10"
set "test5=!test4! 11"
set "test6=!test5! 12"
set "test7=!test6! 13"
set "test8=!test7! 14"
set "test9=!test8! 15"
echo(%test9%
setlocal enableDelayedExpansion
call:ResolveNested "%test9%"
endlocal
echo(End
exit /b

:ResolveNested 
for /f "tokens=*" %%b in (" ") do (
    
)
exit /b

After running for first time, leave console window open and adjust code to:

Code: Select all

@echo off
REM Case study: "variable refences variable references variable..."
set "test1=a value 123"
set "test2=!test1! 456"
set "test3=!test2! 789"
set "test4=!test3! 10"
set "test5=!test4! 11"
set "test6=!test5! 12"
set "test7=!test6! 13"
set "test8=!test7! 14"
set "test9=!test8! 15"
echo(%test9%
echo(End
exit /b

and run again in same console window.
To point out that the routine only needs to be called once with the error to have all variables resolved.

Edit: some typos.
Edit: notice the code adjustment here, the SET commands are required to make it work and they make use of the exclamation mark '!', it won't work without these.
However, there is no longer 'setlocal enableDelayedExpansion' after the adjustment...odd.

[Maylow]

If somehow cmd.exe can continue after ") is unexpected at this time" errors, then this behavior could be used and save a lot of code.

[sst]

This is a known behavior, which was first described by Dave in SETLOCAL continues after batch termination! about 11 years ago. Yes it's extremely hard to find even when one knows about it.

So basically, because of the syntax error, your setlocal enableDelayedExpansion remains in effect after the batch file termination which will propagate to the command line context(This is actually a memory leak), the next time you run your batch file it will start with delayed expansion enabled and so the initialization of the test variables will get resolved even when you have not explicitly enabled the delayed expansion

Yes it have at least one use case, because about a year ago I've developed a batch app to dynamically report and/or manipulate the current state of Extensions/DelayedExpansion at command line context. Something like setlocal but for command line context. But didn't bother to release it because of some necessary cleanups and I'm too lazy for that. Anyway your post reminded of that work and maybe I share a clean version if get some free time and overcome my laziness. Or maybe someone else do it before me. Either way it's OK as I think it would be a very useful and time saving feature for batch file developers/testers.

[Aacini]

Basically, this is a method to terminate a Batch file without closing the child environment, so theoretically it would allow to "export" the variables of a called Batch-file to the calling environment. However, the method used to achieve this behavior is prone to cause errors. Besides, there are better proven methods to do the same...

On the other side, your examples are not really about "nested" variables. One simple and useful example of nested variables could be this one:

Code: Select all

setlocal
set "format=!header[%%i]! = !var[%%i][1]! !var[%%i][2]! !var[%%i][3]!"

setlocal EnableDelayedExpansion
. . .
for /L %%i in (1,1,%n%) do echo %format%

If you want to see what you can perform expanding "nested variables" many levels deep, then you may review this crazy topic...

Antonio

[Maylow]

Aahh thanks for the explanation clearing things up! I will look into those topics you mentioned for further information.

Very interested in the use-case you mentioned sst.

True these are not really nested variables, wanted a simple example to reproduce the behavior I encountered.

Thanks again a lot sst and Aacini!

Edit: changed the topic so others are aware of it.