ReturnVar macro revisited

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Message
Author
jeb
Expert
Posts: 1041
Joined: 30 Aug 2007 08:05
Location: Germany, Bochum

Re: ReturnVar macro revisited

#16 Post by jeb » 04 Nov 2013 06:15

Hi carlsomo,

I was very supressed, that you solved my main problem, to return linefeed AND Carriage return to a disabled context without a helper function :D

But after examing the code I didn't understand why it should work

Code: Select all

if defined _#_$_dis for /f "delims=" %%A in (!%%N!) do @endlocal ^& @set "%%~N=%%A"%\n%

This main line seems to work for you, but my expectations said that this is impossible, as a FOR/F can't tranfser a line feed.

After searching a bit, I found the bug. The macro never returns to a disbaled context :(
You used the clever idea to add the **_#_$_dis** variable, but the for loop never executes anything, as **!%%N!** is always empty :!:

%%N contains in the tests "result" with the quotes, so
!%%N! ->> !"result"! ->> <empty>

I change the test banch a bit, so this types of bugs will be found sooner.

Code: Select all

:TestReturnVar.bat
@echo off
cls
setlocal DisableDelayedExpansion
if not defined ReturnVar call :ReturnVar
set /a test_cnt=0
set "test1=^a!"^^b"
set "test2=caret^"
set "test3="
set "test4====Equals"
@rem        &"&^&!!!^^%^!
set  test5=^&"&^&!!!^^%%^!
setlocal enabledelayedexpansion
set "test6=Hello &^^ "^^^^  ^&" world^!!CR!*!LF!X"
@echo off&echo(

for /L %%n in (1 1 6) do (
   call :Test_Both test%%n
)
exit /b

:Test_Both
setlocal EnableDelayedExpansion
echo Test now                    %1='!%1!'
endlocal
call :Test_context Disable Disable %1
call :Test_context Disable Enable  %1
call :Test_context Enable  Disable %1
call :Test_context Enable  Enable  %1
echo(
exit /b

:Test_context
set /a test_cnt+=1
setlocal %1DelayedExpansion
set "result=not set by macro"
call :startTest %1 %2 %3

REM Output the result
setlocal enabledelayedexpansion
if "!result!"=="!%~3!" (
   set "postfix=OK"
) ELSE (
    set postfix=FAIL
)
set "format1=  Test #!test_cnt! "
set "format2=%1/%2  "
echo !format1:~0,10! !format2:~0,15! result='!result!'    - !postfix! !level!
endlocal

endlocal
exit /b

:startTest
setlocal %2DelayedExpansion
%ReturnVar% result %3 1
exit /b


Now the output looks like
output wrote:Test now test1='^a!"^b'
Test #1 Disable/Disable result='not set by macro' - FAIL
Test #2 Disable/Enable result='not set by macro' - FAIL
Test #3 Enable/Disable result='^a!"^b' - OK
Test #4 Enable/Enable result='^a!"^b' - OK

Test now test2='caret^'
Test #5 Disable/Disable result='not set by macro' - FAIL
Test #6 Disable/Enable result='not set by macro' - FAIL
Test #7 Enable/Disable result='caret^' - OK
Test #8 Enable/Enable result='caret^' - OK

Test now test3=''
Test #9 Disable/Disable result='not set by macro' - FAIL
Test #10 Disable/Enable result='not set by macro' - FAIL
Test #11 Enable/Disable result='' - OK
Test #12 Enable/Enable result='' - OK

Test now test4='===Equals'
Test #13 Disable/Disable result='not set by macro' - FAIL
Test #14 Disable/Enable result='not set by macro' - FAIL
Test #15 Enable/Disable result='===Equals' - OK
Test #16 Enable/Enable result='===Equals' - OK

Test now test5='&"&^&!!!^^%^!'
Test #17 Disable/Disable result='not set by macro' - FAIL
Test #18 Disable/Enable result='not set by macro' - FAIL
Test #19 Enable/Disable result='&"&^&!!!^^%^!' - OK
Test #20 Enable/Enable result='&"&^&!!!^^%^!' - OK

*est now test6='Hello &^ "^ &" world!
X'
Test #21 Disable/Disable result='not set by macro' - FAIL
Test #22 Disable/Enable result='not set by macro' - FAIL
* Test #23 Enable/Disable result='Hello &^ "^ &" world!
X' - OK
* Test #24 Enable/Enable result='Hello &^ "^ &" world!
X' - OK


Currently I suppose to found a solution, that can return to a disabled context each character, even linefeed and carriage return.
But not both at the same time :wink:
Either linefeeds or carriage returns are allowed in a string.

I will post it when it works.

But perhaps you will find a better solution for the problem :idea:

carlsomo
Posts: 91
Joined: 02 Oct 2012 17:21

Re: ReturnVar macro revisited

#17 Post by carlsomo » 05 Nov 2013 02:14

This version handles everything correctly with the exception of the linefeed in the disabled context - Only 2 FAILS :oops:

This segment:

Code: Select all

        if defined _#_$_dis (%\n%
          for /f delims^^=^^ eol^^=  %%A in (""!%%~N!"") do @(%\n%
            if defined _#_$_dis (endlocal ^& set "%%~N=%%~A" ^& set "_#_$_dis=") else (%\n%
              echo %%~A is missing in this string%\n%
            )%\n%
          )%\n%
        )%\n%

processes the first iteration of %%~A before the first linefeed and handles the !CR!
endlocal is only called once. The for /f 're-loops' for each linefeed contained in "!%%~N!", resetting %%~A
I wonder if there is a way to re-insert linefeeds between the %%~A's, then endlocal reassign %%N ??

Output:

Test now test1='^a!"^b'
Test #1 Disable/Disable result='^a!"^b' - OK
Test #2 Disable/Enable result='^a!"^b' - OK
Test #3 Enable/Disable result='^a!"^b' - OK
Test #4 Enable/Enable result='^a!"^b' - OK

Test now test2='caret^'
Test #5 Disable/Disable result='caret^' - OK
Test #6 Disable/Enable result='caret^' - OK
Test #7 Enable/Disable result='caret^' - OK
Test #8 Enable/Enable result='caret^' - OK

Test now test3=''
Test #9 Disable/Disable result='' - OK
Test #10 Disable/Enable result='' - OK
Test #11 Enable/Disable result='' - OK
Test #12 Enable/Enable result='' - OK

Test now test4='===Equals'
Test #13 Disable/Disable result='===Equals' - OK
Test #14 Disable/Enable result='===Equals' - OK
Test #15 Enable/Disable result='===Equals' - OK
Test #16 Enable/Enable result='===Equals' - OK

Test now test5='&"&^&!!!^^%^!'
Test #17 Disable/Disable result='&"&^&!!!^^%^!' - OK
Test #18 Disable/Enable result='&"&^&!!!^^%^!' - OK
Test #19 Enable/Disable result='&"&^&!!!^^%^!' - OK
Test #20 Enable/Enable result='&"&^&!!!^^%^!' - OK

*est now test6='Hello &^ "^ &" world!
X'
X" is missing in this string
*' - FAIL sable/Disable result='Hello &^ "^ &" world!
X" is missing in this string
*' - FAIL sable/Enable result='Hello &^ "^ &" world!
* Test #23 Enable/Disable result='Hello &^ "^ &" world!
X' - OK
* Test #24 Enable/Enable result='Hello &^ "^ &" world!
X' - OK

Code: Select all

:TestReturnVar.bat
@echo off
cls
setlocal DisableDelayedExpansion
if not defined ReturnVar call :ReturnVar
set /a test_cnt=0
set "test1=^a!"^^b"
set "test2=caret^"
set "test3="
set "test4====Equals"
@rem        &"&^&!!!^^%^!
set  test5=^&"&^&!!!^^%%^!
setlocal enabledelayedexpansion
set "test6=Hello &^^ "^^^^  ^&" world^!!CR!*!LF!X"
@echo off&echo(

for /L %%n in (1 1 6) do (
   call :Test_Both test%%n
)
exit /b

:Test_Both
setlocal EnableDelayedExpansion
echo Test now                    %1='!%1!'
endlocal
call :Test_context Disable Disable %1
call :Test_context Disable Enable  %1
call :Test_context Enable  Disable %1
call :Test_context Enable  Enable  %1
echo(
exit /b

:Test_context
set /a test_cnt+=1
setlocal %1DelayedExpansion
set "result=not set by macro"
call :startTest %1 %2 %3

REM Output the result
setlocal enabledelayedexpansion
if "!result!"=="!%~3!" (
   set "postfix=OK"
) ELSE (
    set postfix=FAIL
)
set "format1=  Test #!test_cnt! "
set "format2=%1/%2  "
echo !format1:~0,10! !format2:~0,15! result='!result!'    - !postfix! !level!
endlocal

endlocal
exit /b

:startTest
setlocal %2DelayedExpansion
%ReturnVar% result %3 1
exit /b

:ReturnVar Macro
@if defined ReturnVar @exit /b -1

:initReturnVar
@if "!" equ "" (
  >&2 @echo ERROR: ReturnVar must be initialized with delayed expansion disabled
  @exit /b 1
)

@set LF=^


::  Above 2 blank lines are critical - Do not remove  ::
@set ^"\n=^^^%LF%%LF%^%LF%%LF%^^"

@for /f "usebackq delims= " %%C in (`copy /z "%~f0" nul`) do @set "CR=%%C"

@REM ** ReturnVar <resultVariable> <TransferVariable> [endlocalCount] by jeb and carl
@REM ** return a <TransferVariable> over one or more endlocal barriers to a resultVariable
@REM ** endlocalCount default=1, is the number of endlocals that should be executed
@REM ** All special characters can be transferd from TranferVariable to the result variable,
@REM ** even "<>&|!^"
@REM ** The result is correct, independent of the delayed expansion mode after the endlocals
@set ^"ReturnVar=@for %%# in (1 2) do @if %%#==2 @(%\n%
  setlocal EnableDelayedExpansion%\n%
  set/a safeReturn_count=0%\n%
  for %%C in (!args!) do @(%\n%
    set "safeReturn[!safeReturn_count!]=%%~C" ^& set /a safeReturn_count+=1%\n%
  )%\n%
  if not defined safeReturn[2] set "safeReturn[2]=1"%\n%
  set /a safeReturn[2]+=2%\n%
  for /f "delims=" %%T in ("!safeReturn[1]!") do @set "_#_$_safeReturn=a!%%T!"%\n%
  set ^"_#_$_safeReturn=!_#_$_safeReturn:"=""q!"%\n%
  FOR /F %%R in ("!CR! #") DO @set "_#_$_safeReturn=!_#_$_safeReturn:%%~R=""r!"%\n%
  FOR %%L in ("!LF!") DO @set "_#_$_safeReturn=!_#_$_safeReturn:%%~L=""n!"%\n%
  set "_#_$_safeReturn=!_#_$_safeReturn:^=^^!"%\n%
  call set "_#_$_safeReturn=%%_#_$_safeReturn:^!=""c^!%%"%\n%
  set "_#_$_safeReturn=!_#_$_safeReturn:""c=^!"%\n%
  set ^"_#_$_safeReturn=!_#_$_safeReturn:""q="!"%\n%
  for %%L in ("!LF!") do @(%\n%
    for /f "delims=" %%N in (""!safeReturn[0]!"") do @(%\n%
      for /f "delims=" %%E in (""!_#_$_safeReturn!"") do @(%\n%
        for /l %%n in (1 1 !safeReturn[2]!) do @endlocal%\n%
        if "!" neq "" setlocal enabledelayedexpansion ^& set "_#_$_dis=1"%\n%
        set "%%~N=%%~E" !%\n%
        set "%%~N=!%%~N:""n=%%~L!"%\n%
        FOR /F %%R in ("!CR! #") DO @set "%%~N=!%%~N:""r=%%R!"%\n%
        set "%%~N=!%%~N:~1!"%\n%
        if defined _#_$_dis (%\n%
          for /f delims^^=^^ eol^^=  %%A in (""!%%~N!"") do @(%\n%
            if defined _#_$_dis (endlocal ^& set "%%~N=%%~A") else (%\n%
              echo %%~A is missing in this string%\n%
            )%\n%
          )%\n%
        )%\n%
      )%\n%
    )%\n%
  )%\n%
) else @setlocal ^& @set args="
@exit /b 0

carlsomo
Posts: 91
Joined: 02 Oct 2012 17:21

Re: ReturnVar macro revisited

#18 Post by carlsomo » 08 Nov 2013 02:11

Dear jeb,
I tried very hard not to use a temp file but this version had to resort to that only when called from a disabled environment with linefeeds in the transfer variable. I am not happy with it but it passes the jeb test. It would be much more satisfying to avoid writing to file under any circumstance but I cry 'UNCLE'
At least the goal of avoiding calls to an outside function is met
carl


Test now test1='^a!"^b'
Test #1 Disable/Disable result='^a!"^b' - OK
Test #2 Disable/Enable result='^a!"^b' - OK
Test #3 Enable/Disable result='^a!"^b' - OK
Test #4 Enable/Enable result='^a!"^b' - OK

Test now test2='caret^'
Test #5 Disable/Disable result='caret^' - OK
Test #6 Disable/Enable result='caret^' - OK
Test #7 Enable/Disable result='caret^' - OK
Test #8 Enable/Enable result='caret^' - OK

Test now test3=''
Test #9 Disable/Disable result='' - OK
Test #10 Disable/Enable result='' - OK
Test #11 Enable/Disable result='' - OK
Test #12 Enable/Enable result='' - OK

Test now test4='===Equals'
Test #13 Disable/Disable result='===Equals' - OK
Test #14 Disable/Enable result='===Equals' - OK
Test #15 Enable/Disable result='===Equals' - OK
Test #16 Enable/Enable result='===Equals' - OK

Test now test5='&"&^&!!!^^%^!'
Test #17 Disable/Disable result='&"&^&!!!^^%^!' - OK
Test #18 Disable/Enable result='&"&^&!!!^^%^!' - OK
Test #19 Enable/Disable result='&"&^&!!!^^%^!' - OK
Test #20 Enable/Enable result='&"&^&!!!^^%^!' - OK

*est now test6='Hello &^ "^ &" world!
X'
* Test #21 Disable/Disable result='Hello &^ "^ &" world!
X' - OK
* Test #22 Disable/Enable result='Hello &^ "^ &" world!
X' - OK
* Test #23 Enable/Disable result='Hello &^ "^ &" world!
X' - OK
* Test #24 Enable/Enable result='Hello &^ "^ &" world!
X' - OK



Code: Select all

:TestReturnVar.bat
@echo off
cls
setlocal DisableDelayedExpansion
call :ReturnVar
set /a test_cnt=0
rem echo on
set "test1=^a!"^^b"
set "test2=caret^"
set "test3="
set "test4====Equals"
@rem        &"&^&!!!^^%^!
set  test5=^&"&^&!!!^^%%^!
@setlocal enabledelayedexpansion
set "test6=Hello &^^ "^^^^  ^&" world^!!CR!*!LF!X"
@echo off&echo(
rem set ret

for /L %%n in (1 1 6) do (
   call :Test_Both test%%n
)
exit /b

:Test_Both
setlocal EnableDelayedExpansion
echo Test now                    %1='!%1!'
endlocal
call :Test_context Disable Disable %1
call :Test_context Disable Enable  %1
call :Test_context Enable  Disable %1
call :Test_context Enable  Enable  %1
echo(
exit /b

:Test_context
set /a test_cnt+=1
setlocal %1DelayedExpansion
set "result=not set by macro"
call :startTest %1 %2 %3

REM Output the result
setlocal enabledelayedexpansion
if "!result!"=="!%~3!" (
   set "postfix=OK"
) ELSE (
    set postfix=FAIL
)
set "format1=  Test #!test_cnt! "
set "format2=%1/%2  "
echo !format1:~0,10! !format2:~0,15! result='!result!'    - !postfix! !level!
endlocal

endlocal
exit /b

:startTest
setlocal %2DelayedExpansion
%ReturnVar% result %3 1
exit /b

:ReturnVar Macro
@if defined ReturnVar @exit /b -1

:initReturnVar
@if "!" equ "" (
  >&2 @echo ERROR: ReturnVar must be initialized with delayed expansion disabled
  @exit /b 1
)

@set LF=^


::  Above 2 blank lines are critical - Do not remove  ::
@set ^"\n=^^^%LF%%LF%^%LF%%LF%^^"

@for /f "usebackq delims= " %%C in (`copy /z "%~f0" nul`) do @set "CR=%%C"

@REM ** ReturnVar <resultVariable> <TransferVariable> [endlocalCount] by jeb and carl
@REM ** return a <TransferVariable> over one or more endlocal barriers to a resultVariable
@REM ** endlocalCount default=1, is the number of endlocals that should be executed
@REM ** All special characters can be transferd from TranferVariable to the result variable,
@REM ** even "<>&|!^"
@REM ** The result is correct, independent of the delayed expansion mode after the endlocals

@set ^"ReturnVar=@for %%# in (1 2) do @if %%#==2 @(%\n%
  setlocal EnableDelayedExpansion%\n%
  set/a safeReturn_count=0%\n%
  for %%C in (!args!) do @(%\n%
    set "safeReturn[!safeReturn_count!]=%%~C" ^& set /a safeReturn_count+=1%\n%
  )%\n%
  if not defined safeReturn[2] set "safeReturn[2]=1"%\n%
  set /a safeReturn[2]+=2%\n%
  for /f "delims=" %%T in ("!safeReturn[1]!") do @set "_#_$_safeReturn=a!%%T!"%\n%
  set ^"_#_$_safeReturn=!_#_$_safeReturn:"=""q!"%\n%
  FOR /F %%R in ("!CR! #") DO @set "_#_$_safeReturn=!_#_$_safeReturn:%%~R=""r!"%\n%
  FOR %%L in ("!LF!") DO @set "_#_$_safeReturn=!_#_$_safeReturn:%%~L=""n!"%\n%
  set "_#_$_safeReturn=!_#_$_safeReturn:^=^^!"%\n%
  call set "_#_$_safeReturn=%%_#_$_safeReturn:^!=""c^!%%"%\n%
  set "_#_$_safeReturn=!_#_$_safeReturn:""c=^!"%\n%
  set ^"_#_$_safeReturn=!_#_$_safeReturn:""q="!"%\n%
  for %%L in ("!LF!") do @(%\n%
    for /f "delims=" %%N in (""!safeReturn[0]!"") do @(%\n%
      for /f "delims=" %%E in (""!_#_$_safeReturn!"") do @(%\n%
        for /l %%n in (1 1 !safeReturn[2]!) do @endlocal%\n%
        if "!" neq "" setlocal enabledelayedexpansion ^& set "_#_$_dis=1"%\n%
        set "%%~N=%%~E" !%\n%
        set "%%~N=!%%~N:""n=%%~L!"%\n%
        FOR /F %%R in ("!CR! #") DO @set "%%~N=!%%~N:""r=%%R!"%\n%
        set "%%~N=!%%~N:~1!"%\n%
        if defined _#_$_dis (%\n%
          for /f delims^^=^^ eol^^=  %%A in (""!%%~N!"") do @(%\n%
            if defined _#_$_dis (%\n%
              if "!%%~N!" neq "%%~A" echo !%%~N!^>!temp!\_#_$_dis.txt%\n%
              endlocal ^& set "%%~N=%%~A"%\n%
            ) else (%\n%
              ^<%temp%\_#_$_dis.txt set/p %%~N=""%\n%
            )%\n%
          )%\n%
          if exist %temp%\_#_$_dis.txt del %temp%\_#_$_dis.txt 2^>nul%\n%
        )%\n%
      )%\n%
    )%\n%
  )%\n%
) else @setlocal ^& @set args="
@exit /b 0

jeb
Expert
Posts: 1041
Joined: 30 Aug 2007 08:05
Location: Germany, Bochum

Re: ReturnVar macro revisited

#19 Post by jeb » 08 Nov 2013 02:49

Hmm,

I have two new tests for you.

Code: Select all

set "test7=Line1!LF!Line2!LF!"
set "test8=Line1!CR!!LF!Line2"


Both fails with your temporary file technic.

IMO the set /p technic is a dead end.

My own solution for this part looks like this (It requires one more global variable set "sfr.caret=^")

Code: Select all

for %%L IN ("!\n!") DO (%\n%
   if "!var!" == "!var:%%~L=!" ( %= REM ** No linefeeds found, so use normal FOR/F return =%  %\n%
   ) ELSE if "!var!" NEQ "!var:%%~C=!" ( %= REM ** LF and CR founds, return by calling a helper function is required =%  %\n%
      for /F "delims=" %%P in (""!var!"") DO (%\n%
         for /L %%n in (1 1 !endlocal_cnt!) do endlocal%\n%
         call :safeReturn_Helper%\n%
      )%\n%
   ) ELSE ( %= REM ** Linefeed creation with a CALL SET expansion =%  %\n%
   for /F "delims=: tokens=1,*" %%C IN ("!cr!:"!path!"") DO (%\n%
      %= REM ** Make all special chars safe against one percent expansion =%  %\n%
      set "var=!var:^=^^!"%\n%
      set "var=!var:&=^&!"%\n%
      set "var=!var:|=^|!"%\n%
      set "var=!var:<=^<!"%\n%
      set "var=!var:>=^>!"%\n%
      set "var=!var:"=^^"!" %= REM ** Quotes must be escaped so all other =%  %\n%
      set ^"var=!var:%%~L=^^%%~C!" %= REM ** Replace LF with CR, as CR can survive a FOR/F and it can't be in the string =%  %\n%
      %= REM ** The extra caret in front of the CR is required when the CR is reverted to a linefeed =%  %\n%
      for /F "delims=" %%a in (""!var!"") DO (%\n%
         endlocal%\n%
         endlocal%\n%
         set "path=" %= REM ** Path must be removed, to speed up =%%\n%
         set "pathext=:" %= REM ** PathExt must be removed, to avoid external search for a set.bat,set.exe,... =%%\n%
         set "result=%%~a"%\n%
         %= REM ** The first quote must be escaped twice, first for the %NLM% and then for the escaped content of the result =%%\n%
         call set %%sfr.caret%%^"result=%%result:%%~C=%NLM%%NLM%%%"%\n%
         set "path=%%~D" %= REM ** Restore path and PathExt =%%\n%
         set "pathext=%%~Q"%\n%
      )%\n%
   )%\n%
)%\n%


I changed the test suite a bit for a bit better overview.

Code: Select all

:TestReturnVar.bat
@echo off
cls
setlocal DisableDelayedExpansion
if not defined ReturnVar call :ReturnVar
set /a test_cnt=0
For /F "delims=;" %%# in ('"Prompt;$H;&For %%_ in (1) Do Rem"') Do Set "BS=%%#"
set "test1=^a!"^^b"
set "test2=caret^"
set "test3="
set "test4====Equals"
@rem        &"&^&!!!^^%^!
set  test5=^&"&^&!!!^^%%^!
setlocal enabledelayedexpansion
set "test6=Hello &^^ "^^^^  ^&" world^!!CR!*!LF!X"
set "test7=Line1!LF!Line2!LF!"
set "test8=Line1!CR!!LF!Line2"
@echo off&echo(

for /L %%n in (1 1 8) do (
   call :Test_Both test%%n
)
exit /b

:Test_Both
setlocal EnableDelayedExpansion
echo --- %1 ---                       text='!%1!'
endlocal
call :Test_context Disable Disable %1
call :Test_context Disable Enable  %1
call :Test_context Enable  Disable %1
call :Test_context Enable  Enable  %1
echo(
exit /b

:Test_context
set /a test_cnt+=1
setlocal %1DelayedExpansion
set "result=not set by macro"
call :startTest %1 %2 %3

REM Output the result
setlocal enabledelayedexpansion
set "format1=  Test #!test_cnt! "
set "format2=%1/%2  "
if "!result!"=="!%~3!" (
   set "postfix=   OK "
   echo !format1:~0,10! !postfix! !format2:~0,15!
) ELSE (
    set "postfix=<FAIL>"
   echo !format1:~0,10! !postfix! !format2:~0,15! result='!result!'
)
endlocal

endlocal
exit /b


For your last code it outputs
output wrote:--- test1 --- text='^a!"^b'
Test #1 OK Disable/Disable
Test #2 OK Disable/Enable
Test #3 OK Enable/Disable
Test #4 OK Enable/Enable

--- test2 --- text='caret^'
Test #5 OK Disable/Disable
Test #6 OK Disable/Enable
Test #7 OK Enable/Disable
Test #8 OK Enable/Enable

--- test3 --- text=''
Test #9 OK Disable/Disable
Test #10 OK Disable/Enable
Test #11 OK Enable/Disable
Test #12 OK Enable/Enable

--- test4 --- text='===Equals'
Test #13 OK Disable/Disable
Test #14 OK Disable/Enable
Test #15 OK Enable/Disable
Test #16 OK Enable/Enable

--- test5 --- text='&"&^&!!!^^%^!'
Test #17 OK Disable/Disable
Test #18 OK Disable/Enable
Test #19 OK Enable/Disable
Test #20 OK Enable/Enable

*-- test6 --- text='Hello &^ "^ &" world!
X'
Test #21 OK Disable/Disable
Test #22 OK Disable/Enable
Test #23 OK Enable/Disable
Test #24 OK Enable/Enable

--- test7 --- text='Line1
Line2
'
Test #25 <FAIL> Disable/Disable result='Line1
Line2'
Test #26 <FAIL> Disable/Enable result='Line1
Line2'
Test #27 OK Enable/Disable
Test #28 OK Enable/Enable

--- test8 --- text='Line1
Line2'
Test #29 <FAIL> Disable/Disable result='Line1'
Test #30 <FAIL> Disable/Enable result='Line1'
Test #31 OK Enable/Disable
Test #32 OK Enable/Enable

carlsomo
Posts: 91
Joined: 02 Oct 2012 17:21

Re: ReturnVar macro revisited

#20 Post by carlsomo » 08 Nov 2013 23:38

Ah, very interesting...

My macro fails if there is not a character after a CR or LF other than CR or LF else it works :roll:

so take your latest test 6 (works) and 7 and 8, does not work and try this:

change 7 and 8 to this:

set "test7=Line1!LF!Line2!LF!."
set "test8=Line1!CR!.!LF!Line2"

and you get this:
Test now test1='^a!"^b'
Test #1 Disable/Disable result='^a!"^b' - OK
Test #2 Disable/Enable result='^a!"^b' - OK
Test #3 Enable/Disable result='^a!"^b' - OK
Test #4 Enable/Enable result='^a!"^b' - OK

Test now test2='caret^'
Test #5 Disable/Disable result='caret^' - OK
Test #6 Disable/Enable result='caret^' - OK
Test #7 Enable/Disable result='caret^' - OK
Test #8 Enable/Enable result='caret^' - OK

Test now test3=''
Test #9 Disable/Disable result='' - OK
Test #10 Disable/Enable result='' - OK
Test #11 Enable/Disable result='' - OK
Test #12 Enable/Enable result='' - OK

Test now test4='===Equals'
Test #13 Disable/Disable result='===Equals' - OK
Test #14 Disable/Enable result='===Equals' - OK
Test #15 Enable/Disable result='===Equals' - OK
Test #16 Enable/Enable result='===Equals' - OK

Test now test5='&"&^&!!!^^%^!'
Test #17 Disable/Disable result='&"&^&!!!^^%^!' - OK
Test #18 Disable/Enable result='&"&^&!!!^^%^!' - OK
Test #19 Enable/Disable result='&"&^&!!!^^%^!' - OK
Test #20 Enable/Enable result='&"&^&!!!^^%^!' - OK

*est now test6='Hello &^ "^ &" world!
X'
* Test #21 Disable/Disable result='Hello &^ "^ &" world!
X' - OK
* Test #22 Disable/Enable result='Hello &^ "^ &" world!
X' - OK
* Test #23 Enable/Disable result='Hello &^ "^ &" world!
X' - OK
* Test #24 Enable/Enable result='Hello &^ "^ &" world!
X' - OK

Test now test7='Line1
Line2
.'
Test #25 Disable/Disable result='Line1
Line2
.' - OK
Test #26 Disable/Enable result='Line1
Line2
.' - OK
Test #27 Enable/Disable result='Line1
Line2
.' - OK
Test #28 Enable/Enable result='Line1
Line2
.' - OK

.est now test8='Line1
Line2'
. Test #29 Disable/Disable result='Line1
Line2' - OK
. Test #30 Disable/Enable result='Line1
Line2' - OK
. Test #31 Enable/Disable result='Line1
Line2' - OK
. Test #32 Enable/Enable result='Line1
Line2' - OK


Hmmm... there could be a workable solution here?

jeb
Expert
Posts: 1041
Joined: 30 Aug 2007 08:05
Location: Germany, Bochum

Re: ReturnVar macro revisited

#21 Post by jeb » 10 Nov 2013 05:18

carlsomo wrote:My macro fails if there is not a character after a CR or LF other than CR or LF else it works :roll:
so take your latest test 6 (works) and 7 and 8, does not work and try this:


There was a cause, why I build the test this way :)

The problem of set /p is that it removes all control characters at the end and it can only handle up to ~1021 characters.
The other problem (test8) is that it reads only till ~1021 characters or the first CR/LF.

That's why I said that this technic seems to be a dead end :wink:

Test7 can be solved with the linefeed part of my macro,
it adds the linefeeds just after the FOR/F parameter assignment with a CALL.

jeb wrote:set "var=!var:^=^^!"%\n%
set "var=!var:&=^&!"%\n%
set "var=!var:|=^|!"%\n%
set "var=!var:<=^<!"%\n%
set "var=!var:>=^>!"%\n%
set "var=!var:"=^^"!" %= REM ** Quotes must be escaped so all other =% %\n%
set ^"var=!var:%%~L=^^%%~C!" %= REM ** Replace LF with CR, as CR can survive a FOR/F and it can't be in the string =% %\n%
%= REM ** The extra caret in front of the CR is required when the CR is reverted to a linefeed =% %\n%
for /F "delims=" %%a in (""!var!"") DO (%\n%
endlocal%\n%
endlocal%\n%
set "path=" %= REM ** Path must be removed, to speed up =%%\n%
set "pathext=:" %= REM ** PathExt must be removed, to avoid external search for a set.bat,set.exe,... =%%\n%
set "result=%%~a"%\n%
%= REM ** The first quote must be escaped twice, first for the %NLM% and then for the escaped content of the result =%%\n%
call set %%sfr.caret%%^"result=%%result:%%~C=%NLM%%NLM%%%"%\n%
set "path=%%~D" %= REM ** Restore path and PathExt =%%\n%
set "pathext=%%~Q"%\n%

dbenham
Expert
Posts: 2461
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: ReturnVar macro revisited

#22 Post by dbenham » 01 Dec 2013 14:09

Going back to jeb's original post, I find it interesting that the worst performing return is when the return environment has delayed expansion disabled. Yet the absolute simplest solution exists for disabled delayed expansion if the return value does not contain the newline character - a FOR /F can easily return the value without worrying about any escaping.

I decided to develop different techniques for handling delayed expansion enabled vs disabled, and return value containing or not containing newline. The enabled delayed expansion code is based on jeb's code. I then experimented with ways of dynamically applying the most efficient method possible, depending on the circumstance.

I also added another option to specify the returned ERRORLEVEL. Values of 0 and 1 can be generated very efficiently. But values greater than 1 are more problematic. I started by using cmd /c exit N, but that was unacceptably slow on my home machine. I switched to using a CALLed subroutine with exit /b N.

I now have two optional arguments - the number of active SETLOCAL, and the returned ERRORLEVEL. I opted to make those optional arguments named arguments of the form local=N and err=M. I adapted jeb's original code to my style with the new optional arguments.

I also put all the required macro code in a stand-alone library script. The ugly CALLed functions are embedded within the script. It is very convenient to load the macro from any script as needed.

I wrote 3 new versions: macro.rtnDave1.bat, macro.rtnDave2.bat, and macro.rtnDave3.bat, along with my version of jeb's original code - macro.rtnJeb.bat.

I also wrote a modified test script that provides timing data. The test script expects a single argument value of dave1, dave2, dave3, or jeb. The argument controls which version of the macro gets loaded into memory.


testReturn.bat

Code: Select all

@echo off
cls
setlocal DisableDelayedExpansion
call macro.rtn%1
set /a test_cnt=0
set "value1=%%1^a!"^^b"
set "value2=caret^"
set "value3="
set "value4====Equals"
rem           &"&^&!!!^^%^!
set ^"value5=^&"&^&!!!^^%%^!"
setlocal enabledelayedexpansion
set "value6=Hello &^^ "^^^^  ^&" world^!!CR!*!LF!X"
set "value7=Line1!LF!Line2!LF!"
set "value8=Line1!CR!!LF!Line2"
set "value9=CR!CR!"
echo(

for /L %%n in (1 1 9) do call :Test_Both value%%n

set "input="
set "singleLine=!value1!"
set "multiLine =!value6!"
for %%a in ("SingleLine" "MultiLine ") do for %%b in ("Disable" " Enable") do for %%c in (0 1 2) do (
  set "beg=!time!"
  setlocal %%~bDelayedExpansion
  for /l %%N in (1 1 5000) do (
    setlocal enableDelayedExpansion
    set "input=!%%~a!"
    %returnVar% result input local=1 err=%%c
  )
  endlocal
  set "end=!time!"
  set /a "t1=t2=0"
  for /f "tokens=1-4 delims=:." %%a in ("!beg: =0!") do set /a "t1=(((1%%a*60)+1%%b)*60+1%%c)*100+1%%d-36610100"
  for /f "tokens=1-4 delims=:." %%a in ("!end: =0!") do set /a "t2=(((1%%a*60)+1%%b)*60+1%%c)*100+1%%d-36610100"
  set /a "diff=t2-t1"
  if !diff! lss 0 set /a "diff+=24*60*60*100"
  echo %%~a %%~b err=%%c : !diff!
)
exit /b

:Test_Both
setlocal EnableDelayedExpansion
echo ------- %1 -------------------- text='!%1!'
for %%x in (0 1 2) do (
  call :Test_context Disable Disable %1 %%x
  call :Test_context Disable Enable  %1 %%x
  call :Test_context Enable  Disable %1 %%x
  call :Test_context Enable  Enable  %1 %%x
)
echo(
exit /b

:Test_context
set /a test_cnt+=1
setlocal %1DelayedExpansion
set "result=not set by macro"
call :startTest %2 %3 %4

REM Output the result
set "error=%errorlevel%"
setlocal enabledelayedexpansion
set "input=inputNotSet"
set "format1=  Test #!test_cnt! "
set "format2=%1/%2  "
set "postfix=<FAIL>"
if "!result!"=="!%~3!" if %error% equ %4 set "postfix=   OK "
echo !format1:~0,10! err=%4 !postfix! !format2:~0,15! err=!error! result='!result!'
exit /b


:startTest
setlocal enableDelayedExpansion
set "input=!%2!"
setlocal %1DelayedExpansion
%ReturnVar% result input local=2 err=%3
exit /b


macro.rtnJeb.bat

Code: Select all

@echo off
if not defined returnVar call :loadMacro&exit /b
if "%~1" neq ":returnDisabled" exit /b %2
setlocal enableDelayedExpansion
set "rtn=!%2!"
set "rtn=!rtn:%%=%%3!"
set "rtn=!rtn:""n=%%~L!"
set "rtn=!rtn:""r=%%4!"
set "rtn=!rtn:""q=%%~5!"
for /f "tokens=1-4" %%3 in (^"%% !CR! """") do (
  endlocal
  set "%2=%rtn:~1%"
  exit /b %%X
)


:loadMacro
if "!" equ "" (
  >2 echo ERROR: Delayed expansion must be off when loading the returnVar macro.
  exit /b 1
)
set LF=^


set ^"\n=^^^%LF%%LF%^%LF%%LF%^^"
for /F "usebackq delims= " %%C in (`copy /z "%~f0" nul`) do set "CR=%%C"

set ^"returnVar=for %%# in (1 2) do if %%#==2 (setlocal enableDelayedExpansion%\n%
for /f "tokens=1-4" %%1 in ("!returnVar.args!") do (%\n%
  set "rtn=x!%%2!"%\n%
  set /a "err=!errorlevel!, local=1"%\n%
  for %%A in ("%%~3" "%%~4") do if %%A neq "" set /a "%%~A"%\n%
  set /a "local+=1"%\n%
  for /f %%R in ("!CR!!CR!") do for %%L in ("!LF!") do for %%X in (!err!) do (%\n%
    set ^"rtn=!rtn:"=""q!"%\n%
    set "rtn=!rtn:%%R=""r!"%\n%
    set "rtn=!rtn:%%~L=""n!"%\n%
    set "rtnDis=!rtn!"%\n%
    set "rtn=!rtn:^=^^!"%\n%
    set "path="%\n%
    set "pathExt=;"%\n%
    call set "rtn=%%rtn:^!=""c^!%%"%\n%
    set "rtn=!rtn:""c=^!"%\n%
    for /f "delims=" %%D in (""!rtnDis!"") do for /f "delims=" %%E in (""!rtn!"") do (%\n%
      for /l %%n in (1 1 !local!) do endlocal%\n%
      if "!"=="" (%\n%
        set "%%1=%%~E" !%\n%
        set "%%1=!%%1:""n=%%~L!"%\n%
        set "%%1=!%%1:""r=%%R!"%\n%
        set ^"%%1=!%%1:""q="!"%\n%
        set "%%1=!%%1:~1!"%\n%
        if "%%X" equ "0" (call ) else if "%%X" equ "1" (call) else call "%~f0" :exitErr %%X%\n%
      ) else (%\n%
        set "%%1=%%~D"%\n%
        call "%~f0" :returnDisabled %%1%\n%
      )%\n%
    )%\n%
  )%\n%
)) else set returnVar.args="

exit /b 0


macro.rtnDave1.bat - new code if value does not contain newline (delayed expansion enabled or disabled)

Code: Select all

@echo off
if not defined returnVar call :loadMacro&exit /b
if "%~1" neq ":returnDisabled" exit /b %2
setlocal enableDelayedExpansion
set "rtn=!%2!"
set "rtn=!rtn:%%=%%3!"
set "rtn=!rtn:""n=%%~L!"
set "rtn=!rtn:""r=%%4!"
set "rtn=!rtn:""q=%%~5!"
for /f "tokens=1-4" %%3 in (^"%% !CR! """") do (
  endlocal
  set "%2=%rtn%"
  exit /b %%X
)


:loadMacro
if "!" equ "" (
  >2 echo ERROR: Delayed expansion must be off when loading the returnVar macro.
  exit /b 1
)
set LF=^


set ^"\n=^^^%LF%%LF%^%LF%%LF%^^"
for /F "usebackq delims= " %%C in (`copy /z "%~f0" nul`) do set "CR=%%C"

::returnVar  OutVar  InVar  [Local=N]  [Err=N]
set ^"ReturnVar=for %%# in (1 2) do if %%#==2 (setlocal enableDelayedExpansion%\n%
for /f "tokens=1-4" %%1 in ("!returnVar.args!") do (%\n%
  set "rtn="!%%2!""%\n%
  set /a "err=!errorlevel!, local=1"%\n%
  for %%A in ("%%~3" "%%~4") do if %%A neq "" set /a "%%~A"%\n%
  set /a "local+=1"%\n%
  for %%L in ("!LF!") do for /f %%R in ("!CR!!CR!") do for %%X in (!err!) do (%\n%
    if "!rtn:%%~L=!" equ "!rtn!" (%\n%
      for /f "delims=" %%V in ("!rtn!") do (%\n%
        for /l %%N in (1 1 !local!) do endlocal%\n%
        if "!!" neq "" (%\n%
          set "%%1=%%~V"%\n%
        ) else (%\n%
          setlocal disableDelayedExpansion%\n%
          set "rtn=%%V"%\n%
          setlocal enableDelayedExpansion%\n%
          for /f %%R in ("!CR!!CR!") do (%\n%
            set "rtn=!rtn:@=@A!"%\n%
            set "rtn=!rtn:"=@Q!^"%\n%
            set "rtn=!rtn:%%R=@R!"%\n%
            set "rtn=!rtn:^=@C@C!"%\n%
            setlocal disableDelayedExpansion%\n%
            set "path="%\n%
            set "pathext=;"%\n%
            call set "rtn=%%rtn:!=@C!%%"%\n%
            setlocal enableDelayedExpansion%\n%
            set "rtn=!rtn:@C=^!"%\n%
            set "rtn=!rtn:@R=%%R!"%\n%
            set "rtn=!rtn:@Q="!^"%\n%
            set "rtn=!rtn:@A=@!"%\n%
            for /f "delims=" %%V in ("!rtn!") do (%\n%
              endlocal^&endlocal^&endlocal^&endlocal%\n%
              set "%%1=%%~V"!%\n%
            )%\n%
          )%\n%
        )%\n%
        if "%%X" equ "0" (call ) else if "%%X" equ "1" (call) else call "%~f0" :exitErr %%X%\n%
      )%\n%
    ) else (%\n%
      set "rtn=!rtn:~1,-1!"%\n%
      set ^"rtn=!rtn:"=""q!"%\n%
      set "rtn=!rtn:%%R=""r!"%\n%
      set "rtn=!rtn:%%~L=""n!"%\n%
      set "rtnDis=!rtn!"%\n%
      set "rtn=!rtn:^=^^!"%\n%
      set "path="%\n%
      set "pathExt=;"%\n%
      call set "rtn=%%rtn:^!=""c^!%%"%\n%
      set "rtn=!rtn:""c=^!"%\n%
      for /f "delims=" %%D in ("!rtnDis!") do for /f "delims=" %%E in ("!rtn!") do (%\n%
        for /l %%n in (1 1 !local!) do endlocal%\n%
        if "!"=="" (%\n%
          set "%%1=%%~E" !%\n%
          set "%%1=!%%1:""n=%%~L!"%\n%
          set "%%1=!%%1:""r=%%R!"%\n%
          set ^"%%1=!%%1:""q="!"%\n%
          if "%%X" equ "0" (call ) else if "%%X" equ "1" (call) else call "%~f0" :exitErr %%X%\n%
        ) else (%\n%
          set "%%1=%%~D"%\n%
          call "%~f0" :returnDisabled %%1%\n%
        )%\n%
      )%\n%
    )%\n%
  )%\n%
)) else set returnVar.args=^"

exit /b


macro.rtnDave2.bat - new code only if value does not contain newline and delayed expansion is disabled

Code: Select all

@echo off
if not defined returnVar call :loadMacro&exit /b
if "%~1" neq ":returnDisabled" exit /b %2
setlocal enableDelayedExpansion
set "rtn=!%2!"
set "rtn=!rtn:%%=%%3!"
set "rtn=!rtn:""n=%%~L!"
set "rtn=!rtn:""r=%%4!"
set "rtn=!rtn:""q=%%~5!"
for /f "tokens=1-4" %%3 in (^"%% !CR! """") do (
  endlocal
  set "%2=%rtn%"
  exit /b %%X
)


:loadMacro
if "!" equ "" (
  >2 echo ERROR: Delayed expansion must be off when loading the returnVar macro.
  exit /b 1
)
set LF=^


set ^"\n=^^^%LF%%LF%^%LF%%LF%^^"
for /F "usebackq delims= " %%C in (`copy /z "%~f0" nul`) do set "CR=%%C"

::returnVar  OutVar  InVar  [Local=N]  [Err=N]
set ^"ReturnVar=for %%# in (1 2) do if %%#==2 (setlocal enableDelayedExpansion%\n%
for /f "tokens=1-4" %%1 in ("!returnVar.args!") do (%\n%
  set "rtn="!%%2!""%\n%
  set /a "err=!errorlevel!, local=1"%\n%
  for %%A in ("%%~3" "%%~4") do if %%A neq "" set /a "%%~A"%\n%
  set /a "local+=1"%\n%
  for %%L in ("!LF!") do for /f %%R in ("!CR!!CR!") do for %%X in (!err!) do (%\n%
    if "!rtn:%%~L=!" neq "!rtn!" (set returnVar.part2=1) else set "returnVar.part2="%\n%
    if not defined returnVar.part2 for /f "delims=" %%V in ("!rtn!") do (%\n%
      for /l %%N in (1 1 !local!) do endlocal%\n%
      if %%V equ "" (%\n%
        set "%%1=%%~V"%\n%
        set "returnVar.part2="%\n%
      ) else if "!" neq "" (%\n%
        set "%%1=%%~V"%\n%
        set "returnVar.part2="%\n%
      ) else (%\n%
        setlocal disableDelayedExpansion%\n%
        set "rtn=%%V"%\n%
        setlocal enableDelayedExpansion%\n%
        set "returnVar.part2=1"%\n%
        set local=2%\n%
      )%\n%
      if not defined returnVar.part2 if "%%X" equ "0" (call ) else if "%%X" equ "1" (call) else call "%~f0" :exitErr %%X%\n%
    )%\n%
    if defined returnVar.part2 (%\n%
      set "returnVar.part2="%\n%
      set "rtn=!rtn:~1,-1!"%\n%
      set ^"rtn=!rtn:"=""q!"%\n%
      set "rtn=!rtn:%%R=""r!"%\n%
      set "rtn=!rtn:%%~L=""n!"%\n%
      set "rtnDis=!rtn!"%\n%
      set "rtn=!rtn:^=^^!"%\n%
      set "path="%\n%
      set "pathExt=;"%\n%
      call set "rtn=%%rtn:^!=""c^!%%"%\n%
      set "rtn=!rtn:""c=^!"%\n%
      for /f "delims=" %%D in ("!rtnDis!") do for /f "delims=" %%E in ("!rtn!") do (%\n%
        for /l %%n in (1 1 !local!) do endlocal%\n%
        if "!"=="" (%\n%
          set "%%1=%%~E" !%\n%
          set "%%1=!%%1:""n=%%~L!"%\n%
          set "%%1=!%%1:""r=%%R!"%\n%
          set ^"%%1=!%%1:""q="!"%\n%
          if "%%X" equ "0" (call ) else if "%%X" equ "1" (call) else call "%~f0" :exitErr %%X%\n%
        ) else (%\n%
          set "%%1=%%~D"%\n%
          call "%~f0" :returnDisabled %%1%\n%
        )%\n%
      )%\n%
    )%\n%
  )%\n%
)) else set returnVar.args=^"

exit /b


macro.rtnDave3.bat - new code only if value does not contain newline (or is empty) and delayed expansion is disabled. More escaping done up front before deciding on which technique to use.

Code: Select all

@echo off
if not defined returnVar call :loadMacro&exit /b
if "%~1" neq ":returnDisabled" exit /b %2
setlocal enableDelayedExpansion
set "rtn=!%2!"
set "rtn=!rtn:%%=%%3!"
set "rtn=!rtn:""n=%%~L!"
set "rtn=!rtn:""r=%%4!"
set "rtn=!rtn:""q=%%~5!"
for /f "tokens=1-4" %%3 in (^"%% !CR! """") do (
  endlocal
  set "%2=%rtn:~1%"
  exit /b %%X
)


:loadMacro
if "!" equ "" (
  >2 echo ERROR: Delayed expansion must be off when loading the returnVar macro.
  exit /b 1
)
set LF=^


set ^"\n=^^^%LF%%LF%^%LF%%LF%^^"
for /F "usebackq delims= " %%C in (`copy /z "%~f0" nul`) do set "CR=%%C"

::returnVar  OutVar  InVar  [Local=N]  [Err=N]
set ^"ReturnVar=for %%# in (1 2) do if %%#==2 (setlocal enableDelayedExpansion%\n%
for /f "tokens=1-4" %%1 in ("!returnVar.args!") do (%\n%
  set "rtn=.!%%2!"%\n%
  set /a "err=!errorlevel!, local=1"%\n%
  for %%A in ("%%~3" "%%~4") do if %%A neq "" set /a "%%~A"%\n%
  set /a "local+=1"%\n%
  for %%L in ("!LF!") do for /f %%R in ("!CR!!CR!") do for %%X in (!err!) do (%\n%
    set ^"rtnE=!rtn:"=""q!"%\n%
    set "rtnE=!rtnE:%%R=""r!"%\n%
    set "rtnD=!rtnE:%%~L=""n!"%\n%
    if !rtnD! neq !rtnE! (%\n%
      set multiLn=1%\n%
      set "rtnE=!rtnD!"%\n%
    ) else set "multiLn=0"%\n%
    set "rtnE=!rtnE:^=^^!"%\n%
    set "path="%\n%
    set "pathExt=;"%\n%
    call set "rtnE=%%rtnE:^!=""c^!%%"%\n%
    set "rtnE=!rtnE:""c=^!"%\n%
    set "returnVar.continue=1"%\n%
    for /f "delims=" %%D in ("!rtnD!") do for /f "delims=" %%E in ("!rtnE!") do for %%M in (!multiLn!) do for /f "delims=" %%V in (""!rtn:~1!"") do if defined returnVar.continue (%\n%
      for /l %%n in (1 1 !local!) do endlocal%\n%
      if "!"=="" (%\n%
        set "%%1=%%~E" !%\n%
        set "%%1=!%%1:""n=%%~L!"%\n%
        set "%%1=!%%1:""r=%%R!"%\n%
        set ^"%%1=!%%1:""q="!"%\n%
        set "%%1=!%%1:~1!"%\n%
        if "%%X" equ "0" (call ) else if "%%X" equ "1" (call) else call "%~f0" :exitErr %%X%\n%
      ) else if %%M equ 0 (%\n%
        set "%%1=%%~V"%\n%
        if "%%X" equ "0" (call ) else if "%%X" equ "1" (call) else call "%~f0" :exitErr %%X%\n%
      ) else (%\n%
        set "%%1=%%~D"%\n%
        call "%~f0" :returnDisabled %%1%\n%
      )%\n%
    )%\n%
  )%\n%
)) else set returnVar.args=^"

exit /b


Here are the timing results, two runs for each test. I think I prefer the Dave2 version.

Code: Select all

             Timing for 5000 Iterations (centiseconds)

  TEST                   |   JEB     |   DAVE1   |   DAVE2   |   DAVE3
-------------------------+-----------+-----------+-----------+----------
SingleLine Disable err=0 | 3778 3772 | 1392 1483 | 1194 1196 | 1778 1781
SingleLine Disable err=1 | 3772 3767 | 1366 1370 | 1194 1196 | 1791 1794
SingleLine Disable err=2 | 3777 3755 | 2344 2725 | 2014 2381 | 3105 3120
-------------------------+-----------+-----------+-----------+----------
SingleLine  Enable err=0 | 1236 1226 | 2504 2508 | 2015 2020 | 1873 1876
SingleLine  Enable err=1 | 1246 1237 | 2503 2508 | 1931 1930 | 1876 1882
SingleLine  Enable err=2 | 2381 2376 | 3503 3887 | 2732 3099 | 3257 3267
-------------------------+-----------+-----------+-----------+----------
MultiLine  Disable err=0 | 3819 3800 | 4550 5365 | 3524 4291 | 5246 5270
MultiLine  Disable err=1 | 3822 3814 | 4556 5377 | 3520 4278 | 5260 5298
MultiLine  Disable err=2 | 3806 3804 | 4543 5364 | 3497 4286 | 5263 5300
-------------------------+-----------+-----------+-----------+----------
MultiLine   Enable err=0 | 1274 1273 | 2014 2020 | 1643 1640 | 1989 1990
MultiLine   Enable err=1 | 1283 1283 | 2019 2016 | 1646 1645 | 1990 1994
MultiLine   Enable err=2 | 2433 2426 | 3027 3408 | 2442 2808 | 3368 3366


Dave Benham

Ed Dyreen
Expert
Posts: 1569
Joined: 16 May 2011 08:21
Location: Flanders(Belgium)
Contact:

Re: ReturnVar macro revisited

#23 Post by Ed Dyreen » 02 Dec 2013 03:00

Well of all code I see, mine must be the ugliest duck :P

Funny, I recently tried to get $cr, $lf over endlocal in a single expansion too.
But after haven't coded a single line of DOS in an entire year I forgot this is currently impossible.
So I gave up where jeb stated that even though it can't be done in disDelayed it still is possible for enaDelayed.

I recently had problems with my %replace_% macro replacing linefeeds due to the fact it implements my %endlocal_% macro
that doesn't support $lf, $cr.

I nest the %endlocal_% macro inside other macro's and the outer macro can't exceed 8k.
Though some variables are missing, I'm sure expert can follow.

I've updated my endlocal_ macro from this;

Code: Select all

::--------------------------------------------------------------------------------------------------------------------------
set "$defines=endlocal_"
::--------------------------------------------------------------------------------------------------------------------------
:: last updated       : 2012^11^19
:: support            : naDelayed, no ( $cr, $lf )
::
::   -the number of optional endlocals is the first parameter.
::   -multiple variables can be returned at once.
::   -the maximum return size which is 8k is the maximum of the sum of all sizes of all variables to return.
::
set ^"$usage.%$defines%=^
%=   =% use: ( %%endlocal_%% #eCount, #var, #etc.. )%$n1c%
%=   =% err: Unaffected, panic otherwise"
::
2>nul ( %macroStart_% enableDelayedExpansion )
:: (
%=   =%set ^"!$defines!=!forQ_! (1,2) do if %%?==2 (^
!==^^^!^
%=      =%set ¤=^&!forQ_! (^^^!º^^^!) do if ^^^!¤^^^!.==. set/a¤=0%%~? 2^>nul^&^&(%=   endlocal count    =%^
%=         =%call set º=%%º:*^^^!¤^^^!=%%^&set/a¤+=1^
%=      =%)^|^|set/a¤=2%$n1c%
%=      =%set _=^&!forQ_! (^^^!º^^^!) do (^
%=         =%set ?=^^^!%%~?^^^!^&if defined ? set ?=^^^!?:§=§§^^^!%=                   coded mark set    =%%$n1c%
%=         =%set _=^^^!_^^^!l§f%%~?=^^^!?^^^!%=                                        coded add $lf     =%%$n1c%
%=      =%)%$n1c%
%=      =%set _=^^^!_:^^^"=""^^^!%=                                                    code pre 94, 33   =%%$n1c%
%=      =%set "_=^!_:^^^=9§4^!"^&call set "_=%%^^^_:^^^!=3§3%%"%=                      enco     94, 33   =%%$n1c%
%=      =%for /f "delims=" %%r in ("^!_:~3^!") do (^
%=         =%!forC_! (1,1,^^^!¤^^^!) do endlocal%$n1c%
%=         =%set ?=^^^!^&setlocal enableDelayedExpansion%=                             outer delay       =%%$n1c%
%=         =%set _=%%r%$n1c%
%=         =%call set "_=%%^^^_:3§3=3§3^^^!%%"^&if defined ? (%=                       deco     94, 33   =%^
%=            =%set "_=^!_:9§4=^^^!"^&set _=^^^!_:3§3=^^^!^
%=         =%)else set "_=^!_:9§4=^^^^^!"^&set "_=^!_:3§3=^^^!"%$n1c%
%=         =%set _=^^^!_:""=^^^"^^^!%=                                                 code pst 94, 33   =%%$n1c%
%=         =%!forQ_! ("^!$lf^!") do set _=^^^!_:l§f=%%~?^^^!%=                         deco $lf          =%%$n1c%
%=         =%set _=^^^!_:§§=§^^^!%=                                                    coded mark unset  =%%$n1c%
%=         =%!forLineR_! ("^!_^!") do endlocal^&set "%%r"^^^!^&setlocal enableDelayedExpansion%$n1c%
%=      =%endlocal)^
!==^^^!^
%=   =%)else setlocal enableDelayedExpansion^&set º="
:: )
2>nul %macroEnd%
%endlocalR_% (
%$%
)
to this;

Code: Select all

::--------------------------------------------------------------------------------------------------------------------------
set "$defines=endlocal_"
::--------------------------------------------------------------------------------------------------------------------------
:: last updated       : 2013^12^01
:: support            : naDelayed, disDelayed gives encoded ( $cr, $lf ), enaDelayed supports ( $cr, $lf )
::
::   -the number of optional endlocals is the first parameter.
::   -multiple variables can be returned at once.
::   -the maximum return size which is 8k is the maximum of the sum of all sizes of all variables to return.
::
set ^"$usage.%$defines%=^
%=   =% use: ( %%endlocal_%% #eCount, #var, #etc.. )%$n1c%
%=   =% err: Unaffected, panic otherwise"
::
2>nul ( %macroStart_% enableDelayedExpansion )
:: (
%=   =%set ^"!$defines!=!forQ_! (1,2) do if %%?==2 (^
!==^^^!^
%=      =%set ¤=^&!forQ_! (^^^!º^^^!) do if ^^^!¤^^^!.==. set/a¤=0%%~? 2^>nul^&^&(%=   endlocal count    =%^
%=         =%call set º=%%º:*^^^!¤^^^!=%%^&set/a¤+=1^
%=      =%)^|^|set/a¤=2%$n1c%
%=      =%set _=^&!forQ_! (^^^!º^^^!) do (^
%=         =%set ?=^^^!%%~?^^^!^&if defined ? (^
%=            =%set ?=^^^!?:§=§0^^^!%=                                                 coded mark set    =%%$n1c%
%=            =%set ?=^^^!?:^^^^!$lf!!$lf!=§0l^^^!%=                                   enco $lf          =%%$n1c%
%=            =%for /f %%1 in ("^!$cr^! ") do set ?=^^^!?:%%1=§0c^^^!%=                enco $cr          =%^
%=         =%)%$n1c%
%=         =%set _=^^^!_^^^!§l%%~?=^^^!?^^^!%=                                         coded add $lf     =%^
%=      =%)%$n1c%
%=      =%set _=^^^!_:^^^"=""^^^!%=                                                    code pre 94, 33   =%%$n1c%
%=      =%set "_=^!_:^^^=§9^!"^&call set "_=%%^^^_:^^^!=§3%%"%=                        enco     94, 33   =%%$n1c%
%=      =%for /f delims^^^^= %%r in ("^!_:~2^!") do (^
%=         =%!forC_! (1,1,^^^!¤^^^!) do endlocal%$n1c%
%=         =%set ?=^^^!^&setlocal enableDelayedExpansion%=                             outer delay       =%%$n1c%
%=         =%set _=%%r%$n1c%
%=         =%call set "_=%%^^^_:§3=§3^^^!%%"^&if ^^^!?^^^!.==. (%=                     deco     94, 33   =%^
%=            =%set "_=^!_:§9=^^^^^!"^&set "_=^!_:§3=^^^!"%$n1c%
%=         =%)else set "_=^!_:§9=^^^!"^&set _=^^^!_:§3=^^^!%$n1c%
%=         =%set _=^^^!_:""=^^^"^^^!%=                                                 code pst 94, 33   =%%$n1c%
%=         =%!forQ_! ("^!$lf^!") do set _=^^^!_:§l=%%~?^^^!%=                          deco $lf          =%%$n1c%
%=         =%set _=^^^!_:§0=§^^^!%=                                                    coded mark unset  =%%$n1c%
%=         =%!forLineR_! ("^!_^!") do (^
%=            =%endlocal%$n1c%
%=            =%if ^^^!.==. (^
%=               =%set _=%%r^^^!%$n1c%
%=               =%set _=^^^!_:§l=^^^^!$lf!!$lf!^^^!%=                                 deco $lf          =%%$n1c%
%=               =%for /f %%1 in ("^!$cr^! ") do set _=^^^!_:§c=%%1^^^!%=              deco $cr          =%%$n1c%
%=               =%set "^!_^!"^^^!^
%=            =%)else set "%%r"%$n1c%
%=            =%setlocal enableDelayedExpansion^
%=         =%)%$n1c%
%=         =%endlocal^
%=      =%)^
!==^^^!^
%=   =%)else setlocal enableDelayedExpansion^&set º="
:: )
2>nul %macroEnd%
%endlocalR_% (
%$%
)
I changed;

-FOR /F %%R in ("!CR! #") DO set "%%~N=!%%~N:""r=%%~R!"%\n%
to FOR /F %%R in ("!CR! ") DO set "%%~N=!%%~N:""r=%%~R!"%\n%

-the encoding mark §0 which is sometimes shorter than §§ due to better encrypt, §l instead of l§f etc

-deco 94, 33

A quick verify seems to be bugfree

Code: Select all

set $var0=
setlocal disableDelayedExpansion
setlocal enableDelayedExpansion
setlocal enableDelayedExpansion
:: (
   set "$var0="sim!$lf!sam!$cr!^^^^"ple^^" !
   set "$var1=;:()*<=>^^?^!" !
   %nechon_%
   set $var0
   set $var1
:: )
( %endlocal_% 1, $var0, $var1 )
%echon_%
set $var0
set $var1
endlocal
%echon_%
set $var0
set $var1
endlocal
--- output ---

Code: Select all

$var0="sim
^"ple^
$var1=;:()*<=>^?!

$var0="sim
^"ple^
$var1=;:()*<=>^?!

Omgevingsvariabele $var0 is niet gedefinieerd
Omgevingsvariabele $var1 is niet gedefinieerd
I haven't measured the number of bytes it occupies yet, hope it don't waste memory, having things fit within 8k is such a pain :(

Liviu
Expert
Posts: 470
Joined: 13 Jan 2012 21:24

Re: ReturnVar macro revisited

#24 Post by Liviu » 12 Feb 2014 22:56

dbenham wrote:I adapted jeb's original code to my style with the new optional arguments.

Below is a variation on your adaption of Jeb's returnVar macro. No offense to others ;-) but this was the shortest macro to practice on. The macro has only minor changes, and it may be a few bytes shorter, but otherwise it's functionally and speed-wise (hopefully) equivalent to the original. It appears to pass your testReturn.bat OK. Details are in the fully commented code posted next to the abbreviated version.

Code: Select all

@echo off
if not defined returnVar call :loadMacro&exit /b
if "%~1" neq ":returnDisabled" exit /b %2
setlocal enableDelayedExpansion
set "rtn=!%2!"
set "rtn=!rtn:%%=%%3!"
set "rtn=!rtn:""n=%%~L!"
set "rtn=!rtn:""r=%%~R!"
set "rtn=!rtn:""q=%%4!"
for /f "usebackq tokens=1,2" %%3 in (' %% ^" ') do (
  endlocal
  set "%2=%rtn:~1%"
  exit /b %%X
)

:loadMacro
if "!" equ "" (
  >&2 echo ERROR: Delayed expansion must be off when loading the returnVar macro.
  exit /b 1
)
set LF=^


set ^"\n=^^^%LF%%LF%^%LF%%LF%^^"
for /F "usebackq delims= " %%C in (`copy /z "%~f0" nul`) do set "CR=%%C"

set ^"returnVar=for %%# in (1 2) do if %%#==2 (setlocal enableDelayedExpansion%\n%
for /f "tokens=1-4" %%1 in ("!returnVar.args!") do (%\n%
  set "rtn=x!%%2!"%\n%
  set /a err=!errorlevel!, local=1%\n%
  for %%A in ("%%~3" "%%~4") do if %%A neq "" set /a "%%~A"%\n%
  for %%R in ("!CR!") do for %%L in ("!LF!") do for %%X in (!err!) do (%\n%
    set rtn=!rtn:^"=""q!%\n%
    set rtn=!rtn:%%~R=""r!%\n%
    set rtn=!rtn:%%~L=""n!%\n%
    set rtnDis=!rtn!%\n%
    set "rtn=!rtn:^=^^!"%\n%
    set path=%\n%
    set pathExt=;%\n%
    call set "rtn=%%rtn:^!=""c^!%%"%\n%
    set "rtn=!rtn:""c=^!"%\n%
    for /f "delims=" %%D in ("!rtnDis!") do for /f "delims=" %%E in ("!rtn!") do (%\n%
      for /l %%n in (0 1 !local!) do endlocal%\n%
      if "!"=="" (%\n%
        set "%%1=%%E" !%\n%
        set %%1=!%%1:""n=%%~L!%\n%
        set %%1=!%%1:""r=%%~R!%\n%
        set %%1=!%%1:^""q="!%\n%
        set %%1=!%%1:~1!%\n%
        if "%%X" equ "0" (call ) else if "%%X" equ "1" (call) else call "%~f0" :exitErr %%X%\n%
      ) else (%\n%
        set %%1=%%D%\n%
        call "%~f0" :returnDisabled %%1%\n%
      )%\n%
    )%\n%
  )%\n%
)) else set returnVar.args="

exit /b 0

Same code as above, now with full comments... It's not the first time I "parse" this macro, but it's all those less-than-obvious details that never seem to make it into my long term memory. Maybe this will help me remember faster next time, or perhaps will help someone else follow the mechanics more readily.

Code: Select all

@echo off
if not defined returnVar call :loadMacro&exit /b
if "%~1" neq ":returnDisabled" exit /b %2
setlocal enableDelayedExpansion
set "rtn=!%2!"
set "rtn=!rtn:%%=%%3!"
@rem [12]
set "rtn=!rtn:""n=%%~L!"
@rem [13] :: set "rtn=!rtn:""r=%%4!"
set "rtn=!rtn:""r=%%~R!"
@rem [13] :: set "rtn=!rtn:""q=%%~5!"
set "rtn=!rtn:""q=%%4!"
@rem [e] [13] [14] :: for /f "tokens=1-4" %%3 in (^"%% !CR! """") do (
for /f "usebackq tokens=1,2" %%3 in (' %% ^" ') do (
  endlocal
@rem [f] [1] why :~1
  set "%2=%rtn:~1%"
  exit /b %%X
)

:loadMacro
if "!" equ "" (
@rem [0] :: >2 echo ...
  >&2 echo ERROR: Delayed expansion must be off when loading the returnVar macro.
  exit /b 1
)
set LF=^


set ^"\n=^^^%LF%%LF%^%LF%%LF%^^"
for /F "usebackq delims= " %%C in (`copy /z "%~f0" nul`) do set "CR=%%C"

set ^"returnVar=for %%# in (1 2) do if %%#==2 (setlocal enableDelayedExpansion%\n%
for /f "tokens=1-4" %%1 in ("!returnVar.args!") do (%\n%
@rem [a] [1] why 'x' %\n%
  set "rtn=x!%%2!"%\n%
@rem [2] [3] :: set /a "err=!errorlevel!, local=1"%\n%
  set /a err=!errorlevel!, local=1%\n%
  for %%A in ("%%~3" "%%~4") do if %%A neq "" set /a "%%~A"%\n%
@rem [4] :: set /a "local+=1"%\n%
@rem [5]%\n% :: for /f %%R in ("!CR!!CR!") do for %%L in ("!LF!") do for %%X in (!err!) do (%\n%
  for %%R in ("!CR!") do for %%L in ("!LF!") do for %%X in (!err!) do (%\n%
@rem [6] [7] :: set ^"rtn=!rtn:"=""q!"%\n%
    set rtn=!rtn:^"=""q!%\n%
@rem [5] [3] :: set "rtn=!rtn:%%R=""r!"%\n%
    set rtn=!rtn:%%~R=""r!%\n%
@rem [3] :: set "rtn=!rtn:%%~L=""n!"%\n%
@rem     :: set "rtnDis=!rtn!"%\n%
    set rtn=!rtn:%%~L=""n!%\n%
    set rtnDis=!rtn!%\n%
@rem [b] [8]%\n%
    set "rtn=!rtn:^=^^!"%\n%
    set path=%\n%
    set pathExt=;%\n%
@rem [9]%\n%
    call set "rtn=%%rtn:^!=""c^!%%"%\n%
    set "rtn=!rtn:""c=^!"%\n%
@rem [c] [10] :: for /f "delims=" %%D in (""!rtnDis!"") do for /f "delims=" %%E in (""!rtn!"") do (%\n%
    for /f "delims=" %%D in ("!rtnDis!") do for /f "delims=" %%E in ("!rtn!") do (%\n%
@rem [4] :: for /l %%n in (1 1 !local!) do endlocal%\n%
      for /l %%n in (0 1 !local!) do endlocal%\n%
      if "!"=="" (%\n%
@rem [10] [11] :: set "%%1=%%~E" !%\n%
        set "%%1=%%E" !%\n%
@rem [3] :: set "%%1=!%%1:""n=%%~L!"%\n%
        set %%1=!%%1:""n=%%~L!%\n%
@rem [5] [3] :: set "%%1=!%%1:""r=%%R!"%\n%
        set %%1=!%%1:""r=%%~R!%\n%
@rem [7] :: set ^"%%1=!%%1:""q="!"%\n%
        set %%1=!%%1:^""q="!%\n%
@rem [d] [1] [3] :: set "%%1=!%%1:~1!"%\n%
        set %%1=!%%1:~1!%\n%
        if "%%X" equ "0" (call ) else if "%%X" equ "1" (call) else call "%~f0" :exitErr %%X%\n%
      ) else (%\n%
@rem [10] [3] :: set "%%1=%%~D"%\n%
        set %%1=%%D%\n%
        call "%~f0" :returnDisabled %%1%\n%
      )%\n%
    )%\n%
  )%\n%
)) else set returnVar.args="
exit /b 0

:: :loadMacro defines CR, LF, \n, returnVar variables in caller's context
::
:: %returnVar%  outVar  inVar  [local=#]  [err=#]
:: - 'outVar' is the variable being returned/set in the outer context
:: - 'inVar' is the local/inner context variable whose value is returned
::   (must not be 'returnVar.args' since that name is used by %returnVar%)
:: - optional 'local=#' is the nesting level, default 1
::   %returnVar% will execute endlocal # times
:: - optional 'err=#' is the errorlevel to be returned, default 0
:: - order of optional arguments is irrelevant, names must be 'local'/'err'
:: - %returnVar% does not modify the outer environment except for 'outVar'
::
:: - sample input of 'a " <cr> b <lf> ^ c ! d % e' traced at @rem [a]-[f]
:: [a] !%2!     a " <cr> b <lf> ^ c ! d % e      > input
::     !rtn!    xa " <cr> b <lf> ^ c ! d % e       prepend char
:: [b] !rtnDis! xa ""q ""r b ""n ^ c ! d % e       encode " <cr> <lf>
:: [c] !rtn!    xa ""q ""r b ""n ^^ c ^! d % e     edx: escape ^ !
:: [d] !%1!     xa " <cr> b <lf> ^ c ! d % e     <      return :~1
:: [e] !rtn!    xa %4 %~R b %~L ^ c ! d %3 e       ddx: recode " <cr> <lf> %
:: [f] %rtn%    xa " <cr> b <lf> ^ c ! d % e     <      return :~1
:: ____________________________________________________________________________
::
:: [0] >2 typo, fixed to >&2
::
:: [1] the 'x' is prepended just as a guard against empty strings
::     removing the 'x' and the two :~1 works for all non-empty strings
::
:: [2] local err/local override any namesake variables in caller context
::     even if defaulted/not passed as an argument
::
:: [3] these assignments don't technically require quotes
::     and the %\n% guards against accidental whitespace at the end
::
:: [4] can modify the endlocal loop, instead of incrementing 'local'
::
:: [5] CR in a for/f loop needs be doubled, LF requires a plain non-/f loop
::     http://www.dostips.com/forum/viewtopic.php?p=6930#p6930 !?
::
:: [6] the " -> ""q substitution is losslessly reversible
::   - the double "" makes the entire string parse as unquoted as far as
::     embedded special chars (^&<|>) go
::     (hint: """"q would work as well, while "q, """q, or $$q would not)
::   - ""q also guarantees that no other ""? patterns remain in the string
::
:: [7] can remove outer quotes, but then need to escape first inner quote
::     to preserve the quote balance
::
:: [8] this also works, but is longer and less readable
::     set rtn=!rtn:^^^^=^^^^^^^^!%\n%
::
:: [9] this relies on 'call' doubling the ^ carets inside quoted strings
::     to compensate for '!' halving them under enableDelayedExpansion
::     so the outer quotes are required
::   - and because of that it doesn't seem to be possible to substitute
::     ! -> ^! in one single step, since escaping ^! needs a single caret
::   - also, the two step substitution relies on an unused substring like
::     ""c to be guaranteed available
::
::[10] no need to double the outer double quotes 'in (""!rtn*!"")'
::     since the extra 'x' character [1] is prepended not appended
::  -  prepending the 'x' and trimming with ':~1' works with
::     just 'in ("!rtn*!")', and using %%D inside the loop
::  -  appending the 'x' and removing it with ':~0,-1', instead,
::     would require the double double-quotes 'in (""!rtn*!"")',
::     and using %%~D inside the loop,
::     otherwise lines starting with a " quote or ; semicolon got trashed
::
::[11] the trailing ! ensures that carets are always halved even if
::     the string itself contains no ! so the outer quotes are required
::
::[12] %%~L inherited from caller, becomes available inside the for loop
::     for/? - for variables are single-letter, case sensitive, *global*
::
::[13] can use inherited %%~R instead of local loop variable
::     which reduces the necessary for/f tokens to 1-2
::
::[14] original 'for /f "tokens=1-4" %%3 in (^"%% !CR! """") do'
::     only used tokens=1-3 though 4 were actually declared
::   - both following alternatives work as well
::     'for /f "tokens=1-3" %%3 in (" %% !CR! "^"" ") do'
::     'for /f "usebackq tokens=1-3" %%3 in (' %% !CR! ^" ') do'
::     with the latter returning a single (unquoted) quote in %%5

Any critique much appreciated, about the suggested (cosmetic) code changes, the additional comments, or anything else I missed or may have misunderstood.

Liviu

P.S. [ EDIT ] Corrected comment line ":: - optional 'err=#' is the errorlevel to be returned, default !errorlevel!" to "default 0" since the errorlevel would have been reset to 0 by 'setlocal' at that point.

[ 2nd EDIT ] Updated comment line "::[10] no need to double the outer double quotes 'in (""!rtn*!"")' !?" with a guess as to where the double double-quoting came from.
Last edited by Liviu on 17 May 2014 23:12, edited 2 times in total.

jeb
Expert
Posts: 1041
Joined: 30 Aug 2007 08:05
Location: Germany, Bochum

Re: ReturnVar macro revisited

#25 Post by jeb » 13 Feb 2014 03:23

Hi Liviu,

Liviu wrote:Any critique much appreciated, about the suggested (cosmetic) code changes, the additional comments, or anything else I missed or may have misunderstood.

You used two versions of the macro, one commented one and one uncommented.

I start a new thread to show a way how to use macros with comments without increasing the macro size.
Comments without increasing macro size

jeb

Liviu
Expert
Posts: 470
Joined: 13 Jan 2012 21:24

Re: ReturnVar macro revisited

#26 Post by Liviu » 21 Feb 2014 00:43

jeb wrote:Currently I suppose to found a solution, that can return to a disabled context each character, even linefeed and carriage return.
But not both at the same time :wink:
Either linefeeds or carriage returns are allowed in a string.

I will post it when it works.
I am very much looking forward to that.

For a baseline, below is a macro (heavily based on the generic ones) that doesn't allow either CR or LF - but is otherwise simpler, and is self-contained without external "%~f0" calls.

Code: Select all

@echo off & if defined retLocal goto :eof

if "!" equ "" (
  >&2 echo ERROR: Delayed expansion must be off when loading the retLocal macro.
  exit /b 1
)

set LF=^


set ^"\n=^^^%LF%%LF%^%LF%%LF%^^"

:: macro to return CR/LF-free variable across endlocal barriers
::
:: syntax:  %retLocal%[*cnt] outVar[=inVar]
:: - 'outVar' is the variable being returned/set in the outer context
:: - 'inVar' is the local/inner context variable whose value is returned
:: - optional '=inVar' defaults to 'outVar=outVar'
:: - optional '*cnt' is the nesting level, default 1 (times 'endlocal' called)
::
:: e.g.  %retLocal% var        copies inner 'var' across 1 endlocal barrier
::                             to namesake 'var' in the outer context
::
::       %retLocal%*2 out=in   copies inner 'in' across 2 endlocal barriers
::                             to 'out' in the outer context
::
::       %retLocal%*0 out=in   copies 'in' to 'out' within the same context
::                             like 'set out=%in%' but safe against embedded
::                             quotes, and problem characters except CR/LF
::
:: ! names of variables 'outVar', 'inVar' are expected to be plain quote-less
::   well behaved strings, with no funny (<|>^%!) characters
:: ! 'inVar' name must not be 'cd' since that's used by %retLocal% internally
:: ! outer environment is not modified, except for '%retLocal%*0' with '*0'
::   de-nesting, which clears any 'cd' explicitly set within the same context
:: ! inner errorlevel is not preserved - but the caller can save it on its own
::   if needed via '(%retLocal% outVar=inVar) & set outErr=%errorlevel%'

set ^"retLocal=for %%# in (1 2) do if %%#==2 (%\n%
  setlocal enableDelayedExpansion%\n%
  for /f "tokens=1-2" %%U in ("!cd!") do (%\n%
  for /f "tokens=2 delims=*" %%0 in ("%%U*%%U") do (%\n%
    for /f "tokens=1-2 delims==" %%1 in ("%%V=%%V") do (%\n%
      if not defined %%2 (%\n%
        for /l %%N in (0 1 %%0) do endlocal%\n%
        set "%%1="%\n%
      ) else for /f "delims=" %%D in ("!%%2!") do (%\n%
        set "e=!%%2!"%\n%
        set ^"e=!e:"=""q!"%\n%
        set "e=!e:^=^^!"%\n%
        set "path="%\n%
        set "pathExt=;"%\n%
        call set "e=%%e:^!=""c^!%%"%\n%
        set "e=!e:""c=^!!"%\n%
        for /f "delims=" %%E in ("!e!") do (%\n%
          for /l %%N in (0 1 %%0) do endlocal%\n%
          if "!"=="" (%\n%
            set "%%1=%%E"!%\n%
            set ^"%%1=!%%1:""q="!"%\n%
          ) else (%\n%
            set "%%1=%%D"%\n%
    ))))%\n%
    if "%%~0" equ "0" set "cd="%\n%
))) else set cd=*1^"

exit /b 0

Quick notes:
- macro is named 'retLocal' as to not be confused with the 'returnVar' used here for arbitrary unrestricted strings;
- '*cnt' syntax looks friendlier to me than appending the count at the end;
- 'outVar[=invar]' syntax leaves the door open for future 'outVar1[=invar1] outVar2[=invar2] ...' once someone figures out how to make _that_ work ;-)
- the sticking point in this approach is escaping ! when returning to enableDelayedExpansion - that's what requires the 'call set' line, and in turn that percent expansion requires the previous substitutions of quotes; if some trick were found to make the equivalent of '!var:^!=^^!!' work without percent expansion and without an intermediate ""c variable, the code could be a few good lines shorter.

Liviu

[ EDIT ] P.S. Posted code had a flaw in certain failure cases. For example an input of just CR caused it to not just fail to return the correct value, but actually fail to run the endlocal loop, thus leaving the caller inside an unexpected nested context. While the macro does not support CRs, nor claimed to, it could still fail more graciously in such cases. The variation below does that i.e. always unrolls the given endlocal levels (even if it returns an empty string for CRs).

Code: Select all

@echo off & if defined retLocal goto :eof

if "!" equ "" (
  >&2 echo ERROR: Delayed expansion must be off when loading the retLocal macro.
  exit /b 1
)

set LF=^


set ^"\n=^^^%LF%%LF%^%LF%%LF%^^"
for /F "usebackq delims= " %%C in (`copy /z "%~f0" nul`) do set "CR=%%C"

set ^"retLocal=for %%# in (1 2) do if %%#==2 (%\n%
  setlocal enableDelayedExpansion%\n%
  for /f "tokens=1-2" %%U in ("!cd!") do ^
  for /f "tokens=2 delims=*" %%0 in ("%%U*%%U") do (%\n%
    for /f "tokens=1-2 delims==" %%1 in ("%%V=%%V") do ^
    for /f "usebackq delims=" %%D in ('"!%%2!"') do (%\n%
      set "e=!%%2!"%\n%
      if defined e (%\n%
        set ^"e=!e:"=""!"%\n%
        set "e=!e:^=^^^^!"%\n%
        set "path="%\n%
        set "pathExt=;"%\n%
        call set "e=%%e:^!=^^^!%%"%\n%
        set "e=!e:^^=^!"%\n%
      )%\n%
      for /f "usebackq delims=" %%E in ('"!e!"') do (%\n%
        for /l %%N in (0 1 %%0) do endlocal%\n%
        if "!"=="" (%\n%
          set "%%1=%%~E"!%\n%
          if defined %%1 set ^"%%1=!%%1:""="!"%\n%
        ) else (%\n%
          set "%%1=%%~D"%\n%
  )))%\n%
  if "%%~0" equ "0" set "cd="%\n%
)) else set cd=*1^"

exit /b 0

And for a slightly different approach in the CR/LF-free case, here is another alternative - mostly of technical interest (unquoted 'call set'), since practically it runs more substitutions than the above.

Code: Select all

@echo off & if defined retLocal goto :eof

if "!" equ "" (
  >&2 echo ERROR: Delayed expansion must be off when loading the retLocal macro.
  exit /b 1
)

@rem single linefeed char 0x0A (two blank lines required below)
set LF=^


@rem linefeed macros
set ^"/n=^^^%LF%%LF%^%LF%%LF%"
set ^"//n=^^^^^^%/n%%/n%^^%/n%%/n%"
set ^"///n=^^^^^^^^^^^^%//n%%//n%^^^^%//n%%//n%"

@rem newline macros (linefeed + line continuation)
set ^"\n=%//n%^^"
set ^"\\n=%///n%^^"

set ^"retLocal=for %%# in (1 2) do if %%#==2 (%\n%
  setlocal enableDelayedExpansion%\n%
  for /f "tokens=1-2" %%U in ("!cd!") do ^
  for /f "tokens=2 delims=*" %%0 in ("%%U*%%U") do (%\n%
    for /f "tokens=1-2 delims==" %%1 in ("%%V=%%V") do ^
    for /f "usebackq delims=" %%D in ('"!%%2!"') do (%\n%
      set "e=!%%2!"%\n%
      if defined e (%\n%
        for /f usebackq %%Q in ('%\n%
          ^^^^^^=^^^^^^^^^^^^^^^^ %\\n%
          ^^^"^^=^^^^^^^" %\\n%
          ^^^<^^=^^^^^^^< %\\n%
          ^^^>^^=^^^^^^^> %\\n%
          ^^^|^^=^^^^^^^| %\\n%
          ^^^&^^=^^^^^^^& %\\n%
        ') do set "e=!e:%%Q!"%\n%
        set "path=" ^& set "pathExt=;"%\n%
        call set e=%%e:^^^^!=^^^^^^^^^^^^!%%%\n%
      )%\n%
      for /f "usebackq delims=" %%E in ('"!e!"') do (%\n%
        for /l %%N in (0 1 %%0) do endlocal%\n%
        if "!"=="" (set "%%1=%%~E"!) else (set "%%1=%%~D")%\n%
  ))%\n%
  if "%%~0" equ "0" set "cd="%\n%
)) else set cd=*1^"

exit /b 0
Last edited by Liviu on 27 Feb 2014 01:54, edited 1 time in total.

Liviu
Expert
Posts: 470
Joined: 13 Jan 2012 21:24

Re: ReturnVar macro revisited

#27 Post by Liviu » 27 Feb 2014 01:13

Along the same line as previous post, below is a 'retLocal' macro with CR support (but no LF), also self-contained and without external "%~f0" calls. Comments welcome, since I am still interested in "special case" shortcuts for the "most general case" macro. To make this very clear, I find the generic 'returnVar' to be an admirable tour de force. But there are situations where the input is known to not contain certain characters like LF, and a macro could take advantage of that knowledge.

Code: Select all

@echo off & if defined retLocal goto :eof

if "!" equ "" (
  >&2 echo ERROR: Delayed expansion must be off when loading the retLocal macro.
  exit /b 1
)

@rem single linefeed char 0x0A (two blank lines required below)
set LF=^


@rem linefeed macros
set ^"/n=^^^%LF%%LF%^%LF%%LF%"
set ^"//n=^^^^^^%/n%%/n%^^%/n%%/n%"
set ^"///n=^^^^^^^^^^^^%//n%%//n%^^^^%//n%%//n%"

@rem newline macros (linefeed + line continuation)
set ^"\n=%//n%^^"
set ^"\\n=%///n%^^"
set ^"\\\n=%////n%^^"

@rem single carriage-return char 0x0D
for /F "usebackq delims= " %%C in (`copy /z "%~f0" nul`) do set "CR=%%C"

set ^"retLocal=for %%# in (1 2) do if %%#==2 (%\n%
  setlocal enableDelayedExpansion%\n%
  for /f "tokens=1-2" %%U in ("!cd!") do ^
  for /f "tokens=2 delims=*" %%0 in ("%%U*%%U") do (%\n%
    for /f "tokens=1-2 delims==" %%1 in ("%%V=%%V") do ^
    for /f "usebackq delims=" %%D in ('"!%%2!"') do ^
    for %%R in ("!CR!") do (%\n%
      set "e=!%%2!"%\n%
      if defined e (%\n%
        for /f usebackq %%Q in ('%\n%
          ^^^"^^=^^^"^^^"q %\\n%
          %%~R^^=^^^"^^^"r %\\n%
        ') do set "e=!e:%%Q!"%\n%
        set "path=" ^& set "pathExt=;"%\n%
        set "e=!e:^=^^^^!"%\n%
        call set "e=%%e:^!=^^^!%%"%\n%
        set "e=!e:^^=^!"%\n%
      )%\n%
      for /f "usebackq delims=" %%E in ('"!e!"') do (%\n%
        for /l %%N in (0 1 %%0) do endlocal%\n%
        if "!"=="" (%\n%
          set "%%1=%%~E"!%\n%
          if defined %%1 for /f usebackq %%Q in ('%\n%
            ^^^"^^^"r^^=%%~R %\\n%
            ^^^"^^^"q^^=^^^" %\\n%
          ') do set ^"%%1=!%%1:%%Q!^"%\n%
        ) else (%\n%
          set "%%1=%%~D"%\n%
  )))%\n%
  if "%%~0" equ "0" set "cd="%\n%
)) else set cd=*1^"

exit /b 0

Liviu

jeb
Expert
Posts: 1041
Joined: 30 Aug 2007 08:05
Location: Germany, Bochum

Re: ReturnVar macro revisited

#28 Post by jeb » 27 Feb 2014 02:49

Liviu wrote:Along the same line as previous post, below is a 'retLocal' macro with CR support (but no LF), also self-contained and without external "%~f0" calls.

Dave's and my macro contain the "call %~f0" but it will only be used when a CR AND also a Linefeed was detected, else one of the shortcuts will be used.

So I can't see any relevant drawbacks of the "general case macros"

Btw.
Liviu wrote:if "!" equ "" (
>&2 echo ERROR: Delayed expansion must be off when loading the retLocal macro.
exit /b 1
)

You build here a macro that can return content over an endlocal barrier :!:
Why you don't use it? :D

Code: Select all

setlocal DisableDelayedExpansion
set myReturnMacro=....my macro definition ....

%myReturnMacro% myReturnMacro myReturnMacro


jeb

Liviu
Expert
Posts: 470
Joined: 13 Jan 2012 21:24

Re: ReturnVar macro revisited

#29 Post by Liviu » 28 Feb 2014 00:08

jeb wrote:Dave's and my macro contain the "call %~f0" but it will only be used when a CR AND also a Linefeed was detected, else one of the shortcuts will be used.

So I can't see any relevant drawbacks of the "general case macros"

Don't get me wrong. I certainly appreciate the mastery of the general case macros. But there still are use cases where the full generality is not needed, and those cases could still take advantage of whatever limitations are known to exist in the input data. It may be just one less substitution, or some other code shortcut, and it may not even make any obvious difference performance-wise. Still, from a minimalistic standpoint, why pay for something you know in advance doesn't get used.

As an extreme example in the opposite direction, my default mode is 'disableDelayedExpansion'. That's the first thing needed in order to retrieve command line parameters with some degree of sanity against ^!, and also run well behaved 'for' loops later. Of course, I may switch to 'enableDelayedExpansion' as necessary, but that's usually just for local manipulations. So, to me at least, the following very restricted macro is useful in certain cases - and does not carry any of the unneeded baggage that the completely generic macros do.

Code: Select all

:: macro to return LF-free variable to 'disableDelayedExpansion' context
::
:: syntax:  %retLocal.ddx%[*cnt] outVar[=inVar]
:: - 'outVar' is the variable being returned/set in the outer context
:: - 'inVar' is the local/inner context variable whose value is returned
:: - optional '=inVar' defaults to 'outVar=outVar'
:: - optional '*cnt' is the nesting level, default 1 (times 'endlocal' called)

set ^"retLocal.ddx=for %%# in (1 2) do if %%#==2 (%\n%
  setlocal enableDelayedExpansion%\n%
  for /f "tokens=1-2" %%U in ("!cd!") do ^
  for /f "tokens=2 delims=*" %%0 in ("%%U*%%U") do (%\n%
    for /f "tokens=1-2 delims==" %%1 in ("%%V=%%V") do ^
    for /f "usebackq delims=" %%D in ('"!%%2!"') do (%\n%
      for /l %%N in (0 1 %%0) do endlocal%\n%
      set "%%1=%%~D"%\n%
    )%\n%
    if "%%~0" equ "0" set "cd="%\n%
)) else set cd=*1^"


jeb wrote:You build here a macro that can return content over an endlocal barrier :!:
Why you don't use it? :D
Nice. Too bad the macros I've posted don't do LFs ;-) but it's still a great point to make for the generic ones.

Liviu

spoutnik
Posts: 1
Joined: 03 Jun 2017 03:58

Re: ReturnVar macro revisited

#30 Post by spoutnik » 03 Jun 2017 04:20

dbenham wrote:
[...]

macro.rtnJeb.bat

Code: Select all

@echo off
if not defined returnVar call :loadMacro&exit /b
if "%~1" neq ":returnDisabled" exit /b %2
setlocal enableDelayedExpansion
set "rtn=!%2!"
set "rtn=!rtn:%%=%%3!"
set "rtn=!rtn:""n=%%~L!"
set "rtn=!rtn:""r=%%4!"
set "rtn=!rtn:""q=%%~5!"
for /f "tokens=1-4" %%3 in (^"%% !CR! """") do (
  endlocal
  set "%2=%rtn:~1%"
  exit /b %%X
)


:loadMacro
if "!" equ "" (
  >2 echo ERROR: Delayed expansion must be off when loading the returnVar macro.
  exit /b 1
)
set LF=^


set ^"\n=^^^%LF%%LF%^%LF%%LF%^^"
for /F "usebackq delims= " %%C in (`copy /z "%~f0" nul`) do set "CR=%%C"

set ^"returnVar=for %%# in (1 2) do if %%#==2 (setlocal enableDelayedExpansion%\n%
for /f "tokens=1-4" %%1 in ("!returnVar.args!") do (%\n%
  set "rtn=x!%%2!"%\n%
  set /a "err=!errorlevel!, local=1"%\n%
  for %%A in ("%%~3" "%%~4") do if %%A neq "" set /a "%%~A"%\n%
  set /a "local+=1"%\n%
  for /f %%R in ("!CR!!CR!") do for %%L in ("!LF!") do for %%X in (!err!) do (%\n%
    set ^"rtn=!rtn:"=""q!"%\n%
    set "rtn=!rtn:%%R=""r!"%\n%
    set "rtn=!rtn:%%~L=""n!"%\n%
    set "rtnDis=!rtn!"%\n%
    set "rtn=!rtn:^=^^!"%\n%
    set "path="%\n%
    set "pathExt=;"%\n%
    call set "rtn=%%rtn:^!=""c^!%%"%\n%
    set "rtn=!rtn:""c=^!"%\n%
    for /f "delims=" %%D in (""!rtnDis!"") do for /f "delims=" %%E in (""!rtn!"") do (%\n%
      for /l %%n in (1 1 !local!) do endlocal%\n%
      if "!"=="" (%\n%
        set "%%1=%%~E" !%\n%
        set "%%1=!%%1:""n=%%~L!"%\n%
        set "%%1=!%%1:""r=%%R!"%\n%
        set ^"%%1=!%%1:""q="!"%\n%
        set "%%1=!%%1:~1!"%\n%
        if "%%X" equ "0" (call ) else if "%%X" equ "1" (call) else call "%~f0" :exitErr %%X%\n%
      ) else (%\n%
        set "%%1=%%~D"%\n%
        call "%~f0" :returnDisabled %%1%\n%
      )%\n%
    )%\n%
  )%\n%
)) else set returnVar.args="

exit /b 0


[...]

Dave Benham


Hi,

I like the 'macro.rtnJeb.bat' quite much.
It works quite well at having ONE variable come across the ENDLOCAL barrier, and even for variable containing LF.

However one drawback is that it %returnVar% only ONE variable.

Would there be an extended version of the 'macro.rtnJeb.bat' that would allow to make SEVERAL variables come accross the barrier?

Currently I use:
%returnVar% varInOutsideContext varInInsideContext

And I would like to have something like:
%returnVar% var01InOutsideContext=var01InInsideContext var02InOutsideContext=var02InInsideContext (and so on)
or even force identical name such as:
%returnVar% var01 var02 (and so on)
where var01, var02 and so on are variables declared with same name in the Inside and the Outside context

Thank you very much in advance.

spoutnik

Post Reply