pass-by-value return-by-reference for restricted strings

[Liviu]

Writing even a basic routine which takes a value as input and assigns it to a given variable as output doesn't seem to be entirely trivial. Not even if the input string is known to be moderately well behaved, in particular not contain quotes, and not contain control characters such as CR, LF, SUB etc. Incidentally, this class of strings is a superset of legal filenames, so routines which deal with (possibly wildcarded) paths would be ensured that their arguments fall in this category. Still, eligible strings may include other "poison characters" like ^%!(|) which is what makes things entertaining.

I am pretty sure this must have been discussed and beaten to death before, but the advice I've found most often was to pass by reference and use the most generic return which handles everything from unbalanced quotes to embedded line breaks - and because of that is unnecessarily complex in the case of strings known to be better behaved.

In the simplest case of disableDelayedExpansion it's still needed
- for the caller to double the %s in the passed value (since 'call' would otherwise halve and/or try to expand them as variables), and
- for the callee to halve the ^s inside the received quoted strings (since 'call' would have doubled them).

A minimal example of this "dance" would be the following...

Code: Select all

@rem assume disableDelayedExpansion
:caller  [in,ref] str
setlocal enableDelayedExpansion & set "v=!%1:%%=%%%%!"
endlocal & set "v=%v%"
call :callee "%v%" w
@rem at this point the value of variable %1 has been copied to variable w
goto :eof

:callee  [in,val] src  [out,ref] dst
setlocal disableDelayedExpansion
set "s=%~1"
set "s=%s:^^=^%"
endlocal & set "%2=%s%" & goto :eof

Below is a complete example that covers both disabled/enabled delayed expansion states, and also verifies the input/output values to match the expected test strings. Checked to run without errors in xp.sp3 and win7x64.sp1.

Code: Select all

@echo off & setlocal disableDelayedExpansion
for /f "usebackq tokens=1* delims= " %%a in ("%~f0") do (
  @rem read test strings off ::: comment lines at the bottom
  if "%%~a"==":::" set "u=%%~b" & call :test u
)
endlocal & goto :eof

:test  [in,ref] str
setlocal disableDelayedExpansion
setlocal enableDelayedExpansion & set "v=!%1:%%=%%%%!"
endlocal & set "v=%v%"
call :copy.ddx "%v%" w u
endlocal
setlocal enableDelayedExpansion
set "v=!%1:%%=%%%%!"
call :copy.edx "!v!" w u
endlocal
goto :eof

:copy.ddx  [in,val] src  [out,ref] dst  [in,ref] src  [ret] errorlevel  _______
setlocal disableDelayedExpansion
set "s=%~1"
set "s=%s:^^=^%"

call :comp s %3 %0 "- arg err" || (endlocal & set "%2=" & exit /b 1)
@rem argument verified ok, a real function would now process %s% further

endlocal & set "%2=%s%"
@rem dummy copy function expected to return identical string as inputted
call :comp %2 %3 "" "- ret err"
exit /b %errorlevel%

:copy.edx  [in,val] src  [out,ref] dst  [in,ref] src  [ret] errorlevel  _______
setlocal disableDelayedExpansion
set "s=%~1"
set "s=%s:^^=^%"

call :comp s %3 %0 "- arg err" || (endlocal & set "%2=" & exit /b 1)
@rem argument verified ok, a real function would now process %s% further

if "%s%" equ "%s:!=%" goto :copy.edx.ret
set "s=%s:^=^^%"
set "s=%s:!=^!%"
:copy.edx.ret
endlocal & set "%2=%s%"
@rem dummy copy function expected to return identical string as inputted
call :comp %2 %3 "" "- ret err"
exit /b %errorlevel%

:comp  [in,ref] src  [in,ref] dst  [in,val] tag  [in,val] error-tag  __________
setlocal enableDelayedExpansion
if "%~3" neq "" echo %~3  "!%1!"
if "!%1!" neq "!%2!" echo %~4  "!%2!" & endlocal & exit /b 1
endlocal & exit /b 0

::  strings are being read literally so ^%!(&)<|> need not be escaped  ________
::
::: c:\windows\temp\ntfs.ads:1
::: \\localhost\c$\windows\temp\.\(a&b%!*.??^
::: cmd /c echo /? 1>&2
::: * *. .* *.* ? ?. .? ?.? ?.? . .. ... ^ ^^ ^^^
::: & ^& ^^& | ^| ^^| < ^< ^^< > ^> ^^>
::: & && &&& | || ||| < << <<< > >> >>>
::: , ,, ,,, ; ;; ;;; ' '' ''' ` `` ``` ~ ~~ ~~~
::: + ++ +++ - -- --- / // /// \ \\ \\\ = == ===
::: ( ^( ^^( % %% %%% %%%% %cd% %%cd%% %%%%cd%%%%
::: ) ^) ^^) ! !! !!! !!!! !cd! !!cd!! !!!!cd!!!!
::: ) ^) ^^)
::: ) ^) ^^) !
::: ! ^! ^^! !! !^! ^!! ^!^!
::: !cd! !cd^! ^!cd! ^!cd^!
::: !!cd!! !!cd!^! !!cd^!! !!cd^!^! !^!cd!! !^!cd!^! !^!cd^!! !^!cd^!^!
::: ^!!cd!! ^!!cd!^! ^!!cd^!! ^!!cd^!^! ^!^!cd!! ^!^!cd!^! ^!^!cd^!! ^!^!cd^!^!
::: ) !
::: ) ^!
::: ^) !
::: ^) ^!
::: ^!
::: ^!^!
::: ^!cd^!
::: ^!^!cd^!^!
::: ) ^!
::: ) ^^!
::: ^) ^!
::: ^) ^^!
::: ^ ^^ ^^^ ^^^^ ^^^^^ ^^^^^^ ^^^^^^^^ ^^^^^^^^^

The code is written for readability, rather than real life performance, so I am not looking for tricks to eliminate the 'goto' in :call.edx for example, yet any critique is welcome about additional/failure test cases, or simplifying the logic, or different approaches.

Liviu

[jeb]

Hi Liviu,

the last discussion is at ReturnVar macro revisited

Where the problem is solved with a macro, so it's simple to return a variable from any delayed context to any other context, without any problems.

Code: Select all

setlocal DisableDelayedExpansion
set "myLocalVar=! ^ % &|<>"^! ^^ ^% ^&^|^<^>"
...
%ReturnVar% myLocal myResultVar

This even works inside a FOR loop or generall inside of parenthesis.

Not even if the input string is known to be moderately well behaved, in particular not contain quotes, and not contain control characters such as CR, LF, SUB

But even then you need to convert the variable first, to escape the percents.
To disallow CR,LF,SUB could be ok, but also disallow quotes is a bit too much.

But even without quotes you code doesn't work (or I don't understand how it should work).
How do you get over a endlocal barrier?

And with quotes it seems to be impossible to use the by value technic at all.

Code: Select all

set var="one" two
.... magic converting of var ...
call :someFunc %var%

I can't see any possible way to convert it a safe way.

Btw. I can't see any advantage of by-value vs by-ref, as I don't need any conversion with the by-ref method at all.

jeb

[Liviu]

the last discussion is at ReturnVar macro revisited

Thanks for the pointer. I'll need to study that some more, at first sight it looks like a handful of (slightly) different macros with no clear consensus on the final "winner"

Where the problem is solved with a macro, so it's simple to return a variable from any delayed context to any other context, without any problems.

Guess I wasn't too clear, but the point of my post was to work the mechanics of passing and returning in this decidedly simpler case, rather than solve a particular problem with a ready made macro written for the most general case.

Not even if the input string is known to be moderately well behaved, in particular not contain quotes, and not contain control characters such as CR, LF, SUB

But even then you need to convert the variable first, to escape the percents.
To disallow CR,LF,SUB could be ok, but also disallow quotes is a bit too much.

But even without quotes you code doesn't work (or I don't understand how it should work).
How do you get over a endlocal barrier?

Working on strings without quotes and control characters is a premise of this exercise. It is indeed more restrictive than, say, arbitrary strings. In fact, that was the main idea - see if those restrictions allow perhaps some shortcuts vs. the unrestricted general case. As for real world applicability or usefulness, such strings are not all that uncommon - all legal pathnames fall into this category.

Back to the code I posted, it does the necessary escaping before calling :copy.ddx/.edx, and it also returns the value across the endlocal barrier the old fashioned way. Moreover, the whole roundtrip is checked both for the input value being received correctly, and the output reference being assigned correctly. If you found a case where it fails, please share.

Liviu

[jeb]

Now, I understand your post

If you found a case where it fails, please share.

Sadly I didn't found any failure till now.
I suppose your code is stable, but still unpractical,
as you need to know if you need to use copy.edx or copy.ddx
and for the extra line before the call

jeb