foolproof counting of arguments

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Message
Author
Liviu
Expert
Posts: 470
Joined: 13 Jan 2012 21:24

Re: foolproof counting of arguments

#16 Post by Liviu » 24 Jan 2012 13:28

Yet another variation on the same theme... The code below collects all parameters in a temp file, with the stop condition based on the size of the file. Then, it reads the file back in one shot.

EDIT: code below has changed vs. the one posted earlier today. At the time I edited, there were no followups, so I hope no one minds too much. More details at the bottom.

2nd EDIT: nested the "do rem" parts inside a "setlocal disableextensions" block to protect against accidental expansion of %~a %~dpa etc.

Note that the enclosing (...) is still evaluated at the outer level with extensions enabled, which allows %* to work correctly, but the for loop executes with extensions disabled so %~a, %~dpa etc are no longer expanded.

Code: Select all

@echo off
setlocal disableDelayedExpansion
set "tab=   "

:: can't call with wacky %%*
:: must goto and rely on hardcoded return to :main
::
:: upon return, the following variables are set
::   args is the complete command line
::   argc is the number of arguments
::   arg1, arg2, etc are the individual arguments
::   argsLen, arg1len, arg2len etc are the respective string lengths
:: inside a nested setlocal enableDelayedExpansion
::
goto :getArgs

:: body of actual code would go under :main below
::
:main

echo.
echo ---- copied from %%? ----
echo.
echo args %tab% %tab%{ !args! }
for /l %%n in (1, 1, %argc%) do (
  echo arg%%n %tab% %tab%{ !arg%%n! }
)

:: this part is only needed to get the original cmd line offsets
:: e.g. in order to preserve whitespace with !args:~%arg2off%!
::
:: requires the args, argc, arg?len variables be set by :getArgs
:: sets arg?off upon return to the offset of arg? within args
::
call :getOffs

:: only uses arg1, arg2 etc for verification
::
echo.
echo ---- chopped off %%* ----
echo.
@rem entire command line
echo args %tab%~0,%argsLen% %tab%{ !args! }
set /a delimOff = 0
for /l %%n in (1, 1, %argc%) do (
  @rem whitespace before %%n'th argument
  set /a delimLen = !arg%%noff! - !delimOff!
  for /f "tokens=1*" %%p in ("!delimOff! !delimLen!") do (
    echo %tab%~!delimOff!,!delimLen! %tab%{ !args:~%%p,%%q! }
  )
  set /a delimOff = !arg%%noff! + !arg%%nLen!
  @rem %%n'th argument
  for /f "tokens=1*" %%p in ("!arg%%noff! !arg%%nlen!") do (
    echo arg%%n %tab%~!arg%%noff!,!arg%%nlen! %tab%{ !args:~%%p,%%q! }
    @rem reality check
    if not "!args:~%%p,%%q!"=="!arg%%n!" (
      echo arg%%n ??? %tab%{ !args:~%%p,%%q! } ^^!= { !arg%%n! } 1>&2
    )
  )
)
set /a delimLen = !argsLen! - !delimOff!
if defined args (
  @rem whitespace after last argument
  echo %tab%~!delimOff!,!delimLen! %tab%{ !args:~%delimOff%,%delimLen%! }
)
set "delimOff="
set "delimLen="

goto :eof

::.............................................................................

:getArgs
set "remArgs=%temp%\%random%.tmp"
set prompt=@
>"%remArgs%" (
  setlocal disableextensions
  echo on
  for %%a in (%%a) do rem { %* }
  @echo off
  endlocal
)
call :getFileSize "%remArgs%" sizeOld
set /a argsLen = %sizeOld% - 14

set /a argc = 1
:nextArg
>>"%remArgs%" (
  setlocal disableextensions
  echo on
  for %%a in (%%a) do rem { %1 }
  @echo off
  endlocal
)
call :getFileSize "%remArgs%" sizeNew
set /a argLen = %sizeNew% - %sizeOld% - 14
if %argLen% leq 0 (
  set "sizeOld="
  set "sizeNew="
  set "argLen="
  goto :lastArg
)

set /a arg%argc%len = %argLen%
set /a sizeOld = %sizeNew%
set /a argc += 1
shift /1
goto :nextArg

:lastArg
set /a argc -= 1
prompt

setlocal enableDelayedExpansion
<"%remArgs%" (
  set /p "args="
  set /p "args="
  set "args=!args:~7,-3!"
  for /l %%n in (1, 1, %argc%) do (
    set /p "arg%%n="
    set /p "arg%%n="
    set "arg%%n=!arg%%n:~7,-3!"
  )
)
del "%remArgs%"

goto :main

:getFileSize
set /a %~2 = %~z1
goto :eof

::.............................................................................

:getOffs
set /a argsOff = 0
for /l %%n in (1, 1, %argc%) do (
  call :getOff args !argsOff! arg%%noff
  set /a argsOff = !arg%%noff! + !arg%%nlen!
)
set "argsOff="
goto :eof

:getOff
setlocal enableDelayedExpansion
set /a argOff = %~2
:nextOff
set "chrNext=!%~1:~%argOff%,1!"
if "!chrNext!"=="" goto :lastOff
if not "!chrNext!"==" " ^
if not "!chrNext!"=="%tab%" ^
if not "!chrNext!"=="," ^
if not "!chrNext!"==";" ^
if not "!chrNext!"=="=" goto :lastOff
set /a argOff += 1
goto :nextOff
:lastOff
endlocal & set /a %~3 = %argOff%
goto :eof


Still don't like it completely, because it shifts parameters out thus destroying the original %1/%2/etc. It also leaves all variables stuck in a setlocal block without an easy way to send them back to the enclosing context. But, other than that, it appears to be working as advertised.

Liviu

EDIT: I tidied up some code (the 'for' loop when reading "%remArgs%" back), and added some inline comments (since I seem to be misplacing/forgetting web references way more often than I lose local batch files).

Also added the :getOffs counterpart, which essentially recovers the separators that cmd ignores from the full command line. Nice thing about it is that it basically tokenizes the command line while leaving the actual token parsing to cmd itself. In other words, as long as (1) cmd uses the same escaping logic between %* and %1/%2/etc, and (2) the delimiters remain the known space/tab/comma/semicolon/equal ones, this "manual reparsing" should match cmd's own, regardless of whatever the rules are inside a token.

Below is a sample output using the updated code.

Code: Select all

C:\>eccoargs ; %a %0 %~1 %cd^% !cd! a^|b a^&b "a&b" ^<"&"^> ^^;,=

---- copied from %? ----

args            { ; %a %0 %~1 %cd% !cd! a|b a&b "a&b" <"&"> ^;,= }
arg1            { %a }
arg2            { %0 }
arg3            { %~1 }
arg4            { %cd% }
arg5            { !cd! }
arg6            { a|b }
arg7            { a&b }
arg8            { "a&b" }
arg9            { <"&"> }
arg10           { ^ }

---- chopped off %* ----

args    ~0,46   { ; %a %0 %~1 %cd% !cd! a|b a&b "a&b" <"&"> ^;,= }
        ~0,2    { ;  }
arg1    ~2,2    { %a }
        ~4,1    {   }
arg2    ~5,2    { %0 }
        ~7,1    {   }
arg3    ~8,3    { %~1 }
        ~11,1   {   }
arg4    ~12,4   { %cd% }
        ~16,1   {   }
arg5    ~17,4   { !cd! }
        ~21,1   {   }
arg6    ~22,3   { a|b }
        ~25,1   {   }
arg7    ~26,3   { a&b }
        ~29,1   {   }
arg8    ~30,5   { "a&b" }
        ~35,1   {   }
arg9    ~36,5   { <"&"> }
        ~41,1   {   }
arg10   ~42,1   { ^ }
        ~43,3   { ;,= }

C:\>
Last edited by Liviu on 27 Jan 2012 12:46, edited 1 time in total.

jeb
Expert
Posts: 917
Joined: 30 Aug 2007 08:05
Location: Germany, Bochum

Re: foolproof counting of arguments

#17 Post by jeb » 27 Jan 2012 02:44

Hi,

Liviu wrote:Pretty sure I've seen this noted before, though I can't find a reference handy now... One other corner case is that the 'for' loop would save the incorrect %1 when passed %a or similar as an argument, presumably because of some loop variable name clash. Didn't look into it much yet, but for whatever reason replacing it with
Code:
for %%a in (%%a) do rem . %1.

appears to work better under xp.sp3 at least. Again, not fully understood or tested on my part.


Good idea, but it fails too :wink:

The main problem is to use a FOR-Parameter which can't be expanded.
That's Dave speciality, he made somewhere a list.

As now your code would fail with

Code: Select all

foolProofParam.bat %a...%~a...%~dpa


jeb

dbenham
Expert
Posts: 2288
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: foolproof counting of arguments

#18 Post by dbenham » 27 Jan 2012 11:19

jeb wrote:Hi,

Liviu wrote:for %%a in (%%a) do rem . %1.


Good idea, but it fails too :wink:

The main problem is to use a FOR-Parameter which can't be expanded.
That's Dave speciality, he made somewhere a list.

As now your code would fail with

Code: Select all

foolProofParam.bat %a...%~a...%~dpa


Good catch :!:
I should have seen that one coming once the problem with %a was pointed out :roll:

Unfortunately, I think we've reached a dead end :(

The list of special cases for FOR variables you were referring to is on StackOverflow. There are characters that cannot be directly defined as FOR variables, yet can be expanded. But they don't do us any good.

What we need is a character that can be defined as a FOR variable, but can't be expanded. I've never come up with any. The closest we have is the one you showed me: %

Code: Select all

for %%%% in (%%~%%) do rem . %1.

Command line arguments %% and %~% will be preserved.

But arguments like %~a% %~dp% %~path:% etc. will fail

So % has the fewest possible corner cases. But I don't think those cases are the least likely to occur. We might be better off choosing some extremly rare character for a command line argument. Perhaps a control character like <BackSpace> 0x08.

I believe we can't ever have a "perfect" solution involving FOR.

The only other way I know to capture the echoed command line in a file involves CALL, and that is no good because it can corrupt the command line and it isn't safe.

So it seems we are at a dead end. I don't see how we can ever have an absolutely foolproof method to capture any argument...


unless... :idea:

We could attempt to capture the complete set of arguments at least twice, using a different character for the FOR statement in each attempt. We could then compare the results and if we find discrepencies, we can use logic to try to reconstruct the full argument list. But it sounds like a complicated process. I don't think I want to try to attempt it. I'm not even sure if that could ever be foolproof.

Dave Benham

Liviu
Expert
Posts: 470
Joined: 13 Jan 2012 21:24

Re: foolproof counting of arguments

#19 Post by Liviu » 27 Jan 2012 12:55

jeb wrote:Good idea, but it fails too :wink:
...
As now your code would fail with

Code: Select all

foolProofParam.bat %a...%~a...%~dpa


Thanks for pointing that. Think I have a workaround, see the "2nd EDIT" comments above.
dbenham wrote:What we need is a character that can be defined as a FOR variable, but can't be expanded. I've never come up with any. The closest we have is the one you showed me: %
...
So % has the fewest possible corner cases. But I don't think those cases are the least likely to occur. We might be better off choosing some extremly rare character for a command line argument. Perhaps a control character like <BackSpace> 0x08.

Agreed. I'll remember <BackSpace> as a fallback if the disableextensions workaround proves to be insufficient.
dbenham wrote:unless... :idea:

We could attempt to capture the complete set of arguments at least twice, using a different character for the FOR statement in each attempt. We could then compare the results and if we find discrepencies, we can use logic to try to reconstruct the full argument list. But it sounds like a complicated process. I don't think I want to try to attempt it. I'm not even sure if that could ever be foolproof.

Crossed my mind, too, but I wouldn't go there, either ;-)

Liviu

dbenham
Expert
Posts: 2288
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: foolproof counting of arguments

#20 Post by dbenham » 27 Jan 2012 13:32

Liviu wrote:Note that the enclosing (...) is still evaluated at the outer level with extensions enabled, which allows %* to work correctly, but the for loop executes with extensions disabled so %~a, %~dpa etc are no longer expanded.

disableExtensions - perfect :!: :D

I always forget that option.

Dave Benham

Liviu
Expert
Posts: 470
Joined: 13 Jan 2012 21:24

Re: foolproof counting of arguments

#21 Post by Liviu » 11 Feb 2012 15:29

dbenham wrote:Now I understand why jeb had that # in his REM statement, to protect against /?
And only now do I understand why he had the space after # as well... REM appears to look for a '/?' all across the first argument (unless quoted) i.e. 'REM */?*' and 'REM a^&b%%c^(^!^<^|^>/^?...' both work just as 'REM /?'.

jeb
Expert
Posts: 917
Joined: 30 Aug 2007 08:05
Location: Germany, Bochum

Re: foolproof counting of arguments

#22 Post by jeb » 13 Feb 2012 05:43

dbenham wrote:Now I understand why jeb had that # in his REM statement, to protect against /?


Liviu wrote:And only now do I understand why he had the space after # as well... REM appears to look for a '/?' all across the first argument (unless quoted) i.e. 'REM */?*' and 'REM a^&b%%c^(^!^<^|^>/^?...' both work just as 'REM /?'.


Ok, perhaps I should comment a bit more :)

Liviu wrote:Note that the enclosing (...) is still evaluated at the outer level with extensions enabled, which allows %* to work correctly, but the for loop executes with extensions disabled so %~a, %~dpa etc are no longer expanded.

I didn't understand why you disable extensions, as the REM never use delayed expansion, or better it's hard to see it.

The redirection is done to the output of the echo phase (before the delayed expansion phase).

Code: Select all

setlocal EnableDelayedExpansion
set quest=/?
rem !quest!
rem results to nothing


EDIT: Reading is helpful :!:
I didn't realize that you used the word "EXTENSION" not "EXPANSION"

Now, I understand why you disable extensions :idea:, a really clever trick

dbenham
Expert
Posts: 2288
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: foolproof counting of arguments

#23 Post by dbenham » 13 Feb 2012 09:05

jeb wrote:EDIT: Reading is helpful :!:
I didn't realize that you used the word "EXTENSION" not "EXPANSION"
You would never catch me failing to read properly... noooooo, Not me :lol:

Dave Benham

Liviu
Expert
Posts: 470
Joined: 13 Jan 2012 21:24

Re: foolproof counting of arguments

#24 Post by Liviu » 13 Feb 2012 19:26

dbenham wrote:
jeb wrote:EDIT: Reading is helpful :!:
I didn't realize that you used the word "EXTENSION" not "EXPANSION"
You would never catch me failing to read properly... noooooo, Not me :lol:

In fact, I am pretty sure I've noticed that during an unrelated search, and it must have somehow stuck "disableextensions" in my memory cache ;-) Had never used it before, or thought I'd (need to) ever use it, until this case here where it fit just right. Thanks to both of you, again, and to dostips contributors at large.

Liviu

jeb
Expert
Posts: 917
Joined: 30 Aug 2007 08:05
Location: Germany, Bochum

Re: foolproof counting of arguments

#25 Post by jeb » 14 Feb 2012 04:12

In my opinion there are only two problems left.

We need a temporary file, it would be nicer if we could avoid this.

The REM technic currently fails with embedded linefeeds in the parameters %*.

I create a completly other way of capturing the parameters into a file

Code: Select all

@echo on
@echo Hello1 1>stream1.txt 2>stream2.txt 3>stream3.txt 4>stream4.txt
rem # %* #
@echo off
type stream3.txt > con
exit /b


This is adapted from the code of walid2me SO: Commenting multiple lines in DOS batch file

The only problem is to restore the standard handles :!:
As after the code even the prompt isn't visible anymore

jeb

Liviu
Expert
Posts: 470
Joined: 13 Jan 2012 21:24

Re: foolproof counting of arguments

#26 Post by Liviu » 14 Feb 2012 11:26

jeb wrote:In my opinion there are only two problems left.

We need a temporary file, it would be nicer if we could avoid this.
Right. The other unfortunate side effect of using a temp file is that characters outside the current code page cannot be handled (while a straightforward "set arg1=%1" works, barring of course poison characters in %1).

Code: Select all

C:\>eccoargs ‹αß©∂€›

---- copied from %? ----

args            { <αßc??> }
arg1            { <αßc??> }

---- chopped off %* ----

args    ~0,7    { <αßc??> }
        ~0,0    {  }
arg1    ~0,7    { <αßc??> }
        ~7,0    {  }

jeb wrote:The REM technic currently fails with embedded linefeeds in the parameters %*.
True, and worth noting. Though in practice it only becomes a problem in rather uncommon cases.

jeb wrote:I create a completly other way of capturing the parameters into a file
...
The only problem is to restore the standard handles :!:
As after the code even the prompt isn't visible anymore
Oddly enough, piping the command "| more" brings back the prompt and the ">con" output. That said, I know (and found) virtually no information on handles 3 and 4, so I can't even begin to guess how cmd uses them internally.

Liviu

dbenham
Expert
Posts: 2288
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: foolproof counting of arguments

#27 Post by dbenham » 14 Feb 2012 12:24

Liviu wrote:Oddly enough, piping the command "| more" brings back the prompt and the ">con" output.
:?: :?
I can't figure out what you mean. What is the exact syntax in context?

Dave Benham

Liviu
Expert
Posts: 470
Joined: 13 Jan 2012 21:24

Re: foolproof counting of arguments

#28 Post by Liviu » 14 Feb 2012 12:41

dbenham wrote:What is the exact syntax in context?

Save Jeb's snippet as, say, stream3.cmd. Running it as such results in...

Code: Select all

C:\>stream3

C:\>rem #  #
cls
exit
...where the prompt remains in an odd input state. The last two lines were typed in, 'cls' did nothing, and 'exit' does in fact close the console window once Enter is pressed.

However, piping into "| more" returns to the cmd prompt normally.

Code: Select all

C:\>stream3 | more

C:\>rem #  #


C:\>

Liviu

jeb
Expert
Posts: 917
Joined: 30 Aug 2007 08:05
Location: Germany, Bochum

Re: foolproof counting of arguments

#29 Post by jeb » 15 Feb 2012 00:18

Liviu wrote:...where the prompt remains in an odd input state. The last two lines were typed in, 'cls' did nothing, and 'exit' does in fact close the console window once Enter is pressed.

However, piping into "| more" returns to the cmd prompt normally.


But piping do the same as the exit, as the pipe itself creates two seperate sub-tasks for each side.
The prompt disappears in the child task which is never visible.

So the pipe doesn't solve the problem this way.

jeb

Liviu
Expert
Posts: 470
Joined: 13 Jan 2012 21:24

Re: foolproof counting of arguments

#30 Post by Liviu » 15 Feb 2012 00:39

jeb wrote:But piping do the same as the exit, as the pipe itself creates two seperate sub-tasks for each side.
The prompt disappears in the child task which is never visible.
Haven't followed this in any depth, but it doesn't look quite the same. The pipe closes both ends without any obvious 'exit' being typed in and/or piped through.
jeb wrote:So the pipe doesn't solve the problem this way.
Streams 3/4 redirection is foggy enough to me that I can't even spell out what the "problem" might be, let alone solve it ;-)

Liviu

Post Reply