For loop syntax quirks

[Liviu]

Running a plain loop like 'for %%v in ("X" "Y") do ...' returns X then Y at each of the two steps, respectively, either %%v quoted, or %%~v unquoted. For reasons not entirely clear to me, yet, I can't figure out the syntax to make it work when X is "=^" (quotes are part of the string) and Y is A<LF>B (where <LF> is the single char !LF! linefeed).

The following works, but it returns the correct X in %%v and the correct Y in %%~v. My question is how to write one single for loop which retrieves the values correctly, and both in the same mode - either %%v or %%~v, but not one %%v and the other %%~v.

Code: Select all

@echo off & setlocal enableDelayedExpansion

@rem single linefeed char 0x0A (two blank lines required below)
set LF=^


@rem "=^"
set ^"v1=^"=^^^""
@rem a<lf>b
set ^"v2=a!LF!b"

echo(
for %%v in ( ^^^"^^^=^^^^^" "a!LF!b" ) do (
  if '%%v'=='!v1!'  echo v1 ==  v
  if '%%~v'=='!v1!' echo v1 == ~v
  if '%%v'=='!v2!'  echo v2 ==  v & if not "!v2:%%v=+!"=="+" echo ...subst failed '!v2:%%v=+!' ??
  if '%%~v'=='!v2!' echo v2 == ~v & if not "!v2:%%~v=+!"=="+" echo ...subst failed '!v2:%%~v=+!' ??
)

Output:

Code: Select all

v1 ==  v
v2 == ~v

The obvious escaping doesn't really work, in the sense that the 2nd variable containing the LF remains not fully evaluated (edit: i.e. contains a literal unexpanded !LF!, see jeb's clarification below http://www.dostips.com/forum/viewtopic.php?p=32773#p32773 /end edit) and fails inside a delayed expansion, as shown by the ?? lines in the output.

Code: Select all

echo(
for %%v in ( ^^^"^^^=^^^^^" a^^!LF^^!b ) do (
  if '%%v'=='!v1!'  echo v1 ==  v
  if '%%~v'=='!v1!' echo v1 == ~v
  if '%%v'=='!v2!'  echo v2 ==  v & if not "!v2:%%v=+!"=="+" echo ...subst failed '!v2:%%v=+!' ??
  if '%%~v'=='!v2!' echo v2 == ~v & if not "!v2:%%~v=+!"=="+" echo ...subst failed '!v2:%%~v=+!' ??
)

Output:

Code: Select all

v1 ==  v
v2 ==  v
...subst failed 'a
b' ??
v2 == ~v
...subst failed 'a
b' ??

Difficulty here seems to be that neither the = equal sign nor the <lf> linefeed seem to be readily escape'able outside quotes. And while it's foolish to ever say something is impossible in batch syntax but hints at why the above might not be technically workable would be appreciated, too. After all, this is just a curiosity, and not much of a practical matter.

Liviu

[dbenham]

I think simple FOR parsing and CALL argument parsing are similar in that token delimiters can be protected by quotes, but they cannot be escaped. And <LF> suffers a similar fate, except it cannot be readily passed within a quoted parameter either.

The token delimiters that cannot be escaped are <space> <tab> <comma> <semicolon> <equal> <0xFF>


Dave Benham

[Liviu]

The token delimiters that cannot be escaped are <space> <tab> <comma> <semicolon> <equal> <0xFF>

That looks to be the case, yes. It comes close to a proof that the 1st string "=^" cannot be quoted itself, since that would change the quote-flag balance on the line, and leave the = equal sign outside quotes. But, as I said, one can never be completely sure in batch world

FWIW I did not mean just the plain for loop. I tried variations thereof with for/f, too, both with and without 'usebackq', and did not find any construct that would return both strings correctly in the same loop variable - either full %%v or dequoted %%~v.

Liviu

[dbenham]

FWIW I did not mean just the plain for loop. I tried variations thereof with for/f, too, both with and without 'usebackq', and did not find any construct that would return both strings correctly in the same loop variable - either full %%v or dequoted %%~v.

FOR /F cannot ever parse <LF> as part of a %V variable (or %~V) because FOR /F always breaks lines at <LF> and iterates/parses each line separately. This is partly why jeb structured the safe return technique the way he did. The <LF> had to be stored in a simple quoted FOR variable, whereas the other special characters could all be parsed into variables using a single FOR /F.

The other odd parsing rule for FOR /F is that it will strip the last character of a line prior to parsing if the last character is <CR>. FOR /F never strips more than a single <CR> from the end of a line.


Dave Benham

[Liviu]

FOR /F cannot ever parse <LF> as part of a %V variable (or %~V) because FOR /F always breaks lines at <LF> and iterates/parses each line separately.

Technically right, and all I meant to say was that I tried - but failed

That said, it's still confusing at first sight. For/f can and does parse !LF! literals into variables which then behave "almost" like LFs under enableDelayedExpansion except in particular scenarios like being expanded inside another delayed expansion.

Consider the following variation on my previous example...

Code: Select all

for /f %%v in (^" ^"^^"^=^^^"" !LF! a^!LF^!b ") do (
  if '%%v'=='!v1!'  echo v1 ==  v
  if '%%~v'=='!v1!' echo v1 == ~v
  if '%%v'=='!v2!'  echo v2 ==  v & if not "!v2:%%v=+!"=="+" echo ...subst failed '!v2:%%v=+!' ??
  if '%%~v'=='!v2!' (
    echo v2 == ~v
    set "w=%%~v"
    if '!w!'=='!v2!' (echo ++ copy OK) else (echo -- copy failed)
    for %%w in ("!w!") do (
      if "!v2:%%~w=+!"=="+" (echo ++ nested subst OK) else (echo -- nested subst failed '!v2:%%~v=+!' ??)
    )
    if not "!v2:%%~v=+!"=="+" echo ...subst failed '!v2:%%~v=+!' ??
  )
)

...with output:

Code: Select all

v1 == ~v
v2 ==  v
...subst failed 'a
b' ??
v2 == ~v
++ copy OK
++ nested subst OK
...subst failed 'a
b' ??

The %%~v checks 'if ==' OK against the original v2, can be assigned to another 'w' variable which itself checks OK, but fails to work inside the !v2:%%~v=..! delayed expansion.

Liviu

P.S. Incidentally, this example also shows that for/f can get the first argument properly quoted and correctly dequoted as "=^" in %%~v.

[jeb]

The following works, but it returns the correct X in %%v and the correct Y in %%~v.

Hmm, that's what I expect, as your v1 is defined as "=^", but your v2 is defined without quotes a!LF!b

So, in both cases %%v is enclosed into quotes, but obviously you need for the second case %%~v to match.

Code: Select all

... for %%v in ( ^^^"^^^=^^^^^" "a!LF!b" ) do (
  if '%%v'=='!v1!'  echo v1 ==  v
  if '%%~v'=='!v1!' echo v1 == ~v
  if '%%v'=='!v2!'  echo v2 ==  v & if not "!v2:%%v=+!"=="+" echo ...subst failed '!v2:%%v=+!' ??
  if '%%~v'=='!v2!' echo v2 == ~v & if not "!v2:%%~v=+!"=="+" echo ...subst failed '!v2:%%~v=+!' ??
)

The obvious escaping doesn't really work, in the sense that the 2nd variable containing the LF remains not fully evaluated and fails inside a delayed expansion, as shown by the ?? lines in the output.

A bit hard to see, but in the end the cause was simple.
%%v isn't equal to v2 here.

I reduce your sample

Code: Select all

for %%v in ( a^^!LF^^!b ) do (
   setlocal DisableDelayedExpansion
   echo %%v
   endlocal
)

a!LF!b

Here is !LF! still literally with four character not a real linefeed.
It has nothing to do with quotes

Btw. It's always a good idea to output your parameters with disabled expansion, that can avoid many misinterpretations

jeb

[Liviu]

Here is !LF! still literally with four character not a real linefeed.

Right, I've edited the OP to clarify that point.

Back to the original question, now that neither you nor Dave came up with an easy "obvious" here's-how-to-do-it, I'd say chances are high that the answer is negative

Liviu

[jeb]

Back to the original question, now that neither you nor Dave came up with an easy "obvious" here's-how-to-do-it, I'd say chances are high that the answer is negative

Ok, if I understand the question correct, you ask how to use an unquoted linefeed with a FOR loop?

That's seems not possible as content, as the linefeed is handled as delimiter and you can't escape an delimiter there.

But perhaps you ask something completly different

[Liviu]

Ok, if I understand the question correct, you ask how to use an unquoted linefeed with a FOR loop?

Yes, either of:
- a quoted "=^" in a plain for loop (i.e. returned as ""=^"" in %%v, "=^" in %%~v)
- an unquoted linefeed in a plain for loop
- any linefeed (whether quoted or unquoted) in a for/f loop
would settle the matter - if only they worked, but none of them seems to be possible.

Liviu

[jeb]

Ok.
- You can't receive an unquoted linefeed in a normal for loop, as it's handled as delimter
- You can't receive a linefeed at all in a FOR/F loop, as it's handled as the line end delimiter

- I can't see any problems with a quoted "=^" in a plain loop

Code: Select all

@echo off
setlocal DisableDelayedExpansion
for %%v in ("=^") do (
   echo %%v
)

"=^"

jeb

[Liviu]

oops, edited your post instead of quoting it, sorry Liviu, must have looked something like this

I can't see any problems with a quoted "=^" in a plain loop

"=^" is the actual (unquoted) test string. Once quoted, it would have to be ""=^"". That's what I meant by this - and I still don't see a way to have a plain for loop return a value of ""=^"" in %%v on the first iteration.

- a quoted "=^" in a plain for loop (i.e. returned as ""=^"" in %%v, "=^" in %%~v)

[Ed Dyreen]

<equals> acts as a delimiter, therefor it can never be passed byValue unquoted.

Code: Select all

@echo off &prompt $G &setlocal DisableDelayedExpansion
@echo on
for %%v in (""="") do (
   echo _%%v_
)
@echo off
pause

Code: Select all

> for %v in ("" "") do (echo _%v_ )

> (echo _""_ )
_""_

> (echo _""_ )
_""_

and I still don't see a way to have a plain for loop return a value of ""=^"" in %%v on the first iteration.

Code: Select all

@echo off &prompt $G
setlocal enableDelayedExpansion
@echo on
for %%v in (^^^!"^!""=^^^^""^!"^^^!) do (
   echo %%v
)
@echo off

Code: Select all

>for %v in (^!"^!""=^^^^""^!"^!) do (echo %v )

>(echo !"!""=^^""!"! )
""=^""

%%v really contains !"!""=^^""!"! but since !"! is undefined, it is lost the moment you set or print %%v

[penpen]

As the "a!LF!b" works on v2 with %%~v:
Is there a reason why you don't just add doublequotes to e temporary variable v2_2 and work on it using %%v?

Code: Select all

@echo off
cls
setlocal enableDelayedExpansion
@rem single linefeed char 0x0A (two blank lines required below)
set LF=^


@rem "=^"
set ^"v1=^"=^^^""
@rem a<lf>b
set ^"v2=a!LF!b"

for %%v in ("a!LF!b") do set "v2_2=!v2:%%~v=%%v!"
for %%v in ( ^^^"^^^=^^^^^" "a!LF!b" ) do (
  if '%%v'=='!v1!'  echo v1 ==  v
  if '%%~v'=='!v1!' echo v1 == ~v
  if '%%v'=='!v2!'  echo v2 ==  v & if not "!v2:%%v=+!"=="+" echo ...subst failed '!v2:%%v=+!' ??
  if '%%~v'=='!v2!' echo v2 == ~v & if not "!v2:%%~v=+!"=="+" echo ...subst failed '!v2:%%~v=+!' ??
  if '%%v'=='!v2_2!' echo v2_2 == v & if not "!v2_2:%%v=+!"=="+" echo ...subst2 failed '!v2_2:%%v=+!' ??
  if '%%~v'=='!v2_2!' echo v2_2 == ~v & if not "!v2_2:%%~v=+!"=="+" echo ...subst2 failed '!v2_2:%%~v=+!' ??
)
endlocal
goto :eof

penpen

[Liviu]

Is there a reason why you don't just add doublequotes to e temporary variable v2_2 and work on it using %%v?

Basically, you are changing Y to a different Y (as used in the 1st sentence of the OP). That makes it an entirely different question, and one which happens to have an easy answer, as you note. And, as I said to begin with, this was a curiosity more than a practical matter, so I was not looking for workarounds. Even the original case can be solved with additional variables and/or for loops, but the question was whether someone saw an "obvious" answer without any of that overhead.

<equals> acts as a delimiter, therefor it can never be passed byValue unquoted.

That's the difficulty, indeed. As Dave noted, the same applies to the other token delimiters <space> <tab> <comma> <semicolon> <equal> <0xFF>. Still, I am secretly disappointed that there isn't a backdoor cheat to be found. Not even this one works

Code: Select all

for %%^" in ( ""^=^^"" ) do echo [%%^"]

Liviu

[Ed Dyreen]

The backdoor cheat I posted works , no seriously if jeb hasn't figured it out, I ain't even gonna try