Problem with delayed expansion after ECHO. or ECHO:

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Message
Author
dbenham
Expert
Posts: 2289
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Problem with delayed expansion after ECHO. or ECHO:

#1 Post by dbenham » 26 May 2011 10:04

Can anyone explain why the first four delayed expansions fail, but the rest succeed?

Code: Select all

@echo off
setlocal enableDelayedExpansion
set test=JUNK
echo:!test:~0,2!
echo:!test:J=F!
echo.!test:~0,2!
echo.!test:J=F!
echo !test:~0,2!
call echo:!test:~0,2!
echo:!test!
echo:!test:~0!
echo !test:J=F!
call echo:!test:J=F!

Output:

Code: Select all

test:~0,2
test:J=F
test:~0,2
test:J=F
JU
JU
JUNK
JUNK
FUNK
FUNK

This seems like a DOS bug.

Dave Benham

aGerman
Expert
Posts: 3778
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Problem with delayed expansion after ECHO. or ECHO:

#2 Post by aGerman » 26 May 2011 10:34

Hi Dave,

jeb already answered this issue in this thread.

Regards
aGerman

dbenham
Expert
Posts: 2289
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Problem with delayed expansion after ECHO. or ECHO:

#3 Post by dbenham » 26 May 2011 11:02

Ahh, I had seen that post before, but had never waded far enough to see the part relevent to this post. Next time I get insomnia I'll try to parse Jeb's explanation. It looks like it is going to take some serious thinking.

Thanks aGerman

aGerman
Expert
Posts: 3778
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Problem with delayed expansion after ECHO. or ECHO:

#4 Post by aGerman » 26 May 2011 11:32

Yeah, jeb spent a lot of time to figure out how the command interpreter works. Months ago I asked him to write a book about his knowledge. But he was sure it would be a shelf warmer and he never took the bait even as I told him that I would buy it beyond any doubt :lol:

Regards
aGerman

orange_batch
Expert
Posts: 442
Joined: 01 Aug 2010 17:13
Location: Canadian Pacific
Contact:

Re: Problem with delayed expansion after ECHO. or ECHO:

#5 Post by orange_batch » 26 May 2011 11:57

Actually to go even further, it's about character interpretation. This is a command prompt error, not an echo error.

This works, but puts quotation marks around the output:

Code: Select all

set var=hello
echo:"!var:~2,4!"

If that's not acceptable, at least it's a good solution for:

Code: Select all

if "!var:~2,4!"=="el" (echo:Match.) else echo:Not match.

Not using quotations around !var:~2,4! will fail.

dbenham
Expert
Posts: 2289
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Problem with delayed expansion after ECHO. or ECHO:

#6 Post by dbenham » 26 May 2011 12:26

@orange_batch
I never thought it was an ECHO error - I always assumed it was a parsing issue. It just happens to be a very arcane parsing issue.

I have seen and dealt with the IF issue before. I think I stumbled on the solution and never thought much of it.

I never thought to relate the issues, but it makes sense.

Dave

aGerman
Expert
Posts: 3778
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Problem with delayed expansion after ECHO. or ECHO:

#7 Post by aGerman » 26 May 2011 14:07

The backspace character could help to solve that problem.

Code: Select all

@echo off &setlocal enabledelayedexpansion
set var=hello
for /f "delims=#" %%a in ('"prompt #$H# &echo on &for %%b in (1) do rem"') do set "bs=%%a"
echo:%bs%!var:~2,4!

But I assume a better way is to avoid ":" and use "(" instead.

Regards
aGerman

jeb
Expert
Posts: 918
Joined: 30 Aug 2007 08:05
Location: Germany, Bochum

Re: Problem with delayed expansion after ECHO. or ECHO:

#8 Post by jeb » 26 May 2011 15:02

A little nice problem :)

you can solve it also with carets

Code: Select all

setlocal enableDelayedExpansion
set test=JUNK
echo:!test:~0^,2!
echo:!test:J^=F!
echo.!test:~0^,2!


The cause is the well known command token splitting effect. :wink:

When the line is parsed, in phase 2 (special characters ^&<> and so on) also the line is split into two tokens.
The first one is expected as the command the second is the rest of the line.
The tokenizer splits the tokens at "<space>,;=&|<>(", so in your samples the command tokens look like

Code: Select all

echo:!test:~0
echo:!test:J
echo.!test:~0

As the command token and the rest of the line are (delayed) expanded separatly both parts fails to expands.

btw. Even the delayed effect to carets is separated for both parts

Code: Select all

setlocal EnableDelayedExpansion
set a=####
set test=CONTENT
echo:^^^^!  ^^^^
echo:^^^^!  ^^^^!
echo:!test:~2,3!a!
echo !test:~2,3!a!
----- OUTPUT ----
^  ^^
^  ^
test:~2,3####
NTEa



Ok, but why it fails with an IF !test:~1,2! command :?:
IF,FOR and also REM are detected in phase 2 and activate a special token handler.
IF have special token rules to detect all the different IF-Syntax styles like
IF a==b
if a EQU b
if NOT a==b
if defined var
if exist f

Some of these tokens are not affected by delayed expansion, here only a,b and f are affected.

Code: Select all

set myNot=NOT
set myDefined=defined
set var=content
set indVar=var
set a=#
set b=#
if !a!==!b! echo "==" works
if !a! EQU !b! echo "EQU" works
if NOT a==b echo "NOT" works
if defined !indVar! echo "DEF" works
if !myDefined! var echo SyntaxError
if !myNot! a==b echo SyntaxError


hope it helps
jeb

dbenham
Expert
Posts: 2289
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Problem with delayed expansion after ECHO. or ECHO:

#9 Post by dbenham » 26 May 2011 15:57

%BS% is interesting, but not good because it produces 0x08 0x20 0x08 in the output, easily seen if redirected to a file and looked at with hexdump or hex editor.

Thanks Jeb - it definitely helps. It will help even more when I really dig into all your posts I can find dealing with parsing. I've bookmarked a few, including some on some other sites. Are you sure you don't want to write that book aGerman suggested? :wink:

In my mind, a perfect solution should echo the appropriate substring, or echo a blank line if the result is empty or if the variable is undefined. My last requirement is a killer. But is it so unreasonable? If var is undefined and !var! and %var% return an empty string, then why shouldn't some form of !var:~0,4! or %var:~0,4% also return an empty string? (because its "DOS" with its arcane parsing rules - I know).

None of the methods work in all cases. Some fail if the substring result is an empty string. All fail if the variable is undefined.

Even this doesn't work because the assignment of var2 fails if var is undefined.

Code: Select all

set var=
set "var2=!var:~2,4!"
echo There should be a blank line after this.
echo:!var2!
echo Ouch!


I'm amazed I have to resort to an IF statement to get my desired behavior.

Code: Select all

@echo off &setlocal enableDelayedExpansion
::I want to ECHO the results of a substring operation on var, which happens to be undefined.
set var=
if defined var (set "tempVar=!var:~2,4!") else set "tempVar="
echo There should be a blank line after this.
echo:!tempVar!
echo Was that so hard?... YES^^!


Dave Benham

jeb
Expert
Posts: 918
Joined: 30 Aug 2007 08:05
Location: Germany, Bochum

Re: Problem with delayed expansion after ECHO. or ECHO:

#10 Post by jeb » 27 May 2011 05:55

dbenham wrote:%BS% is interesting, but not good because it produces 0x08 0x20 0x08 in the output, easily seen if redirected to a file and looked at with hexdump or hex editor.


Good observation! (I see you wrote it in "Re: new functions: :chr, :asc, :asciiMap)
I never noticed this, I thought that the code produced a single <Backspace>, but $H creates the sequence <backspace><space><backspace>, to ensure that the character is deleted.
I have to change my code.

To get only one <bs> the code should look like (I add a space to the delims).

Code: Select all

for /f "delims=# " %%a in ('"prompt #$H# &echo on &for %%b in (1) do rem"') do set "bs=%%~a"


But then the echo:%BS%!var:~1,2! will fail again, as <bs> isn't a token delimiter (It was the hidden space).

jeb

Ed Dyreen
Expert
Posts: 1569
Joined: 16 May 2011 08:21
Location: Flanders(Belgium)
Contact:

Re: Problem with delayed expansion after ECHO. or ECHO:

#11 Post by Ed Dyreen » 28 May 2011 14:37

Noobish question, When do we need backspace ?, in general I mean.
I already learn't that linefeed is handy in macros, but backspace ????
Where do I find that information :|

jeb
Expert
Posts: 918
Joined: 30 Aug 2007 08:05
Location: Germany, Bochum

Re: Problem with delayed expansion after ECHO. or ECHO:

#12 Post by jeb » 01 Jun 2011 15:17

Ed Dyreen wrote:Noobish question, When do we need backspace ?, in general I mean.
I already learn't that linefeed is handy in macros, but backspace ????

@Ed

It's nice for a a progress counter, counting from 0 to 100 at the same position,
using the <CR> character here, creates the same result.

Code: Select all

@echo off
setlocal EnableDelayedExpansion
call :BL.String.CreateBS_ESC
for /L %%n in (1,1,10000) DO (
   <nul set /p "=!BS!!BS!!BS!!BS!!BS!%%n"
)
goto :Eof


:BL.String.CreateBS_ESC
:: Creates two variables with one character BS=Ascii-08 and ESC=Ascii-27
:: BS and ESC can be used  with and without DelayedExpansion
:: @attention $H produce a <BS><space><BS>, so we need # and <space> as delims
setlocal
for /F "tokens=1,3 delims=# " %%a in ('"prompt #$H#$E# & echo on & for %%b in (1) do rem"') do (
  ENDLOCAL
  set "BS=%%a"
  set "ESC=%%b"
  goto :EOF
)
goto :eof


dbenham wrote:In my mind, a perfect solution should echo the appropriate substring, or echo a blank line if the result is empty or if the variable is undefined.


You didn't need an IF statement.

Code: Select all

@echo off
setlocal EnableDelayedExpansion
set var=
set "var2=x!var!"
set "var2=!var2:~2,4!"
echo There should be a blank line after this.
echo(!var2!
echo .. as expeced


jeb

dbenham
Expert
Posts: 2289
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Problem with delayed expansion after ECHO. or ECHO:

#13 Post by dbenham » 01 Jun 2011 17:29

jeb wrote:
dbenham wrote:In my mind, a perfect solution should echo the appropriate substring, or echo a blank line if the result is empty or if the variable is undefined.


You didn't need an IF statement.

Code: Select all

@echo off
setlocal EnableDelayedExpansion
set var=
set "var2=x!var!"
set "var2=!var2:~2,4!"
echo There should be a blank line after this.
echo(!var2!
echo .. as expeced


Fair enough - I don't "have" to resort to an IF. However your code seems more obfuscated to me. I suppose there must be other "solutions" as well. But I don't think that matters. The important thing is I learned some limitations of the batch parser in this thread.

Thanks

Dave

Ed Dyreen
Expert
Posts: 1569
Joined: 16 May 2011 08:21
Location: Flanders(Belgium)
Contact:

Re: Problem with delayed expansion after ECHO. or ECHO:

#14 Post by Ed Dyreen » 01 Jun 2011 22:20

@jeb
You must be some kind of god or something, well,.. you are definetly my guru.
Just give me 3 years to process your wisdom.

You may have noticed I am digging all posts that have your name in it.
My god man, how did you become so good. :shock:

orange_batch
Expert
Posts: 442
Joined: 01 Aug 2010 17:13
Location: Canadian Pacific
Contact:

Re: Problem with delayed expansion after ECHO. or ECHO:

#15 Post by orange_batch » 02 Jun 2011 01:46

He's very thorough and has the patience to test and research and test and test and... with that form logical conclusions. The head R&D of DOStips.

Post Reply