ECHO. FAILS to give text or blank line - Instead use ECHO/

[penpen]

:shock: My brain hurts
(...)
Also, within parentheses, it only works as a comment if it follows a label or label comment.

If the above causes headaches (sorry for that), then try this proof that no label (comment) is needed, but another caret (at least under win 7 home 64 bit):

Code: Select all

@echo off
(
   echo(What do you expect to see? It's magic.
^
 ^

   echo(Test 1: Do you think you can see me?.
   echo(Hello caret1!
^
  ^

   echo(Test 2: Do you think you can see me?.
   echo(Hello caret2!
)

penpen

[Sponge Belly]

Hi Dave and Penpen!

Good effort, but I’ve uncovered something even more obscure about echo.

Turns out if the last character in a file is SUB (Ctrl-Z) and echo is used to append to the file, the SUB is clobbered.

If anyone can find a use for this, please let us know.

- SB

PS: Same goes for appending to a file with set /p.

[Liviu]

no label (comment) is needed, but another caret (at least under win 7 home 64 bit)

Confirmed under xp.sp3 as well.

Code: Select all

@echo off

^

echo this line is ignored
echo this line is executed

(
^
^

echo this line is ignored
echo this line is executed
)

Turns out if the last character in a file is SUB (Ctrl-Z) and echo is used to append to the file, the SUB is clobbered.

It's the append ">>" redirection that loses the Ctrl-Z, not echo or set/p in particular. For example, if file "some.txt" ends with a Ctrl-Z and you do a "dir >>some.txt" then the Ctrl-Z is dropped, too. Not very surprising, since Ctrl-Z a.k.a. EOF (end of file) in DOS-world is known to have special treatment in text related scenarios.

Liviu

P.S. [ EDIT - added this paragraph ] One other thing to note is that the lines above marked as "ignored" seem to be fully parsed, in fact, just not executed for whatever reason. Replacing "echo this line is ignored" with "echo this line is & ignored" causes an error "'ignored' is not recognized..." - showing that the line was parsed, and was even parsed beyond just the first token.

This is relevant because it lessens the chances that the trick could be useful for macro comments. In that case, it's about what's parsed on the right-hand side of the "set" command, not execution. Surely doesn't mean it's impossible, and wish someone proves me wrong but I couldn't find the right incantation to get the "rem" lines ignored even in a plain vanilla example like this.

Code: Select all

@echo off

set var=line-1^

line-2^

rem comment-2^

line-3^

rem comment-3.a^

rem comment-3.b^

line-4

set var

[jeb]

There isn't much magic here.

I think it's more or less obvious

Mainly it's the normal escaping logic.

Code: Select all

^

something

Will be shrinked to <LF>something

So you get commands preceded with a <LF>.

Code: Select all

@echo off
SET LF=^


REM ***
echo #1
^

#1 ignore this

echo #2
^%LF%%LF%Same as #1

(
echo #3
:label3
^

ignore this

echo #4
:label4
^%LF%%LF%Same as #3
)

I suppose the <LF> doesn't stop the parser in the special character phase as the <LF> is an escaped version.
But later a <LF> seems to be a valid command without any effect.

If the above causes headaches (sorry for that), then try this proof that no label (comment) is needed, but another caret (at least under win 7 home 64 bit):
Code:
@echo off
(
echo(What do you expect to see? It's magic.
^
^

echo(Test 1: Do you think you can see me?.
echo(Hello caret1!
^
^

echo(Test 2: Do you think you can see me?.
echo(Hello caret2!
)

Even this can be reduced to

Test1
<escaped space><LF>echo Test1

Test2
<escaped space><normal space><LF>echo Test1

There are only some cases left with <LF> in combination with labels and parenthesis.
There it's possible that the first token of a line can be dropped.

Code: Select all

@echo off
(
^

rem echo You can see this!
)

jeb

[Sponge Belly]

Hi Liviu!

Thanks for the explanation.

But why doesn’t…

Code: Select all

(call;>>file.txt)

clobber the SUB at the end of file.txt?

- SB

[Liviu]

But why doesn’t…

Code: Select all

(call;>>file.txt)

clobber the SUB at the end of file.txt?

Because 'call;' outputs nothing at all. You can check the timestamps of 'file.txt' to see that it wasn't modified. The SUB is only clobbered if there is something actually being appended to the file.

Liviu

[dbenham]

While trying to refine the rules for how the batch parser works in general, I discovered some new info about this topic.

First, at the bottom of this post I began to suspect some previous statements in this thread were wrong. jeb then confirmed the inaccuracy and provided corrected information.

I then refined some proposed rules on how the batch parser determines if a command is an internal or external command. These rules enable me to predict the following:

A safe character to follow ECHO should be any token delimiter that is not white space. That leads me to predict that all of the following should be good:

Code: Select all

echo(
echo=
echo,
echo;

But ( is only a command token delimiter in phase 2. So ECHO( cannot be used within a delayed expansion macro.
Also, ( is not a command token delimiter when it follows an unexecuted label within a parenthesized block. So ECHO( cannot be used there.

But ECHO= ECHO, and ECHO; should always be safe.

I've run some quick tests, and so far, this all seems to be true

Edit
Actually, they are not always safe, because I don't think a safe form exists for CALL ECHO.
All of the following will call ECHO.BAT if it exists in the current directory or the PATH:

Code: Select all

call echo=
call echo,
call echo;
call echo(

Dave Benham

[jeb]

A safe character to follow ECHO should be any token delimiter that is not white space. That leads me to predict that all of the following should be good:

Code: Select all

echo(
echo=
echo,
echo;

But ( is only a command

Quite a nice try , but not safe for content.

Code: Select all

echo(/?
echo=/?
echo,/?
echo;/?

Only "echo(/?" is able to show "/?", all others show the HELP of echo.

jeb
PS: I edited the incorrect description in the old post

[dbenham]

Of course. That makes sense.

When ECHO sees ;/? or ,/? or =/? it sees a token delimiter and then a string beginning with help option. So it prints help.

But (/? does not start with a token delimiter, so the /? is masked. Then the leading character is stripped and the remainder is printed.

Oh well. At least things make sense now.


Dave Benham

[CJM]

This subject seemed to tail off without resolution, yet using ECHO to reliably display information is such a necessity.

...
fails, if a file in the current directory exists named my.bat

Code: Select all

echo\..\my.bat
echo:\..\my.bat
echo.\..\my.bat

...

...
The echo:\..\my.bat behavior really blows my mind.
...

Not only that, it's a seemingly unexploited code-injection security vulnerability! This had me searching for a character that did not exhibit the ability to invoke file system action at all.

Perhaps if the known issues were summarized in some way. This lead me to layout the issues with each character in some objective/digestible form, so I started tabulating my own efforts and verifying some of those that may've have been discussed in various forums.

I believe these are issues toward consistently using internal ECHO command to display intended content:

• avoid searching for or unintentionally invoking external commands [CRITICAL--could inadvertently cause loss of data]
• prevent unpredictable failure behavior due to a particular file name/path existing in the file system
• suppress unintended invocation of ECHO's HELP [or CALL's HELP] when line begins with /? or ?, instead output those characters as intended
• suppress ECHO's tendency to report status [ECHO is on / off] when nothing follows, and output a blank/new line as intended

Here's the reference I've been developing in table form. Basically, one would want to choose the character that checks the most "Ok" boxes, but I'd also avoid the insecure ones, those that have the potential for the file system vulnerability:

Code: Select all

                Test _________________________________________  Type__________

        Char    Echo    %Macro% Call    For     ECHO /? ECHO ?  Ext     File    Batch   Prompt  isDelim Note
        _______ _______ _______ _______ _______ _______ _______ _______ _______ _______ _______ _______ _______________________________
    ECHO[Space] On/Off  On/Off  bat/cmd On/Off  Help    Ok      Ext     Valid           9009    Token   CALL ECHO runs ECHO.bat/.cmd if exist in dir, ignores PATHEXT
    ECHO[Tab]   On/Off  On/Off  On/Off  On/Off  Help    Ok      Ok      Ok              Invalid Token
    ECHO,       Ok      Ok      Ok      Ok      Help    Ok      Ok      Valid           Ok      Token
    ECHO;       Ok      Ok      Ok      Ok      Help    Ok      Ok      Valid           Ok      Token
    ECHO=       Ok      Ok      Ok      Ok      Help    Ok      Ok      Valid           Ok      Token
    ECHO/       Ok      Ok      Ok      Ok      -Help   +Help   Ok      Ok      Ok      Ok      Ok      -+Prints ECHO's Help if content begins with ? or /? follows any token [CALL's Help if CALLed and contncontains /? anywhere]
    ECHO\       Ok      Ok      Ok      Ok      Ok-Call Ok      Ext     Path    .ext    9009    Ok      -CALL ECHO\/? produces CALL's Help; Executes ECHO\batchfilename.bat if path exists
    ECHO:       Ok      Ok      Ok      Ok      Ok-Call Ok      Ext     Ok      .ext    9009-2K Ok      -CALL ECHO:/? produces CALL's Help; Executes if \..\my.pathext exists
    ECHO+       Ok      Ok      Ok      Ok      Ok-Call Ok      Call    Valid   bat/cmd Ok      Ok      -CALL ECHO+/? produces CALL's Help
    ECHO.       *9009   *9009   File    Ok      Ok-Call Ok      Ext     Valid   bat/cmd 9009    Ok      -CALL ECHO./? produces CALL's Help; *9009 if path to bat/cmd/js exists, ignores PATHEXT
    ECHO[       Ok      Ok      9009    Ok      Ok-Call Ok      Call    Valid   bat/cmd Ok      Ok      -CALL ECHO[/? produces CALL's Help
    ECHO]       Ok      Ok      9009    Ok      Ok-Call Ok      Call    Valid   bat/cmd Ok      Ok      -CALL ECHO]/? produces CALL's Help
    ECHO(       Ok      Ok      9009    9009    Ok%Call Ok%9009 Call    Valid   Ok      Ok      Command -CALL ECHO(/? produces CALL's Help; %9009 in FOR command variable: FOR %%a in ("(")do @ECHO%%~a/?
    ECHO)       9009    9009    9009    9009    9009    9009    Ext     Valid           9009    Command
    ECHO^&      9009    9009    On/Off  Ok      9009    9009    Ext     Valid           9009    Command
    ECHO^|      9009    9009    On/Off  Ok      9009    9009    Ext     Valid           9009    Pipe
    ECHO^<      9009    9009    9009    9009    9009    9009    Ext?    Ok              9009    Ok
    ECHO^>      9009    9009    9009    9009    9009    9009    Ext?    Ok              9009    Ok
    ECHO*       9009    9009    9009    9009    9009    9009    Ext?    Ok              9009    Ok
    ECHO?       9009    9009    9009    9009    9009    9009    Ext?    Ok              9009    Ok
    ECHO_       9009    9009    9009    9009    9009    9009    Ext     Valid           9009    Ok
    ECHO-       9009    9009    9009    9009    9009    9009    Ext     Valid           9009    Ok
    ECHO!       9009    9009    9009    9009    9009    9009    Ext     Valid           9009    Ok
    ECHO@       9009    9009    9009    9009    9009    9009    Ext     Valid           9009    Ok
    ECHO#       9009    9009    9009    9009    9009    9009    Ext     Valid           9009    Ok
    ECHO$       9009    9009    9009    9009    9009    9009    Ext     Valid           9009    Ok
    ECHO%       9009    9009    9009    9009    9009    9009    Ext     Valid           9009    Ok
    ECHO^       9009    9009    9009    9009    9009    9009    Ext     Valid           9009    Ok
    ECHO{       9009    9009    9009    9009    9009    9009    Ext     Valid           9009    Ok
    ECHO}       9009    9009    9009    9009    9009    9009    Ext     Valid           9009    Ok
    ECHO'       9009    9009    9009    9009    9009    9009    Ext     Valid           9009    Ok
    ECHO`       9009    9009    9009    9009    9009    9009    Ext     Valid           9009    Ok
    ECHO~       9009    9009    9009    9009    9009    9009    Ext     Valid           9009    Ok
    ECHO"       9009    9009    9009    Unknown 9009    9009    Ext     Valid           9009    Command

Legend ("Ok" passes the test):
Echo    Displays intended blank line without giving status: ECHO is [on/off].
%Macro% Displays as intended when used as an environment variable macro
Call    Displays as intended without attempting external path search
For     Displays as intended when used as a FOR command variable
Help /? Displays as intended despite beginning with /?
Help ?  Displays as intended despite beginning with ?
Ext     Most likely remains internal, Ext means content can trigger external access to the file system: ()[]{}^=;!'+,`~ and [space] and/or .\:
File    Character cannot exist in File System preventing introduction of potential conflicts--exclude if Valid 
Path    Displays as intended despite content beginning with valid path reference \..\name.bat
Batch   Results when executed from batch file
Prompt  Results typing at Command Prompt
isDelim is a delimiter, potential conflict as a:
        Command delimiter
        Pipe delimiter
        Token delimiter

After years of wrestling with this, I think we can categorize the adverse issues into a few main categories:

• [Potentially] Destructive - Any character that can cause ECHO to inadvertently trigger invocation of any file system entry. This must be avoided at all costs.
• Confusing/Deceiving - Characters that may cause inconsistent/undesirable/unbalanced actions either due to code readability by humans or machine [yes, I'm specifically thinking of "ECHO(" which makes the code appear to have unbalanced parenthesis, or worse making it look deceivingly like a certain code block should/(not) be executed when it actually isn't].
• Cosmetic - Everything else that may result in the wrong information being displayed. [Granted, the display of wrong information could potentially cause subsequent destruction from actions taken due to incorrect information being displayed, it is not immediately destructive through the act of displaying the information itself].

There were some posts here referring to "safe" characters, which I now believe is a relative term. I'm going to refer as:

• "insecure" any character that could cause inadvertent invocation
• "safe" one that reliably displays the intended information

... I prefer to avoid any future confusion of : and \ as elements of a file path, and am now using
ECHO/

The OP's original suggestion of "/" as the ECHO delimiter actually may appear to be the most secure one, in that (at least in my. testing), I've been unable to have it inadvertently try to invoke anything from the file system. In my reference table above, ECHO/ emerges as the one that has the most "Ok" columns and doesn't have the file system invocation vulnerability.

When I look at ECHO/ as my choice of ECHO delimiter, there is really only one problem to tackle: the ? anomalies.

...
they show the HELP of the ECHO command

Code: Select all

echo/?
echo,/?
echo;/?
echo=/?

...

While I know this refers to the specific characters listed, I found these not shown interesting:

Correctly displays "/?":

Code: Select all

>ECHO//?
/?

Correctly displays "?":

Code: Select all

>ECHO,?
?
>ECHO;?
?
>ECHO=?
?

On the other hand, this show ECHO's Help:

...
ECHO/ ? WHAT'S UP DOC ?
gives a result I never expected ! !

More...
Using "/":

Code: Select all

>echo//?
/?
>echo/   / ?
   / ?
>echo/    / . ?
    / . ?
>echo/    .   /  ?
...  ECHO [ON | OFF] ...
>echo/    ?  .   /  ?
...  ECHO [ON | OFF] ...
>echo/    /  ?  .     ?
    /  ?  .     ?

Using "=":

Code: Select all

>echo=   / ?
...  ECHO [ON | OFF] ...
>echo=    / . ?
    / . ?
>echo=    .   /  ?
    .   /  ?
>echo=    ?  .   /  ?
    ?  .   /  ?
>echo=    /  ?  .     ?
...  ECHO [ON | OFF] ...

This suggests there's some sort of escaping possible:

Code: Select all

>echo/?
...  ECHO [ON | OFF] ...
>echo//?
/?
>echo///?
...  ECHO [ON | OFF] ...
>echo////?
///?
>echo/////?
...  ECHO [ON | OFF] ...
>echo//////?
/////?

So it seems that ECHO/ can be used unless the content begins with "?", in which case these work equally inconsistent:

• ECHO;
• ECHO,
• ECHO=

Has anyone found a way to reliably ECHO a line that contains a "/?" or "?" character? Using ECHO/ as a delimiter-of-choice, this would seem to be the only unsolved mystery. Except...

... because I don't think a safe form exists for CALL ECHO. ...

I agree. When CALL ECHO invokes CALL's Help, I've found no amount of escaping to stop it.

[jeb]

Hi CJM,

This subject seemed to tail off without resolution, yet using ECHO to reliably display information is such a necessity.

In my opinion, there is a champion: echo(
It's the most reliable in most of the cases.
It fails only with:

Code: Select all

:case1
call echo( /?

:case2
setlocal EnableDelayedExpansion
set "cmd=echo("
!cmd! text

:case3
for /F %%C in ("echo(") do %%C text

But the cases are not really important, because using `call echo <raw text>` is per se unreliable.
It should be only used with double percent expansion like

Code: Select all

set "var=/?"
call echo(%%var%%

And using delayed expansion or FOR-metavars with commands is unusual and the only place where it could be used is in flexible macros.

[CJM]

...
In my opinion, there is a champion: echo(
It's the most reliable in most of the cases.
It fails only with:

Code: Select all

:case1
call echo( /?

...

I see.

"call echo( /?" apparently suffers from the same issue I mentioned which suggests there might be a way to escape it somehow:

Code: Select all

Fails:	call echo( /?
Works:	call echo( //?
Fails:	call echo( ///?
Works:	call echo( ////?
Fails:	call echo( /////?
---

I thought I had test cases where "ECHO(" suffered from the external file invocation vulnerability, but I can't seem to reproduce it at the moment.

I guess I just don't want to believe that ECHO( is the best [or least worst] because of the appearance of unbalanced parenthesis.

[dbenham]

We have all been down that road - it simply looks wrong. But the countless experiments don't lie - it really is the only option that always works (except for some esoteric CALL situations, which should rarely be needed)

[jeb]

"call echo( /?" apparently suffers from the same issue I mentioned which suggests there might be a way to escape it somehow:

It's a problem of the CALL, it's independent of the command, because the command is never be used, when the CALL detects the /? before.

Code: Select all

CALL echo /?
CALL REM /?
CALL :: /?
CALL unsupported /?

[CJM]

...
It's a problem of the CALL, it's independent of the command, because the command is never be used, when the CALL detects the /? before.
...

Agreed, since CALL is the one intercepting with its own help (or not passing on the CALLed command I should say).

I was pointing out the interesting behavior where CALL's help is invoked with an odd number of slash (/) characters, while CALL ECHO succeeds in ECHOing the content with an even number of slashes. The even/odd number of slashes to me suggests there might be an as-yet-undiscovered method of escaping. There is also seemingly no way CALL ECHO's help (via CALL that is), but I can't think of a need for that anyway.