mixed results escaping space without quotes

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Post Reply
Message
Author
taripo
Posts: 228
Joined: 01 Aug 2011 13:48

mixed results escaping space without quotes

#1 Post by taripo » 31 Jul 2015 09:19

C:\>c:\a^ b\a.bat<ENTER>

it runs a.bat

I know

Code: Select all

cd a b
works, CD is clearly taking all parameters to it as one parameter.

But I notice that
C:\>c:\a b\a.bat <ENTER> <-- doesn't run "c:\a b\a.bat" it understandably, says c:\a is not recognized as an internal or extenal command.

That tells me though, that the ^ in c:\a^ b\a.bat is escaping the space.

Why then does ^ escape a space for that. But not appear to work here.

b.bat does echo %1

C:\blah>b foo^ foo<ENTER>

C:\blah>ECHO foo
foo

Do batch parameters somehow not allow ^ as an escape character?

I know echo displays all parameters or the command line, even if it receives multiple parameters. So I know that's not it.

Is there any way to escape a space besides quotes.

And why does it work for the one example but not the other?

Meerkat
Posts: 89
Joined: 19 Jul 2015 02:27
Location: Philippines

Re: mixed results escaping space without quotes

#2 Post by Meerkat » 01 Aug 2015 01:52

I am not really sure, but I think it is because of the 'weird' parsing of CMD to scripts/commands...

Here is a sample CMD session: (For the sake of clarity...)

Code: Select all

C:\Users\Public\Samp>dir /b
a b.bat
a.bat
b.bat

C:\Users\Public\Samp>type a.bat b.bat "a b".bat 2>nul
@echo HELLO!!
@echo.%1
@echo HI!!

C:\Users\Public\Samp>a
HELLO!!

C:\Users\Public\Samp>a ARGS
HELLO!!

C:\Users\Public\Samp>a^ ARGS
HELLO!!

C:\Users\Public\Samp>::a.bat works
C:\Users\Public\Samp>
C:\Users\Public\Samp>::but...
C:\Users\Public\Samp>
C:\Users\Public\Samp>%cd%\a^ b
HI!!

C:\Users\Public\Samp>::The previous command runs "a b".bat
C:\Users\Public\Samp>


I do not really know... "escaping the space to be part of file name" works if the Batch file is run in full path... :?:

...but if not, it will treat the space as a delimiter for parameters, even though it is 'escaped'.

On the b.bat part... (another sample CMD session)

Code: Select all

C:\Users\Public\Samp>type b.bat
@echo.%1

C:\Users\Public\Samp>b


C:\Users\Public\Samp>b SAMPLE
SAMPLE

C:\Users\Public\Samp>b^ HELLO
HELLO

C:\Users\Public\Samp>%cd%\b HELLO^ HELLO
HELLO

C:\Users\Public\Samp>%cd%\b^ HELLO HELLO
'C:\Users\Public\Samp\b HELLO' is not recognized as an internal or external comm
and,
operable program or batch file.

C:\Users\Public\Samp>


I am not really sure.... but I hope the CMD sessions help... :?:

Meerkat

taripo
Posts: 228
Joined: 01 Aug 2011 13:48

Re: mixed results escaping space without quotes

#3 Post by taripo » 01 Aug 2015 02:23

Good findings. So caret works in the path part where the path is a full path but doesn't ever seem to work in the arguments.

"a b.bat" has @echo %1


Code: Select all

C:\blah>%cd%\a^ b.bat foo^ bar
foo

C:\blah>

OperatorGK
Posts: 66
Joined: 13 Jan 2015 06:55

Re: mixed results escaping space without quotes

#4 Post by OperatorGK » 02 Aug 2015 14:26

AFAIK, unquoted delimiter characters ( ,;) aren't allowed in arguments for obvious reasons: echo doesn't care about delimiters, but put del or rmdir instead and results may be absolutely devastating. Imagine someone uses

Code: Select all

del file^ *

in such manner:

Code: Select all

CALL del file^ *

expecting only all files that start with "file " will be deleted. 1 argument SHOULDN'T split in 2 after re-parsing, unless user KNOWS how his output should be parsed and designed special variable.

taripo
Posts: 228
Joined: 01 Aug 2011 13:48

Re: mixed results escaping space without quotes

#5 Post by taripo » 02 Aug 2015 15:58

so it'd be safer if

Code: Select all

del file^ *


behaved like

Code: Select all

del file" "*


rather than it being interpreted as

Code: Select all

del file *


deleting the file "file", and prompting re deleting all files in the current directory.

Why are you in defense of the less safe option?

And as to echo not caring about delimiters.. yeah but there's no reason why cmd couldn't apply the rule that ^ escapes space universally. And then ^ can be done with ^^ cmd wasn't built only for echo, as you point out it has DEL too.

Also bear in mind that I used echo because of its just dumping what is put after it... because it shows how the parameters of the batch file are getting split up, by not interpreting things itself.

Also I notice, that perhaps, it is eating the caret it's just not operating the caret
(i.e. it's eating the caret, as if the caret were an escape character being applied and even removed, but it's not making the ; literal so what follows the caret, the semi colon, is not being escaped, it seems)

Code: Select all

C:\>dir r;s /b
r
s

C:\>dir r^;s /b
r
s

C:\>

OperatorGK
Posts: 66
Joined: 13 Jan 2015 06:55

Re: mixed results escaping space without quotes

#6 Post by OperatorGK » 03 Aug 2015 03:19

You don't understand: it's because argument count isn't changing with call, but it could be so if unescaped delimiters were allowed. One less caveat with call is always better. And it's eating the caret cause argument split phase is done after normal parsing phase, otherwise set "a=b c" and %a% will be parsed as 1 argument, not as 2.
Also echo follows standard rules, it just takes %* instead of %1 %2 %3....

taripo
Posts: 228
Joined: 01 Aug 2011 13:48

Re: mixed results escaping space without quotes

#7 Post by taripo » 03 Aug 2015 07:15

OperatorGK wrote:You don't understand: it's because argument count isn't changing with call, but it could be so if unescaped delimiters were allowed. One less caveat with call is always better. And it's eating the caret cause argument split phase is done after normal parsing phase, otherwise set "a=b c" and %a% will be parsed as 1 argument, not as 2.
Also echo follows standard rules, it just takes %* instead of %1 %2 %3....


You are right that i'm not understanding you.

I never even mentioned call, only you did.

And here i'm just trying to understand what you said earlier.

You describe the way the functionality works, with the words "unquoted delimiter characters ( ,;) aren't allowed in arguments"

SPACE is a delimiter

DEL a b<ENTER> <--- deletes a, and deletes b

^^^ That space has no quotes.

It's what I guess you might mean by "an unquoted delimiter".

It delimits. If it was quoted, it wouldn't delimit.

Maybe you mean quoted delimiter characters aren't allowed in arguments?

well, if the space is quoted it wouldn't be a delimiter, would it

DEL a" "b<ENTER> <--- that works too

And a quoted space is allowed and works as expected.

(My issue had nothing to do with quoted space anyway!! It was to do with a space escaped with a caret)

Maybe when you said that... "unquoted delimiter characters ( ,;) aren't allowed in arguments" you meant it doesn't allow unquoted "delimiter characters" such as ; or space, that are in the arguments to be treated as literals.. So ^SPACE isn't accepted. That is what i've said though I didn't call them delimiters when i'm talking about them being treated as literals.. And yeah the only way is for those characters to be literals in arguments is for them to be quoted ";" or " " since caret doesn't escape them.. though then they're not there alone they're there with quotes around them.. But that is just basically what I said, I didn't contradict that. I said space can be quoted in the arguments portion, just not escaped with a caret.

You write "it's eating the caret [be]cause" <-- You seem to be telling me the reason for it eating the caret as if perhaps you thought I didn't expect it to eat the caret, but I always expected it to eat the caret, and it is. My issue is it eats it but doesn't preserve the space. So eating the caret didn't seem to do anything. Whereas it does for the file path part / the filename that precedes the arguments. And you can escape a space in the arguments, just not with caret, and that seems strange.


You make an interesting point about ECHO taking %* while DEL and others use %1 %2 %3 (so you imply that those internal commands have the same choice that batch files have.. that they see arguments the same way, that's interesting)

You use an example of SET "a=b c" (I can see set, like echo, must take %* hence set a=b works, not just set a"="b) unlike dir or del which takes %1 %2 etc I get that, that's interesting.


You mention a normal phase and a splitting phase, i'm not sure what aspects of them you refer to, and I have no doubt that if there are those two phases then the normal phase(involving eating carets), and deciding which spaces are preserved, is done before argument splitting. But you then say that if it didn't work like that then set "a=b c" and %a% will be parsed as 1 argument, not as 2. (As if you say, lucky it is passed as 2 arguments!) So you seem to imply that set "a=b c" is 2 arguments passed to set, but it's not, it is 1! Even set a=b" "c is one argument passed to set.

I can see that you can't caret escape delimiters in arguments %1 %2 %3

so, not just space but ; too..

but you can quote space or ; or whatever.

so whatever the rule is.. e.g. one rule for file path, another for %1 %2 %3 parameters at least it's easy to test, because if a command like DEL or DIR sees its parameters as %1 %2 %3 then I can echo %1 %2 and work out what it is seeing.

But it really seems the file path rule for carets is different from the argument one(as in, for batch files or internal commands).. it'd be interesting to see that rule written anywhere.

Code: Select all

C:\blah>type a.bat
echo %1
C:\blah>a t;g;h

C:\blah>echo t
t

C:\blah>a t^;g^;h

C:\blah>echo t
t

C:\blah>a t";"g";"h

C:\blah>echo t";"g";"h
t";"g";"h

OperatorGK
Posts: 66
Joined: 13 Jan 2015 06:55

Re: mixed results escaping space without quotes

#8 Post by OperatorGK » 03 Aug 2015 09:34


SPACE is a delimiter

DEL a b<ENTER> <--- deletes a, and deletes b

^^^ That space has no quotes

Code: Select all

DEL a b

I said "delimiter characters". Delimiter characters in cmd are " ,;" quoted or unquoted. It's just a name for group.
"a","b" is arguments. Space between them is delimiter character. It's unquoted, and because of that it is delimiter.
I'll drop reasons why xxx is better and just explain how cmd parses arguments.
I'll assume echo uses %* and other use %1 %2 %3 or their equivalents.
Also I drop everything unneeded (stream redirection, delayed variable expansion, etc).

First, cmd.exe searches for unescaped and unquoted "&|><" characters. If any found, they are specially treated.
Second, it expands all %variables%.
Third, cmd.exe removes all unescaped and unquoted carets.
Fourth, it splits "%*" into "%1 %2 %3...".
This is all I worked out.

taripo
Posts: 228
Joined: 01 Aug 2011 13:48

Re: mixed results escaping space without quotes

#9 Post by taripo » 03 Aug 2015 09:47

OperatorGK wrote:First, cmd.exe searches for unescaped and unquoted "&|><" characters. If any found, they are specially treated.
Second, it expands all %variables%.
Third, cmd.exe removes all unescaped and unquoted carets.
Fourth, it splits "%*" into "%1 %2 %3...".
This is all I worked out.


So where does Space and comma and semicolon fit into those rules? At the Fourth stage?

And what about a space comma or semicolon that occurs in the file path or filename (I know what happens but it doesn't seem to be covered by your rules) e.g.

C:\blah>a^ b.bat<ENTER>
or
C:\blah>a^ b.bat hello^ bob<ENTER>

At stage 3 if I apply that rule it turns

C:\blah>a^ b.bat hello^ bob<ENTER>

into

C:\blah>a b.bat hello bob<ENTER>

which still doesn't explain / cover / determine why you get a preserved/literal space from the first ^SPACE on the line.

It might explain why you get a delimiting space from the second ^SPACE on the line. 'cos the caret got removed and it makes no difference (by your rules).. But it does make a difference in the case of the first ^SPACE on the line.

OperatorGK
Posts: 66
Joined: 13 Jan 2015 06:55

Re: mixed results escaping space without quotes

#10 Post by OperatorGK » 03 Aug 2015 11:04

taripo wrote:
OperatorGK wrote:First, cmd.exe searches for unescaped and unquoted "&|><" characters. If any found, they are specially treated.
Second, it expands all %variables%.
Third, cmd.exe removes all unescaped and unquoted carets.
Fourth, it splits "%*" into "%1 %2 %3...".
This is all I worked out.


So where does Space and comma and semicolon fit into those rules? At the Fourth stage?

And what about a space comma or semicolon that occurs in the file path or filename (I know what happens but it doesn't seem to be covered by your rules) e.g.

C:\blah>a^ b.bat<ENTER>
or
C:\blah>a^ b.bat hello^ bob<ENTER>

At stage 3 if I apply that rule it turns

C:\blah>a^ b.bat hello^ bob<ENTER>

into

C:\blah>a b.bat hello bob<ENTER>

which still doesn't explain / cover / determine why you get a preserved/literal space from the first ^SPACE on the line.

It might explain why you get a delimiting space from the second ^SPACE on the line. 'cos the caret got removed and it makes no difference (by your rules).. But it does make a difference in the case of the first ^SPACE on the line.

Yes, at fourth stage.
I don't know how to figure out file/internal command name parsing rules. I only know that these rules are complicated and there isn't known way to ABSOLUTELY safe echo something in EVERY situation. Also, even these stages that I wrote may be COMPLETELY different, it's only what they seem to be.

Meerkat
Posts: 89
Joined: 19 Jul 2015 02:27
Location: Philippines

Re: mixed results escaping space without quotes

#11 Post by Meerkat » 04 Aug 2015 01:50

Hmm... just an extra piece of more weirdness. :mrgreen:

Code: Select all

C:\Users\Public\Samp>echo @echo THIS WORKED.>>"A   B.BAT"

C:\Users\Public\Samp>:::Three spaces:::
C:\Users\Public\Samp>
C:\Users\Public\Samp>%cd%\A^ ^ ^ B
THIS WORKED.

C:\Users\Public\Samp>A^ ^ ^ B
'A' is not recognized as an internal or external command,
operable program or batch file.

C:\Users\Public\Samp>\A^ ^ ^ B
'\A' is not recognized as an internal or external command,
operable program or batch file.

C:\Users\Public\Samp>.\A^ ^ ^ B
'.\A' is not recognized as an internal or external command,
operable program or batch file.

C:\Users\Public\Samp>:::Failed, but...
C:\Users\Public\Samp>
C:\Users\Public\Samp>^ \A^ ^ ^ B
THIS WORKED.

C:\Users\Public\Samp>:::Yeah!!!
C:\Users\Public\Samp>
C:\Users\Public\Samp>


The weird thing is ^<space>\ and then the file name...

-----

Of course, I do not recommend this to be used, especially when we will execute an EXE file that needs parameters, because they will treat the arguments differently. For instance, if the file is "A B.EXE" and we used ^ \A^ ^ ^ B <1st param> <2nd param>. Then, the fist param interpreted by the "A B.EXE" might be "\A^", then the second might be "^", so on...


Meerkat

OperatorGK
Posts: 66
Joined: 13 Jan 2015 06:55

Re: mixed results escaping space without quotes

#12 Post by OperatorGK » 04 Aug 2015 09:33

Guys, @jeb showed me link how the CMD.EXE parses nearly everything. I hope it helps:
http://stackoverflow.com/questions/4094699/how-does-the-windows-command-interpreter-cmd-exe-parse-scripts/4095133#4095133

taripo
Posts: 228
Joined: 01 Aug 2011 13:48

Re: mixed results escaping space without quotes

#13 Post by taripo » 06 Aug 2015 12:37

[quote="operator]
Guys, @jeb showed me link how the CMD.EXE parses nearly everything. I hope it helps:
http://stackoverflow.com/questions/4094 ... 33#4095133
[/quote]

I'm aware of jeb and dave benham's posts on the batch parser at that link, but I couldn't find the answer there, so maybe I am not looking at the right thing there. I know they contribute here so i'm hoping they drop by over here 'cos this is clearly a tough one.

Note:

A comment on that SO link mentions a google translation of jeb's post

http://www.administrator.de/detail/Die_ ... erpreters/

it's really good
Last edited by taripo on 07 Aug 2015 05:03, edited 2 times in total.

jeb
Expert
Posts: 1055
Joined: 30 Aug 2007 08:05
Location: Germany, Bochum

Re: mixed results escaping space without quotes

#14 Post by jeb » 07 Aug 2015 02:06

Hi taripo,

you are right, there is nothing about this in the batch parser description.

Some time ago, at dostips was a discussion about quotes and unknown wildcard characters in filenames.

ex. All these variants starts mybatch.bat

Code: Select all

myBatch
"myBatch.bat"
m"yB"""atch"."bat


It was more or less generally known that a single parameter, created from the command line or by a CALL, can't contain unquoted spaces.
But the carets are still usefull to escape special characters.

Code: Select all

myBatch.bat "one parameter"
myBatch.bat ^"one parameter^"
myBatch.bat two parameters
myBatch.bat two^ parameters
myBatch.bat two^^ parameters
myBatch.bat two^^^ parameters
myBatch.bat one^&parameter


Therefore, I was really surprised that the absolute path variant respects the escaped spaces, but it still fails with the relative paths.

As this only works for the command itself, I suppose there could be different parsers for fetching the command from the command line.

Post Reply