New/unknown behaviour in percent/delayed expansion

[jeb]

Unbelievable

From the post [url="http://www.dostips.com/forum/viewtopic.php?f=3&t=7233#p47191"]One-liners for logical OR operator[/url]

Probably not as you (sambul35) expected (you should use delayed expansion and replace the outer percentage characters with exclamation marks; i've also added a semicolon),
but it is the first time i saw three percentage characters in one variable working:
Code:

:: test.bat
@echo off
setlocal enableDelayedExpansion
set "cities=;York;London;Brussels;%%1;%%2;%%3;"
echo %cities:;%2;=;%
echo !cities:;%2;=;!
endlocal
goto :eof


Code:

Z:\>test.bat London London
;York;London;Brussels;%1;%3;
;York;Brussels;%1;%2;%3;

Why does this work?

First I thought, that it's a simple case where the parser rules can explain it.
But, thats not true it's a new behaviour.
I have to expand the percent/delayed expansion rules

When a variable replace expression begins with any character, except a percent, then the search pattern will be build up to the the first equal sign.

Code: Select all

%var:A??????=replace%

For the question marks is any character allowed, except the equal sign, but even percent signs.

But in the case you use percent signs in the search pattern, they will not expand anything, they are handled as normal characters.
The same rules applies to delayed expansion for the exclamation marks.

Code: Select all

set "var1=Begin #%%~ End"
echo 1: %var1:#%~=Invalid%

set var2=#Begin #%%X%% End
set "X=Begin"
echo 2: %var2:#%X%=REPLACE%

1: Begin Invalid End
2: #Begin REPLACE End

It's still not possible to replace a single percent sign with this behaviour, as you always need a prefixed character.
But now it's possible to replace all character up to the first percent sign.

Code: Select all

set "var=One percent%% sign"
echo %var:*%=No%

jeb

[sambul35]

It's still not possible to replace a single percent sign with this behavior, as you always need a prefixed character. But now it's possible to replace all character up to the first percent sign.

Or after when a single percent sign identifies a batch or function argument like %4, if I got the above post right.

[jeb]

jeb wrote:
It's still not possible to replace a single percent sign with this behavior, as you always need a prefixed character.

Except when a single percent sign identifies a batch or function argument like %4, if I got the above post right.

You doesn't got it right

If you have a variable containing some percent signs that you want to replace with the text "PERCENT", that's still not possible this way.

Code: Select all

set "var=This text%% contains %% some percent %% signs"
set "fail=%var:%=PERCENT%"   -- This one fails
set "good=%var: %=PERCENT%"   -- This one works, but it needs the space before the percent sign and it replaces both signs
setlocal EnableDelayedExpansion
set "oldWay=!var:%%=PERCENT!"  -- This one works as expected

[sambul35]

Great! Now its possible to use either an argument or variable in Replace expression. It adds to the known use of CALL SET in Replace expressions mentioned here.

And the variable to be replaced by a string can be delayed expended !var! or only static %var%?

And the target replacement string can also be a variable - delayed expended or static?

[penpen]

I've created an applicationexample (not obvious, but simple enough i think):
http://www.dostips.com/forum/viewtopic.php?p=47219#p47219.


penpen

[dbenham]

Unbelievable
. . .
First I thought, that it's a simple case where the parser rules can explain it.
But, thats not true it's a new behaviour.
I have to expand the percent/delayed expansion rules

I was as shocked as you. But the %VAR% expansion rules that I laid out (with your help) actually were consistent with the behavior
The only issue with those rules (before my edit today) is they did not specify which characters can be used within the search and replace strings. But they certainly didn't exclude the %.

To clarify the rule, all we need is the following:

<LF> aborts the search=replace parsing. The first = terminates the search string, after which the next % terminates the replace string.

So the search string can contain any characters except <LF> or =
The first search character cannot be % because an earlier rule will have already attempted to expand %VAR:%

The replacement string can contain any characters except <LF> or %

Note that both search and replace can contain <CR> because the %VAR% expansion in phase 1.3 occurs before <CR> is stripped in phase 1.5
But <CR> in the replace string is pointless because phase 1.5 will simply strip any <CR> that was inserted

I've updated rule 1.3 at StackOverflow to account for the new information.


Dave Benham

[sambul35]

I've created an application example (not obvious, but simple enough i think):
http://www.dostips.com/forum/viewtopic.php?p=47219#p47219.

It depends on who reads it, likewise the Rules at Stackoverflow.

I still can't understand, if the above discovery affects only regular %var% variables, or also delayed expanded variables !var!, used as a substring in Replace expressions?
Can someone give a clear answer to these questions:

1. Can the variable to be replaced by a string be delayed expended !var! or only static %var%? Any escapes required for ! or %?
2. Can the replacement string also be a delayed expended or static variable? Any escapes required for ! or %?

[penpen]

I'm unsure if i do understand you right.
The new discovery is that within the search string (at position 2 or later) you could use:
- percentage characters during the percentage expansion, and
- exclamation marks for delayed Expansion.

Code: Select all

@echo off
setlocal enableExtensions enableDelayedExpansion
set "_A= - %% - %% - %% - %%"
set "_B= ^! _ ^! _ ^! _ ^! _"

set _A
echo    %_A: %= +%
echo(

set _B
echo    !_B: != ^^!

endlocal
goto :eof

Ad 1)
You could replace a string like " %var%" (without doublequotes) stored within a variable A with any other string in percentage/normal expansion.
Also you could replace a string " !var!" (without doublequotes) stored within a variable A with any other string in exclamation mark/delayed expansion.
For both you don't need to escape these characters.

Ad 2)
The replacement string could hold both a delayed expanded, and a default expandable variable string, but the desired expansion step may be already processed.
Note that an expression "%variable:search=replace%" percentage characters are only allowed within the "search" part (at the second or later position); the percentage character is not allowed at the replace string - but you could use exclamation marks where you want.
(Delayed Expansion: Analogous.)

Code: Select all

@echo off
setlocal enableExtensions enableDelayedExpansion
set "A=a"
set "B=%%A%% %%A%%"
echo %B: %A%= !A!%
endlocal & echo %B: %A%= !A!%
goto :eof

penpen

[sambul35]

Thanks for the explanation, sounds like very effective approach to variable match and replacement techniques.