Help me escape special characters used in a %VAR% variable

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Post Reply
Message
Author
alan_b
Expert
Posts: 357
Joined: 04 Oct 2008 09:49

Help me escape special characters used in a %VAR% variable

#1 Post by alan_b » 03 Dec 2011 06:13

These fail to work for me :-

Code: Select all

SET "COMMAND=('findstr /n "^" %2')" & CALL :NEW_PROC %1 & GOTO :EOF
SET ""COMMAND=('findstr /n "^" %2')"" & CALL :NEW_PROC %1 & GOTO :EOF

Is there any alternative to

Code: Select all

SET "COMMAND=('findstr /n "^^" %2')" & CALL :NEW_PROC %1 & GOTO :EOF

NB The above is used in conjunction with

Code: Select all

:NEW_PROC
<%1 (
  for /f %%i in %COMMAND% do (
    set "ln=" & set /p "ln=" & echo(!ln!
  )
)
GOTO :EOF


My problem is that I am starting with proven code with many instances such as

Code: Select all

:_D
<%1 (
  for /f %%i in ('findstr /n "^" %2') do (
    set "ln=" & set /p "ln=" & echo(!ln!
  )
)
GOTO :EOF

:_P
<%1 (
  for /f %%i in ('findstr /n /v "^" ^< %2') do (
    set "ln=" & set /p "ln=" & echo(!ln!
  )
)
GOTO :EOF

Adding an extra ^ before each existing ^ needs attention to detail,
and there may be other special characters that could trip me up.
I was hoping that two sets of double quotes around ""COMMAND=('findstr /n "^" %2')"" might fix the problem - but it fails me.

I need to add a large block of code to convert each processed file into an indexed array of text
and then perform some additional processing.
I intend to have just one screen full of code executed in
:NEW_PROC
rather than several dozen instances inserted into each portion of my present "in-line" code.

Regards
Alan

Ed Dyreen
Expert
Posts: 1569
Joined: 16 May 2011 08:21
Location: Flanders(Belgium)
Contact:

Re: Help me escape special characters used in a %VAR% variab

#2 Post by Ed Dyreen » 03 Dec 2011 06:22

:D
Hi alan

Code: Select all

@echo off

SET ^"COMMAND=('findstr /n ^"^^^" %%2')"

SET COMMAND
pause
exit

Code: Select all

COMMAND=('findstr /n "^" %2')
Druk op een toets om door te gaan. . .

jeb
Expert
Posts: 967
Joined: 30 Aug 2007 08:05
Location: Germany, Bochum

Re: Help me escape special characters used in a %VAR% variab

#3 Post by jeb » 03 Dec 2011 07:27

Hi Alan,
alan_b wrote:Adding an extra ^ before each existing ^ needs attention to detail,
and there may be other special characters that could trip me up.
I was hoping that two sets of double quotes around ""COMMAND=('findstr /n "^" %2')"" might fix the problem - but it fails me.

There is a simple solution with delayed expansion!

Code: Select all

@echo off
setlocal EnableDelayedExpansion
SET "COMMAND='findstr /n "^^" <%2'"
CALL :NEW_PROC %1
GOTO :EOF

:NEW_PROC
<%1 (
  for /f %%i in (!COMMAND!) do (
    set "ln=" & set /p "ln=" & echo(!ln!###%%i
  )
)
GOTO :EOF

The big difference is to expand the command with !command! instead of %command%, as the content of the delayed expansion can't be changed by the parser anymore.
But therefore you need the parenthesis directly in the FOR loop, not in the !command!, otherwise the parsing of the for-loop will fail.

hope it helps
jeb

alan_b
Expert
Posts: 357
Joined: 04 Oct 2008 09:49

Re: Help me escape special characters used in a %VAR% variab

#4 Post by alan_b » 03 Dec 2011 09:00

Sorry but I need to explain again.

This works perfectly

Code: Select all

SET "COMMAND=('findstr /n "^^" %2')" & CALL :NEW_PROC %1 & GOTO :EOF
....
:NEW_PROC
<%1 (
  for /f %%i in %COMMAND% do (
    set "ln=" & set /p "ln=" & echo(!ln!
  )
)
GOTO :EOF

My problem is the need to mentally think about and manually change my existing fully debugged and working code from
('findstr /n "^" %2')
to
('findstr /n "^^" %2')

The above works exactly as required

I also have the inconvenience of handling
('type %2 ^| findstr /n /v "^"')
and
('find /n /v "" ^< %2')
and
('findstr /n /v "^" ^< %2')

I assume that a simple sweep with NotePad through the whole script will instantly change every instance of a single ^ into a double ^^
But I have yet to test that for validity,
and I do not know if ^ precedes anything else which might go wrong should a ^^ replacement be performed.

Perhaps you can advise me if it is safe to blindly substitute every instance of s single ^ into a double ^^.

My original hope was a simple replacement of
every =( with =~(
and every ) with )~
where ~ denotes some magic character that removes the need to double up on each single ^

Additionally, I do not know if single ^ are the only characters to think about.

Regards
Alan

dbenham
Expert
Posts: 2383
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Help me escape special characters used in a %VAR% variab

#5 Post by dbenham » 03 Dec 2011 13:49

Alan - I think the solution is insanely simple: :D

I think this will work:

Code: Select all

SET COMMAND=('findstr /n "^" %2') & CALL :NEW_PROC %1 & GOTO :EOF


I believe any string that works in your existing inline code should work using the above.


Dave Benham

alan_b
Expert
Posts: 357
Joined: 04 Oct 2008 09:49

Re: Help me escape special characters used in a %VAR% variab

#6 Post by alan_b » 03 Dec 2011 17:09

Dave

I was really surprised that it was so easy.
SET COMMAND=('findstr /n "^" %2') & CALL :NEW_PROC
does indeed result in :NEW_PROC performing exactly like the in-line code
<%1 ( for /f %%i in ('findstr /n "^" %2') do (

There are absolutely no manipulations required with the round ( ) brackets.

I found a problem with
<%1 ( for /f %%i in ('type %2 ^| findstr /n /v "^"')
The ^| requires quotes, i.e.
SET COMMAND=('type %2 "^|" findstr /n /v "^"')
problem fixed.

I am stuck with this problem :-
This was original working code

Code: Select all

:_S
echo :==========="('find /n /v "" ^< %2')"==========
<%1 (  for /f %%i in ('find /n /v "" ^< %2') do (
    set "ln=" & set /p "ln=" & echo(!ln!
  )
)
GOTO :EOF

It produced an output table
MODE = _S. Files :- OUTER = File3.TXT ; INNER = File3.TXT.
file 3 line 1 of 6 but missing line 2 ####

file 3 line 3 of 6 but missing line 2 ####
file 3 line 4 of 6 but missing line 2 ####
file 3 line 5 of 6 but missing line 2 ####
file 3 line 6 of 6 but missing line 2 ####
SUCCESS ---MODE = _S : File3.TXT File3.TXT PROCESSED 6 LINES

[/quote]
I used the exact same ('find /n /v "" ^< %2') within replacement code

Code: Select all

:_S
  SET COMMAND=('find /n "" ^< %2') & CALL :NEW_PROC %1 & GOTO :EOF

The result was
MODE = _S. Files :- OUTER = File3.TXT ; INNER = File3.TXT.
< was unexpected at this time.

I wrapped quotes around the ^<

Code: Select all

:_S
  SET COMMAND=('find /n "" "^<" %2') & CALL :NEW_PROC %1 & GOTO :EOF

The result was
MODE = _S. Files :- OUTER = File3.TXT ; INNER = File3.TXT.
File not found - ^<


My code concludes with

Code: Select all

:NEW_PROC
<%1 (
  for /f %%i in %COMMAND% do (
    set "ln=" & set /p "ln=" & echo(!ln!
  )
)
GOTO :EOF


DEBUG TWEAK :-
:NEW_PROC
set command
<%1 (
for /f %%i in %COMMAND% do (
set "ln=" & set /p "ln=" & echo(!ln!
)
)
GOTO :EOF

THE RESULT IS NOW
MODE = _S. Files :- OUTER = File3.TXT ; INNER = File3.TXT.
COMMAND=('find /n "" "^<" File3.TXT')
File not found - ^<


Conclusion :-
I desperately need a convenient way to cause either :NEW_PROC or some variant to execute
for /f %%i in ('find /n /v "" ^< %2') do (

Regards
Alan

OJBakker
Expert
Posts: 32
Joined: 12 Aug 2011 13:57

Re: Help me escape special characters used in a %VAR% variab

#7 Post by OJBakker » 03 Dec 2011 19:12

I am not entirely sure but these might work:
SET COMMAND=('find /n "" < %2')
SET COMMAND=('find /n "" %2')


The first: leave out the caret
The second: leave out the redirect, find does not need it

dbenham
Expert
Posts: 2383
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Help me escape special characters used in a %VAR% variab

#8 Post by dbenham » 03 Dec 2011 21:38

Try combining my approach with jeb's. See if this does not solve all the problems (I think this is what jeb was intending all along):

Code: Select all

setlocal disableDelayedExpansion
set COMMAND='findstr /n "^" %2'&call :new_proc %1
set COMMAND='find /n /v "" ^< %2'&call :new_proc %1
set COMMAND='type %2 ^| find /n /v ""'&call :new_proc %1
exit /b

:NEW_PROC
<%1 (
  setlocal enableDelayedExpansion
  for /f %%i in (!COMMAND!) do (
    set "ln=" & set /p "ln=" & echo(!ln!
  )
)
GOTO :EOF

I postponed enabling delayed expansion just in case %1 or %2 contain !


Dave Benham

alan_b
Expert
Posts: 357
Joined: 04 Oct 2008 09:49

Re: Help me escape special characters used in a %VAR% variab

#9 Post by alan_b » 04 Dec 2011 03:03

Thank you both

I went to bed dreading the thought I might have to resort to chopping out the
^< %2')
from every SET COMMAND=.... which concluded with the above, and call a new special

Code: Select all

:NEW_PROC_YUK
<%1 (
  for /f %%i in (%COMMAND_START% ^< %2') do (
    set "ln=" & set /p "ln=" & echo(!ln!
  )
)
GOTO :EOF


I am a lot happier now an will try out your suggestions immediately after lunch.
Right now it is time to prepare for church

Regards
Alan

dbenham
Expert
Posts: 2383
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Help me escape special characters used in a %VAR% variab

#10 Post by dbenham » 04 Dec 2011 03:50

OJBakker wrote:I am not entirely sure but these might work:
SET COMMAND=('find /n "" < %2')
SET COMMAND=('find /n "" %2')

The first: leave out the caret
The second: leave out the redirect, find does not need it

first - The SET command fails without the caret
second - Unfortunately FIND prefixes the output with the file name using that syntax.


Dave Benham

alan_b
Expert
Posts: 357
Joined: 04 Oct 2008 09:49

Re: Help me escape special characters used in a %VAR% variab

#11 Post by alan_b » 05 Dec 2011 05:00

Dave

Many thanks, your solution worked perfectly.

There was one problem with my secondary target :-
"I need to add a large block of code to convert each processed file into an indexed array of text"

I found that the variables created by previous use of :NEW_PROC were destroyed each time :NEW_PROC was used.
That was fixed by removing "setlocal enableDelayedExpansion" from :NEW_PROC

This is code OutPut demonstration :-
Mode _S was my original in-line code that took 6 lines.
Mode _T is your single line that takes only 1 line to call :NEW_PROC
Modes _U and _V are your (Type ...) and (Findstr ...) variants which do the same perfect job.
--- starting --- --- starting --- --- starting ---


Original 6 lines of code, using "^<"
:*********************************************
MODE = _S. Files :- OUTER(%1) = File3.TXT; INNER(%2) = File4.TXT.
:==========="('find /n /v "" ^< File4.TXT')"==========
file 3 line 1 of 6 but missing line 2 ####

file 3 line 3 of 6 but missing line 2 ####
file 3 line 4 of 6 but missing line 2 ####
file 3 line 5 of 6 but missing line 2 ####
file 3 line 6 of 6 but missing line 2 ####

New 1 line code, creating variables, using "^<"
:*********************************************
MODE = _T. Files :- OUTER(%1) = File3.TXT; INNER(%2) = File4.TXT.
:=========== ('find /n /v "" < File4.TXT') --------------
file 3 line 1 of 6 but missing line 2 ####

file 3 line 3 of 6 but missing line 2 ####
file 3 line 4 of 6 but missing line 2 ####
file 3 line 5 of 6 but missing line 2 ####
file 3 line 6 of 6 but missing line 2 ####

New 1 line code, creating variables, using "^"
:*********************************************
MODE = _U. Files :- OUTER(%1) = File3.TXT; INNER(%2) = File4.TXT.
:=========== ('findstr /n "^" File4.TXT') --------------
file 3 line 1 of 6 but missing line 2 ####

file 3 line 3 of 6 but missing line 2 ####
file 3 line 4 of 6 but missing line 2 ####
file 3 line 5 of 6 but missing line 2 ####
file 3 line 6 of 6 but missing line 2 ####

New 1 line code, creating variables, using "^|"
:*********************************************
MODE = _V. Files :- OUTER(%1) = File3.TXT; INNER(%2) = File4.TXT.
:=========== ('type File4.TXT | find /n /v ""') --------------
file 3 line 1 of 6 but missing line 2 ####

file 3 line 3 of 6 but missing line 2 ####
file 3 line 4 of 6 but missing line 2 ####
file 3 line 5 of 6 but missing line 2 ####
file 3 line 6 of 6 but missing line 2 ####
ALL TESTS DONE

Generated Variables :-
A_1010=MODE = _S. Files :- OUTER(%1) = File3.TXT; INNER(%2) = File4.TXT.
A_1021=MODE = _T. Files :- OUTER(%1) = File3.TXT; INNER(%2) = File4.TXT.
A_1032=Mode-_T
A_1034=file 3 line 1 of 6 but missing line 2 ####
A_1036=file 3 line 3 of 6 but missing line 2 ####
A_1037=file 3 line 4 of 6 but missing line 2 ####
A_1038=file 3 line 5 of 6 but missing line 2 ####
A_1039=file 3 line 6 of 6 but missing line 2 ####
A_1049=MODE = _U. Files :- OUTER(%1) = File3.TXT; INNER(%2) = File4.TXT.
A_1060=Mode-_U
A_1062=file 3 line 1 of 6 but missing line 2 ####
A_1064=file 3 line 3 of 6 but missing line 2 ####
A_1065=file 3 line 4 of 6 but missing line 2 ####
A_1066=file 3 line 5 of 6 but missing line 2 ####
A_1067=file 3 line 6 of 6 but missing line 2 ####
A_1077=MODE = _V. Files :- OUTER(%1) = File3.TXT; INNER(%2) = File4.TXT.
A_1088=Mode-_V
A_1090=file 3 line 1 of 6 but missing line 2 ####
A_1092=file 3 line 3 of 6 but missing line 2 ####
A_1093=file 3 line 4 of 6 but missing line 2 ####
A_1094=file 3 line 5 of 6 but missing line 2 ####
A_1095=file 3 line 6 of 6 but missing line 2 ####
Press any key to continue . . .


As you can see - I even get indexed arrays to play with.

This is the code that created the above

Code: Select all

@echo off & setlocal EnableDelayedExpansion & CLS

echo --- starting ---  --- starting ---  --- starting ---    & ECHO(

SET /A Ndx=1000
ECHO( & ECHO Original 6 lines of code, using "^<"
CALL :TESTX File3.TXT File4.TXT _S
ECHO( & ECHO New 1 line code, creating variables, using "^<"
CALL :TESTX File3.TXT File4.TXT _T
ECHO( & ECHO New 1 line code, creating variables, using "^"
CALL :TESTX File3.TXT File4.TXT _U
ECHO( & ECHO New 1 line code, creating variables, using "^|"
CALL :TESTX File3.TXT File4.TXT _V

ECHO ALL TESTS DONE & ECHO(
ECHO( Generated Variables :- & SET A_

PAUSE
GOTO :EOF


:TESTX
echo :*********************************************

SET /A Ndx+=10 & SET A_!Ndx!=MODE = %3.  Files :-  OUTER(%%1) = %1; INNER(%%2) = %2.
ECHO !A_%Ndx%! & SET /A Ndx+=1
GOTO :%3

:_S
echo :==========="('find /n /v "" ^< %2')"==========
<%1 (  for /f %%i in ('find /n /v "" ^< %2') do (
    set "ln=" & set /p "ln=" & echo(!ln!
  )
)
GOTO :EOF

:_T
set COMMAND='find /n /v "" ^< %2'&call :new_proc %* & GOTO :EOF
:_U
set COMMAND='findstr /n "^" %2'&call :new_proc %* & GOTO :EOF
:_V
set COMMAND='type %2 ^| find /n /v ""'&call :new_proc %* MODE %1 & GOTO :EOF


:NEW_PROC
echo :=========== (!COMMAND!) --------------
REM
SET /A Ndx+=10 & SET "A_!Ndx!=Mode-%3" & SET /A Ndx+=1
<%1 (
  for /f %%i in (!COMMAND!) do (
    set "ln=" & set /p "ln=" & echo(!ln!
    SET /A Ndx+=1 & SET "A_!Ndx!=!ln!"
  )
)
GOTO :EOF

I now have a perfect single line replacement for the "Mode :_S" test
I have 27 other test modes to convert,
but I expect no trouble due to any variant that includes ^<, "^", or ^=

Very many thanks Dave, perfect solution to my needs.

Regards
Alan

Post Reply