SemiColons, String Literals, Delimiters and Function Calls

[m.y.cowan]

Hi, I'm new to writing batch files, but I come across an interesting problem when passing lines of HTML code to a function. Basically I noticed that my lines of html string literals were being happily processed by my functions, until a semicolon (;) in between two double quotation marks. At that point the semicolon acted as a delimiter and the rest of the text in the line got passed to the next param in the function. I also noticed the same problem with full colons
I hope I'm just missing a newbie mistake, that can be easily fix. Is there a workaround this? Otherwise I have to do some clever explaining to my boss why I've been wasting the last week writing a batch code that will never work.

Below is quick batch file that illustrates the problem:

Thank you for your time.

BatchFunctionCallDelimitBug.bat

@echo off

REM -- Prepare the Command Processor --
SETLOCAL ENABLEEXTENSIONS
SETLOCAL DISABLEDELAYEDEXPANSION

REM.-- Version History:
REM XX.XXX YYYYMMDD Author Description
REM Version01.001-beta &:20130730 m.y.cowan to find delimiter bug when string literals are passed to a function

:BatchFunctionCallDelimitBug- parses a File line by line"
::syntax: BatchFunctionCallDelimitBug.bat "word" File
:: "word" - a throw away param
:: File [in] - file to be parsed
:: e.g. TYPE sourceText.txt|BatchFunctionCallDelimitBug.bat "word">result.txt

set "param1=param1"
set "param2=param2"
set "param3=param3"

if defined param1 set "param1=%param1:""="%"

echo param1 is %param1%
echo param2 is %param2%
echo param3 is %param3%
echo.

if "%~1"=="" findstr "^::" "%~f0"&GOTO:EOF

for /f "delims=" %%A in ('"findstr /n ^^ %3"') do (
set "line=%%A"
setlocal EnableDelayedExpansion

set "line=!line:*:=!"

echo.line is !line!

call :stringFunction "!line!" %param2% %param3%

echo.

)
goto:eof

:::::::::::::::::::::::::::::::
:stringFunction -- returns position of first occurrence of a string in another
string, case sensitive, maximum string length is 1023 characters
:: -- %~1: in - varible name of a string to be searched
:: -- %~2: in - string to be found
:: -- %~3: out- return variable name, will be set to position or undefined if string not found


SetLocal EnableDelayedExpansion
set "pos="
set "sf_param_1=%~1"
set "sf_param_2=%~2"
set "sf_param_3=%~3"

echo.findString sf_param_1 !sf_param_1! sf_param_2 !sf_param_2! sf_param_3 !sf_param_3!
EXIT /b

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

sourceText.txt
"word1" "word2" "word3" "word4" "word5" "word6"
"word1;" "word2" "word3" "word4" "word5" "word6" :: everything after the semicolon gets passed to the 2nd param
"word1"; "word2" "word3" "word4" "word5" "word6"
"word1;" "word2" "word3;" "word4" "word5" "word6" :: word2 and word3 get sent to the 2nd param, word 4, 5, 6 are sent to the 3rd

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The output should be something like this

result.txt
param1 is param1
param2 is param2
param3 is param3

line is "word1" "word2" "word3" "word4" "word5" "word6"
findString sf_param_1 "word1" "word2" "word3" "word4" "word5" "word6"
sf_param_2 param2 sf_param_3 param3

line is "word1;" "word2" "word3" "word4" "word5" "word6"
findString sf_param_1 "word1
sf_param_2 "word2" "word3" "word4" "word5" "word6" sf_param_3 param2

line is "word1"; "word2" "word3" "word4" "word5" "word6"
findString sf_param_1 "word1"; "word2" "word3" "word4" "word5" "word6" sf_param_2 param2 sf_param_3 param3

line is "word1;" "word2" "word3;" "word4" "word5" "word6"
findString sf_param_1 "word1 sf_param_2 "word2" "word3 sf_param_3 "word4" "word5" "word6"

[foxidrive]

Some characters are treated as whitespace when used on a command line or within a call - see this example. Full colons are not.

if you double quote them then they will be preserved as a single argument.

Code: Select all

@echo off
call :routine a,b;c=d:e
call :routine "a,b;c=d:e"
pause
goto :eof
:routine
echo %1 %2 %3 %4 %5

[m.y.cowan]

foxidrive,

Thanks for the reply.

But, if you notice I pass the first parameter in with double " " inside a lope

call :stringFunction "!line!" %param2% %param3%

double " " seem to work fine in with a normal function call, but I get this breakdown inside a loop and when I use ! !

Also I noticed today that with the line

line is "word1;" "word2" "word3" "word4" "word5" "word6"
findString sf_param_1 "word1 sf_param_2 "word2" "word3" "word4" "word5" "word6" sf_param_3 param2

not only does the remaining text is passed to the 2nd parameter, but the variable for the 2nd parameter gets passed to the 3rd parameter.

Weird.

[foxidrive]

It helps to discuss what you are trying to get the batch file to do.

I would comment too that your sample code is not clear enough to show the issue - simplify the code so that it is only what is required.

[m.y.cowan]

Umm okay.
This is a stripped down sample of some code I'm trying to build. It'll parse through a page of html and delete substrings i.e. jQuerry tags of varying lengths. (Yes I know there are plenty of IDEs out there that can do this, but I'm working in a very secure environment - annoyingly so) My design is have a loop parse through the lines of html. Sending each line off to some functions that will find the beginning of a jQuerry substring, then the end of the jQuery substring to get the length and then finally remove the substring.

In the code I posted here the loop parses through the lines of text from the sourceText.txt and then passes that line to :stringFunction which then prints the line out. I'm just trying to demonstrate the problem I've found - that when string literals that have semicolons inside of paired quotation marks, the semicolons seem to act like delimiters.

I would like to know how to get around this problem.

Cheers.

[jeb]

This is the most problematic line

call :stringFunction "!line!" %param2% %param3%

As it's not possible to pass a string in a safe way with CALL.

It's better only to pass the names of the variables, like

Code: Select all

call :stringFunction line param2 param3

And later to access them by

Code: Select all

:stringFunction
...
set "arg1=!%1!"
set "arg2=!%2!"
set "arg3=!%3!"

[m.y.cowan]

Thanks jeb,

I hope I can give this a try this weekend.

Cheers.