 |
new functions: :chr, :asc, :asciiMap
| Author |
Message |
|
jeb
Expert
Joined: 30 Aug 2007 08:05 Posts: 567 Location: Germany
|
 Re: new functions: :chr, :asc, :asciiMap
I suppose that !k doesn't see any "E" with a cyrillic font.
But perhaps you could be right, that shutdown always have an unlocaized part.
jeb (sleeping already)
|
| 04 Apr 2011 17:04 |
|
 |
|
aGerman
Expert
Joined: 22 Jan 2010 18:01 Posts: 1395 Location: Germany
|
 Re: new functions: :chr, :asc, :asciiMap
Good morning, jeb I couldn't figure out whether the E is language independent, but now I know that shutdown /? doesn't work in general. The output on the XP machine at work: Code: Usage: shutdown [-i | -l | -s | -r | -a] [-f] [-m \\computername] [-t xx] [-c "comment"] [-d up:xx:yy]
No args Display this message (same as -?) -i Display GUI interface, must be the first option -l Log off (cannot be used with -m option) -s Shutdown the computer -r Shutdown and restart the computer -a Abort a system shutdown -m \\computername Remote computer to shutdown/restart/abort -t xx Set timeout for shutdown to xx seconds -c "comment" Shutdown comment (maximum of 127 characters) -f Forces running applications to close without warning -d [u][p]:xx:yy The reason code for the shutdown u is the user code p is a planned shutdown code xx is the major reason code (positive integer less than 256) yy is the minor reason code (positive integer less than 65536)
There is no TAB at all Regards aGerman
|
| 05 Apr 2011 10:37 |
|
 |
|
dbenham
Expert
Joined: 12 Feb 2011 21:02 Posts: 888 Location: United States (east coast)
|
 Re: new functions: :chr, :asc, :asciiMap
Well I finally did what should have been obvious, and downloaded a free Hex editor. (thanks Jeb) It's much easier to investigate and work things out now. I added two additional related functions: :hex2str converts a string of hex digits into a string (for example 414243 -> ABC) :str2hex converts a string into a string of hex digits (for example ABC -> 414243) The code is at the end of this post. Using Jeb's tips (with a few modifications) the functions now support all but the following two characters: NUL 0x00 and LF 0x0A. Most of the added characters are embedded directly in the batch file. I reconfigured my editor to preserve Tabs (I use Context - it was configured to convert tabs into 2 spaces). By default, only the characters that are embedded in the batch file are available. The optional /X switch adds support for CR 0x0D and SUB 0x1A by adding them programatically. I structured things this way because the /X option takes significantly longer to build the ASCII map, and variables with CR can only be accessed with delayed expansion. I don't want to penalize calls that don't need CR or SUB. The /X option requires writing and reading a temporary file. I used the DOS Tips :Unique function plus %random% to generate a temporary file name that should prevent collisions even during concurrent use on a shared drive. I would love to add support for LF 0x0A. I tried many variations of Jeb's for loop idea, but I could never get it to work when combined with the full ASCII map. I think if I restrict the code to :asc and :chr I could make things work with ugly code. But :hex2str really causes problems. The problem I have is when trying to pass a string that requires delayed expansion across the function return boundry. I need a technique that will allow me to pass any combination of characters across the boundry. Usually the problem involves the for loop parsing the string into two or more strings such that the loop has multiple iterations. The technique I am currently using splits the string at each LF but works for all other characters. Some notes on what I discovered: 1) CR passes through my variation of the function for loop return technique as long as it is not the last character in the string. If it is the last character it gets dropped. I solved this by simply appending an extra CR on the end if and only if the string ends in CR. 2) I ended up not using this, but I tested Jeb's code to generate a BS 0x08 and I found one small flaw that is easy to fix. Jeb's current DEL variable actually has length 3. It consists of <BS><Space><BS> (hex 08 20 08). A simple substring operation trims it down to the desired single character. Code: :BL.String.CreateDEL_ESC :: Creates two variables with one character DEL=Ascii-08 and ESC=Ascii-27 :: DEL and ESC can be used with and without DelayedExpansion setlocal for /F "tokens=1,2 delims=#" %%a in ('"prompt #$H#$E# & echo on & for %%b in (1) do rem"') do ( ENDLOCAL set "DEL=%%a" set "DEL=%DEL:~0,1% set "ESC=%%b" goto :EOF ) goto :eof
The prompt $H works the way it does because there is no guarantee there will be a subsequent character after the $H. The first <BS> moves the screen cursor back one position, but does not erase the character there. The <Space> "erases" the character but moves the cursor forward, and the final <BS> moves the cursor back again. ----- And now the revised code. As before, the test cases are at the top. There are two tests. First loop through all valid ASCII codes and convert to char and back to code using :chr and :asc. The end code should match the original code. The second test builds the ASCII map, converts the entire ASCII map to a hex string and then back to a map string using :asciiMap, :str2hex and :hex2str. The starting map should match the ending map. The tests are run both with and without the /X option. I won't include the output because it is so long, but it does give the expected results on my Vista machine. The /X option would be much faster if the ASCII map were preserved as a global variable, but I want each function to stand on its own without depending on global variables (in other words no side effects). WARNING - I believe the web site is corrupting the ASCII map in the code block. The 10th character after the = should be a TAB but I think it is getting converted into 4 spaces. Below is the actual line (hopefully intact) set %~1= !^"#$%%^&'^(^)*+,-./0123456789:;^<=^>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^^_`abcdefghijklmnopqrstuvwxyz{^|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ¡¢£¤¥¦§¨©ª«¬®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖרÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ Code: @echo off :top setlocal disableDelayedExpansion for /l %%n in (0,1,255) do ( call :chr %1 %%n char if not errorlevel 1 ( call :asc %1 char 0 n call echo "%%n:%%char%%:%%n%%" ) else echo chr %%n produced above error ) echo: call :asciiMap %1 map call :strLen map len set map echo len=%len% echo: call :str2hex %1 map hex set hex echo: call :hex2str %1 hex str set str echo: setlocal enableDelayedExpansion if "!map!"=="!str!" (echo map matches str) else (echo map does not match str) if "%1"=="" call :top /X exit /b
:str2hex [/X] StrVar [RtnVar] :: :: Converts the string contained within variable StrVar into a string of :: ASCII codes, with each code represented as a pair of hexadecimal digits. :: The length of the result will always be exactly twice the length of :: the original string. :: :: Sets RtnVar=result :: or displays result if RtnVar not specified :: :: If one of the following problematic characters appears within StrVar :: then the corresponding hex pair is set to 00 and errorlevel is set to 1. :: :: C H A R A C T E R S U P P O R T E D ? :: Dec Hex Oct Char Normal /X Option :: --- ---- ---- ---- ------ --------- :: 0 0x00 00 NUL (null) No No :: 10 0x0A 012 LF (line feed) No No :: 13 0x0D 015 CR (carriage return) No Yes :: 26 0x1A 032 SUB (substitute) No Yes :: :: If the case insensitive /X option is specified then CR (0x0D) and :: SUB (0x1A) are supported properly as input. :: :: Note: The /X option requires writing and reading a small temporary file. ::: ::: Dependencies - :asciiMap, :strLen, :Unique ::: setlocal disableDelayedExpansion if /i "%~1"=="/X" ( set option=%1 shift /1 ) else set option= call :asciiMap %option% asciiMap set "hexMap=0123456789ABCDEF" call :strLen %~1 len set /a len-=1 set rtn= set err= setlocal enableDelayedExpansion for /l %%n in (0,1,%len%) do ( set c=!%~1:~%%n,1! set d=0 for /l %%n in (0,1,255) do if "!asciiMap:~%%n,1!"=="!c!" set d=%%n set /a h1=d/16, h2=d%%16 for %%a in (!h1!) do for %%b in (!h2!) do set rtn=!rtn!!hexMap:~%%a,1!!hexMap:~%%b,1! ) ( endlocal endlocal if "%~2" neq "" (set %~2=%rtn%) else echo:%rtn% exit /b %err% ) exit /b
:hex2str [/X] HexVar RtnVar :: :: Converts a string of hexadecimal digits contained within variable HexVar :: into a string, where each pair of hex digits in the input represents the :: ASCII code of a character in the result. :: :: Stores the result in the variable RtnVar. :: :: If one of the following problematic characters is specified within the Hex :: string then a space is substituted and errorlevel is set to 1: :: :: C H A R A C T E R S U P P O R T E D ? :: Dec Hex Oct Char Normal /X Option :: --- ---- ---- ---- ------ --------- :: 0 0x00 00 NUL (null) No No :: 10 0x0A 012 LF (line feed) No No :: 13 0x0D 015 CR (carriage return) No Yes :: 26 0x1A 032 SUB (substitute) No Yes :: :: If the case insensitive /X option is specified then CR (0x0D) and :: SUB (0x1A) may appear in the output. If the result contains CR (0x0D) :: then RtnVar should only be accessed via delayed expansion. :: :: Note: The /X option requires writing and reading a small temporary file. :: :: If an invalid hexadecimal digit is detected within the Hex string then the :: corresponding result character is set to a space and errorlevel is set :: to 2. :: :: Aborts with an error message to stderr and errorlevel 3 if the Hex string :: length is not divisible by 2. ::: ::: Dependencies - :asciiMap, :strLen, :Unique ::: setlocal disableDelayedExpansion if /i "%~1"=="/X" ( set option=%1 shift /1 ) else set option= call :asciiMap %option% map call :strLen %1 len set /a mod=len%%2 if %mod%==1 1>&2 echo "ERROR: Hex string length not a multiple of 2" & exit /b 3 set rtn= set /a len-=1 setlocal enableDelayedExpansion set err=0 for /l %%n in (0,2,%len%) do ( 2>nul set /a d=0x!%~1:~%%n,2! || (set d=32&set err=2) for %%d in (!d!) do set c=^!map:~%%d,1! if "!c!"==" " if not !d!==32 if !err!==0 set err=1 set "rtn=!rtn!!c!" ) if defined option if "!rtn:~-1!"=="!map:~13,1!" set "rtn=!rtn!!map:~13,1!" for /f "delims=" %%s in ("!rtn!") do ( endlocal endlocal if "%~2" neq "" set %~2=%%s exit /b %err% ) exit /b
:asc [/X] StrVar IntVal [RtnVar] :: :: Computes the ASCII code for a specified character within the string :: contained by variable StrVar. The position within the string is specified :: by the IntVal argument. A non-negative value is relative to the beginning :: of the string, with 0 specifiying the first character. A negative value is :: relative to the end of the string, with -1 specifying the last character. :: :: Sets RtnVar=result :: or displays result if RtnVar not specified :: :: IntVal may be passed as a variable without enclosing the name in percent :: symbols. :: :: If one of the following problematic characters is specified then RtnVar :: will be undefined and errorlevel will be set to 1. :: :: C H A R A C T E R S U P P O R T E D ? :: Dec Hex Oct Char Normal /X Option :: --- ---- ---- ---- ------ --------- :: 0 0x00 00 NUL (null) No No :: 10 0x0A 012 LF (line feed) No No :: 13 0x0D 015 CR (carriage return) No Yes :: 26 0x1A 032 SUB (substitute) No Yes :: :: If the case insensitive /X option is specified then CR (0x0D) and :: SUB (0x1A) may successfully be specified as input. :: :: Note: The /X option requires writing and reading a small temporary file. :: :: If StrVar is not defined then aborts with an error message to stderr and :: errorlevel 2. :: :: If IntVal is greater than or equal to the length of the string then aborts :: with an error message to stderr and errorlevel 3. :: :: Negative IntVal values will never result in errorlevel 2: Positions earlier :: than the 1st character are treated as the 1st character. ::: ::: Dependencies - :asciiMap, :Unique ::: setlocal disableDelayedExpansion if /i "%~1"=="/X" ( set option=%1 shift /1 ) else set option= set /a n=%~2 2>nul if errorlevel 1 1>&2 echo "ERROR: Invalid numeric value"&exit /b 11 if not defined %~1 1>&2 echo "ERROR: Variable not defined"&exit /b 2 call :asciiMap %option% ascii setlocal enableDelayedExpansion set "chr=!%~1:~%n%,1!" if not defined chr 1>&2 echo "ERROR: String position not found"&exit /b 3 if "!chr!"==" " (set /a rtn=32) else ( if defined rtn set rtn= for /l %%n in (0,1,255) do if "!ascii:~%%n,1!"=="!chr!" set rtn=%%n ) if defined rtn (set err=0) else set err=1 (endlocal & rem -- return values endlocal if "%~3" neq "" (set %~3=%rtn%) else (echo:%rtn%) exit /b %err% ) exit /b
:chr [/X] IntVal [RtnVar] :: :: Converts ASCII code IntVal into the corresponding character. :: :: Sets RtnVar=result :: or displays result if RtnVar not specified :: :: IntVal must be a value between 0 and 255. :: :: Aborts with an error message to stderr and errorlevel 11 if IntVal is not :: a valid ASCII code. :: :: Aborts with an error message to stderr and errorlevel 1 if IntVal :: corresponds to one of the following problematic characters: :: :: C H A R A C T E R S U P P O R T E D ? :: Dec Hex Oct Char Normal /X Option :: --- ---- ---- ---- ------ --------- :: 0 0x00 00 NUL (null) No No :: 10 0x0A 012 LF (line feed) No No :: 13 0x0D 015 CR (carriage return) No Yes :: 26 0x1A 032 SUB (substitute) No Yes :: :: If the case insensitive /X option is specified then CR (0x0D) and :: SUB (0x1A) are supported. If the result is CR (0x0D) then RtnVar should :: only be accessed via delayed expansion. :: :: Note: The /X option requires writing and reading a small temporary file. :: :: IntVal may be passed as a variable without enclosing the name in percent :: symbols. ::: ::: Dependencies - :asciiMap, :Unique ::: setlocal disableDelayedExpansion if /i "%~1"=="/X" ( set option=%1 shift /1 ) else set option= set /a n=%~1 2>nul if errorlevel 1 1>&2 echo "ERROR: Invalid ASCII Code"&exit /b 11 if %n%==32 set "c= "&setlocal enableDelayedExpansion&goto :chr.end if %n% lss 0 1>&2 echo "ERROR: Invalid ASCII Code"&exit /b 11 if %n% gtr 255 1>&2 echo "ERROR: Invalid ASCII Code"&exit /b 11 call :asciiMap %option% ascii setlocal EnableDelayedExpansion set "c=!ascii:~%n%,1!" if "!c!"==" " ( 1>&2 echo "ERROR: Problematic ASCII Code"&exit /b 1 ) if %n%==13 set "c=!c!!c!" :chr.end for /f "delims=" %%c in ("!c!") do ( endlocal endlocal if "%~2" neq "" (set %~2=%%c) else (echo:%%c) ) exit /b
:asciiMap [/X] rtnVar :: :: Sets variable rtnVar to a 256 character string containing the complete :: extended ASCII character set except a space has been substituted for each :: of the following problematic characters: :: :: C H A R A C T E R S U P P O R T E D ? :: Dec Hex Oct Char Normal /X Option :: --- ---- ---- ---- ------ --------- :: 0 0x00 00 NUL (null) No No :: 10 0x0A 012 LF (line feed) No No :: 13 0x0D 015 CR (carriage return) No Yes :: 26 0x1A 032 SUB (substitute) No Yes :: :: If the case insensitive /X option is specified then CR (0x0D) and :: SUB (0x1A) characters are included in the map. However, a map with :: these characters can only be accessed via delayed expansion. :: :: Note: The /X option requires writing and reading a small temporary file. ::: ::: Dependencies - :Unique ::: setlocal disableDelayedExpansion if /i "%~1"=="/X" shift /1 & goto :asciiMap.extend (endlocal call :asciiMap.setMap %~1 exit /b ) :asciiMap.extend call :Unique file if defined temp (set filePath=%temp%) else if defined tmp (set filePath=%tmp%) else set filePath=. set file="%filePath%\_asciiMap_%file%_%random%.tmp" for /f %%a in ('copy /Z "%~dpf0" nul') do set "cr=%%a" copy /a nul+nul %file% > nul for /f "usebackq" %%a in (%file%) do set "sub=%%a" del %file% call :asciiMap.setMap map setlocal enableDelayedExpansion set "map=!map:~0,13!!cr!!map:~14,12!!sub!!map:~27!" for /f "delims=" %%a in ("!map!") do ( endlocal endlocal set %~1=%%a exit /b ) :asciiMap.setMap :: WARNING - THE 10th character after the = should be a single TAB character. I believe the web site is converting the TAB into 4 spaces set %~1= !^"#$%%^&'^(^)*+,-./0123456789:;^<=^>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^^_`abcdefghijklmnopqrstuvwxyz{^|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ¡¢£¤¥¦§¨©ª«¬®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖרÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ exit /b
::----------------------------------------------------------------------------- :: The following are existing dostips functions ::-----------------------------------------------------------------------------
:strLen string len -- returns the length of a string :: -- string [in] - variable name containing the string being measured for length :: -- len [out] - variable to be used to return the string length :: Many thanks to 'sowgtsoi', but also 'jeb' and 'amel27' dostips forum users helped making this short and efficient :$created 20081122 :$changed 20101116 :$categories StringOperation :$source http://www.dostips.com ( SETLOCAL ENABLEDELAYEDEXPANSION set "str=A!%~1!"&rem keep the A up front to ensure we get the length and not the upper bound rem it also avoids trouble in case of empty string set "len=0" for /L %%A in (12,-1,0) do ( set /a "len|=1<<%%A" for %%B in (!len!) do if "!str:~%%B,1!"=="" set /a "len&=~1<<%%A" ) ) ( ENDLOCAL & REM RETURN VALUES IF "%~2" NEQ "" SET /a %~2=%len% ) EXIT /b
:Unique ret -- returns a unique string based on a date-time-stamp, YYYYMMDDhhmmsscc :: -- ret [out,opt] - unique string :$created 20060101 :$changed 20080219 :$categories StringOperation,DateAndTime :$source http://www.dostips.com SETLOCAL for /f "skip=1 tokens=2-4 delims=(-)" %%a in ('"echo.|date"') do ( for /f "tokens=1-3 delims=/.- " %%A in ("%date:* =%") do ( set %%a=%%A&set %%b=%%B&set %%c=%%C)) set /a "yy=10000%yy% %%10000,mm=100%mm% %% 100,dd=100%dd% %% 100" for /f "tokens=1-4 delims=:. " %%A in ("%time: =0%") do @set UNIQUE=%yy%%mm%%dd%%%A%%B%%C%%D ENDLOCAL & IF "%~1" NEQ "" (SET %~1=%UNIQUE%) ELSE echo.%UNIQUE% EXIT /b
|
| 06 Apr 2011 23:38 |
|
 |
|
dbenham
Expert
Joined: 12 Feb 2011 21:02 Posts: 888 Location: United States (east coast)
|
 Re: new functions: :chr, :asc, :asciiMap
Correction - the code block is converting the TAB into 3 spaces. The TAB in the map outside the code block is correct. But the 161st character after the = is supposed to be ASCII decimal 160, but it has been corrupted into a space in both places.
Dave
|
| 06 Apr 2011 23:53 |
|
 |
|
jeb
Expert
Joined: 30 Aug 2007 08:05 Posts: 567 Location: Germany
|
 Re: new functions: :chr, :asc, :asciiMap
Hi dbenham, look nice now, but some comments... the NUL character seems to be impossible to handle with batch, as I suppose cmd.exe works internally with null-terminated strings. So you can't store a NUL in a variable. The character searching in :asc or in :str2hex could be enhanced by a map (similar to :strLen) Code: set "ch=a" set "ascMap=000#101#202#A41#B42" set "split=!ascMap:*%ch%=!" set /a "hexVal=0x!split:~0,2!"
You only have to care about case sensitive. The <LF> and exclams should work with a obvious escape sequence Code: @echo off setlocal EnableDelayedExpansion call :lfTest result echo the result is: echo !result!-- The end goto :eof
:lfTest setlocal set "NotDelayedFlag=!" setlocal EnableDelayedExpansion set LF=^
rem TWO Empty lines are neccessary
rem Test result set "var=one&line!LF!two with exclam^! !LF!three with "quotes^&""
rem ** Prepare for return if not defined NotDelayedFlag ( for %%a in ("!LF!") do set "var=!var:%%~a=""L!" set "var=!var:^=^^^^!" set "var=!var:"=""Q!" ) if not defined NotDelayedFlag ( set "var=%var:!=^^^^^!%" ! set "var=!var:""Q="!" for %%a in ("!LF!") do set "var=!var:""L=%%~a!" ) set ^"var=!var:"=^"!" set "var=!var:&=^&!" for %%a in ("!LF!") do set "var=!var:%%~a=^%%~a%%~a!" ( ENDLOCAL ENDLOCAL set ^"%~1=%var%" ! goto :eof )
jeb
|
| 07 Apr 2011 03:24 |
|
 |
|
dbenham
Expert
Joined: 12 Feb 2011 21:02 Posts: 888 Location: United States (east coast)
|
 Re: new functions: :chr, :asc, :asciiMap
Quote: The character searching in :asc or in :str2hex could be enhanced by a map (similar to :strLen) Code: set "ch=a" set "ascMap=000#101#202#A41#B42" set "split=!ascMap:*%ch%=!" set /a "hexVal=0x!split:~0,2!"
Great idea Jeb. I had hoped to use one map that would convert biderectionally, but I think speed improvements using your strategy are worth the added effort of maintaining two maps. I don't understand how your strategy is similar to :strLen. In my earlier post I was referring to a failed strategy outlined below that was similar to strLen (pseudo-code): Code: build my original map length 256 set rtn=0 For each 1 bit in a byte going from high to low ( XOR the bit with rtn if CHR < %map:~rtn,1% then remove the bit (XOR the bit with rtn again) )
Upon completion of the loop, rtn should be the ASCII code for CHR. The loop has only 8 iterations, so it would be very fast. But the algorithm FAILS because the characters do not sort in ASCII code sequence. ================= Regarding the 2nd part of your reply: Close, but I still see problems. I added support for ^ | < > ( ) that was missing in your original. But there are still problems if input contains CR. Also, there is a problem with calling while using delayed expansion if input contains ""L. Code: @echo off setlocal DisableDelayedExpansion call :lfTest result setlocal EnableDelayedExpansion echo The result is: echo !result! call :lfTest result echo The result is: echo !result!
goto :eof
:lfTest setlocal set "NotDelayedFlag=!" echo: if defined NotDelayedFlag (echo lfTest was called with Delayed Expansion DISABLED) else echo lfTest was called with Delayed Expansion ENABLED setlocal EnableDelayedExpansion for /f %%a in ('copy /Z "%~dpf0" nul') do set "CR=%%a" set LF=^
rem TWO Empty lines are neccessary
rem Test result set "var=one&line!LF!two with exclam^! !LF!three with "quotes^&"&"!LF!four with ^^^^ ^| ^< ^> ( ) ^& ^^^! ^"!LF!xxxxxwith CR!CR!five !LF!six with ^"^"Q ^"^"L still six " echo the input is: echo !var! echo:
rem ** Prepare for return set "var=!var:^=^^!" if defined NotDelayedFlag goto :NotDelayed for %%a in ("!LF!") do set "var=!var:%%~a=""L!" set "var=!var:^=^^^^!" set "var=!var:"=""Q!" set "var=%var:!=^^^^^!%" ! set "var=!var:""Q="!" for %%a in ("!LF!") do set "var=!var:""L=%%~a!" :NotDelayed set ^"var=!var:"=^"!" set "var=!var:&=^&!" set "var=!var:|=^|!" set "var=!var:<=^<!" set "var=!var:>=^>!" set "var=!var:(=^(!" set "var=!var:)=^)!" for %%a in ("!LF!") do set "var=!var:%%~a=^%%~a%%~a!" for %%a in ("!CR!") do set "var=!var:%%~a=^%%~a%%~a!" ( ENDLOCAL ENDLOCAL set ^"%~1=%var%" ! goto :eof )
Here is the output from above Code: lfTest was called with Delayed Expansion DISABLED the input is: one&line two with exclam! three with "quotes&"&" four with ^ | < > ( ) & ! " five with CR six with ""Q ""L still six
The result is: one&line two with exclam! three with "quotes&"&" four with ^ | < > ( ) & ! " xxxxxwith CRfive six with ""Q ""L still six
lfTest was called with Delayed Expansion ENABLED the input is: one&line two with exclam! three with "quotes&"&" four with ^ | < > ( ) & ! " five with CR six with ""Q ""L still six
The result is: one&line two with exclam! three with "quotes&"&" four with ^ | < > ( ) & ! " xxxxxwith CRfive six with ""Q still six
So it looks like the functions can support CR or LF, but not both.  Unless you have any other ideas? I had already given up on calling functions while delayed expansion is enabled. But it looks like you have discovered a "universal" technique that works as long as returned value does not contain CR or LF! Awsome! Dave
|
| 07 Apr 2011 15:04 |
|
 |
|
dbenham
Expert
Joined: 12 Feb 2011 21:02 Posts: 888 Location: United States (east coast)
|
 Re: new functions: :chr, :asc, :asciiMap
I have fixed two bugs in my last posted code- :chr and :hex2str both failed with ; (0x59) due to the default FOR /F "eol=;" option. Unfortunately eol= option cannot be completely disabled like "delims=", there is always an eol character. (The description of eol within "help for" or "for /?" is terrible!) Fixed :hex2str at end of function: Code: if "!rtn:~0,1!"==";" (set "eol=eol= ") else set "eol=" for /f "%eol%delims=" %%s in ("!rtn!") do ( endlocal endlocal if "%~2" neq "" set %~2=%%s exit /b %err% ) exit /b
Fixed :chr at end of function Code: if "!c!"==";" (set "eol=eol= ") else set "eol=" for /f "%eol%delims=" %%c in ("!c!") do ( endlocal endlocal if "%~2" neq "" (set %~2=%%c) else (echo:%%c) ) exit /b
The bug was masked, but not completely hidden, by a poor choice in variable naming in the test case. improved test case at top: Code: for /l %%n in (0,1,255) do ( call :chr %1 %%n char if not errorlevel 1 ( call :asc %1 char 0 rtn call echo "%%n:%%char%%:%%rtn%%" ) else echo chr %%n produced above error )
I'm still working on Jeb's idea for speeding up :str2hex and :asc. I'm close, but still some things to overcome. So far there is some improved performance, but I'm not sure yet if it's worth the effort. Dave
|
| 08 Apr 2011 06:56 |
|
 |
|
jeb
Expert
Joined: 30 Aug 2007 08:05 Posts: 567 Location: Germany
|
 Re: new functions: :chr, :asc, :asciiMap
dbenham wrote: Close, but I still see problems. I added support for ^ | < > ( ) that was missing in your original. But there are still problems if input contains CR. Also, there is a problem with calling while using delayed expansion if input contains ""L. Ok, I was a bit imprecise. The ""L problem can be solved by the correct order of the commands. And the support for | < > ( ) is equal to &. But I didn't look at this way anymore, as I think it comes to a dead end. dbenham wrote: So it looks like the functions can support CR or LF, but not both.  Unless you have any other ideas? I had already given up on calling functions while delayed expansion is enabled. But it looks like you have discovered a "universal" technique that works as long as returned value does not contain CR or LF! Awsome!  It was a bit tricky, but I created a slightly other solution. It can handle CR and LF and all the other characters And it's shorter and doesn't need to handle the & | < > characters. Code: @echo off setlocal EnableDelayedExpansion cls for /f %%a in ('copy /Z "%~dpf0" nul') do set "CR=%%a" set LF=^
rem TWO Empty lines are neccessary set "original=zero*? %%~A%%~B%%~C%%~L!LF!one&line!LF!two with exclam^! !LF!three with "quotes^&"&"!LF!four with ^^^^ ^| ^< ^> ( ) ^& ^^^! ^"!LF!xxxxxwith CR!CR!five !LF!six with ^"^"Q ^"^"L still six "
setlocal DisableDelayedExpansion call :lfTest result original
setlocal EnableDelayedExpansion echo The result with disabled delayed expansion is: if !original! == !result! (echo OK) ELSE echo !result!
call :lfTest result original echo The result with enabled delayed expansion is: if !original! == !result! (echo OK) ELSE echo !result! echo ------------------ echo !original!
goto :eof
:::::::::::::::::::: :lfTest setlocal set "NotDelayedFlag=!" echo( if defined NotDelayedFlag (echo lfTest was called with Delayed Expansion DISABLED) else echo lfTest was called with Delayed Expansion ENABLED setlocal EnableDelayedExpansion set "var=!%~2!"
rem echo the input is: rem echo !var! echo(
rem ** Prepare for return set "var=!var:%%=%%~A!" set "var=!var:"=%%~B!" for %%a in ("!LF!") do set "var=!var:%%~a=%%~L!" for %%a in ("!CR!") do set "var=!var:%%~a=%%~C!"
rem ** It is neccessary to use two IF's else the %var% expansion doesn't work as expected if not defined NotDelayedFlag set "var=!var:^=^^^^!" if not defined NotDelayedFlag set "var=%var:!=^^^!%" !
set "replace=%% """ !CR!!CR!" for %%L in ("!LF!") do ( for /F "tokens=1,2,3" %%A in ("!replace!") DO ( ENDLOCAL ENDLOCAL set "%~1=%var%" ! @echo off goto :eof ) )
It replaces % with %~A " with %~B <CR> with %~C <LF> with %~L And in the return statement it conterts them back to the original values. I need two FOR statements, as the FOR /F can't assign <LF> to a parameter, but a simple FOR %%a. The % substituition to %~A is neccessary, else there could be content with something like "100%C" which would expand the wrong way. And in the enabled delayed expansion mode, there are two more substituitions ^ with ^^ ! with ^! The caret doubling is neccessary, as in a line is one or more exclamation marks the ^ works a second time as an escape character, but now in and outside of quotes. To assure that the doubled carets will always reduced to a single caret, I append an exlamation mark after the last quote in the set statements. Hope this is the end ... jeb
|
| 10 Apr 2011 15:05 |
|
 |
|
aGerman
Expert
Joined: 22 Jan 2010 18:01 Posts: 1395 Location: Germany
|
 Re: new functions: :chr, :asc, :asciiMap
jeb wrote: To assure that the doubled carets will always reduced to a single caret, I append an exlamation mark after the last quote in the set statements.
That's not clear to me. How can an exclamation mark behind the last quote influence the variable content? Regards aGerman
|
| 10 Apr 2011 15:59 |
|
 |
|
jeb
Expert
Joined: 30 Aug 2007 08:05 Posts: 567 Location: Germany
|
 Re: new functions: :chr, :asc, :asciiMap
The simple rule for delayed expansion is. For each character in the line do: - If it is a caret (^) the next character has no special meaning, the caret itself is removed - If it is an exclamation mark, search for the next exclamation mark (carets are not observed here), then expands to the content of the variable - If no exclamation mark is found in this phase, the result is discarded, the result of the phase before is used instead (important for the carets) So, at this point the difference should be clear, the carets are removed even if the exclamation mark have no other effect in a line. A sample Code: @echo off setlocal EnableDelayedExpansion
echo one caret^^ echo none caret^^ !
set "var1=one caret^" set "var2=none caret^" !
echo !var1! echo !var2! ----- OUTPUT ---- one caret^ none caret one caret^ one caret
But these was only the simple rule. There is a bit more to know about it. - The line is split into two parts, the command-token and the rest of the line, for each part this phase is executed separatly Sample for the "two part" rule Code: setlocal EnableDelayedExpansion set "x=######" echo:^^^^"^^^^" ^^^^"^^^^" echo:^^^^"^^^^" ^^^^"^^^^"!x! echo:^^^^"^^^^"!x! ^^^^"^^^^" echo:^^^^"^^^^"!x! ^^^^"^^^^"!x! --- Output --- ^^"^^^^" ^^"^^^^" ^^"^^^^" ^"^^"###### ^"^^"###### ^^"^^^^" ^"^^"###### ^"^^"###### This uses a trick to append something to the first/commando token, but after the parsing phases, the echo splits and prints it. jeb
|
| 10 Apr 2011 16:19 |
|
 |
|
aGerman
Expert
Joined: 22 Jan 2010 18:01 Posts: 1395 Location: Germany
|
 Re: new functions: :chr, :asc, :asciiMap
Thanks for your explanation. I always appreciate your comments because there's so much stuff I never saw before.
Regards aGerman
|
| 10 Apr 2011 16:56 |
|
 |
|
dbenham
Expert
Joined: 12 Feb 2011 21:02 Posts: 888 Location: United States (east coast)
|
 Re: new functions: :chr, :asc, :asciiMap
Oh my goodness Jeb! Your solution is insanely clever. I'm able to follow the substitutions, but I'll have to spend more time looking at your explanation to fully understand the games you play with ! Thankfully I don't have to fully understand it to use it. I've finished incorporating your improved map lookup for asc and str2hex. It doesn't make much difference for :asc, but str2hex is now nearly 7 times faster! I solved the case issue by doubling each character after # and testing the 1st remaining character after substitution against the target character. If no match then I just perform one more substitution to get the correct case. I've eliminated the clunky /X option and now create the special characters only as needed. For :hex2str I added options to perform substitutions for NUL, CR, LF, and Errors. By default NUL is represented as <NUL>, LF as <LF>, invalid hex as <ERR>, and CR as itself. But the options allow setting each representation to any string you want. The only thing that was missing was support for a true LF in :hex2str. I was all ready to post what I had but now you've provided the last missing piece. I'll incorporate your solution and post the final result in the next few days. Dave
|
| 10 Apr 2011 20:58 |
|
 |
|
dbenham
Expert
Joined: 12 Feb 2011 21:02 Posts: 888 Location: United States (east coast)
|
 Re: new functions: :chr, :asc, :asciiMap
I think this project is now complete!  (with the exception of bugs which are bound to crop up  ). The library now supports 255 ASCII characters (only 00 NUL not supported). I've fully incorporated Jeb's new found technique allowing a function to return any string value, regardless whether the function was called with delayed expansion enabled or disabled. I've turned the library into a complete, self-contained, callable library, complete with built-in help functionality. There is no need to copy the routines into your own batch file - you can simply call the library directly. Of course there is nothing stopping you from copying the routines if that is your preference. To avoid corruption of the code that I experienced earlier, I've made the file available on a free Google web site. Download the CharLib_bat.txt file found here: https://sites.google.com/site/dbenhamfiles/home/CharLib_bat.txtand rename it to CharLib.bat. Ideally the file should be in a directory that is in your path. Run CharLib without any options to get basic syntax and other general info. CharLib help will provide a list of available functions. CharLib test will run a test suite that exercises most of the functionality. One more time I have to thank Jeb for all his work and advice that enabled me to make these routines fully functional. I hope others find this library useful. Please post a bug report here if you find problems. However I am not interested in adding additional functionality (except for maybe some form of versioning info in the case of bug fix updates) Dave Benham
|
| 13 Apr 2011 22:03 |
|
 |
|
plp626
Joined: 17 Apr 2009 00:36 Posts: 5 Location: China
|
 Re: new functions: :chr, :asc, :asciiMap
|
| 19 May 2011 07:50 |
|
 |
|
dbenham
Expert
Joined: 12 Feb 2011 21:02 Posts: 888 Location: United States (east coast)
|
 Re: new functions: :chr, :asc, :asciiMap
Hi plp626. I'm assuming the functions failed miserably for you. There is an interesting discussion at "universal" %DATE% parser concerning multi-national byte representation of DOS strings. I think I have a handle on how to work with single byte character sets. But I have no idea if functions like :asc, :chr, :parseDate etc. can be made to work with multibyte character sets like GBK. I would like to see if the functions can be made to work for you, but I will need your help. I don't have a machine on which I can test GBK. One quick question - how does DOS substring work with Chinese characters like your test string? In other words, if variable str contains your string, would %str:~0,1% return the 1st Chinese character consisting of two bytes? Dave Benham
|
| 21 May 2011 15:50 |
|
|
Who is online |
Users browsing this forum: Aacini, Bing [Bot], Ed Dyreen, Endoro, Google [Bot] and 21 guests |
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum
|
|
 |