Macro $RTrim (trim trailing spaces) / also $LTrim and $Trim

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Post Reply
Message
Author
aGerman
Expert
Posts: 4654
Joined: 22 Jan 2010 18:01
Location: Germany

Macro $RTrim (trim trailing spaces) / also $LTrim and $Trim

#1 Post by aGerman » 25 Dec 2011 19:42

EDIT: I was starting with that RTrim macro. dbenham helped to debug it and he changed the code to define various characters for trimming. He also developed macros for left-trimming ($LTRIM) and trimming both sides of the string ($TRIM). See http://www.dostips.com/forum/viewtopic.php?f=3&t=2697&p=12327#p12327

Hi folks,

alan_b asked for a way to trim trailing spaces. See http://www.dostips.com/forum/viewtopic.php?f=3&t=2694
I wrote a basic macro that I try to complete. That's the current status:

Code: Select all

@echo off

:: Predefine variables and the macro $RTrim
setlocal DisableDelayedExpansion
  :: LineFeed
set LF=^


set ^"\n=^^^%LF%%LF%^%LF%%LF%^^"&rem TWO EMPTY LINES ABOVE REQUIRED!
  :: create variable containing 4096 spaces
setlocal EnableDelayedExpansion
set "spcs=                "
for /l %%i in (1 1 8) do (
  set "spcs=!spcs!!spcs!"
)
endlocal &set "spcs=%spcs%"
  :: macro for right-trim
set $RTrim=for /l %%I in (1 1 2) do if %%I==2 (%\n%
  for /f %%j in ("!_arg!") do set "_str=!%%j!"%\n%
  set /a "k=4096"%\n%
  set "spc=%spcs%"%\n%
  for /l %%j in (1 1 13) do (%\n%
    for %%k in (!k!) do (%\n%
      if "!_str:~-%%k!"=="!spc!" (%\n%
        set "_str=!_str:~0,-%%k!"%\n%
      )%\n%
    )%\n%
    set /a "k/=2"%\n%
    for %%k in (!k!) do set "spc=!spc:~%%k!"%\n%
  )%\n%
  ECHO "!_str!"%\n%
  for /f "delims=*" %%j in ("!_arg!") do for /f "delims=" %%k in ("!_str!") do endlocal ^&set "%%j=%%k"%\n%
) else setlocal EnableDelayedExpansion ^&set _arg=

:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: Test
set "file=test.txt"
setlocal EnableDelayedExpansion
<"%file%" (
  for /f %%n in ('type "%file%"^|find /c /v ""') do (
    for /L %%i in (1 1 %%n) do (
      set "LN=" &set /p "LN="
      echo "!LN!"&rem BEFORE
      %$RTrim% LN
      echo "!LN!"&rem AFTER
      echo(
    )
  )
)
endlocal
pause

Output:

Code: Select all

"qwert                        "
"qwert"
"qwert"

"qwert !                        "
"qwert !"
"qwert "

"qwert ^!                      "
"qwert ^!"
"qwert !"

Each first line of the blocks above (but whithout quotation marks) is one line in test.txt. The second lines of each block are outputted from inside of the macro before I try to pass the endlocal command. The third lines are the results after endlocal.

As you can see the problem is that I try to transfer the variable into a "DelayedExpansion" environment. For that reason the exclamation marks and carets are stripped.
Is there a way to avoid that?
That's the relevant line of the macro:

Code: Select all

  for /f "delims=*" %%j in ("!_arg!") do for /f "delims=" %%k in ("!_str!") do endlocal ^&set "%%j=%%k"%\n%

Regards
aGerman

aGerman
Expert
Posts: 4654
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Macro RTrim (trim trailing spaces)

#2 Post by aGerman » 26 Dec 2011 13:27

Well, I found a possible way ...

Code: Select all

  :: macro for right-trim
set $RTrim=for /l %%I in (1 1 2) do if %%I==2 (%\n%
  for /f %%j in ("!_arg!") do set "_str=!%%j!"%\n%
  set /a "k=4096"%\n%
  set "spc=%spcs%"%\n%
  for /l %%j in (1 1 13) do (%\n%
    for %%k in (!k!) do (%\n%
      if "!_str:~-%%k!"=="!spc!" (%\n%
        set "_str=!_str:~0,-%%k!"%\n%
      )%\n%
    )%\n%
    set /a "k/=2"%\n%
    for %%k in (!k!) do set "spc=!spc:~%%k!"%\n%
  )%\n%
  echo "!_str!"%\n%
  if !_flag!==1 (%\n%
    for /f "delims=*" %%j in ("!_arg!") do for /f "delims=" %%k in ("!_str!") do endlocal ^&set "%%j=%%k"%\n%
  ) else set "!_arg!=!_str!"%\n%
) else set "_flag=0" ^&(if "0" neq "!_flag!" set "_flag=1" ^&setlocal EnableDelayedExpansion) ^&set _arg=

... but I'm not happy with that solution. If it is executed in a "DelayedExpansion" environment the macro does not create it's own sub environment. That means all of the variables inside of the macro are still valid after it was processed and (even worse) if one of the variables existed before then its value is changed now :(

Regards
aGerman

dbenham
Expert
Posts: 2461
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Macro RTrim (trim trailing spaces)

#3 Post by dbenham » 26 Dec 2011 21:37

Hi aGerman

I have various versions of safe return for macros using the old macro style at Batch "macros" with arguments - Major Update The safe return macros are found in file macroLib_Return.bat. They use techniques I learned from jeb in the new functions: :chr, :asc, :asciiMap thread.

I adapted techniques from my macro.AnyRtn1 to your $RTRIM macro with the new macro style. It does not support <CR> or <LF> in the returned string. It could be added, but the only way I know dynamically builds a temporary macro that must be called after the main macro completes. And the temp macro cannot be called within the same code block as the main macro. Also, the concept of trimming spaces at the end of a string that includes <LF> seems weird to me.

Besides adding support for a safe return, I also fixed a couple bugs and optimized a bit

- You needed extra code to deal with potential empty _str throughout the life-cycle of the macro.

- I don't see the need to parse _arg twice, so I put the entire macro body within one %%J for loop.

- I eliminated the need for the spc variable which enabled me to eliminate one of the %%k for loops.

- The pesky EOL option during return was a problem. I set EOL=<LF> with the awful looking syntax that jeb figured out.

Code: Select all

set $RTrim=for /l %%I in (1 1 2) do if %%I==2 (%\n%
  for /f %%J in ("!_arg!") do (%\n%
    set "_str=!%%J!"%\n%
    set /a "k=4096"%\n%
    for /l %%j in (1 1 13) do (%\n%
      if defined _str for %%k in (!k!) do (%\n%
        if "!_str:~-%%k!"=="!spcs:~-%%k!" set "_str=!_str:~0,-%%k!"%\n%
        set /a "k/=2"%\n%
      )%\n%
    )%\n%
    ECHO "!_str!"%\n%
    if defined _str (%\n%
      if not defined _notDelayed (%\n%
        set "_str=!_str:^=^^!"%\n%
        set "_str=!_str:"=""Q!^"%\n%
        call set "_str=%%^_str:^!=""E^!%%" ! %\n%
        set "_str=!_str:""E=^!"%\n%
        set "_str=!_str:""Q="!^"%\n%
      )%\n%
      for /f ^^^"eol^^^=^^^

^

^^^ delims^^^=^^^" %%k in ("!_str!") do endlocal^&endlocal^&set "%%J=%%k"!%\n%
    ) else endlocal^&endlocal^&set "%%J="%\n%
  )%\n%
) else setlocal^&set "_notDelayed=!"^&setlocal EnableDelayedExpansion^&set _arg=


Dave Benham

aGerman
Expert
Posts: 4654
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Macro RTrim (trim trailing spaces)

#4 Post by aGerman » 27 Dec 2011 04:09

Hi Dave,

many thanks for your support!
I hoped there was a way without all these replacements of special characters. Especially the call set is bugging me because it slows down tremendously.

dbenham wrote:- The pesky EOL option during return was a problem. I set EOL=<LF> with the awful looking syntax that jeb figured out.

I already fixed that yesterday (besides some other fixes), but I used the LF variable instead. That makes it more handsome.

I'm going to do some tests. I come back later with the results and probably a (tentatively) final version.

Thanks again
aGerman

dbenham
Expert
Posts: 2461
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Macro RTrim (trim trailing spaces)

#5 Post by dbenham » 27 Dec 2011 08:04

aGerman wrote:Especially the call set is bugging me because it slows down tremendously.
Amen to that. Unfortunately I can't think of any other good solution within a single macro. The ! must be escaped. Search and replace involving ! does not work with delayed expansion. Immediate expansion within a block of code cannot be used without a CALL if the variable was defined within the code block. The macro is a block of code and the variable is defined within it. I'm stuck.

The only way forward that I can see is to use something other than expansion search and replace to introduce the escape. The only pure batch way to do that is character by character search and replace - that seems much worse than CALL. That leaves a non-batch solution like VB Script, java script, or a 3rd party tool - I don't think you are interested in going down that road.

You could avoid the CALL if you split the single macro into a pair of macros that would have to be called one after the other. But then it could never be called within a block of code, and that would be extremely restrictive. (That is the problem I faced with my RtnAny1 and RtnAnyN macros.)

I still think your RTRIM macro is a major step forward, even with the CALL. The algorithm could be implemented as a "traditional" function, and then CALL is not needed to escape the !. But I think the macro form would still outperform the function form.


Dave Benham

aGerman
Expert
Posts: 4654
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Macro RTrim (trim trailing spaces)

#6 Post by aGerman » 27 Dec 2011 10:04

Thanks Dave.

dbenham wrote:Unfortunately I can't think of any other good solution within a single macro.

Neither do I. For that reason I assume your solution should be the best.

dbenham wrote:The only way forward that I can see is to use something other than expansion search and replace to introduce the escape. The only pure batch way to do that is character by character search and replace - that seems much worse than CALL. That leaves a non-batch solution like VB Script, java script, or a 3rd party tool - I don't think you are interested in going down that road.

I'm quite familiar with VBScript and JScript. I already suggested alan_b that I could write a script that would replace Lf by CrLf and trim trailing spaces at once. Writing a macro was more or less out of curiosity.

Well, finally I decided to do only 2 minor changes on your version.
- I inserted the creation of the spcs variable into the macro. Hence the predefined variables are reduced to what is needed by default. (And 2 milli seconds more ore less should not make any difference.)
- As I told before I used the LF variable to define the EOL option.

Code: Select all

@echo off

:: Predefine variables and the macro $RTrim
setlocal DisableDelayedExpansion
  :: LineFeed
set LF=^


set ^"\n=^^^%LF%%LF%^%LF%%LF%^^"&rem TWO EMPTY LINES ABOVE REQUIRED!

  :: macro for right-trimming
set $RTrim=for /l %%I in (1 1 2) do if %%I==2 (%\n%
  set "_spcs=                                                                "%\n%
  for /l %%i in (1 1 6) do set "_spcs=!_spcs!!_spcs!"%\n%
  for /f %%J in ("!_arg!") do (%\n%
    set "_str=!%%J!"%\n%
    set /a "k=4096"%\n%
    for /l %%j in (1 1 13) do (%\n%
      if defined _str for %%k in (!k!) do (%\n%
        if "!_str:~-%%k!"=="!_spcs:~-%%k!" set "_str=!_str:~0,-%%k!"%\n%
        set /a "k/=2"%\n%
      )%\n%
    )%\n%
    if defined _str (%\n%
      if not defined _notDelayed (%\n%
        set "_str=!_str:^=^^!"%\n%
        set "_str=!_str:"=""Q!^"%\n%
        call set "_str=%%^_str:^!=""E^!%%" ! %\n%
        set "_str=!_str:""E=^!"%\n%
        set "_str=!_str:""Q="!^"%\n%
      )%\n%
      for /f ^^^"eol^^=^^^%LF%%LF%^%LF%%LF%^^ delims^^=^^^" %%k in ("!_str!") do endlocal^&endlocal^&set "%%J=%%k"!%\n%
    ) else endlocal^&endlocal^&set "%%J="%\n%
  )%\n%
) else setlocal^&set "_notDelayed=!"^&setlocal EnableDelayedExpansion^&set _arg=

:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: Test
set "LN=;"%%* ^^^^ ^^! !)                         "

setlocal DisableDelayedExpansion
  echo Disabled Delayed Expansion
  echo BEFORE "%LN%"
  %$RTrim% LN
  echo AFTER  "%LN%"
endlocal
echo(

setlocal EnableDelayedExpansion
  echo Enabled Delayed Expansion
  echo BEFORE "!LN!"
  %$RTrim% LN
  echo AFTER  "!LN!"
endlocal
pause

Regards
aGerman

dbenham
Expert
Posts: 2461
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Macro RTrim (trim trailing spaces)

#7 Post by dbenham » 27 Dec 2011 14:51

Looks great :)

I thought it might be useful to have a version that allows you to specify the character that should be trimmed. So I modified your $RTRIM macro to accept an optional 2nd argument. The 2nd argument should specify the name of a variable that contains the character to trim in the 1st position. If the 2nd argument is missing then it defaults to trimming spaces as before.

Then I thought it would be useful to have an $LTRIM (and $TRIM) as well. The FOR /f "EOL=%char% DELIMS=%char%" solution works really well in an LTRIM function. But in a macro we can't use immediate expansion and you can't specify the DELIMS and EOL options using delayed expansion. So I can't use the FOR solution if I want to support a user defined trim character.

So here are versions of $RTRIM, $LTRIM and $TRIM that support a user defined trim character, all derived from your $RTRIM.

Code: Select all

::%$RTRIM% strVar [charVar] -- macro for right-trimming
set $RTrim=for /l %%I in (1 1 2) do if %%I==2 (%\n%
  set "_spcs= "%\n%
  for /f "tokens=1,2" %%J in ("!_arg!") do (%\n%
    set "_str=!%%J!"%\n%
    if "%%~K" neq "" if defined %%~K set "_spcs=!%%K:~0,1!"%\n%
    for /l %%i in (1 1 12) do set "_spcs=!_spcs!!_spcs!"%\n%
    set /a "k=4096"%\n%
    for /l %%j in (1 1 13) do (%\n%
      if defined _str for %%k in (!k!) do (%\n%
        if "!_str:~-%%k!"=="!_spcs:~-%%k!" set "_str=!_str:~0,-%%k!"%\n%
        set /a "k/=2"%\n%
      )%\n%
    )%\n%
    if defined _str (%\n%
      if not defined _notDelayed (%\n%
        set "_str=!_str:^=^^!"%\n%
        set "_str=!_str:"=""Q!^"%\n%
        call set "_str=%%^_str:^!=""E^!%%" ! %\n%
        set "_str=!_str:""E=^!"%\n%
        set "_str=!_str:""Q="!^"%\n%
      )%\n%
      for /f ^^^"eol^^=^^^%LF%%LF%^%LF%%LF%^^ delims^^=^^^" %%k in ("!_str!") do endlocal^&endlocal^&set "%%J=%%k"!%\n%
    ) else endlocal^&endlocal^&set "%%J="%\n%
  )%\n%
) else setlocal^&set "_notDelayed=!"^&setlocal EnableDelayedExpansion^&set _arg=

::%$LTRIM% strVar [charVar] -- macro for left-trimming
set $LTrim=for /l %%I in (1 1 2) do if %%I==2 (%\n%
  set "_spcs= "%\n%
  for /f "tokens=1,2" %%J in ("!_arg!") do (%\n%
    set "_str=!%%J!"%\n%
    if "%%~K" neq "" if defined %%~K set "_spcs=!%%K:~0,1!"%\n%
    for /l %%i in (1 1 12) do set "_spcs=!_spcs!!_spcs!"%\n%
    set /a "k=4096"%\n%
    for /l %%j in (1 1 13) do (%\n%
      if defined _str for %%k in (!k!) do (%\n%
        if "!_str:~0,%%k!"=="!_spcs:~-%%k!" set "_str=!_str:~%%k!"%\n%
        set /a "k/=2"%\n%
      )%\n%
    )%\n%
    if defined _str (%\n%
      if not defined _notDelayed (%\n%
        set "_str=!_str:^=^^!"%\n%
        set "_str=!_str:"=""Q!^"%\n%
        call set "_str=%%^_str:^!=""E^!%%" ! %\n%
        set "_str=!_str:""E=^!"%\n%
        set "_str=!_str:""Q="!^"%\n%
      )%\n%
      for /f ^^^"eol^^=^^^%LF%%LF%^%LF%%LF%^^ delims^^=^^^" %%k in ("!_str!") do endlocal^&endlocal^&set "%%J=%%k"!%\n%
    ) else endlocal^&endlocal^&set "%%J="%\n%
  )%\n%
) else setlocal^&set "_notDelayed=!"^&setlocal EnableDelayedExpansion^&set _arg=

::%$TRIM% strVar [charVar] -- macro for trimming both left and right
set $Trim=for /l %%I in (1 1 2) do if %%I==2 (%\n%
  set "_spcs= "%\n%
  for /f "tokens=1,2" %%J in ("!_arg!") do (%\n%
    set "_str=!%%J!"%\n%
    if "%%~K" neq "" if defined %%~K set "_spcs=!%%K:~0,1!"%\n%
    for /l %%i in (1 1 12) do set "_spcs=!_spcs!!_spcs!"%\n%
    set /a "k=4096"%\n%
    for /l %%j in (1 1 13) do (%\n%
      if defined _str for %%k in (!k!) do (%\n%
        if "!_str:~-%%k!"=="!_spcs:~-%%k!" set "_str=!_str:~0,-%%k!"%\n%
        if "!_str:~0,%%k!"=="!_spcs:~-%%k!" set "_str=!_str:~%%k!"%\n%
        set /a "k/=2"%\n%
      )%\n%
    )%\n%
    if defined _str (%\n%
      if not defined _notDelayed (%\n%
        set "_str=!_str:^=^^!"%\n%
        set "_str=!_str:"=""Q!^"%\n%
        call set "_str=%%^_str:^!=""E^!%%" ! %\n%
        set "_str=!_str:""E=^!"%\n%
        set "_str=!_str:""Q="!^"%\n%
      )%\n%
      for /f ^^^"eol^^=^^^%LF%%LF%^%LF%%LF%^^ delims^^=^^^" %%k in ("!_str!") do endlocal^&endlocal^&set "%%J=%%k"!%\n%
    ) else endlocal^&endlocal^&set "%%J="%\n%
  )%\n%
) else setlocal^&set "_notDelayed=!"^&setlocal EnableDelayedExpansion^&set _arg=


Dave Benham

aGerman
Expert
Posts: 4654
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Macro RTrim (trim trailing spaces)

#8 Post by aGerman » 27 Dec 2011 16:18

Great idea, Dave :!:

What do you think which character was my first attempt? :wink:

Regards
aGerman

Ed Dyreen
Expert
Posts: 1569
Joined: 16 May 2011 08:21
Location: Flanders(Belgium)
Contact:

Re: Macro $RTrim (trim trailing spaces) / also $LTrim and $T

#9 Post by Ed Dyreen » 27 Jan 2012 03:06

'
The algorithm is incredibly clever :P

Code: Select all

if "!_str:~-%%k!"=="!_spcs:~-%%k!" set "_str=!_str:~0,-%%k!"
a++German

@ben, very nice, only thing I change is jeb's faster/smaller suggestion.

Code: Select all

set $Trim=for %%I in (1,2) do if %%I==2 (%\n%
...
 ) else ...

Ed Dyreen
Expert
Posts: 1569
Joined: 16 May 2011 08:21
Location: Flanders(Belgium)
Contact:

Re: Macro $RTrim (trim trailing spaces) / also $LTrim and $T

#10 Post by Ed Dyreen » 27 Jan 2012 04:46

'
I require a little different functionality most of the time, the one that eats until Left/Right/Outer.
Excuse my foolishness, didn't realize you guys were after something else :oops:
Left

Code: Select all

%=      =%set $=^&set "$=^!%%~a^!"^&^&if 1%%~b neq +1%%~b (set "$=^!$:*%%~b=^!") else set "$=^!$:~0,-%%~b^!"%$n1c%
%=      =%set "%%~a=^!$^!"%$n1c%
Right

Code: Select all

%=      =%if "%%~b"=="" (set "$f= ") else set "$f=%%~b"%$n1c%
%=      =%set $$=^&set "$$=^!%%~a^!"^&^&!forQ_! ("^!$f^!") do if 1%%~? neq +1%%~? (%$n1c%
%=         =%set "?=^!$$:%%~?=" "^!"^&set ?="^!?^!"%$n1c%
%=         =%!forQ_! (^^^!?^^^!) do if "%%~?" neq "" set "$=%%~?"%$n1c%
%=         =%!forQ_! (^^^!$^^^!) do set "?=^!$$:*%%~?=^!###"%$n1c%
%=         =%if "^!?^!"=="###" set "?=%%~?^!$^!###"%$n1c%
%=         =%set "$=^!$$^!###"^&!forQ_! ("^!?^!") do set "$$=^!$:%%~?=^!"%$n1c%
%=      =%) else set "$$=^!$$:~0,-%%~?^!"%$n1c%
%=      =%set "%%~a=^!$$^!"%$n1c%

Aacini
Expert
Posts: 1886
Joined: 06 Dec 2011 22:15
Location: México City, México
Contact:

Re: Macro $RTrim (trim trailing spaces) / also $LTrim and $T

#11 Post by Aacini » 01 Apr 2012 19:19

Ed Dyreen wrote:'
@ben, very nice, only thing I change is jeb's faster/smaller suggestion.

Code: Select all

set $Trim=for %%I in (1,2) do if %%I==2 (%\n%
...
 ) else ...

Excuse me Ed, where did you see that jeb's suggestion?

I thought that this suggestion was mine! :wink: (near the end of this post):

Aacini wrote:Note: for %%n in (1 2) execute faster than for /L %%n in (1,1,2).

Post Reply