The "ultimate" file search and replace batch utility

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Post Reply
Message
Author
dbenham
Expert
Posts: 2461
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

The "ultimate" file search and replace batch utility

#1 Post by dbenham » 28 Dec 2011 15:01

There are a number of file search and replace routines written, but all that I have seen are case insensitive and also have limitations on the search string (no =, can't start with * or ~). Also neither string can contain % or !, depending on whether delayed expansion is used or not.

Ideally a non batch solution should be used. My favorite is the sed command found in the GNU Utilities for Win32.
2013-04-07 EDIT: My new favorite utility is a hybrid JScript/batch utility that I wrote that does not require downloading a non-native executable: JREPL.BAT

But sometimes a pure batch solution is still desired.

Here is a batch script that can do a case sensitive search, and there are no restrictions on the contents of the search or replacement string. It uses a brute force character by character search, but the performance is not bad as long as the number of replacements is relatively small. The routine only does the character by character search on lines that are known to contain the search string.

I learned something new while developing this: FIND search strings may include double quotes (") if they are escaped as "". :!: :D If only the rest of Windows batch used that standard, but alas, no. :(

Full documentation and limitations are listed at the top of the file in comments. Please let me know if you find any bugs. (I've done some testing, but I wouldn't call it rigorous testing)

2012-09-11 Edit - Removed useless computation of replacement length :oops:

Code: Select all

@echo off
:modFile File SearchVar [ReplaceVar] [/I]
::
::  Perform a search and replace operation on each line within File.
::
::  SearchVar = A variable containing the search string.
::
::  ReplaceVar = A variable containing the replacement string.
::               If ReplaceVar is missing or is not defined then the
::               search string is replaced with an empty string.
::
::  The /I option specifies a case insensitive search.
::
::  A backup of the original File is made with an extension of .bak
::  prior to making any changes.
::
::  The number of replacements made is returned as errorlevel.
::
::  If an error occurs then no changes are made and
::  the errorlevel is set to -1.
::
::  Limitations
::    - File must use Windows style line terminators <CR><LF>.
::    - Trailing control characters will be stripped from each line.
::    - The maximum input line length is 1021 characters.
::
setlocal enableDelayedExpansion

  ::error checking
  if "%~2"=="" (
    >&2 echo ERROR: Insufficient arguments
    exit /b -1
  )
  if not exist "%~1" (
    >&2 echo ERROR: Input file "%~1" does not exist
    exit /b -1
  )
  2>nul pushd "%~1" && (
    popd
    >&2 echo ERROR: Input file "%~1" does not exist
    exit /b -1
  )
  if not defined %~2 (
    >&2 echo ERROR: searchVar %2 not defined
    exit /b -1
  )
  if /i "%~3"=="/I" (
    >&2 echo ERROR: /I option can only be specified as 4th argument
    exit /b -1
  )
  if "%~4" neq "" if /i "%~4" neq "/I" (
    >&2 echo ERROR: Invalid option %4
    exit /b -1
  )

  ::get search and replace strings
  set "_search=!%~2!"
  set "_replace=!%~3!"

  ::build list of lines that must be changed, simply exit if none
  set "replaceCnt=0"
  set changes="%temp%\modFileChanges%random%.tmp"
  <"%~1" find /n %~4 "!_search:"=""!^" >%changes% || goto :cleanup

  ::compute length of _search
  set "str=A!_search!"
  set searchLen=0
  for /l %%A in (12,-1,0) do (
    set /a "searchLen|=1<<%%A"
    for %%B in (!searchLen!) do if "!str:~%%B,1!"=="" set /a "searchLen&=~1<<%%A"
  )

  ::count number of lines + 1
  for /f %%N in ('find /v /c "" ^<"%~1"') do set /a lnCnt=%%N+1

  ::backup source file
  if exist "%~1.bak" del "%~1.bak"
  ren "%~1" "%~nx1.bak"

  ::initialize
  set "skip=2"

  <"%~1.bak" (

    %=for each line that needs changing=%
    for %%l in (!searchLen!) do for /f "usebackq delims=[]" %%L in (%changes%) do (

      %=read and write preceding lines that don't need changing=%
      for /l %%N in (!skip! 1 %%L) do (
        set "ln="
        set /p "ln="
        if defined ln if "!ln:~1021!" neq "" goto :lineLengthError
        echo(!ln!
      )

      %=read the line that needs changing=%
      set /p "ln="
      if defined ln if "!ln:~1021!" neq "" goto :lineLengthError

      %=compute length of line=%
      set "str=A!ln!"
      set lnLen=0
      for /l %%A in (12,-1,0) do (
        set /a "lnLen|=1<<%%A"
        for %%B in (!lnLen!) do if "!str:~%%B,1!"=="" set /a "lnLen&=~1<<%%A"
      )

      %=perform search and replace on line=%
      set "modLn="
      set /a "end=lnLen-searchLen, beg=0"
      for /l %%o in (0 1 !end!) do (
        if %%o geq !beg! if %~4 "!ln:~%%o,%%l!"=="!_search!" (
          set /a "len=%%o-beg"
          for /f "tokens=1,2" %%a in ("!beg! !len!") do set "modLn=!modLn!!ln:~%%a,%%b!!_replace!"
          set /a "beg=%%o+searchLen, replaceCnt+=1"
        )
      )
      for %%a in (!beg!) do set "modLn=!modLn!!ln:~%%a!"

      %=write the modified line=%
      echo(!modLn!

      %=prepare for next iteration=%
      set /a skip=%%L+2
    )

    %=read and write remaining lines that don't need changing=%
    for /l %%N in (!skip! 1 !lnCnt!) do (
      set "ln="
      set /p "ln="
      if defined ln if "!ln:~1021!" neq "" goto :lineLengthError
      echo(!ln!
    )

  ) >"%~1"

  :cleanup
  del %changes%
exit /b %replaceCnt%

:lineLengthError
  del %changes%
  del "%~1"
  ren "%~nx1.bak" "%~1"
  >&2 echo ERROR: Maximum input line length exceeded. Changes aborted.
exit /b -1


Dave Benham
Last edited by dbenham on 24 Jan 2015 18:01, edited 6 times in total.

aGerman
Expert
Posts: 4654
Joined: 22 Jan 2010 18:01
Location: Germany

Re: The "ultimate" file search and replace batch utility

#2 Post by aGerman » 28 Dec 2011 15:51

:shock:
Double+ from my side :wink:

Minor suggestion - You could always write to a temporary file and use MOVE to overwrite the origin file later.
Pros: the origin file keeps unchanged in case the process would interrupt (for whatever reason) and you don't have to delete this temporary file if the process was successful.

Regards
aGerman

Post Reply