string find and replace batch file

[mcranmer]

I am wanting to build a batch file which finds and replaces a given string in over a hundred text files.

I have plucked this code from the web but all it does it open the text file in the BATCHTEST folder. This BATCHTEST folder contains a test text file which contains only the word BOB. The code is supposed to change all BOBs the FLUTEs. Once I get this working I will try to implement a more complex and real-world string replacement solution.

sorry if this is very basic.

thanks,

Mark

# Script txtedit.txt
# .txt files are in folder V:\Mark\VBA_Excel\BATCHTEST. Collect a list of them.
var str list ; lf -n "*.txt" "V:\Mark\VBA_Excel\BATCHTEST" > $list
# Process files one by one.
while ( $list <> "")
do
# Get the next file.
var str file ; lex "1" $list > $file
# Read file contents into a string variable.
var str content ; cat $file > $content
# Keep replacing "BOB" with "FLUTE" until there are
# no more instances left of "BOB".
while ( { sen -c "^BOB^" $content } > 0 )
sal "^BOB^" "FLUTE" $content > null
# All instances are replaced. Write file back.
echo $content > { echo $file }
done

[mcranmer]

Just found this link in the DosTips examples, and will take a look:


http://www.dostips.com/DtCodeBatchFiles ... AndReplace

[foxidrive]

This should handle *.txt in the current folder. Add /s to the dir command to process all folders in the tree. Test it on sample files first.

Use this batch file to call the batch file below:
(untested:)

@echo off
for /f "delims=" %%a in ('dir *.txt /b /a-d') do (
call SAR.BAT "%%a" "%%a.tmp" "BOB" "FLUTE"
move /y "%%a.tmp" "%%a" >nul
)



SAR.BAT

Code: Select all

@echo off
if "%~3"=="" (
echo.Search and replace
echo Syntax: 
echo %0 "filein.txt" "fileout.ext" "regex" "replace_text" [first]
echo.
echo. if [first] is present only the first occurrence is changed
goto :EOF
)
if "%~5"=="" (set global=true) else (set global=false)
set s=regex.replace(wscript.stdin.readall,"%~4")
 >_.vbs echo set regex=new regexp
>>_.vbs echo regex.global=%global%
>>_.vbs echo regEx.IgnoreCase=True            
>>_.vbs echo regex.pattern="%~3"
>>_.vbs echo wscript.stdOut.write %s%
cscript /nologo _.vbs <"%~1" >"%~2"
del _.vbs

[mcranmer]

thanks! That was a huge help.

mark

[mcranmer]

I'm hugely impressed with your code, and am learning a lot from studying it.

I'd like to tweak your code so that removes all empty rows, including all carriage returns at the end of the text file.

I have found code which performs this function on a single file and outputs the result to a new file. I simple want all existing text files in the current folder and subfolders to be changed, and not copied.

This is the carriage return and empty row batch I've have found:

Code: Select all

@echo off
setlocal DisableDelayedExpansion
set "firstLineReady="
(
    for /F "eol=$ delims=" %%a in (a.txt) DO (
        if defined firstLineReady (echo()
        set "firstLineReady=1"
        <nul set /p "=%%a"
    )
) > out.txt

Thanks immensely for your help on this. It is so frustrating being a newbie. I'm spending every spare minute poring over this stuff in order to get up to speed.

Mark

[foxidrive]

EDIT:

This should do it using the same techniques. The files are re-written as that is the only way to do it, but they are given the original filenames again.

Test this on some sample files first.

All lines of text in the files must be less than 1023 bytes in length

The other thing to note is the eol=$ segment of the inner for in do loop. This will cause it to skip any lines that start with $
You can use any character here but it will skip lines starting with that character. If you remove eol=$ it will skip lines starting with ; as that is the default.

Code: Select all

@echo off
:: Please note eol=$ will skip lines starting with $
:: All lines of text must be less than 1023 bytes in length
setlocal
set "firstLineReady="
for /f "delims=" %%a in ('dir *.txt /b /s') do (
echo processing %%a
(
    for /f "eol=$ delims=" %%b in ('type "%%a"') do (
        if defined firstLineReady (echo.)
        if not defined firstLineReady set firstLineReady=1
        <nul set /p "=%%b"
    )
) > out.txt
move /y out.txt "%%a" >nul
set "firstLineReady="
)
echo.
echo done.
pause

[mcranmer]

PERFECT! Thanks.

I am using my new vba expertise at work and have saved the team many days of work by building an appliction which makes various changes to thousands of text files based on regex pattern matching, but only just realised that my application is adding a carriage return. This batch file will enable immediate remedial action.

thanks again,

Mark

[foxidrive]

Excellent, Mark. :thumbsup:

[mcranmer]

by batch file is working in certain folders but not others. I'm curious to know why. It appears to hang on the first file.

the folder path is:

V:\Mark\PandR\Projects\TestData\T&C\test

sample filenames are:

B53_RP1_190212.txt
D42_RP1_190212.txt
DF3_RP1_080811.txt
DM3_RP1_190212.txt
DY3_RP1_080811.txt
FG_RP1_030811.txt
GU2_RP1_16112011.txt
IE_RP1_150911.txt
J72_RP1_030811.txt
J82_RP1_030811.txt
J92_RP1_030811.txt


and sample file content is:


header-PolicyDocumentation-100993203
header-policydetails
WaitPeriod=14
TotalvalueofDiscounts=£0.00
TotalPreviousValue=£0.00
TotalPremiumPencePerDay=0p
TotalPremiumExcServiceCostQtrBF=£38.97
TotalPremiumExcServiceCostQtr=£38.97
TotalPremiumExcServiceCostMonBF=£12.99
TotalPremiumExcServiceCostMon=£12.99
TotalPremiumExcServiceCostAnnBF=£155.88
TotalPremiumExcServiceCostAnn=£155.88
TotalPaymentDuePencePerDay=43p
TotalPaymentDue=£155.88
ToDate=03/03/2013
TandCCode=T000
SchemeCode=HS147
RenCont=Non
QuarterlyInstalmentNote=4 instalments of £38.97
ProductCode=DY3
ProductCategory=IDE
PolicyStartDate=04/03/2012
PolicyReference=100993203
PolicyHolderTitle=Mr
PolicyHolderSurname=Product DY3
PolicyHolderInitials=R
PolicyHolderFullName=Mr R Product DY3
PolicyHistoricClaim=no
PolicyExpiryDate=04/03/2013
PolicyClaimFromDate=22/07/11
PayType=CHQ
PaymentReference=HS147DY3/100993203
PaymentMethod=CHQ
PaymentFrequency=Annual
NonConsent9NoEMail3rd=NO
NonConsent8NoEMailHS=NO
NonConsent7NoEMailHS3rd=NO
NonConsent6NoTel3rd=NO
NonConsent5NoTelHS=NO
NonConsent4NoTelHS3rd=NO
NonConsent3NoMail3rd=NO
NonConsent2NoMailHS=NO
NonConsent1NoMailHS3rd=NO
NonConsent12NoSMS3rd=NO
NonConsent11NoSMSHS=NO
NonConsent10NoSMSHS3rd=NO
MonthlyInstalmentNote=12 instalments of £12.99
Mailcode=QIIIA4A
LetterExtension=rp1
LetterAmount=£155.88
LapseDate=20/09/2011
InsuredPostCode=RO3 5OR
InsuredHouseNumber=2
InsuredCountry=Great Britain & NI
InsuredAddress1=Rock Rock
FirstInstalmentValue=£0.00
ExcessIndicator=no
DocsByEmail=No
CustomerReference=43001730
CoverNumber=HS147DY3/100993203
CoolingOffDay=28 days
AverageMonthlyAmount=£12.99
AffinityPartnerName=Homeserve Branded
AffinityPartnerCode=HS
AdvisedSale=n
end-policydetails
end-PolicyDocumentation-100993203

any thoughts?

Mark

[foxidrive]

Place a PAUSE command before the move command.
See if the pause is executed, and if the out.txt file is created when the pause command is on the screen.

Maybe the file is open for exclusive use in another program?

Which filename is on the screen when it hangs?
Can you post that file to a webspace somewhere?

[mcranmer]

At present it appears to with the first file, whichever is on the list.

eg.

filename:DF3_RP1_080811.txt


contents:

header-PolicyDocumentation-100998710
header-policydetails
WaitPeriod=14
TotalvalueofDiscounts=£0.00
TotalPreviousValue=£0.00
TotalPremiumPencePerDay=11p
TotalPremiumExcServiceCostQtrBF=£9.87
TotalPremiumExcServiceCostQtr=£9.87
TotalPremiumExcServiceCostMonBF=£3.29
TotalPremiumExcServiceCostMon=£3.29
TotalPremiumExcServiceCostAnnBF=£39.48
TotalPremiumExcServiceCostAnn=£39.48
TotalPaymentDuePencePerDay=11p
TotalPaymentDue=£39.48
ToDate=03/03/2013
TandCCode=T000
SchemeCode=HS147
RenCont=Non
QuarterlyInstalmentNote=4 instalments of £9.87
ProductCode=DF3
ProductCategory=IDE
PolicyStartDate=04/03/2012
PolicyReference=100998710
PolicyHolderTitle=Mr
PolicyHolderSurname=Product DF3
PolicyHolderInitials=Y
PolicyHolderFullName=Mr Y Product DF3
PolicyHistoricClaim=no
PolicyExpiryDate=04/03/2013
PolicyClaimFromDate=02/08/11
PayType=CHQ
PaymentReference=HS147DF3/100998710
PaymentMethod=CHQ
PaymentFrequency=Annual
NonConsent9NoEMail3rd=YES
NonConsent8NoEMailHS=NO
NonConsent7NoEMailHS3rd=NO
NonConsent6NoTel3rd=YES
NonConsent5NoTelHS=NO
NonConsent4NoTelHS3rd=NO
NonConsent3NoMail3rd=YES
NonConsent2NoMailHS=NO
NonConsent1NoMailHS3rd=NO
NonConsent12NoSMS3rd=YES
NonConsent11NoSMSHS=NO
NonConsent10NoSMSHS3rd=NO
MonthlyInstalmentNote=12 instalments of £3.29
Mailcode=STJA2CD
LetterExtension=rp1
LetterAmount=£39.48
LapseDate=01/10/2011
InsuredPostCode=WS5 5TY
InsuredHouseNumber=89
InsuredCountry=Great Britain & NI
InsuredAddress1=Slords Ave
FirstInstalmentValue=£39.48
ExcessIndicator=no
DocsByEmail=No
CustomerReference=43007161
CoverNumber=HS147DF3/100998710
CoolingOffDay=28 days
AverageMonthlyAmount=£3.29
AffinityPartnerName=Heeee Branded
AffinityPartnerCode=HS
AdvisedSale=n
end-policydetails
end-PolicyDocumentation-100998710


could the security settings on the network, or other factors be relevant? (can't post paste file online at present)

mark

[foxidrive]

It works here with the contents you pasted before and this one.

Try copying the file to a single folder, and run the batch file in that folder. I expect it will work but if it does then there is something else causing the issue.

Re Network security - you must have read and modify permissions for the files.

[foxidrive]

Also try this version with a PAUSE statement after creation of the file - see if it reaches the pause statement (and tell me so if there is still an issue)

and the pause after the move command with the >nul removed so you can see if there is an error there.

So press a key twice for each file. See which file it fails on and try that file in a local folder as a test.

Code: Select all

@echo off
:: Please note eol=$ will skip lines starting with $
:: All lines of text must be less than 1023 bytes in length
setlocal
set "firstLineReady="
for /f "delims=" %%a in ('dir *.txt /b /s') do (
echo processing %%a
(
    for /f "eol=$ delims=" %%b in ('type "%%a"') do (
        if defined firstLineReady (echo.)
        if not defined firstLineReady set firstLineReady=1
        <nul set /p "=%%b"
    )
) > out.txt
pause
move /y out.txt "%%a"
pause
set "firstLineReady="
)
echo.
echo done.
pause

[mcranmer]

just discovered that the batch runs perfectly if run locally, rather than on a network drive. So all's well.

thanks
mark

[severn]

Sorry for posting in an old thread, but my query didn't seem worth starting a new one for...

foxidrive's rename batch files are almost exactly what I'm looking for but I would like to be able to amend text files in multiple subfolders, would any one be able to tell me how to modify the code todo this, I expected it to be a simple /s flag but that didn't work.

Thanks in advance.