Replacing a "|" in an ordinary text file

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Post Reply
Message
Author
hanneshar
Posts: 6
Joined: 02 Sep 2014 05:50

Replacing a "|" in an ordinary text file

#1 Post by hanneshar » 02 Sep 2014 06:41

Hallo,

I'm permanently working with a terminal console on a very very old Unix DB system. If I want to paste texts directly in the terminal application I have to replace the German special characters by different chars. This is the 'translation table':

[ = Ä
] = Ü
{ = ä
} = ü
\ = Ö
~ = ß
| = ö

Currently, I'm first pasting the text in MS Word, use a Macro to change all "Ä" to "[" (and so on), copy the text again and paste it in the terminal application.

Because it would my life easier I've tried now to write a batch program to to this job for me. I was looking and found an example I was able to adapt to my particular needs:

echo off
setlocal enabledelayedexpansion

set "filename=TestText.txt"
set "tempfile=tmptxt.txt"

for /F "delims=" %%a in (%filename%) do (
set text=%%a
set text=!text:^ä={!
set text=!text:^ü=}!
set text=!text:^Ä=[!
set text=!text:^Ü=]!
set text=!text:^Ö=\!
set text=!text:^?=~!
echo !text!>>%tempfile%
)

del %filename%

This batch is working perfectly - but I can't apply it to the "ö -> |" problem... When I add this line to the batch:

set text=!text:^ö=|!

I just receive an error message and the job is terminated.


Big question: what to I have to modify to replace all "ö" by a "|" in my text file?




Greetings,




Hannes

foxidrive
Expert
Posts: 6031
Joined: 10 Feb 2012 02:20

Re: Replacing a "|" in an ordinary text file

#2 Post by foxidrive » 02 Sep 2014 07:45

Double quotes protect several poison characters - this may help:

You may find that a codepage change is needed for some regions but your codepage may already handle German characters.
A related issue is that the characters you see in a text editor can be different to what you see in a cmd prompt.

You may not need the ^ characters there either.

Code: Select all


echo off
setlocal enabledelayedexpansion

set "filename=TestText.txt"
set "tempfile=tmptxt.txt"

for /F "delims=" %%a in (%filename%) do (
set "text=%%a"
set "text=!text:^ä={!"
set "text=!text:^ü=}!"
set "text=!text:^Ä=[!"
set "text=!text:^Ü=]!"
set "text=!text:^Ö=\!"
set "text=!text:^?=~!"
set "text=!text:^ö=|!"
echo !text!>>"%tempfile%"
)

del "%filename%"


hanneshar
Posts: 6
Joined: 02 Sep 2014 05:50

Re: Replacing a "|" in an ordinary text file

#3 Post by hanneshar » 03 Sep 2014 02:19

Thank you for your help! :-)

Your batch is now running without producing an error message - however, the script does not exchange the German special characters by the required chars...

The only way I currently found is to add this line

set text=!text:^ö="|"!

Unfortunately, this is switching (for example) the word >>Höhe<< to >>H"|"he<<... :-( The only combination that currently is doing it's job is this one:

set text=!text:^ö="|"!
set text=!text:^"=!

By this I'm correctly switching the word >>Höhe<< to >>H|he<<. However, this way I'm loosing also the >>"<< chars that are regulatory included in text, for example in quotations.


Do you (or anybody) has another idea how to transfer all >>ö<< to a >>|<< ?



Greetings from Hannover, Germany,




Hannes

hanneshar
Posts: 6
Joined: 02 Sep 2014 05:50

Re: Replacing a "|" in an ordinary text file

#4 Post by hanneshar » 03 Sep 2014 04:16

Sorry for communicating a second problem with the batch file listed above by me... I just realized that this batch script is removing all 'multiple carriage returns' from my processed text. For example I start with this text phrase:

Entnahmen

Wir

My batch script transfers this to

Entnahmen
Wir

If I look in a HexDump of the files my batch script starts with this (reduced to the CR part from 'n' to 'W'):

6E 2E 0D 0A 0D 0A 57

After the batch run this is reduced to

6E 2E 0D 0A 57

In other words: the current batch is removing '0D 0A' from the text. If I send for example 10x '0D 0A' in a line to the batch file it will also end up with 1x '0D 0A', removing the other 9x '0D 0A'.


New question: which line in the script listed above is killing my multiple '0D 0A' ?



Greetings from Hannover, Germany,




Hannes

foxidrive
Expert
Posts: 6031
Joined: 10 Feb 2012 02:20

Re: Replacing a "|" in an ordinary text file

#5 Post by foxidrive » 03 Sep 2014 07:12

The for /f command itself removes blank lines, without extra processing.

When I copy and pasted your code then the characters changed due to codepage differences.
Retype the correct German characters in the script below and see how that goes.

Code: Select all

@echo off
setlocal enabledelayedexpansion

set "filename=TestText.txt"
set "tempfile=tmptxt.txt"

(
for /F "tokens=1,* delims=:" %%a in ('findstr /n "^" ^< "%filename%"') do (
set "text=%%b"
   if defined text (
      set "text=!text:a={!"
      set "text=!text:b=}!"
      set "text=!text:c=[!"
      set "text=!text:d=]!"
      set "text=!text:e=\!"
      set "text=!text:f=~!"
      set "text=!text:g=|!"
   )
echo(!text!
)
)>"%tempfile%"
del "%filename%"


It's interesting that if you remove the if defined text then a bug sets the variable to a={ when the variable is empty.

hanneshar
Posts: 6
Joined: 02 Sep 2014 05:50

Re: Replacing a "|" in an ordinary text file

#6 Post by hanneshar » 04 Sep 2014 11:31

Thank you again for your patience with a medical doctor who tries to improve his daily workflow :-)

I have now copied your modified script, changed the chars to the German special letters and started the batch... it hangs. The file tmptxt.txt is generated, but the size is "0". What might be the problem? When I press Ctrl-C the resulting question "stop patch process (Y/N)" is written to tmptxt.txt.

=========
I now modified my initial script to this one here:

echo off
setlocal enabledelayedexpansion

set "filename=TestText.txt"
set "tempfile=tmptxt.txt"

for /F "delims=" %%a in (%filename%) do (
set text=%%a
set text=!text:^ä={!
set text=!text:^ü=}!

set text=!text:^"=$!
set text=!text:^ö="|"!
set text=!text:^"=!
set text=!text:^$="!

set text=!text:^Ä=[!
set text=!text:^Ü=]!
set text=!text:^Ö=\!
set text=!text:^?=~!
echo !text!>>%tempfile%
)

del %filename%

This way I can indeed exchange the >>|<< character - but I loose the >>§<< char in all of my texts. Okay, you can argue that a medical doctors does not need this symbol... I guess this batch script would be already perfect for me - but I still have the problem with the removed 'multiple carriage returns' ('0D 0A') from my text. Maybe you have an idea how I can retain those 'multiple carriage returns' in my texts?




Greetings,





CH

Squashman
Expert
Posts: 4465
Joined: 23 Dec 2011 13:59

Re: Replacing a "|" in an ordinary text file

#7 Post by Squashman » 04 Sep 2014 11:35

You should put quotes around your set statements.

Code: Select all

set "text=!text:^ö=|!"

foxidrive
Expert
Posts: 6031
Joined: 10 Feb 2012 02:20

Re: Replacing a "|" in an ordinary text file

#8 Post by foxidrive » 04 Sep 2014 18:08

hanneshar wrote:Thank you again for your patience with a medical doctor who tries to improve his daily workflow :-)


You're welcome. It's always useful to automate tasks. :)

Copy and paste the code in my last reply, because that is different to the one you used.

hanneshar
Posts: 6
Joined: 02 Sep 2014 05:50

Re: Replacing a "|" in an ordinary text file

#9 Post by hanneshar » 05 Sep 2014 02:40

Thank you again & again for spending your time with me :-)

I have directly taken your batch script and exchanged only the characters to the German special ones. This is the script I'm currently using:

====================================================
@echo off
setlocal enabledelayedexpansion

set "filename=TestText.txt"
set "tempfile=tmptxt.txt"

(
for /F "tokens=1,* delims=:" %%a in ('findstr /n "^" ^< "%filename%"') do (
set "text=%%b"
if defined text (
set "text=!text:ä={!"
set "text=!text:ü=}!"
set "text=!text:Ä=[!"
set "text=!text:Ü=]!"
set "text=!text:Ö=\!"
set "text=!text:ß=~!"
set "text=!text:ö=|!"
)
echo(!text!
)
)>"%tempfile%"
del "%filename%"
====================================================

When I run this script on a Win7 64bit German OS in a command shell (cmd.exe) it hangs without occupying CPU load as described above...



Sorry for my stupidity - I'm sure I'm doing some very simple wrong but your script 'hangs' in my hands. I just tested this script as well on a Server 2003 32bit German and a Server 2012 64bit German OS - as well on these machines the script hangs. The same happens when I start this batch script in powershell.exe with the command >>.\script.bat<<.



Greetings,




Hannes

foxidrive
Expert
Posts: 6031
Joined: 10 Feb 2012 02:20

Re: Replacing a "|" in an ordinary text file

#10 Post by foxidrive » 05 Sep 2014 05:59

The most likely problem is that your file doesn't have a trailing carriage return. Is that normal for your files?

Can you open it up and add an enter at the bottom and save it - to test this?

This addition to the code should fix it - the last line added below adds a trailing CRLF to the file before processing.

Code: Select all

@echo off
setlocal enabledelayedexpansion

set "filename=TestText.txt"
set "tempfile=tmptxt.txt"

echo(>>"%filename%"


foxidrive
Expert
Posts: 6031
Joined: 10 Feb 2012 02:20

Re: Replacing a "|" in an ordinary text file

#11 Post by foxidrive » 07 Sep 2014 11:39

Did this help you?

hanneshar
Posts: 6
Joined: 02 Sep 2014 05:50

Re: Replacing a "|" in an ordinary text file

#12 Post by hanneshar » 08 Sep 2014 10:34

Sorry for my late answer - I was the weekend at home and had no access to the PC I'm working on the batch.

Now I've added this line at the position in the script:

echo(>>"%filename%"

However, the batch file still removes all additional CR's.


The source is a text generated by notepad.exe.

I guess I have to keep my old word macro...





Greetings,





Hannes

foxidrive
Expert
Posts: 6031
Joined: 10 Feb 2012 02:20

Re: Replacing a "|" in an ordinary text file

#13 Post by foxidrive » 08 Sep 2014 16:14

hanneshar wrote:Now I've added this line at the position in the script:

echo(>>"%filename%"

However, the batch file still removes all additional CR's.


It works fine here.

Try this once more - use the code below and see if it changes the a,b,c,d,e,f,g and preserves the blank lines.

If it does, retype the German characters and try it again.


Code: Select all

@echo off
setlocal enabledelayedexpansion

set "filename=TestText.txt"
set "tempfile=tmptxt.txt"
echo(>>"%filename%"

(
for /F "tokens=1,* delims=:" %%a in ('findstr /n "^" ^< "%filename%"') do (
set "text=%%b"
if defined text (
set "text=!text:a={!"
set "text=!text:b=}!"
set "text=!text:c=[!"
set "text=!text:d=]!"
set "text=!text:e=\!"
set "text=!text:f=~!"
set "text=!text:g=|!"
)
echo(!text!
)
)>"%tempfile%"
del "%filename%"

pieh-ejdsch
Posts: 239
Joined: 04 Mar 2014 11:14
Location: germany

Re: Replacing a "|" in an ordinary text file

#14 Post by pieh-ejdsch » 09 Sep 2014 01:09

foxidrive wrote:It's interesting that if you remove the if defined text then a bug sets the variable to a={ when the variable is empty.

Undefined vars everytimes prompt as their abrevs.
If you want a filled Variable fill with the Line number

Code: Select all

@echo off
setlocal enabledelayedexpansion

set "filename=TestText.txt"
set "tempfile=tmptxt.txt"

(
for /F "tokens=*delims=" %%a in ('findstr /n "^" ^< "%filename%"') do (
  set "text=%%a"
  set "text=!text:a={!"
  set "text=!text:b=}!"
  set "text=!text:c=[!"
  set "text=!text:d=]!"
  set "text=!text:e=\!"
  set "text=!text:f=~!"
  set "text=!text:g=|!"
  echo(!text:*:=!
))>"%tempfile%"
del "%filename%"


Phil

Post Reply