JREPL.BAT v8.3 - regex text processor with support for text highlighting and alternate character sets

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Message
Author
dbenham
Expert
Posts: 2254
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: JREPL.BAT v8.2 - regex text processor with support for text highlighting and alternate character sets

#421 Post by dbenham » 11 Jul 2019 11:30

onlinestatements wrote:
11 Jul 2019 11:22
Is it possible yet to tell jrepl to only search line numbers 3 thru 10?
I have the latest 8.2 version
It is all documented within the built in help.

Option /INC 3:10 will restrict the search to lines 3 through 10


Dave Benham

onlinestatements
Posts: 10
Joined: 24 Oct 2018 09:54

Re: JREPL.BAT v8.2 - regex text processor with support for text highlighting and alternate character sets

#422 Post by onlinestatements » 15 Jul 2019 06:08

It is not possible to use the /INC switch because my data has NULL bytes.
How can I use the PFLAG "i" option to only process the first found match and quit after that?

Code: Select all

for /R %%I in ("*.out") do (
call C:\qsi\jrepl.bat "\f" "" /xseq /INC "3:10" /F "%%I" /L /O -
call C:\qsi\jrepl.bat "ESCE" "" /xseq /INC "3:10" /F "%%I" /L /O -
)

dbenham
Expert
Posts: 2254
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: JREPL.BAT v8.2 - regex text processor with support for text highlighting and alternate character sets

#423 Post by dbenham » 15 Jul 2019 12:31

OK, I remember this situation now.

No, I don't have a solution for you. JScript has an inherent problem reading lines if the data contains null bytes. The only way JScript can reliably read null bytes is to read the entire file into memory as binary data, which is how the JREPL /M option works.

I can see why it would be useful to be able to read lines containing null bytes, but I am not eager to perform the significant design and coding required to circumvent the inherent JScript limitation.

I have a rough idea how it might be done within the current JREPL architecture, but I am not ready to tackle that task at the moment. I'll think about it, and I may change my mind over the next couple weeks.


Dave Benham

Izya Kurvitch
Posts: 2
Joined: 15 Jul 2019 15:14

Re: JREPL.BAT v8.2 - regex text processor with support for text highlighting and alternate character sets

#424 Post by Izya Kurvitch » 15 Jul 2019 16:51

Hi Mr. Dave. Excuse my English. Really respect what are you doing, not only in IT sphere but in native music :) .
I have a question with my batch script replacing data from a file or list of files which doesn't work and unfortunately exits cycle loop... Could you explain where is a mistake? Here the code:

Code: Select all

@echo off
setlocal enabledelayedexpansion
set "int=000"
set "int_new=111"
for %%i in ("c:\text.txt") do (
jrepl "!int!" "!int_new!" /m /f "c:\text.txt" /o "c:\text_2.txt"
)
pause
It replaces the text but exits of the script. Thank you very much!

onlinestatements
Posts: 10
Joined: 24 Oct 2018 09:54

Re: JREPL.BAT v8.2 - regex text processor with support for text highlighting and alternate character sets

#425 Post by onlinestatements » 15 Jul 2019 21:15

Do you think SED or GREP from GNUWIN32 tools might help me with my problem?

dbenham
Expert
Posts: 2254
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: JREPL.BAT v8.2 - regex text processor with support for text highlighting and alternate character sets

#426 Post by dbenham » 16 Jul 2019 05:12

Izya Kurvitch wrote:
15 Jul 2019 16:51
It replaces the text but exits of the script. Thank you very much!
JREPL.BAT is a batch script (that just happens to incorporate JSCRIPT). Like any other batch script, you must use CALL JREPL if you want to execute JREPL within another batch script and return.


Dave Benham

dbenham
Expert
Posts: 2254
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: JREPL.BAT v8.2 - regex text processor with support for text highlighting and alternate character sets

#427 Post by dbenham » 16 Jul 2019 05:24

onlinestatements wrote:
15 Jul 2019 21:15
Do you think SED or GREP from GNUWIN32 tools might help me with my problem?
Certainly not grep, as it only does searches, not replace.

I believe sed will work, but I wouldn't know how to do it. You should probably look elsewhere for help with sed.


Dave Benham

dbenham
Expert
Posts: 2254
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: JREPL.BAT v8.2 - regex text processor with support for text highlighting and alternate character sets

#428 Post by dbenham » 16 Jul 2019 10:50

@onlinestatements

Oh my goodness - I just realized there is a simple solution for reading files with null bytes without using the /M option :!: :D

It is the Windows Script Host's TextStream.ReadLine method that is unable to read binary data with null bytes. But JREPL already gives you the option to read/write files via ADO instead of TextStream. ADO has no problem reading the binary data with null bytes 8)

So you can drop the /M option and use ADO by appending the correct character set name to the file name. For example, if your input is ASCII, then you can simply use

Code: Select all

jrepl "search" "replace" ...options... /F "input.txt|ascii"
You can also use ADO for the output, but it is not necessary - TextStream.WriteLine has no problem writing null bytes.


I need to update the JREPL documentation to account for this issue. Currently the docs say binary input requires the /M option.


Dave Benham

dbenham
Expert
Posts: 2254
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: JREPL.BAT v8.3 - regex text processor with support for text highlighting and alternate character sets

#429 Post by dbenham » 16 Jul 2019 11:36

I replaced v8.2 with v8.3 - The only change is to the documentation - Binary files with null bytes may be read with either the /M option, or else via ADO.


Dave Benham

Izya Kurvitch
Posts: 2
Joined: 15 Jul 2019 15:14

Re: JREPL.BAT v8.2 - regex text processor with support for text highlighting and alternate character sets

#430 Post by Izya Kurvitch » 16 Jul 2019 15:49

dbenham wrote:
16 Jul 2019 05:12
Izya Kurvitch wrote:
15 Jul 2019 16:51
It replaces the text but exits of the script. Thank you very much!
JREPL.BAT is a batch script (that just happens to incorporate JSCRIPT). Like any other batch script, you must use CALL JREPL if you want to execute JREPL within another batch script and return.


Dave Benham
Thank you very much for your answer! I've just found the same solution of the problem. Anyway, like to you for right realization!
Image
Izya Kurvitch

onlinestatements
Posts: 10
Joined: 24 Oct 2018 09:54

Re: JREPL.BAT v8.3 - regex text processor with support for text highlighting and alternate character sets

#431 Post by onlinestatements » 16 Jul 2019 18:52

how would I implement that solution into my code?

Code: Select all

for /R %%I in ("*.out") do (
call C:\qsi\jrepl.bat "\f" "" /xseq /INC "3:10" /F "%%I" /L /O -
call C:\qsi\jrepl.bat "ESCE" "" /xseq /INC "3:10" /F "%%I" /L /O -
)
Also I posted my question on stack overflow website and somebody gave me a solution using binary

for /R %%I in ("*.out") do call C:\qsi\jrepl.bat "\x1BE([\s\S]+?)\f" "$1" /M /F "%%I" /O -

see the page here that describes the above solution:
https://stackoverflow.com/questions/569 ... 5#57012495

onlinestatements
Posts: 10
Joined: 24 Oct 2018 09:54

Re: JREPL.BAT v8.2 - regex text processor with support for text highlighting and alternate character sets

#432 Post by onlinestatements » 18 Jul 2019 07:16

When I do this code it:

Code: Select all

call C:\qsi\jrepl.bat "\f" "" /F "%%I|ascii" /O -
it eliminates all the FF and keeps the null bytes intact

But if I do this code with /INC option added like:

Code: Select all

call C:\qsi\jrepl.bat "\f" "" /INC "3:9" /F "%%I|ascii" /O -
it doesn't eliminate anything at all.

What am I doing wrong?
dbenham wrote:
16 Jul 2019 10:50
@onlinestatements

Oh my goodness - I just realized there is a simple solution for reading files with null bytes without using the /M option :!: :D

It is the Windows Script Host's TextStream.ReadLine method that is unable to read binary data with null bytes. But JREPL already gives you the option to read/write files via ADO instead of TextStream. ADO has no problem reading the binary data with null bytes 8)

So you can drop the /M option and use ADO by appending the correct character set name to the file name. For example, if your input is ASCII, then you can simply use

Code: Select all

jrepl "search" "replace" ...options... /F "input.txt|ascii"
You can also use ADO for the output, but it is not necessary - TextStream.WriteLine has no problem writing null bytes.


I need to update the JREPL documentation to account for this issue. Currently the docs say binary input requires the /M option.


Dave Benham

mwaychoff
Posts: 1
Joined: 18 Jul 2019 11:06

Re: JREPL.BAT v8.3 - regex text processor with support for text highlighting and alternate character sets

#433 Post by mwaychoff » 18 Jul 2019 11:13

I am writing MediaWiki to PMwiki converter and am finding JREPL extremely useful. I have found two syntax that v8.3 does not handle correctly. Any suggestions would be greatly appreciated.

::SYNTAX EXAMPLES MEDIAWIKI PMWIK

::Images [[Image:wiki.jpg]] Attach:foo.jpg
jrepl_v8_3.bat "[[Image:" "Attach:" /f C:\Temp\jrepl\test.txt /o -

::Internal Link [[a link|with title]] [[page name | link text]]
jrepl_v8_3.bat "|" " | " /f C:\Temp\jrepl\test.txt /o -

dbenham
Expert
Posts: 2254
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: JREPL.BAT v8.3 - regex text processor with support for text highlighting and alternate character sets

#434 Post by dbenham » 18 Jul 2019 21:12

mwaychoff wrote:
18 Jul 2019 11:13
I am writing MediaWiki to PMwiki converter and am finding JREPL extremely useful. I have found two syntax that v8.3 does not handle correctly. Any suggestions would be greatly appreciated.

::SYNTAX EXAMPLES MEDIAWIKI PMWIK

::Images [[Image:wiki.jpg]] Attach:foo.jpg
jrepl_v8_3.bat "[[Image:" "Attach:" /f C:\Temp\jrepl\test.txt /o -

::Internal Link [[a link|with title]] [[page name | link text]]
jrepl_v8_3.bat "|" " | " /f C:\Temp\jrepl\test.txt /o -
[ and | are regular expression meta characters. If you want to search for those literal characters, then you must either add the /L option (literal option), or else escape them in your search string as \[ and \|

Code: Select all

jrepl_v8_3.bat "[[Image:" "Attach:" /f C:\Temp\jrepl\test.txt /o - /l
jrepl_v8_3.bat "|" " | " /f C:\Temp\jrepl\test.txt /o - /l
or

Code: Select all

jrepl_v8_3.bat "\[\[Image:" "Attach:" /f C:\Temp\jrepl\test.txt /o -
jrepl_v8_3.bat "\|" " | " /f C:\Temp\jrepl\test.txt /o -
If you are not using regular expressions, then you are missing the real power of JREPL. They are worth learning.


Dave Benham

Post Reply