replacing a hex sequence in a binary file with a hybrid .bat?

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Message
Author
catalinnc
Posts: 39
Joined: 12 Jan 2015 11:56

replacing a hex sequence in a binary file with a hybrid .bat?

#1 Post by catalinnc » 02 Mar 2017 13:11

i have a 16 bytes binary file (hex.bin) with this content:

Code: Select all

F1F2F3F4F5A0A0FFFFA0A0F1F2F3F4F5


is there a way to use a hybird .bat to replace the 0xA0A0 hex sequences with 0xBBBBBB?

the final result will have 18 bytes and will look like this:

Code: Select all

F1F2F3F4F5BBBBBBFFFFBBBBBBF1F2F3F4F5

_

Squashman
Expert
Posts: 4465
Joined: 23 Dec 2011 13:59

Re: replacing a hex sequence in a binary file with a hybrid .bat?

#2 Post by Squashman » 02 Mar 2017 13:44

Certutil comes to mind.

Aacini
Expert
Posts: 1885
Joined: 06 Dec 2011 22:15
Location: México City, México
Contact:

Re: replacing a hex sequence in a binary file with a hybrid .bat?

#3 Post by Aacini » 02 Mar 2017 13:53

Same solution for your previous question:

Code: Select all

@set @a=0 /* & echo off

cscript //nologo //E:JScript "%~F0" < file.txt > file.tmp
move /Y file.tmp file.txt > NUL

goto :EOF */


WScript.Stdout.Write(WScript.Stdin.ReadAll().replace(/\x0A0\x0A0/g,"\x0BB\x0BB\x0BB"));

Antonio

aGerman
Expert
Posts: 4654
Joined: 22 Jan 2010 18:01
Location: Germany

Re: replacing a hex sequence in a binary file with a hybrid .bat?

#4 Post by aGerman » 02 Mar 2017 13:57

I would use certutil for that little amount of data, too.

Code: Select all

@echo off &setlocal EnableDelayedExpansion
certutil -f -encodehex "hex.bin" "hex.hex"
set "hex="
for /f "usebackq tokens=2-17" %%a in ("hex.hex") do set "hex=!hex!%%a %%b %%c %%d %%e %%f %%g %%h %%i %%j %%k %%l %%m %%n %%o %%p "
>"hex.hex" echo(!hex:a0 a0=bb bb bb!
certutil -f -decodehex "hex.hex" "hex.bin"
del "hex.hex"

Steffen

dbenham
Expert
Posts: 2461
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: replacing a hex sequence in a binary file with a hybrid .bat?

#5 Post by dbenham » 02 Mar 2017 14:06

You have already been shown how you can use JREPL.BAT to edit a file with hex sequences in the replace value at viewtopic.php?f=3&t=6044&start=270#p51373.

And Aacini's post directly following that one shows how to do the same thing with a dedicated hybrid script.

It is not much of a leap to extend either solution to include hex sequences in the search term.

Using JREPL.BAT

Code: Select all

jrepl "\xA0\xA0" "\xBB\xBB\xBB" /m /x /f hex.bin /o -

The /M option is only needed if the input might contain \x00 bytes. If you know that it cannot contain null bytes, then you can remove the /M option.

Aacini has already shown how to modify his dedicated hybrid script to solve this.

Hopefully you will show some initiative and attempt to solve the problem yourself before you ask a question the next time you have a file content find/replace task.


Dave Benham

catalinnc
Posts: 39
Joined: 12 Jan 2015 11:56

Re: replacing a hex sequence in a binary file with a hybrid .bat?

#6 Post by catalinnc » 02 Mar 2017 14:24

thanks a lot to everybody for your kind help...
_

p.s. i opened this thread because i find that replacing hex strings is much more useful for my work than the text replacing...
_

stondesign
Posts: 22
Joined: 23 Oct 2018 17:28

Re: replacing a hex sequence in a binary file with a hybrid .bat?

#7 Post by stondesign » 23 Oct 2018 17:31

Hi...

jrepl "\xAA\xBB\xCC "\x11\x22\x33" /m /x /f hex.bin /o -

is it possible to save the file with a new name?

dbenham
Expert
Posts: 2461
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: replacing a hex sequence in a binary file with a hybrid .bat?

#8 Post by dbenham » 24 Oct 2018 05:36

Of course - simply substitute the file name (file path) for the dash in the /O option.

I assume you have not yet gotten a copy of JREPL. Before you ask too many questions, be sure you read the built in help.

Below are a few useful help commands to get you started.
JREPL /?? - All available help, one page at a time - lengthy!
JREPL /?HELP - All available forms of help
JREPL /?OPTIONS - A brief summary of all options
JREPL /?/O - Detailed help for the /O option. The same general form works for all options.


Dave Benham

stondesign
Posts: 22
Joined: 23 Oct 2018 17:28

Re: replacing a hex sequence in a binary file with a hybrid .bat?

#9 Post by stondesign » 24 Oct 2018 07:47

Hello, I created a program in Batch that basically should modify a Binary file to which the bit values should change

00084750: 37 B9
00084751: AC 92

but when I go to use the string
jrepl "x37 \ xAC \ x01" "xB9 \ x92 \ x01" / m / x / f original.bin / O modified.bin

Saving the File

When I then compare with
fc / b original.bin modified.bin

There are no differences on the file, so it has not been changed ...
it is possible to have a script that works even on a longer list and maybe give it the possibility to intervene as below also on a .txt file

Example
00084750: 00 01
00084751: 00 02
00084752: 00 03
00084753: 00 04
00084754: 00 05
00084755: 00 06
....

dbenham
Expert
Posts: 2461
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: replacing a hex sequence in a binary file with a hybrid .bat?

#10 Post by dbenham » 24 Oct 2018 10:29

@stondesign

Please learn to use [code][/code] blocks for showing code. It is much easier to read.

The syntax is horribly wrong in the commands that you listed. For example, there is never a space after the / when specifying an option. For example, it should be "/m", not "/ m". And your hex strings are also completely bogus. For example, I suspect you intended your find string to be "\x37\xAC\x01".

But beyond that, I don't understand what your goal is. Are you trying to modify a specific sequence of bytes into another sequence throughout the entire file? That is what this thread is about.

Or are you trying to set the byte value at specific offsets within a file? That is an entirely different problem.

Please be more precise when you describe your problem.


Dave Benham

stondesign
Posts: 22
Joined: 23 Oct 2018 17:28

Re: replacing a hex sequence in a binary file with a hybrid .bat?

#11 Post by stondesign » 24 Oct 2018 14:45

Hi, basically I have 2 car maps in Binary files
1 original (original.ori)
1 already modified without EGR valve (egroff.bin)

I would like to create a small program that dragging the file (originale.ori) on the .Bat let me intervene on the bits that I can edit.

Example I would like to transform the bits of the offsets as below ...
00084750: 37 -----> B9
00084751: AC -----> 92

Obviously I know that for 4 bits I can do it manually but on a file with greater offset changes I would like to do it with a small program ...

dbenham
Expert
Posts: 2461
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: replacing a hex sequence in a binary file with a hybrid .bat?

#12 Post by dbenham » 24 Oct 2018 19:46

Here is how JREPL could do your example task, assuming your offsets are 0 based:

Code: Select all

call jrepl "([\s\S]{84750})\x37\xAC([\s\S]*)" "$1\xB9\x92$2" /m /xseq /f original.ori /o egroff.bin
Dave Benham

Aacini
Expert
Posts: 1885
Joined: 06 Dec 2011 22:15
Location: México City, México
Contact:

Re: replacing a hex sequence in a binary file with a hybrid .bat?

#13 Post by Aacini » 24 Oct 2018 22:09

This topic is rather interesting to me, so I wrote a complete solution for it. Please, note that does not matter the original value of the bytes in the file, just the offset and the new value at such offset.

This is the code. An explanation and example of its use is below.

Code: Select all

@echo off
setlocal EnableDelayedExpansion

set "file=myCmd.exe"
for %%a in ("%file%") do set "hex=%%~Na.hex"

rem Specify the desired changes as offset:newValue pairs
rem Add left zeros in offsets with less than five digits *only*
rem E.G.: to change "1C8608  E.C.H.O." by "S.H.O.W.":
set "changes=1C8608:53 1C860A:48 1C860C:4F 1C860E:57"

rem Encode file from bin to text hex
certutil -f -encodehex "%file%" "%hex%"

rem Create a list of line-offsets to search for
set "offsets="
for %%c in (%changes%) do for /F "tokens=1,2 delims=:" %%a in ("%%c") do (
   set "offset=%%a"
   set "offsets=!offsets! !offset:~0,-1!"
)

rem Convert line-offsets to a list of line numbers
set "lines="
for /F "delims=:" %%a in ('findstr /N /I "%offsets%" "%hex%"') do set "lines=!lines! %%a"

rem Process the hex file and perform the desired changes
set "tokens=%%A %%B %%C %%D %%E %%F %%G %%H  %%I %%J %%K %%L %%M %%N %%O %%P   %%Q"
set "letter=ABCDEFGHIJKLMNOP"
for /F "tokens=1*" %%a in ("%changes%") do set "change=%%a" & set "changes=%%b"
set "last=1"
< "%hex%" (

   rem Process target lines
   for %%l in (%lines%) do (

      rem Copy the lines between target ones
      set /A skip=%%l-last, last=%%l+1
      for /L %%i in (1,1,!skip!) do (
         set /P "line="
         echo !line!
      )

      rem Process the target line
      set /P "line="
      for /F "tokens=1*" %%x in ("!line!") do (
         set "ofs=%%x" & set "values=%%y"
         for /L %%# in (1,1,16) do for /F "tokens=1,2 delims=:" %%a in ("!change!") do (
            set "offset=%%a"
            if /I "!offset:~0,-1!" equ "!ofs!" (
               set /A "token=0x!offset:~-1!"
               for /F %%x in ("!token!") do for /F %%y in ("!letter:~%%x,1!") do call :ChangeValue %%y
               if defined changes (
                  for /F "tokens=1*" %%x in ("!changes!") do set "change=%%x" & set "changes=%%y"
               ) else (
                  set "change="
               )
            )
         )
      )
      echo !ofs!	!values!
   )

   rem Copy the lines after the last target one
   findstr "^"

) > temp.hex
del "%hex%"

rem Decode file back from hex to bin
certutil -f -decodehex temp.hex "%file%"
del temp.hex

goto :EOF


:ChangeValue token
set "newVals=!tokens:%1=b!"
for /F "tokens=1-16*" %%A in ("%values%") do set "values=%newVals%"
exit /B
Example of a cmd.exe session that uses previous code:

Code: Select all

C:\Antonio\tests> rem Copy cmd.exe as new local file

C:\Antonio\tests> copy "%comspec%" myCmd.exe
        1 archivo(s) copiado(s).

C:\Antonio\tests> rem Encode it as text hex file

C:\Antonio\tests> certutil -encodehex myCmd.exe myCmd.hex
Longitud de entrada = 404992
Longitud de salida = 1894304
CertUtil: -encodehex comando completado correctamente.

C:\Antonio\tests> rem Search for the place where "echo" command is located

C:\Antonio\tests> findstr /I "e.c.h.o" myCmd.hex
1c860   00 00 00 00 00 00 00 00  45 00 43 00 48 00 4f 00   ........E.C.H.O.

C:\Antonio\tests> rem At this place, insert appropriate data in test.bat

C:\Antonio\tests> rem Modify "ECHO" by "SHOW" in myCmd.exe

C:\Antonio\tests> test
Longitud de entrada = 404992
Longitud de salida = 1894304
CertUtil: -encodehex comando completado correctamente.

Longitud de entrada = 1894304
Longitud de salida = 404992
CertUtil: -decodehex comando completado correctamente.

C:\Antonio\tests> rem Test it

C:\Antonio\tests> myCmd
Microsoft Windows [Versión 6.2.9200]
(c) 2012 Microsoft Corporation. Todos los derechos reservados.

C:\Antonio\tests> show Hello world
Hello world
Antonio

dbenham
Expert
Posts: 2461
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: replacing a hex sequence in a binary file with a hybrid .bat?

#14 Post by dbenham » 24 Oct 2018 22:35

I like it :D
Once I saw your idea to use CERTUTIL, I began thinking of all the optimizations that could be done, and you hit all that I could think of, especially the use of SET /P to read through all the changes, and FINDSTR to read/write the remainder.

I'm not sure this makes a substantive difference, but you might consider generating the hex file without offsets or ASCII by adding 4 as a 3rd paramater to CERTUTIL -encodeHex, as described at viewtopic.php?f=3&t=8521


Dave Benham

Aacini
Expert
Posts: 1885
Joined: 06 Dec 2011 22:15
Location: México City, México
Contact:

Re: replacing a hex sequence in a binary file with a hybrid .bat?

#15 Post by Aacini » 25 Oct 2018 03:55

dbenham wrote:
24 Oct 2018 22:35
I like it :D
Once I saw your idea to use CERTUTIL, I began thinking of all the optimizations that could be done, and you hit all that I could think of, especially the use of SET /P to read through all the changes, and FINDSTR to read/write the remainder.
An "optimization" I thought on, but not implemented it, is to use the same SET /P command to both output the current line and input the next line: 8)

Code: Select all

set /P "line=!line!" & echo/
dbenham wrote:
24 Oct 2018 22:35
I'm not sure this makes a substantive difference, but you might consider generating the hex file without offsets or ASCII by adding 4 as a 3rd paramater to CERTUTIL -encodeHex, as described at viewtopic.php?f=3&t=8521


Dave Benham
If the hex file is generated with no offsets then there is no way to match its lines with the desired changes (that are based on offsets)...

Antonio

Post Reply