Create nul and all ascii characters with only batch

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Message
Author
dbenham
Expert
Posts: 2388
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Create nul and all ascii characters with only batch

#46 Post by dbenham » 29 Jan 2014 14:01

I agree with penpen - awesome work :!:
I also have had little time to contribute. And I had a similar idea for iterating hex notation of byte codes :)

Here is a version that uses Carlos's new discovery.

I think it is fairly optimized for both performance and code length. I didn't bother skipping makecab for 0x1A.

Code: Select all

REM This code creates 256 files containing one single Byte each from 0x00 until 0xFF
REM Teamwork of carlos, penpen, aGerman, dbenham
REM Tested under Win2000, XP, Win7, Win8
@echo off
setlocal EnableDelayedExpansion
2>nul md "characters"
pushd "characters"

>"t.tmp" type nul
set "hex=0 1 2 3 4 5 6 7 8 9 A B C D E F"
for %%A in (%hex%) do for %%B in (%hex%) do (
  set /a "N=0x%%A%%B"
  >nul makecab /d compress=off /d reserveperfoldersize=!N! /d reserveperdatablocksize=26 "t.tmp" "%%A%%B.chr"
  type "%%A%%B.chr" | ((for /l %%N in (1 1 38) do pause)>nul&findstr "^">"temp.tmp")
  >nul copy /y "temp.tmp" /a "%%A%%B.chr" /b
)
>nul copy /y nul + nul /a "1A.chr" /a
del "t.tmp" "temp.tmp"
popd


Dave Benham
Last edited by dbenham on 29 Jan 2014 22:32, edited 4 times in total.

einstein1969
Expert
Posts: 776
Joined: 15 Jun 2012 13:16
Location: Italy, Rome

Re: Create nul and all ascii characters with only batch

#47 Post by einstein1969 » 29 Jan 2014 14:03

I saw that you are trying to create a binary file that contains the characters.

Why are you using the makecab?

Should not use what you have developed to create an ad hoc binary file?

einstein1969

penpen
Expert
Posts: 1859
Joined: 23 Jun 2013 06:15
Location: Germany

Re: Create nul and all ascii characters with only batch

#48 Post by penpen » 29 Jan 2014 14:26

The makecab is needed to create all binary byte values.
I'm not sure if i understand your second question, you could create a binary [0xAB, 0xBA] file using:

Code: Select all

copy "characters\AB.chr" /B + "characters\BA.chr" /B "adhoc.binary" /B
Or you could do something like that (but this won't work for all characters, so you sometimes need the single files):

Code: Select all

set "table="
set /P "table=" < characters\table.dat
set /P"=%table:~0xAB,1%%table:~0xBA,1%" > adhoc.bin < nul

penpen

Edit1: Added 1 comment: (but this won't work...).
Edit2: corrected a flaw.
Last edited by penpen on 29 Jan 2014 14:28, edited 3 times in total.

carlos
Expert
Posts: 501
Joined: 20 Aug 2010 13:57
Location: Chile
Contact:

Re: Create nul and all ascii characters with only batch

#49 Post by carlos » 29 Jan 2014 14:27

@dbenham. Very nice modification, thanks. I tested ok in Windows xp and 8. Please, you can keep the teamwork header adding your name, description header and the system tested header. All is ok, but it not have the headers.

dbenham
Expert
Posts: 2388
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Create nul and all ascii characters with only batch

#50 Post by dbenham » 29 Jan 2014 14:41

@Carlos -Done. I did the development on Win7

@einstein1969 - True, most of the characters can be generated using CHARLIB.BAT. But that only works because CHARLIB.BAT has binary values embedded within it. That causes problems when posting to/copying from forum sites due to code page issues. It also can cause problems with things like tabs.

The makecab solution is a universal solution that should post well to forum sites, and work on any machine from XP forward without need of any 3rd party tools.

The files need only be generated once. Then most of the byte codes can be read into variables for faster access. I've got some ideas on how to incorporate this into CHARLIB.BAT, but I'm waiting for some free time to implement it.


Dave Benham

aGerman
Expert
Posts: 3944
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Create nul and all ascii characters with only batch

#51 Post by aGerman » 29 Jan 2014 15:27

Thumbs up guys :D That was really a fine step forward.

Regards
aGerman

einstein1969
Expert
Posts: 776
Joined: 15 Jun 2012 13:16
Location: Italy, Rome

Re: Create nul and all ascii characters with only batch

#52 Post by einstein1969 » 29 Jan 2014 15:49

@penpen & dbenham
ok. I did not realize. But now it is more clear.

@aGerman
True! :D

dbenham
Expert
Posts: 2388
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Create nul and all ascii characters with only batch

#53 Post by dbenham » 29 Jan 2014 19:34

I've come up with yet another variation using MAKECAB :twisted:

SET /P will terminate the input upon reading a NULL byte. :)
What really blows me away is that SET /P can read both newline (0x0A) and carriage return (0x0D) into a variable if they are followed by 0x00 :shock:

So I just set the MAKECAB ReservePerDataBlockSize directive to 0 instead of 26, and then use SET /P to read the value into a variable instead of using COPY to write it to a file.

I reverted back to the earlier method for generating a null byte file.

Code: Select all

::This is a demo showing one way to generate all 256 byte codes
::using only batch and native external commands.
::Authors: carlos, aGerman, penpen, dbenham
::Tested on XP, Win7, and Win8

@echo Off
setlocal enableDelayedExpansion

:: Create file null.dat containing a single null byte (0x00).
:: A null byte can only be written to a file.
:: It cannot be read into a variable
cmd /u /c set/p"=a"<nul >bytes.dat
copy /y bytes.dat+nul bytes.dat >nul
type bytes.dat |(pause>nul&findstr "^" >t1.tmp)
copy /y t1.tmp /a null.dat /b >nul


:: Load byte codes 0x01 - 0xFF into a variable with a space
:: at position 0 to take the place of the null byte.
:: Also print out intermediate results.
:: Note that the display of the final bytes variable will appear
:: to be missing values because of the embedded carriage return
type nul >"t.tmp"
set "bytes= "
for /l %%N in (1 1 255) do (
  makecab /d compress=off /d reserveperfoldersize=%%N /d reserveperdatablocksize=0 "t.tmp" "t1.tmp" >nul
  type "t1.tmp" | ((for /l %%N in (1 1 38) do pause)>nul&findstr "^">"t2.tmp")
  <t2.tmp set /p "chr="
  set "bytes=!bytes!!chr!"
  echo   char(%%N^)=[!chr!]
)
del "t.tmp" "t1.tmp" "t2.tmp"
echo bytes=!bytes!


:: Create file bytes.dat containing all 256 byte codes
copy null.dat bytes.dat >nul
<nul set /p "=!bytes:~1!" >>bytes.dat

The hexDump.bat utility is a handy way to examine the resulting files :)
See the post above the link for examples on how to format the output of hexDump.bat


Dave Benham
Last edited by dbenham on 30 Jan 2014 00:20, edited 1 time in total.

carlos
Expert
Posts: 501
Joined: 20 Aug 2010 13:57
Location: Chile
Contact:

Re: Create nul and all ascii characters with only batch

#54 Post by carlos » 29 Jan 2014 21:17

@dave: your demo code works very well. But it is different, not creates the 256 .chr files, it creates the bytes.dat file. I not know if use 00 character + set /p instead the 1a character + copy cutoff will works for create the .chr files separetely.
Because this, I prefer the previous code (linked in the first post of this topic).
But, if you want optimize more your demo code, you you save two pause>nul using this:

Code: Select all

makecab /d compress=off /d reservepercabinetsize=%%N "t.tmp" "t1.tmp" >nul
  type "t1.tmp" | ((for /l %%N in (1 1 36) do pause)>nul&findstr "^">"t2.tmp")


Edit 1: what is the purpose of bytes.dat ?

Edit 2: I test your demo code again, but using cmd /u and with it bytes.dat have 511 bytes.

Edit 3: I prefer this version

Code: Select all

http://www.dostips.com/forum/viewtopic.php?p=32184#p32184
because it Works well using cmd /a and cmd /u. Please you can add a note of this on it? Also I prefer it, because it not need call to cmd /a or cmd /u, for the purpose of create the .chr files is for me the better code.

Edit 4: My prefered version (on the link of edit 3), also Works on Windows 2000. I tested it using cmd /a and cmd /u and the code Works, and also the .chr files have 1 byte correctly (I check this with a c program). Please, you can add that Works on win 2000 to the header?.

dbenham
Expert
Posts: 2388
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Create nul and all ascii characters with only batch

#55 Post by dbenham » 29 Jan 2014 22:24

carlos wrote:@dave: your demo code works very well. But it is different, not creates the 256 .chr files, it creates the bytes.dat file. ...what is the purpose of bytes.dat ?
Yes they are different, and they serve slightly different purposes. The earlier versions using 0x1A as a stop byte absolutely are best for creating single byte files with any byte value. But the last version using a 0x00 stop byte is better if you want to generate any given byte code in a variable.

The bytes variable is very convenient for extracting any character given the byte value. This was the main purpose of the demo.

The bytes.dat is more of an after thought for demo purposes, but not for any specific use. I just wanted a place to collect all the byte values, including 0x00. And it is easy to prove the content is correct using my HexDump.bat. But I don't see the bytes.dat file as having any real value.


carlos wrote:My prefered version (on the link of edit 3), also Works on Windows 2000. I tested it using cmd /a and cmd /u and the code Works, and also the .chr files have 1 byte correctly (I check this with a c program). Please, you can add that Works on win 2000 to the header?.
I don't understand your point, or what you want me to do.


Dave Benham

carlos
Expert
Posts: 501
Joined: 20 Aug 2010 13:57
Location: Chile
Contact:

Re: Create nul and all ascii characters with only batch

#56 Post by carlos » 29 Jan 2014 22:26

@Dave: please sorry for my english. This is my request.
Add:

Code: Select all

REM Tested under 2000, XP, Win7, Win8 using cmd with /u and /a

or similar.

dbenham
Expert
Posts: 2388
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Create nul and all ascii characters with only batch

#57 Post by dbenham » 29 Jan 2014 22:34

Ah, got it. I updated your preferred link.

Did you test my most recent version on Win8 and/or Win2000?

carlos
Expert
Posts: 501
Joined: 20 Aug 2010 13:57
Location: Chile
Contact:

Re: Create nul and all ascii characters with only batch

#58 Post by carlos » 29 Jan 2014 22:47

dbenham wrote:Ah, got it. I updated your preferred link.

Did you test my most recent version on Win8 and/or Win2000?


When i test your code called demo was on Win8 and Works only using cmd /a, using cmd /u the bytes.dat have 511 bytes

Sponge Belly
Posts: 200
Joined: 01 Oct 2012 13:32
Location: Ireland
Contact:

Re: Create nul and all ascii characters with only batch

#59 Post by Sponge Belly » 31 Jan 2014 10:57

Hi Guys, :-)

First off, thanks to aGerman for explaining the makecab hack. I wouldn’t have thought of that in a million years! How did Carlos even think of it? I hereby nominate Carlos for promotion to Expert. Any seconds?

Next up, Dave Benham wrote:

What really blows me away is that SET /P can read both newline (0x0A) and carriage return (0x0D) into a variable if they are followed by 0x00.


True, but that’s not the whole story. I did some testing and it turns out that there has to be at least one non-control character after the null character or set /p will fail. Btw, Tab is treated as a control character.

Lastly, I’ve come up with yet another variant of CreateNul.cmd:

Code: Select all

cmd /d /u /c echo(>xnlutf.tmp
findstr /v ^"^

^" xnlutf.tmp >xnul.chr
del xnlutf.tmp


Bye for now!

- SB

carlos
Expert
Posts: 501
Joined: 20 Aug 2010 13:57
Location: Chile
Contact:

Re: Create nul and all ascii characters with only batch

#60 Post by carlos » 31 Jan 2014 12:38

nice code sponge belly. It now is the most shorter way of create the nul character. In my personal case, I always avoid use the crlf from multiline for codes to be posted on forums, because in the past when I post some code to alt.msdos.batch.nt it add a space to the end of every line of the code, then the lf was damaged.

Post Reply