Create n amount of random dummy files

[noprogrammer]

Hi,

a few situations require random binary files of varying size to be quickly generated.
Although there might be tools for this, I'd like to do this via cmd. As the first approach, fsutil came into my mind:

Code: Select all

fsutil file createnew somefile.dat 2046

I then found the (better) auxiliary rdfc (program size: 39 kb).
However, I'd prefer having a script using Windows 7 tools merely -- if possible...

Here's what I want to do:

1. Get random filesize as

   Code: Select all

   Set /a %RANDOM% % 19999999

   if filesize is smaller than 1 kb
2. Create new file with random filename (of reasonable length) and calculated filesize
   (Real random content, not just zero-filled like fsutil does, something like dd¹ if=/dev/urandom .. for Linux does)
3. Optionally compress the file with an archiver (7z/rar); nice to have only

My main problems as of now:

1. I need some kind of while condition in step 1.
2. Unsure about randomizing name strings & automization concept, maybe using a wordlist?
   Maybe better approach for creating random file content than rdfc?

I'm aware of the limits of Batch language, I guess anybody would suggest using bash instead...

Thanks for your ideas.


¹ Apparently there's some sort of dd port for Windows (chrysocome net), but it seems just to be a poor copy of the Unix command.

[penpen]

A.I need some kind of while condition in step 1.

Why do you need such a condition, just pick a random number out of [1024 : 19999999].

This may help, although i create files and zip them in @todo-pseudocode only:

Code: Select all

@echo off
cls
setlocal enableDelayedExpansion
:: loading file names from names.txt in the same directory as this batch file
:: assumed: filename/index array length <= 2^15 (maximum random number)
call :loadFileNames names "%~p0names.txt"
call :initIndices names indices

:: create a random number of files with pseudo random filenames (selected from a list randomly)
set /A "createFileCount=(%RANDOM%%%indices.length)+1"
echo(create %createFileCount% files:

for /L %%a in (1,1,%createFileCount%) do (
   call :pickRandom index indices
   set /A "fileSize=(((!RANDOM!<<15)|!RANDOM!)%%(19999999-1024))+1024"
   set /A "zip=%RANDOM%%%2"
   for %%b in (!index!) do   (
      if !zip! == 1 (
         echo(@todo= create the file !names[%%b]! with size of !fileSize! bytes and zip it.
      ) else (
         echo(@todo= create the file !names[%%b]! with size of !fileSize! bytes.
      )
   )

)
goto :eof


:pickRandom
::   %~1 variable name to store the picked element
::   %~2 index array name to pick the random element
::   assumed: index array length <= 2^15 (maximum random number)
::   returns 0 on error
::   picked element is deleted
   set /A "%~1=!%~2.length!"
   if not "!%~2.length!" == "!%~1!" set "%~1=0" & goto :eof
   if !%~1! LEQ 0 set "%~1=0" & goto :eof
   set /A "%~1=(%RANDOM%%%!%~2.length!)+1"
   set /A "%~2.pick=%~2[!%~1!]"
   set /A "%~2[!%~1!]=%~2[!%~2.length!]"
   set "%~2[!%~2.length!]="
   set /A "%~1=%~2.pick"
   set "%~2.pick="
   set /A "%~2.length-=1"
   goto :eof


:initIndices
::   %~1 filename array name
::   %~2 index array name
   set /A "%~2.length=!%~1.length!"
   for /L %%a in (1,1,!%~1.length!) do set "%~2[%%a]=%%a"
   goto :eof


:loadFileNames
::   %~1 array name
::   %~2 "filename to load file names from"
   set /A "%~1.length=0"
   for /F "tokens=* usebackq delims=" %%a in ("%~2") do (
      set /A "%~1.length+=1"
      set "%~1[!%~1.length!]="%%~a""
   )
   goto :eof

The code above computes the number of files, the pseudo random filenames,
their random filesize in [1024 : 19999999] and if they should be zipped.

The above batch script needs a file named "names.txt" with a content similar to this:

Code: Select all

a.txt
b.txt
c.txt

But you may define the filenames explicitely within the batch script, instead of loading it from a file.

To create the random filecontent i recommend you to use a compiled (executable) file,
as interpreted scripts are much to slow for creating a huge number of random files.

penpen

Edit: Maybe you want to use the compiler packages of the .NET, if installed (optional updates), to create the random file content.
Here is n example batch file to compile C# code:

Code: Select all

::csc.bat
:main
   @echo off
   setlocal
   cls

   set "csc="

   pushd "%SystemRoot%\Microsoft.NET\Framework"
   for /f "tokens=* delims=" %%i in ('dir /b /o:n "v*"') do (
      dir /a-d /b "%%~fi\csc.exe" >nul 2>&1 && set "csc="%%~fi\csc.exe""
   )
   popd

   if defined csc (
      echo most recent C#.NET compiler located in:
      echo %csc%.
   ) else (
      echo C#.NET compiler not found.
      goto :eof
   )

   
   %csc% /nologo /target:exe /out:"%~f1.exe" "%~f1.cs"
   goto :eof

Just create a C# source code file "test.cs", and compile it using the above batch, to create the test.exe:

Code: Select all

csc.bat "test"

You may use the VB, JScript compilers (vbc.exe, jsc.exe) instead of the C# compiler (csc.exe).

Edit2: changed the first block from code to quote.

[noprogrammer]

Thank you, penpen. (I'll take a closer look.)

Albeit I did a little programming (Ansi C, Javascript, VBscript, AutoIt), I don't know C# at all, so for the nonce I'd rather stick to rdfc. If you happen to know a trick how to imitate dd if=/dev/urandom.. using csc.exe, I'm all ears!

I think I should explain my idea in greater detail. I said I'd like to generate a varying number of binary files of variable size.

To get the big picture, I'm looking for something similar to this great draft from DerbyCon 2011.

Actually, int0x80 held a Anti Forensics talk about how to give Forensic guys a hard time analyzing a such enhanced system.
There are more cool ideas available, but back to the topic.

I just thought it would be nice having some modular script to generate binary rubbish ("files"), provide a random "file name" including "extension", apply a date stamp (I already got that part), and optionally compress them (nice to have, though).

Although I'm not into anti forensics that much, I thought of using that for e.g. testing purposes (backup and compression software).

[penpen]

So you're looking for something like this?

Code: Select all

@echo off
setlocal
set "wordlist=\usr\share\dict\american-english-insane"
goto :entryPoint

:intToHex
::   %~1 variable to store the hex value
::   %~2 int32 string value
::   needed only to simulate the "random_pass=$(dd..." line in procedure ":do_a_file"
   setlocal enableDelayedExpansion
   set /A "num=%~2"
   set "hex="
   set "digits=0123456789ABCDEF"
   for /L %%a in (0,1,7) do (
      set /A "nibble=num&0xF", "num>>=4"
      for %%b in (!nibble!) do set "hex=!digits:~%%b,1!!hex!"
   )
   endlocal & set "%~1=%hex%"
   goto :eof



:do_a_file
   setlocal
   set "filename=%~1"
   set "filename=!filename:\\=\!"
   set /A "fileSize=(((!RANDOM!<<15)|!RANDOM!)%%(19999999-1024))+1024"

echo @   "dd if=/dev/urandom of="!filename!" bs=1 count=!filesize! status=noxfer conv=notrunc,noerror 2>nul"
rem   useless waste of resources: compute a sha512 sum over a random 128 byte field.
rem   (128*8)%512==0 => sha512 sum is also random over the same (uniform) distribution function ...
rem !   random_pass=$(dd if=/dev/urandom bs=128 count=1 status=noxfer 2> /dev/null | sha512sum - | cut -f 1 -d ' ')
rem   instead create a random 64 byte sha 512 sum as a stream of 16 random sint32 in hex:
   for /L %%a in (1,1,16) do (
      call :intToHex "hex" "(((!RANDOM!<<15)+!RANDOM!)<<15)+!RANDOM!"
      set "random_pass=!random_pass!!hex!"
   )
   set /A "extension=!RANDOM! %% 3"
   goto :random_!extension!

:random_0
echo @   "gpg -q --yes --passphrase "!random_pass!" -c "!filename!" >nul"
   goto :end_random
:random_1
   set "tempfile=!filename!.rar"
echo @   "rar a -o+ -hp"!random_pass!" "!tempfile!" "!filename!" >nul"
   goto :end_random
:random_2
   set "tempfile=!filename!.7z"
echo @   "7za a -y -mhe=on -p"!random_pass!" "!tempfile!" "!filename!" >nul"
   goto :end_random

:end_random
echo @   "shred -n 1 -zu "!filename!""
   goto :eof



:pick_durr
   set "subdir_path=!%~1!"
   set "durr="
   set "subdir_count=0"

   for %%a in ("!subdir_path:\=" "!") do (
      set /A "subdir_count+=1"
      set "durr[!subdir_count!]=%%~a"
   )
   set /A "num_subdir=!RANDOM! %% !subdir_count!"

   for /L %%a in (1, 1, !num_subdir!) do set "durr=!durr!\!durr[%%a]!"
   if defined durr set "durr=!durr:~1!"

   set "%~2=!durr!"
   goto :eof


:build_filename
   set "filename="
   set /A "num_namewords=(!RANDOM! %% 5)"
   for /F "tokens=1 delims=:" %%a in ('findstr /N "^^" !wordlist!') do set num_words=%%a
   for /L %%a in (0, 1, !num_namewords!) do (
      set /A "rand_line=!RANDOM! %% !num_words!"
      (
         for /L %%b in (1, 1, !rand_line!) do (
            set "rand_word="
            set /P "rand_word="
         )
      ) < "!wordlist!"
      if %%a == 0 (
         set "filename=!rand_word!"
      ) else (
         set "filename=!filename!_!rand_word!"
      )
   )

   set "%~1=!filename!"
   goto :eof


:build_path
   set /A "num_subdirs=!RANDOM! %% 13"
   if !num_subdirs! == 0 (
      echo(
      exit /b 0
   )

   set "subdir_path="
   for /F "tokens=1 delims=:" %%a in ('findstr /N "^^" !wordlist!') do set num_words=%%a
   for /L %%a in (1, 1, !num_subdirs!) do (
      set /A "rand_line=!RANDOM! %% !num_words!"
      (
         for /L %%b in (1, 1, !rand_line!) do (
            set "rand_word="
            set /P "rand_word="
         )
      ) < "!wordlist!"
      if %%a == 1 (
         set "subdir_path=!rand_word!"
      ) else (
         set "subdir_path=!subdir_path!\!rand_word!"
      )
   )

   set "%~1=!subdir_path!"
   goto :eof


:main
   setlocal enableDelayedExpansion
   for /L %%a in (1, 1, %~2) do (
      call :build_path "subdurr_path"
echo "      mkdir "%~1\!subdurr_path!""
      set /A "num_files=(!RANDOM!%%(32768-9001))+9001"

      for /L %%b in (!num_files!, -1, 1) do (
         call :build_filename "filename"
         call :pick_durr "subdurr_path" "subdurr"
         call :do_a_file "%~1\!subdurr!\!filename!"
      )
   )
echo "   ((del /S /F "%~1" > nul) 2> nul)"

   endlocal
   goto :eof



:entryPoint
if not "%~1" == "" if not "%~2" == "" (
   call :main "%~1" "%~2"
   endlocal
   exit /B 0
)

echo "Usage: %~nx0 <directory> <iterations>"
endlocal
exit /B 9001

The above code only simulates a run for testing.
To make it fully work, just delete the encapsulating [echo "]["] in the procedure :main,
and replace the "echo @" lines with external commands, that will do the desired.
The procedure :intToHex is only needed to simulate the rem! line.
Replace the loop, where it is called with the program of your choice, to solve the task.

Note:
The code above is translated from bash file linked in the post above.
It may be highly optimized, but maybe the speed is wanted: i don't know, so i doesn't changed it.

penpen

Edit: Removed posting my opinion to that, as it was inappropriate.
Edit2: Added the full code, read the comment below the code to make it fully work.

[penpen]

Albeit I did a little programming (Ansi C, Javascript, VBscript, AutoIt), I don't know C# at all, so for the nonce I'd rather stick to rdfc. If you happen to know a trick how to imitate dd if=/dev/urandom.. using csc.exe, I'm all ears!

It is no trick, but a simple C# algorithm:

Code: Select all

using System;
using System.IO;
using System.Security.Cryptography;



class RndFile {
   private static RNGCryptoServiceProvider rngCsp = null;
   private const int NO_ERROR = 0;
   private const int WRONG_ARGS = 1;
   private const int ERROR = 2;


   /*
    * filename is to be assumed non null, not empty and a valid filename string,
    * check it prior using.
    */
   public static int CreateRandomFile (string filename, int filesize) {
      const int BUFFER = 2097152;         // 2 MB
      byte [] buffer1 = new byte [BUFFER];
      byte [] buffer2 = new byte [filesize % BUFFER];

      if (buffer1 == null) {   
      } else if (buffer2 == null) {
      } else try {
         using (FileStream fs = System.IO.File.Create (filename)) {
            for (;BUFFER <= filesize; filesize -= BUFFER) {
               rngCsp.GetBytes (buffer1);
                    fs.Write (buffer1, 0, buffer1.Length);
            }
            rngCsp.GetBytes (buffer2);
                 fs.Write(buffer2, 0, buffer2.Length);
            fs.Flush ();
         }

         return NO_ERROR;
      } catch (CryptographicException) {
         System.Console.WriteLine ("CSP fail: Could not be acquired.");
      } catch (ArgumentNullException) {
         System.Console.WriteLine ("Buffer fail: Is null.");
      } catch (IOException) {
         System.Console.WriteLine ("Write fail: IOException.");
      } catch (ObjectDisposedException) {
         System.Console.WriteLine ("Write fail: DisposedException");
      }

      return ERROR;
   }

   public static int Main (string [] args) {
      string args0 = null;
      int argi1 = -1;
      int retValue = WRONG_ARGS;

      if (args.Length != 2) {
         System.Console.WriteLine ("Arguments: Wrong argument count.");
      } else if ((args0 = args [0]) == null) {
         System.Console.WriteLine ("Argument 1: ArgumentNullException.");
      } else if (args0.Length == 0) {
         System.Console.WriteLine ("Argument 1: Empty string.");
      } else if (args0.IndexOfAny (Path.GetInvalidPathChars()) >= 0) {
         System.Console.WriteLine ("Argument 1: Contains forbidden chars.");
      } else if (File.Exists (args0)) {
         return NO_ERROR;
      } else try {
         argi1 = System.Convert.ToInt32 (args [1]);
         rngCsp = new RNGCryptoServiceProvider ();
         retValue = NO_ERROR;
      } catch (System.ArgumentNullException) {
         System.Console.WriteLine ("Argument 2: ArgumentNullException.");
      } catch (System.FormatException) {
         System.Console.WriteLine ("Argument 2: Not an int value.");
      } catch (System.OverflowException) {
         System.Console.WriteLine ("Argument 2: OverflowException.");
      } catch (CryptographicException) {
         System.Console.WriteLine ("CSP fail: Could not be acquired.");
         retValue = ERROR;
      }

      if (retValue == NO_ERROR) {
         if (rngCsp != null) {
            retValue = CreateRandomFile (args0, argi1);
            rngCsp.Dispose ();
         } else {
            System.Console.WriteLine ("RNGCryptoServiceProvider: Could not be instanciated.");
            retValue = ERROR;
         }
      } else if (retValue == WRONG_ARGS) {
         System.Console.WriteLine ("Usage: " + Environment.GetCommandLineArgs () [0] + " filename filesize(sint32+)");
      }

      return retValue;
   }
}

penpen

[noprogrammer]

penpen, the cmd code is really impressing and good news is that I ran parts of it successfully.

Ignoring the dir sub for a start, the creation/compression of bogus files works fine.
After that I got caught in the :build_filename sub, even after replacing

Code: Select all

For /f "tokens=1 delims=:" %%a In ('findstr /N "^^" !wordlist!') Do Set num_words=%%a

by the line

Code: Select all

For /f "tokens=2 delims=:" %%a In ('find /c /v "" !wordlist!') Do Set/a num_words=%%a

I assume parsing of the wordlist file fails...
A guess: It should be within the For /l %%a In (0, 1, !num_namewords!) loop, right after the Set/a line..
(As for the wordlist, I grabbed a dic file with 1382512 lines having 1 word each.)

By the way, I noticed your script lacks "enableExtensions". Did you do that on purpose?
I had to add it trying to run subs like :do_a_file ...

Instead of the dd if=/dev/urandom.. command I used the following line

Code: Select all

rdfc.exe !filename! !filesize! B overwrite

which seems to run fine...

Thank you for the great ideas!

I haven't used your C# example yet, but it might be something like

Code: Select all

binfill.exe !filename! !filesize!

with the !filesize! option similar to rdfc.

[penpen]

After that I got caught in the :build_filename sub, even after replacing

Code: Select all

For /f "tokens=1 delims=:" %%a In ('findstr /N "^^" !wordlist!') Do Set num_words=%%a

by the line

Code: Select all

For /f "tokens=2 delims=:" %%a In ('find /c /v "" !wordlist!') Do Set/a num_words=%%a

I assume parsing of the wordlist file fails...
A guess: It should be within the For /l %%a In (0, 1, !num_namewords!) loop, right after the Set/a line..
(As for the wordlist, I grabbed a dic file with 1382512 lines having 1 word each.)

The code above is fully functional, i have tested it.
I'm not sure, why you have replaced the above line, but you should note, that this line does NOT initialize the wordlist,
it just computes the number of lines (words) that the wordlist holds:
The 'findstring /N "^^" wordlist' Searches for the "start of line" and passes the numbered lines with the format "N: text"
to the command line parser (CLP). Because of "tokens=1 delims=:" the CLP grabs the first token, that ends with a colon
(excluded) and sets %%a to the line number, which is then stored to num_words.
The lst entry stays, so when the for loop finished num_words contains the numbers of lines/words in the wordlist dictionary.
I've lazy programmed it, to see the bash script shimmering through easily.
So i didn't initiate the variable num_words to 0, as you also have other problems, if the dictionary is empty.
I also didn't any optimiziation the bash code (expect for the sha512, and set variables to random parts) as posted above:
For example especially this variable (num_words) only needs to be initiated once, and not ten thousands of times.
Its all the same for the words in the wordlist... .
But as assumed, the slow speed of the batch is not the problem, as creating of the randomfiles is the time dominating part.

By the way, I noticed your script lacks "enableExtensions". Did you do that on purpose?
I had to add it trying to run subs like :do_a_file ...

In all windows versions the extensions should be enabled by default, except in protected mode, and when installing.
I assumed you don't wanted to use it in such cases.
If your admin has disabled it by default, just add it to the second line, and check if they've been enabled,
or if the enabling is also disabled, but i doubt that, but if, you have to revise the code.

I haven't used your C# example yet, but it might be something like

Code: Select all

binfill.exe !filename! !filesize!

with the !filesize! option similar to rdfc.

Yes, in this example it creates a file named "!filename!" with !filesize! bytes and a full random content.

penpen

[noprogrammer]

I'm not sure, why you have replaced the above line, but you should note, that this line does NOT initialize the wordlist,
it just computes the number of lines (words) that the wordlist holds:
The 'findstring /N "^^" wordlist' Searches for the "start of line" and passes the numbered lines with the format "N: text"
to the command line parser (CLP).

Okay, this 1st For statement works for a 2nd dic (2865 lines), it seems the first one was too big.

Copying sub content of :build_filename into a test file and running that, the above For statement runs.
Then I receive a Syntax error, as soon as it comes to

Code: Select all

For /l %%a In (0, 1, !num_namewords!) Do (
   Set/a "rand_line=!RANDOM! %% !num_words!"
   (

presumably. (I'm unsure about the Set/p "rand_word=" line, too.)

[penpen]

Okay, this 1st For statement works for a 2nd dic (2865 lines), it seems the first one was too big.

This sounded curious, so i've tested it with a 2 GB dictionary: It takes some time, but it works.
So the size is not the problem there, but it might give problems if the file is not in ANSI code with MS-DOS text format.

Copying sub content of :build_filename into a test file and running that, the above For statement runs.
Then I receive a Syntax error, as soon as it comes to

Code: Select all

For /l %%a In (0, 1, !num_namewords!) Do (
   Set/a "rand_line=!RANDOM! %% !num_words!"
   (

presumably. (I'm unsure about the Set/p "rand_word=" line, too.)

You should post the sytax error, as i can't say what might have gone wrong.
You should rem "@echo off" out and post some lines prior the error, too.
The only error i can imagine in the set/a line could be a "missing operand",
or a "divide by 0" error.
That might mean, that num_words is not initiated correctly
You may then display its content using an "echo(num_words=!num_words!", prior to the error line, please post it, too.

Beside:

Code: Select all

      (
         for /L %%b in (1, 1, !rand_line!) do (
            set "rand_word="
            set /P "rand_word="
         )
      ) < "!wordlist!"

The set /P reads the input from STDIN to the variable "rand_word" in this case.
The STDIN is redirected to the file "!wordlist!".
The loop assigns the first !rand_line! lines of the wordlist to the rand_word,
so that after the loop rand_word contains the !rand_line!. line/word.

Btw: I've tested the above code with this wordlist

Code: Select all

that
is
a
test

It runs without any errors, so i wonder why it doesn't run on your system.

penpen

[noprogrammer]

It turned out to be the first dic file I used. Coming from Ubuntu it had the wrong line breaks (LF instead of CRLF).
After fixing this the script ran fine. (However, it's a good idea to use small wordlists with <20,000 entries.)

It's interesting to compare bash and cmd (masterly done by penpen) in terms of programming...

I have to admit that reading shell programs is easier for me than "advanced" cmd scripts and I haven't even done that as much.
The Set/p trick is a good one, and you were right about everything, it's just Cmd throwing stupid error messages at times.

So last, not least: Thanks a bunch for your invaluable help!
It was a great lesson for me.