quickly write a large file?

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Post Reply
Message
Author
taripo
Posts: 227
Joined: 01 Aug 2011 13:48

quickly write a large file?

#1 Post by taripo » 12 Dec 2017 03:07

Hi

How can I quickly write a large file where each line has the string "aaa"?

this method is slow

I let it run for about 20 seconds and stopped it 'cos it was slow and it wrote maybe 20,000 lines far from 200,000
C:\blah>for /L %f in (1,1,200000) do @echo aaa >>abigfile1.txt
I could write a program test.rb in eg ruby
i=0
while i < 200000 do
puts("aaa")
i+=1
end
and run ruby test.rb >bigfile

that runs fast, does the whole task within a second

But ruby is a third party program that not everybody would have installed.

i'm wondering if there is a way to do it without a third party program, that is as fast ore pretty fast like within <= 3 seconds.. That somebody can run and produce that file?

TIA

penpen
Expert
Posts: 1995
Joined: 23 Jun 2013 06:15
Location: Germany

Re: quickly write a large file?

#2 Post by penpen » 12 Dec 2017 04:25

First, you should avoid creating thousands of filestreams where one filestream suffices:

Code: Select all

(for /L %f in (1,1,200000) do @echo aaa) >>abigfile1.txt
Then you should use a batch file, because if you execute this command in "cmd.exe" you also change the title 400,002 times (in your case).

At last you should avoid many for loop iterations as possible.

So this batch file ("test.bat") might help you:

Code: Select all

@echo off
setlocal enableExtensions disableDelayedExpansion
echo %time%
for %%a in ("abigfile1.txt") do (
	>"%%~a" (for /L %%f in (1,1,3125) do echo aaa)
	for /L %%f in (1,1,6) do >>"%%~a" findstr "^" "%%~a"
)
echo %time%

endlocal
goto :eof
This batch takes round about 0.2 seconds on my system.

penpen

taripo
Posts: 227
Joined: 01 Aug 2011 13:48

Re: quickly write a large file?

#3 Post by taripo » 14 Dec 2017 16:58

thanks

Post Reply