Page 1 of 1

slice.cmd

Posted: 26 Jan 2014 15:19
by carlos
Hello. I found this in my archive. Maybe it can be useful for someone.

It can print a text file from a specific line to specific line.

Usage:
Usage: slice.cmd filename [<int>start] [<int>end]


slice.cmd

Code: Select all

@Echo Off
:: Usage: slice.cmd filename [<int>start] [<int>end]
:: Note: if end is 0 then prints until the end of file
Setlocal EnableDelayedExpansion
Set "f=%~f1"
Set /A "b=%~2,e=%~3,#=0" 2>Nul
If !b! Leq 0 Set "b=1"
If !e! Leq 0 Set "e=2147483647"
For /f "delims=" %%$ In (
'^(Type "!f!" ^&Echo:^)^|Findstr.exe /n "^^"') Do (
Set /A "#=#+1"
Set "$=%%$"
Set "$=!$:*:=!"
If !#! Geq %b% Echo(!$!
If !#! Geq %e% Goto :Eof
)
Goto:Eof


Re: slice.cmd

Posted: 29 Jan 2014 16:34
by Sponge Belly
Hello Carlos! :-)

Slice was one of the first Batch programs I tried to write. I couldn’t output the lines verbatim so I asked for help on the alt.msdos.batch.nt newsgroup and a long, rambling thread unfolded. Your program was one of several submitted.

Now that I have a better idea of what I’m doing, I can optimise some of your code for a change! :lol:

Code: Select all

@Echo Off & Setlocal EnableExtensions DisableDelayedExpansion
Set /A b=%~2+0,e=%~3+0,#=0 2>Nul
If %b% Leq 0 Set "b=1"
If %e% Leq 0 Set "e=2147483647"

For /f "delims=" %%$ In ('Findstr /n "^" "%~dpf1"') Do (
Set /A #+=1 & Set "$=%%$"
Setlocal EnableDelayedExpansion
Set "$=!$:*:=!"
If !#! Geq %b% Echo(!$!
If !#! Geq %e% (Endlocal & Goto End)
Endlocal
)

:End
Endlocal & Goto:Eof


The main difference is the use of the expansion-toggling technique as explained by Jeb in SO: How to Read a File (third answer).

Edit: Implemented suggestion made by Carlos and program now exits more cleanly when last line is reached.

- SB

Re: slice.cmd

Posted: 29 Jan 2014 19:06
by carlos
@Sponge Belly: I test your code and it needs a simple fix. Use this:

Code: Select all

Set /A b=%~2+0,e=%~3+0,#=0 2>Nul

I sum 0 for ensure that the b and e would defined, else if it not defined this cause fail If Leq 0.

I think that your code the improvement is the not use of type command. Type command break on 26 ascii character. But because slice is for text files it would not be a problem.

But you turn off the delayed expansion for avoid that text lines with text like this !windir! would be interpreted and printed C:\Windows
My code have this problem.

Your code fix this problem.

Thanks for share.

Re: slice.cmd

Posted: 31 Jan 2014 16:02
by Sponge Belly
Hi Carlos,

I updated my code with your suggestion. And the program now exits more cleanly when the last line of output is reached.

I’ve often thought of writing a better version with features like inverted ranges, numbered lines with padded zeros, and support for long lines, etc. But to implement some of these items on my wishlist would be next to impossible using pure Batch. And even if I did, the end result would be painfully slow.

Or so I thought up until recently. With the discovery of how to write binary using only Batch, Slice 2.0 just might escape from Development Limbo after all. :-)

- SB

Re: slice.cmd

Posted: 03 Feb 2014 12:00
by einstein1969
Hi Sponge Belly and Carlos,

I have tested on 700kb about and there are about 7 seconds of delay.
With a file of 1.2MB the time is about 20 seconds.
For higher size the time increase exponentially.

I have a 16Mb text file and the time is over five minutes. I have break before finish.

This version is more fast for bigger files. Only 1second for 16MB file

The code not delete temporary files.

Code: Select all

@Echo Off
:: Usage: slice.cmd filename [<int>start] [<int>end]
:: Note: if end is 0 then prints until the end of file
:: Add trick for bigger files. Tested on Win7 32bit monocore

Setlocal EnableExtensions DisableDelayedExpansion
Set /A b=%~2+0,e=%~3+0,#=0 2>Nul
If %b% Leq 0 Set "b=1"
If %e% Leq 0 Set "e=2147483647"

set "tmp_file=%tmp%\slice_%random%_%random%.dat"
Findstr /n "^" "%~dpf1">%tmp_file%

rem For /f "delims=" %%$ In ('Findstr /n "^" "%~dpf1"') Do (
For /f "delims=" %%$ In (%tmp_file%) Do (
Set /A #+=1 & Set "$=%%$"
Setlocal EnableDelayedExpansion
Set "$=!$:*:=!"
If !#! Geq %b% Echo(!$!
If !#! Geq %e% (Endlocal & Goto End)
Endlocal
)

:End
Endlocal & Goto:Eof


einstein1969

Re: slice.cmd

Posted: 03 Feb 2014 15:52
by carlos
thanks einstein1969.
Please you can explain the trick.
Because you are handling bigger files.
I thinking how handle if the new lines are greater than 2147483647 (improbable) ?

Re: slice.cmd

Posted: 03 Feb 2014 17:00
by Squashman
I would be more worried about the file size then it having over 2 Billion lines.

Re: slice.cmd

Posted: 03 Feb 2014 18:01
by einstein1969
the trick is simple.

I'm using temporary file istead execute the findstr in the for /F

added:

Code: Select all

set "tmp_file=%tmp%\slice_%random%_%random%.dat"
Findstr /n "^" "%~dpf1">%tmp_file%


and substitute this

Code: Select all

For /f "delims=" %%$ In ('Findstr /n "^" "%~dpf1"') Do (


with:

Code: Select all

For /f "delims=" %%$ In (%tmp_file%) Do (


The for /f for bigger file is more slow if use the execute command. I think that the buffering is more expensive.

My file of 16MB has about 250.000 lines. (<< 2.147.483.647)

I'm studing for fast seek that is the other problem of slice.

maybe just use a simple "skip" in for.

this version has very fast seek

Code: Select all

@Echo Off
:: Usage: slice.cmd filename [<int>start] [<int>end]
:: Note: if end is 0 then prints until the end of file
:: Add trick for bigger files and fast seek

Setlocal EnableExtensions DisableDelayedExpansion
Set /A b=%~2+0,e=%~3+0,#=0 2>Nul
If %b% Leq 0 Set "b=1"
If %e% Leq 0 Set "e=2147483647"

rem using temporary file for faster execute on big file in size
set "tmp_file=%tmp%\slice_%random%_%random%.dat"
Findstr /n "^" "%~dpf1">%tmp_file%

rem fast seek
set /A #=%b%-1
For /f "skip=%#% delims=" %%$ In (%tmp_file%) Do (

Set /A #+=1 & Set "$=%%$"
Setlocal EnableDelayedExpansion
Set "$=!$:*:=!"
If !#! Geq %b% Echo(!$!
If !#! Geq %e% (Endlocal & Goto End)

rem debug on title
if !random! geq 32700 title !#!
Endlocal
)

:End
Endlocal & Goto:Eof


EDIT:
this trick can be used for reading binary(any content) files too!
In this case the limit of 2147483647 is more important.

I'm studing the trick of Sponge Belly in this post for truncate a file / split and this

einstein1969

Re: slice.cmd

Posted: 04 Feb 2014 00:17
by carlos
@einstein1969: skip less or equal to 0 fail.

instead of this:

Code: Select all

set /A #=%b%-1
For /f "skip=%#%


maybe some like this will work:

Code: Select all

set "skip="
if %b% gtr 1 set /A "skip=%b%-1"
if defined skip set "skip=skip=%skip%"
For /f "skip=%skip%

Re: slice.cmd

Posted: 04 Feb 2014 13:13
by einstein1969
carlos wrote:@einstein1969: skip less or equal to 0 fail.

instead of this:

Code: Select all

set /A #=%b%-1
For /f "skip=%#%


maybe some like this will work:

Code: Select all

set "skip="
if %b% gtr 1 set /A "skip=%b%-1"
if defined skip set "skip=skip=%skip%"
For /f "skip=%skip%


Thanks, I see.

now works. :)

einstein1969