Reverse Parsing

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Post Reply
Message
Author
shadeclan
Posts: 61
Joined: 02 Jun 2011 11:29
Location: USA - Somewhere between Albany NY and Bennington VT

Reverse Parsing

#1 Post by shadeclan » 16 Nov 2011 14:23

I'm just full of questions today!

Is it possible to parse a file starting at the end instead of the beginning, similar to using a negative number when iterating a range of values in a FOR loop?

Hi, Ed! :D

Ed Dyreen
Expert
Posts: 1569
Joined: 16 May 2011 08:21
Location: Flanders(Belgium)
Contact:

Re: Reverse Parsing

#2 Post by Ed Dyreen » 16 Nov 2011 15:00

'
In DOS that is not possible ( in assembly it is ), it is also not logical.
In any language you begin by enumerating the file
saving it to an array
reversing the array
writing out the reversed array to a file.

some files are simply too big though, then things get complicated :|
A solution would be to read the last line of file A and appending it to file B
Delete the last line of file A and repeat all previous steps until fileA is empty.

OJBakker
Expert
Posts: 31
Joined: 12 Aug 2011 13:57

Re: Reverse Parsing

#3 Post by OJBakker » 16 Nov 2011 15:38

If you enumerate the file with leading zeroes the reverse ordering can be done with Sort /R

!k
Expert
Posts: 378
Joined: 17 Oct 2009 08:30
Location: Russia

Re: Reverse Parsing

#4 Post by !k » 16 Nov 2011 15:46


shadeclan
Posts: 61
Joined: 02 Jun 2011 11:29
Location: USA - Somewhere between Albany NY and Bennington VT

Re: Reverse Parsing

#5 Post by shadeclan » 17 Nov 2011 07:44

Ed Dyreen wrote:In DOS that is not possible ( in assembly it is ), it is also not logical.
Good morning, Mr. Spock. :twisted:

I'm not sure why you'd say that, Ed. After all, in the often-used phrase:

Code: Select all

@echo.some text>> SomeLogFile.txt
DOS must know where the end of the file in order to append the line to it.

I have a running log-file. Rather than start at the beginning and go to the end, where the pertinent text is, I was thinking "why not start at the end and go backwards? Now, I do have a work-around - I can write the log to a temp file and work with that, then append it to the log-file at the end of the program. I just thought that was a little clunky, is all.

shadeclan
Posts: 61
Joined: 02 Jun 2011 11:29
Location: USA - Somewhere between Albany NY and Bennington VT

Re: Reverse Parsing

#6 Post by shadeclan » 17 Nov 2011 07:51

OJBakker wrote:If you enumerate the file with leading zeroes the reverse ordering can be done with Sort /R
That's an interesting idea but I think it will eat up more processor time and memory than my work-around.

orange_batch
Expert
Posts: 442
Joined: 01 Aug 2010 17:13
Location: Canadian Pacific
Contact:

Re: Reverse Parsing

#7 Post by orange_batch » 01 Dec 2011 20:55

Ed Dyreen wrote:In any language you begin by enumerating the file
saving it to an array
reversing the array
writing out the reversed array to a file.

This is the solution for DOS as well. You could also do this using a call to VBScript (which is likely more stable/efficient as it has some built-in ability to process arrays).

Code: Select all

@echo off&setlocal enabledelayedexpansion

set "in_file=in.txt"
set "out_file=out.txt"

for /f "delims=] tokens=1*" %%a in ('type "%in_file%"^|find /v /n ""') do (
set /a counter+=1
set "array!counter!=%%b"
)

for /l %%a in (%counter%,-1,1) do echo:!array%%a!>>"%out_file%"

Keeps blank lines. Beware possible special character bugs as usual with DOS.

This version will reverse the input file (no output file):

Code: Select all

@echo off&setlocal enabledelayedexpansion

set "file=mytext.txt"

for /f "delims=] tokens=1*" %%a in ('type "%file%"^|find /v /n ""^&type nul^>"%file%"') do (
set /a counter+=1
set "array!counter!=%%b"
)

for /l %%a in (%counter%,-1,1) do echo:!array%%a!>>"%file%"

shadeclan
Posts: 61
Joined: 02 Jun 2011 11:29
Location: USA - Somewhere between Albany NY and Bennington VT

Re: Reverse Parsing

#8 Post by shadeclan » 02 Dec 2011 08:25

orange_batch wrote:... This is the solution for DOS as well ...
Interesting solution - I will need to file this in my notes in case I need to reverse the order of lines in a file - but it's still less efficient in this application where the goal is to read a few lines at the end of a large file without going through the older data ahead of it. My temporary log file solution, where I keep the info from the current run for searching, then append it to the end of the permanent log is still the more efficient solution.

orange_batch
Expert
Posts: 442
Joined: 01 Aug 2010 17:13
Location: Canadian Pacific
Contact:

Re: Reverse Parsing

#9 Post by orange_batch » 02 Dec 2011 17:24

Yes it would be of course. If the permanent log is a fixed length, you could use the skip option of for. If not, you could always count the lines to a file, read that for your skip, and with each appended log, count the lines of that log and add its length to that file.

dbenham
Expert
Posts: 2383
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Reverse Parsing

#10 Post by dbenham » 03 Dec 2011 20:54

It's kind of funny that I just answered the same question (reversing the order of lines in a file) on StackOverflow: http://stackoverflow.com/a/8369403/1012053

My 2nd solution is basically the same as orange_batch's except it should not have any problem with special characters. The main improvement over orange_batch is the support for ! within the text file.

Here is a routine that will echo the last n lines of a file:

Code: Select all

@echo off
:tail  FilePath  LineCount
::
:: Prints the last LineCount lines of file FilePath
::
setlocal disableDelayedExpansion
set file="%~1"
set /a "cnt=%~2"
for /f %%n in ('type %file% ^| find /c /v ""') do set /a "skip=%%n-cnt"
if %skip% leq 0 (set "skip=") else set "skip=skip=%skip%"
for /f "%skip%delims=" %%a in ('type %file% ^| find /n /v ""') do (
  set "ln=%%a"
  setlocal enableDelayedExpansion
  echo(!ln:*]=!
  endlocal
)
exit /b


Dave Benham

dbenham
Expert
Posts: 2383
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Reverse Parsing

#11 Post by dbenham » 08 Dec 2011 12:00

I did not realize MORE does not pause if the output is redirected to a file until I saw this post by !k

This is a MUCH faster version of tail - very noticeable difference for large files.

Code: Select all

@echo off
:tail  FilePath  LineCount
::
:: Prints the last LineCount lines of file FilePath
::
setlocal disableDelayedExpansion
set file="%~1"
set /a "cnt=%~2"
set tempFile="%temp%\tail%random%.txt"
for /f %%n in ('^<%file% find /c /v ""') do set /a "skip=%%n-cnt"
if %skip% leq 0 set "skip=0"
more +%skip% %file% >%tempFile%
type %tempFile%
del %tempFile%
exit /b


Dave Benham

Post Reply