View unanswered posts | View active topics It is currently 24 Oct 2014 02:30



Post new topic Reply to topic  [ 20 posts ]  Go to page 1, 2  Next
New bug in FOR command? 
Author Message
Expert

Joined: 06 Dec 2011 22:15
Posts: 734
Location: México City, México
Post New bug in FOR command?
I found a strange bug in FOR command at this SO topic. I thought that before a FOR command start the iterations, it saves the status that controlls such iterations in order to avoid endless loops. However, in this case the execution of commands in the FOR body modify the limits of the iterations, but just for one more time!. Here it is:
Code:
C:>dir /b
monsieur_lazhar_.jpg
monsieur_lazhar__ver2.jpg
test.bat

C:>type test.bat
@echo off

   rem Rename "*.jpg" files to 'Poster 1', 'Poster 2', 'Poster 3' ...
   set i=0
   for %%b in (*.jpg) do (
      set /A i+=1
      ren "%%b" "Poster !i!%%~Xb"
      echo File "%%b" renamed to "Poster !i!%%~Xb"
   )

C:>test
File "monsieur_lazhar_.jpg" renamed to "Poster 1.jpg"
File "monsieur_lazhar__ver2.jpg" renamed to "Poster 2.jpg"
File "Poster 1.jpg" renamed to "Poster 3.jpg"

C:>dir /b
Poster 2.jpg
Poster 3.jpg
test.bat
Is this bug documented? Is there a direct way to avoid it? I want to avoid an ugly patch at end of the FOR: :?
Code:
if not exist "Poster 1.jpg" ren "Poster %i%.jpg" "Poster 1.jpg"


Antonio


26 Dec 2012 16:34
Profile

Joined: 26 Oct 2011 22:38
Posts: 1007
Location: Egypt
Post Re: New bug in FOR command?
You can use:
Code:
For /F "delims=" %%b in ('DIR /B *.jpg') Do (
instead.

But it seems that it do what you need if you run the code twice :o , so you can do something like that:
Code:
@echo off
setlocal enabledelayedexpansion
rem Rename "*.jpg" files to 'Poster 1', 'Poster 2', 'Poster 3' ...
:loop
set i=0
set i2=0
for %%b in (*.jpg) do (
    set /A i+=1
    set /A i2+=1
    ren "%%b" "Poster !i!%%~Xb"
    echo File "%%b" renamed to "Poster !i!%%~Xb"
  )
if "!i2!" neq "3" goto :loop
pause

!i2! is for counting how many times the for loop was executed, we need it to run twice so we set the last if to go to loop as it is not equal to 3

or just create a file with the original file names and use it to rename them, but that will require more that one for loop


26 Dec 2012 18:29
Profile

Joined: 26 Oct 2011 22:38
Posts: 1007
Location: Egypt
Post Re: New bug in FOR command?
another idea, the problem as you said is that it doesn't keep track, so instead of storing the original file names in a file, we can store them in a variable (memory), then iterate through that variable and name that files one after the other.
Code:
@echo off
setlocal enabledelayedexpansion

:: Look here, this 2 lines
set "x="
for %%a in (*.jpg) do set "x=!x!%%a "

rem Rename "*.jpg" files to 'Poster 1', 'Poster 2', 'Poster 3' ...
set i=0
for %%b in (!x!) do (
    set /A i+=1
    ren "%%b" "Poster !i!%%~Xb"
    echo File "%%b" renamed to "Poster !i!%%~Xb"
  )
pause

the !x! variable will hold the names of the files (we append the file names to it) like this
Quote:
file1.jpg file2.jpg file3.jpg

then in your code, the for command will iterate throw these names as they are seprated by a space ( a file-set )

Edit:
BUT it won't work if the files has spaces in it's names :(

Edit: again :D
I found a work arround, instead of :
Quote:
for %%a in (*.jpg) do set "x=!x!%%a "

it should be like this:
Quote:
for %%a in (*.jpg) do set "x=!x!^"%%a^" "
all the file names will be qouted, whether it has space or not it will always be qouted


26 Dec 2012 18:40
Profile

Joined: 10 Feb 2012 02:20
Posts: 4321
Post Re: New bug in FOR command?
Aacini wrote:
I found a strange bug in FOR command


It's been a bug since msdos day, and it can affect more than one file.

using for /f with a dir command is the way I would get around it too.


26 Dec 2012 20:30
Profile
Expert

Joined: 13 Jan 2012 21:24
Posts: 441
Post Re: New bug in FOR command?
Aacini wrote:
I found a strange bug in FOR command

Seems to be in the same family as http://www.dostips.com/forum/viewtopic.php?p=22897#p22897. Don't know that I'd call it a "bug", since there is no "expected" behavior mandated or guaranteed in any docs I've seen about what "should" happen once the code in the loop modifies the fileset the loop itself works on. This goes all the way from batch 'for' loops down to the underlying system level FindFirst/NextFile APIs.

foxidrive wrote:
using for /f with a dir command is the way I would get around it too

That might get around the case in point here, indeed (at the expense of unicode support, as noted in the other thread). Still not foolproof against other programs outside the batch deleting/renaming files while 'dir /b' executes - but then one would have to define what the "expected outcome" would be, first.

Liviu


26 Dec 2012 22:21
Profile
Expert

Joined: 16 May 2011 08:21
Posts: 1262
Location: Flanders_(Belgium)
Post Re: New bug in FOR command?
'
I agree with Liviu,

In java an exception is thrown when I try to change the object I am enumerating.

Now I know what to watch out for, thanks for posting this Aacini, :)


27 Dec 2012 01:20
Profile WWW

Joined: 10 Feb 2012 02:20
Posts: 4321
Post Re: New bug in FOR command?
Liviu wrote:
Still not foolproof against other programs outside the batch deleting/renaming files while 'dir /b' executes


*NO* executable can survive that kind of action while it is running - unless it is actually written into the program for that scenario.

FWIW It's certainly not a new bug as I was aware of it in the dreaded days of MSDOS. The fact that it still hasn't been fixed speaks loudly.


27 Dec 2012 01:47
Profile
Expert

Joined: 12 Feb 2011 21:02
Posts: 1232
Location: United States (east coast)
Post Re: New bug in FOR command?
As others have said, this is a long standng "feature" of FOR, FOR /D, and FOR /R.

Recently I was fooling around with the feature to see if I could create an "infinite" directory based FOR loop. I succeeded, but it is slow :twisted:

The following loop continues until %RANDOM% generates 0.
Code:
@echo off
setlocal enableDelayedExpansion

:: initialize
md dirLoop
pushd dirloop
set "pad=         "
for /l %%N in (101 1 800) do copy nul "%pad%1.%%N" >nul

set /a n=0
for %%F in (*) do (
  set /a n+=1
  2>nul set /a "1/(n%%100)" || echo !n!
  if !random!==0 (
    echo(  exiting at !time!
    goto :break
  )
  set /a new=%%~nF+1
  set "new=%pad%!new!"
  ren "%%F" "!new:~-10!%%~xF" >nul
)

:break
popd
rd /s /q dirLoop
echo(completed at !time!
echo iterations: !n!

The longer the loop runs, the longer it takes to break out. I'm not sure, but I suspect it may have something to do with temporarily preserving a list of all names that have already been iterated. I suspect it needs to do this to make sure that the same file is not iterated twice due to the short 8.3 name. It could take time to clear out the temporary list from memory. But this is all guess work.


Dave Benham


27 Dec 2012 15:37
Profile

Joined: 20 Aug 2010 13:57
Posts: 370
Location: Chile
Post Re: New bug in FOR command?
I understand the problem. For /f and For works different.

For example if you use:
For /F %%a in ('Dir /b *.jpg')
this execute the command and save all the text output. Then you have a list static.

When you use:
For %%a in (*.jpg)
this not keep in memory a list, it look in the hard disk every time, then it have a dinamic list.

When you use ren you affect this dinamic list.
I test put the renamed files in other directory. Check this:

Code:

C:\Users\Carlos\Desktop\test>dir /b
monsieur_lazhar_.jpg
monsieur_lazhar__ver2.jpg
test.bat

C:\Users\Carlos\Desktop\test>type test.bat
@echo off
   md images
   setlocal enabledelayedexpansion

   rem Rename "*.jpg" files to 'Poster 1', 'Poster 2', 'Poster 3' ...
   set i=0
   for %%b in (*.jpg) do (
      set /A i+=1
      move "%%b" "images\Poster !i!%%~Xb"
      echo File "%%b" renamed to "images\Poster !i!%%~Xb"
   )

C:\Users\Carlos\Desktop\test>test.bat
Se han movido         1 archivos.
File "monsieur_lazhar_.jpg" renamed to "images\Poster 1.jpg"
Se han movido         1 archivos.
File "monsieur_lazhar__ver2.jpg" renamed to "images\Poster 2.jpg"

C:\Users\Carlos\Desktop\test>dir /b
images
test.bat

C:\Users\Carlos\Desktop\test>dir /b images
Poster 1.jpg
Poster 2.jpg

C:\Users\Carlos\Desktop\test>


27 Dec 2012 17:34
Profile WWW

Joined: 20 Aug 2010 13:57
Posts: 370
Location: Chile
Post Re: New bug in FOR command?
The difference is similar to this:

You open a file for read. You read entirely and keep in memory, then if other application do a change in the file you are not reading from the file, you are reading from you buffer that is a copy of the file when you read it.

Other way is open the file for read and write, then you read the file byte to byte, and in the end of the file you add other character, then you read a new character out of the original file.

The first is similar to For /F and the second is similar to For and Ren together. All this applied to a filesystem.


27 Dec 2012 17:43
Profile WWW
Expert

Joined: 13 Jan 2012 21:24
Posts: 441
Post Re: New bug in FOR command?
foxidrive wrote:
*NO* executable can survive that kind of action while it is running - unless it is actually written into the program for that scenario.
Correct. And such "the program" can't be done in batch language, as far as I am aware of.

foxidrive wrote:
FWIW It's certainly not a new bug as I was aware of it in the dreaded days of MSDOS. The fact that it still hasn't been fixed speaks loudly.
Still not sure why call it a "bug", since I can't find any reference documentation to build expectations on as to what's supposed to happen in such cases . Also not sure what you'd propose as a "fix", since it's not always as clear cut. In the general case, it's simply not possible to do an arbitrary rename in one pass. Assume for example that Aacini's original 'dir /b' returned
Code:
C:>dir /b images
Poster 2.jpg
Poster 1.jpg
(which is possible if C:\images is on a file system that returns unsorted names e.g. FAT). Then it would take non-trivial code to rename the "first" file to "Poster 1" then the "second" one to "Poster 2".

Liviu


28 Dec 2012 01:35
Profile

Joined: 10 Feb 2012 02:20
Posts: 4321
Post Re: New bug in FOR command?
Liviu wrote:
Assume for example that Aacini's original 'dir /b' returned
Code:
C:>dir /b images
Poster 2.jpg
Poster 1.jpg
(which is possible if C:\images is on a file system that returns unsorted names e.g. FAT). Then it would take non-trivial code to rename the "first" file to "Poster 1" then the "second" one to "Poster 2".


dir /b /o:n would fix it. :)


28 Dec 2012 02:45
Profile
Expert

Joined: 13 Jan 2012 21:24
Posts: 441
Post Re: New bug in FOR command?
foxidrive wrote:
dir /b /o:n would fix it. :)

Only on lucky days ;-)
Code:
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\>dir /b /on images
Poster 1.jpg
Poster 10.jpg
Poster 2.jpg


28 Dec 2012 10:30
Profile
Expert

Joined: 12 Feb 2011 21:02
Posts: 1232
Location: United States (east coast)
Post Re: New bug in FOR command?
Liviu wrote:
foxidrive wrote:
dir /b /o:n would fix it. :)

Only on lucky days ;-)
Code:
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\>dir /b /on images
Poster 1.jpg
Poster 10.jpg
Poster 2.jpg


True, /o:n may not give the desired results, but at least the results will be determinate. A given input set of file names will always produce the same result, regardless how, or in what order, the original files were created.


Dave Benham


28 Dec 2012 10:49
Profile
Expert

Joined: 06 Dec 2011 22:15
Posts: 734
Location: México City, México
Post Re: New bug in FOR command?
Ok. The core of this problem is clearly defined now: a FOR-set that remain static while the FOR is executed vs. a FOR-set that change because a command in DO part. Note that this difference is NOT necessarily related to FOR vs. FOR /F; for example:
Code:
for /F "delims=" %%a in (thefile.txt) do (
   echo %%a >> thefile.txt
)

I think previous FOR may read lines created by itself if the file size before the FOR is larger than the buffer used to read the file. The possible explanations of FOR behavior may give us the way to understand it and plan methods to avoid the problem. However, I want to emphasize a particular point of this problem.

Liviu wrote:
Don't know that I'd call it a "bug", since there is no "expected" behavior mandated or guaranteed in any docs I've seen about what "should" happen once the code in the loop modifies the fileset the loop itself works on...

...but then one would have to define what the "expected outcome" would be, first.

Liviu

I think the "expected behavior" should be the same in any case. In the original example I used to start this topic, if FOR-command would check that this condition not happen, then "Poster 1.jpg" file would not be renamed to "Poster 3.jpg". On the other hand, if FOR-command does NOT have this checking, then it would enter into an endless rename loop ("Poster 2" to "Poster 4", "Poster 3" to "Poster 5", etc). Why FOR-command rename an already renamed file just one time? This is what I call BUG!

Antonio


28 Dec 2012 18:47
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 20 posts ]  Go to page 1, 2  Next


Who is online

Users browsing this forum: Bing [Bot], Google [Bot], Yahoo [Bot] and 16 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Forum style by Vjacheslav Trushkin for Free Forums/DivisionCore.