Optimizing These Loops

[Samir]

I have the current segment of code:

Code: Select all

:REPORTSTART
FOR %%f IN (DAY*.RPT) DO (
SET filedatetime=%%~tf
REM ECHO !filedatetime!
IF "!filedatetime:~0,10!" == "%currentDate%" SET DAY=%%f
REM ECHO !DAY!
)

FOR %%f IN (PLU*.RPT) DO (
SET filedatetime=%%~tf
REM ECHO !filedatetime!
IF "!filedatetime:~0,10!" == "%currentDate%" SET PLU=%%f
REM ECHO !PLU!
)

FOR %%f IN (SKU*.RPT) DO (
SET filedatetime=%%~tf
REM ECHO !filedatetime!
IF "!filedatetime:~0,10!" == "%currentDate%" SET SKU=%%f
REM ECHO !SKU!
)

ECHO !DAY!
ECHO !PLU!
ECHO !SKU!

What it does is search files of a certain type for a certain date (in the format mm/dd/yyyy) and then puts the filename into a variable if it matches. All the files are in the same place, so it's looping through these files three times to get the three file names. (There is only one of each of these files per day.)

What I wanted to see is if there was a way to optimize the amount of time spent in looping through the files by just looping through all of them once (*.RPT) and then assigning each variable based on what type of file is found on that date.

Some additional information

• The number of files the current implementation has to go through is <300 (for each loop)
• The total number of files (*.RPT) is over 3400

Possible?

[aGerman]

Does %currentDate% contain the current date?

[Samir]

Does %currentDate% contain the current date?

It can, but it is usually a day or two before, and could be one in the past few months from current. It is always in the format of MM/DD/YYYY.

[aGerman]

I see. Not sure if that would increase the performance but at least something you could try

Code: Select all

FOR %%f IN (*.RPT) DO (
  SET "filedatetime=%%~tf"
  SET "file=%%f"
  IF "!filedatetime:~0,10!"=="%currentDate%" (
    IF /I "!file:~,3!"=="DAY" (
      SET "DAY=%%f"
    ) ELSE IF /I "!file:~,3!"=="PLU" (
      SET "PLU=%%f"
    ) ELSE IF /I "!file:~,3!"=="SKU" (
      SET "SKU=%%f"
    )
  )
)

[Samir]

I see. Not sure if that would increase the performance but at least something you could try

Code: Select all

FOR %%f IN (*.RPT) DO (
  SET "filedatetime=%%~tf"
  SET "file=%%f"
  IF "!filedatetime:~0,10!"=="%currentDate%" (
    IF /I "!file:~,3!"=="DAY" (
      SET "DAY=%%f"
    ) ELSE IF /I "!file:~,3!"=="PLU" (
      SET "PLU=%%f"
    ) ELSE IF /I "!file:~,3!"=="SKU" (
      SET "SKU=%%f"
    )
  )
)

The more I've thought about performance, I think this is why I originally coded it in separate loops. I think the only way a singular loop might be faster is if it stopped working once all three files were found.

But then the same methodology could be applied to the three individual loops to speed them up as well.

The only reason I'm focusing on this is because this seems to be the biggest area that this particular batch spends time. If there was a way to significantly speed up the execution, then the entire batch would be faster.

[aGerman]

I think the only way a singular loop might be faster is if it stopped working once all three files were found.

You could jump out.

Code: Select all

SET "DAY="&SET "PLU="&SET "SKU="
FOR %%f IN (*.RPT) DO (
  SET "filedatetime=%%~tf"
  SET "file=%%f"
  IF "!filedatetime:~0,10!"=="%currentDate%" (
    IF /I "!file:~,3!"=="DAY" (
      SET "DAY=%%f"
    ) ELSE IF /I "!file:~,3!"=="PLU" (
      SET "PLU=%%f"
    ) ELSE IF /I "!file:~,3!"=="SKU" (
      SET "SKU=%%f"
    )
  )
  IF DEFINED DAY IF DEFINED PLU IF DEFINED SKU GOTO EXITLOOP
)
:EXITLOOP

[Squashman]

Maybe Robocopy would be faster at finding them.

[Samir]

Maybe Robocopy would be faster at finding them.

On the systems where this executes, it's not installed. That was actually my first inclination. Or maybe outputting the directory to a file if that can be searched faster.

[penpen]

I don't know if the following is much faster, but you could try to avoid using environment variables (instead this uses a tempfile; not tested):

Code: Select all

@echo off
setlocal
:: ... set current date ... 

>list.temp (
   for %%f in ("DAY*.RPT" "PLU*.RPT" "SKU*.RPT" "*.txt") do echo(%%~tf %%~nx
)
for /F "token=2" %%f in ('findstr "^%currentdate%" "list.tmp"') do (
   for /F "token=1 delims=YU" %%t in ("%%~f") do (
      if        "DA" == "%%~t" ( SET DAY=%%f
      ) else if "PL" == "%%~t" ( SET PLU=%%f
      ) else if "SK" == "%%~t" ( SET SKU=%%f
      )
   )
)
del "list.temp"
endlocal
goto :eof

The first loop may be replaced by a dir command (may be faster); in that case you probably have to adjust the tokens in the other loops.


penpen

[Squashman]

Maybe Robocopy would be faster at finding them.

On the systems where this executes, it's not installed. That was actually my first inclination. Or maybe outputting the directory to a file if that can be searched faster.

Ugh!!! You realize there is no support for Windows XP anymore and you can install Robocopy to Windows XP.
Regardless you are way behind the curve:
Windows 10
Windows 8.1
Windows 8
Windows 7
Windows Vista

[mcnd]

If your system has short names enabled, then you could try this

Code: Select all

@echo off
    setlocal enableextensions enabledelayedexpansion
    
    set "currentdate=11/03/2015"
    
    for /f "tokens=1,2 eol=|" %%a in ('
        dir *.rpt /a-d /-n ^| findstr /c:"%currentdate%"
    ') do for %%c in ("%%~a.%%~b?") do (
        set "filetype=%%~nc"
        set "_!filetype:~0,3!=%%~nxc"
    )

    echo DAY: %_day%
    echo PLU: %_plu%
    echo SKU: %_sku%

[Samir]

Thank you for the many solutions. I'll parse through them when I get a chance.

Maybe Robocopy would be faster at finding them.

On the systems where this executes, it's not installed. That was actually my first inclination. Or maybe outputting the directory to a file if that can be searched faster.

Ugh!!! You realize there is no support for Windows XP anymore and you can install Robocopy to Windows XP.
Regardless you are way behind the curve:
Windows 10
Windows 8.1
Windows 8
Windows 7
Windows Vista

Kinda ironic when you're telling me about being 'behind' and we're essentially talking on a site about DOS batch files--an operating system that's been extinct now since windows 98.

[Squashman]

Kinda ironic when you're telling me about being 'behind' and we're essentially talking on a site about DOS batch files--an operating system that's been extinct now since windows 98.

You are incorrect. We are not talking about DOS batch files. We are talking about Windows (NT) batch files which are the equivalent of using a shell in any Unix or Linux environment. Majority of batch file we write today would NEVER run on DOS.

[Samir]

Kinda ironic when you're telling me about being 'behind' and we're essentially talking on a site about DOS batch files--an operating system that's been extinct now since windows 98.

You are incorrect. We are not talking about DOS batch files. We are talking about Windows (NT) batch files which are the equivalent of using a shell in any Unix or Linux environment. Majority of batch file we write today would NEVER run on DOS.

The name of the site is "DOStips". Yes, we're working with the NT-based command.com equivalent, but it's still the same antiquated language that's been around since the late 1980s. Many of the bats you may write won't work in traditional DOS, but I try to stick to normal DOS functions unless I have something more complicated to work on (like the above code).

I'm not worried about what the rest of the world is running on their desktop. I have something that works and that's all that matters to me. I could care less how 'old' it is. I'm not trying to keep up with the Jones's; I'm just trying to get the job done.

[Samir]

I think the only way a singular loop might be faster is if it stopped working once all three files were found.

You could jump out.

Code: Select all

SET "DAY="&SET "PLU="&SET "SKU="
FOR %%f IN (*.RPT) DO (
  SET "filedatetime=%%~tf"
  SET "file=%%f"
  IF "!filedatetime:~0,10!"=="%currentDate%" (
    IF /I "!file:~,3!"=="DAY" (
      SET "DAY=%%f"
    ) ELSE IF /I "!file:~,3!"=="PLU" (
      SET "PLU=%%f"
    ) ELSE IF /I "!file:~,3!"=="SKU" (
      SET "SKU=%%f"
    )
  )
  IF DEFINED DAY IF DEFINED PLU IF DEFINED SKU GOTO EXITLOOP
)
:EXITLOOP

Good point. I should probably also try to exit the loops in my original code and see how that helps with speed.