Process files one by one

[karaziki]

Hope to have a light here.
I have a batch file to compress PNG files. When I drag and drop some files on batch, it starts to process all of them in the same time. And it becomes useless when I tried with hundreds of files (sure) :)

The main part is something like that (simplified)

Code: Select all

START "" /W truepng.exe /out "%~n1_out1.png" %1
START "" /W pngwolf.exe --in="%~n1_out1.png" --out="%~n1_out2.png"

Problem is the "%1" part.
I'm trying to make batch to process files with 4-file blocks. Thinking of 2 different aproach, and both fine.
1. Will process the first 4 file, after continue with other 4.
2. Or it will run maximum 4 "pngwolf.exe" instances in the same time (best option)

In my several tests, I started to try with a counter logic %1 after %2 after %n...
But even in that phase I stucked and couldn't create a counter system.
No need to ask, sure I'm newbie on batch coding. But even to be a good designer, looks like I need to learn more on batch.
Thanks for anyone will give a hand. Hope there is a solution that I can understand and adaptate.

All Bests
Evren

[ShadowThief]

Dude, I responded to your post on Stack Overflow almost 20 minutes ago.

[karaziki]

Thanks dude now I finished trying.
I can remove the /W flag only from pngwolf. Truepng had to h /W because pngwolf using truepng's output. And after removing the wait flag from pngwolf, actually not too much of things changed as expected.

[karaziki]

Actually even I'm not sure is there a way or not with batch files.
Just I hope and wait for a possibility to be able to manage instances.

For a possible bad news (like not possible to do) I already started to try rename actions under loop. I think at least I can solve it myself. Hope :)

[ShadowThief]

I edited my SO post and it should work now, unless truepng does something stupid like delete the folder after processes or something.

[karaziki]

Hell no, truepng is the most smart one I see so far. Works straight.
By the way I always found this forum more comfortable. Why not to write just over here?

And when it comes to reply;
The standart way of executing can not help since;
1. The process time of truepng can be seconds to several minutes. A fixed delay can not work. I had to handle each file with /W flag.

2. My reason to need a batch script is to be able to use with hundreds of files each time. Can not know how many for a fixed definition. This is somethin like "start the progress, go to sleep and pray for it to be finished until morning" :)

[ShadowThief]

Here is fine.

For some reason I was under the impression that the script would wait for the ping to finish, but I didn't take into account that one process that was running long could still be active (when I tested with pings of varying lengths, my testing had the long ping after the short ping, so my script was falsely running correctly - I discovered my error when I flipped the commands around).

My next move was to replace the ping with a pause, but

• it would have the same issue
• you mentioned that you have a tendency to go do other things while the code is running

Next up is going to be an attempt at a variation of http://stackoverflow.com/questions/1257 ... 8#12665498 that uses a counter variable to generate as many lock files as necessary. The overall idea of performing all of the truepng commands before the pngwolf commands is still going to work, though.

Also, I notice that you're optimizing an optimized png file. Is that really necessary? That seems like archiving an archive file - if you do it properly the first time, the second time gives you no added benefit.

[karaziki]

There is a reply out there I think following your last logic over lock files (I don't know how it works ??)
But I think your way will be a bit different to handle priority of truepng after pngwolf order. What I see so far is, seems like you love to deal with batch stuff :)

Anyway. And when it comes to double compression...
Actually not. There are tons of PNG compressors out there and all of them have their unique sides. Some handles quantization better, some does flawless dithering, some have great logic of palette ordering etc. etc. But the truepng and pngwolf couple handles nearly all what other can do.

Main logic is to benefit all their advanced parts. Important rule is to command correctly to not repeat an action twice. Like saying truepng you do just these, do not touch these points. After pngwolf you fix just these, do not alter (and loose time) with what truepng already did etc.

The actual code is like that in this moment.

Code: Select all

ECHO File Path  [ %fpath% ]
ECHO Exe  Path  [ %epath%bin ]
ECHO ____________________________________________________________
ECHO.

FOR %%A in (%*) do (

COPY /Y %%A "%fpath%\%%~nxA.bak" >NUL
bin\truepng.exe /f4 /g0 /a0 /l q=4 m=1 /i0 /md remove all /zc9 /zm9 /zs1 /quiet /y /out "%fpath%\%%~nxA" %%%A >NUL
timeout 1 >NUL
bin\pngwolf.exe --in="%fpath%\%%~nxA" --out="%fpath%\%%~nxA" --exclude-singles --exclude-heuristic --max-stagnate-time=2 --max-evaluations=1 --7zip-mpass=2 >NUL
)

Still checking over balance of speed and performance. And trying to minimize actions.

[ShadowThief]

If you like this, you should wait until somebody like Squashman, Aacini, or dbenham comes in here and corrects my code

Is the 4-at-a-time thing for speed issues or do the programs have a limit to how many instances can be run at a time? Because this will run truepng on all files, wait for all instances of true png to finish, and then run pngwolf on all the files output by truepng.

Code: Select all

:: Heavily utilizes dbenham's code from http://stackoverflow.com/questions/12577442/waiting-for-parallel-batch-scripts/12665498#12665498
@echo off
setlocal enabledelayedexpansion

set "lock=%temp%\wait%random%.lock"
set counter=0

:: Assign each parameter to a counter value and give it its own lock file
:: This way, the script does not continue until all processing is completed
for %%A in (%*) do (
   set /a counter+=1
   start "" cmd /c 9>"!lock!!counter!" truepng.exe /out "%%~nA_out1.png" %%A
)

:waitLoop
1>nul 2>nul ping -n 2 localhost
for /L %%A in (1,1,!counter!) do (
   ( rem
   ) 9>"!lock!%%A" || goto :waitLoop
) 2>nul

:: Delete the lock files
del "%lock%*"

:: Run pngwolf on the truepng output
for %%A in (%*) do start "" pngwolf.exe --in="%%~nA_out1.png" --out="%%~nA_out2.png"

[karaziki]

Edit: Forget to reply, 4 instance is about performance. Programs doesn't have limitation on it. Actually best aproach is to follow number of cores. I'm using 8 but think 4 is good because of 2 software will run for each file. Hope to have it work.
_________________________________________
While I started to give up, got replies ;)

I was over an another aproach and while thinking that it is simple and straight, got your reply. Looks like far straightforward. I'll test in a short while. And after test I will share the complete code with other variants available on internet.

PNG compression is a hard topic and I couldnt satisfied of samples on internet. I was on thousands of tries and tests for a while. With the batch code it will become useful in any manner. Don't want to be smart ass but I think I did the best results out there in alternatives with 2 software. Anyone interested must try, results surprising.

[foxidrive]

Irfanview has command line functions and PNG options.

You haven't described what you need to do - only that you want to process files and keep it to 4 of them at one time.

Dave Benham (dbenham at SO) has written solutions for that and also posts here. I seem to recall him posting them at SO, and I'm not sure if they are here too.

ShadowThief's solution may be fine too, I haven't tested Dave's or ShadowThief's as it's not a task I need to date.

[Squashman]

Dave has done some great stuff with process control. I am just a hack who moderates the forum. But I have always wanted to test out using the WAITFOR command to control process flow.

[foxidrive]

I have always wanted to test out using the WAITFOR command to control process flow.

I looked at the help for that when it was mentioned a while back - it isn't very clear on how it functions.

My motivation was low and I didn't look further to see how it works.

[Squashman]

I have tested it a few times with some of my automation but mostly the software I use is controlled by trigger files. WAITFOR can be used locally and between computers on the same network.

[karaziki]

@foxidrive
Thanks for your reply.
Irfan view is a great piece of software for casual usage. But when it comes to professional design it's way of encoding and compressing can not satisfy you.

@Squashman
I've seen the solution of Dave Benham but my knowledge is not enough to adaptate it to my script. My shame :)