Help with only downloading new files

[brandonrit]

All -

I am trying to make a batch file that only downloads the new files from an ftp site that are not in a local directory.. then use scheduler to run that bat file once ever 5 mins.

When i run this - it tries to download the files that are already there and overwrite them...

Where is my error.. i am pulling out my hair (what i have left) here...


Thanks!





Here is what i have ..

@Echo Off

REM -- Define File Filter, i.e. files with extension .txt
Set FindStrArgs=/E /C:".pdf"

REM -- Extract Ftp Script to create List of Files
Set "FtpCommand=ls"
Call:extractFileSection "[Ftp Script 1]" "-">"%temp%\%~n0.ftp"
Rem Notepad "%temp%\%~n0.ftp"

REM -- Execute Ftp Script, collect File Names
Set "FileList="
For /F "Delims=" %%A In ('"Ftp -v -i -s:"%temp%\%~n0.ftp"|Findstr %FindStrArgs%"') Do (
Call Set "FileList=%%FileList%% "%%A""
)

REM -- Extract Ftp Script to download files that don't exist in local folder
Set "FtpCommand=mget"
For %%A In (%FileList%) Do If Not Exist "%%~A" Call Set "FtpCommand=%%FtpCommand%% "%%~A""
Call:extractFileSection "[Ftp Script 1]" "-">"%temp%\%~n0.ftp"
Rem Notepad "%temp%\%~n0.ftp"

For %%A In (%FtpCommand%) Do Echo.%%A

REM -- Execute Ftp Script, download files
ftp -i -s:"%temp%\%~n0.ftp"
Del "%temp%\%~n0.ftp"
GOTO:EOF


:extractFileSection StartMark EndMark FileName -- extract a section of file that is defined by a start and end mark
:: -- [IN] StartMark - start mark, use '...:S' mark to allow variable substitution
:: -- [IN,OPT] EndMark - optional end mark, default is first empty line
:: -- [IN,OPT] FileName - optional source file, default is THIS file
:$created 20080219 :$changed 20100205 :$categories ReadFile
:$source http://www.dostips.com
SETLOCAL Disabledelayedexpansion
set "bmk=%~1"
set "emk=%~2"
set "src=%~3"
set "bExtr="
set "bSubs="
if "%src%"=="" set src=%~f0& rem if no source file then assume THIS file
for /f "tokens=1,* delims=]" %%A in ('find /n /v "" "%src%"') do (
if /i "%%B"=="%emk%" set "bExtr="&set "bSubs="
if defined bExtr if defined bSubs (call echo.%%B) ELSE (echo.%%B)
if /i "%%B"=="%bmk%" set "bExtr=Y"
if /i "%%B"=="%bmk%:S" set "bExtr=Y"&set "bSubs=Y"
)
EXIT /b


[Ftp Script 1]:S
!Title Connecting...
open ftp.directsmile.de
HPcloud
Hp2012

!Title Preparing...
cd public_html\rhombus
lcd c:\test\rhombus
binary
hash

!Title Processing... %FtpCommand%
%FtpCommand%

!Title Disconnecting...
disconnect
bye

[foxidrive]

I am trying to make a batch file that only downloads the new files from an ftp site that are not in a local directory.. then use scheduler to run that bat file once ever 5 mins.

When i run this - it tries to download the files that are already there and overwrite them...

REM -- Extract Ftp Script to download files that don't exist in local folder
Set "FtpCommand=mget"
For %%A In (%FileList%) Do echo "%%~A" &pause
Call:extractFileSection "[Ftp Script 1]" "-">"%temp%\%~n0.ftp"
Rem Notepad "%temp%\%~n0.ftp"

Try the modification above and examine the drive:\path\filename on the screen. It would seem that is the place were the decision is made to download it or not and either the current directory is not correct or the path is fubar.

[brandonrit]

Thanks I'll try it and let you know!

You rock!

[Squashman]

I was looking at this script for a while last night and could not figure out the purpose of that For Loop. The only thing it is doing is echoing what is in that variable. It is not doing anything with the FTP process itself.

I am going to test it out at work today to see if I can get it too work. I might have a use for it some day.

[brandonrit]

My issue appears to be in the comparing of the list of files on the ftp vs the local drive

any ideas?

[foxidrive]

We need details to fix yours, but try this (untested):

EDITED: details changed

Code: Select all

@echo off
set server_name=mydomain.com
set username=myname
set password=mypassword
set local_folder=c:\test

echo ---------------------------------------------------------------------------
echo %date% %time% Begin Transfer

(
echo open %server_name%
echo %username%
echo %password%
echo cd /
echo ls 
echo quit
)>list.ftp

FTP -s:list.ftp >filelist.txt

for /f "skip=2 delims=" %%a in (
'type "filelist.txt"^|find /i /v "ftp>" ^|find /i /v "ftp:" ^|find /i /v "226 Transfer complete"'
) do if not exist "%local_folder%\%%a" >>filelist2.txt echo %%a

(
echo %open server_name%
echo %username%
echo %password%
echo cd /
)>list2.ftp


for /f "delims=" %%a in ('type "filelist2.txt"') do (
>>list2.ftp echo get %%a
)

echo quit>>list2.ftp

FTP -s:list2.ftp

del list.ftp
del list2.ftp
del filelist.txt
del filelist2.txt

echo %date% %time% Done FTP Transfer
echo.
echo.

[brandonrit]

Paths all appear to be correct
files are being written to the correct directories -it just is not doing the do if not exist action properly..

time for more coffee

[foxidrive]

Show us what it reports when it prints to the console, in the debugging step. Also show us the local folder contents from DIR /B


Also try the code I posted above.

[brandonrit]

Figured it out - the batch file actually works fine - it is comparing the files on the ftp to the local directory of the batch file.

The batch file was actually writing the file to another location - hence the constant downloading..

One more cup of coffee was all i needed....

[foxidrive]

You're welcome.