WinSCP vs FTP.exe vs NETUSE w/Robocopy [WinSCP is Winner!]

[julesverne]

Hi guys, so I'm almost ready to start my speed differential tests between windows cmd ftp / winscp ftp / and net use w/robocopy. I've come across this issue though I need some help with concerning the windows ftp, and after much googling can't find anything, which baffles my mind. I've got scripts ready for the robocopy and winscp ftp test, just not win ftp.

I wanna do my tests based on the needs that I need rather than setting up a generic file transfer test.

I want to take a folder on my computer that contain files and subdirectories with files in them and transfer them to a LAN server using ftp.

the main directory looks like this below

project-20140514-083015-Jules

now the word "project" will be the only consistent word used for the main directory. The rest is date time and project name.

Inside that main folder there will be a few files and can have two subdirectories in there. There may not always be. It will have at least one, but sometimes two. They would always be called "video" and "audio" and if they are there they will have files in them.

So an example local path could look like C:\Program Files\Program\project-20140514-083015-Jules\video\*.*

I want to mirror this same directory tree on the remote server to the path \\server\hostname\client\clientexample\projectexample\project-20140514-083015-Jules

So that all subdirectories and files in project-20140514-083015-Jules are mirrored on the server.

Again, the date and time and project name would always be different. and subdirectory video and audio may or may not be there, but if they are they need to be included, and they will have files in them.

Preferably.. this ftp script would be included in a batch file rather than this script being called from a batch file. But if that's not possible. As far as username and pw one will be hardcoded the pw will be a variable that can be stored or passed to the ftp script.

Thanks!

[Squashman]

The native ftp.exe in Windows can not copy an entire folder structure up to an FTP site.
You would be better off with a 3rd party utility that can synchronize your local drive with the ftp server.

[julesverne]

WinSCP is the winner based on the criteria in the first post with the addition that I forgot about, that sometimes there are subfolders in the video folder with files in them.

However, this is just based on the scripts that I was able to come up with. Any other ideas are very welcome.

The only way that ftp.exe might be more competitive is possibly a better FOR loop than the one I created. As far as I can tell I have to build (by echoing out in the batch) and execute the ftp scripts multiple times in the for loop to ensure ftp'ing all subfolders and files in regards to squashman's comment.

The native ftp.exe in Windows can not copy an entire folder structure up to an FTP site.

RESULTS OF TEST
(171,532,288 bytes)
9 Files, 9 Folders

WinSCP ** FTP.exe ** NETUSE
****************w/ ROBOCOPY
_____________________________
seconds ** seconds ** seconds
15.07 *****16.32******16.76
15.08 *****16.21******16.41
15.05 *****16.30******16.47
15.07 *****16.17******16.55
15.07 *****16.05******16.54

Code: Select all

TITLE WinSCP TEST BAT

set start=%time%

for /f "delims=*" %%T in ('dir /ad /b "C:\Program Files\program\project*" ') do (
   START "winscp" /MIN /WAIT c:\winscp\winscp.exe /command "option batch continue" "open ftp://user:pw@server:21" "cd /client/clientname/projectname" "put ""C:\Program Files\program\%%T"" /client/clientname/projectname/%%T" "close" "exit" /xmllog="%~dp0%fileverification.xml"   
)


set end=%time%
set options="tokens=1-4 delims=:."
for /f %options% %%a in ("%start%") do set start_h=%%a&set /a start_m=100%%b %% 100&set /a start_s=100%%c %% 100&set /a start_ms=100%%d %% 100
for /f %options% %%a in ("%end%") do set end_h=%%a&set /a end_m=100%%b %% 100&set /a end_s=100%%c %% 100&set /a end_ms=100%%d %% 100

set /a hours=%end_h%-%start_h%
set /a mins=%end_m%-%start_m%
set /a secs=%end_s%-%start_s%
set /a ms=%end_ms%-%start_ms%
if %hours% lss 0 set /a hours = 24%hours%
if %mins% lss 0 set /a hours = %hours% - 1 & set /a mins = 60%mins%
if %secs% lss 0 set /a mins = %mins% - 1 & set /a secs = 60%secs%
if %ms% lss 0 set /a secs = %secs% - 1 & set /a ms = 100%ms%
if 1%ms% lss 100 set ms=0%ms%

:: mission accomplished
set /a totalsecs = %hours%*3600 + %mins%*60 + %secs% 
echo
echo Converting %filecounter% file(s) took %hours% hours %mins% minutes %secs%.%ms% seconds (%totalsecs%.%ms%s total)
echo.

pause

Code: Select all

Title WINDOWS FTP SCRIPT TEST BAT
set start=%time%

for /f %%a in ('dir /a:D /b "C:\Program Files\program\project*"') do (
   echo open server>mkorigscript.ftp
   echo user>>mkorigscript.ftp
   echo pw>>mkorigscript.ftp
   echo ^!:---ftp commands below here --- >>mkorigscript.ftp
   echo mkdir /client/clientname/projectname/%%a>>mkorigscript.ftp
   echo bye>>mkorigscript.ftp
   @ftp -v -i -s:"mkorigscript.ftp"
   for /f %%F in ('dir /a:-d /b "C:\Program Files\program\%%a"') do (
      echo open server>mkorigscript.ftp
      echo user>>mkorigscript.ftp
      echo pw>>mkorigscript.ftp
      echo ^!:---ftp commands below here --->>mkorigscript.ftp
      echo lcd "C:\Program Files\program\%%a">>mkorigscript.ftp
      echo cd /client/clientname/projectname/%%a>>mkorigscript.ftp
      echo mput *.*>>mkorigscript.ftp
      echo bye>>mkorigscript.ftp
      @ftp -v -i -s:"mkorigscript.ftp"
   )
   for /f %%F in ('dir /a:D /b "C:\Program Files\program\%%a"') do (
      echo open server>mkorigscript.ftp
      echo user>>mkorigscript.ftp
      echo pw>>mkorigscript.ftp
      echo ^!:---ftp commands below here --->>mkorigscript.ftp
      echo mkdir /client/clientname/projectname/%%a/%%F>>mkorigscript.ftp
      echo lcd "C:\Program Files\program\%%a\%%F">>mkorigscript.ftp
      echo cd /client/clientname/projectname/%%a/%%F>>mkorigscript.ftp
      echo mput *.*>>mkorigscript.ftp
      echo bye >>mkorigscript.ftp
      @ftp -v -i -s:"mkorigscript.ftp"
   )
   if exist "C:\Program Files\program\%%a\video" (
      for /f %%F in ('dir /a:D /b "C:\Program Files\program\%%a\video"') do (
         echo open server>mkorigscript.ftp
         echo user>>mkorigscript.ftp
         echo pw>>mkorigscript.ftp
         echo ^!:---ftp commands below here --->>mkorigscript.ftp
         echo mkdir /client/clientname/projectname/%%a/video/%%F>>mkorigscript.ftp
         echo lcd "C:\Program Files\program\%%a\video\%%F">>mkorigscript.ftp
         echo cd /client/clientname/projectname/%%a/video/%%F>>mkorigscript.ftp
         echo mput *.*>>mkorigscript.ftp
         echo bye >>mkorigscript.ftp
         @ftp -v -i -s:"mkorigscript.ftp"
      )
   )
)

set end=%time%
set options="tokens=1-4 delims=:."
for /f %options% %%a in ("%start%") do set start_h=%%a&set /a start_m=100%%b %% 100&set /a start_s=100%%c %% 100&set /a start_ms=100%%d %% 100
for /f %options% %%a in ("%end%") do set end_h=%%a&set /a end_m=100%%b %% 100&set /a end_s=100%%c %% 100&set /a end_ms=100%%d %% 100

set /a hours=%end_h%-%start_h%
set /a mins=%end_m%-%start_m%
set /a secs=%end_s%-%start_s%
set /a ms=%end_ms%-%start_ms%
if %hours% lss 0 set /a hours = 24%hours%
if %mins% lss 0 set /a hours = %hours% - 1 & set /a mins = 60%mins%
if %secs% lss 0 set /a mins = %mins% - 1 & set /a secs = 60%secs%
if %ms% lss 0 set /a secs = %secs% - 1 & set /a ms = 100%ms%
if 1%ms% lss 100 set ms=0%ms%

:: mission accomplished
set /a totalsecs = %hours%*3600 + %mins%*60 + %secs% 
echo
echo Converting %filecounter% file(s) took %hours% hours %mins% minutes %secs%.%ms% seconds (%totalsecs%.%ms%s total)
echo.

pause

Code: Select all

TITLE NET USE WITH ROBOCOPY BAT TEST

set start=%time%

net use X: "\\server\share" /USER:user pw /PERSISTENT:NO


for /f "delims=*" %%T in ('dir /ad /b "C:\Program Files\program\project*" ') do (
   MD X:\client\clientname\projectname\%%T
   ROBOCOPY "C:\Program Files\program\%%T" "X:\client\clientname\projectname\%%T" /E /NJS /NDL /NC /NFL /NS /NJH
)

net use x: /d
klist purge

set end=%time%
set options="tokens=1-4 delims=:."
for /f %options% %%a in ("%start%") do set start_h=%%a&set /a start_m=100%%b %% 100&set /a start_s=100%%c %% 100&set /a start_ms=100%%d %% 100
for /f %options% %%a in ("%end%") do set end_h=%%a&set /a end_m=100%%b %% 100&set /a end_s=100%%c %% 100&set /a end_ms=100%%d %% 100

set /a hours=%end_h%-%start_h%
set /a mins=%end_m%-%start_m%
set /a secs=%end_s%-%start_s%
set /a ms=%end_ms%-%start_ms%
if %hours% lss 0 set /a hours = 24%hours%
if %mins% lss 0 set /a hours = %hours% - 1 & set /a mins = 60%mins%
if %secs% lss 0 set /a mins = %mins% - 1 & set /a secs = 60%secs%
if %ms% lss 0 set /a secs = %secs% - 1 & set /a ms = 100%ms%
if 1%ms% lss 100 set ms=0%ms%

:: mission accomplished
set /a totalsecs = %hours%*3600 + %mins%*60 + %secs% 

echo Converting %filecounter% file(s) took %hours% hours %mins% minutes %secs%.%ms% seconds (%totalsecs%.%ms%s total)
echo.

pause

[foxidrive]

15 seconds isn't a good test. Copy 20 GB and then see what the results are.

Also be aware that robocopy has switches to increase efficiency on a LAN with the /J switch from memory.

[Squashman]

Your ftp script could be improved upon by alot. You could just create one large ftp script instead of ecexuting multiple ftp scripts.

I am schocked that SCP would be faster considering it should have overhead with the encryption.