I have tested and fixed the code from abc0502 and it works great. The code you posted wasn't doing recurrive checks on existing files but it helped me figure out a lot of things I didn't know. One very important thing I learned that some REM and :: lines will still execute if special characters are included in them, causing errors and ERRORLEVEL items should be checked in reverse order.
Now I need to understand how to create a log file from all of my ECHO statements, and how to submit the directory names in a command statement like: c:>MergeDirectory M:\ScrDir M:DestDir M:DelDir /L or /M for move or list. Any help would be appreciated.
Here is the new, errorfree, revised code for anyone that is interested in this kind of batch file. I'm not sure how fast it will be - I want to be able to write to a log before running. Also, I am saving files to a DELETE folder instead of deleting them right away, but will get rid of that code once I feel comfortable that everything works great. Thanks for helping.
Code: Select all
@echo OFF
setLOCAL DisableDelayedExpansion
REM setLOCAL EnableDelayedExpansion
REM EnableDelayedExpansion is Disabled by default.
REM .. A BATCH WITH EXAMPLES TO HELP YOU LEARN DOS
REM .. comments with .. tell you what the code does in general english and sometimes why
REM ,, comments with ,, explain specific code syntax
REM -- comments with -- are lines of code that should have worked
REM -- but did not work because of specials characters so
REM -- they are usually followed with alternate code that did work
REM .. MergeDirs.bat: Merge a source tree into a destination tree, renaming duplicated files
REM .. MergeDirs C:\sourcedir D:\destdir C:\deletedDir
REM .. ListOrMove L=List M=Move
REM .. Get source and destination drives from parameters in the command line or calling batch file
set "from=%1"
set "to=%2"
set "DeleteFiles=%3"
set "ListOrMove=%4"
REM .. Delete quotes from front of directory names and back of directory names
REM .. In order to pass the directory names from the batch, quotes are used, but
REM .. the quotes will not work in later code, so they are removed here and added back
REM .. as needed
REM ,, for /F loops against a set of files or strings delims=xxx delimiter character(s) (default = a space)
REM ,, usebackq is used when you need to put double quotes around a filename -
REM ,, it checks for double quotes and automatically removes them when processing the
REM ,, strings and placing the value into the token
REM ,, tokens=n Specifies which/how many variables will be needed to read items from
REM ,, each line (default=1, * means automatically add as many as needed)
REM ,, so when your string is '"Filename"', usebackq causes this to be seen as 'Filename'
for /F "usebackq tokens=*" %%a in ('%from%') do set from=%%~a
for /F "usebackq tokens=*" %%a in ('%to%') do set to=%%~a
for /F "usebackq tokens=*" %%a in ('%DeleteFiles%') do set DeleteFiles=%%~a
REM .. Delete trailing backslash "\" from directory names. Our code adds a "\"
REM .. between directories and filenames, so we need to make sure "\" deleted here.
REM ,, When you delete characters in a string you can use substrings to do it. The
REM ,, following code should have worked in the steps above to strip our quotes, but
REM ,, the quote itself caused the code to not work, so we had to use the FOR loop
REM ,, to remove the quotes in the code above. Now we can use substrings to check a
REM ,, trailing backslash at the end of each directory name and then remove it.
REM ,, The IF statement does not work for quotes in the string so use a FOR loop
REM -- THIS DOES NOT WORK BECAUSE OF QUOTES: if %from:~0,1%==" set from=%from:~0,1%
REM ,, to use substrings: %variable:~num_chars_to_skip, num_chars_to_keep%
REM ,, ==> 0,1 keep first letter; -1 keep last letter
REM ,, if second num is blank keep everything after first num is found
REM ,, "-" means to start from the right and go backward, other wise strart from the left
if %from:~-1%==\ set from=%from:~-1%
if %to:~-1%==\ set to=%to:~-1%
if %DeleteFiles:~-1%==\ set DeleteFiles=%DeleteFiles:~-1%
REM .. Use WMIC to retrieve date and time for use in log filename
REM ,, for /F loops against a set of files or strings delims=x delimiter character(s)
REM ,, (in this case, use a comma)
REM ,, skip=2 means to skip the first two lines of the strings that will be processed (default = 0)
REM ,, tokens=n Specifies which/how many variables will be needed to read items from
REM ,, each line (default=1, * means automatically add as many as needed)
REM ,, In this case use the second item through the seventh item in the string
REM ,, %%A is a replaceable parameter (a variable name) so each item can be processed
REM ,, on the command line you would use %A instead of %%A used in this batch file
REM ,, WMIC Path Win32_LocalTime Get returns the date in a particluar format.
REM ,, Day^,Hour^,Minute^,Month^,Second^,Year is the list of expected items. The ^ is
REM ,, needed because comma "," is the delimiter.
REM ,, /Format:csv returns the date string in a table format with items separated by commas.
for /F "skip=2 tokens=2-7 delims=," %%A IN ('WMIC Path Win32_LocalTime Get Day^,Hour^,Minute^,Month^,Second^,Year /Format:csv') DO (
REM ,, Use the /A (arithmetic) when calculating with numbers. All parts of the above date string are numbers.
set /A SortDate = 10000 * %%F + 100 * %%D + %%A
set /A SortTime = 10000 * %%B + 100 * %%C + %%E
)
REM .. Add a timestamp to the log filename;
REM ,, %cd% puts the logfile in current directory the batch/command was
REM ,, called/started from
set "MergeLogFile=%cd%\MergeDirectoryLog-%SortDate%%SortTime%.txt"
set "SkippedLogFile=%cd%\MergeDirectoryLog-%SortDate%%SortTime%-Skipped.txt"
set "DeletedLogFile=%cd%\MergeDirectoryLog-%SortDate%%SortTime%-Deleted.txt"
set "ErrorLogFile=%cd%\MergeDirectoryLog-%SortDate%%SortTime%-Errors.txt"
set "MovedLogFile=%cd%\MergeDirectoryLog-%SortDate%%SortTime%-Moved.txt"
REM .. Make sure the source directory exists before executing the batch job
if NOT Exist "%from%" (
echo "Directory %from% does not exist. Please enter a valid directory name."
echo "Directory %from% does not exist. Please enter a valid directory name." >> %MergeLogFile%
REM ,, goto :eof will return back to the code/script/function/place that called
REM ,, this script, in this case, either another batch file or the command line.
goto :eof
)
REM .. Display the Directory names now that they have been cleaned up.
echo Move From: %from%
echo Move From: %from% >> %MergeLogFile%
echo Move To: %to%
echo Move To: %to% >> %MergeLogFile%
echo Deleted Files: %DeleteFiles%
echo Deleted Files: %DeleteFiles% >> %MergeLogFile%
echo List or Move: %ListOrMove%
echo List or Move: %ListOrMove% >> %MergeLogFile%
echo Log file: MergeLogFile=%cd%MergeDirsLog-%SortDate%%SortTime%.txt
echo Log file: MergeLogFile=%cd%MergeDirsLog-%SortDate%%SortTime%.txt >> %MergeLogFile%
REM .. Set up some tracking variables
set MoveTo=%to%
set "LogMsg="
set /A "from_len=0"
set /A "LimitLoops=100"
set "MsgLvl=*"
echo MoveTo %MoveTo%
echo MoveTo %MoveTo% >> %MergeLogFile%
echo.
echo. >> %MergeLogFile%
REM .. Create Destination Folder if not Exist
if not Exist "%to%" MD "%to%" >nul
if not Exist "%DeleteFiles%" MD "%DeleteFiles%" >nul
REM .. Get Length of the characters in "from" variable to remove them later
for /F "usebackq delims=" %%A in ('"%from%"') Do (Call :strLen from from_len)
REM .. Add "1" to the length to avoid problems later
REM .. += is shorthand used to add the current variable value to the number
REM .. following it and store it in the variable (from_len = from_len + number)
set /A from_len+=1
echo "===== BEGIN Robocopy MOVE FILES"
echo "===== BEGIN Robocopy MOVE FILES" >> %MergeLogFile%
REM .. Move new folders and new files from "Source" to "Destination"
REM .. This Robocopy is set to ignore any existing duplicate folder names and files
REM .. Duplicates will be addressed later because robocopy can not rename files
REM ,, /L List only - don’t copy, timestamp, move or delete any files.
REM ,, /TEE Output to console window, as well as the log file.
REM ,, /E : Copy Subfolders, including Empty Subfolders.
REM ,, /MOVE Move files and dirs (delete from source after copying).
REM ,, /XC eXclude Changed; /XN Newer files; /XO eXclude Older files;
REM ,, /XX eXclude "eXtra" files and dirs (present in destination but not source)
if %ListOrMove% EQU "M" (
echo "** %ListOrMove% Moving files **" >> %MergeLogFile%
ROBOCOPY "%from%" "%to%" /TEE /E /MOVE /XC /XN /XO /XX /LOG+:"%MergeLogFile%" /R:3
) ELSE (
echo "** %ListOrMove% Listing Files **" >> %MergeLogFile%
ROBOCOPY "%from%" "%to%" /L /TEE /E /MOVE /XC /XN /XO /XX /LOG+:"%MergeLogFile%" /R:3
)
echo "===== Robocopy MOVE FILES COMPLETED"
echo "===== Robocopy MOVE FILES COMPLETED" >> %MergeLogFile%
echo.
echo. >> %MergeLogFile%
REM .. Compare and Move the rest of the files.
REM ,, for /F loops against a set of files delims=xxx delimiter character(s)
REM ,, (default = a space); usebackq Used to put quotes around a
REM ,, filename and use the alternate quoting style:
REM ,, - Use double quotes for long file names in "filenameset".
REM ,, - Use single quotes for 'Text string to process'
REM ,, - Use back quotes for `command to process`
REM ,, DIR list files and subfolders /B=bare with no header or file info;
REM ,, /S=show subdirectories; /A:D Folders only; /A:-D NO Folders;
set /A "numFilesProcessed=0"
set /A "numFilesDeleted=0"
set /A "numFilesMoved=0"
set /A "numFilesSkipped=0"
set /A "numErrors=0"
SETLOCAL EnableDelayedExpansion
REM --------------- Begin primary loop to process files ----------------------
for /F "delims=" %%A In ('DIR /B /S /A:-D "%from%"') Do (
REM --------------------------------------------------------------------------
echo.
echo.
echo "==== NEW FILE ==== !Date! !Time!" "%%A"
echo.
echo. >> %MergeLogFile%
echo. >> %MergeLogFile%
echo "==== NEW FILE ==== !Date! !Time!" "%%A" >> %MergeLogFile%
echo. >> %MergeLogFile%
REM .. Reset skip flag each time a new file is processed
set /A "skipfile=0"
REM .. Increment total number for files processed by 1
set /A "numFilesProcessed+=1"
REM .. Check for exclamation point in the filename and skip file if
REM .. there are any - set enable delayed and any ! will disappear in filename
REM .. so an easy check is to check to see if the file exists
if NOT Exist "%%A" (
set /A "skipFile=1"
echo *************** CheckName !CheckName!
) ELSE (
REM .. Checking the filename for more special characters that can cause
REM .. the script to fail.
set /A "OrigfnLen=0"
set "fFullFileName=%%A"
call :strlen "fFullFileName" OrigfnLen
set /A "spclCharLen=0"
set "Checkname=%%A"
REM -- These don't work (! ^%% ^")
REM .. Don't check for ":" because it is in the drive path name
For %%I In (® ^| ^& ^< ^> ^^ + @ # $ { } [ ] ' ` ^%% ^" ) Do (Set CheckName=!CheckName:%%I=!)
call :strlen CheckName spclCharLen
if !spclCharLen! NEQ !OrigfnLen! set /A "skipfile=1"
)
if !skipfile!==1 (
echo "+++++ File has bad/unusual character in name, so skipping. +++++" "%%A"
echo "+++++ File has bad/unusual character in name, so skipping. +++++" "%%A">> %MergeLogFile% 2>&1
echo "%%A">> %SkippedLogFile% 2>&1
set /A "numFilesSkipped+=1"
) ELSE (
REM .. set variables with parts of filename and path
REM .. For a top level, fsubdir should be null.
REM .. ex: fromPath=M:\DirSrc, fFullPath=M:\DirSrc, then fsubdir=""
set "fFullPath=%%~dpA"
set "fsubdir=!fFullPath:~%from_len%!"
set "fFileNmExt=%%~nxA"
set "fFileName=%%~nA"
set "fExt=%%~xA"
set "fSize=%%~zA"
set "fdate=%%~tA"
rem echo "These were used while testing"
rem echo "from - %from% - !from!" - %%A"
rem echo "fFullFileName - %fFullFileName% - !fFullFileName! - %%A"
rem echo "fFullPath - %fFullPath% - !fFullPath! - %%~dpA"
rem echo "fsubdir - %fsubdir% - !fsubdir!"
rem echo "fFileNmExt - %fFileNmExt% - !fFileNmExt! - %%~nxA"
rem echo "fFileName - %fFileName% - !fFileName! - %%~nA"
rem echo "fExt - %fExt% - !fExt! - %%~xA"
rem echo "fsize - %fsize% - !fsize!"
rem echo "fdate - %fdate% - !fdate!"
rem echo "OrigfnLen %OrigfnLen% - !OrigfnLen!"
rem echo.
set /a "FileNumber=-1"
set "MoveTo=!to!"
if Not Exist "%to%\!fsubdir!!fFileNmExt!" (
set /a "FileNumber=0"
) ELSE (
REM .. :FileIncrNum Checks for Duplicates - Name, Date/Time, and Size. If one
REM .. is not the same it adds extra number at the end of the filename.
call :FileIncrNum FileNumber,!fsize!,"!fdate!","!fFullPath!",%from_len%,"!fFileName!","!fExt!",MoveTo,%LimitLoops%
)
REM cmd /c exit 0 is used to set errorlevel to 0 so that error checking will work
cmd /c exit 0
if %ListOrMove% EQU "M" (
if not Exist "!MoveTo!\!fsubdir!" (
echo "Make directory !MoveTo!\!fsubdir!"
echo "Make directory !MoveTo!\!fsubdir!" >> %MergeLogFile%
MD "!MoveTo!\!fsubdir!" >> %MergeLogFile% 2>&1
if ERRORLEVEL 1 if NOT ERRORLEVEL 2 echo "An ERROR occurred while creating the directory." >> %MergeLogFile%
)
)
echo FileNumber !FileNumber!
if !FileNumber! EQU 0 (
set movetoFileName=!MoveTo!\!fsubdir!!fFileNmExt!
) Else (
set movetoFileName=!MoveTo!\!fsubdir!!fFileName! ^(!FileNumber!^)!fExt!
)
echo movetoFileName !movetoFileName!
echo "=!numFilesProcessed!= Moving !fFullFileName! TO !movetoFileName!"
echo "=!numFilesProcessed!= Moving !fFullFileName! TO !movetoFileName!" >> %MergeLogFile%
if %ListOrMove% EQU "M" (
MOVE /Y "!fFullFileName!" "!movetoFileName!" >> %MergeLogFile% 2>&1
if ERRORLEVEL 0 if NOT ERRORLEVEL 1 (
if !MoveTo! EQU %DeleteFiles% (
set /a "numFilesDeleted+=1"
echo "%%A">> %DeletedLogFile% 2>&1
) ELSE (
set /a "numFilesMoved+=1"
echo "%%A">> %MovedLogFile% 2>&1
)
echo "ERRORLEVEL %ERRORLEVEL% no errors occurred, move succeeded."
echo "ERRORLEVEL %ERRORLEVEL% no errors occurred, move succeeded." >> %MergeLogFile%
) ELSE (
if ERRORLEVEL 1 if NOT ERRORLEVEL 2 (
echo "ERRORLEVEL 1 -- an error occurred."
echo "ERRORLEVEL 1 -- an error occurred." >> %MergeLogFile%
) ELSE (
echo "ERRORLEVEL 2 or greater occurred. %ERRORLEVEL%"
echo "ERRORLEVEL 2 or greater occurred. %ERRORLEVEL%" >> %MergeLogFile%
)
set /a "numErrors+=1"
echo "%%A">> %ErrorLogFile% 2>&1
)
)
)
)
if %ListOrMove% EQU "M" (
echo.
echo.
echo. >> %MergeLogFile%
echo. >> %MergeLogFile%
echo "** List Subdirectories in %from% **"
echo "** List Subdirectories in %from% **" >> %MergeLogFile%
REM .. Sort /R= Sort in reverse order
DIR /B /S /A:D "%from%" ^| sort /R >> %MergeLogFile% 2>&1
echo "** Delete Subdirectories in %from% **"
echo "** Delete Subdirectories in %from% **" >> %MergeLogFile%
for /F "usebackq delims=" %%d in (`DIR /B /S /A:D "%from%" ^| sort /R`) do echo %%d >> %MergeLogFile% 2>&1
for /F "usebackq delims=" %%d in (`DIR /B /S /A:D "%from%" ^| sort /R`) do rd "%%d" >> %MergeLogFile% 2>&1
echo "** Delete Subdirectories deleted **"
echo "** Delete Subdirectories deleted **" >> %MergeLogFile%
)
echo.
echo.
echo. >> %MergeLogFile%
echo. >> %MergeLogFile%
echo Number of Files Processed %numFilesProcessed% !numFilesProcessed!
echo Number of Files Processed %numFilesProcessed% >> %MergeLogFile%
echo Number of Files Deleted %numFilesDeleted% !numFilesDeleted!
echo Number of Files Deleted %numFilesDeleted% >> %MergeLogFile%
echo Number of Files Moved %numFilesMoved% !numFilesMoved!
echo Number of Files Moved %numFilesMoved% >> %MergeLogFile%
echo Number of Files Skipped %numFilesSkipped% !numFilesSkipped!
echo Number of Files Skipped %numFilesSkipped% >> %MergeLogFile%
echo Number of Errors %numErrors% !numErrors!
echo Number of Errors %numErrors% >> %MergeLogFile%
goto :eof
:: ==========================================================::
:FileIncrNum <1IncrNum> <2fromSIZE_var> <3fromDATE_var> <4fromFullPath> <5fromPathLen> <6fromFilename> <7fromExt> <8destPath> <9looplimit_var>
:: ----------------------------------------------------------::
REM .. Function to calculate recursively the next number to use in the file name
setLOCAL ENABLEDELAYEDEXPANSION
set /a "IncrNum=%~1+1"
set fromSIZE=%~2
set fromDATE=%~3
set fromFullPath=%~4
set fromPathLen=%~5
set fromFileName=%~6
set fromExt=%~7
set destPath=!%~8!
set /a "looplimit=0+%~9"
set fromFullFileName=%fromFullPath%%fromFileName%%fromExt%
set subPath=!fromFullPath:~%fromPathLen%!
rem echo IncrNum !IncrNum!
If !IncrNum! EQU 0 (
set destFileName=!destPath!\!subPath!!fromFileName!!fromExt!
) Else (
set destFileName=!destPath!\!subPath!!fromFileName! ^(!IncrNum!^)!fromExt!
)
rem echo destFileName !destFileName! %destFileName%
if %looplimit% EQU 0 (
if Exist "!destFileName!" (
call :FileIncrNum IncrNum,%fromSIZE%,"%fromDATE%","%fromFullPath%",%fromPathLen%,"%fromFileName%","%fromExt%",destPath,0
)
) ELSE (
echo "PROPOSED FILE EXISTS SO RENAME THE FILE !IncrNum! !destFileName!" >> %MergeLogFile%
if !IncrNum! LSS !looplimit! (
if Exist "!destFileName!" (
Call :Info "!destFileName!" "toSIZE" "toDATE"
if "!toSIZE!" EQU "%fromSIZE%" (
set "LogMsg=THE FILES HAVE SAME SIZES !toSIZE! NEQ %fromSIZE% - SAVE FILE AND COMPARE - %fromFullFileName%"
if "!toDATE!" EQU "%fromDATE%" (
set "LogMsg=THE FILES HAVE SAME DATES !toDATE! NEQ %fromDATE% - SAVE FILE AND COMPARE - %fromFullFileName%"
FC /B /LB1 "%fromFullFileName%" "!destFileName!" | FIND "FC: no dif">nul
if ERRORLEVEL 0 if NOT ERRORLEVEL 1 (
set "LogMsg=THE FILE %fromFullFileName% HAS THE SAME DATE, SAME SIZE, NO BINARY DIFFERENCE FROM !destFileName! - SO DELETE THE FILE "
set "destPath=!DeleteFiles!"
set /a "IncrNum=0"
set "destFileName=!destPath!\%subPath%%fromFileName%%fromExt%"
) ELSE (set "LogMsg=THE FILE %fromFullFileName% HAS THE SAME DATE, SAME SIZE, BUT HAS A BINARY DIFFERENCE - SAVE FILE AND MANUALLY COMPARE ")
) ELSE (set "LogMsg=THE FILES HAVE DIFFERENT DATES !toDATE! NEQ %%~tA - SAVE FILE AND MANUALLY COMPARE - %fromFullFileName% ")
) Else (set "LogMsg=THE FILES HAVE DIFFERENT SIZES !toSIZE! NEQ %%~zA - SAVE FILE AND MANUALLY COMPARE - %fromFullFileName% ")
echo "*!LogMsg! - !destFileName!" >> %MergeLogFile%
if "!destPath!" EQU "%DeleteFiles%" (
if Exist "!destFileName!" (
echo "FILE !destFileName! exists, so increment file number and try again. IncrNum %IncrNum% !IncrNum! " >> %MergeLogFile%
call :FileIncrNum IncrNum,%fromSIZE%,"%fromDATE%","%fromFullPath%",%fromPathLen%,"%fromFileName%","%fromExt%",destPath,0
) ELSE (echo "Delete file !destFileName! does not exist so go back and move file to delete IncrNum %IncrNum% !IncrNum!" >> %MergeLogFile%)
) ELSE (
if "!destPath!" EQU "%to%" (
if Exist "!destFileName!" (
echo "FILE EXISTS !destFileName!, increment file number IncrNum %IncrNum% !IncrNum! - destPath %destPath% !destPath!" >> %MergeLogFile%
call :FileIncrNum IncrNum,%fromSIZE%,"%fromDATE%","%fromFullPath%",%fromPathLen%,"%fromFileName%","%fromExt%",destPath,%LimitLoops%
) ELSE (echo "File !destFileName! does not exist so go back and move file IncrNum %IncrNum% !IncrNum!" >> %MergeLogFile%)
) ELSE (echo "not equal %destPath% !destPath! EQU %to%" >> %MergeLogFile%)
)
)
) ELSE (echo "****ERROR**** LOOP LIMIT REACHED !IncrNum! !Looplimit!" >> %MergeLogFile%)
)
(
ENDLOCAL
set "%~1=%IncrNum%"
set "%~8=%destPath%"
)
goto :eof
:: ==========================================================::
:strLen <string> <len>
:: ----------------------------------------------------------::
::-- returns the length of a string
:: -- string [in] - variable name containing the string being measured for length
:: -- len [out] - variable to be used to return the string length
:: Many thanks to 'sowgtsoi', but also 'jeb' and 'amel27' dostips forum users helped making this short and efficient
::$created 20081122 :$changed 20101116 :$categories StringOperation
::$source http://www.dostips.com
(SETLOCAL ENABLEDELAYEDEXPANSION
set "str=A!%~1!"
set "len=0"
for /L %%C in (12,-1,0) do (
set /a "len|=1<<%%C"
for %%D in (!len!) do (if "!str:~%%D,1!"=="" set /a "len&=~1<<%%C")
)
)
( ENDLOCAL & REM RETURN VALUES
IF "%~2" NEQ "" SET /a "%~2=%len%"
if "%~2" EQU "" set /a echo %len%
)
goto :eof
:: ==========================================================::
:Info <file> <size_var> <time/date_var>
:: ----------------------------------------------------------::
Set "Info_file=%~1"
For /F "delims=" %%A In ("%Info_File%") Do (
Set "%~2=%%~zA"
Set "%~3=%%~tA"
)
goto :eof
