Thanks in advance for your help.
I am using ROBOCOPY to copy some files from on drive to another on the same computer. Every once in awhile, I get an error. And instead of retrying 999 times as it should, it retries once and fails. I have a couple questions:
Why would this error be happening in the first place?
Why isn't ROBOCOPY retrying the 999 times as defined?
Commands are below:
mkdir C:\Users\tempuser\AppData\Local\temp\test1
robocopy /R:999 /W:5 /NP /E /XO /NFL /NDL E:\test1 C:\Users\tempuser\AppData\Local\temp\test1 test*
-------------------------------------------------------------------------------
ROBOCOPY :: Robust File Copy for Windows
-------------------------------------------------------------------------------
Started : Monday, March 25, 2013 4:20:51 AM
Source : E:\test1
Dest : C:\Users\tempuser\AppData\Local\temp\test1
Files : test*
Options : /NDL /NFL /S /E /DCOPY:DA /COPY:DAT /NP /XO /R:999 /W:5
------------------------------------------------------------------------------
2013/03/25 04:20:51 ERROR 32 (0x00000020) Accessing Destination Directory C:\Users\tempuser\AppData\Local\temp\test1
The process cannot access the file because it is being used by another process.
Waiting 5 seconds... Retrying...
------------------------------------------------------------------------------
Total Copied Skipped Mismatch FAILED Extras
Dirs : 1 0 0 0 1 0
Files : 0 0 0 0 0 0
Bytes : 0 0 0 0 0 0
Times : 0:00:05 0:00:00 0:00:05 0:00:00
Ended : Monday, March 25, 2013 4:20:56 AM
This is quite probably a bug in robocopy. It has at least one other bug surrounding Error 32 / in use files: /b (backup) mode will fail with this error even if a file is not exclusively locked (and is copyable with 'copy', 'xcopy', windows explorer, and robocopy without /b), so I suspect there are bugs in how it handles in-use files in general.
There is no "bug" in ROBOCOPY.
Something is "locking" your source folders and files from time-to-time. Not always, as that's proof based on the fact that your copies work "at times".
I would place a script to copy "out" your source files into another location (drive preferably), and then use ROBOCOPY from that new location into your other or final location. Use the move option to keep the intermediate location free for the next backups, etc.
I use this...
robocopy <source path> <target path> <files> /s /j /r:2 /w:5 /log+:robocopy.log
I hope someone finds this helpful. This "fix" has worked for me several times so good luck.
Simply logout of the server with file explorer and PowerShell running... force close any apps running at the sign-out Windows page and log back into the server . Run your robocopy script and your locked process would have been released.
Sorry if it doesn't work for you.
Related
I have a large directory I'm using Robocopy to sync between new and old storage. Even if there isn't much data to copy and not many files on the destination not on the source I still am getting a very large amount of time in the Extras column. For example, on last sync less than one hour was in the Copied column (around 10K files), but over 30 hours in Extras (only 23 files and 0 directories in Extras though)! What could take so much time in Extras? This is the options being used.
/FFT /S /E /DCOPY:AT /COPY:DAT /PURGE /Z /NP /R:1 /W:1
I am using below command inside the batch file and running via task scheduler in my server , it deletes most of the nested files/folders but when path length exceed 255 it does not delete those files/folders. Is this issue with RMDIR , What are other alternatives , Robocopy Delete ? Has somebody faced this issue and robocopy was able to solve this issue for them , something like below ?
RMDIR "Path-location" /S /Q -- Does not delete files/folder > 255 length
Robocopy : You need to create empty folder before executing below commands :
robocopy "C:\empty" "C:\Test Folder" /purge
rmdir "C:\Test Folder" // Once all files/folders are deleted now remove your parent/root directory
This is normal behaviour. To be exact, the limit is defined as 260 characters (source). You have multiple options to circumvent this limit:
You can raise this limit to 65535 characters by adding/setting the registry value HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled of type REG_DWORD to 1. This works since Windows 10, version 1607 (read more). Reboot your machine after this change.
You can use robocopy in combination with the /PURGE parameter to synchronize an empty folder to the folder that contains the long file paths as you already found out. This might get complicated if you do not want to delete everything, but only certain files.
You can use UNC paths to describe long paths. Change
C:\very\long\...\path
to
\\?\C:\very\long\...\path
I have several tasks using Robocopy.exe, On One of those tasks, I get a error result 0X3 but the log file doesn't display any error.
Here is the command :
ROBOCOPY \\X\L$\FORMATION \\Y\O$\FORMATION /MIR /sec /W:2 /R:5 /log:c:\O_journal.log
I tried to launch it without the task scheduler, it worked fine without ant errors.
I know that the error 0X3 is : ERROR_PATH_NOT_FOUND.But the log file doesn't display any error.
If someone have any clue or any methods...
This is a standard behavior and relates to exit codes from Robocopy:
0×00 0 No errors occurred, and no copying was done.
The source and destination directory trees are completely synchronized.
0×01 1 One or more files were copied successfully (that is, new files have arrived).
0×02 2 Some Extra files or directories were detected. No files were copied
Examine the output log for details.
0×04 4 Some Mismatched files or directories were detected.
Examine the output log. Housekeeping might be required.
0×08 8 Some files or directories could not be copied
(copy errors occurred and the retry limit was exceeded).
Check these errors further.
0×10 16 Serious error. Robocopy did not copy any files.
Either a usage error or an error due to insufficient access privileges
on the source or destination directories.
And a combination of them, ie: 0x03 = 0x02 + 0x01
See https://blogs.technet.microsoft.com/deploymentguys/2008/06/16/robocopy-exit-codes/ for a complete explanation.
This is an issue to the path configured on the robocopy script. I have fixed this issue by adding the root directory or the main drive to both source and destination.
Note: Please make sure that you are an administrator to both Servers (source and destination)
Example:
Source: "\\192.168.1.2\e$\my files"
Destination: "\\192.168.1.5\d$\backup"
The root directory or the main drive that I have referred is the "e$" and "d$" as shown above. If this will not included in your script, you will be encountering the "0X3" error in the Scheduler and the scheduler will not works.
Note: This script is to synchronize/replicate the file from the source to destination.
Parameters used. /MIR /E /Z /R:5 /W:5
This is how the script looks as a whole:
robocopy "\\\192.168.1.2\e$\my files" "\\\192.168.1.5\d$\backup" /MIR /E /Z /R:5 /W:5
Hope this might help.
Thanks.
Xian
I have executed the following command:
robocopy.exe /E /ZB /COPY:DAT /R:5 /W:7 /TBD /NP /TEE /LOG:
The intended file was copied, along with some other files which had the exact same name and were residing in the folder beneath the source directory. but this appeared in the output:
Options : /TBD /TEE /S /E /COPY:DAT /ZB /NP /R:5 /W:7
*EXTRA Dir -1 \computer\path\to\file
New Dir 0 \computer\path\to\folder
New Dir 0 \computer\path\to\folder
New Dir 1 \computer\path\to\folder
New File 1.5 m filename
-OUTPUT TRIMMED-
Total Copied Skipped Mismatch FAILED Extras
Dirs : 15 14 1 0 0 2
Files : 4 4 0 0 0 0
I understand the *EXTRA dir stands for: An “extra” file is present in destination but not source. Can anyone explain the -1?
two folders were copied to the destination, without either of them containing the file specified in the command, or the file name in the path. They were not empty in the source folder but were copied empty. Why were those folders copied?
I'm trying to find an easy way of deploying only changed files to the webserver for deployment purpose. In times past I've used MSBuild, which could be told to only copy files that were newer than the ones on the target, but I'm in a hurry and don't want to try to figure out the newer version of MSBuild.
Can I use ROBOCOPY for this? There is a list of options for exclusion, which is:
/XC :: eXclude Changed files.
/XN :: eXclude Newer files.
/XO :: eXclude Older files.
/XX :: eXclude eXtra files and directories.
/XL :: eXclude Lonely files and directories.
What exactly does it mean to exclude? Exclude copying, or exclude overwriting? For example, if I wrote:
ROBOCOPY C:\SourceFolder\ABC.dll D:\DestinationFolder /XO
would this copy only newer files, not files of the same age?
Or is there a better tool to do this?
To answer all your questions:
Can I use ROBOCOPY for this?
Yes, RC should fit your requirements (simplicity, only copy what needed)
What exactly does it mean to exclude?
It will exclude copying - RC calls it skipping
Would the /XO option copy only newer files, not files of the same age?
Yes, RC will only copy newer files. Files of the same age will be skipped.
(the correct command would be robocopy C:\SourceFolder D:\DestinationFolder ABC.dll /XO)
Maybe in your case using the /MIR option could be useful. In general RC is rather targeted at directories and directory trees than single files.
You can use robocopy to copy files with an archive flag and reset the attribute. Use /M command line, this is my backup script with few extra tricks.
This script needs NirCmd tool to keep mouse moving so that my machine won't fall into sleep. Script is using a lockfile to tell when backup script is completed and mousemove.bat script is closed. You may leave this part out.
Another is 7-Zip tool for splitting virtualbox files smaller than 4GB files, my destination folder is still FAT32 so this is mandatory. I should use NTFS disk but haven't converted backup disks yet.
backup-robocopy.bat
#REM https://technet.microsoft.com/en-us/library/cc733145.aspx
#REM http://www.skonet.com/articles_archive/robocopy_job_template.aspx
set basedir=%~dp0
del /Q %basedir%backup-robocopy-log.txt
set dt=%date%_%time:~0,8%
echo "%dt% robocopy started" > %basedir%backup-robocopy-lock.txt
start "Keep system awake" /MIN /LOW cmd.exe /C %basedir%backup-robocopy-movemouse.bat
set dest=E:\backup
call :BACKUP "Program Files\MariaDB 5.5\data"
call :BACKUP "projects"
call :BACKUP "Users\Myname"
:SPLIT
#REM Split +4GB file to multiple files to support FAT32 destination disk,
#REM splitted files must be stored outside of the robocopy destination folder.
set srcfile=C:\Users\Myname\VirtualBox VMs\Ubuntu\Ubuntu.vdi
set dstfile=%dest%\Users\Myname\VirtualBox VMs\Ubuntu\Ubuntu.vdi
set dstfile2=%dest%\non-robocopy\Users\Myname\VirtualBox VMs\Ubuntu\Ubuntu.vdi
IF NOT EXIST "%dstfile%" (
IF NOT EXIST "%dstfile2%.7z.001" attrib +A "%srcfile%"
dir /b /aa "%srcfile%" && (
del /Q "%dstfile2%.7z.*"
c:\apps\commands\7za.exe -mx0 -v4000m u "%dstfile2%.7z" "%srcfile%"
attrib -A "%srcfile%"
#set dt=%date%_%time:~0,8%
#echo %dt% Splitted %srcfile% >> %basedir%backup-robocopy-log.txt
)
)
del /Q %basedir%backup-robocopy-lock.txt
GOTO :END
:BACKUP
TITLE Backup %~1
robocopy.exe "c:\%~1" "%dest%\%~1" /JOB:%basedir%backup-robocopy-job.rcj
GOTO :EOF
:END
#set dt=%date%_%time:~0,8%
#echo %dt% robocopy completed >> %basedir%backup-robocopy-log.txt
#echo %dt% robocopy completed
#pause
backup-robocopy-job.rcj
:: Robocopy Job Parameters
:: robocopy.exe "c:\projects" "E:\backup\projects" /JOB:backup-robocopy-job.rcj
:: Source Directory (this is given in command line)
::/SD:c:\examplefolder
:: Destination Directory (this is given in command line)
::/DD:E:\backup\examplefolder
:: Include files matching these names
/IF
*.*
/M :: copy only files with the Archive attribute and reset it.
/XJD :: eXclude Junction points for Directories.
:: Exclude Directories
/XD
C:\projects\bak
C:\projects\old
C:\project\tomcat\logs
C:\project\tomcat\work
C:\Users\Myname\.eclipse
C:\Users\Myname\.m2
C:\Users\Myname\.thumbnails
C:\Users\Myname\AppData
C:\Users\Myname\Favorites
C:\Users\Myname\Links
C:\Users\Myname\Saved Games
C:\Users\Myname\Searches
:: Exclude files matching these names
/XF
C:\Users\Myname\ntuser.dat
*.~bpl
:: Exclude files with any of the given Attributes set
:: S=System, H=Hidden
/XA:SH
:: Copy options
/S :: copy Subdirectories, but not empty ones.
/E :: copy subdirectories, including Empty ones.
/COPY:DAT :: what to COPY for files (default is /COPY:DAT).
/DCOPY:T :: COPY Directory Timestamps.
/PURGE :: delete dest files/dirs that no longer exist in source.
:: Retry Options
/R:0 :: number of Retries on failed copies: default 1 million.
/W:1 :: Wait time between retries: default is 30 seconds.
:: Logging Options (LOG+ append)
/NDL :: No Directory List - don't log directory names.
/NP :: No Progress - don't display percentage copied.
/TEE :: output to console window, as well as the log file.
/LOG+:c:\apps\commands\backup-robocopy-log.txt :: append to logfile
backup-robocopy-movemouse.bat
#echo off
#REM Move mouse to prevent maching from sleeping
#rem while running a backup script
echo Keep system awake while robocopy is running,
echo this script moves a mouse once in a while.
set basedir=%~dp0
set IDX=0
:LOOP
IF NOT EXIST "%basedir%backup-robocopy-lock.txt" GOTO :EOF
SET /A IDX=%IDX% + 1
IF "%IDX%"=="240" (
SET IDX=0
echo Move mouse to keep system awake
c:\apps\commands\nircmdc.exe sendmouse move 5 5
c:\apps\commands\nircmdc.exe sendmouse move -5 -5
)
c:\apps\commands\nircmdc.exe wait 1000
GOTO :LOOP
Looks like /e option is what you need, it'll skip same files/directories.
robocopy c:\data c:\backup /e
If you run the command twice, you'll see the second round is much faster since it skips a lot of things.