I'm trying to find an easy way of deploying only changed files to the webserver for deployment purpose. In times past I've used MSBuild, which could be told to only copy files that were newer than the ones on the target, but I'm in a hurry and don't want to try to figure out the newer version of MSBuild.
Can I use ROBOCOPY for this? There is a list of options for exclusion, which is:
/XC :: eXclude Changed files.
/XN :: eXclude Newer files.
/XO :: eXclude Older files.
/XX :: eXclude eXtra files and directories.
/XL :: eXclude Lonely files and directories.
What exactly does it mean to exclude? Exclude copying, or exclude overwriting? For example, if I wrote:
ROBOCOPY C:\SourceFolder\ABC.dll D:\DestinationFolder /XO
would this copy only newer files, not files of the same age?
Or is there a better tool to do this?
To answer all your questions:
Can I use ROBOCOPY for this?
Yes, RC should fit your requirements (simplicity, only copy what needed)
What exactly does it mean to exclude?
It will exclude copying - RC calls it skipping
Would the /XO option copy only newer files, not files of the same age?
Yes, RC will only copy newer files. Files of the same age will be skipped.
(the correct command would be robocopy C:\SourceFolder D:\DestinationFolder ABC.dll /XO)
Maybe in your case using the /MIR option could be useful. In general RC is rather targeted at directories and directory trees than single files.
You can use robocopy to copy files with an archive flag and reset the attribute. Use /M command line, this is my backup script with few extra tricks.
This script needs NirCmd tool to keep mouse moving so that my machine won't fall into sleep. Script is using a lockfile to tell when backup script is completed and mousemove.bat script is closed. You may leave this part out.
Another is 7-Zip tool for splitting virtualbox files smaller than 4GB files, my destination folder is still FAT32 so this is mandatory. I should use NTFS disk but haven't converted backup disks yet.
backup-robocopy.bat
#REM https://technet.microsoft.com/en-us/library/cc733145.aspx
#REM http://www.skonet.com/articles_archive/robocopy_job_template.aspx
set basedir=%~dp0
del /Q %basedir%backup-robocopy-log.txt
set dt=%date%_%time:~0,8%
echo "%dt% robocopy started" > %basedir%backup-robocopy-lock.txt
start "Keep system awake" /MIN /LOW cmd.exe /C %basedir%backup-robocopy-movemouse.bat
set dest=E:\backup
call :BACKUP "Program Files\MariaDB 5.5\data"
call :BACKUP "projects"
call :BACKUP "Users\Myname"
:SPLIT
#REM Split +4GB file to multiple files to support FAT32 destination disk,
#REM splitted files must be stored outside of the robocopy destination folder.
set srcfile=C:\Users\Myname\VirtualBox VMs\Ubuntu\Ubuntu.vdi
set dstfile=%dest%\Users\Myname\VirtualBox VMs\Ubuntu\Ubuntu.vdi
set dstfile2=%dest%\non-robocopy\Users\Myname\VirtualBox VMs\Ubuntu\Ubuntu.vdi
IF NOT EXIST "%dstfile%" (
IF NOT EXIST "%dstfile2%.7z.001" attrib +A "%srcfile%"
dir /b /aa "%srcfile%" && (
del /Q "%dstfile2%.7z.*"
c:\apps\commands\7za.exe -mx0 -v4000m u "%dstfile2%.7z" "%srcfile%"
attrib -A "%srcfile%"
#set dt=%date%_%time:~0,8%
#echo %dt% Splitted %srcfile% >> %basedir%backup-robocopy-log.txt
)
)
del /Q %basedir%backup-robocopy-lock.txt
GOTO :END
:BACKUP
TITLE Backup %~1
robocopy.exe "c:\%~1" "%dest%\%~1" /JOB:%basedir%backup-robocopy-job.rcj
GOTO :EOF
:END
#set dt=%date%_%time:~0,8%
#echo %dt% robocopy completed >> %basedir%backup-robocopy-log.txt
#echo %dt% robocopy completed
#pause
backup-robocopy-job.rcj
:: Robocopy Job Parameters
:: robocopy.exe "c:\projects" "E:\backup\projects" /JOB:backup-robocopy-job.rcj
:: Source Directory (this is given in command line)
::/SD:c:\examplefolder
:: Destination Directory (this is given in command line)
::/DD:E:\backup\examplefolder
:: Include files matching these names
/IF
*.*
/M :: copy only files with the Archive attribute and reset it.
/XJD :: eXclude Junction points for Directories.
:: Exclude Directories
/XD
C:\projects\bak
C:\projects\old
C:\project\tomcat\logs
C:\project\tomcat\work
C:\Users\Myname\.eclipse
C:\Users\Myname\.m2
C:\Users\Myname\.thumbnails
C:\Users\Myname\AppData
C:\Users\Myname\Favorites
C:\Users\Myname\Links
C:\Users\Myname\Saved Games
C:\Users\Myname\Searches
:: Exclude files matching these names
/XF
C:\Users\Myname\ntuser.dat
*.~bpl
:: Exclude files with any of the given Attributes set
:: S=System, H=Hidden
/XA:SH
:: Copy options
/S :: copy Subdirectories, but not empty ones.
/E :: copy subdirectories, including Empty ones.
/COPY:DAT :: what to COPY for files (default is /COPY:DAT).
/DCOPY:T :: COPY Directory Timestamps.
/PURGE :: delete dest files/dirs that no longer exist in source.
:: Retry Options
/R:0 :: number of Retries on failed copies: default 1 million.
/W:1 :: Wait time between retries: default is 30 seconds.
:: Logging Options (LOG+ append)
/NDL :: No Directory List - don't log directory names.
/NP :: No Progress - don't display percentage copied.
/TEE :: output to console window, as well as the log file.
/LOG+:c:\apps\commands\backup-robocopy-log.txt :: append to logfile
backup-robocopy-movemouse.bat
#echo off
#REM Move mouse to prevent maching from sleeping
#rem while running a backup script
echo Keep system awake while robocopy is running,
echo this script moves a mouse once in a while.
set basedir=%~dp0
set IDX=0
:LOOP
IF NOT EXIST "%basedir%backup-robocopy-lock.txt" GOTO :EOF
SET /A IDX=%IDX% + 1
IF "%IDX%"=="240" (
SET IDX=0
echo Move mouse to keep system awake
c:\apps\commands\nircmdc.exe sendmouse move 5 5
c:\apps\commands\nircmdc.exe sendmouse move -5 -5
)
c:\apps\commands\nircmdc.exe wait 1000
GOTO :LOOP
Looks like /e option is what you need, it'll skip same files/directories.
robocopy c:\data c:\backup /e
If you run the command twice, you'll see the second round is much faster since it skips a lot of things.
Related
I am creating a batch file to be run later which will be used to copy files from one location to another while renaming the files, skipping any existing files, and without prompting the user. Examples of files to be copied:
00021001.txt
00021001.xyz
00021001.abc
00021001001.jpg
Copied files will have the names:
00022001.txt
00022001.xyz
00022001.abc
00022001001.jpg
Things I have tried:
xcopy C:\Testing\1000012\21\00021*.* C:\Testing\1000013\22\00022*.* /D
This almost does it. It copies all the files starting with "00021" in the first location into the second location while properly renaming them to start with "00022". It skips all the files with the same name and date stamp, but ends up prompting to copy any files from the source which are newer than the target.
robocopy C:\Testing\1000012\21\ C:\Testing\1000013\22\ 00021*.* /xo /xn /xc
I was hoping that by excluding older, newer, and same date files it would work (even if it doesn't rename - I would just do that in a separate step.) Unfortunately, this just ends up overwriting newer source files over existing target files if they are a different filesize.
I have even tried the Copy-Item command in PowerShell. But it doesn't do the renaming like Xcopy, and it doesn't skip existing files (although I can get it to confirm and say "No to All".)
Copy-Item -Path "C:\CWUImageCompare\Testing\1000012\CWU\chemistry\129\21\00021*.*" -Destination "C:\CWUImageCompare\Testing\1000013\CWU\chemistry\129\20\" -Confirm
If xcopy had the "Skip if existing" flag I'd be all set, but it doesn't.
#ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
rem The following settings for the source directory & destination directory are names
rem that I use for testing and deliberately includes spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=u:\your files"
SET "destdir=u:\your results"
FOR /f "skip=1delims=" %%b IN ('xcopy /L /Y "%sourcedir%\00021*.*" "%destdir%\00022*.*" ^|sort') DO (
SET "oname=%%~nxb"
IF EXIST "%destdir%\00022!oname:~5!" (ECHO "%%b" skipped) ELSE (ECHO COPY /y "%%b" "%destdir%\00022!oname:~5!")
)
GOTO :EOF
Always verify against a test directory before applying to real data.
Seems a little complicated, but essentially, execute the xcopy (I omitted the /D for testing) with /L /Y to simply produce a list.
Since the list has a last line that begins with a numeric, whereas the other lines start with a drive-letter, sort the list and skip the first line.
This would implement the date-requirement.
Then grab the part after the first 5 characters of the name+extension, test whether the new name exists and either report or copy as appropriate (The copy command is disarmed for testing)
I am using omake for my project build
As part of my delivery build I need to copy some delivery related files into a folder
How I can use robocopy execution where I can only copy the interested files(which are selected by running a for loop on a list I have)
Into the folder.
List_paths = filepath1
Filepath2
.
FilepathN
For file in list_paths
%exec robocopy $(file,A) $(destin)
End for
By doing that robocopy always complains me that there is no such file by adding \ at the end of eCh file and for destin folder too.
I Understand that robocopy just works fine for copying files from one folder to other but I have condition to copy only selective files from folders (for which I have the list of paths)
Please help on this.
If you have a textfile with complete patnames (i.e list.txt), with content:
d:\temp\a.sql
d:\temp\b.sql
d:\temp\c.sql
and a batchfile roboDo.bat like this:
#echo off
SET dest=d:\temp\temp\
FOR /f "useback tokens=1" %%f IN (`type list.txt`) DO (
ECHO copying %%~nxf from %%~dpf to %dest%
ROBOCOPY /NP /NJH /NJS %%~dpf %dest% %%~nxf >NUL
)
running the batchfile wil output this:
D:\TEMP>robodo
copying a.sql from d:\TEMP\ to d:\temp\temp\
copying b.sql from d:\TEMP\ to d:\temp\temp\
copying c.sql from d:\TEMP\ to d:\temp\temp\
I have a job that periodically runs and archives files into a folder structure that looks like this:
ArchiveFolder
TimestampFolder
JobnameFolder
job<timestamp>.xml
For a given job, I'd like to collect all xml files in the archive folder into a flat directory (no subdirectories, just all the files) without having to drill down into each one, examine for the proper job, then copy the file.
It seems there should be a fairly straigtforward way of doing this. Any suggestions?
EDIT: I guess I wasn't clear here. The TimeStampFolder will have a name of something like 2011-07-24, the JobnameFolder will have a name like FooFeed or BarFeed, and the job file will have a name like job2011-07-24.xml. There are hundreds to thousands of TimeStampFolders, and each one may have one or more job folders in it. Given a specific job name, I want to collect all the files in all the directories that match that job type, and dump them into the local folder, with no subdirectories.
EDIT1:
SET JOB=JobName
SET OF=OutputFolder
START /wait NET USE Z: "\\ServerName\Sharename\ArchiveFolder" password password_here /USER:domainname\username /P:NO
PUSHD Z:\
FOR /F "USEBACKQ tokens=*" %%A IN (`DIR /b /a:d /s ^| FIND /I "%JOB%"`) DO (
FOR /R %%F IN (%%A) DO (
COPY /Y "%%~fF" "%OF%"
)
)
POPD
It basically locates each subdirectory of ArchiveFolder that includes the JobName in it, then digs into each one that it finds to copy the files out of them.
EDIT2:
Added NET USE to access your network share to perform tasks on the files. If your local machine already has the UNC assigned to a driveletter, you can remove the NET USE command line and change Z: to the assigned driveletter.
#ECHO OFF
FOR /R %%v IN (job*.xml) DO COPY "%%v" c:\out\
I'm sending commands to a remote computer in order to have it copy a file.
I want the file to be copied, but not to overwrite the previous file with the same name (if it exists).
I also need the command to run without any prompts (xcopy likes to prompt whether the target name I've specified is file or directory, and it will also prompt about overwriting a file).
I have good results with xcopy /d.
It will copy NEWER files, and since we can assume that existing files have same time-stamp, you will copy only files that don't exist.
just in case anyone else finds this:
robocopy x:\sourcefolder Y:\destfolder /s /e /r:0 /z
much better than xcopy, even gives you a table at the end informing of any failed or skipped files. Doesn't prompt to not overwrite.
Well, there's a certain remedy! It has helped me with saving much of my effort and time on Win10 while writing a setup for our product demo.
Just try to use piping:
#ECHO N|COPY /-Y SourceFiles Destination
As an example I used this piece of code so that I would have a clean gentle quiet and safe copy!
#FOR /D %%F in ("FooPath") DO #(
#ECHO N|COPY /-Y ^"%%~npdxF\*.*^" ^"GooPath^" 3>NUL 2>NUL >NUL
)
where obviously FooPath is the source and GooPath is the destination.
Enjoy!
(main source: https://ss64.com/nt/copy.html)
Following command copy files and folder but not override file if already exist.
xcopy "*.*" "C:\test\" /s /y /d
No way to make it NOT overwrite as far as I know. but /Y will make it overwrite. and /I will get rid of the file/dict prompt. See xcopy /? for all options
You can also use the replace command. It has two modes: to add files that don't exist there or replace files that do exist. You want the previous mode:
replace <path1> <path2> /A
I had to copy AND rename files, so I got the prompt about creating a file or a directory.
This is the, rather "hackish" way I did it:
ECHO F | XCOPY /D "C:\install\dummy\dummy.pdf" "C:\Archive\fffc810e-f01a-47e8-a000-5903fc56f0ec.pdf"
XCOPY will use the "F" to indicate it should create the target as a file:
C:\install>ECHO F | XCOPY /D "C:\install\dummy\dummy.html" "C:\Archive\aa77cd6e-1d19-4eb4-b2a8-3f8fe60daf00.html"
Does C:\Archive\aa77cd6e-1d19-4eb4-b2a8-3f8fe60daf00.html specify a file name or directory name on the target
(F = file, D = directory)? F
C:\install\dummy\dummy.html
1 File(s) copied
I've also verified this command leaves existing files alone. (You should too :-)
I need to delete files of a certain type (.zip files, say) from a folder, and all of its sub-folders, using the command line. Ideally I am looking for something that can be run as a .bat file in Windows.
I know there is a /S switch for the DEL command to look in sub-folders, but to add to the challenge I need to exclude files of a certain name ("Backup.zip" as an example).
Is there a way to delete files recursively but exclude files of a certain name. It will not be practical in my situation to explicitly list all the file names I want to delete, only the files of the matching type I don't want to delete.
A nice trick: make the files you want to exclude read-only!
DEL /S will not delete read-only file.
The following script does not do exactly what you want (see my remarks below) but shows you how read-only files can be used to avoid deletion.
#ECHO OFF
:: This example supposes your first parameter is
:: the relative path to the folder to be deleted
:: and the second is the relative path (from the
:: the target folder) of the one to be excluded
:: Notice that this will only work if the folders
:: are in the working drive, if not, you ll
:: have to specify full paths
IF "%1"=="" GOTO ERROR
IF "%2"=="" GOTO ERROR
IF NOT EXIST %1\NUL GOTO ERROR
CD %1
IF NOT EXIST %2\NUL GOTO ERROR
ECHO Starting up the deletion process
ECHO. * Setting attributes
attrib %1\*.mp3 -r -s -h > NUL
attrib %2\*.mp3 +r > NUL
ECHO. * Deleting files
del /s %1\*.mp3
ECHO. * Reseting attributes
attrib %2\*.mp3 -r > NUL
ECHO.
ECHO Operation completed!
ECHO.
GOTO END
:ERROR
ECHO Parameters:
ECHO. Param1 -> target folder
ECHO. Param2 -> folder to be ignored
ECHO.
GOTO END
:END
Note: you can adapt this script in order to ignore not just a sub-folder but all files of given type:
attrib /S *.xxx +r > NUL
will in effect help you to exclude all 'xxx' files of the current directory and all sub-directories (hence the /S option).
Note: the "> NUL" part is a redirection often used to hide standard output, instead of displaying it on screen.
It can be dangerous if used too often (in a large loop with different paths involved, for instance) since it is a device, and like all devices (AUX, COMn, LPTn, NUL and PRN):
opening a device will claim one file handle. However, unlike files, devices will never be closed until reboot.
each device exists in every directory on every drive, so if you used redirection to NUL in, say, C:\ and after that you use it again in C:\TEMP, you'll lose another file handle.
Just do this, easy
windows button + r
type cmd and hit enter
Navigate to parent directory:
type c: or d: (or letter of the drive you want to navigate to)
type dir to see a list of that directory's contents ( dir /ah to see hidden files )
then to change directory, type cd xxxx ( xxxx = directory name )
Repeat 4&5 until you get to the directory where you want to run the batch delete
then type your pattern. Something like: del /S /ah *.jpg and hit enter. It will run through all sub-directories, and remove all visible and hidden files that are .jpg files
* is a wildcard
/S deletes from all subfolders
/ah a = select files based on attribute, h = hidden
Example: to delete those annoying .DS_Store files that appear when you copy from Mac to Windows, run:
del /S /ah .DS_Store
or
del /S /ah ._*
which will get all the 'duplicate' hidden files that are also created when copying from Mac to Windows.
You can easily loop a set of files and perform a command on each one, like this:
set match=D:\blah\M*.zip
for %%x in (%match%) do (
del %%x
)
Then I think you need to read this article on how to manipulate strings in DOS:
http://www.dostips.com/DtTipsStringManipulation.php
You can simply use below:
forfiles /p C:\temp-new /s /c "cmd /c if #isdir==FALSE del #file"
TechNet Referenceenter link description here
Perhaps the 'forfiles' command could be of use
http://technet.microsoft.com/en-us/library/cc753551.aspx
Hope that helps.
This script will delete all .zip files from a folder (and subfolders), BUT ONLY IF the file name does not contain the word "backup", or "Backup" or "BACKUP", etc.
# Script DeleteZip.txt
var str folder, filelist, file
cd $folder
lf -r -n "*.zip" > $filelist
while ($filelist <> "")
do
lex "1" $filelist > $file
# Does this $file contain the word "backup" ? Do case independent search (-c option).
if ( { sen -c "^backup^" $file } <= 0 )
system del ("\""+$file+"\"")
endif
done
Script is in biterscripting ( http://www.biterscripting.com ). Save the script in file C:/Scripts/DeleteZip.txt. Run the script as
script "C:/Scripts/DeleteZip.txt" folder("C:/testfolder")
Will do the delete operation on folder C:/testfolder and all its subfolders, at all levels.
A simple way that you need:
FORFILES /P "DIRECTORY" /S /M SEARCHMASK /C "cmd /c if #file neq \"FILE_TO_EXCLUDE\" del /f #file"
You still can adapt the command to your need to exclude specific files and/or by date. More details in the 'forfiles' documentation below.
https://learn.microsoft.com/pt-br/windows-server/administration/windows-commands/forfiles