I am putting together a Powershell script that syncs a source code tree into another directory. It creates a list of directories to ignore, per the output of robocopy /?:
# ----------------------------------------------------------------------------
# Source file directories
# ----------------------------------------------------------------------------
$reactjs = "C:\Users\me\Projects\prj\reactjs"
# ----------------------------------------------------------------------------
# Target directories
# ----------------------------------------------------------------------------
$synergy = "C:\Users\me\Projects\sync\reactjs"
$synReactjs = $synergy + "\prj-JS"
# ----------------------------------------------------------------------------
# Directories to be ignored
# ----------------------------------------------------------------------------
$reactIgnores = #(
$reactjs + "\.git",
$reactjs + "\node_modules",
$reactjs + "\build"
)
# ----------------------------------------------------------------------------
# Copy/sync files
# ----------------------------------------------------------------------------
Robocopy /l $reactjs $synReactjs /mir /xd $reactIgnores
However, robocopy immediately descends into the .git directory, though the output header looks like it parsed my switches correctly:
-------------------------------------------------------------------------------
ROBOCOPY :: Robust File Copy for Windows
-------------------------------------------------------------------------------
Started : Wednesday, October 31, 2018 12:00:32 PM
Source : C:\Users\me\Projects\prj\reactjs\
Dest : C:\Users\me\Projects\sync\reactjs\prj-JS\
Files : *.*
Exc Dirs : C:\Users\me\Projects\prj\reactjs\.git C:\Users\me\Projects\prj\reactjs\node_modules C:\Users\me\Projects\prj\reactjs\build
Options : *.* /L /S /E /DCOPY:DA /COPY:DAT /Z /R:1000000 /W:30
------------------------------------------------------------------------------
New Dir 8 C:\Users\me\Projects\prj\reactjs\
New File 634 .eslintrc.json
New File 206 .gitignore
New File 55 .npmrc
New File 521255 package-lock.json
New File 1015 package.json
New File 1001 package.json.react16
New File 10391 README
New File 1639 README.md
New Dir 9 C:\Users\me\Projects\prj\reactjs\.git\
New File 135 COMMIT_EDITMSG
New File 319 config
New File 73 description
New File 125 FETCH_HEAD
<...>
What am I missing with the /xd switch? It works fine with a single directory.
Edit: I included the Powershell tag because /xd ignores the list of files if I include them in the command line explicitly. Loading them via the array does not work. This results in an exc line of (compare to above):
Exc Dirs : C:\Users\me\Projects\prj\reactjs\.git
C:\Users\me\Projects\prj\reactjs\node_modules
C:\Users\me\Projects\prj\reactjs\build
In
$reactIgnores = #(
$reactjs + "\.git",
$reactjs + "\node_modules",
$reactjs + "\build"
)
try putting the full path.
This array worked for me, escaping double quotes with single quotes. You don't need the full path and I used a single /XD $eXcludeDirectories command:
$eXcludeDirectories = #(
'"Folder01"',
'"Folder02"',
'"Folder03"'
)
Hopefully this will help someone else.
Related
I have to fetch the files from SFTP account from directory "/sourcedir/Test/TRANS*/transferfiles.csv" using PUTTY and transfer them over to my local destination dir. Having trouble using the wildcard "TRANS*" in the directory path. How do i use multiple wildcards in the directory path?? I'm getting the error "/sourcedir/Test/TRANS*/*transferfiles*.csv": multiple-level wildcards unsupported.".
TIA
[string]$TransferResults = & 'C:\Program Files\PuTTY\pscp.exe' -l 'username' -pw 'password' "username#$(IPaddress):/sourcedir/Test/TRANS*/*transferfiles*.csv" $Destination
I tried the solution that #Cpt.Whale suggested.
Output:
Listing directory /sourcedir/Test/ drwxr--r-- 1 - - 0 Apr 28 14:43 TRANS_whiteplain drwxr--r-- 1 - - 0 Apr 28 14:43 TRANS_test_1
Code snippet to parse in foreach loop.
foreach($file in $parsedfolders){
[string]$fpath = $file.fullName
[string]$TransferResults = & 'C:\Program Files\PuTTY\pscp.exe' -l 'username' -pw 'password' "username#$(IPaddress):$fpath/*transferfiles*.csv"
$Destination
i get the error : unable to identify /transferfiles.csv: no such file or directory
I'll assume your output from pscp -ls looks like this, based on your comment:
Listing directory /sourcedir/Test/
drwxr--r-- 1 - - 0 Apr 28 14:43 TRANS_whiteplain
drwxr--r-- 1 - - 0 Apr 28 14:43 TRANS_test_1
drwxr--r-- 1 - - 0 Apr 28 14:43 TRANS test spaces
From that output, we can use regex to get the folder names:
# First, list the folder names in the upper level folder
$folders = & 'C:\Program Files\PuTTY\pscp.exe' -l 'username' -pw 'password' -ls "username#10.0.0.1:/sourcedir/Test/"
# only lines starting with d, select everything after time "0:00"
$pattern = '^d.+\d:\d{2} (.*)'
# parse output into folder names using regex
$parsedFolders = foreach ($folder in $folders) {
# trim whitespace
[regex]::Match($folder,$pattern).Groups[1].Value.trim() |
# discard empty results
Where { -not [string]::IsNullOrWhiteSpace($_) }
}
Now the parsed folders should be usable:
$parsedFolders
TRANS_whiteplain
TRANS_test_1
TRANS test spaces
So try and do your copy for each folder:
# Finally, copy files from each parsed folder to destination
$results = foreach ($parsedFolder in $parsedFolders) {
& 'C:\Program Files\PuTTY\pscp.exe' -l 'username' -pw 'password' "username#10.0.0.1:/sourcedir/Test/$parsedfolder/*transferfiles*.csv" $Destination
}
I have .msu and .exe files in a folder. I'm creating INI files for each. For the .msu files it runs just fine. For the .exe files it processes the first one then it skips the rest of them. The code
$INIFileName = ($strItem -replace ".$fileExtention",".ini")
seems to be the issue. It doesn't replace any of following .exe extentions to the .ini format after the initial .exe file
Output:
** Creating .INI files for any msu's that don't have one **
Original file name with .exe: kb0000001.msu
New INI file name with .ini : kb0000001.ini
Original file name with .exe: kb0000002.msu
New INI file name with .ini : kb0000002.ini
Original file name with .exe: kb0000003.msu
New INI file name with .ini : kb0000003.ini
** Creating .INI files for any exe's that don't have one **
Original file name with .exe: kb0000004.exe
New INI file name with .ini : kb0000004.ini
Original file name with .exe: kb0000005.exe
New INI file name with .ini : kb0000005.exe
C:\Temp\PatchBundle\kb0000005.exe exists. Skipping.
Original file name with .exe: kb0000006.exe
New INI file name with .ini : kb0000006.exe
C:\Temp\PatchBundle\kb0000006.exe exists. Skipping.
PS C:\temp\PatchBundle>
Test files: (Created in code)
kb0000001.msu
kb0000002.msu
kb0000003.msu
kb0000004.exe
kb0000005.exe
kb0000006.exe
Code:
## Create test files
Function CreateTestFiles ($strDestPath)
{
If (-not(Test-Path "$strDestPath\kb0000001.msu"))
{
New-Item ("$strDestPath\kb0000001.msu")
New-Item ("$strDestPath\kb0000002.msu")
New-Item ("$strDestPath\kb0000003.msu")
New-Item ("$strDestPath\kb0000004.exe")
New-Item ("$strDestPath\kb0000005.exe")
New-Item ("$strDestPath\kb0000006.exe")
}
}
## Create INI's for files
Function CreateINI ($fileExtention, $SPNumber, $strDestPath)
{
Write-Host -ForegroundColor Magenta `n " ** Creating .INI files for any $fileExtention's that don't have one **" `n
## Obtain a list of KB*.extention files.
$arrList = get-childitem -path $strDestPath -name -filter "kb*.$fileExtention"
## If extention list is empty, abort
If ($arrList.Count -eq 0)
{
write-host -foregroundcolor "red" " No KB*.$fileExtention files found to work with."
}
Else {
## Start looping through the list stored in the array
Foreach ($strItem in $arrList) {
## Determine the patch INI name with path
Write-Host "Original file name with .exe: $strItem"
$INIFileName = ($strItem -replace ".$fileExtention",".ini")
$strDestINI = "$strDestPath\$INIFileName"
Write-Host "New INI file name with .ini : $INIFileName"
## If the destination patch INI already exists, skip processing to avoid overwriting.
If (Test-Path $strDestINI) {
write-host -ForegroundColor Red " $strDestINI exists. Skipping."
}
## Else, create a new patch INI from the template.
Else {
## Gets KB Number from file name
$KBNumber = ((($strItem -replace ".$fileExtention","").ToUpper()).TrimStart("KB"))
## Creates INI file
#OutputINIFile $fileExtention $KBNumber $SPNumber $strDestINI
## If File Extention is MSU, checks if file needs the wsusscan.cab file pulled and renamed
## Checks: Ignores file names with v. Example: KB1234567v2.msu
## Checks: Length of file name is greater than 13 chararacters. Example: KB1234567IE7.msu
If (($fileExtention = 'msu') -and `
($strItem -notlike '*v*') -and `
($strItem.Length -gt 13))
{
## Create the -scan.cab file
#Create-scanCab $KBNumber $strDestPath $strItem
}
}
}
}
}
Clear
## Main
## Variables in the script
$strDestPath = "C:\Temp\PatchBundle"
[Void][System.IO.Directory]::CreateDirectory($strDestPath)
$fileExtentions = "msu", "exe"
## Gets the service pack # based on the OS selected
$SPNumber = 2
CreateTestFiles $strDestPath
## Create .INI for all extentions files
Foreach ($extention in $fileExtentions)
{CreateINI $extention $SPNumber $strDestPath}
In line 55 you are assigning extention as msu inside if condition(for msu files this won't effect but for the second exe file extension replacement won't happen because of that),like this:
If (($fileExtention = 'msu') -and `
I think you meant
If (($fileExtention -eq 'msu') -and `
During my analysis why my script isn't working I could find some solutions that solves some basic parts (also with the help of Stackoverflow), but there is still an issue for which I cannot find a solution.
When I create a temp-directory-variable $PARAM_TEMP and check this with Test-Path everything is working fine (wether the directory exists or not). But on the next part, when I use the same technique for the arguments-parameter $PARAM_DESTINATION the statement-result is wrong (I already found the hint for adding single-quotes - after this, at least the scripts runs through without errors).
When I test my test-output in console in the Windows-Explorer, it finds the directory (thus it definitely exists)
here's the content of my script BeforeInstallationScript2.ps1:
#
# BeforeInstallationScript2.ps1
#
param(
[Parameter(
Mandatory=$true,
Position=0,
HelpMessage='Path to targetDir')]
[string] $PARAM_DESTINATION
)
"1st Argument: $PARAM_DESTINATION"
# create temporary directory if not exists
$PARAM_TEMP=$env:TEMP+"\MyApp"
"temporary Directory: $PARAM_TEMP"
if(!(Test-Path -Path "$PARAM_TEMP" )){
"temp-dir does NOT exist and will be created"
New-Item -ItemType directory -Path $PARAM_TEMP
} else {
"temp-dir exist"
}
# create Timestamp-variable for saving configs
$a = Get-Date
$DATETIME= "" + $a.Year + $a.Month + $a.Day + $a.Hour + $a.Minute
"Timestamp: $DATETIME"
# if there exists already a myApp-Installation, copy config-files
"Parameter-Path=: $PARAM_DESTINATION"
"exists? = " + (Test-Path "'$PARAM_DESTINATION'" )
if((Test-Path -Path "'$PARAM_DESTINATION'" )) {
"param-path exists"
if((Test-Path -Path "'$PARAM_DESTINATION\configuration\MyApp.conf'" )) {
"copy file to $PARAM_TEMP\$DATETIME-MyApp.conf"
Copy-Item "$PARAM_DESTINATION\configuration\MyApp.conf" "$PARAM_TEMP\$DATETIME-MyApp.conf"
}
} else {
"not existing, no files to copy/save"
}
I this script in powershell get the output as follows:
PS X:\MyApp-Setup> C:\Windows\system32\WindowsPowerShell\v1.0\powershell.exe -Version 2.0 -NoProfile -NonInteractive -InputFormat None -ExecutionPolicy Bypass -File ".\BeforeInstallationScript2.ps1" "C:\Program Files (x86)\Internet Explorer"
1st Argument: C:\Program Files (x86)\Internet Explorer
temporary Directory: C:\Users\ixmid\AppData\Local\Temp\MyApp
temp-dir does NOT exist and will be created
Verzeichnis: C:\Users\USER\AppData\Local\Temp
Mode LastWriteTime Length Name
---- ------------- ------ ----
d---- 16.11.2016 11:01 MyApp
Timestamp: 20161116111
Parameter-Path=: C:\Program Files (x86)\Internet Explorer
exists? = False
not existing, no files to copy/save
PS X:\MyApp-Setup>
As you can see, first Test-Path works fine, and creates the missing directory. But at the second part it doesn't work fine.
Any suggestions, why the second (and third) Test-Path-statements are working wrong?
For completion: second output of executing the script (when MyApp-directory now exists) looks as follows:
1st Argument: C:\Program Files (x86)\Internet Explorer
temporary Directory: C:\Users\USER\AppData\Local\Temp\MyApp
temp-dir exist
Timestamp: 201611161113
Parameter-Path=: C:\Program Files (x86)\Internet Explorer
exists? = False
not existing, no files to copy/save
You use both " quotes and ' quotes. If you want variables, $PARAM_DESTINATION in this case, to expand you need to use double quotes only. So: "$PARAM_DESTINATION".
Read up more on double and single quotes
http://windowsitpro.com/blog/single-quotes-vs-double-quotes-powershell-whats-difference
I use a similar script to generate and move files into folders.
$ToFolder = "$env:USERPROFILE\Desktop\to"
$FromFolder = "$env:USERPROFILE\Desktop\From"
#Create the sample folder on your desktop
#This line can be commented out if your ToFolder exists
New-Item $ToFolder -ItemType directory -Force
GCI -Path $FromFolder *.torrent | % {
if ($_.Name -match "(19|20)\d{2}") {
#Check to see if year folder already exists at the destination
#If not then create a folder based on this year
if (!(Test-Path "$ToFolder\$($Matches[0])")) {
New-Item -Path "$ToFolder\$($Matches[0])" -ItemType directory
}
#Transfer the matching file to its new folder
#Can be changed to Move-Item if happy with the results
Copy-Item -Path $_.FullName -Destination "$ToFolder\$($Matches[0])" -Force
}
}
but in my case I want move PDF and different files name and I don't know how solve. Key is title like "Il Corriere dello Sport" withouth suffixes like 02-08-2016 and without -.
Il_Corriere_dello_Sport_SICILIA_-_02-08-2016HQ
Il_Corriere_dello_Sport_STADIO_-_02-08-2016HQ
Il_Corriere_di_Arezzo_-_31-08-2016MQ
Il_Giornale_Di_Vicenza_-_23-08-2016
Il_Mattino_di_Padova_-_23-08-2016
Il_Messaggero_-_02-08-2016
Il_Messaggero_-_23-08-2016
Il__Messaggero_Veneto_-_31-08-2016HQ
Il__Tirreno_-_31-08-2016HQ
Il_Centro_-_30-08-2016
Il_Centro_CHIETI_-_23-08-2016HQ
So I need create folders like
Il_Corriere_dello_Sport_SICILIA
Il_Corriere_di_Arezzo
Il_Giornale_Di_Vicenza
Il_Mattino_di_Padova
Il_Messaggero
Il__Tirreno
P.S: underscore sign _ is not necessary so I prefer replace with space.
and then script must move files relative folders. Finally result should be
├─Il Messaggero [folder]
│ ├─Il_Messaggero_-_02-08-2016 [file]
│ └─Il_Messaggero_-_23-08-2016 [file]
├─Il Messaggero Veneto [folder]
│ └─Il__Messaggero_Veneto_-_31-08-2016HQ [file]
:
Here is a pure batch-file solution -- see all the explanatory remarks (rem):
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem // Define constants here:
set "SPLITCHAR=-" & rem // (a single character to split the file names)
set "SEARCHSTR=_" & rem // (a certain string to be replaced by another)
set "REPLACSTR= " & rem // (a string to replace all found search strings)
set "OVERWRITE=" & rem // (set to non-empty value to force overwriting)
rem // Get file location and pattern from command line arguments:
set "LOCATION=%~1" & rem // (directory containing files to process)
set "PATTERNS=%~2" & rem // (file pattern; match all files if empty)
rem /* Prepare overwrite flag (if defined, set to character forbidden
rem in file names; this affects later check for file existence): */
if defined OVERWRITE set "OVERWRITE=|"
rem // Continue only if location is given:
if defined LOCATION (
rem // Change current working directory to given location:
pushd "%LOCATION%" && (
rem // Loop through all files matching the given pattern:
for /F "eol=| delims=" %%F in ('dir /B "%PATTERNS%"') do (
rem // Process each file in a sub-routine:
call :PROCESS "%%F" "%SPLITCHAR%" "%SEARCHSTR%" "%REPLACSTR%"
)
rem // Restore former working directory:
popd
)
)
endlocal
exit /B
:PROCESS
rem // Retrieve first argument of sub-routine:
set "FILE=%~1"
rem // Split name at (first) split character and get portion in front:
for /F "delims=%~2" %%E in ("%~1") do (
rem // Append a split character to partial name:
set "FOLDER=%%E%~2"
)
setlocal EnableDelayedExpansion
rem // Right-trim partial name:
if not "%~3"=="" set "FOLDER=!FOLDER:%~3%~2=!"
set "FOLDER=!FOLDER:%~2=!"
rem /* Check whether partial name is not empty
rem (could happen if name began with split character): */
if defined FOLDER (
rem // Replace every search string with another:
if not "%~3"=="" set "FOLDER=!FOLDER:%~3=%~4!"
rem // Create sub-directory (surpress error if it already exists):
2> nul md "!FOLDER!"
rem /* Check if target file already exists; if overwrite flag is
rem set (to an invalid character), the target cannot exist: */
if not exist "!FOLDER!\!FILE!%OVERWRITE%" (
rem // Move file finally (surpress `1 file(s) moved.` message):
1> nul move /Y "!FILE!" "!FOLDER!"
)
)
endlocal
exit /B
The script requires the directory containing all the files to process as the first command line argument. The created sub-directories are placed therein. An optional second command line argument defines a file name pattern to filter certain file types/names. Supposing it is saved as D:\Script\build-folder-hierarchy.bat, the files are contained in D:\Data, and you want to handle *.pdf files only, run it as follows:
"D:\Script\build-folder-hierarchy.bat" "D:\Data" "*.pdf"
This is a very similar approach, but with a slightly different directory handling:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem // Define constants here:
set "SPLITCHAR=-" & rem // (a single character to split the file names)
set "SEARCHSTR=_" & rem // (a certain string to be replaced by another)
set "REPLACSTR= " & rem // (a string to replace all found search strings)
set "OVERWRITE=" & rem // (set to non-empty value to force overwriting)
rem // Get file location and pattern from command line arguments:
set "LOCATION=%~1" & rem // (directory to move the processed files into)
set "PATTERNS=%~2" & rem // (file pattern; match all files if empty)
rem /* Prepare overwrite flag (if defined, set to character forbidden
rem in file names; this affects later check for file existence): */
if defined OVERWRITE set "OVERWRITE=|"
rem // Continue only if target location is given:
if defined LOCATION (
rem // Create target location (surpress error if it already exists):
2> nul md "%LOCATION%"
rem /* Loop through all files matching the given pattern
rem in the current working directory: */
for /F "eol=| delims=" %%F in ('dir /B "%PATTERNS%"') do (
rem // Process each file in a sub-routine:
call :PROCESS "%%F" "%LOCATION%" "%SPLITCHAR%" "%SEARCHSTR%" "%REPLACSTR%"
)
)
endlocal
exit /B
:PROCESS
rem // Retrieve first argument of sub-routine:
set "FILE=%~1"
rem // Split name at (first) split character and get portion in front:
for /F "delims=%~3" %%E in ("%~1") do (
rem // Append a split character to partial name:
set "FOLDER=%%E%~3"
)
setlocal EnableDelayedExpansion
rem // Right-trim partial name:
if not "%~4"=="" set "FOLDER=!FOLDER:%~4%~3=!"
set "FOLDER=!FOLDER:%~3=!"
rem /* Check whether partial name is not empty
rem (could happen if name began with split character): */
if defined FOLDER (
rem // Replace every search string with another:
if not "%~4"=="" set "FOLDER=!FOLDER:%~4=%~5!"
rem // Create sub-directory (surpress error if it already exists):
2> nul md "%~2\!FOLDER!"
rem /* Check if target file already exists; if overwrite flag is
rem set (to an invalid character), the target cannot exist: */
if not exist "%~2\!FOLDER!\!FILE!%OVERWRITE%" (
rem // Move file finally (surpress `1 file(s) moved.` message):
1> nul move /Y "!FILE!" "%~2\!FOLDER!"
)
)
endlocal
exit /B
This script uses the current working directory to find the files to process. It requires the target directory as the first command line argument, where the created sub-directories are placed in. An optional second command line argument defines a file name pattern to filter certain file types/names. Supposing it is saved as D:\Script\build-folder-hierarchy.bat, the files are contained in D:\Data and need to be moved to D:\Target, and you want to handle *.pdf files only, run it as follows:
cd /D "D:\Data"
"D:\Script\build-folder-hierarchy.bat" "D:\Target" "*.pdf"
Another solution is to use this powershell script
$pdfs = Get-ChildItem -Filter "*.pdf"
foreach ($pdf in $pdfs) {
# Extract the journal name from the file name
$journalName = ($pdf.Name -split "_-_")[0] -replace "_", " "
# Create a folder with the name of the journal if it doesn't already exist
$targetFolder = "$journalName"
if (!(Test-Path $targetFolder)) {
New-Item -ItemType Directory -Path $targetFolder
}
# Move the file to the journal folder
Move-Item $pdf.FullName $targetFolder
}
I´ve a small problem i didnt understand...
i want to fire-up robocopy with source/target information from a variable.
$source = $TDU2Backup_Path+$dir_array[$i]+$file_array[$i]
$target = $TDU2Unpacked_Path+$dir_array[$i]+$file_array[$i]
when i print the var. content with write-host everything is fine, i get something like that:
D:\spiele\TDU2Community\Euro\Bnk\islands\ibiza\level\area-1\area-1-3\sector-1-3-4-2.bnk
D:\spiele\TDU2Community\Backup\gas-station-xandernl\Euro\Bnk\islands\ibiza\level\area-1\area-1-3\sector-1-3-4-2.bnk
but, if i use the var. with robocopy (or xcopy or something like that), i have always a backslash at the end
---------------------------------------------------------------------------
Quelle : D:\spiele\TDU2Community\Euro\Bnk\islands\ibiza\level\area-1\area-1-3\sector-1-3-4-2.bnk\
Ziel : D:\spiele\TDU2Community\Backup\gas-station-by-xandernl\Euro\Bnk\islands\ibiza\level\area-1\area-1-3\sector-1-3-4-2.bnk\
Dateien : * . *
Optionen: * . * /DCOPY:DA /COPY:DAT /R:1000000 /W:30
Here is a screenshot: