How to do custom file copy in Powershell? - powershell

Below power shell script will copy and replace all the contents from source to destination.
Copy-Item -Path $sourcePath -Destination $installationPath -Recurse -Force
This looks simple. But in my case, I need to implement a custom file copy logic.
Logging everything about files being copied.
Skipping certain set of folders.
Sample script:
[Array]$files=Get-ChildItem ($sourceDirectoryPath) -Recurse | Format-Table Name, Directory, FullName
for($i=0;$i -le $files.Length-1;$i++)
{
. . . . . .
# Build destination path based on the source path.
# Check and create folders if it doesn't exists
# Add if cases to skip certain parts.
Copy-Item -Force $sourcePath -Destination $destinationPath
}
Any ideas on how to achieve this? Any other ideas also will help.
Thanks.

Something like this:
$ErrorActionPreference = "Stop"
Set-StrictMode -Version Latest
$sourceDir = "c:\temp\source"
$targetDir = "c:\temp\target"
$skipFiles = #(
"skip.me",
"and.me"
)
Get-ChildItem -Path $sourceDir -Recurse | ForEach-Object {
# ignore folders
if ($_.PSIsContainer)
{
return
}
# skip this file?
if ($skipFiles -contains $_.Name)
{
Write-Verbose "Skipping '$_.FullName'"
return
}
# create target folder when needed
$path = $_.DirectoryName -replace "^$([RegEx]::Escape($sourceDir))",$targetDir
if (!(Test-Path $path))
{
Write-Verbose "Creating target path '$path'..."
New-Item -Path $path -ItemType Directory
}
# copy the file
Write-Verbose "Copying '$_.FullName' to '$path'..."
Copy-Item $_.FullName $path | Out-Null
}

Related

CSV - Piping to Copy-Item

When I try to import a CSV, and take a source filename/path and destination folder ref, copy-item seems to not copy the file in question.
I have a folder full of files in C:\Dir1\Test\Files\ and I need to copy them to individual folders in C:\Dir1\Test, based on what is in the csv.
$SourceDir = 'C:\Dir1\Test\Files\'
$DestDir = 'C:\Dir1\Test\'
Import-Csv C:\Dir1\Test\FileList.csv | ForEach-Object {
$Source = $SourceDir + $($_.'FilePath')
$Dest = $DestDir + "$($_.'Folder Ref')\"
Copy-Item $Source -Destination $Dest
}
If I switch out the Copy-Item to Write-Host, it reads to me correctly, am I doing something wrong?
Nothing happens, it returns me to the prompt with no output
Constructing file paths using string concatenation as you are doing is never a good idea..
Better use PowerShells cmdlet Join-Path for that or .Net [System.IO.Path]::Combine() method.
As mklement0 already commented, Copy-Item by default does not procude any visual output unless you add -Verbose.
You can also append switch -PassThru and in that case, the cmdlet returns an object that represents the copied item.
In your case, why not add an informative message yourself, something like:
$SourceDir = 'C:\Dir1\Test\Files'
$DestDir = 'C:\Dir1\Test'
Import-Csv -Path 'C:\Dir1\Test\FileList.csv' | ForEach-Object {
# construct the source path
$Source = Join-Path -Path $SourceDir -ChildPath $_.FilePath
if (Test-Path -Path $source -PathType Leaf) {
# construct the destination path
$Dest = Join-Path -Path $DestDir -ChildPath $_.'Folder Ref'
# make sure the target path exists before trying to copy to it
$null = New-Item -Path $Dest -ItemType Directory -Force
# now copy the file
Write-Host "Copying file '$Source' to '$Dest'"
Copy-Item -Path $Source -Destination $Dest
}
else {
Write-Warning "File '$Source' could not be found"
}
}

Powershell script to create a single folder based of the 5 digits in the name of the pdf's

So far I have tried the following script:
$SourceFolder = "D:\WORK\JetLetter\LKY\LKY_jV_004\"
$TargetFolder = "D:\WORK\JetLetter\LKY\LKY_jV_004\Final\"
Get-ChildItem -Path $SourceFolder -Filter *.pdf |
ForEach-Object {
$ChildPath = Join-Path -Path $_.Name.Replace('.pdf','') -ChildPath $_.Name
[System.IO.FileInfo]$Destination = Join-Path -Path $TargetFolder -ChildPath $ChildPath
if( -not ( Test-Path -Path $Destination.Directory.FullName )){
New-Item -ItemType Directory -Path $Destination.Directory.FullName
}
Copy-Item -Path $_.FullName -Destination $Destination.FullName
}
This creates a folder for every pdf in the folder.
I need it create a single folder based on the 5 digit in the name and move those files into the new folder.
For example: I could have 10 pdf's that have the number "30565" in them and the new folder should be named "30565"
Here are some file names to explain:
LKY_20974_Pr01_1-5000.pdf
to
D:\WORK\JetLetter\LKY\LKY_jV_004\Final\20974
LKY_20974_Pr02_5001-10000.pdf
to
D:\WORK\JetLetter\LKY\LKY_jV_004\Final\20974
LKY_20974_Pr03_10001-15000.pdf
to
D:\WORK\JetLetter\LKY\LKY_jV_004\Final\20974
I have tried to include an else block to the best answer script and haven't had much success. I did however create a separate script that will archive the files before creating a new file. I just have to run it before the main powershell script.
$SourceDir = 'D:\WORK\JetLetter\LKY\LKY_jV_004_9835'
$DestDir = 'D:\WORK\JetLetter\LKY\#Print_Production_Files'
$ArchiveDir = 'D:\WORK\JetLetter\LKY\#Print_Production_Files\#archive'
$Filter = '*.pdf'
$FileList = Get-ChildItem -LiteralPath $SourceDir -Filter $Filter -File
foreach ($FL_Item in $FileList)
{
# this presumes the target dir number is ALWAYS the 2nd item in the split string
$TargetDir = $FL_Item.BaseName.Split('_')[1]
$FullTargetDir = Join-Path -Path $DestDir -ChildPath $TargetDir
if (Test-Path -LiteralPath $FullTargetDir)
{
# the "$Null =" is to suppress unwanted output about what was done
$null = Move-Item -Path $FullTargetDir -Destination $ArchiveDir -Force
}
}
This has made the files and folders a lot more organized.
i think this does what you want done. [grin] the comments seem adequate, but if you have any questions, please ask.
$SourceDir = 'c:\temp\JetLetter\LKY\LKY_jv_004'
$DestDir = 'c:\temp\JetLetter\LKY\LKY_jv_004\Final'
$Filter = '*.pdf'
#region >>> make the dirs and sample files to work with
# remove the entire "#region/#endregion" block when you are ready to work with real data
# make the dirs
$Null = mkdir -Path $SourceDir, $DestDir -ErrorAction 'SilentlyContinue'
# make the test files
$SampleFiles = #(
'LKY_11111_Pr11_1-11111.pdf'
'LKY_22222_Pr22_2-22222.pdf'
'LKY_22222_Pr22_2222-2222.pdf'
'LKY_33333_Pr33_3-3333.pdf'
'LKY_33333_Pr33_33333-33333.pdf'
'LKY_55555_Pr55_5-5555.pdf'
'LKY_77777_Pr77_7-77777.pdf'
'LKY_77777_Pr77_77777-77777.pdf'
'LKY_99999_Pr99_9-99999.pdf'
)
foreach ($SF_Item in $SampleFiles)
{
# the "$Null =" is to suppress unwanted output about what was done
$Null = New-Item -Path $SourceDir -Name $SF_Item -ItemType 'File' -ErrorAction 'SilentlyContinue'
}
#endregion >>> make the dirs and sample files to work with
$FileList = Get-ChildItem -LiteralPath $SourceDir -Filter $Filter -File
foreach ($FL_Item in $FileList)
{
# this presumes the target dir number is ALWAYS the 2nd item in the split string
$TargetDir = $FL_Item.BaseName.Split('_')[1]
$FullTargetDir = Join-Path -Path $DestDir -ChildPath $TargetDir
if (-not (Test-Path -LiteralPath $FullTargetDir))
{
# the "$Null =" is to suppress unwanted output about what was done
$Null = New-Item -Path $FullTargetDir -ItemType 'Directory'
}
$NewFullFileName = Join-Path -Path $FullTargetDir -ChildPath $FL_Item.Name
# leave the file in the source dir if it already is in the final target dir
# you may want to save the not-copied info to a file for later review
if (-not (Test-Path -LiteralPath $NewFullFileName))
{
# the "Move-Item" cmdlet on win7ps5.1 is wildly unreliable
# so i used copy & then remove
$Null = Copy-Item -LiteralPath $FL_Item.FullName -Destination $NewFullFileName
Remove-Item -LiteralPath $FL_Item.FullName
}
else
{
Write-Warning (' {0} already exists in {1}' -f $FL_Item.Name, $FullTargetDir)
Write-Warning ' The file was not moved.'
Write-Warning ''
}
}
screen output only exists for "not moved" files. again, you may want to save the list to a $Var or to a file for later work.
one of the moved files ...
C:\Temp\JetLetter\LKY\LKY_jv_004\Final\22222\LKY_22222_Pr22_2222-2222.pdf

Merge two directories keeping any files with same name

I am looking for some help to create a PowerShell script to merge or copy one directory to another that the destination directory has files with the same name as the source.
I need to keep both, the script can append a number to the source file if it has a file of duplicate name in the destination.
Here is a sample script that deals with one file, but I need to set a directory and let it loose recursively on the entire directory.
$SourceFile = "C:\Temp\File.txt"
$DestinationFile = "C:\Temp\NonexistentDirectory\File.txt"
if ((Test-Path $DestinationFile) -eq $false) {
New-Item -ItemType File -Path $DestinationFile -Force
}
Copy-Item -Path $SourceFile -Destination $DestinationFile
try this
$SourceDir = "C:\Temp"
$DestinationDir = "C:\Temp2\NonexistentDirectory"
#create dir if not exists (dont remove if exist)
New-Item -ItemType Directory -Path $DestinationDir -Force
#get list files destination dir
$DestinationFiles=gci $DestinationDir -File
#loop on file source and create newname for copy while name exist already
gci $SourceDir -File | %{
$counter=0
$name=$_.Name
while ($name -in $DestinationFiles.Name)
{
$counter++;
$name="{0}_{1:d6}{2}" -f $_.BaseName, $counter, $_.Extension
}
Copy-Item -Path $_.FullName -Destination "$DestinationDir\$name"
}

Copy items from Source to Destination if they don't already exist

I have a pretty basic powershell copy script that copies items from a source folder to a destination folder. However this is moving way too much data, and I'd like to check if the filename already exists so that file can be ignored. I don't need this as complex as verifying created date/checksum/etc.
Currently it's along the lines of:
Copy-Item source destination -recurse
Copy-Item source2 destination2 -recurse
I'd imagine I need to add the Test-Path cmdlet, but I'm uncertain how to implement it.
You could always call ROBOCOPY from PowerShell for this.
Use the /xc (exclude changed) /xn (exclude newer) and /xo (exclude older) flags:
robocopy /xc /xn /xo source destination
This will ONLY copy those files that are not in the destination folder.
For more option type robocopy /?
$exclude = Get-ChildItem -recurse $dest
Copy-Item -Recurse $file $dest -Verbose -Exclude $exclude
While I agree that Robocopy is the best tool for something like this, I'm all for giving the customer what they asked for and it was an interesting PowerShell exercise.
This script should do just what you asked for: copy a file from Source to Destination only if it does not already exist in the Destination with a minimum of frills. Since you had the -recurse option in your example, that made for a bit more coding than just simply testing for the filename in the Destination folder.
$Source = "C:\SourceFolder"
$Destination = "C:\DestinationFolder"
Get-ChildItem $Source -Recurse | ForEach {
$ModifiedDestination = $($_.FullName).Replace("$Source","$Destination")
If ((Test-Path $ModifiedDestination) -eq $False) {
Copy-Item $_.FullName $ModifiedDestination
}
}
Building off of Wai Ha Lee's post, here's an example that worked for me:
$Source = "<your path here>"
$Dest = "<your path here>"
$Exclude = Get-ChildItem -recurse $Dest
Get-ChildItem $Source -Recurse -Filter "*.pdf" | Copy-Item -Destination $Dest -Verbose -Exclude $Exclude
This builds a list to exclude, then copies any pdf in the source directory and sub-directories to the destination in a single folder...excluding the existing files. Again, this is an example from my needs, but similar to yours. Should be easy enough to tweak to your hearts content.
Function Copy-IfNotPresent will accept one file at a time but it's easy to loop for all files you want to copy. Here's an example:
gci c:\temp\1\*.* -Recurse -File | % { Copy-IfNotPresent -FilePath $_ -Destination "C:\temp\2\$(Resolve-Path $_ -relative)" -Verbose }
Here's the function. It will generate the folder tree if necessary. Here's the gists link: https://gist.github.com/pollusb/cd47b4afeda8edbf8943a8808c880eb8
Function Copy-IfNotPresent {
<#
Copy file only if not present at destination.
This is a one file at a time call. It's not meant to replace complex call like ROBOCOPY.
Destination can be a file or folder. If it's a folder, you can use -Container to force Folder creation when not exists
#>
[CmdletBinding()]
Param (
[Parameter(Mandatory)]
$FilePath,
[Parameter(Mandatory)]
[string]$Destination,
[switch]$Container,
[switch]$WhatIf
)
#region validations
if ($FilePath -isnot [System.IO.FileInfo]){
$File = Get-ChildItem $FilePath -File
} else {
$File = $FilePath
}
if (!$File.Count){
Write-Warning "$FilePath no file found."
return
} elseif ($File.Count -gt 1) {
Write-Warning "$FilePath must resolve to one file only."
return
}
#endregion
# Destination is a folder
if ($Container -or (Test-Path -Path $Destination -PathType Container)) {
if (!(Test-Path $Destination)) {
New-Item -Path $Destination -ItemType Container | Out-Null
}
$Destination += "\$($File.Name)"
}
# Destination is a file
if (!(Test-Path $Destination)) {
if ($WhatIf) {
Write-Host "WhatIf:Copy-IfNotPresent $FilePath -> $Destination"
} else {
# Force creation of parent folder
$Parent = Split-Path $Destination -Parent
if (!(Test-Path $Parent)) {
New-Item $Parent -ItemType Container | Out-Null
}
Copy-Item -Path $FilePath -Destination $Destination
Write-Verbose "Copy-IfNotPresent $FilePath -> $Destination (is absent) copying"
}
} else {
Write-Verbose "Copy-IfNotPresent $Destination (is present) not copying"
}
}
$source = "c:\source"
$destination = "c:\destination"
Create a list of files to exclude, i.e. files already existing in the destination.
$exclude = Get-Childitem -Recurse $destination | ForEach-Object { $_.FullName -replace [Regex]::Escape($destination ), "" }
Recursively copy all contents from the source to the destination excluding the previously collected files.
Copy-Item -Recurse -Path (Join-Path $source "*") -Destination $destination -Exclude $exclude -Force -Verbose
(Join-Path $source "*") add a wildcard at end ensuring that you get the children of the source folder instead of the source folder itself.
Force is used because I don't mind that there are already existing folders (results in error messages). Use with caution.
ForEach-Object { $_.FullName -replace [Regex]::Escape($destination ), "" } transforms the existing file full names into values which can be used as Exclude parameter
Here is a recursive script that syncronizes 2 folders ignoring existing files:
function Copy-FilesAndFolders([string]$folderFrom, [string]$folderTo) {
$itensFrom = Get-ChildItem $folderFrom
foreach ($i in $itensFrom)
{
if ($i.PSIsContainer)
{
$subFolderFrom = $folderFrom + "\" + $i.BaseName
$subFolderTo = $folderTo + "\" + $i.BaseName
Copy-FilesAndFolders $subFolderFrom $subFolderTo | Out-Null
}
else
{
$from = $folderFrom + "\" + $i.Name
$to = $folderTo + "\" + $i.Name
if (!(Test-Path $from)) # only copies non-existing files
{
if (!(Test-Path $folderTo)) # if folder doesn't exist, creates it
{
New-Item -ItemType "directory" -Path $folderTo
}
Copy-Item $from $folderTo
}
}
}
}
To call it:
Copy-FilesAndFolders "C:\FromFolder" "C:\ToFolder"
Lots of great answers in here, here's my contribution as it relates to keeping an mp3 player in sync with a music library.
#Tom Hubbard, 10-19-2021
#Copy only new music to mp3 player, saves time by only copying items that don't exist on the destination.
#Leaving the hardcoded directories and paths in here, sometimes too much variable substitution is confusing for newer PS users.
#Gets all of the albums in the source directory such as your music library
$albumsInLibrary = gci -Directory -path "C:\users\tom\OneDrive\Music" | select -ExpandProperty Name
#Gets all of the albums of your destination folder, such as your mp3 player
$albumsOnPlayer = gci -Directory -Path "e:\" | select -ExpandProperty name
#For illustration, it will list the differences between the music library and the music player.
Compare-Object -DifferenceObject $albumsInLibrary -ReferenceObject $albumsOnPlayer
#Loop through each album in the library
foreach ($album in $albumsInLibrary)
{
#Check to see if the music player contains this directory from the music library
if ($albumsOnPlayer -notcontains $album)
{
#If the album doesn't exist on the music player, copy it and it's child items from the library to the player
write-host "$album is not on music player, copying to music player" -ForegroundColor Cyan
Copy-Item -path "C:\users\Tom\OneDrive\music\$album" -Recurse -Destination e:\$album
}
}

Copy-item Files in Folders and subfolders in the same directory structure of source server using PowerShell

I am struggling really hard to get this below script worked to copy the files in folders and sub folders in the proper structure (As the source server).
Lets say, there are folders mentioned below:
Main Folder: File aaa, File bbb
Sub Folder a: File 1, File 2, File 3
Sub Folder b: File 4, File 5, File 6
Script used:
Get-ChildItem -Path \\Server1\Test -recurse | ForEach-Object {
Copy-Item -LiteralPath $_.FullName -Destination \\server2\test |
Get-Acl -Path $_.FullName | Set-Acl -Path "\\server2\test\$(Split-Path -Path $_.FullName -Leaf)"
}
Output:
File aaa, File bbb
Sub Folder a (Empty Folder)
Sub Folder b (Empty Folder)
File 1, File 2, File 3, File 4, File 5, File 6.
I want the files to get copied to their respective folders (Like the source folders). Any further help is highly appreciated.
This can be done just using Copy-Item. No need to use Get-Childitem. I think you are just overthinking it.
Copy-Item -Path C:\MyFolder -Destination \\Server\MyFolder -recurse -Force
I just tested it and it worked for me.
edit: included suggestion from the comments
# Add wildcard to source folder to ensure consistent behavior
Copy-Item -Path $sourceFolder\* -Destination $targetFolder -Recurse
If you want to mirror same content from source to destination, try following one.
function CopyFilesToFolder ($fromFolder, $toFolder) {
$childItems = Get-ChildItem $fromFolder
$childItems | ForEach-Object {
Copy-Item -Path $_.FullName -Destination $toFolder -Recurse -Force
}
}
Test:
CopyFilesToFolder "C:\temp\q" "c:\temp\w"
one time i found this script, this copy folder and files and keep the same structure of the source in the destination, you can make some tries with this.
# Find the source files
$sourceDir="X:\sourceFolder"
# Set the target file
$targetDir="Y:\Destfolder\"
Get-ChildItem $sourceDir -Include *.* -Recurse | foreach {
# Remove the original root folder
$split = $_.Fullname -split '\\'
$DestFile = $split[1..($split.Length - 1)] -join '\'
# Build the new destination file path
$DestFile = $targetDir+$DestFile
# Move-Item won't create the folder structure so we have to
# create a blank file and then overwrite it
$null = New-Item -Path $DestFile -Type File -Force
Move-Item -Path $_.FullName -Destination $DestFile -Force
}
I had trouble with the most popular answer (overthinking). It put AFolder in the \Server\MyFolder\AFolder and I wanted the contents of AFolder and below in MyFolder. This didn't work.
Copy-Item -Verbose -Path C:\MyFolder\AFolder -Destination \\Server\MyFolder -recurse -Force
Plus I needed to Filter and only copy *.config files.
This didn't work, with "\*" because it did not recurse
Copy-Item -Verbose -Path C:\MyFolder\AFolder\* -Filter *.config -Destination \\Server\MyFolder -recurse -Force
I ended up lopping off the beginning of the path string, to get the childPath relative to where I was recursing from. This works for the use-case in question and went down many subdirectories, which some other solutions do not.
Get-Childitem -Path "$($sourcePath)/**/*.config" -Recurse |
ForEach-Object {
$childPath = "$_".substring($sourcePath.length+1)
$dest = "$($destPath)\$($childPath)" #this puts a \ between dest and child path
Copy-Item -Verbose -Path $_ -Destination $dest -Force
}
Here you go.
Function Backup-Files {
[CmdletBinding()]
Param (
[Parameter(Mandatory)]
[System.IO.FileInfo[]]$Source,
[Parameter(Mandatory)]
[String]$Destination
)
if (!(Test-Path $Destination)) {[void][System.IO.Directory]::CreateDirectory($Destination)}
ForEach ($File in $Source) {
$SourceRoot = $(Convert-Path $File.PSParentPath).split('\')[0]
$NewFile = $($File.FullName).Replace($SourceRoot,$Destination)
$NewDir = $($File.DirectoryName).Replace($SourceRoot,$Destination)
[void][System.IO.Directory]::CreateDirectory($NewDir)
Copy-Item -Path $File.FullName -Destination $NewFile -Force
}
}
Examples
<#
.SYNOPSIS
Copy FileInfo object or array to a new destination while retaining the original directory structure.
.PARAMETER Source
FileInfo object or array. (Get-Item/Get-ChildItem)
.PARAMETER Destination
Path to backup source data to.
.NOTES
Version (Date): 1.0 (2023-02-04)
Author: Joshua Biddle (thebiddler#gmail.com)
Purpose/Change: Initial script development.
Known Bugs:
.EXAMPLE
Backup-Files -Source $(Get-ChildItem -Path 'C:\Users\*\Documents' -Recurse -Force -Exclude 'My Music','My Pictures','My Videos','desktop.ini' -ErrorAction SilentlyContinue) -Destination "C:\Temp\UserBackup"
.EXAMPLE
Backup-Files -Source $(Get-ChildItem -Path 'C:\Users\*\Desktop' -Exclude "*.lnk","desktop.ini" -Recurse -Force -ErrorAction SilentlyContinue) -Destination "C:\Temp\UserBackup"
#>
I wanted a solution to copy files modified after a certain date and time which mean't I need to use Get-ChildItem piped through a filter. Below is what I came up with:
$SourceFolder = "C:\Users\RCoode\Documents\Visual Studio 2010\Projects\MyProject"
$ArchiveFolder = "J:\Temp\Robin\Deploy\MyProject"
$ChangesStarted = New-Object System.DateTime(2013,10,16,11,0,0)
$IncludeFiles = ("*.vb","*.cs","*.aspx","*.js","*.css")
Get-ChildItem $SourceFolder -Recurse -Include $IncludeFiles | Where-Object {$_.LastWriteTime -gt $ChangesStarted} | ForEach-Object {
$PathArray = $_.FullName.Replace($SourceFolder,"").ToString().Split('\')
$Folder = $ArchiveFolder
for ($i=1; $i -lt $PathArray.length-1; $i++) {
$Folder += "\" + $PathArray[$i]
if (!(Test-Path $Folder)) {
New-Item -ItemType directory -Path $Folder
}
}
$NewPath = Join-Path $ArchiveFolder $_.FullName.Replace($SourceFolder,"")
Copy-Item $_.FullName -Destination $NewPath
}