Copy-Item not including subfolder files of source directory in PowerShell - powershell

I have a script which moves files with a specific extension from a source directory to a destination directory.
My source directory looks like:
FILESFOLDER
File1.td
File2.td
SUBFOLDER
File3.td
File4.td
My script is short and looks like:
if(!(Test-Path $SourceDirPath) -or !(Test-Path $DestinationDirPath))
{
Write-Host "The source- or destination path is incorrect, please check "
break
}else
{
Write-Host "Success"
Copy-Item -path $SourceDirPath -Filter "*$extension" -Destination $DestinationDirPath -Recurse
}
I have to mention that the $SourceDirPath comes from a config file and only works if I declare it as C:\FILESFOLDER\*
The script works but doesn't copy the files from the subfolder to destination. The destination only has File1 and File2.
What's wrong with my Copy-Item command?

The -Recurse switch has no effect in your case, because it's only applied to the items matched by the combination of $SourceDirPath, which uses a trailing \* to match the items in the source directory, and -Filter, which in your case are only the *.td files located directly in your source directory.
Omitting the trailing \* to target the directory itself (e.g., C:\FOLDER rather than C:\FOLDER\*), solves that problem in principle, but, due to an annoying quirk only works as intended if the destination directory does not exist yet.
If it does exist, the items are placed in a subdirectory of the destination directory, named for the source directory.
If there's nothing in the existing destination directory to preserve before copying (and no special attributes of directory itself need preserving, you can work around the problem by deleting the preexisting destination directory beforehand.
if(!(Test-Path $SourceDirPath) -or !(Test-Path $DestinationDirPath))
{
Write-Host "The source or destination path is incorrect, please check "
break
}
else
{
Write-Host "Success"
# Delete the preexisting destination directory,
# which is required for the Copy-Item command to work as intended.
# BE SURE THAT IT IS OK TO DO THIS.
Remove-Item $DestinationDirPath -Recurse
# Note: $SourceDirPath must NOT end in \*
Copy-Item -Recurse -LiteralPath $SourceDirPath -Filter "*$extension" -Destination $DestinationDirPath
}
If you do need to preserve existing $DestinationDirPath content or attributes (ACLs, ...), more work is needed.

So this will compare the directory source versus the destination and then copy over the content. Ignore my funky variables, I've amended them below on a edit
#Release folder
$Source = ""
#Local folder
$Destination = ""
#Find all the objects in the release
get-childitem $Source -Recurse | foreach {
$SrcFile = $_.FullName
$SrcHash = Get-FileHash -Path $SrcFile -Algorithm MD5 # Obtain hash
$DestFile = $_.Fullname -replace [RegEx]::escape($Source),$Destination #Escape the hash
Write-Host "comparing $SrcFile to $DestFile" -ForegroundColor Yellow
if (Test-Path $DestFile)
{
#Check the hash sum of the file copied
$DestHash = Get-FileHash -Path $DestFile -Algorithm MD5
#compare them up
if ($SrcHash.hash -ne $DestHash.hash) {
Write-Warning "$SrcFile and $DestFile Files don't match!"
Write-Warning "Copying $SrcFile to $DestFile "
Copy-Item $SrcFile -Destination $DestFile -Force
}
else {
Write-Host "$SrcFile and $DestFile Files Match" -ForegroundColor
Green
}
}
else {
Write-host "$SrcFile is missing! Copying in" -ForegroundColor Red
Copy-Item $SrcFile -Destination $DestFile -Force
}
}

You should use the Get-ChildItem cmdlet with the -recurse parameter to retrieve all files based on your filter condition. Then you can pipe the result to the Copy-Item cmdlet and only have to specify a destination.

Try this:
$source = "C:\Folder1"
$destination = "C:\Folder2"
$allfiles = Get-ChildItem -Path $source -Recurse -Filter "*$extension"
foreach ($file in $allfiles){
if (test-path ($file.FullName.replace($source,$destination))) {
Write-Output "Seccess"
Copy-Item -path $file.FullName -Destination ($file.FullName.replace($source,$destination))
}
else {
Write-Output "The source- or destination path is incorrect, please check "
break
}
}

Related

How to backup these files into specific folders using powershell

I've finally have given up googling and come here out of desperation. Go easy on me I'm fairly new to Powershell.
So, the objective of the code below was to first look through the source folder, then read through each .zip file and move to the directory specified by the value in the hashtable. Unfortunately, this is not how they want it to work anymore.
Now I need to retain the parent folder from source: for example "DAL" and then create the proceeding folders based on the file names and finally move each .zip to its file specified folder. Also, it needs to go through each folder under source which will be at least 20 other folders with a unique 3 character names.
$srcRoot = "C:\Cloud\source\dal"
$dstRoot = "C:\Cloud\Destination"
##$map = #{}; dir -recurse | ? { !$_.psiscontainer} | % { ##$map.add($_.name,$_.PSChildName) }
# DAT and DEV will have to be excluded from folder creation
$map = {
#AEODDAT_201901 = "AEOD\2019\01"
#AEOMDEV_201902 = "AEOM\2019\01"
#AEOYDAT_201902 = "AEOY\2019\01"
}
$fileList = Get-ChildItem -Path $srcRoot -Filter "*.zip*" -File -Force -Recurse
foreach ($file in $fileList)
{
#Go through each file up to mapped string
$key = $file.BaseName.Substring(0,14)
if ($key -in $map.Keys)
{
$fileName = $file.Name
$dstDir = Join-Path -Path $dstRoot -ChildPath $map[$key]
#create direcotory if not in path
if (-not (Test-Path -Path $dstDir))
{
mkdir -Path $dstDir
}
Write-Verbose "Moving $($file.FullName)"
if (Test-Path -Path (Join-Path -Path $dstDir -ChildPath $fileName))
{
#Write error if name exists
Write-Error -Message "File $fileName already exists at $dstDir"
#move path
} else {
Move-Item -Path $($file.FullName) -Destination $dstDir
}
}
}
So C:\Cloud\source\DAL\AEODDAT20190101.zip should create folders in C:\Cloud\Destination\DAL\AEOD\2019\01\AEODDAT20190101.zip would be my desired output.
Welcome, Matt! (no pun intended) One of the habits I have in similar situations with destination folders is to Set-Location $dstRoot and create folders from the relative path. You can execute New-Item with the relative path and the syntax is simpler. For example, your If statement could look like this and it would work the same way (with a slightly different error message):
if ($key -in $map.Keys){
Set-Location $dstRoot
New-Item -ItemType Directory $map[$key] -ErrorAction Ignore #won't raise an error if it exists
Write-Verbose "Moving $($file.FullName)"
#this will raise an error if the file already exists, unless you specify -Force
Move-Item "$($file.FullName)" $map[$key]
}
EDIT: Found 2 issues.
$map is a Hashtable literal that should be preceded with #:
$map = #{
AEODDAT20190101 = "AEOD\2019\01"
You were missing the last character of the base file name by taking only the first 14 characters. AEODDAT2019010 didn't match AEODDAT20190101. This should fix it:
$key = $file.BaseName.Substring(0,15)

Avoiding File Duplication before Copying files to the folder and Subfolder

I am trying to create a code on Powershell that will actually Copy files from one Location( Lets say A) to location B. Now Location B have two subfolders (lets say X and Y). I need to copy the file from A to B but before copying I need to make sure that the files which I am copying should not be there in X or Y in order to avoid file duplication. If the file exist, it should not copy that particular file.
$PathS = Get-ChildItem -Path "\\sc-y-ap-swt-1\AutoClientFiles\reception\*.txt" |
Where-Object { $_.CreationTime -gt (Get-Date).AddDays(-1) }
$PathD = "C:\OCM\data\EverestSwift\inbound\"
$pathtest = Get-ChildItem -path "C:\OCM\data\EverestSwift\inbound\" -Recurse -File
If((Test-Path -Path "\\sc-y-ap-swt-1\AutoClientFiles\reception\*.txt") -eq $false) {
Exit
} Else {
Try {
Foreach ($File in $Pathtest){
if ($File -eq $PathS ){
Write-Host "Duplicate Files"
exit 1
}
Copy-Item -Path $PathS -Destination $PathD -Force
Exit 0
}
} catch [Exception]{
Write-Host $_.Exception.Message
Exit 1
}
}
You can do this, but why. As Cory said, this is why robocopy exists.
What do you mean by same?
The filename can be the same, but the timestamps can be different, thus making it a different file, even if the name is the same. So, you should be looking at name and timestamp or file hashes.
So, see these Q&A about such a use case.
Does Robocopy SKIP copying existing files by default?
How to skip existing and/or same size files when using robocopy
RoboCopy "%%F" %destination% *.srt *.pdf *.mp4 *.jpg /COPYALL /XO /R:0
Yet, doing this with powerShell, your post could be a duplicate of this one.
Copy items from Source to Destination if they don't already exist
Examples from the above:
$Source = 'C:\SourceFolder'
$Destination = 'C:\DestinationFolder'
Get-ChildItem $Source -Recurse | ForEach {
$ModifiedDestination = $($_.FullName).Replace("$Source","$Destination")
If ((Test-Path $ModifiedDestination) -eq $False) {
Copy-Item $_.FullName $ModifiedDestination
}
}
# Or
$Source = '<your path here>'
$Dest = '<your path here>'
$Exclude = Get-ChildItem -recurse $Dest
Get-ChildItem $Source -Recurse -Filter '*' |
Copy-Item -Destination $Dest -Verbose -Exclude $Exclude

How to do custom file copy in Powershell?

Below power shell script will copy and replace all the contents from source to destination.
Copy-Item -Path $sourcePath -Destination $installationPath -Recurse -Force
This looks simple. But in my case, I need to implement a custom file copy logic.
Logging everything about files being copied.
Skipping certain set of folders.
Sample script:
[Array]$files=Get-ChildItem ($sourceDirectoryPath) -Recurse | Format-Table Name, Directory, FullName
for($i=0;$i -le $files.Length-1;$i++)
{
. . . . . .
# Build destination path based on the source path.
# Check and create folders if it doesn't exists
# Add if cases to skip certain parts.
Copy-Item -Force $sourcePath -Destination $destinationPath
}
Any ideas on how to achieve this? Any other ideas also will help.
Thanks.
Something like this:
$ErrorActionPreference = "Stop"
Set-StrictMode -Version Latest
$sourceDir = "c:\temp\source"
$targetDir = "c:\temp\target"
$skipFiles = #(
"skip.me",
"and.me"
)
Get-ChildItem -Path $sourceDir -Recurse | ForEach-Object {
# ignore folders
if ($_.PSIsContainer)
{
return
}
# skip this file?
if ($skipFiles -contains $_.Name)
{
Write-Verbose "Skipping '$_.FullName'"
return
}
# create target folder when needed
$path = $_.DirectoryName -replace "^$([RegEx]::Escape($sourceDir))",$targetDir
if (!(Test-Path $path))
{
Write-Verbose "Creating target path '$path'..."
New-Item -Path $path -ItemType Directory
}
# copy the file
Write-Verbose "Copying '$_.FullName' to '$path'..."
Copy-Item $_.FullName $path | Out-Null
}

Move-item "the path is not supported."

We have a script which is successfully compressing folders to 7z and then deleting the folder once it has been compressed.
What we would like to do is move the compressed 7z file to another location which is on cheaper storage and also for backup / offsite archiving purposes.
I've tried the below but I'm getting an error "the path is not supported."
Is anyone able to assist?
Get-ChildItem 'E:\AbleyTest\TestFolder\_Archived\*' | Where-Object {
$_.PSIsContainer
} | Select-Object -Expand FullName | ForEach-Object {
& 7z.exe a -mx=9 -t7z "$_.7z" "$_"
if ($LastExitCode -eq 0) {
$folder = (Get-Item $_).Parent.Parent.Name
if (Test-Path "e:\archived\$folder") {
Move-Item -Path "$_.7z" -Destination "e:\archived\$folder\$_.7z"
} else {
New-Item "e:\archived\$folder" -Type directory
Move-Item -Path "$_.7z" -Destination "e:\archived\$folder\$_.7z"
}
Remove-Item -Path $_ -Force -Recurse
} else {
Add-Content "e:\scripts\archivelog $(get-date -f dd-MM-yyyy).txt" "$_ ran into error $LastExitCode while archiving"
}
}
Your destination path is not valid. Consider that $_ is the full path to some folder (such as E:\AbleyTest\TestFolder\_Archived\SomeFolder, your destination path of:
"e:\archived\$folder\$_.7z"
suddenly becomes:
e:\archived\TestFolder\E:\AbleyTest\TestFolder\_Archived\SomeFolder.7z
Edit: I just realized that I told you the problem, but didn't really help you solve it. The destination path doesn't need the name of the file (it retains the file name to use at the destination), so simply stop after $folder, and you should be just fine:
Move-Item -Path "$_.7z" -Destination "e:\archived\$folder"

Copy items from Source to Destination if they don't already exist

I have a pretty basic powershell copy script that copies items from a source folder to a destination folder. However this is moving way too much data, and I'd like to check if the filename already exists so that file can be ignored. I don't need this as complex as verifying created date/checksum/etc.
Currently it's along the lines of:
Copy-Item source destination -recurse
Copy-Item source2 destination2 -recurse
I'd imagine I need to add the Test-Path cmdlet, but I'm uncertain how to implement it.
You could always call ROBOCOPY from PowerShell for this.
Use the /xc (exclude changed) /xn (exclude newer) and /xo (exclude older) flags:
robocopy /xc /xn /xo source destination
This will ONLY copy those files that are not in the destination folder.
For more option type robocopy /?
$exclude = Get-ChildItem -recurse $dest
Copy-Item -Recurse $file $dest -Verbose -Exclude $exclude
While I agree that Robocopy is the best tool for something like this, I'm all for giving the customer what they asked for and it was an interesting PowerShell exercise.
This script should do just what you asked for: copy a file from Source to Destination only if it does not already exist in the Destination with a minimum of frills. Since you had the -recurse option in your example, that made for a bit more coding than just simply testing for the filename in the Destination folder.
$Source = "C:\SourceFolder"
$Destination = "C:\DestinationFolder"
Get-ChildItem $Source -Recurse | ForEach {
$ModifiedDestination = $($_.FullName).Replace("$Source","$Destination")
If ((Test-Path $ModifiedDestination) -eq $False) {
Copy-Item $_.FullName $ModifiedDestination
}
}
Building off of Wai Ha Lee's post, here's an example that worked for me:
$Source = "<your path here>"
$Dest = "<your path here>"
$Exclude = Get-ChildItem -recurse $Dest
Get-ChildItem $Source -Recurse -Filter "*.pdf" | Copy-Item -Destination $Dest -Verbose -Exclude $Exclude
This builds a list to exclude, then copies any pdf in the source directory and sub-directories to the destination in a single folder...excluding the existing files. Again, this is an example from my needs, but similar to yours. Should be easy enough to tweak to your hearts content.
Function Copy-IfNotPresent will accept one file at a time but it's easy to loop for all files you want to copy. Here's an example:
gci c:\temp\1\*.* -Recurse -File | % { Copy-IfNotPresent -FilePath $_ -Destination "C:\temp\2\$(Resolve-Path $_ -relative)" -Verbose }
Here's the function. It will generate the folder tree if necessary. Here's the gists link: https://gist.github.com/pollusb/cd47b4afeda8edbf8943a8808c880eb8
Function Copy-IfNotPresent {
<#
Copy file only if not present at destination.
This is a one file at a time call. It's not meant to replace complex call like ROBOCOPY.
Destination can be a file or folder. If it's a folder, you can use -Container to force Folder creation when not exists
#>
[CmdletBinding()]
Param (
[Parameter(Mandatory)]
$FilePath,
[Parameter(Mandatory)]
[string]$Destination,
[switch]$Container,
[switch]$WhatIf
)
#region validations
if ($FilePath -isnot [System.IO.FileInfo]){
$File = Get-ChildItem $FilePath -File
} else {
$File = $FilePath
}
if (!$File.Count){
Write-Warning "$FilePath no file found."
return
} elseif ($File.Count -gt 1) {
Write-Warning "$FilePath must resolve to one file only."
return
}
#endregion
# Destination is a folder
if ($Container -or (Test-Path -Path $Destination -PathType Container)) {
if (!(Test-Path $Destination)) {
New-Item -Path $Destination -ItemType Container | Out-Null
}
$Destination += "\$($File.Name)"
}
# Destination is a file
if (!(Test-Path $Destination)) {
if ($WhatIf) {
Write-Host "WhatIf:Copy-IfNotPresent $FilePath -> $Destination"
} else {
# Force creation of parent folder
$Parent = Split-Path $Destination -Parent
if (!(Test-Path $Parent)) {
New-Item $Parent -ItemType Container | Out-Null
}
Copy-Item -Path $FilePath -Destination $Destination
Write-Verbose "Copy-IfNotPresent $FilePath -> $Destination (is absent) copying"
}
} else {
Write-Verbose "Copy-IfNotPresent $Destination (is present) not copying"
}
}
$source = "c:\source"
$destination = "c:\destination"
Create a list of files to exclude, i.e. files already existing in the destination.
$exclude = Get-Childitem -Recurse $destination | ForEach-Object { $_.FullName -replace [Regex]::Escape($destination ), "" }
Recursively copy all contents from the source to the destination excluding the previously collected files.
Copy-Item -Recurse -Path (Join-Path $source "*") -Destination $destination -Exclude $exclude -Force -Verbose
(Join-Path $source "*") add a wildcard at end ensuring that you get the children of the source folder instead of the source folder itself.
Force is used because I don't mind that there are already existing folders (results in error messages). Use with caution.
ForEach-Object { $_.FullName -replace [Regex]::Escape($destination ), "" } transforms the existing file full names into values which can be used as Exclude parameter
Here is a recursive script that syncronizes 2 folders ignoring existing files:
function Copy-FilesAndFolders([string]$folderFrom, [string]$folderTo) {
$itensFrom = Get-ChildItem $folderFrom
foreach ($i in $itensFrom)
{
if ($i.PSIsContainer)
{
$subFolderFrom = $folderFrom + "\" + $i.BaseName
$subFolderTo = $folderTo + "\" + $i.BaseName
Copy-FilesAndFolders $subFolderFrom $subFolderTo | Out-Null
}
else
{
$from = $folderFrom + "\" + $i.Name
$to = $folderTo + "\" + $i.Name
if (!(Test-Path $from)) # only copies non-existing files
{
if (!(Test-Path $folderTo)) # if folder doesn't exist, creates it
{
New-Item -ItemType "directory" -Path $folderTo
}
Copy-Item $from $folderTo
}
}
}
}
To call it:
Copy-FilesAndFolders "C:\FromFolder" "C:\ToFolder"
Lots of great answers in here, here's my contribution as it relates to keeping an mp3 player in sync with a music library.
#Tom Hubbard, 10-19-2021
#Copy only new music to mp3 player, saves time by only copying items that don't exist on the destination.
#Leaving the hardcoded directories and paths in here, sometimes too much variable substitution is confusing for newer PS users.
#Gets all of the albums in the source directory such as your music library
$albumsInLibrary = gci -Directory -path "C:\users\tom\OneDrive\Music" | select -ExpandProperty Name
#Gets all of the albums of your destination folder, such as your mp3 player
$albumsOnPlayer = gci -Directory -Path "e:\" | select -ExpandProperty name
#For illustration, it will list the differences between the music library and the music player.
Compare-Object -DifferenceObject $albumsInLibrary -ReferenceObject $albumsOnPlayer
#Loop through each album in the library
foreach ($album in $albumsInLibrary)
{
#Check to see if the music player contains this directory from the music library
if ($albumsOnPlayer -notcontains $album)
{
#If the album doesn't exist on the music player, copy it and it's child items from the library to the player
write-host "$album is not on music player, copying to music player" -ForegroundColor Cyan
Copy-Item -path "C:\users\Tom\OneDrive\music\$album" -Recurse -Destination e:\$album
}
}