PowerShell search recursively for files and folder which the name contains "..." - powershell

I need help to create a script that can save my life.
My backup software got it wrong because I misdesigned my backup plan and I have lots of named files; filename or folder name (2) (encoding conflict).
I would like to recursively search my network share to find folders and files with "(encode conflict)" in their name and first export them for verification.
Then, if all goes well, I would like to move them to another place while keeping the folder's hierarchy and that's where I stuck.
Get-ChildItem -LiteralPath '\\?\E:\Network Shares\Commun' -Recurse -Filter "*(encode*" #| move-item -Destination 'C:\Users\Desktop\Conflits\'
For the export I found the script here :
https://stackoverflow.com/a/15261816/19493679
Thanks to OP
The files from my test are moving without folder's and file's hierarchy ...
Can you help me please ? :)

Move-Item doesn't know about the directory structure, it just gets paths fed one by one from Get-ChildItem. For preserving directory structure, Move-Item would have to know a common base path, but there is currently no way to specify one.
So I've created a helper function New-DestinationPath that can be chained in between Get-ChildItem and Move-Item (or Copy-Item). The function creates the destination directory structure and outputs the source path and the fully resolved destination path.
Function New-DestinationPath {
[CmdletBinding(SupportsShouldProcess)]
param (
[Parameter(Mandatory, ValueFromPipeline, ValueFromPipelineByPropertyName)]
[Alias('Fullname', 'PSPath')]
[String] $Path,
[Parameter(Mandatory, Position=0)]
[String] $CommonPath,
[Parameter(Mandatory, Position=1)]
[String] $Destination
)
process {
# Temporarily change current directory -> base directory for Resolve-Path -Relative
Push-Location $CommonPath
try {
# Resolve input path relative to $CommonPath (current directory)
$relativePath = Resolve-Path $Path -Relative
}
finally {
Pop-Location # Restore current directory
}
# Resolve full target file path and directory
$targetPath = Join-Path $Destination $relativePath
$targetDir = Split-Path $targetPath -Parent
# Create target dir if not already exists (-Force).
$null = New-Item $targetDir -ItemType Directory -Force
# Output the full source and destination paths (for use with Copy-Item or Move-Item)
[PSCustomObject]#{
Path = $Path
Destination = $targetPath
}
}
}
Usage:
$searchPath = '\\?\E:\Network Shares\Commun'
Get-ChildItem -LiteralPath $searchPath -Recurse -File |
Where-Object Fullname -like '*(encode*' |
New-DestinationPath -CommonPath $searchPath -Destination 'C:\Users\Desktop\Conflits' -WhatIf |
Move-Item -LiteralPath { $_.Path } -Destination { $_.Destination } -WhatIf
By using the common parameter -WhatIf the functions only output what they would do. Remove both -WhatIf arguments when you have confirmed that the right paths are used.
In the Move-Item call we are using delay-bind scriptblocks to avoid a ForEach-Object command. Using the automatic variable $_ we can directly refer to the output of the previous command in the pipeline (New-DestinationPath).

Related

Copy a file with its parent directory structure in Powershell

I am trying to create a little patcher in PowerShell which first backs up the files before replacing them. But I can't get it to cooperate when the file is in a subdirectory. To simplify:
First, it creates the backup directory:
$Null = New-Item -Path 'C:\Backup' -ItemType 'Directory'
Next, I check for the file:
[System.IO.FileInfo]$FileInfo = Get-Item -LiteralPath 'C:\App\bin\executable.exe'
Now, I want this file to end up in C:\Backup\bin\executable.exe
The first thing I tried was to get the FullName and replace C:\App with C:\Backup resulting in $FileInfo | Copy-Item -Destination 'C:\Backup\bin\executable.exe'
But it keeps throwing an exception because C:\Backup\bin does not exist. I tried combinations of the -Recurse and -Container switches.
I assume the issue is that I'm setting the full path as my destination, resulting in no relative paths being created. But I don't see another way to get that target path set. I can't tell it to copy to C:\Backup\bin because there's no logical way for me to know about bin without extracting it. At which point I might as well create it in the backup directory.
But I kind of want this to be automagic (for the C:\App\multi\level\path\file.dll at a later stage). This makes the script more flexible if I change the file locations later on.
I'm probably missing something really obvious here. Any suggestions?
This is what I'm doing now, compensating for having to make the parent directories myself. I'm open to ways to optimize and of course still looking for a way to not have to explicitly create the parent directories.
Function Backup-Item {
Param (
[Parameter(Mandatory = $True)] [System.String]$Source,
[Parameter(Mandatory = $True)] [System.String]$SourceRoot,
[Parameter(Mandatory = $False)][System.Management.Automation.SwitchParameter]$Backup = $True,
[Parameter(Mandatory = $False)][System.String]$BackupDirectory
) # Param
Begin {
If (Test-Path -LiteralPath $Source) { [System.IO.FileInfo]$SourceFile = Get-Item -LiteralPath $Source }
Else { } # TODO: Break
If ($Backup) {
If (Test-Path -LiteralPath $BackupDirectory) { } # TODO: Break
Else { $Null = New-Item -Path $BackupDirectory -ItemType 'Directory' }
[System.String]$Destination = $SourceFile.FullName.Replace($SourceRoot,$BackupDirectory)
} # If
} # Begin
Process {
If ($Backup) {
[System.String]$TargetDirectory = $SourceFile.DirectoryName.Replace($SourceRoot,$BackupDirectory)
If (-not (Test-Path -LiteralPath $TargetDirectory)) { $Null = New-Item -Path $TargetDirectory -ItemType 'Directory' }
$SourceFile | Copy-Item -Destination $Destination
} # If
} # Process
End {}
} # Function Backup-Item
I don't like having to provide the SourceRoot but it's the only way to deduce the relative paths I could think of.
I will also need to make BackupDirectory mandatory but only if Backup is True (which is the default and a bit dirty since a switch should be off by default, but I wanted the ability to do -Backup:$False to override).
I would use this as a helper script in the larger patcher-script.
Why not use robocopy to copy the entire C:\Apps folder to the C:\Backup folder, like with
robocopy C:\Apps C:\Backup /MIR
If you want pure PowerShell, you can do
$sourcePath = 'C:\Apps'
$backupPath = 'C:\Backup'
Get-ChildItem $sourcePath -File -Recurse | ForEach-Object {
# create the destination folder, even if it is empty
$targetFolder = Join-Path -Path $backupPath -ChildPath $_.DirectoryName.Substring($sourcePath.Length)
$null = New-Item -ItemType Directory -Path $targetFolder -Force
# next copy the file
$_ | Copy-Item -Destination $targetFolder
}

Powershell Variable for dynamic Move-Item

I have a function that looks at some registry settings for SQL Server which just pulls put the data and log location and puts them in variables. But I run into an issue when I pass them into Move-Item. Basically:
fnGetDataNLog
Returns $datalocation, $loglocation
When I run Move-Item -Path $datalocaton -Destination $loglocation I get
Cannot bind argument to parameter 'path' because it does not exist.
Is that due to its passing a runtime variable? Is there another way to do that then.
You can run a Test-Path to ensure the location exists.
If you $Datalocation is a UNC path to a directory you can do the below. eg. $Datalocation = '\\UNC\Path\To\Folder'
If(-not (Test-Path -Path $Datalocation)) {
New-Item -Path $Datalocation -ItemType Directory
}
Move-Item -Path $Datalocation-Destination $loglocation
This will test if the path at $Datalocation exists, if -not then it will create it.
if the UNC path is a string that goes to a file, ie. not an object result from a Get-ChildItem, then you can use a bit of regex to get the parent folder and then do the below. The regex will remove everything after the last \.
eg. \\UNC\Path\To\Folder\File.txt becomes \\UNC\Path\To\Folder\
If(-not (Test-Path -Path ($Datalocation -replace '[^\\]+$'))) {
New-Item -Path $Datalocation -ItemType Directory
}
Move-Item -Path $Datalocation -Destination $loglocation

Move File to new location where Parent Folder Name Matches

Problem
I am working on a rollback feature in my application in which I copy the files from a backup/rollback directory to a destination folder. As simple as that sounds, this is where it gets complicated. Due to all the files sharing the same or similar name, I used the parent folder as the anchor to help enforce unique locations.
I want to essentially recursively search a directory and wherever a folder name matches a parent directory of an object, paste a copy of the object into that folder, overwriting whatever file(s) share a name with said object.
A more visible way to represent this would be:
$Path = C:\Temp\MyBackups\Backup_03-14-2017
$destination = C:\SomeDirectory\Subfolder
$backups = GCI -Path "$Path\*.config" -Recursive
foreach ($backup in $backups) {
Copy-Item -Path $backup -Destination $destination | Where-Object {
((Get-Item $backup).Directory.Name) -match "$destination\*"
}
}
However, the above doesn't work and none of my research is finding anything remotely similar to what I'm trying to do.
Question
Does anyone know how to Copy an Item from one location to another in which the parent folder of the copied item matches a folder in the destination using PowerShell?
Enumerate the backed-up files, replace the source base path with the destination base path, then move the files. If you only want to replace existing files, test if the destination exists:
Get-ChildItem -Path $Path -Filter '*.config' -Recursive | ForEach-Object {
$dst = $_.FullName.Replace($Path, $destination)
if (Test-Path -LiteralPath $dst) {
Copy-Item -Path $_.FullName -Destination $dst -Force
}
}
If you want to restore files that are missing in the destination make sure to create missing directories first:
Get-ChildItem -Path $Path -Filter '*.config' -Recursive | ForEach-Object {
$dst = $_.FullName.Replace($Path, $destination)
$dir = [IO.Path]::GetDirectoryName($dst)
if (-not (Test-Path -LiteralPath $dir -PathType Container)) {
New-Item -Type Directory -Path $dir | Out-Null
}
Copy-Item -Path $_.FullName -Destination $dst -Force
}
You may be over thinking this. If you are backing up a web.config file from a website, I'd highly advise using the SiteID as the backup folder. Then simply utilize this as the means to find the right folder to copy the web.config file to when you want to rollback.
Ideally when working with any group of items (in this instance websites) try to find a unique identifier for the items. SiteIDs are ideal for this.
$Path = C:\Temp\MyBackups\Backup_03-14-2017 #In this directory store the web.config's in directories that match the SiteID of the site they belong to
#For example, if the site id was 5, then the full backup directory would be: C:\Temp\MyBackups\Backup_03-14-2017\5
$backups = Get-ChildItem -Path $Path -Include *.config -Recurse
foreach ($backup in $backups)
{
$backupId = $backup.Directory.Name
$destination = (Get-Website | where {$_.id -eq $backupId}).physicalPath
Copy-Item -Path $backup -Destination $destination
}

Using Powershell to move documents to a folder based on file name

I have searched your database for answers and find things that are similar, but not quite what I am needing. I do not know how to code well enough to take some of the examples and modify them to fit my needs.
I have documents that I need to move to a folder that has the same name as the beginning of the document name. Here are a couple of example of my document names (ParcelNum_CountyNum_Year_DocType.pdf)
188777_014_2013_NOAV.pdf
242353_227_2014_HRGNOTICE.pdf
R506275_246_2013_NOAV.pdf
I currently have the documents residing in a single folder named by the county number (the 3 digit number between the first _ and the 2nd _)
I have folders created that are named by the ParcelNum, which is the first set of characters/numbers at the beginning and before the first _. I need Powershell to read the first portion of the document name stopping at the first _ and place it in the folder that matches that numchar.
So for example the first document example above is named 188777_014_2013_NOAV.pdf and resides at the root of the CountyNum folder named 014.
For that example, I would need it to be moved from the root folder 014, into its subfolder named 188777.
I hope that makes sense. Please feel free to ask questions.
Thank you,
Here's what I would do. This is under the assumption that the files you want to move are in a parent folder and the root & sub folders you want to sort them to are also in the same folder, i.e. if the example PDF's path is C:\test\Docs\188777_014_2013_NOAV.pdf and you want it to end up in C:\test\Docs\014\188777.
Update the path in the $basePath variable to your liking. Comments are included above each step with explanations. Enjoy!
# This is the parent directory for the files and sorted folders
$basePath = "C:\test\Docs"
# This sets that folder as your CWD (Current working directory)
Set-Location $basePath
# This grabs the *files* underneath the parent directory, ignoring sub directories
$files = Get-ChildItem $basePath | Where-Object {$_.PSIsContainer -eq $false}
# Starts iterating through each file
foreach ($file in $files) {
# Split the base file name by underscore (creates an array with each part of the name seperated)
$split = $file.BaseName -split "_"
# Store the second item [1] in the array $split as the root folder name
$root = "$basePath\$($split[1])"
# Store the first item [0] in the array $split as the sub folder name
$sub = "$root\$($split[0])"
# Check if the root folder exists, create it if not
if (!(Test-Path $root -ErrorAction SilentlyContinue)) {
New-Item $root -ItemType Directory | Out-Null
}
# Check if the sub folder exists, create it if not
if (!(Test-Path $sub -ErrorAction SilentlyContinue)) {
New-Item $sub -ItemType Directory | Out-Null
}
# Move the file to the sub folder
Move-Item $file.FullName -Destination $sub -Verbose
}
Tried to solve your problem:
#Requires -Version 4.0
function Move-FileToDirectoryWithSamePrefix
{
[CmdletBinding()]
Param(
[Parameter(Mandatory = $true, Position = 0)]
[ValidateScript( {Test-Path $_ -PathType Container})]
[string]
$DirectoryToMoveFileTo,
[Parameter(Mandatory = $true, Position = 1)]
[ValidateScript( {Test-Path $_ -PathType Container})]
[string]
$DirectoryToSearchFiles
)
Begin {
# Needed to find all files for movement
$fileRegex = ".*_[0-9]{3}_[0-9]{4}_.*\.pdf"
$currentLocation = Get-Location
}
Process{
# find files to move
$matchingFile = Get-ChildItem -Path $DirectoryToSearchFiles -Recurse | Where-Object { $_.Name -match $fileRegex }
# Change to destination directory
Set-Location $DirectoryToMoveFileTo
foreach ($file in $matchingFile) {
$directoryName = ($file -split "_")[0]
# Find the director to move to
$dirToMoveFile = Get-ChildItem -Recurse -Include $directoryName
if ($dirToMoveFile -eq $null) {
# Create directory if not existing
$dirToMoveFile = New-Item -Name $directoryName -Path $DirectoryToMoveFileTo -ItemType Directory
}
Move-Item -Path $file.FullName -Destination $dirToMoveFile.fullName
}
}
End{
Set-Location $currentLocation
}
}
Store above in a ps1 file, e.g. moveFile.ps1. Afterwards open a PowerShell and source the file:
PS C:\> . C:\PathToYourPs1File\moveFile.ps1
Afterwards you can the function like:
PS C:\temp> Move-FileToDirectoryWithSamePrefix -DirectoryToMoveFileTo .\filesDestination -DirectoryToSearchFiles .\files
If you need to debug it open the ps1 in the Powershell ISE, set a breakpoint and start the script. For further information see this link.
Hope that helps.

Copy items from Source to Destination if they don't already exist

I have a pretty basic powershell copy script that copies items from a source folder to a destination folder. However this is moving way too much data, and I'd like to check if the filename already exists so that file can be ignored. I don't need this as complex as verifying created date/checksum/etc.
Currently it's along the lines of:
Copy-Item source destination -recurse
Copy-Item source2 destination2 -recurse
I'd imagine I need to add the Test-Path cmdlet, but I'm uncertain how to implement it.
You could always call ROBOCOPY from PowerShell for this.
Use the /xc (exclude changed) /xn (exclude newer) and /xo (exclude older) flags:
robocopy /xc /xn /xo source destination
This will ONLY copy those files that are not in the destination folder.
For more option type robocopy /?
$exclude = Get-ChildItem -recurse $dest
Copy-Item -Recurse $file $dest -Verbose -Exclude $exclude
While I agree that Robocopy is the best tool for something like this, I'm all for giving the customer what they asked for and it was an interesting PowerShell exercise.
This script should do just what you asked for: copy a file from Source to Destination only if it does not already exist in the Destination with a minimum of frills. Since you had the -recurse option in your example, that made for a bit more coding than just simply testing for the filename in the Destination folder.
$Source = "C:\SourceFolder"
$Destination = "C:\DestinationFolder"
Get-ChildItem $Source -Recurse | ForEach {
$ModifiedDestination = $($_.FullName).Replace("$Source","$Destination")
If ((Test-Path $ModifiedDestination) -eq $False) {
Copy-Item $_.FullName $ModifiedDestination
}
}
Building off of Wai Ha Lee's post, here's an example that worked for me:
$Source = "<your path here>"
$Dest = "<your path here>"
$Exclude = Get-ChildItem -recurse $Dest
Get-ChildItem $Source -Recurse -Filter "*.pdf" | Copy-Item -Destination $Dest -Verbose -Exclude $Exclude
This builds a list to exclude, then copies any pdf in the source directory and sub-directories to the destination in a single folder...excluding the existing files. Again, this is an example from my needs, but similar to yours. Should be easy enough to tweak to your hearts content.
Function Copy-IfNotPresent will accept one file at a time but it's easy to loop for all files you want to copy. Here's an example:
gci c:\temp\1\*.* -Recurse -File | % { Copy-IfNotPresent -FilePath $_ -Destination "C:\temp\2\$(Resolve-Path $_ -relative)" -Verbose }
Here's the function. It will generate the folder tree if necessary. Here's the gists link: https://gist.github.com/pollusb/cd47b4afeda8edbf8943a8808c880eb8
Function Copy-IfNotPresent {
<#
Copy file only if not present at destination.
This is a one file at a time call. It's not meant to replace complex call like ROBOCOPY.
Destination can be a file or folder. If it's a folder, you can use -Container to force Folder creation when not exists
#>
[CmdletBinding()]
Param (
[Parameter(Mandatory)]
$FilePath,
[Parameter(Mandatory)]
[string]$Destination,
[switch]$Container,
[switch]$WhatIf
)
#region validations
if ($FilePath -isnot [System.IO.FileInfo]){
$File = Get-ChildItem $FilePath -File
} else {
$File = $FilePath
}
if (!$File.Count){
Write-Warning "$FilePath no file found."
return
} elseif ($File.Count -gt 1) {
Write-Warning "$FilePath must resolve to one file only."
return
}
#endregion
# Destination is a folder
if ($Container -or (Test-Path -Path $Destination -PathType Container)) {
if (!(Test-Path $Destination)) {
New-Item -Path $Destination -ItemType Container | Out-Null
}
$Destination += "\$($File.Name)"
}
# Destination is a file
if (!(Test-Path $Destination)) {
if ($WhatIf) {
Write-Host "WhatIf:Copy-IfNotPresent $FilePath -> $Destination"
} else {
# Force creation of parent folder
$Parent = Split-Path $Destination -Parent
if (!(Test-Path $Parent)) {
New-Item $Parent -ItemType Container | Out-Null
}
Copy-Item -Path $FilePath -Destination $Destination
Write-Verbose "Copy-IfNotPresent $FilePath -> $Destination (is absent) copying"
}
} else {
Write-Verbose "Copy-IfNotPresent $Destination (is present) not copying"
}
}
$source = "c:\source"
$destination = "c:\destination"
Create a list of files to exclude, i.e. files already existing in the destination.
$exclude = Get-Childitem -Recurse $destination | ForEach-Object { $_.FullName -replace [Regex]::Escape($destination ), "" }
Recursively copy all contents from the source to the destination excluding the previously collected files.
Copy-Item -Recurse -Path (Join-Path $source "*") -Destination $destination -Exclude $exclude -Force -Verbose
(Join-Path $source "*") add a wildcard at end ensuring that you get the children of the source folder instead of the source folder itself.
Force is used because I don't mind that there are already existing folders (results in error messages). Use with caution.
ForEach-Object { $_.FullName -replace [Regex]::Escape($destination ), "" } transforms the existing file full names into values which can be used as Exclude parameter
Here is a recursive script that syncronizes 2 folders ignoring existing files:
function Copy-FilesAndFolders([string]$folderFrom, [string]$folderTo) {
$itensFrom = Get-ChildItem $folderFrom
foreach ($i in $itensFrom)
{
if ($i.PSIsContainer)
{
$subFolderFrom = $folderFrom + "\" + $i.BaseName
$subFolderTo = $folderTo + "\" + $i.BaseName
Copy-FilesAndFolders $subFolderFrom $subFolderTo | Out-Null
}
else
{
$from = $folderFrom + "\" + $i.Name
$to = $folderTo + "\" + $i.Name
if (!(Test-Path $from)) # only copies non-existing files
{
if (!(Test-Path $folderTo)) # if folder doesn't exist, creates it
{
New-Item -ItemType "directory" -Path $folderTo
}
Copy-Item $from $folderTo
}
}
}
}
To call it:
Copy-FilesAndFolders "C:\FromFolder" "C:\ToFolder"
Lots of great answers in here, here's my contribution as it relates to keeping an mp3 player in sync with a music library.
#Tom Hubbard, 10-19-2021
#Copy only new music to mp3 player, saves time by only copying items that don't exist on the destination.
#Leaving the hardcoded directories and paths in here, sometimes too much variable substitution is confusing for newer PS users.
#Gets all of the albums in the source directory such as your music library
$albumsInLibrary = gci -Directory -path "C:\users\tom\OneDrive\Music" | select -ExpandProperty Name
#Gets all of the albums of your destination folder, such as your mp3 player
$albumsOnPlayer = gci -Directory -Path "e:\" | select -ExpandProperty name
#For illustration, it will list the differences between the music library and the music player.
Compare-Object -DifferenceObject $albumsInLibrary -ReferenceObject $albumsOnPlayer
#Loop through each album in the library
foreach ($album in $albumsInLibrary)
{
#Check to see if the music player contains this directory from the music library
if ($albumsOnPlayer -notcontains $album)
{
#If the album doesn't exist on the music player, copy it and it's child items from the library to the player
write-host "$album is not on music player, copying to music player" -ForegroundColor Cyan
Copy-Item -path "C:\users\Tom\OneDrive\music\$album" -Recurse -Destination e:\$album
}
}