Powershell copying specific files from source directory, excluding several folders, but then recursive with wildcard for files - powershell

Here is my current script and it works fine. Not efficient running same code twice but I don't know how to combine the wildcards... anyway on to the bigger issue.
The below code searches through my $sourceDir, excludes the files listed in $ExclusionFiles, copies all folders and structure as well as any .jpg or any .csv files, then puts them into the $targetDir.
$sourceDir = 'c:\sectionOne\Graphics\Data'
$targetDir = 'C:\Test\'
$ExclusionFiles = #("InProgress.jpg", "input.csv", "PCMCSV2.csv")
# Get .jpg files
Get-ChildItem $sourceDir -filter "*.jpg" -recurse -Exclude $ExclusionFiles | `
foreach{
$targetFile = $targetDir + $_.FullName.SubString($sourceDir.Length);
New-Item -ItemType File -Path $targetFile -Force;
Copy-Item $_.FullName -destination $targetFile
}
# Get .csv files
Get-ChildItem $sourceDir -filter "*.csv" -recurse -Exclude $ExclusionFiles | `
foreach{
$targetFile = $targetDir + $_.FullName.SubString($sourceDir.Length);
New-Item -ItemType File -Path $targetFile -Force;
Copy-Item $_.FullName -destination $targetFile
}
My list of files in the main $sourceDir that I need to exclude is getting longer and there are folders I want to exclude as well. Can someone tell me how to,
Copy only a list of specific files in the $sourceDir
Exclude certain folders in $sourceDir from copying
Combine the wildcard search for .jpg and .csv into one statement
I'm still learning so any help would be greatly appreciated!

This is a case where a little bit of Regex will go a long way:
You can filter multiple extensions by using a pretty basic match:
$extensions = 'jpg', 'csv'
$endsWithExtension = "\.(?>$($extensions -join '|'))$"
Get-ChildItem -Recurse |
Where-Object Name -Match $endsWithExtension
You can exclude a list of specific files with one more Where-Object and the -In parameter:
$extensions = 'jpg', 'csv'
$endsWithExtension = "\.(?>$($extensions -join '|'))$"
$ExcludeFileNames = #("InProgress.jpg", "input.csv", "PCMCSV2.csv")
Get-ChildItem -Recurse |
Where-Object Name -Match $endsWithExtension |
Where-Object Name -NotIn $ExcludeFileNames
From there on in, your Foreach-Object is basically correct (nice touch making sure the file exists by using New-Item, though I'd personally assign it's output to null and -PassThru the Copy-Item).
Get-ChildItem $sourceDir -Recurse |
Where-Object Name -Match $endsWithExtension |
Where-Object Name -NotIn $ExcludeFileNames |
Foreach-Object {
$targetFile = $targetDir + $_.FullName.SubString($sourceDir.Length);
New-Item -ItemType File -Path $targetFile -Force;
Copy-Item $_.FullName -destination $targetFile
}

Related

Excluding Folders with Get-ChildItem - Need Help Debugging a Script

I've searched through both StackOverflow and SuperUser to try to figure this out, and I'm still getting plagued by a problem I can't figure out how to fix. I know it's something simple, but after playing with it for an hour I'm still stumped. Simple question: how the heck do I tell Get-Childitem to exclude folders?
Right up front here's the code that doesn't work:
$sourceDir="E:\Deep Storage"
$targetDir="W:\Deep Storage"
$excludeThese = 'Projects2','Projects3','Projects4';
Get-ChildItem -Path $sourceDir -Directory -Recurse |
where {$_.fullname -notin $excludeThese} |
Get-ChildItem -Path $sourceDir | ForEach-Object {
$num=1
$nextName = Join-Path -Path $targetDir -ChildPath $_.name
while(Test-Path -Path $nextName)
{
$nextName = Join-Path $targetDir ($_.BaseName + "_$num" + $_.Extension)
$num+=1
}
$_ | Move-Item -Destination $nextName -Force -Verbose -WhatIf
}
}
The underlying concept here already works:
$sourceDir="E:\Deep Storage"
$targetDir="W:\Deep Storage"
Get-ChildItem -Path $sourceDir -File -Recurse | ForEach-Object {
$num=1
$nextName = Join-Path -Path $targetDir -ChildPath $_.name
while(Test-Path -Path $nextName)
{
$nextName = Join-Path $targetDir ($_.BaseName + "_$num" + $_.Extension)
$num+=1
}
$_ | Copy-Item -Destination $nextName -Verbose
}
Basically what this does is it moves folders from one place to another, and if files exist in both places, it renames files coming in. It helps keep my archive drive clear. But there are three folders there that I want to exclude because I still pull assets from them regularly, so I don't need those files moved.
Hence the difference between the two code samples: in the first one, I'm trying to get Get-Childitem to exclude a specific trio of folders, while this second one just grabs everything all at once.
I tried just doing a straight -Exclude with $excludeThese as the variable, without success; I tried skipping the variable approach altogether and just putting the folder names in after -Exclude. Still didn't work. I also tried putting in the entire path to the folders I wanted to exclude. No good--no matter what I did, the -WhatIf showed that the script was trying to move everything, including the folders I was theoretically excluding.
The last trick I tried was one I came across here on SO, and that was to go a gci with the exclude argument first, then do another gci after it. That still failed, so now I have to turn to the experts for help.
I would use a regex string created from the (escaped) directory names to exclude to make sure files withing these folders are ignored.
Also, by using a lookup Hashtable of all file names already present in the target folder, figuring out if a file with a certain name already exists is extremely fast.
$sourceDir = 'E:\Deep Storage'
$targetDir = 'W:\Deep Storage'
$excludeThese = 'Projects2','Projects3','Projects4';
# create a regex string with all folder names to exclude combined with regex OR (|)
$excludeDirs = ($excludeThese | ForEach-Object { [Regex]::Escape($_) }) -join '|'
# create a lookup Hashtable and store the filenames already present in the destination folder
$existingFiles = #{}
Get-ChildItem -Path $targetDir -File | ForEach-Object { $existingFiles[$_.Name] = $true }
Get-ChildItem -Path $sourceDir -File -Recurse |
Where-Object {$_.DirectoryName -notmatch $excludeDirs} |
ForEach-Object {
# construct the new filename by appending an index number if need be
$newName = $_.Name
$count = 1
while ($existingFiles.ContainsKey($newName)) {
$newName = "{0}_{1}{2}" -f $_.BaseName, $count++, $_.Extension
}
# add this new name to the Hashtable so it exists in the next run
$existingFiles[$newName] = $true
# use Join-Path to create a FullName for the file
$newFile = Join-Path -Path $targetDir -ChildPath $newName
$_ | Move-Item -Destination $newFile -Force -Verbose -WhatIf
}
Assuming the excluded directories are at the top:
$sourceDir="E:\Deep Storage"
$excludeThese = 'Projects2','Projects3','Projects4'
get-childitem $sourcedir -exclude $excludethese | get-childitem -recurse

Copy all latest files from folders/sub to the same name folders/sub in powershell

I am trying to copy the latest file from every folder/sub-folder into a same file structure on a different drive.
Latest file from source copied to the same name corresponding destination.
The destination folder hierarchy already exists & cannot be copied over or recreated. This & other versions are not behaving. Can anyone help?
$sourceDir = 'test F Drive\Shares\SSRSFileExtract\'
$destDir = 'test X Drive\SSRSFileExtract\'
$date = Get-Date
$list = Get-ChildItem -Path $sourceDir | Sort-Object -Property LastWriteTime -Descending | Select-Object -First 1
foreach ($item in $list)
{
Copy-Item -Verbose -LiteralPath $item.FullName -Destination $destDir -Force |
Get-Acl -Path $item.FullName | Set-Acl -Path $destDir\$(Split-Path -Path $item.FullName -Leaf)
}
Get-ChildItem –Path $destDir -Recurse | Where-Object {($_.LastWriteTime -lt (Get-Date).AddDays(-5))} | Remove-Item -Verbose -Recurse -Force
I found a solution for this which copies/moves all the files from all sub folders in to all corresponding sub folders:
Powershell: Move all files from folders and subfolders into single folder
The way your code retrieves the list of files will only return one single object because of Select-Object -First 1. Also, because you don't specify the -File switch, Get-ChildItem will also return DirectoryInfo objects, not just FileInfo objects..
What you could do is get an array of FileInfo objects recursively from the source folder and group them by the DirectoryName property
Then loop over these groups of files and from each of these groups, select the most recent file and copy that over to the destination folder.
Try:
$sourceDir = 'F:\Shares\SSRSFileExtract'
$destDir = 'X:\SSRSFileExtract'
Get-ChildItem -Path $sourceDir -File -Recurse | Group-Object DirectoryName | ForEach-Object {
# the $_ automatic variable represents one group at a time inside the loop
$newestFile = $_.Group | Sort-Object -Property LastWriteTime -Descending | Select-Object -First 1
# construct the target sub directory
# you could also use $_.Name (the name of this group) instead of $newestFile.DirectoryName here, because
# we grouped on that DirectoryName file property.
$targetDir = Join-Path -Path $destDir -ChildPath $newestFile.DirectoryName.Substring($sourceDir.Length)
# if you're not sure the targetpath exists, uncomment the next line to have it created first
# if (!(Test-Path -Path $targetDir -PathType Container)) { $null = New-Item -Path $target -ItemType Directory }
# copy the file
$newestFile | Copy-Item -Destination $targetDir -Force -Verbose
# copy the file's ACL
$targetFile = Join-Path -Path $targetDir -ChildPath $newestFile.Name
$newestFile | Get-Acl | Set-Acl -Path $targetFile
}
Apparently you would also like to clean up older files in the destination folder
Get-ChildItem –Path $destDir -File -Recurse |
Where-Object {$_.LastWriteTime -lt (Get-Date).AddDays(-5).Date} |
Remove-Item -Verbose -Recurse -Force
Be aware that the final code to remove older files could potentially remove all files from a subfolder if all happen to be older than 5 days..

Powershell copy files with specific name

I tried to copy files from one folder to another which have word HIGH at the end of name of files in their names but didn't get it. Any suggestion?
$dest = "C:\transform"
$source = "D:\result"
get-childitem $source - filter ".jpg" -recurse | Where-Object {$_.DirectoryName -match "HIGH" | ForEach-Object { Copy-Item $.fullname $dest}
$_.DirectoryName holds the folder name, $_.Name the file name :
$dest = "C:\transform"
$source = "D:\result"
Get-ChildItem $source -Filter ".jpg" -Recurse |
? { $_.BaseName -match "HIGH$" } |
% { Copy-Item $_.FullName $dest}
Or, as pointed by #Walter Mitty, a simpler :
Copy-Item -Path $source -Filter "*HIGH.jpg" -Destination $dest –Recurse
(in this case -Filter and -Include seem to behave the same)
The simplest way to copy files from one folder to another is the Copy-Item cmdlet.
Take a look at the -Path -Include -Destination and -Recurse parameters.
https://technet.microsoft.com/library/60a19812-67ab-4b58-a6f5-34640edafbb0(v=wps.630).aspx

PowerShell - Copy specific files from specific folders

So, the folder structure looks like this:
SourceFolder
file1.txt
file1.doc
Subfolder1
file2.txt
file2.doc
SubSubFolder
file3.txt
doc3.txt
What I want to do is copy all .txt files from folders, whose (folder) names contains the eng, to a destination folder. Just all the files inside the folder - not the file structure.
What I used is this:
$dest = "C:\Users\username\Desktop\Final"
$source = "C:\Users\username\Desktop\Test1"
Copy-Item $source\eng*\*.txt $dest -Recurse
The problem is that it copies the .txt files only from each parent folder but not the sub-folders.
How can I include all the sub-folders in this script and keep the eng name check as well? Can you please help me?
I am talking about PowerShell commands. Should I use robocopy instead?
Yet another PowerShell solution :)
# Setup variables
$Dst = 'C:\Users\username\Desktop\Final'
$Src = 'C:\Users\username\Desktop\Test1'
$FolderName = 'eng*'
$FileType = '*.txt'
# Get list of 'eng*' file objects
Get-ChildItem -Path $Src -Filter $FolderName -Recurse -Force |
# Those 'eng*' file objects should be folders
Where-Object {$_.PSIsContainer} |
# For each 'eng*' folder
ForEach-Object {
# Copy all '*.txt' files in it to the destination folder
Copy-Item -Path (Join-Path -Path $_.FullName -ChildPath '\*') -Filter $FileType -Destination $Dst -Force
}
You can do this :
$dest = "C:\NewFolder"
$source = "C:\TestFolder"
$files = Get-ChildItem $source -File -include "*.txt" -Recurse | Where-Object { $_.DirectoryName -like "*eng*" }
Copy-Item -Path $files -Destination $dest
Another take:
$SourceRoot = <Source folder path>
$TargetFolder = <Target folder path>
#(Get-ChildItem $SourceRoot -Recurse -File -Filter *.txt| Select -ExpandProperty Fullname) -like '*\eng*\*' |
foreach {Copy-Item $_ -Destination $TargetFolder}
It may be easier to first get a list of all the folders that contain eng in the name.
$dest = "C:\Users\username\Desktop\Final"
$source = "C:\Users\username\Desktop\Test1"
$engFolders = Get-ChildItem $source -Directory -Recurse | Where { $_.BaseName -match "^eng" }
Foreach ($folder In $engFolders) {
Copy-Item ($folder.FullName + "\*.txt") $dest
}
Fine to do that with powershell. Try:
$dest = "C:\Users\username\Desktop\Final"
$source = "C:\Users\username\Desktop\Test1"
Get-ChildItem $source -filter "*.txt" -Recurse | Where-Object { $_.DirectoryName -match "eng"} | ForEach-Object { Copy-Item $_.fullname $dest }

Move-Item -Exclude -Recursive Is Not leaving excluded items in place within sub directories

I'm trying to move all items except a certain type of file. In this case *.msg. It does fine if the excluded file resides within the parent folder. However, the moment that same type of file is located within a subdirectory, it fails to leave the file in place and instead moves it to the new location.
username = Get-Content '.\users.txt'
foreach ($un in $username)
{
$destA = "c:\users\$un\redirectedfolders\mydocuments"
$destB = "c:\users\$un\redirectedfolders\desktop"
$sourceA = "C:\users\$un\mydocuments"
$sourceB = "C:\users\$un\desktop"
New-Item -ItemType Directory -Path $destA, $destB
Get-ChildItem $sourceA -Exclude '*.msg' -Recurse | Move-Item -Destination {Join-Path $destA $_.FullName.Substring($sourceA.length)}
Get-ChildItem $sourceB -Exclude '*.msg' -Recurse | Move-Item -Destination {Join-Path $destB $_.FullName.Substring($sourceB.length)}
}
This is due to the filtering done by the Get-ChildItem exclude filter. It's kind of a known issue, and if you really want I could probably dig up some reference documentation, but it may take some time. Regardless, GCI doesn't handle wildcards very well when it comes to filtering things. What you are probably better off doing is piping it to a Where command like this:
$username = Get-Content '.\users.txt'
foreach ($un in $username)
{
$destA = "c:\users\$un\redirectedfolders\documents"
$destB = "c:\users\$un\redirectedfolders\desktop"
$sourceA = "C:\users\$un\documents"
$sourceB = "C:\users\$un\desktop"
New-Item -ItemType Directory -Path $destA, $destB
GCI $sourceA -recurse | ?{$_.Extension -ne ".msg" -and !$_.PSIsContainer} | %{
$CurDest = Join-Path $destA $_.FullName.Substring($sourceA.length)
If(!(Test-Path $CurDest.SubString(0,$CurDest.LastIndexOf("\")))){New-Item -Path $CurDest -ItemType Directory|Out-Null}
$_ | Move-Item -Destination $CurDest
}
GCI $sourceB -recurse | ?{$_.Extension -ne ".msg" -and !$_.PSIsContainer} | %{
$CurDest = Join-Path $destB $_.FullName.Substring($sourceB.length)
If(!(Test-Path $CurDest.SubString(0,$CurDest.LastIndexOf("\")))){New-Item -Path $CurDest -ItemType Directory|Out-Null}
$_ | Move-Item -Destination $CurDest
}
}
Edit: Ok, now excludes folders, and also keeps folder structure.
Edit2: Re-designed to do a ForEach loop on the files, build the destination path as $CurDest, test to make sure it exists and make it if it doesn't, then move the files. Also changed mydocuments to documents which is the path to a user's My Documents folder.