I have a directory A in which containing the sub directories a, b, c, d, each sub-directory contains timestamped folders, example 20230120, and files whose names end with the date and time of day example file-202301200545
What I want is to copy the files whose names end with today's date to the folder with today's date.
Sorry, it's a bit complicated to explain but I hope you understand it.
I wrote a script that performs this action but on the condition that I fix each directory.
$pattern = Get-Date -Format yyyyMMdd
$path = "D:\A\b\"
$Dest = "D:\A\b\\$pattern"
Get-ChildItem -Path $path -Recurse -Filter *$pattern* | Where-Object {$_.PSIsContainer -eq $false} | Move-Item -Destination $Dest
$path = "D:\A\c\"
$Dest = "D:\A\c\\$pattern"
Get-ChildItem -Path $path -Recurse -Filter *$pattern* | Where-Object {$_.PSIsContainer -eq $false} | Move-Item -Destination $Dest
$path = "D:\A\d\"
$Dest = "D:\A\d\\$pattern"
Get-ChildItem -Path $path -Recurse -Filter *$pattern* | Where-Object {$_.PSIsContainer -eq $false} | Move-Item -Destination $Dest
I need help to use a loop or any other techinique that will allow me to do this more easily
Assuming that the target dir. already exists:
Get-ChildItem -Directory -Path D:\A\[a-d] |
Get-ChildItem -File -Recurse -Filter *$pattern* |
Move-Item -Destination {
Join-Path ($_.DirectoryName -split '\\')[0..2] $pattern
} -WhatIf
Note: The -WhatIf common parameter in the command above previews the operation. Remove -WhatIf and re-execute once you're sure the operation will do what you want.
If you want to create the target dir. on demand, use
New-Item -Type Directory -Force (Join-Path ($_.DirectoryName -split '\\')[0..2] $pattern) - New-Item's -Force switch, when combined with -Type Directory, ensures that a preexisting directory, if any, is returned; otherwise, the directory is created; in either case, a System.IO.DirectoryInfo instance representing the directory is returned, which stringifies to its .FullName property, i.e. to its full, file-system-native path.
Note:
The nested Get-ChildItem calls are meant to promote conceptual clarity:
The outer call uses a PowerShell wildcard expression in combination with the -Directory switch to match the top-level directories of interest - note the use of character range [a-d] to match directories names a, b, c, or d.
The inner call acts on each resulting directory and recursively searches only for files matching the pattern, via the -File switch.
This approach avoids the need for your Where-Object {$_.PSIsContainer -eq $false} call, which - incidentally - can be more PowerShell-idiomatically expressed as Where-Object { -not $_.PSIsContainer } and - in PowerShell (Core) 7+ - using simplified syntax, Where-Object -not PSIsContainer.
Note: It is tempting to try a single Get-ChildItem call such as Get-ChildItem -Path D:\A\[a-d] -File -Recurse -Filter *$pattern*, but that doesn't actually work, because the -File switch is then applied to whatever the D:\A\[a-d] wildcard expression resolves to, and since the results are directories, -File effectively excludes them from further processing, resulting in an effective no-op.
That is, -File in combination with -Recurse only works meaningfully for literal input paths, to whose child (descendant) items the -File switch is then applied.
The Move-Item call uses a delay-bind script block to dynamically determine the destination directory:
$_.DirectoryName contains each input file's full directory path, and (... -split '\\')[0..2] extracts the ancestral path that is composed of 3 components, such as D:\A\c.
I have a a bunch of language folders present in a directory under E:\Data\ like hu-hu, de-de etc.. on the other hand i have a bunch of file names in G:\ that contain the part of folder name for e.g.
amd64.de-de_OCR.cab,amd64.handwriting.de-de.cab
I need to copy all matching file names based on the foldername
for e.g. de-de should copy all matching files in G:\ i.e. both amd64.de-de_OCR.cab,amd64.handwriting.de-de.cab
This is the code i have so far but it is not copying over the files, and i am not sure how to proceed next, any help is appreciated.
$listfoldername = Get-ChildItem -Path "E:\Data" -Recurse -Directory -Force -ErrorAction SilentlyContinue | Select-Object Name
$destfolder = Get-ChildItem -Path "E:\Data" -Recurse -Directory -Force -ErrorAction SilentlyContinue | Select-Object FullName
$filename = Get-ChildItem -file G:\
if($filename -like $listfoldername)
{
Copy-Item -Path $filename -Destination $destfolder
}
There's a few issues with your code
The main issue with your code is that you are trying to use the -like operator to compare two objects (your object containing the directories you wish to move files to, and the object containing the files.
What you need to do is loop through each file and directory, one by one, to determine if the directory name (e.g. "hu-hu" is found in the filename (e.g. amd64.hu-hu_OCR.cab)
You'll want to use the wildcard indicator "*" with the -like operator (e.g. "*hu-hu*")
This below code snippet should do the trick. I tested using the file and folder names you've provided.
"G:" contains the folders:
de-de
hu-hu
us-us (note, I added this to make sure the code did not match this directory)
"E:\Data" contains the files
amd64.de-de_OCR.cab
amd64.handwriting.de-de.cab
amd64.handwritinghu-hu.cab
amd64.hu-hu_OCR.cab
$FileDirectory = "G:" # Change to "G:\", the trailing slash breaks syntax highlight on SO
$DataDirectory = "E:\Data"
$listfoldername = Get-ChildItem -Path "$DataDirectory" -Recurse -Directory -Force -ErrorAction SilentlyContinue | Select-Object Name
$filename = Get-ChildItem -file "$FileDirectory"
#Loop through each file one at a time
foreach ($file in $filename) {
# Then, loop through each folder one at a time
foreach ($folder in $listfoldername) {
# Set the current filename and listfoldername to variables for later -like operator
$FileString = $file.Name
$FolderString = $folder.Name
# If the current file "is like" the current folder name
if($FileString -like "*$FolderString*")
{
# Set the name of the current folder to a variable
$DataFolder = $folder.Name
Copy-Item -Path "$FileDirectory\$FileString" -Destination "$DataDirectory\$DataFolder"
} else {
Write-Output ("$FolderString pattern not found in $FileString")
}
}
}
I think you should start off by getting a list of possible language target folders. Then loop over the path where the files are, filtering their names to have at least the dash in it and next test if any of the language target folders matches the filename.
Something like this:
$langFolder = 'E:\Data'
$fileFolder = 'G:\' #'# dummy comment to fix syntax highlighting in SO
# get a list of the language folders
# if the languages folder has multiple subdirectories to include, add -Recurse here
$targetFolders = Get-ChildItem -Path $langFolder -Directory
# get a list of FileInfo objects for the files in the G:\ path
# if you need to search subdirectories aswell, add -Recurse here
$files = Get-ChildItem -Path $fileFolder -File -Filter '*-*.*'
foreach($file in $files) {
# check if a language name matches the file name
foreach($folder in $targetFolders) {
if ($file.BaseName -like "*$($folder.Name)*") {
# we have found a matching language target directory
$file | Copy-Item -Destination $folder.FullName
break # exit this folder foreach loop and get on with the next file
}
}
}
P.S. If all the files are .cab files you could speed up by setting the Filter to '*-*.cab' in line $files = Get-ChildItem ...
I need to add a safety net in my script. I'm trying to do a copy job based on a list of users provided through a txt file. Copy the files from that users home directory to a new location. Once the files are copied, check if the file exists in the new location. If yes, then Remove-Item.
Can someone help me? I just don't know how to implement the "if file exists" logic.
$username = Get-Content '.\users.txt'
foreach ($un in $username)
{
$dest = "\\server\homedirs\$un\redirectedfolders"
$source = "\\server\homedirs\$un"
New-Item -ItemType Directory -Path $dest\documents, $dest\desktop
Get-ChildItem $source\documents -Recurse -Exclude '*.msg' | Copy-Item -Destination $dest\documents
Get-ChildItem $source\desktop -Recurse -Exclude '*.msg' | Copy-Item -Destination $dest\desktop
Get-ChildItem $source\mydocuments, $source\desktop -Recurse -Exclude '*.msg' | Remove-Item -Recurse
}
The shortest way to delete file if it doesn't exist is NOT to use Test-Path but:
rm my_file.zip -ea ig
This is short version of
rm my_file.zip -ErrorAction Ignore
which is much more readable and more DRY then
if (Test-Path my_file.zip) { rm my_file.zip }
To answer your question per se, you can do it like this:
Get-ChildItem $source\mydocuments, $source\desktop -Recurse -Exclude '*.msg' | %{
if (Test-Path ($_. -replace "^$([regex]::escape($source))","$dest")) {
Remove-Item $_ -Recurse
}
}
Test-Path returns $true if the file at the given path exists, otherwise $false.
$_ -replace "^$([regex]::escape($source))","$dest" converts the path of each source item you're enumerating with the corresponding destination path, by replacing $source at the beginning of the path with $dest.
The basic regex for the first argument to the -replace operator is ^$source (which means "match the value of $source at the beginning of the string"). However, you need to use [regex]::escape in case $source contains any regex special characters, which is in fact extremely likely with Windows paths, since they contain backslashes. For example, the value you've given here for $source contains \s, which in a regex means "any whitespace character". $([regex]::escape($source)) will interpolate the value of $source with any regex special characters properly escaped, so that you're matching the explicit value.
That said, if your purpose is to copy each item to a new location, and remove the original only if the copy to the new location is successful, it seems like you're reinventing the wheel. Why not just use Move-Item instead of Copy-Item?
Not directly related to the question, but rather than repeating the same command for each subdirectory, you can use a foreach loop:
foreach ($subdir in (echo documents desktop)) {
# Whatever command you end up using to copy or move the items,
# using "$source\$subdir" and "$dest\$subdir" as the paths
}
Test-Path commandlet will help you check if the file exists
http://technet.microsoft.com/en-us/library/ee177015.aspx
#Adi Inbar, I need to use a function like this because I need to move files to a remote session, and the Move-Item does not work when I tried -ToSession... only Copy-Item.
The Key is that if the power or internet goes down, the script will delete the file even if it wasn't copied.
$username = "name"
$password = ConvertTo-SecureString "password" -AsPlainText -Force
$credential = New-Object System.Management.Automation.PSCredential -ArgumentList ($username, $password)
$Session = New-PSSession -ComputerName "IPAdress" -Credential $credential
Copy-Item -Path C:\userPC_1\csv-output\*.csv -Destination C:\userPC_2\Documents\Test_Scripts -ToSession $Session -Verbose
Get-PSSession | Remove-PSSession
Get-ChildItem -Path C:\userPC_1\csv-output\*.csv | Remove-Item -Force
This script to remove some docs doesn't work:
$includeExtensions = #("*.doc", "*.docx")
get-childitem -Path "C:\somedir" -Include $includeExtensions | remove-item
Why not? And how do you specify an array of extensions which can then be passed as an argument?
You can pipe the results of Get-ChildItem to filter on just the extensions you want.
$includeExtensions = #(".doc", ".docx")
Get-ChildItem -Path "C:\somedir" | ?{$includeExtensions -contains $_.Extension} | Remove-Item
Try specifying it like so:
Get-ChildItem c:\somedir\* -Inc $includeExtensions | remove-item -whatif
From the docs on Get-ChildItem:
The Include parameter is effective only when the command includes the
Recurse parameter or the path leads to the contents of a directory,
such as C:\Windows*, where the wildcard character specifies the
contents of the C:\Windows directory.
I have the following snippet of PowerShell script:
$source = 'd:\t1\*'
$dest = 'd:\t2'
$exclude = #('*.pdb','*.config')
Copy-Item $source $dest -Recurse -Force -Exclude $exclude
Which works to copy all files and folders from t1 to t2, but it only excludes the exclude list in the "root"/"first-level" folder and not in sub-folders.
How do I make it exclude the exclude list in all folders?
I think the best way is to use Get-ChildItem and pipe in the Copy-Item command.
I found that this worked:
$source = 'd:\t1'
$dest = 'd:\t2'
$exclude = #('*.pdb','*.config')
Get-ChildItem $source -Recurse -Exclude $exclude | Copy-Item -Destination {Join-Path $dest $_.FullName.Substring($source.length)}
Basically, what is happening here is that you're going through the valid files one by one, then copying them to the new path. The 'Join-Path' statement at the end is so that the directories are also kept when copying over the files. That part takes the destination directory and joins it with the directory after the source path.
I got the idea from here, and then modified it a bit to make it work for this example.
I hope it works!
I had this problem, too, and spent 20 minutes with applying the solutions here, but kept having problems.So I chose to use robocopy - OK, it's not powershell, but should be available everywhere where powershell runs.
And it worked right out of the box:
robocopy $source $dest /S /XF <file patterns to exclude> /XD <directory patterns to exclude>
e.g.
robocopy $source $dest /S /XF *.csproj /XD obj Properties Controllers Models
Plus, it has tons of features, like resumable copy.
Docs here.
As comments format code badly I'll post as answer but it's just an addition to #landyman's answer.
The proposed script has a drawback - it will create double-nested folders. For example for 'd:\t1\sub1' it will create empty directory 'd:\t2\sub1\sub1'. That's due to the fact that Copy-Item for directories expects parent directory name in -Destination property not directory name itself.
Here's a workaround I found:
Get-ChildItem -Path $from -Recurse -Exclude $exclude | Copy-Item -Force -Destination {
if ($_.GetType() -eq [System.IO.FileInfo]) {
Join-Path $to $_.FullName.Substring($from.length)
} else {
Join-Path $to $_.Parent.FullName.Substring($from.length)
}
}
Note that the syntax spec calls for a STRING ARRAY; ala String[]
SYNTAX
Copy-Item [[-Destination] <String>] [-Confirm] [-Container] [-Credential <PSCredential>] [-Exclude <String[]>] [-Filter <String>] [-Force] [-FromSession <PSSession>] [-Include
<String[]>] -LiteralPath <String[]> [-PassThru] [-Recurse] [-ToSession <PSSession>] [-UseTransaction] [-WhatIf] [<CommonParameters>]
If you're not explicit in your array generation, you end up with an Object[] - and that is ignored in many cases, leaving the appearance of "buggy behavior" because of type-safety. Since PowerShell can process script-blocks, evaluation of other than a type-specific variable (so that a valid string could be determined) would leave an opening for the potential of an injection mode attack on any system whose execution policy were lax.
So this is unreliable:
PS > $omissions = #("*.iso","*.pdf","*.zip","*.msi")
PS > $omissions.GetType()
Note the result....
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True Object[] System.Array
And this works.... for example:
PS > $omissions = [string[]]#("*.iso","*.pdf","*.zip","*.msi")
**or**
PS > [string[]]$omissions = ("*.iso,*.pdf,*.zip,*.msi").split(',')
PS > $omissions.GetType()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True String[] System.Array
Note that even a "single" element would still require the same cast, so as to create a 1-element array.
If you're trying this at home, be sure to use the Replace-Variable "omissions" to clean out the existence of $omissions before recasting it in the examples shown above.
And as far as a pipeline that works reliably that I've tested....
--------------------------------------------------------------------------------------- cd $sourcelocation
ls | ?{$_ -ne $null} | ?{$_.BaseName -notmatch "^\.$"} | %{$_.Name} | cp -Destination $targetDir -Exclude $omissions -recurse -ErrorAction silentlycontinue
---------------------------------------------------------------------------------------
The above does a directory listing of the source files in the base (selected "current") directory, filters out potential problem items, converts the file to the basename and forces cp (copy-item alias) to re-access the file "by name" in the "current directory" - thus reacquiring the file object, and copies it. This will create empty directories, including those that may even contain excluded files (less the exclusions of course). Note also that "ls" (get-childitem) does NOT -recurse - that is left to cp. Finally - if you're having problems and need to debug, remove the -ErrorAction silentlycontinue switch and argument, which hides a lot of nuisances that might interrupt the script otherwise.
For those whose comments were related to "\" inclusions, keep in mind that you're working over the .NET sub-layer via an interpreter (i.e. PowerShell), and in c# for example, the inclusion of a single "\" (or multiple singles in a string), results in the compiler demanding you correct the condition by using either "\\" to escape the backslash, or precede the string with an # as in #"\"; with the other remaining option being the enclosure of the string in single quotes, as '\'. All of this is because of ASCII interpolation of character combinations like "\n" etc.
The latter is a much bigger subject, so I'll leave you with that consideration.
The exclude parameter won't work with dirs. A variant of Bo's script does the trick:
$source = 'c:\tmp\foo'
$dest = 'c:\temp\foo'
$exclude = '\.bak'
Get-ChildItem $source -Recurse | where {$_.FullName -notmatch $exclude} |
Copy-Item -Destination {Join-Path $dest $_.FullName.Substring($source.length)}
I was looking for a way to copy files modified after a certain date/timestamp so as to archive them. This way I could save off exactly what files I worked on (assuming I know when I started). (Yes, I know this is what SCM is for, but there are times when I just want to snapshot my work without checking it in.)
Using landyman's tip, and stuff I found elsewhere, I found that this worked:
$source = 'c:\tmp\foo'
$dest = 'c:\temp\foo'
$exclude = #('*.pdb', '*.config')
Get-ChildItem $source -Recurse -Exclude $exclude |
where-object {$_.lastwritetime -gt "8/24/2011 10:26 pm"} |
Copy-Item -Destination {Join-Path $dest $_.FullName.Substring($source.length)}
Get-ChildItem with Join-Path was working mostly for me, but I realized it was copying root directories inside the other root directories, which was bad.
For example
c:\SomeFolder
c:\SomeFolder\CopyInHere
c:\SomeFolder\CopyInHere\Thing.txt
c:\SomeFolder\CopyInHere\SubFolder
c:\SomeFolder\CopyInHere\SubFolder\Thin2.txt
Source Directory: c:\SomeFolder\CopyInHere
Destination Directory: d:\PutItInHere
Goal:
Copy every childitem Inside c:\SomeFolder\CopyInHere to the root of d:\PutItInHere, but not including c:\SomeFolder\CopyInHere itself.
- E.g. Take all the children of CopyInHere and make them Children of PutItInHere
The above examples do this most of the way, but what happens is It Creates a folder Called SubFolder, and Creates a Folder in Folder called SubFolder.
That's because Join-Path Calculates a destination path of d:\PutItInHere\SubFolder for the SubFolder child item, so SubFolder get's created in a Folder called SubFolder.
I got around this by Using Get-ChildItems to bring back a collection of the items, then using a loop to go through it.
Param(
[Parameter(Mandatory=$True,Position=1)][string]$sourceDirectory,
[Parameter(Mandatory=$True,Position=2)][string]$destinationDirectory
)
$sourceDI = [System.IO.DirectoryInfo]$sourceDirectory
$destinationDI = [System.IO.DirectoryInfo]$destinationDirectory
$itemsToCopy = Get-ChildItem $sourceDirectory -Recurse -Exclude #('*.cs', 'Views\Mimicry\*')
foreach ($item in $itemsToCopy){
$subPath = $item.FullName.Substring($sourceDI.FullName.Length)
$destination = Join-Path $destinationDirectory $subPath
if ($item -is [System.IO.DirectoryInfo]){
$itemDI = [System.IO.DirectoryInfo]$item
if ($itemDI.Parent.FullName.TrimEnd("\") -eq $sourceDI.FullName.TrimEnd("\")){
$destination = $destinationDI.FullName
}
}
$itemOutput = New-Object PSObject
$itemOutput | Add-Member -Type NoteProperty -Name Source -Value $item.FullName
$itemOutput | Add-Member -Type NoteProperty -Name Destination -Value $destination
$itemOutput | Format-List
Copy-Item -Path $item.FullName -Destination $destination -Force
}
What this does in short, is it uses the current item's full name for the destination calculation. However it then checks to see if it is a DirectoryInfo object. If it is it checks if it's Parent Folder is the Source Directory, that means the current folder being iterated is a direct child of the source directory, as such we should not append it's name to the destination directory, because we want that folder to be created in the destination directory, not in a folder of it's in the destination directory.
Following that, every other folder will work fine.
$sourcePath="I:\MSSQL\Backup\Full"
$excludedFiles=#("MASTER", "DBA", "MODEL", "MSDB")
$sourceFiles=(ls $sourcePath -recurse -file) | where-object { $_.directory.name -notin $excludedFiles }
this is what i did, i needed to copy out a bunch of backup files to a separate location on the network for client pickup. we didn't want them to have the above system DB backups.
I had a similar problem extending this a bit. I want a solution working for sources like
$source = "D:\scripts\*.sql"
too. I found this solution:
function Copy-ToCreateFolder
{
param(
[string]$src,
[string]$dest,
$exclude,
[switch]$Recurse
)
# The problem with Copy-Item -Rec -Exclude is that -exclude effects only top-level files
# Copy-Item $src $dest -Exclude $exclude -EA silentlycontinue -Recurse:$recurse
# http://stackoverflow.com/questions/731752/exclude-list-in-powershell-copy-item-does-not-appear-to-be-working
if (Test-Path($src))
{
# Nonstandard: I create destination directories on the fly
[void](New-Item $dest -itemtype directory -EA silentlycontinue )
Get-ChildItem -Path $src -Force -exclude $exclude | % {
if ($_.psIsContainer)
{
if ($Recurse) # Non-standard: I don't want to copy empty directories
{
$sub = $_
$p = Split-path $sub
$currentfolder = Split-Path $sub -leaf
#Get-ChildItem $_ -rec -name -exclude $exclude -Force | % { "{0} {1}" -f $p, "$currentfolder\$_" }
[void](New-item $dest\$currentfolder -type directory -ea silentlycontinue)
Get-ChildItem $_ -Recurse:$Recurse -name -exclude $exclude -Force | % { Copy-item $sub\$_ $dest\$currentfolder\$_ }
}
}
else
{
#"{0} {1}" -f (split-path $_.fullname), (split-path $_.fullname -leaf)
Copy-Item $_ $dest
}
}
}
}
The below snippet will copy all files and folders from $source to $dest, excluding .pdb and .config files from the root folder and sub-folders:
Get-ChildItem -Path $source | Copy-Item -Destination $dest -Recurse -Container -Exclude #('*.pdb','*.config')
One way of copying items from one folder to another using regular expressions for exclusion:
$source = '.\source'
$destination = '.\destination'
$exclude = '.*\.pdf$|.*\.mp4$|\\folder1(\\|$)|\\folder2(\\|$)'
$itemsToCopy = Get-ChildItem $source -Recurse |
Where-Object FullName -notmatch $exclude | Select-Object -Expand FullName
$sourceFullNameLength = (Get-Item $source).FullName.Length
foreach ($item in $itemsToCopy) {
$relativeName = $item.Substring($sourceFullNameLength + 1)
Copy-Item -Path $item -Destination "$destination\$relativeName"
}
I wrote this for daily use and packaged it in the script module, it maintains all the directory structure and supports wildcards:
function Copy-Folder {
[CmdletBinding()]
param(
[Parameter(Mandatory)]
[String]$FromPath,
[Parameter(Mandatory)]
[String]$ToPath,
[string[]] $Exclude
)
if (Test-Path $FromPath -PathType Container) {
New-Item $ToPath -ItemType Directory -ErrorAction SilentlyContinue | Out-Null
Get-ChildItem $FromPath -Force | ForEach-Object {
# avoid the nested pipeline variable
$item = $_
$target_path = Join-Path $ToPath $item.Name
if (($Exclude | ForEach-Object { $item.Name -like $_ }) -notcontains $true) {
if (Test-Path $target_path) { Remove-Item $target_path -Recurse -Force }
Copy-Item $item.FullName $target_path
Copy-Folder -FromPath $item.FullName $target_path $Exclude
}
}
}
}
Just call the Copy-Folder -FromPath 'fromDir' -ToPath 'destDir' -Exclude *.pdb,*.config
The -FromPath and -ToPath can be omitted,
Copy-Folder -FromPath 'fromDir destDir -Exclude *.pdb,*.config