Move-Item -Exclude -Recursive Is Not leaving excluded items in place within sub directories - powershell

I'm trying to move all items except a certain type of file. In this case *.msg. It does fine if the excluded file resides within the parent folder. However, the moment that same type of file is located within a subdirectory, it fails to leave the file in place and instead moves it to the new location.
username = Get-Content '.\users.txt'
foreach ($un in $username)
{
$destA = "c:\users\$un\redirectedfolders\mydocuments"
$destB = "c:\users\$un\redirectedfolders\desktop"
$sourceA = "C:\users\$un\mydocuments"
$sourceB = "C:\users\$un\desktop"
New-Item -ItemType Directory -Path $destA, $destB
Get-ChildItem $sourceA -Exclude '*.msg' -Recurse | Move-Item -Destination {Join-Path $destA $_.FullName.Substring($sourceA.length)}
Get-ChildItem $sourceB -Exclude '*.msg' -Recurse | Move-Item -Destination {Join-Path $destB $_.FullName.Substring($sourceB.length)}
}

This is due to the filtering done by the Get-ChildItem exclude filter. It's kind of a known issue, and if you really want I could probably dig up some reference documentation, but it may take some time. Regardless, GCI doesn't handle wildcards very well when it comes to filtering things. What you are probably better off doing is piping it to a Where command like this:
$username = Get-Content '.\users.txt'
foreach ($un in $username)
{
$destA = "c:\users\$un\redirectedfolders\documents"
$destB = "c:\users\$un\redirectedfolders\desktop"
$sourceA = "C:\users\$un\documents"
$sourceB = "C:\users\$un\desktop"
New-Item -ItemType Directory -Path $destA, $destB
GCI $sourceA -recurse | ?{$_.Extension -ne ".msg" -and !$_.PSIsContainer} | %{
$CurDest = Join-Path $destA $_.FullName.Substring($sourceA.length)
If(!(Test-Path $CurDest.SubString(0,$CurDest.LastIndexOf("\")))){New-Item -Path $CurDest -ItemType Directory|Out-Null}
$_ | Move-Item -Destination $CurDest
}
GCI $sourceB -recurse | ?{$_.Extension -ne ".msg" -and !$_.PSIsContainer} | %{
$CurDest = Join-Path $destB $_.FullName.Substring($sourceB.length)
If(!(Test-Path $CurDest.SubString(0,$CurDest.LastIndexOf("\")))){New-Item -Path $CurDest -ItemType Directory|Out-Null}
$_ | Move-Item -Destination $CurDest
}
}
Edit: Ok, now excludes folders, and also keeps folder structure.
Edit2: Re-designed to do a ForEach loop on the files, build the destination path as $CurDest, test to make sure it exists and make it if it doesn't, then move the files. Also changed mydocuments to documents which is the path to a user's My Documents folder.

Related

Excluding Folders with Get-ChildItem - Need Help Debugging a Script

I've searched through both StackOverflow and SuperUser to try to figure this out, and I'm still getting plagued by a problem I can't figure out how to fix. I know it's something simple, but after playing with it for an hour I'm still stumped. Simple question: how the heck do I tell Get-Childitem to exclude folders?
Right up front here's the code that doesn't work:
$sourceDir="E:\Deep Storage"
$targetDir="W:\Deep Storage"
$excludeThese = 'Projects2','Projects3','Projects4';
Get-ChildItem -Path $sourceDir -Directory -Recurse |
where {$_.fullname -notin $excludeThese} |
Get-ChildItem -Path $sourceDir | ForEach-Object {
$num=1
$nextName = Join-Path -Path $targetDir -ChildPath $_.name
while(Test-Path -Path $nextName)
{
$nextName = Join-Path $targetDir ($_.BaseName + "_$num" + $_.Extension)
$num+=1
}
$_ | Move-Item -Destination $nextName -Force -Verbose -WhatIf
}
}
The underlying concept here already works:
$sourceDir="E:\Deep Storage"
$targetDir="W:\Deep Storage"
Get-ChildItem -Path $sourceDir -File -Recurse | ForEach-Object {
$num=1
$nextName = Join-Path -Path $targetDir -ChildPath $_.name
while(Test-Path -Path $nextName)
{
$nextName = Join-Path $targetDir ($_.BaseName + "_$num" + $_.Extension)
$num+=1
}
$_ | Copy-Item -Destination $nextName -Verbose
}
Basically what this does is it moves folders from one place to another, and if files exist in both places, it renames files coming in. It helps keep my archive drive clear. But there are three folders there that I want to exclude because I still pull assets from them regularly, so I don't need those files moved.
Hence the difference between the two code samples: in the first one, I'm trying to get Get-Childitem to exclude a specific trio of folders, while this second one just grabs everything all at once.
I tried just doing a straight -Exclude with $excludeThese as the variable, without success; I tried skipping the variable approach altogether and just putting the folder names in after -Exclude. Still didn't work. I also tried putting in the entire path to the folders I wanted to exclude. No good--no matter what I did, the -WhatIf showed that the script was trying to move everything, including the folders I was theoretically excluding.
The last trick I tried was one I came across here on SO, and that was to go a gci with the exclude argument first, then do another gci after it. That still failed, so now I have to turn to the experts for help.
I would use a regex string created from the (escaped) directory names to exclude to make sure files withing these folders are ignored.
Also, by using a lookup Hashtable of all file names already present in the target folder, figuring out if a file with a certain name already exists is extremely fast.
$sourceDir = 'E:\Deep Storage'
$targetDir = 'W:\Deep Storage'
$excludeThese = 'Projects2','Projects3','Projects4';
# create a regex string with all folder names to exclude combined with regex OR (|)
$excludeDirs = ($excludeThese | ForEach-Object { [Regex]::Escape($_) }) -join '|'
# create a lookup Hashtable and store the filenames already present in the destination folder
$existingFiles = #{}
Get-ChildItem -Path $targetDir -File | ForEach-Object { $existingFiles[$_.Name] = $true }
Get-ChildItem -Path $sourceDir -File -Recurse |
Where-Object {$_.DirectoryName -notmatch $excludeDirs} |
ForEach-Object {
# construct the new filename by appending an index number if need be
$newName = $_.Name
$count = 1
while ($existingFiles.ContainsKey($newName)) {
$newName = "{0}_{1}{2}" -f $_.BaseName, $count++, $_.Extension
}
# add this new name to the Hashtable so it exists in the next run
$existingFiles[$newName] = $true
# use Join-Path to create a FullName for the file
$newFile = Join-Path -Path $targetDir -ChildPath $newName
$_ | Move-Item -Destination $newFile -Force -Verbose -WhatIf
}
Assuming the excluded directories are at the top:
$sourceDir="E:\Deep Storage"
$excludeThese = 'Projects2','Projects3','Projects4'
get-childitem $sourcedir -exclude $excludethese | get-childitem -recurse

Powershell copying specific files from source directory, excluding several folders, but then recursive with wildcard for files

Here is my current script and it works fine. Not efficient running same code twice but I don't know how to combine the wildcards... anyway on to the bigger issue.
The below code searches through my $sourceDir, excludes the files listed in $ExclusionFiles, copies all folders and structure as well as any .jpg or any .csv files, then puts them into the $targetDir.
$sourceDir = 'c:\sectionOne\Graphics\Data'
$targetDir = 'C:\Test\'
$ExclusionFiles = #("InProgress.jpg", "input.csv", "PCMCSV2.csv")
# Get .jpg files
Get-ChildItem $sourceDir -filter "*.jpg" -recurse -Exclude $ExclusionFiles | `
foreach{
$targetFile = $targetDir + $_.FullName.SubString($sourceDir.Length);
New-Item -ItemType File -Path $targetFile -Force;
Copy-Item $_.FullName -destination $targetFile
}
# Get .csv files
Get-ChildItem $sourceDir -filter "*.csv" -recurse -Exclude $ExclusionFiles | `
foreach{
$targetFile = $targetDir + $_.FullName.SubString($sourceDir.Length);
New-Item -ItemType File -Path $targetFile -Force;
Copy-Item $_.FullName -destination $targetFile
}
My list of files in the main $sourceDir that I need to exclude is getting longer and there are folders I want to exclude as well. Can someone tell me how to,
Copy only a list of specific files in the $sourceDir
Exclude certain folders in $sourceDir from copying
Combine the wildcard search for .jpg and .csv into one statement
I'm still learning so any help would be greatly appreciated!
This is a case where a little bit of Regex will go a long way:
You can filter multiple extensions by using a pretty basic match:
$extensions = 'jpg', 'csv'
$endsWithExtension = "\.(?>$($extensions -join '|'))$"
Get-ChildItem -Recurse |
Where-Object Name -Match $endsWithExtension
You can exclude a list of specific files with one more Where-Object and the -In parameter:
$extensions = 'jpg', 'csv'
$endsWithExtension = "\.(?>$($extensions -join '|'))$"
$ExcludeFileNames = #("InProgress.jpg", "input.csv", "PCMCSV2.csv")
Get-ChildItem -Recurse |
Where-Object Name -Match $endsWithExtension |
Where-Object Name -NotIn $ExcludeFileNames
From there on in, your Foreach-Object is basically correct (nice touch making sure the file exists by using New-Item, though I'd personally assign it's output to null and -PassThru the Copy-Item).
Get-ChildItem $sourceDir -Recurse |
Where-Object Name -Match $endsWithExtension |
Where-Object Name -NotIn $ExcludeFileNames |
Foreach-Object {
$targetFile = $targetDir + $_.FullName.SubString($sourceDir.Length);
New-Item -ItemType File -Path $targetFile -Force;
Copy-Item $_.FullName -destination $targetFile
}

move files with specific extension to folder in higher hierarchy

All my files are in specific folders:
17\1\1\PRO
17\1\2\PRO
17\2\1\PRO
xx\xx\xx\PRO
17 is the year (so 18 for next year etc)
the first 1 is the folder specifying the case number (can be up to 100).
The second 1 is the sub parts on the case number.
That last 1 has a folder PRO in it where all data resides.
We need to move these files, but the files need to stay inside their respective "PRO" folders.
For example:
a file in 17\1\1\pro\xxx\www\ needs to go to 17\1\1\pro\movies
a file in 17\2\2\pro\xxdfsdf\eeee\ needs to go to 17\2\2\pro\movies.
The movies folder should get created if there are files to move.
I need to get a part of the full name of a file and move the file there to the "movie" folder. The problem is I do not know how to split the full name, add \movies to it and move the files there.
This is my code so far:
Get-ChildItem -Path $mypath -Recurse -File -Filter $extension | select $_Fullname |
Move-Item -Force -Destination ($_Fullname.Split("pro"))
If the destination is always "movies subdirectory of the grandparent directory of the file's directory" you can build the destination path relative to the file's location:
Get-ChildItem ... | ForEach-Object {
$dst = Join-Path $_.Directory '..\..\movies'
if (-not (Test-Path -LiteralPath $dst -PathType Container)) {
New-Item -Type Directory -Path $dst | Out-Null
}
Move-Item $_.FullName -Destination $dst
}
If the PRO directory is your anchor you could use a regular expression replacement like this instead:
Get-ChildItem ... | ForEach-Object {
$dst = $_.Directory -replace '^(.*\\\d+\\\d+\\\d+\\PRO)\\.*', '$1\movies'
if (-not (Test-Path -LiteralPath $dst -PathType Container)) {
New-Item -Type Directory -Path $dst | Out-Null
}
Move-Item $_.FullName -Destination $dst
}
If you don't know how many directories there are, I would do something like this:
Get-ChildItem -Path $mypath -Recurse -File -Filter $extension | ForEach-Object {
if ($_.FullName.IndexOf('\PRO\') -gt 0) {
$Destination = Join-Path -Path $_.FullName.Substring(0,$_.FullName.IndexOf('\PRO\') + 5) -ChildPath 'movies';
New-Item $Destination -ItemType Directory -ea Ignore;
$_ | Move-Item -Destination $Destination;
} else {
throw ("\PRO\ path not found in '$($_.FullName)'");
}
}
This will work fine as long as your paths only have \pro\ once. If they have it more than once like customer\pro\17\pro\17\1\1\pro\xx\yy\zz\www and you need the last index, then use $_.FullName.LastIndexOf('\pro\').
If you've got \pro\ directories both before and after the directory that .\pro\movies\ is in, well, you're in trouble. You'll probably have to find a different point of reference.
With a set of folders
17\1\1\PRO
17\1\2\PRO
17\2\1\PRO
You could try the following
$RootPaths = Get-ChildItem -Path C:\folder\*\*\*\pro
$RootPaths will then contain all 3 paths mentioned above and the code below will move all files to the appropriate directory.
ForEach( $Path in $RootPaths)
{
$Movies = Join-Path $Path -Child "Movies"
If( -not (Test-Path $Movies ) ) { New-Item -Path $Movies -ItemType Directory }
Get-ChildItem -Path $Path -Recurse -File -Filter $Extension |
Move-Item -Path $_.FullName -Destination "$( $Path )\Movies"
}
This way it doesn't matter how many levels down your files are. They always get moved to the same directory.

How to create folder structure skeleton using Powershell?

we are having folder/sub folder structure for our application.
Whenever we are adding new modules, we have to copy the folder structures exactly without copying files.
How to copy the folders and subfolders in the hierarchy but leaving the files alone?
This is a bit 'hacky' as Powershell doesn't handle this very well... received wisdom says to use xCopy or Robocopy but if you really need to use Powershell, this seems to work OK:
$src = 'c:\temp\test\'
$dest = 'c:\temp\test2\'
$dirs = Get-ChildItem $src -recurse | where {$_.PSIsContainer}
foreach ($dir in $dirs)
{
$target = ($dir.Fullname -replace [regex]::Escape($src), $dest)
if (!(test-path $target))
{
New-Item -itemtype "Directory" $target -force
}
}
$source = "<yoursourcepath>"
$destination = "<yourdestinationpath>"
Get-ChildItem -Path $source -Recurse -Force |
Where-Object { $_.psIsContainer } |
ForEach-Object { $_.FullName -replace [regex]::Escape($source), $destination } |
ForEach-Object { $null = New-Item -ItemType Container -Path $_ }

Copy a file including it's relative path

I need to copy a large number of files to a backup folder but I want to maintain their relative paths. I only need specific files; i.e.
C:\scripts\folder\File.ext1
C:\scripts\folder2\file2.ext2
C:\scripts\file3.ext1
But I only need to copy the ext1 files like so:
C:\backup\folder\File.ext1.bak
C:\backup\file3.ext1.bak
The source paths are of multiple depths.
This is what I have to copy the files:
$files = gci -path C:\scripts\ -recurse -include *.ext1
$files | % { Copy-Item $_ "$($_).bak"; move-item $_ -destination C:\backup\ }
This just dumps all the files into C:\backup\ and does not appear to get any of the paths. Not sure how that part would be done.
Something like this could work:
gci -path C:\scripts\ -recurse -include *.ext1 |
% { Copy-Item $_.FullName "$($_.FullName).bak"
move-item $_.FullName -destination ($_.FullName -replace 'C:\\scripts\\','C:\backup\') }
It is not clever, but it's quick & dirty and works without a lot of effort.
get-childitem returns absolute paths, but you can make them relative to the current working directory as follows:
resolve-path -relative
So to copy a filtered set of files from the current directory recursively to a destination directory:
$dest = "c:\dest"
$filter = "*.txt"
get-childitem -recurse -include $filter | `
where-object { !$_.PSIsContainer } | `
resolve-path -relative | `
% { $destFile = join-path $dest $_; new-item -type f $destFile -force | out-null; copy-item $_ $destFile; get-item $destfile; }
new-item is needed to create the parent directories
get-item provides a display of all the new files it created
Of course robocopy does all this, but there will be times when you want to do more special filtering or filename mangling...
Use robocopy.
robocopy c:\scripts c:\backup *.ext1 /s
Oops. I failed to notice you wanted to add the .bak extension too. I still think it is a good idea to use robocopy to copy the files then:
dir c:\backup -recurse -include *.ext1 | % { ren $_ "$_.bak" }
You can try this
Clear-Host
$from = "'C:\scripts\"
$to = "'C:\backup\"
$inc = #('*.ext1', '*.extx')
$files = get-childItem -path $from -include $inc -Recurse
$files | % {$dest = (Join-Path $to $($_.FullName+".bak").SubString($from.length)); $dum = New-Item -ItemType file $dest -Force; Copy-Item -Path $_ -Destination $dest -Recurse -Force }
the new-item is there in order to force path creation.
Jean Paul