I've searched through both StackOverflow and SuperUser to try to figure this out, and I'm still getting plagued by a problem I can't figure out how to fix. I know it's something simple, but after playing with it for an hour I'm still stumped. Simple question: how the heck do I tell Get-Childitem to exclude folders?
Right up front here's the code that doesn't work:
$sourceDir="E:\Deep Storage"
$targetDir="W:\Deep Storage"
$excludeThese = 'Projects2','Projects3','Projects4';
Get-ChildItem -Path $sourceDir -Directory -Recurse |
where {$_.fullname -notin $excludeThese} |
Get-ChildItem -Path $sourceDir | ForEach-Object {
$num=1
$nextName = Join-Path -Path $targetDir -ChildPath $_.name
while(Test-Path -Path $nextName)
{
$nextName = Join-Path $targetDir ($_.BaseName + "_$num" + $_.Extension)
$num+=1
}
$_ | Move-Item -Destination $nextName -Force -Verbose -WhatIf
}
}
The underlying concept here already works:
$sourceDir="E:\Deep Storage"
$targetDir="W:\Deep Storage"
Get-ChildItem -Path $sourceDir -File -Recurse | ForEach-Object {
$num=1
$nextName = Join-Path -Path $targetDir -ChildPath $_.name
while(Test-Path -Path $nextName)
{
$nextName = Join-Path $targetDir ($_.BaseName + "_$num" + $_.Extension)
$num+=1
}
$_ | Copy-Item -Destination $nextName -Verbose
}
Basically what this does is it moves folders from one place to another, and if files exist in both places, it renames files coming in. It helps keep my archive drive clear. But there are three folders there that I want to exclude because I still pull assets from them regularly, so I don't need those files moved.
Hence the difference between the two code samples: in the first one, I'm trying to get Get-Childitem to exclude a specific trio of folders, while this second one just grabs everything all at once.
I tried just doing a straight -Exclude with $excludeThese as the variable, without success; I tried skipping the variable approach altogether and just putting the folder names in after -Exclude. Still didn't work. I also tried putting in the entire path to the folders I wanted to exclude. No good--no matter what I did, the -WhatIf showed that the script was trying to move everything, including the folders I was theoretically excluding.
The last trick I tried was one I came across here on SO, and that was to go a gci with the exclude argument first, then do another gci after it. That still failed, so now I have to turn to the experts for help.
I would use a regex string created from the (escaped) directory names to exclude to make sure files withing these folders are ignored.
Also, by using a lookup Hashtable of all file names already present in the target folder, figuring out if a file with a certain name already exists is extremely fast.
$sourceDir = 'E:\Deep Storage'
$targetDir = 'W:\Deep Storage'
$excludeThese = 'Projects2','Projects3','Projects4';
# create a regex string with all folder names to exclude combined with regex OR (|)
$excludeDirs = ($excludeThese | ForEach-Object { [Regex]::Escape($_) }) -join '|'
# create a lookup Hashtable and store the filenames already present in the destination folder
$existingFiles = #{}
Get-ChildItem -Path $targetDir -File | ForEach-Object { $existingFiles[$_.Name] = $true }
Get-ChildItem -Path $sourceDir -File -Recurse |
Where-Object {$_.DirectoryName -notmatch $excludeDirs} |
ForEach-Object {
# construct the new filename by appending an index number if need be
$newName = $_.Name
$count = 1
while ($existingFiles.ContainsKey($newName)) {
$newName = "{0}_{1}{2}" -f $_.BaseName, $count++, $_.Extension
}
# add this new name to the Hashtable so it exists in the next run
$existingFiles[$newName] = $true
# use Join-Path to create a FullName for the file
$newFile = Join-Path -Path $targetDir -ChildPath $newName
$_ | Move-Item -Destination $newFile -Force -Verbose -WhatIf
}
Assuming the excluded directories are at the top:
$sourceDir="E:\Deep Storage"
$excludeThese = 'Projects2','Projects3','Projects4'
get-childitem $sourcedir -exclude $excludethese | get-childitem -recurse
I'm trying to have all the files in the folder structure copied to a folder that is also part of that structure. So the destination folder is excluded form the search. I also want to exclude any folder that has in its path ".thumbnails" but when I replace the full path in the $Skip with a wild card path such as 'D:\ZZZ_Phone_test*.thumbnails' it won't work.
Secondly, I'd like to make this more efficient if possible so the job can be finished quicker. When the script is running it is mostly the CPU working not so much the harddrive.
Thirdly, is there any way how to generate some output of what was copied, skipped, errors... and save it to a logfile?
$Source = 'D:\ZZZ_Phone_test'
$Dest = 'D:\ZZZ_Phone_test\1\1\BackUp'
$Skip = 'D:\ZZZ_Phone_test\4\.thumbnails'
Get-ChildItem $Source -Directory -Recurse | ? FullName -ne $Dest | ? FullName -ne $Skip | get-ChildItem -File | Copy-Item -Exclude `
*.0,*.1,*.nomedia,*.thumbnail,*.chck,*.crypt12,*.tmp,*.db,*.crypt1,*.ini,*.pdrproj,*.pkpass,*.dat,*.enc,*.lck,*.xml,*.json,*.LOCK,*.443,*.preference `
-Destination $Dest
.
EDIT: the following works but it will only exclude files in directories whose names end with "thumbnails" or "BackUp". If there are any directories with files inside of "thumbnails" folder they will all be processed. I'd like to define the folders to be excluded the way that even if there are subdirectories with files in a directory defined in $Skip they would not be processed.
$Source = 'D:\ZZZ_Phone_test'
$Dest = 'D:\ZZZ_Phone_test\1\1\BackUp'
$Skip = '*thumbnails', '*BackUp'
(Get-ChildItem -Path $Source -Directory -Recurse -Exclude $Skip).FullName |
get-ChildItem -File |
Copy-Item -WhatIf -Exclude `
*.0,*.1,*.nomedia,*.thumbnail,*.chck,*.crypt12,*.tmp,*.db,*.crypt1,*.ini,*.pdrproj,*.pkpass,*.dat,*.enc,*.lck,*.xml,*.json,*.LOCK,*.443,*.preference `
-Destination $Dest
Try this and modify as you wish for that file exclusion section...
$Source = 'D:\Temp'
$Dest = 'D:\Destination'
$Skip = '*est', '*here'
<#
Always build you code one use case at a time to ensure you are getting what
you'd expect before moving ot the next.
#>
# Get all directories off a give path
(Get-ChildItem -Path $Source -Directory -Recurse).FullName |
Select-Object -First 5
# Results
<#
D:\Temp\AddressFiles
D:\Temp\ChildFolder
D:\Temp\est
D:\Temp\here
D:\Temp\hold
#>
# Exclude named directories
(Get-ChildItem -Path $Source -Directory -Recurse -Exclude $Skip).FullName |
Select-Object -First 5
# Results
<#
D:\Temp\AddressFiles
D:\Temp\ChildFolder
D:\Temp\ChildFolder\New folder
D:\Temp\ChildFolder\temp
D:\Temp\hold
#>
# Or include only what you want
(Get-ChildItem -Path $Source -Directory -Recurse -Include $Skip).FullName |
Select-Object -First 5
# Results
<#
D:\Temp\ChildFolder\New folder\est
D:\Temp\est
D:\Temp\here
#>
# Loop directories and process files, trap errors
(Get-ChildItem -Path $Source -Directory -Recurse -Exclude $Skip).FullName |
Select-Object -First 5 |
ForEach {
Try
{
"Processing $PSItem"
$CopyItemSplat = #{
Path = (Get-ChildItem -Path $PSItem -ErrorAction Stop).FullName
Destination = $Dest
Verbose = $true
WhatIf = $true
}
}
Catch
{
Write-Warning -Message 'An error was encountered.'
$PSitem.Exception.Message
}
}
# Results
<#
Processing D:\Temp\AddressFiles
Processing D:\Temp\ChildFolder
Processing D:\Temp\ChildFolder\New folder
Processing D:\Temp\ChildFolder\temp
WARNING: An error was encountered.
The property 'FullName' cannot be found on this object. Verify that the property exists.
Processing D:\Temp\hold
WARNING: An error was encountered.
The property 'FullName' cannot be found on this object. Verify that the property exists.
#>
For complex filtering needs, divide et impera is often useful an approach. That is, simplify the problem in multiple steps instead of trying to write an one-liner.
Let's take a directory listing of all the files and exclude the destination directory $dest. As % (shorthand for Where-Object) parameter -notmatch expects a regular expression, the $dest path is escaped with [regex]::Escape. This needs to be done, as backslash \ is a reserved character in regular expressions. One could write the path in escaped form in the first hand, like c:\\my\\path\\to\\somewhere, but Escape does all the work needed.
Get-ChildItem $source\* -Recurse -File | ? { $_.psparentpath -notmatch [regex]::escape($dest) }
Now that we have all the files except destination, start pruning the list. Since there are lots of file extensions, let's put those on an array. Loop through the array, and remove each match from the $files array.
$excluded = #("*.0", "*.1", "*.nomedia", "*.thumbnail", "*.chck", "*.crypt12", "*.tmp", "*.db",
"*.crypt1", "*.ini", "*.pdrproj", "*.pkpass", "*.dat", "*.enc", "*.lck", "*.xml", "*.json",
"*.LOCK", "*.443", "*.preference")
foreach($ex in $excluded) {
$files = $files | ? {$_.extension -notlike $ex}
}
To remove the $skip, filter the collection again:
$files = $files | ? {$_.DirectoryName -ne $skip)
At this point, all you have is an array that contains files that are to be copied into $dest. Before copying, use -WhatIf switch to see what Copy-Item would do to be sure the copy works as intended:
$files | % { Copy-Item -WhatIf $_ $dest }
To wrap up a complete example,
$source = "c:\temp\phonetest"
$dest = "c:\temp\phonetest\1\1\BackUp"
$Skip = "c:\temp\phonetest\skipme"
# Get list of all the files
Get-ChildItem $source\* -Recurse | ? { $_.psparentpath -notmatch [regex]::escape($dest) }
# Filter by extension
$excluded = #("*.0", "*.1", "*.nomedia", "*.thumbnail", "*.chck", "*.crypt12", "*.tmp", "*.db", "*.crypt1", "*.ini", "*.pdrproj", "*.pkpass", "*.dat", "*.enc", "*.lck", "*.xml", "*.json", "*.LOCK", "*.443", "*.preference")
foreach($ex in $excluded) {
$files = $files | ? {$_.extension -notlike $ex}
}
# Skip specific dir
$files = $files | ? {$_.DirectoryName -ne $skip)
# See what would be copied
$files | % { Copy-Item -WhatIf $_ $dest }
Get-ChildItem -Path $dir -Recurse -File | ForEach-Object `
{
Write-Host $_.fullname
}
Get-ChildItem : A parameter cannot be found that matches
parameter name 'File'.
Version
-------
5.1.17763.771
I have many other ps scripts running that use this and it works fine, but for some reason it's not working in this new script I wrote. What's going on?
Thank you everyone for helping.
The value of $dir was
" S:\folder\folder\"
Removing the space before the drive letter resolved the issue.
In addition to the OP's particular instance, there are other cases where this error can occur. Get-ChildItem can be used on other providers besides the file provider (e.g., the registry), and the -File switch is not valid in those cases.
Example:
$dir = 'C:\Temp'
Get-ChildItem -Path $dir -File # Works just fine
$dir = 'HKCU:\Software'
Get-ChildItem -Path $dir # Works without -File switch
Get-ChildItem -Path $dir -File # Throws "parameter cannot be found" error
That wasn't the source of the problem for me.
All I did was surround the variable with ( ) and that resolve it.
In my case I wanted the full filename of the 'found' file
$curdir = "C:\Program Files\"
(Get-Childitem -Path ($curdir) -File -filter "node.exe" -recurse -Force -ErrorAction SilentlyContinue).Directory.FullName
In your case I'm sure just using this would work for you:
Get-ChildItem -Path ($dir) -Recurse -File | ForEach-Object `
{
Write-Host $_.fullname
}
Im testing out a script on Pester. I keep getting a Path cannot be found error.
Apparently something is wrong with some of my lines.
$userFolders = Get-ChildItem C:\Users -Directory
foreach($folder in $userFolders)
{
Get-ChildItem C:\Users\$folder\AppData\Local\Temp -Recurse | Remove-Item -Recurse -Force
Get-ChildItem C:\Users\$folder\AppData\Local\Google\Chrome\User Data\Default\Cache -Recurse | Remove-Item -Recurse -Force
}
Get-ChildItem C:\Windows\Temp -Recurse | Remove-Item -Recurse -Force
I can't seem to find what is wrong, but I know it is somewhere here.
Any help would be greatly appreciated.
$folder is an object. Casting to string will give you the folder name. You can do this by using quotes.
As Mark has identified, you would also need quotes if there a space in the path.
$userFolders = Get-ChildItem C:\Users -Directory
foreach($folder in $userFolders) {
Write-Host "This is the folder object:"
$folder
Write-Host "This is folder object cast to string: $folder"
Get-ChildItem "C:\Users\$folder\AppData\Local\Temp" -Recurse | Remove-Item -Recurse -Force
Get-ChildItem "C:\Users\$folder\AppData\Local\Google\Chrome\User Data\Default\Cache" -Recurse | Remove-Item -Recurse -Force
}
Get-ChildItem C:\Windows\Temp -Recurse | Remove-Item -Recurse -Force
One of your paths has a space in it so needs to wrapped in quotes:
Get-ChildItem "C:\Users\$folder\AppData\Local\Google\Chrome\User Data\Default\Cache" -Recurse | Remove-Item -Recurse -Force
It needs to be double quotes so that the $folder variable is still expanded.
The other answers have pointed out the main flaw in your script which is the lack of quotes. There is also some redundancy in two ways.
Repetition of Get-ChildItem <something> | Remove-Item <something>
Recursion on both sides of the pipe.
Consider the following:
function Get-TempFolderItem {
foreach ($Local in (Get-ChildItem -Path "C:\Users\*\AppData\Local")) {
'Temp','Google\Chrome\User Data\Default\Cache' |
ForEach-Object { Get-ChildItem -Path "$Local\$_" -Recurse }
}
}
Get-TempFolderItem | Remove-Item -Force
Here there is less repetition which simplifies debugging if you make a mistake. And if the getting of directory content is exhaustive, the removal shouldn't have to be.
Because you are saying that this is in the context of testing you should probably look into mocking for something like removal of files. With temp-files it doesn't matter much if you delete something you didn't intend to, but generally that is something you need to avoid when testing.
You can do this by writing
Mock Remove-Item {}
inside the Describe block.
Hope this answer is useful to you.
Probably not every user folder has a 'Google' sub-directory
C:\Users\$folder\AppData\Local\**Google**\...
I need to traverse a folder (ParentLogFolder), find logs and move them from the relative folder (SUBLOG) to a subfolder (CompressedLogFolder)
ParentLogfolder
---SubLog
------CompressedLogFolder
---SubLog
------CompressedLogFolder
Folder structure, not limited to 2 SubLogfolders, can be 100+
$Path = "C:\ParentLogFolder"
$Pattern = "*.log"
$date = get-date
$AgeLimit = $date.AddDays(-31)
Get-ChildItem -Filter $Pattern -path $path -recurse | Where-Object
{$_.LastWriteTime -lt $AgeLimit} | Move-Item -Path
I can't figure out how to get the files of the parent folder into a variable and add the subfolder as destination in the Move-Item part.
Anyone out there, who can help answer this?
Don't mind the $Date part of the code, it works, it's just there because it needs to be in the solution.
Instead of onliner, consider emphasis on code clarity. That is, break the process in substeps by divide and conquer approach. It's easier to check variables for incorrect values when the pipeline isn't used too much. Like so,
$allFiles = Get-ChildItem -Filter $Pattern -path $path -recurse
$oldFiles = $allFiles | ? { $_.LastWriteTime -lt $ageLimit }
foreach($file in $oldFiles) {
$archiveFolder = join-path $file.DirectoryName 'someArchiveFolder'
$destination = join-path $archiveFolder $file.Name
move-item -whatif $file.FullName $destination
}
The -whatif switch will print what the move command would do. Remove it to actually move files.