Get-ChildItem and wildcards and filtering - powershell

I have two different ways of getting files with a wildcard pattern:
Get-ChildItem "$ActivityLogDirectory/*.csv"
and
Get-ChildItem "$ActivityLogDirectory" -Filter *.csv
I prefer to use the latter instead of the former because the former (Get-ChildItem "$ActivityLogDirectory/*.csv") has, on occasion, given me a permission denied error.
They both appear to return the same results, but when I try to compress the resulting files with this command:
Compress-Archive -Update -Path $CsvFiles -DestinationPath C:\Users\admin\Downloads\foo.zip
the former succeeds while the latter fails with the following error:
Compress-Archive : The path 'rgb dev automation store a_1-1_2194_20181120.csv'
either does not exist or is not a valid file system path.
At line:1 char:1
+ Compress-Archive -Update -Path $CsvFiles -DestinationPath C:\Users\ad ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (rgb dev automat...94_20181120.csv:String) [Compress-Archive], InvalidOperationException
+ FullyQualifiedErrorId : ArchiveCmdletPathNotFound,Compress-Archive
So what's the difference between these two ways of getting a listing of files using wildcards? Or perhaps asked another way, why does using -Filter *.csv cause the Compress-Archive cmdlet to fail?

The reason you're seeing different behavior is the - obscurely situational - stringification behavior of the objects output by Get-ChildItem:
This answer details when Get-ChildItem output happens to stringify to a mere filename vs. a full path, and it so happens that Get-ChildItem "$ActivityLogDirectory" -Filter *.csv stringifies to mere filenames.
The workaround is to explicitly stringify the objects as their full paths via their FullName property (PSv3+ syntax):
$CsvFiles = (Get-ChildItem "$ActivityLogDirectory" -Filter *.csv).FullName

If you are running this from a shell with the location of the folder where the CSV files are located then this will work. What you are doing by passing the $CsvFiles variable into Compress-Archive is trying to run against the file name in the current context. To fix this pass the full path $CsvFiles.FullName:
$Csvfiles = (Get-Childitem $ActivityLogDirectory -Filter *.csv)
Compress-Archive -Update -Path $Csvfiles.fullname -DestinationPath C:\Users\admin\Downloads\foo.zip

Related

I want to take the output of Get-PSDrive to get the drive letter of all drives and do a search for ISO's and use the path from the Get-PSDrive

I am trying to use Get-PSDrive to put all the drive letters used by disks into a string that returns each letter (That works)
I am having problems using the $ISODeviceLetter variable to use it as a path to search all drives for .ISO files.
$ISODeviceLetter = Get-PSDrive | Select-Object -ExpandProperty 'Name' | Select-String -Pattern '^[a-z]:$'
$ISOLocation = Get-ChildItem -Path $ISODeviceLetter -Recurse -Include *.ISO
This is the error I get:
Get-ChildItem : Cannot find path 'C:\Users\Administrator\C' because it does not exist.
At line:1 char:16
+ ... OLocation = Get-ChildItem -Path $ISODeviceLetter -Recurse -Include *. ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (C:\Users\Administrator\C:String) [Get-ChildItem], ItemNotFoundException
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetChildItemCommand
I see by the error the get-childitem is using the default location which is the C:\Users\Administrator.
The desired output is to pass $ISODriveLetter as the Path for all drives on the server "C:","D:","E:", so all the drives can be searched for .ISO files.
Thanks ahead of time for the help.
The issue is that when referencing drives, they have to be distinguished using a colon (:) to disambiguate the drive from a regular folder/file:
Use the Foreach-Object command to append the Colon to each drive letter returned.
Changed the filter from -Include to -Filter, to speed up the process as -Filter is implemented by the FileSystemProvider, and you're also not looking for multiple files.
Added -File switch for a more efficient query as well, since an .ISO extension is associated to a file.
$ISODeviceLetter = [array](Get-PSDrive |
Where-Object -FilterScript {$_.Name -match "^.$"}).Foreach{
$_.Name + ':'
}
$ISOLocation = Get-ChildItem -Path $ISODeviceLetter -File -Recurse -Filter *.ISO
Bonus:
You can use Simplified Syntax to achieve the same results as above for better readability:
$ISODeviceLetter = (Get-PSDrive).Where{$_.Name -match "^.$"} |
Select -exp Name | Foreach Insert -ArgumentList 1,':'
Only real "downside" of this is, that you're using the pipeline even more. Even though it may not affect the speed in your return for this example, other time sensitive tasks can take quite a performance hit using the pipeline.

How to skip error of Get-ChildItem returning Access denied

I have a problem with my powershell script, i'm trying to find a file in a folder recursivly. The folder where i am looking is %temp%. Unfortunatly in this folder, there is some folder protected by admin right. Then when i am using Get-ChildItem, it return nothing exept an error (UnauthorizedAccessException).
Here is my code :
$path= (Get-ChildItem -path $ENV:TEMP -force -Recurse -Include logMyApp.txt).FullName
I also tryed with -ErrorAction SilentlyContinue but it does not work.
Thank you for your time :)
Edit : Trying to say Hello, but it does not work, stackoverflow policy ?
Avoid combining the -Recurse and -Include/-Exclude parameters when using Get-ChildItem against the FileSystem provider - they are not mutually exclusive in a technical sense, but their behavior is partially redundant (-Include/-Exclude tries to recurse the file tree independently) and this can sometimes lead to unexpected, buggy and slow enumeration behavior.
For simple inclusion patterns, use -Filter in place of -Include:
$path = (Get-ChildItem -path $ENV:TEMP -force -Recurse -Filter logMyApp.txt -ErrorAction SilentlyContinue).FullName

PowerShell 7Zip4Powershell zipping .zip error

Currently have a PSscript that does 1.65tb of .bmp files sorted into folders by yyyy/MM/dd/HH, replaces the # in the file name then converted to .jpg.
What I can't get past is the next step which gives me an error because it tries to zip the zip file that was created.
How to I prevent that? I tried the -Exclude but I think it has to be done just before the actual compression happens.
tried -Append and it fails for file not found.
(last step is delete the files that were zipped but I believe that can be done with a Remove-Item)
Directory structure looks like this with files in youngest child directory:
D:\Test\Processed\2020\01\13\13
D:\Test\Processed\2020\01\13\14
D:\Test\Processed\2020\01\13\15
D:\Test\Processed\2020\01\13\16
D:\Test\Processed\2020\01\13\17
D:\Test\Processed\2020\01\13\18
D:\Test\Processed\2020\01\13\19
D:\Test\Processed\2020\01\13\20
D:\Test\Processed\2020\01\13\21
D:\Test\Processed\2020\01\13\22
D:\Test\Processed\2020\01\13\23
PowerShell script
$sourceRootPath = "D:\Test\Processed\2020\01\13"
$targetRootPath = "D:\Test\Processed\2020\Archived"
Get-ChildItem -Path $sourceRootPath -Recurse -Exclude *.zip | Where-Object {$_.PSIsContainer} | ForEach-Object {
$directoryFullName = $_.FullName
$directoryName = $_.Name
#$folderPathToCompress - redundant but keeps my thinking straight for now
$folderPathToCompress = $directoryFullName
# This creates an error that the .zip file does not exist and exits
#Compress-7Zip -Path $folderPathToCompress -ArchiveFileName $directoryFullName\$directoryName.zip -Format Zip -CompressionLevel Ultra -Append
# This creates the .zip file BUT creates an error when it tries to zip the .zip file it is creating
#Compress-7Zip -Path $folderPathToCompress -ArchiveFileName $directoryFullName\$directoryName.zip -Format Zip -CompressionLevel Ultra
}
I will get this posting stuff figure out, sorry for the errors, I am open to learning two things at once but it takes 4x longer... so back at it.
1.) I am using the 7Zip4Powershell Module - it looked like a good idea, but maybe I should stick with passing everything to a variable and then Invoke-Expression thus not use the module but the 7zip command directly(?)
2.) Made a few attempts with same results but issue not resolved ... yet
This is what the error looks like and the different attempts below.
Compress-7Zip : The process cannot access the file 'D:\Test\Processed\2020\01\13\13\13.zip' because it is being used by another process.
At C:\Users\moe3srv\Desktop\test2.ps1:19 char:6
+ (Compress-7Zip -Path $folderPathToCompress -ArchiveFileName $dire ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (SevenZip4PowerS...+CompressWorker:CompressWorker) [Compress-7Zip], IOException
+ FullyQualifiedErrorId : err01,SevenZip4PowerShell.Compress7Zip
(Get-ChildItem -Path $sourceRootPath -Recurse)
(Get-ChildItem -Path $sourceRootPath -Recurse -Exclude .zip )
(Get-ChildItem -Path $sourceRootPath -Recurse -Exclude *.zip )
Alternatively in PowerShell we can user Compress-Archive
eg,
Compress-Archive -Path C:\Reference\* -DestinationPath C:\Archives\Draft.zip

Compress-Archive Error: Cannot access the file because it is being used by another process

I would like to zip a path (with a service windows running inside).
When the service is stopped, it works perfectly, when the service is running, I have the exception:
The process cannot access the file because it is being used by another
process.
However, when I zip with 7-zip, I don't have any exception.
My command:
Compress-Archive [PATH] -CompressionLevel Optimal -DestinationPath("[DEST_PATH]") -Force
Do you have any idea to perform the task without this exception?
Copy-Item allows you to access files that are being used in another process.
This is the solution I ended up using in my code:
Copy-Item -Path "C:\Temp\somefolder" -Force -PassThru |
Get-ChildItem |
Compress-Archive -DestinationPath "C:\Temp\somefolder.zip"
The idea is that you pass through all the copied items through the pipeline instead of having to copy them to a specific destination first before compressing.
I like to zip up a folder's content rather than the folder itself, therefore I'm using Get-ChildItem before compressing in the last line.
Sub-folders are already included. No need to use -recurse in the first line to do this
A good method to access files being used by another process is by creating snapshots using Volume Shadow Copy Service.
To do so, one can simply use PowerShells WMI Cmdlets:
$Path = "C:/my/used/folder"
$directoryRoot = [System.IO.Directory]::GetDirectoryRoot($Path).ToString()
$shadow = (Get-WmiObject -List Win32_ShadowCopy).Create($directoryRoot, "ClientAccessible")
$shadowCopy = Get-WmiObject Win32_ShadowCopy | ? { $_.ID -eq $shadow.ShadowID }
$snapshotPath = $shadowCopy.DeviceObject + "\" + $Path.Replace($directoryRoot, "")
Now you can use the $snapshotPath as -Path for your Compress-Archive call.
This method can also be used to create backups with symlinks.
From there on you can use the linked folders to copy backed up files, or to compress them without those Access exceptions.
I created a similiar function and a small Cmdlet in this Gist: Backup.ps1
There was a similar requirement where only few extensions needs to be added to zip.
With this approach, we can copy the all files including locked ones to a temp location > Zip the files and then delete the logs
This is bit lengthy process but made my day!
$filedate = Get-Date -Format yyyyMddhhmmss
$zipfile = 'C:\Logs\logfiles'+ $filedate +'.zip'
New-Item -Path "c:\" -Name "Logs" -ItemType "directory" -ErrorAction SilentlyContinue
Robocopy "<Log Location>" "C:\CRLogs\" *.txt *.csv *.log /s
Get-ChildItem -Path "C:\Logs\" -Recurse | Compress-Archive -DestinationPath $zipfile -Force -ErrorAction Continue
Remove-Item -Path "C:\Logs\" -Exclude *.zip -Recurse -Force

Using Get-ChildItem -Exclude or -Include returns nothing

I have a list in my C: directory that has many files. If I try to run an -Exclude on it, I get no returns. Same with -Include. If I use -Filter, it returns what I expected to get back. Am I not understanding what it should be doing?
Here is an example of what I am running and getting nothing:
Get-ChildItem -Path C: -Exclude "*.txt"
I get nothing back. If I run
Get-Childitem -filter "*.txt"
I get this back:
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 11/7/2007 8:00 AM 17734 eula.1028.txt
-a---- 11/7/2007 8:00 AM 17734 eula.1031.txt
-a---- 11/7/2007 8:00 AM 10134 eula.1033.txt
-a---- 11/7/2007 8:00 AM 17734 eula.1036.txt
-a---- 11/7/2007 8:00 AM 17734 eula.1040.txt
-a---- 11/7/2007 8:00 AM 118 eula.1041.txt
-a---- 11/7/2007 8:00 AM 17734 eula.1042.txt
-a---- 11/7/2007 8:00 AM 17734 eula.2052.txt
-a---- 11/7/2007 8:00 AM 17734 eula.3082.txt
7/7/2016 8:50 AM 93 HaxLogs.txt
-a---- 7/8/2016 8:30 AM 0 Test.txt
To summarize and complement gravity's and Brian Reynolds's helpful answers:
There are two distinct problems with your approach:
Targeting C: probably doesn't (always) do what you want, because C: refers to whatever happens to be the current location (working dir.) on drive C: at the moment.
To target the root folder of drive C:, you must use C:\, which I'll assume is what you meant in the remainder of this answer.
Using the -Exclude (and also -Include) parameter with neither -Recurse nor a -Path value ending in \* often yields NO results. Unexpected? Indeed - see below for more.
Thus, Get-Item -Path C:\* -Exclude *.txt - note the switch from Get-ChildItem to Get-Item and the * after C:\ - is needed to make your command work for the items located directly in C:\ only.
Background information:
Using the provider-native -Filter parameter is generally preferable to -Include, because:
it is much faster than -Include due to the provider itself performing the filtering at the source, as opposed to letting PowerShell apply the filter later, after all objects have been received.
it doesn't require you to switch to Get-Item and append \* to the -Path parameter value.
Get-ChildItem -Path C:\ -Filter *.txt works fine for matching all *.txt files in the root directory of C:, for instance.
That said, there are caveats:
The wildcard pattern language supported by -Filter has fewer features than PowerShell's - notably, it doesn't support character sets/ranges such as [0-9] - may unexpectedly match short (8.3) filenames, and has other legacy quirks - see this well-researched answer for the gory details.
-Filter supports only a single pattern, whereas -Include supports multiple ones (an array of patterns).
Unfortunately, -Filter is always a positive (inclusionary) filter and therefore cannot be used to provide the functionality of -Exclude.
The implementation of -Include / -Exclude with Get-ChildItem is unintuitive and has pitfalls:
Side note: if you only use one -Include pattern (as opposed to -Exclude), it's easier to append the pattern directly to the -Path argument; e.g.: Get-ChildItem C:\*.txt
tl;dr:
To get predictable behavior with -Include / -Exclude if you're not also using -Recurse (if you are using -Recurse this workaround is not needed):
# IMPORTANT: Workaround isn't needed if you're using -Recurse.
# * "\*" was appended to the input path
# * Get-*ChildItem* was switched to Get-*Item*
Get-Item C:\path\to\* -Include ...
Get-Item C:\path\to\* -Exclude ...
In PowerShell (Core) v7+, if your input paths are literal ones, you can alternatively use Get-ChildItem with -LiteralPath rather than (possibly positionally implied) -Path (use . for the current dir.):
# IMPORTANT: Works in PowerShell (Core) only.
# Note the use of -LiteralPath.
Get-ChildItem -LiteralPath C:\path\to -Include ...
Get-ChildItem -Literalpath C:\path\to -Exclude ...
There's an outright bug in Windows PowerShell where, when -LiteralPath is used, -Include / -Exclude are quietly ignored. This has been fixed in PowerShell (Core) as of (at least) v7.0 (and it's fair to assume that it will not be fixed in Windows PowerShell, which will only receive critical fixes going forward).
-Include and -Exclude do not work as one would intuitively expect, which is the subject of GitHub issue #3304:
-Include and -Exclude modify the leaf (last) path component of the -Path argument, i.e. file or directory names in the case of file-system paths.
That means that the patterns are first applied to the leaf component of the specified folder path itself, before getting applied to the child items, if at all.
If the input path doesn't end in \* and -Recurse is not specified, the implications are as follows:
-Include: If the input path's last path component does not match the -Include pattern(s), the input path itself is excluded (not included), and the path's child items are never looked at - nothing is output.
-Exclude: Analogously, if the input path's last path component does match the -Exclude pattern(s), the input path itself is excluded, and the path's child items are never looked at - nothing is output.
Targeting a root directory - e.g., Get-ChildItem -Path C:\ -Exclude Windows) appears to be broken altogether as of v7.0: it either produces no output at all, or fails on Unix-like platforms, both with -Include and -Exclude, irrespective of the patterns used - see GitHub issue #11649.
As stated, the problem doesn't surface if -Recurse is used, because that forces descending into the input path's subtree, even if the input path itself is not included / excluded.
Unless -Recurse is needed, the only way to get expected behavior is to replace
Get-ChildItem C:\path\to -Include / -Exclude with
Get-Item C:\path\to\* -Include / -Exclude - note the use of Get-Item instead of Get-ChildItem, and that \* was appended to the -Path argument.
By contrast, if you use Get-ChildItem * in combination with -Exclude and there are directories among the non-excluded items, Get-ChildItem will unexpectedly output their contents instead; this does not happen with -Include and generally doesn't happen with subdirectory matches by a wildcard expression (-Path argument and/or -Filter argument).
A summary of the problems as of PowerShell 7.2:
GitHub issue #3304 (discussed above): counterintuitive application of -Include / -Exclude patterns only to the input themselves rather than to their children.
GitHub issue #11649: Get-ChildItem -Path <rootPath> -Exclude <anyPattern> unexpectedly produces no output (even though nothing should be excluded, given that the exclusion pattern is normally applied to the input paths, such as / or c:\).
GitHub issue #9126: -Include / -Exclude unexpectedly follow symlinks when -Recurse is used.
GitHub issue #8662: performance problem: -Include / -Exclude are slower(!) than after-the-fact filtering with Where-Object.
A related feature request is GitHub issue #15159, which suggests introducing the ability to exclude subfolder subtrees (as opposed to just excluding the items matching the patterns themselves, but not their children), with a new parameter such as -ExcludeSubtree.
Examples: problematic uses of -Include / -Exclude
Note: To make all commands below work as one would intuitively expect, replace Get-ChildItem C:\Windows with Get-Item C:\Windows\* - note the use of a different cmdlet, Get-Item, and the appended \*.
# HAPPENS TO WORK, BUT IS NOT ROBUST:
# Matches all w* items *inside* C:\Windows, but
# ONLY because w* happens to match 'Windows' - the last input
# path component - too.
Get-ChildItem C:\Windows -Include w*
# HAPPENS TO WORK, BUT IS NOT ROBUST:
# Matches all items whose names *don't* start with a-v *inside* C:\Windows, but
# ONLY because [a-v]* happens not to exclude 'Windows' - the last input
# path component - too.
Get-ChildItem C:\Windows -Exclude [a-v]*
# OUTPUTS NOTHING:
# Because t* doesn't match 'Windows', the child items of
# 'C:\Windows' are not considered.
Get-ChildItem C:\Windows -Include t*
# OUTPUTS NOTHING:
# Because w* matches 'Windows', it is excluded, and
# the child items of 'C:\Windows' are not considered.
Get-ChildItem C:\Windows -Exclude w*
Get-ChildItem -Path "C:\*" -Include "*.txt"
This example, of how -Include should work, will give you the results you were expecting. Note that I provided a wildcard in the path parameter as well, to explicitly define the path as "any file in the root C:" as opposed to "C:" itself.
Source:
https://technet.microsoft.com/library/hh849800.aspx
Example 3 from this link, in case it goes defunct (note the wildcard in path here, as well):
C:\> Get-ChildItem –Path "C:\Windows\Logs\*" -Include "*.txt" -Exclude "A*"
Using 'C:' with no slash after it is the reason you're not getting the results you want. This is being interpreted as a relative path to the current directory instead of the root of the C drive. (See: Path with no slash after drive letter and colon - what does it point to?)
If you use 'C:\' instead it should work as expected.
Edit: My mistake; I was responding specifically to the examples which only use '-exclude'.
This seems to be a bug (because the string version of the parent properties is empty string?). With subdirectories, it works fine:
get-childitem c:\windows -directory -exclude *.dll
You can specify the root directory in a different way, and get a strange error message:
get-childitem Microsoft.PowerShell.Core\FileSystem::C:\ -exclude *.txt
get-childitem : Cannot process argument because the value of argument "path" is not valid. Change the value of the "path" argument and run the operation again.
At line:1 char:1
+ get-childitem Microsoft.PowerShell.Core\FileSystem::C:\ -exclude wind ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Get-ChildItem], PSArgumentException
+ FullyQualifiedErrorId : Argument,Microsoft.PowerShell.Commands.GetChildItemCommand
Or, in osx:
get-childitem / -directory -exclude *.txt
get-childitem : Cannot process argument because the value of argument "path" is not valid. Change the value of the "path" argument and run the operation again.
At line:1 char:1
+ get-childitem / -directory -exclude var
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Get-ChildItem], PSArgumentException
+ FullyQualifiedErrorId : Argument,Microsoft.PowerShell.Commands.GetChildItemCommand
This is a workaround for powershell 6 and above:
get-item c:\ | get-childitem -exclude *.txt