I am trying to exclude the "recycle bin", "Windows", and "Program Files" folders in my recursive call to Get-ChildItem.
I figured it would just be -Exclude '$Recycle.Bin', 'Windows', 'Program Files' or -Exclude 'C:\$Recycle.Bin', 'C:\Windows', 'C:\Program Files', but neither of these give me the wanted result. Any ideas? Thanks
Exclude does not filter out child objects found within excluded directories when you use the Recurse option. The answer over here contains a good explanation about this:
How to exclude files and folders from Get-ChildItem in PowerShell?
You can accomplish what you want by stringing together multiple calls as the other answer suggested. Here is a sample one-liner PowerShell command that lists all files greater than 1GB in size, excluding the directories you listed:
Get-ChildItem C:\ -Directory | Where-Object Name -NotIn #('Windows','Program Files','$Recycle.Bin') | % { Get-ChildItem -File $_.FullName -Recurse} | Where-Object {($_.Length /1GB) -gt 1} | Sort -Descending -Property Length | Format-Table Length, FullName -AutoSize -Wrap
Here is a breakdown of how this one-liner works:
The first Get-ChildItem returns all Top Level directories under your C:\ drive.
These directories are then piped (|) into the Where-Object cmdlet, where we use the -NotIn parameter to specify an array of directory names to exclude.
This filtered set of Top Level directories is then passed into the ForEach-Object cmdlet denoted here with the % alias.
Each of the directories is then passed in one-by-one into another Get-ChildItem cmdlet which does the recursive search you desire. (Note that I used the -File filter to only return files)
That should cover what you asked for in your original question. The second Where-Object cmdlet filters files over 1GB in size, then those get sorted by file size, and the result is printed out in table format. You probably want to replace these parts with whatever you were trying to do.
If you want to use different types of filters instead of the NotIn filter, take a look at all the other options availabe in the Where-Object doc.
You could use this syntax instead to achieve the same effect.
$Directories = Get-ChildItem c:\ -Directory | Where-Object Name -NotIn #('Windows','Program Files','$Recycle.Bin')
$Output = $Directories | % { Get-ChildItem $_.FullName -Recurse}
Related
I'm working on a script to output some data from multiple files based on a string search. It outputs the string found, followed by the following six characters. I can get this to work for an exact location. However, I want to search across files inside multiple subfolders in the path. Using the below script, I get PermissionDenied errors...
[regex] $pattern = '(?<=(a piece of text))(?<chunk>.*)'
Get-Content -Path 'C:\Temp\*' |
ForEach-Object {
if ($_ -match $pattern) {
$smallchunk = $matches.chunk.substring(0, 6)
}
}
"$smallchunk" | Out-File 'C:\Temp\results.txt'
If I change -Path to one of the subfolders, it works fine, but I need it to go inside each subfolder and execute the get-content.
e.g., look inside...
C:\Temp\folder1\*
C:\Temp\folder2\*
C:\Temp\folder3\*
And so on...
Following up on boxdog's suggestion of Select-String, the only limitation would be folder recursion. Unfortunately, Select-String only allows the searching of multiple files in one directory.
So, the way around this is piping the output of Get-ChildItem with a -Recurse switch into Select-String:
$pattern = "(?<=(a piece of text))(?<chunk>.*)"
Get-ChildItem -Path "C:\Temp\" -Exclude "results.txt" -File -Recurse |
Select-String -Pattern $pattern |
ForEach-Object -Process {
$_.Matches[0].Groups['chunk'].Value.Substring(0,6)
} | Out-File -FilePath "C:\Temp\results.txt"
If there's a need for the result to be saved to $smallchunk you can still do so inside the loop if need be.
Abraham Zinala's helpful answer is the best solution to your problem, because letting Select-String search your files' content is faster and more memory-efficient than reading and processing each line with Get-Content.
As for what you tried:
Using the below script I get PermissionDenied errors...
These stem from directories being among the file-system items output by Get-ChildItem, which Get-Content cannot read.
If your files have distinct filename extensions that your directories don't, one option is to pass them to the (rarely used with Get-Content) -Include parameter; e.g.:
Get-Content -Path C:\Temp\* -Include *.txt, *.c
However, as with Select-String, this limits you to a single directory's content, and it doesn't allow you to limit processing to files fundamentally, if extension-based filtering isn't possible.
For recursive listing, you can use Get-ChildItem with -Recurse, as in Abraham's answer, and pipe the file-info objects to Get-Content:
Get-ChildItem -Recurse C:\Temp -Include *.txt, *.c | Get-Content
If you want to simply limit output to files, whatever their name is, use the -File switch (similarly, -Directory limits output to directories):
Get-ChildItem -File -Recurse C:\Temp | Get-Content
I am wanting to filter through a list of folders that include a certain criteria while excluding another set of criteria in PowerShell.
The code below is what is currently being played around with that has had partial succession; -Exclude seems to exclude folders that contain *EDS* in the name but not *eng*. Below is a sample output of what is still being included:
C:\Users\USERNAME\Box\CTRL Active Projects\project1\project1_OPS\06-Software\04-Database Backups\01-Record Backup
C:\Users\USERNAME\Box\CTRL Active Projects\project1\project1_OPS\06-Software\04-Database Backups
C:\Users\USERNAME\Box\CTRL Active Projects\project1\project1_OPS\01-Eng\03-Submittals WIP\Backups
The -Exclude *eng*, *EDS* might eventually be converted over to a variable/named arraylist.
# Find sub-folder within job folder
$DatabaseFolder = (Get-ChildItem -Path $JobFolder -Filter "*Backup*" -Exclude *eng*, *EDS* -Recurse -Directory).Fullname | Sort-Object -Descending -Property LastWriteTime
Write-Output $DatabaseFolder
The $JobFolder returns the path to search the sub directories:
C:\Users\USERNAME\Box\CTRL Active Projects\project1\
Some other background info: PowerShell 5.1 is being used as Administrator, Windows 10 OS
Could someone help me understand why the code is still including some of the excluded parameters?
Both -Filter and -Include / -Exclude only ever operate on the last path component - i.e. an input file-system item's name (.Name property).
When -Exclude is combined with -Recurse, only the matching items themselves are excluded, not also their subtrees. That is, the contents (child items) of an excluded directory are still enumerated (unless their names too happen to match the exclusion pattern).
Therefore, ...\01-Eng\03-Submittals WIP\Backups, for instance, was still included in your output: the *eng* -Exclude filter excluded ...\01-Eng itself, but kept enumerating its subdirectories.
As of PowerShell (Core) 7.2.1, there is no direct way to exclude directory subtrees by wildcard patterns from a Get-ChildItem -Recurse enumeration; GitHub issue #15159 proposes adding this functionality.
Therefore, you'll have to perform your exclusions by post-filtering with a Where-Object call that matches against the full path (.FullName property):
Get-ChildItem -Recurse -Directory -Path $JobFolder -Filter *Backup* |
Where-Object FullName -notmatch 'eng|EDS'
Note: For brevity, -notmatch with a regex that uses alternation (|) is used to match multiple substrings.
To use wildcard expressions, you'd have to use the -notlike operator multiple times, once for each pattern (which would also require you to use a script block argument, with explicit use of $_, i.e. you then couldn't use the simplified syntax shown above; in other words:
Where-Object { $_.FullName -notlike '*eng*' -and $_.FullName -notlike '*EDS*' }).
I don't see anything in the documentation about using Filter and Exclude:
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/get-childitem?view=powershell-7.2
However, since it's not working as you intended, you can always manually perform the exclude:
gci -recurse -directory | ? { $_.Name -notlike "*eng*" -and $_.Name -notlike "*EDS*" }
I am using the below to recursively list all files in a folder that contains the $pattern
Get-ChildItem $targetDir -recurse | Select-String -pattern "$pattern" | group path | select name
But it seems it both list files having the $pattern in its name and in its content, e.g. when I run the above where $pattern="SAMPLE" I get:
C:\tmp\config.include
C:\tmp\README.md
C:\tmp\specs\SAMPLE.data.nuspec
C:\tmp\specs\SAMPLE.Connection.nuspec
Now:
C:\tmp\config.include
C:\tmp\README.md
indeed contains the SAMPLE keywords/text but I don't care about that, I only need the command to list file names not file with content matching the pattern. What am I missing?
Based on the below answers I have also tried:
$targetDir="C:\tmp\"
Get-ChildItem $targetDir -recurse | where {$_.name -like "SAMPLE"} | group path | select name
and:
$targetDir="C:\tmp\"
Get-ChildItem $targetDir -recurse | where {$_.name -like "SAMPLE"} | select name
but it does not return any results.
Select-String is doing what you told it to. Emphasis mine.
The Select-String cmdlet searches for text and text patterns in input strings and files.
So if you are just looking to match with file names just use -Filter of Get-ChildItem or post process with Where-Object
Get-ChildItem -Path $path -Recurse -Filter "*sample*"
That should return all files and folders that have sample in their name. If you just wanted files or directories you would be able to use the switches -File or -Directory to return those specific object types.
If your pattern is more complicated than a simple word then you might need to use Where-Object like in Itchydon's answer with something like -match giving you access to regex.
The grouping logic in your code should be redundant since you are returning single files that all have unique paths. Therefore I have not included that here. If you just want the paths then you can pipe into Select-Object -Expand FullName or just (Get-ChildItem -Path $path -Recurse -Filter "*sample*").Fullname
get-ChildItem $targetDir -recurse | where {$_.name -like $pattern} | select name
To complement Matt's helpful answer:
Specifically, because what you're piping to Select-String are [System.IO.FileInfo] objects - which is what Get-ChildItem outputs - rather than strings, it is the contents of the files represented by these objects is being searched.
Assuming that you need to match only the file name part of each file's path and that your pattern can be expressed as a wildcard expression, you do not need Select-String at all and can instead use Get-ChildItem with -Filter, as in Matt's answer, or the slower, but slightly more powerful -Include.
Caveat:
Select-String -Pattern accepts a regular expression (e.g., .*sample.*; see Get-Help about_Regular_Expressions),
whereas Get-ChildItem -Filter/-Include accepts a wildcard expression (e.g., *sample*; see Get-Help about_Wildcards) - they are different things.
On a side note: If your intent is to match files only, you can tell Get-ChildItem to restrict output to files (as opposed to potentially also directories) using -File (analogously, you can limit output to directories with -Directory).
Group-Object path (group path) will not work as intended, because the .Path property of the match-information objects output by Select-String contains the full filename, so you'd be putting each file in its own group - essentially, a no-op.
When using just Get-ChildItem, the equivalent property name would be .FullName, but what you're looking for is to group by parent path (the containing directory's path), .DirectoryName), I presume, therefore:
... | Group-Object DirectoryName | Select-Object Name
This outputs the full path of each directory that contains at least 1 file with a matching file name.
(Note that the Name in Select-Object Name refers to the .Name property of the group objects returned by Group-Object, which in this case is the value of the .DirectoryName property on the input objects.)
To complement the excellent answer by #mklement0, you can ask Powershell to print the full path by appending a pipe as follows:
Get-ChildItem -Recurse -ErrorAction SilentlyContinue -Force -Filter "*sample*" | %{$_.FullName}
Note: When searching folders where you might get an error based on security, hence we use the SilentlyContinue option.
I went through the answer by #Itchydon
but couldn't follow the use of '-like' $pattern.
I was trying to list files having 32characters(letters and numbers) in the filename.
PS C:> Get-ChildItem C:\Users\ -Recurse | where {$_.name -match "[a-zA-Z0-9]{32}"} | select name
or
PS C:> Get-ChildItem C:\Users\010M\Documents\WindowsPowerShell -Recurse | Where-Object {$_.name -match "[A-Z0-9]{32}"} | select name
So, in this case it doesn't matter whether you use where or where-object.
You can use select-string directly to search for files matching a certain string, yes, this will return the filename:count:content ... etc, but, internally these have names that you can chose or omit, the one you need is the "filename" to do this pipe this into "select-object" choosing the "FileName" from the output.
So, to select all *.MSG files that has the pattern of "Subject: Webservices restarted", you can do the following:
Select-String -Path .*.MSG -Pattern 'Subject: WebServices Restarted'
-List | select-object Filename
Also, to remove these files on the fly, you could pip into a ForEach statement with the RM command as follows:
Select-String -Path .*.MSG -Pattern 'Subject: WebServices Restarted'
-List | select-object Filename | foreach { rm $_.FileName }
I am trying to compare two sets of folders to determine discrepancies in file and folder counts. I have found a command that will output the data I am looking for, but cannot find a way to print it to a file. Here is the command I am using currently:
dir -recurse | ?{ $_.PSIsContainer } | %{ Write-Host $_.FullName (dir $_.FullName | Measure-Object).Count }
This is getting me the desired data but I need to find a way to print this to a text file. Any help would be greatly appreciated.
The problem is the use of the Write-Host cmdlet, which bypasses almost all pipeline handling. In this case, it is also unnecessary, as any output that isn't used by a cmdlet is automatically passed into the pipeline (or to the console if there's nothing further).
Here is your code rewritten to output a string to the pipeline instead of using Write-Host. This uses PowerShell's string subexpression operator $(). At the console, it will look the same, but it can be piped to a file or other cmdlet.
gci -Recurse -Directory | %{ "$($_.FullName) $((gci $_.FullName).Count)" }
You may also find it useful to put the data into a PSCustomObject. Once you have the object, you can do further processing such as sorting or filtering based on the count.
$folders = gci -Recurse -Directory | %{ [PSCustomObject]#{Name=$_.FullName; Count=(dir $_.FullName).Count }}
$folders | sort Count
$folders | where Count -ne 0
Some notes on idioms: dir is an alias for Get-Childitem, as is gci. Using gci's -Directory parameter is the best way to list only directories, rather than the PSIsContainer check. Finally, Measure-Object is unnecessary. You can take the Count of the file listing directly.
See also Write-Host Considered Harmful from the inventor of PowerShell
I am trying to select files that have identical names except for the extenstion...
IE:
idiotCode.dll, idiotCode.pdb, idiotCode.xml, StupidFool.dll, StupidFool.pdb, StupidFool.xml
et cetera.
take a gander at my where-Object call in the below script line...
gci -path $FromPath | ? {$_.Name -match "idiotCode|StupidFool|YourAnIdiot|TheSuckIstHeMatterWhichU" -and $_.Name -like "*.dll"} | foreach{write-host("Do I have the files here? : "+ $_.Fullname + " -destination" + $ToPath) }
Can I use the like parameter to do that? Is there another way to do that? Maybe something in my get-childItem method call which I could pipe into my Where-Object call?
There is a lot that you can do with just the Get-ChildItem Cmdlet. If you look at the help Get-ChildItem, you can do a lot of the filtering there. Specifically using the filters -Filter, -Include and -Exclude
For ex:
Get-ChildItem -Path $FromPath -Include "idiotCode.*","StupidFool.*","YourAnIdiot.*","TheSuckIstHeMatterWhichU.*" -Filter "*.dll"
Ok, I'm a little confused as to the whole concept here because you say you want to select multiple files of the same name but different extensions. Then you pull a directory listing of PathA, filter out everything except files in a kind of "approved names list", and only allow .DLL files to show, and then reference PathB for reasons unknown.
I'm guessing here, but I think you want to query .DLL files from PathA, and then check for matching files from PathB.
$Reference = (GCI $FromPath -Filter "*.DLL").basename
GCI $ToPath|?{$_.BaseName -Match $Reference}|FT -Group BaseName