Editor's note:
The gist of this question is:
* How do I add custom properties to objects output by Get-ChildItem that contain derived path information, namely the parent folder path (as Folder and name (as Foldername)?
* How do I export the resulting objects to a CSV file?
Currently have a script running that I took from StackOverflow and modified for my own use. The purpose of the script is to look at a directory, grab file names, then export them to a directory as a .csv file.
I was able to modify the script to pull through just the file name as originally it only pulled through the path, user etc. Was able to do this by adding ,Name. For some reason I can't get the script to pull through the parent folder it's in, also wanting the parents parent folder.
I believe this part of the code is what I am having most trouble with. Upon the select I was able to add ,Name but adding ,folder or ,foldername after the select doesn't pull through properly.
ForEach-Object {$_ | Add-Member -Name "Owner" -MemberType NoteProperty -Value (Get-Acl $_.FullName).Owner -PassThru} |
Sort-Object fullname |
Select FullName,CreationTime,LastWriteTime,Owner,Name,Folder,Foldername
The [System.IO.FileInfo] instances returned by Get-ChildItem for files do not have Folder or FolderName properties.
Get-ChildItem -File $HOME\Desktop | Get-Member, for instance, will show you the available properties, and will show you that the desired information can be derived from the PSPath and PSParentPath properties.
Select-Object allows hashtable-based property definitions, so-called calculated properties, which allow you to rename and/or transform properties and/or add custom properties derived from other property values by way of a script block.
Note: You can also use calculated properties with the Format-Table and Format-List cmdlets for creating output for display only (as well as with other cmdlets - see the linked help topic).
A simplified example of what you're looking for (includes output to a CSV file):
Get-ChildItem $HOME\Desktop | Select-Object Name,
#{ Name = 'Folder'; Expression = { Convert-Path $_.PSParentPath } },
#{ Name = 'Foldername'; Expression = { ($_.PSPath -split '\\')[-2] } } |
Export-Csv Out.csv -Encoding Utf8 -NoTypeInformation
Note that, alternatively, you could add Folder and FolderName properties to the input objects via Add-Member, as you did with the Owner property in your question.
Explanation:
Note that you can get more detailed information about any of the commands mentioned by running Get-Help <command-name> -Full; add -online to view the help topic in a browser; to learn more about the -split operator, run Get-Help about_split; to learn about PowerShell's help system in general, run Get-Help Get-Help -online.
Each #{ ... } construct passed to Select-Object is a hash table that defines a calculated property to attach to each output object:
The hash table must have two entries:
Name or Label, which defines the property's name; for brevity, you may use a (case-insensitive) prefix of the key name, such as just n or l.
Expression, which defines the property's value; again, a (case-insensitive) prefix of the key name works too, such as just e.
The expression can be a mere property name (a string), in case you simply want to rename an input property, but is more typically a script block ({ ... }), which is a piece of code that gets executed for each input object, and whose output becomes the value of the property being defined; inside that script block, automatic variable $_ (or $PSItem) refers to the input object at hand.
Definition of the Folder property: Convert-Path $_.PSParentPath converts the fully qualified PowerShell path that the PSParentPath property contains - which includes a prefix identifying the drive provider - to a regular filesystem path; e.g., Microsoft.PowerShell.Core\FileSystem::C:\Users\jdoe\Desktop -> C:\Users\jdoe\Desktop.
Definition of the Foldername property: ($_.PSPath -split '\\')[-2] splits the full path into components by path separator \, and then accesses the next-to-last component (-2), which is the parent folder name; e.g., C:\Users\jdoe\Desktop\file.txt -> Desktop
'\\' must be used to represent \, because -split's 1st RHS operand is a regular expression, where \ has special meaning and must therefore be doubled to be taken as a literal.
If you wanted to support / as the path separator as well for cross-platform support, you'd use ($_.PSPath -split '[\\/]')[-2].
Export-Csv exports the objects output by Select-Object to CSV Out.csv, where the input objects' property names become the header row, and the property values the data rows.
-Encoding Utf8, required in Windows PowerShell only (PowerShell (Core) 7+ now fortunately consistently uses BOM-less UTF-8), ensures that non-ASCII characters are properly encoded; by default, Export-Csv uses ASCII encoding and simply replaces non-ASCII characters, such as foreign letters, with verbatim ? characters, resulting in loss of information; note that -Encoding Utf8 in Windows PowerShell invariably creates UTF-8 files with a BOM.
-NoTypeInformation, again required in Windows PowerShell only, suppresses a line that Export-Csv by defaults adds as the first line of the output file, which contains the full type name (class name) of the input objects (e.g., #TYPE System.Management.Automation.PSCustomObject; this is meant to facilitate later reconversion into objects).
Related
I have this working, but need LastWriteTime and can't get it.
Get-ChildItem -Recurse | Select-String -Pattern "CYCLE" | Select-Object Path, Line, LastWriteTime
I get an empty column and zero Date-Time data
Select-String's output objects, which are of type Microsoft.PowerShell.Commands.MatchInfo, only contain the input file path (string), no other metadata such as LastWriteTime.
To obtain it, use a calculated property, combined with the common -PipelineVariable parameter,
which allows you to reference the input file at hand in the calculated property's expression script block as a System.IO.FileInfo instance as output by Get-ChildItem, whose .LastWriteTime property value you can return:
Get-ChildItem -File -Recurse -PipelineVariable file |
Select-String -Pattern "CYCLE" |
Select-Object Path,
Line,
#{
Name='LastWriteTime';
Expression={ $file.LastWriteTime }
}
Note how the pipeline variable, $file, must be passed without the leading $ (i.e. as file) as the -PipelineVariable argument . -PipelineVariable can be abbreviated to -pv.
LastWriteTime is a property of System.IO.FileSystemInfo, which is the base type of the items Get-ChildItem returns for the Filesystem provider (which is System.IO.FileInfo for files). Path and Line are properties of Microsoft.PowerShell.Commands.MatchInfo, which contains information about the match, not the file you passed in. Select-Object operates on the information piped into it, which comes from the previous expression in the pipeline, your Select-String in this case.
You can't do this as a (well-written) one-liner if you want the file name, line match, and the last write time of the actual file to be returned. I recommend using an intermediary PSCustomObject for this and we can loop over the found files and matches individually:
# Use -File to only get file objects
$foundMatchesInFiles = Get-ChildItem -Recurse -File | ForEach-Object {
# Assign $PSItem/$_ to $file since we will need it in the second loop
$file = $_
# Run Select-String on each found file
$file | Select-String -Pattern CYCLE | ForEach-Object {
[PSCustomObject]#{
Path = $_.Path
Line = $_.Line
FileLastWriteTime = $file.LastWriteTime
}
}
}
Note: I used a slightly altered name of FileLastWriteTime to exemplify that this comes from the returned file and not the match provided by Select-String, but you could use LastWriteTime if you wish to retain the original property name.
Now $foundMatchesInFiles will be a collection of files which have CYCLE occurring within them, the path of the file itself (as returned by Select-String), and the last write time of the file itself as was returned by the initial Get-ChildItem.
Additional considerations
You could also use Select-Object and computed properties but IMO the above is a more concise approach when merging properties from unrelated objects together. While not a poor approach, Select-Object outputs data with a type containing the original object type name (e.g. Selected.Microsoft.PowerShell.Commands.MatchInfo). The code may work fine but can cause some confusion when others who may consume this object in the future inspect the output members. LastWriteTime, for example, belongs to FileSystemInfo, not MatchInfo. Another developer may not understand where the property came from at first if it has the MatchInfo type referenced. It is generally a better design to create a new object with the merged properties.
That said this is a minor issue which largely comes down to stylistic preference and whether this object might be consumed by others aside from you. I write modules and scripts that many other teams in my organization consume so this is a concern for me. It may not be for you. #mklement0's answer is an excellent example of how to use computed properties with Select-Object to achieve the same functional result as this answer.
There are a lot of questions and answers about comparing the hash of two folder for integrity like this one. Assuming I have a folder that I copied to a backup medium (External drive, flash, optical disc) and would like to delete the original to save space.
What is the best way to save the original's folder hashes (before deletion) in a text file perhaps and check the backup's integrity much later against that file.
Note that if you delete the originals first and later find that the backup lacks integrity, so to speak, all you'll know is that something went wrong; the non-corrupted data will be gone.
You can create a CSV file with 2 columns, RelativePath (the full file path relative to the input directory) and Hash, and save it to a CSV file with Export-Csv:
$inputDir = 'C:\path\to\dir' # Note: specify a *full* path.
$prefixLen = $inputDir.Length + 1
Get-ChildItem -File -Recurse -LiteralPath $inputDir |
Get-FileHash |
Select-Object #{
Name='RelativePath'
Expression={ $_.Path.Substring($prefixLen) }
},
Hash |
Export-Csv originalHashes.csv -NoTypeInformation -Encoding utf8
Note: In PowerShell (Core) 7+, neither -NoTypeInformation nor -Encoding utf8 are needed, though note that the file will have no UTF-8 BOM; use -Encoding utf8bom if you want one; conversely, in Windows PowerShell you invariably get a BOM.
Note:
The Microsoft.PowerShell.Commands.FileHashInfo instances output by Get-FileHash also have an .Algorithm property naming the hashing algorithm that was used ('SHA256' by default, or as specified via the -Algorithm parameter).
If you want this property included (whose value will be the same for all CSV rows), you simply add Algorithm to the array of properties passed to Select-Object above.
Note how a hashtable (#{ ... }) passed as the second property argument to Select-Object serves as a calculated property that derives the relative path from each .Path property value (which contains the full path).
You can later apply the same command to the backup directory tree, saving to, say, backupHashes.csv, and compare the two CSV files with Compare-Object:
Compare-Object (Import-Csv -LiteralPath originalHashes.csv) `
(Import-Csv -LiteralPath backupHashes.csv) `
-Property RelativePath, Hash
Note: There's no strict need to involve files in the operation - one or both output collections can be captured in memory and can be used directly in the comparison - just omit the Export-Csv call in the command above and save to a variable ($originalHashes = Get-ChildItem ...)
Editor's note:
The gist of this question is:
* How do I add custom properties to objects output by Get-ChildItem that contain derived path information, namely the parent folder path (as Folder and name (as Foldername)?
* How do I export the resulting objects to a CSV file?
Currently have a script running that I took from StackOverflow and modified for my own use. The purpose of the script is to look at a directory, grab file names, then export them to a directory as a .csv file.
I was able to modify the script to pull through just the file name as originally it only pulled through the path, user etc. Was able to do this by adding ,Name. For some reason I can't get the script to pull through the parent folder it's in, also wanting the parents parent folder.
I believe this part of the code is what I am having most trouble with. Upon the select I was able to add ,Name but adding ,folder or ,foldername after the select doesn't pull through properly.
ForEach-Object {$_ | Add-Member -Name "Owner" -MemberType NoteProperty -Value (Get-Acl $_.FullName).Owner -PassThru} |
Sort-Object fullname |
Select FullName,CreationTime,LastWriteTime,Owner,Name,Folder,Foldername
The [System.IO.FileInfo] instances returned by Get-ChildItem for files do not have Folder or FolderName properties.
Get-ChildItem -File $HOME\Desktop | Get-Member, for instance, will show you the available properties, and will show you that the desired information can be derived from the PSPath and PSParentPath properties.
Select-Object allows hashtable-based property definitions, so-called calculated properties, which allow you to rename and/or transform properties and/or add custom properties derived from other property values by way of a script block.
Note: You can also use calculated properties with the Format-Table and Format-List cmdlets for creating output for display only (as well as with other cmdlets - see the linked help topic).
A simplified example of what you're looking for (includes output to a CSV file):
Get-ChildItem $HOME\Desktop | Select-Object Name,
#{ Name = 'Folder'; Expression = { Convert-Path $_.PSParentPath } },
#{ Name = 'Foldername'; Expression = { ($_.PSPath -split '\\')[-2] } } |
Export-Csv Out.csv -Encoding Utf8 -NoTypeInformation
Note that, alternatively, you could add Folder and FolderName properties to the input objects via Add-Member, as you did with the Owner property in your question.
Explanation:
Note that you can get more detailed information about any of the commands mentioned by running Get-Help <command-name> -Full; add -online to view the help topic in a browser; to learn more about the -split operator, run Get-Help about_split; to learn about PowerShell's help system in general, run Get-Help Get-Help -online.
Each #{ ... } construct passed to Select-Object is a hash table that defines a calculated property to attach to each output object:
The hash table must have two entries:
Name or Label, which defines the property's name; for brevity, you may use a (case-insensitive) prefix of the key name, such as just n or l.
Expression, which defines the property's value; again, a (case-insensitive) prefix of the key name works too, such as just e.
The expression can be a mere property name (a string), in case you simply want to rename an input property, but is more typically a script block ({ ... }), which is a piece of code that gets executed for each input object, and whose output becomes the value of the property being defined; inside that script block, automatic variable $_ (or $PSItem) refers to the input object at hand.
Definition of the Folder property: Convert-Path $_.PSParentPath converts the fully qualified PowerShell path that the PSParentPath property contains - which includes a prefix identifying the drive provider - to a regular filesystem path; e.g., Microsoft.PowerShell.Core\FileSystem::C:\Users\jdoe\Desktop -> C:\Users\jdoe\Desktop.
Definition of the Foldername property: ($_.PSPath -split '\\')[-2] splits the full path into components by path separator \, and then accesses the next-to-last component (-2), which is the parent folder name; e.g., C:\Users\jdoe\Desktop\file.txt -> Desktop
'\\' must be used to represent \, because -split's 1st RHS operand is a regular expression, where \ has special meaning and must therefore be doubled to be taken as a literal.
If you wanted to support / as the path separator as well for cross-platform support, you'd use ($_.PSPath -split '[\\/]')[-2].
Export-Csv exports the objects output by Select-Object to CSV Out.csv, where the input objects' property names become the header row, and the property values the data rows.
-Encoding Utf8, required in Windows PowerShell only (PowerShell (Core) 7+ now fortunately consistently uses BOM-less UTF-8), ensures that non-ASCII characters are properly encoded; by default, Export-Csv uses ASCII encoding and simply replaces non-ASCII characters, such as foreign letters, with verbatim ? characters, resulting in loss of information; note that -Encoding Utf8 in Windows PowerShell invariably creates UTF-8 files with a BOM.
-NoTypeInformation, again required in Windows PowerShell only, suppresses a line that Export-Csv by defaults adds as the first line of the output file, which contains the full type name (class name) of the input objects (e.g., #TYPE System.Management.Automation.PSCustomObject; this is meant to facilitate later reconversion into objects).
I'm parsing filenames in Powershell, and when I use Get-ChildItem | select name, I get a clean output of the files:
file1.txt
file2.txt
file3.txt
But when I try to narrow down those files with Select-String, I'm getting a weird # and { in front of my output:
Get-ChildItem | select name | Select-String -Pattern "1"
#{file1.txt}
Is there a parameter I'm missing? If I pipe with findstr rather than Select-String it works like a charm:
Get-ChildItem | select name | Findstr "1"
file1.txt
You can simplify and speed up your command as follows:
#((Get-ChildItem).Name) -match '1'
Note: #(), the array-subexpression operator, is needed to ensure that -match operates on an array, even if only one file happens to exist in the current dir.
(...).Name uses member-access enumeration to extract all Name property values from the file-info objects returned by Get-ChildItem.
-match, the regular-expression matching operator, due to operating on an array of values, returns the sub-array of matching values.
To make your original command work:
Get-ChildItem | select -ExpandProperty Name |
Select-String -Pattern "1" | select -ExpandProperty Line
select -ExpandProperty Name makes select (Select-Object) return only the Name property values; by default (implied -Property parameter), a custom object that has a Name property is returned.
select -ExpandProperty line similarly extracts the Line property value from the Microsoft.PowerShell.Commands.MatchInfo instances that Select-String outputs.
Note that in PowerShell [Core] v7+ you could omit this step by instead using Select-String's (new) -Raw switch to request string-only output.
As for what you tried:
As stated, by not using -ExpandProperty, select name (implied -Property parameter) created a custom object ([pscustomobject] instance) with a Name property.
Select-String stringifies its input objects, if necessary, so it can perform a string search on them, which results in the representation you saw; here's a simulation:
# Stringify a custom object via an expandable string ("...")
PS> "$([pscustomobject] #{ Name = 'file1.txt' })"
#{Name=file1.txt}
As an aside:
The above stringification method is essentially like calling .ToString() on the input objects[1], which often results in useless string representations (by default, just the type name); a more useful and intuitive stringification would be to use PowerShell's rich output-formatting system, i.e. to use the string representation you would see in the console; changing Select-String's behavior to do that is the subject of this feature request on GitHub.
[1] Calling .ToString() directly on a [pscustomobject] instance is actually still broken as of PowerShell Core 7.0.0-rc.2, due to this bug; the workaround is to call .psobject.ToString() or to use an expandable string, as shown above.
Editor's note:
The gist of this question is:
* How do I add custom properties to objects output by Get-ChildItem that contain derived path information, namely the parent folder path (as Folder and name (as Foldername)?
* How do I export the resulting objects to a CSV file?
Currently have a script running that I took from StackOverflow and modified for my own use. The purpose of the script is to look at a directory, grab file names, then export them to a directory as a .csv file.
I was able to modify the script to pull through just the file name as originally it only pulled through the path, user etc. Was able to do this by adding ,Name. For some reason I can't get the script to pull through the parent folder it's in, also wanting the parents parent folder.
I believe this part of the code is what I am having most trouble with. Upon the select I was able to add ,Name but adding ,folder or ,foldername after the select doesn't pull through properly.
ForEach-Object {$_ | Add-Member -Name "Owner" -MemberType NoteProperty -Value (Get-Acl $_.FullName).Owner -PassThru} |
Sort-Object fullname |
Select FullName,CreationTime,LastWriteTime,Owner,Name,Folder,Foldername
The [System.IO.FileInfo] instances returned by Get-ChildItem for files do not have Folder or FolderName properties.
Get-ChildItem -File $HOME\Desktop | Get-Member, for instance, will show you the available properties, and will show you that the desired information can be derived from the PSPath and PSParentPath properties.
Select-Object allows hashtable-based property definitions, so-called calculated properties, which allow you to rename and/or transform properties and/or add custom properties derived from other property values by way of a script block.
Note: You can also use calculated properties with the Format-Table and Format-List cmdlets for creating output for display only (as well as with other cmdlets - see the linked help topic).
A simplified example of what you're looking for (includes output to a CSV file):
Get-ChildItem $HOME\Desktop | Select-Object Name,
#{ Name = 'Folder'; Expression = { Convert-Path $_.PSParentPath } },
#{ Name = 'Foldername'; Expression = { ($_.PSPath -split '\\')[-2] } } |
Export-Csv Out.csv -Encoding Utf8 -NoTypeInformation
Note that, alternatively, you could add Folder and FolderName properties to the input objects via Add-Member, as you did with the Owner property in your question.
Explanation:
Note that you can get more detailed information about any of the commands mentioned by running Get-Help <command-name> -Full; add -online to view the help topic in a browser; to learn more about the -split operator, run Get-Help about_split; to learn about PowerShell's help system in general, run Get-Help Get-Help -online.
Each #{ ... } construct passed to Select-Object is a hash table that defines a calculated property to attach to each output object:
The hash table must have two entries:
Name or Label, which defines the property's name; for brevity, you may use a (case-insensitive) prefix of the key name, such as just n or l.
Expression, which defines the property's value; again, a (case-insensitive) prefix of the key name works too, such as just e.
The expression can be a mere property name (a string), in case you simply want to rename an input property, but is more typically a script block ({ ... }), which is a piece of code that gets executed for each input object, and whose output becomes the value of the property being defined; inside that script block, automatic variable $_ (or $PSItem) refers to the input object at hand.
Definition of the Folder property: Convert-Path $_.PSParentPath converts the fully qualified PowerShell path that the PSParentPath property contains - which includes a prefix identifying the drive provider - to a regular filesystem path; e.g., Microsoft.PowerShell.Core\FileSystem::C:\Users\jdoe\Desktop -> C:\Users\jdoe\Desktop.
Definition of the Foldername property: ($_.PSPath -split '\\')[-2] splits the full path into components by path separator \, and then accesses the next-to-last component (-2), which is the parent folder name; e.g., C:\Users\jdoe\Desktop\file.txt -> Desktop
'\\' must be used to represent \, because -split's 1st RHS operand is a regular expression, where \ has special meaning and must therefore be doubled to be taken as a literal.
If you wanted to support / as the path separator as well for cross-platform support, you'd use ($_.PSPath -split '[\\/]')[-2].
Export-Csv exports the objects output by Select-Object to CSV Out.csv, where the input objects' property names become the header row, and the property values the data rows.
-Encoding Utf8, required in Windows PowerShell only (PowerShell (Core) 7+ now fortunately consistently uses BOM-less UTF-8), ensures that non-ASCII characters are properly encoded; by default, Export-Csv uses ASCII encoding and simply replaces non-ASCII characters, such as foreign letters, with verbatim ? characters, resulting in loss of information; note that -Encoding Utf8 in Windows PowerShell invariably creates UTF-8 files with a BOM.
-NoTypeInformation, again required in Windows PowerShell only, suppresses a line that Export-Csv by defaults adds as the first line of the output file, which contains the full type name (class name) of the input objects (e.g., #TYPE System.Management.Automation.PSCustomObject; this is meant to facilitate later reconversion into objects).