Export CSV. Folder, subfolder and file into separate column - powershell

I created a script that lists all the folders, subfolders and files and export them to csv:
$path = "C:\tools"
Get-ChildItem $path -Recurse |select fullname | export-csv -Path "C:\temp\output.csv" -NoTypeInformation
But I would like that each folder, subfolder and file in pfad is written into separate column in csv.
Something like this:
c:\tools\test\1.jpg
Column1
Column2
Column3
tools
test
1.jpg
I will be grateful for any help.
Thank you.

You can split the Fullname property using the Split() method. The tricky part is that you need to know the maximum path depth in advance, as the CSV format requires that all rows have the same number of columns (even if some columns are empty).
# Process directory $path recursively
$allItems = Get-ChildItem $path -Recurse | ForEach-Object {
# Split on directory separator (typically '\' for Windows and '/' for Unix-like OS)
$FullNameSplit = $_.FullName.Split( [IO.Path]::DirectorySeparatorChar )
# Create an object that contains the splitted path and the path depth.
# This is implicit output that PowerShell captures and adds to $allItems.
[PSCustomObject] #{
FullNameSplit = $FullNameSplit
PathDepth = $FullNameSplit.Count
}
}
# Determine highest column index from maximum depth of all paths.
# Minus one, because we'll skip root path component.
$maxColumnIndex = ( $allItems | Measure-Object -Maximum PathDepth ).Maximum - 1
$allRows = foreach( $item in $allItems ) {
# Create an ordered hashtable
$row = [ordered]#{}
# Add all path components to hashtable. Make sure all rows have same number of columns.
foreach( $i in 1..$maxColumnIndex ) {
$row[ "Column$i" ] = if( $i -lt $item.FullNameSplit.Count ) { $item.FullNameSplit[ $i ] } else { $null }
}
# Convert hashtable to object suitable for output to CSV.
# This is implicit output that PowerShell captures and adds to $allRows.
[PSCustomObject] $row
}
# Finally output to CSV file
$allRows | Export-Csv -Path "C:\temp\output.csv" -NoTypeInformation
Notes:
The syntax Select-Object #{ Name= ..., Expression = ... } creates a calculated property.
$allRows = foreach captures and assigns all output of the foreach loop to variable $allRows, which will be an array if the loop outputs more than one object. This works with most other control statements as well, e. g. if and switch.
Within the loop I could have created a [PSCustomObject] directly (and used Add-Member to add properties to it) instead of first creating a hashtable and then converting to [PSCustomObject]. The choosen way should be faster as no additional overhead for calling cmdlets is required.

While a file with rows containing a variable number of items is not actually a CSV file, you can roll your own and Microsoft Excel can read it.
=== Get-DirCsv.ps1
Get-Childitem -File |
ForEach-Object {
$NameParts = $_.FullName -split '\\'
$QuotedParts = [System.Collections.ArrayList]::new()
foreach ($NamePart in $NameParts) {
$QuotedParts.Add('"' + $NamePart + '"') | Out-Null
}
Write-Output $($QuotedParts -join ',')
}
Use this to capture the output to a file with:
.\Get-DirCsv.ps1 | Out-File -FilePath '.\dir.csv' -Encoding ascii

Related

Export CSV: file structure with folders as columns

My question is quite similar to one posted here: Export CSV. Folder, subfolder and file into separate column
I have a file and folder structure containing possibly up to 10 folders deep and I want to run PowerShell to create a hash table that writes each file into a row, with each of the folders as a separate column, and then the filename at a dedicated column.
I start off with
gci -path C:\test -file -recurse|export-csv C:\temp\out.csv -notypeinformation
But this produces the standard table with some of the info I need but the directory is of course presented as one long string.
I'd like to get an output where each folder and its subfolder that houses the file to be presented as a column.
C:\Test\Folder1\Folder2\Folder3\file.txt
to be presented as
Name
Parent1
Parent2
Parent3
Parent4
Parent5
Parent6
Filename
file.txt
Folder1
Folder2
Folder3
file.txt
image1.png
Folder1
image.1png
Doc1.docx
Folder1
Folder2
Folder3
Folder4
Folder5
Folder6
Doc1.docx
table3.csv
Folder1
Folder2
table3.csv
As you can see there are some files which have just one folder whereas others could stored in several folders deep.
I need to keep this consistent, as I want to use Power Automate and the File system connector to read the file paths using the Excel table and then parse and create the file into SharePoint using the parent/folder levels as metadata/column in the document library.
I took zett42's code from the linked question and modified it.
$allItems = Get-ChildItem C:\Test -File -Recurse | ForEach-Object {
# Split on directory separator (typically '\' for Windows and '/' for Unix-like OS)
$FullNameSplit = $_.FullName.Split( [IO.Path]::DirectorySeparatorChar )
# Create an object that contains the splitted path and the path depth.
# This is implicit output that PowerShell captures and adds to $allItems.
[PSCustomObject] #{
FullNameSplit = $FullNameSplit
PathDepth = $FullNameSplit.Count
Filename = $_.Name
}
}
# Determine highest column index from maximum depth of all paths.
# Minus one, because we'll skip root path component.
$maxColumnIndex = ( $allItems | Measure-Object -Maximum PathDepth ).Maximum - 1
$allRows = foreach( $item in $allItems ) {
# Create an ordered hashtable
$row = [ordered]#{}
# Add all path components to hashtable. Make sure all rows have same number of columns.
foreach( $i in 1..$maxColumnIndex ) {
$row[ "Filename" ] = $item.Filename
$row[ "Column$i" ] = if( $i -lt $item.FullNameSplit.Count ) { $item.FullNameSplit[ $i ] } else { $null }
}
# Convert hashtable to object suitable for output to CSV.
# This is implicit output that PowerShell captures and adds to $allRows.
[PSCustomObject] $row
}
I can get the filename to show as a separate column but I don't want the script to add the filename at the last column.
PowerShell allrows output screenshot
Thanks
I've answered my own question.
Modified zett42's script, and included a few variables around splitting around just the Name of from GetChild-Item as opposed to the FullName and then of course the fixed column with just the filename in the hash table.
$allItems = Get-ChildItem C:\Test -File -Recurse | ForEach-Object {
# Split on directory separator (typically '\' for Windows and '/' for Unix-like OS)
# $FullNameSplit = $_.FullName.Split( [IO.Path]::DirectorySeparatorChar )
$FullNameSplit = split-path -Path $_.FullName
$DirNameSplit = $FullNameSplit.Split( [IO.Path]::DirectorySeparatorChar )
# Create an object that contains the splitted path and the path depth.
# This is implicit output that PowerShell captures and adds to $allItems.
[PSCustomObject] #{
#FullNameSplit = $FullNameSplit
#PathDepth = $FullNameSplit.Count
DirNameSplit = $DirNameSplit
PathDepth = $DirNameSplit.Count
Filename = $_.Name
}
}
# Determine highest column index from maximum depth of all paths.
# Minus one, because we'll skip root path component.
$maxColumnIndex = ( $allItems | Measure-Object -Maximum PathDepth ).Maximum - 1
$allRows = foreach( $item in $allItems ) {
# Create an ordered hashtable
$row = [ordered]#{}
# Add all path components to hashtable. Make sure all rows have same number of columns.
foreach( $i in 1..$maxColumnIndex ) {
$row[ "Filename" ] = $item.Filename
#$row[ "Column$i" ] = if( $i -lt $item.FullNameSplit.Count ) { $item.FullNameSplit[ $i ] } else { $null }
$row[ "Parent$i" ] = if( $i -lt $item.DirNameSplit.Count ) { $item.DirNameSplit[ $i ] } else { $null }
# $row[ "Column$i" ] = $item.DirNameSplit[$i]
}
# Convert hashtable to object suitable for output to CSV.
# This is implicit output that PowerShell captures and adds to $allRows.
[PSCustomObject] $row
}

Retrieving file information of specific date using powershell

I have a base folder as:
D:\St\Retail\AMS\AMS\FTP-FromClient\AMS
It contains various folders of dates:
2022-04-01
2022-04-02
...
...
2022-02-02
2021-05-05
2019-04-12
And each of these folders contains own files inside the folder. So, I want to retrieve all the filename inside the folder if it has 2022-04. So if the folder has '2022-04' as the base name ,I need to retreive all the file inside the folder like '2022-04-01','2022-04-02','2022-04-03'. The way I tried is:
cls
$folerPath = 'D:\St\Retail\AMS\AMS\FTP-FromClient\AMS'
$files = Get-ChildItem $folerPath
[System.Collections.ArrayList]$data = #()
foreach ($f in $files) {
$a = Get-ChildItem $f.FullName
foreach ($inner in $a) {
echo $inner.FullName
$outfile = $inner.FullName -match '*2022-04*'
$datepart = $inner.FullName.split('\')[-1]
if ($outfile) {
$data.add($datepart + '\' + $inner.Name.Trim())
}
}
}
My final $data may contains like this:
2022-04-01/abc.txt
2022-04-02/cde.pdf
2022-04-03/e.xls
You can do this by first collecting the directories you want to explore and then loop over these to get the files inside.
Using a calculated property you can output in whatever format you like:
$folderPath = 'D:\St\Retail\AMS\AMS\FTP-FromClient\AMS'
$data = Get-ChildItem -Path $folderPath -Filter '2022-04*' -Directory | ForEach-Object {
$dir = $_.Name
(Get-ChildItem -Path $_.FullName -File |
Select-Object #{Name = 'FolderFile'; Expression = {'{0}\{1}' -f $dir, $_.Name}}).FolderFile
}
After this, $data would be a string array with this content:
2022-04-01\abc.txt
2022-04-02\cde.pdf
2022-04-03\e.xls
By using wildcards for both directory and file name, you only need a single Get-ChildItem call:
$folderPath = 'D:\St\Retail\AMS\AMS\FTP-FromClient\AMS'
$folderDate = '2022-04'
[array] $data = Get-ChildItem "$folderPath/$folderDate*/*" -File | ForEach-Object{
# Join-Path's implicit output will be captured as an array in $data.
Join-Path $_.Directory.Name $_.Name
}
$data will be an array of file paths like this:
2022-04-01\abc.txt
2022-04-02\cde.pdf
2022-04-03\e.xls
Notes:
[array] $data makes sure that the variable always contains an array. Otherwise PowerShell would output a single string value when only a single file is found. This could cause problems, e. g. when you want to iterate over $data by index, you would iterate over the characters of the single string instead.
To make this answer platform-independent I'm using forward slashes in the Get-ChildItem call which work as path separators under both Windows and *nix platforms.
Join-Path is used to make sure the output paths use the expected default path separator (either / or \) of the platform.

Deleting CSV the entire row if text in a column matches a specific path or a file name

I'm new to Powershell so please try to explain things a little bit too if you can. I'm trying to export the contents of a directory along with some other information in a CSV .
The CSV file contains information about the files however, I just need to match the FileName column (which contains the full path). If it's matched, I need to delete the entire row.
$folder1 = OldFiles
$folder2 = Log Files\January
$file1 = _updatehistory.txt
$file2 = websites.config
In the CSV file, if any of these is matched, the entire row must be deleted. The CSV file contains FileName in this manner:
**FileName**
C:\Installation\New Applications\Root
I've tried doing this:
Import-csv -Path "C:\CSV\Recursion.csv" | Where-Object { $_.FileName -ne $folder2} | Export-csv -Path "C:\CSV\RecursionUpdated.csv" -NoTypeInformation
But it's not working out. I would really appreciate help here.
It looks like you want to match only parts of the full path, so you should use -like or -match operators (or their negated variants) which can do non-exact matching:
$excludes = '*\OldFiles', '*\Log Files\January', '*\_updatehistory.txt', '*\websites.config'
Import-csv -Path "C:\CSV\Recursion.csv" |
Where-Object {
# $matchesExclude Will be $true if at least one exclude pattern matches
# against FileName. Otherwise it will be $null.
$matchesExclude = foreach( $exclude in $excludes ) {
# Output $true if pattern matches, which will be captured in $matchesExclude.
if( $_.FileName -like $exclude ) { $true; break }
}
# This outputs $true if the filename is not excluded, thus Where-Object
# passes the row along the pipeline.
-not $matchesExclude
} | Export-csv -Path "C:\CSV\RecursionUpdated.csv" -NoTypeInformation
This code makes heavily use of PowerShell's implicit output behaviour. E. g. the literal $true in the foreach loop body is implicit output which will be automatically captured in $matchesExclude. If it were not for the assignment $matchesExclude = foreach ..., the value would have been written to the console instead (if not captured somewhere else in the callstack).

How to seperate CSV values within a CSV into new rows in PowerShell

I'm receiving an automated report from a system that cannot be modified as a CSV. I am using PowerShell to split the CSV into multiple files and parse out the specific data needed. The CSV contains columns that may contain no data, 1 value, or multiple values that are comma separated within the CSV file itself.
Example(UPDATED FOR CLARITY):
"Group","Members"
"Event","362403"
"Risk","324542, 340668, 292196"
"Approval","AA-334454, 344366, 323570, 322827, 360225, 358850, 345935"
"ITS","345935, 358850"
"Services",""
I want the data to have one entry per line like this (UPDATED FOR CLARITY):
"Group","Members"
"Event","362403"
"Risk","324542"
"Risk","340668"
"Risk","292196"
#etc.
I've tried splitting the data and I just get an unknown number of columns at the end.
I tried a foreach loop, but can't seem to get it right (pseudocode below):
Import-CSV $Groups
ForEach ($line in $Groups){
If($_.'Members'.count -gt 1, add-content "$_.Group,$_.Members[2]",)}
I appreciate any help you can provide. I've searched all the stackexchange posts and used Google but haven't been able to find something that addresses this exact issue.
Import-Csv .\input.csv | ForEach-Object {
ForEach ($Member in ($_.Members -Split ',')) {
[PSCustomObject]#{Group = $_.Group; Member = $Member.Trim()}
}
} | Export-Csv .\output.csv -NoTypeInformation
# Get the raw text contents
$CsvContents = Get-Content "\path\to\file.csv"
# Convert it to a table object
$CsvData = ConvertFrom-CSV -InputObject $CsvContents
# Iterate through the records in the table
ForEach ($Record in $CsvData) {
# Create array from the members values at commas & trim whitespace
$Record.Members -Split "," | % {
$MemberCount = $_.Trim()
# Check if the count is greater than 1
if($MemberCount -gt 1) {
# Create our output string
$OutputString = "$($Record.Group), $MemberCount"
# Write our output string to a file
Add-Content -Path "\path\to\output.txt" -Value $OutputString
}
}
}
This should work, you had the right idea but I think you may have been encountering some syntax issues. Let me know if you have questions :)
Revised the code as per your updated question,
$List = Import-Csv "\path\to\input.csv"
foreach ($row in $List) {
$Group = $row.Group
$Members = $row.Members -split ","
# Process for each value in Members
foreach ($MemberValue in $Members) {
# PS v3 and above
$Group + "," + $MemberValue | Export-Csv "\path\to\output.csv" -NoTypeInformation -Append
# PS v2
# $Group + "," + $MemberValue | Out-File "\path\to\output.csv" -Append
}
}

Powershell - Pass list of directory paths to FOR Loop - Output results to CSV

The code below works. Rather than specify the path manually I would like to pass a list of values from a csv file E:\Data\paths.csv and then output individual csv files for each path processed displaying the $Depth for that directory......
$StartLevel = 0 # 0 = include base folder, 1 = sub-folders only, 2 = start at 2nd level
$Depth = 10 # How many levels deep to scan
$Path = "E:\Data\MyPath" # starting path
For ($i=$StartLevel; $i -le $Depth; $i++) {
$Levels = "\*" * $i
(Resolve-Path $Path$Levels).ProviderPath | Get-Item | Where PsIsContainer |
Select FullName
}
Thanks,
Phil
Get-Help Import-Csv will help you in this regards.
regards,
kvprasoon
I assume you want something like the following:
# Create sample input CSV
#"
Path,StartLevel,Depth
"E:\Data\MyPath",0,10
"# > PathSpecs.csv
# Loop over each input CSV row (object with properties
# .Path, .StartLevel, and .Depth)
foreach ($pathSpec in Import-Csv PathSpecs.csv) {
& { For ([int] $i=$pathSpec.StartLevel; $i -le $pathSpec.Depth; $i++) {
$Levels = "\*" * $i
Resolve-Path "$($pathSpec.Path)$Levels" | Get-Item | Where PsIsContainer |
Select FullName
} } | # Export paths to a CSV file named "Path-<input-path-with-punct-stripped>.csv"
Export-Csv -NoTypeInformation "Path-$($pathSpec.Path -replace '[^\p{L}0-9]+', '_').csv"
}
Note that your approach to breadth-first enumeration of subdirectories in the subtree works, but will be quite slow with large subtrees.