I have the following code to add new columns to a csv file. I would like to amend the code to run on multiple csv files within a folder and output it to a different folder.
$source = "C:\input_folder\input.csv"
$destination = "C:\output_folder\output.csv"
(Import-CSV $source |
Select-Object *,#{Name='column1';Expression={'data1'}} |
Select-Object *,#{Name='column2';Expression={'data2'}} |
ConvertTo-csv -NoTypeInformation |
Select-Object -Skip 0) -replace '"' | Set-Content $destination
You could do something like this. You might also want to add a check that the files in the folder are .csv type and not something else.
$source = "C:\input_folder"
$destinationFolder = "C:\output_folder"
$folderContents = Get-ChildItem $source
foreach ($item in $folderContents) {
if (Test-Path -Path $item -PathType Leaf == True) {
(Import-CSV $item |
Select-Object *,#{Name='column1';Expression={'data1'}} |
Select-Object *,#{Name='column2';Expression={'data2'}} |
ConvertTo-csv -NoTypeInformation |
Select-Object -Skip 0) -replace '"' | Set-Content ("$destinationFolder\$item" + "_formatted.csv")
}
}
Related
I'm trying (badly) to work through combining CSV files into one file and prepending a column that contains the file name. I'm new to PowerShell, so hopefully someone can help here.
I tried initially to do the well documented approach of using Import-Csv / Export-Csv, but I don't see any options to add columns.
Get-ChildItem -Filter *.csv | Select-Object -ExpandProperty FullName | Import-Csv | Export-Csv CombinedFile.txt -UseQuotes Never -NoTypeInformation -Append
Next I'm trying to loop through the files and append the name, which kind of works, but for some reason this stops after the first row is generated. Since it's not a CSV process, I have to use the switch to skip the first title row of each file.
$getFirstLine = $true
Get-ChildItem -Filter *.csv | Where-Object {$_.Name -NotMatch "Combined.csv"} | foreach {
$filePath = $_
$collection = Get-Content $filePath
foreach($lines in $collection) {
$lines = ($_.Basename + ";" + $lines)
}
$linesToWrite = switch($getFirstLine) {
$true {$lines}
$false {$lines | Select -Skip 1}
}
$getFirstLine = $false
Add-Content "Combined.csv" $linesToWrite
}
This is where the -PipelineVariable parameter comes in real handy. You can set a variable to represent the current iteration in the pipeline, so you can do things like this:
Get-ChildItem -Filter *.csv -PipelineVariable File | Where-Object {$_.Name -NotMatch "Combined.csv"} | ForEach-Object { Import-Csv $File.FullName } | Select *,#{l='OriginalFile';e={$File.Name}} | Export-Csv Combined.csv -Notypeinfo
Merging your CSVs into one and adding a column for the file's name can be done as follows, using a calculated property on Select-Object:
Get-ChildItem -Filter *.csv | ForEach-Object {
$fileName = $_.Name
Import-Csv $_.FullName | Select-Object #{
Name = 'FileName'
Expression = { $fileName }
}, *
} | Export-Csv path/to/merged.csv -NoTypeInformation
I have the following code that prints the file system rights of each account enabled on the folder with path "C:\Temp\CSM\*" & "C:\Temp\CSM\*\*" . How do I write the output in a comma-separated CSV? As this is for PowerShell 2.0 Export-Csv -Append parameter cannot be used.
$FolderPath = dir -Directory -Path "C:\Temp\CSM\*", "C:\Temp\CSM\*\*" -Force
foreach ($Folder in $FolderPath) {
$Acl = Get-Acl -Path $Folder.FullName
foreach ($Access in $acl.Access) {
Write-Host $Folder.FullName "," $Access.IdentityReference "," $Access.FileSystemRights
}
}
Prior to PowerShell v3 if you wanted to append to an existing CSV you need something like this:
... | ConvertTo-Csv -NoType | Select-Object -Skip 1 | Add-Content
However, in your scenario that probably isn't necessary. If you replace your foreach loops with a pipeline you can write the CSV in one go without having to append to it in a loop (which isn't recommended anyway).
$folders = "C:\Temp\CSM\*", "C:\Temp\CSM\*\*"
Get-ChildItem -Path $folders -Directory -Force | ForEach-Object {
$path = $_.FullName
Get-Acl -Path $path |
Select-Object -Expand Access |
Select-Object #{n='Path';e={$path}}, IdentityReference, FileSystemRights
} | Export-Csv 'C:\output.csv' -NoType
Using PowerShell, I have created a script that adds headers to a CSV, select unique value within specified columns, and exports it as a new CSV.
That script is:
import-csv -Header (1..39) -Path 'J:\consult evv\splitfile_385.csv' | select '30', '31', '39' -Unique | Export-Csv -Path "C:\Users\CFOSTER\Desktop\stuff\modsplitfile_385.csv" -NoTypeInformation
This script only works for one item: splitfile_385.csv. I'd like to use this for all files in a directory.
I can loop through items in the specified path but am having trouble combining the loop with the script above.
Currently, I have:
$files = Get-ChildItem "J:\consult evv"
for ($i=0; $i -lt $files.Count; $i++) {
$outfile = "C:\Users\CFOSTER\Desktop\Stuff\new" + $files[$i]
Get-Content $files[$i].FullName | Where-Object { !($_ -match 'step4' -or $_ -match 'step9') } | import-csv -Header (1..39) -Path $files[$i].FullName | select '30', '31', '39' -Unique | Set-Content $outfile
}
Using this gives me the error: import-csv : You must specify either the -Path or -LiteralPath parameters, but not both.
How can I combine my script with a loop for all files in a directory?
You can use ForEach-Object and import one at a time.
Get-ChildItem "C:\CSV Files\*.csv" | ForEach-Object {
Import-Csv $_ | Where-Object { ... }
}
If you want to output to a new CSV file, make sure the output filename doesn't match the pattern used with Get-ChildItem or you'll run into trouble.
try somethng like this:
Get-ChildItem "J:\consult evv" -file -Filter "*.csv" | %{
$outfile = "C:\Users\CFOSTER\Desktop\Stuff\new" + $_.Name
Get-Content $_.FullName | select -skip 1 | Where { !($_ -match 'step4' -or $_ -match 'step9') } |
ConvertFrom-Csv -Header (1..39) | select '30', '31', '39' -Unique | Out-File $outfile -Append
}
I wish to search for specific files listed in searchFiles and pipe their locations to TestFileLocation.CSV. However, my current script only generates an empty CSV. What am I missing?
My TestFindFile.csv is of the form:
Name
123.pdf
321.pdf
aaa.pdf
SNIPPET
$searchFiles = Import-CSV 'C:\Data\SCRIPTS\PS1\TestFindFile.csv' -Header ("Name")
$source = 'C:\Data'
ForEach($File in $searchFiles)
{
Get-ChildItem $source -Filter $File -rec | where {!$_.PSIsContainer} | select-object FullName | export-csv -notypeinformation -delimiter '|' -path c:\data\scripts\ps1\TestFileLocation.csv
}
You were overwriting the CSV for each iteration of the loop.
$searchFiles = Import-CSV 'C:\Data\SCRIPTS\PS1\TestFindFile.csv' -Header ("Name")
$source = 'C:\Data'
$outputPath = 'c:\data\scripts\ps1\TestFileLocation.csv'
$searchFiles | ForEach-Object {
# Silently continue to try to ignore error like
# not being able to read path's which are too long
Get-ChildItem $source -Filter $_ -rec -ErrorAction SilentlyContinue | where {!$_.PSIsContainer} | select-object FullName
} | export-csv -notypeinformation -delimiter '|' -path $outputPath
Example using AlphaFS
A comment asked for an example using AlphaFS because it claims to overcome the long path issue. I'm not going into all the details, but here is how I got it to work.
# download and unzip to c:\alpahfs
# dir C:\AlphaFS\* -Recurse -File | Unblock-File
[System.Reflection.Assembly]::LoadFrom('C:\AlphaFS\lib\net451\AlphaFS.dll')
$searchFiles = Import-CSV 'C:\Data\SCRIPTS\PS1\TestFindFile.csv' -Header ("Name")
$source = 'C:\Data'
$outputPath = 'c:\data\scripts\ps1\TestFileLocation.csv'
$searchFiles | ForEach-Object {
$files = [Alphaleonis.Win32.Filesystem.Directory]::EnumerateFiles($source,'*',[System.IO.SearchOption]::AllDirectories)
$files | ForEach-Object { [PSCustomObject] #{FileName = $_} }
} | export-csv -notypeinformation -delimiter '|' -path $outputPath
# type $outputPath
If your .csv file contains the header "Name", there is no need to again declare it when running Import-Csv.
The reason the output is empty is that you are searching for an Object which contains the property Name (imported from the TestFindFile.csv). Search for $File.Name. Also pull commands outside the loop that don't need to be there:
$searchFiles | Select -ExpandProperty Name | % {
Get-ChildItem $source -Filter $_ -Recurse | where {!$_.PSIsContainer}
} | select-object FullName | export-csv -notypeinformation -delimiter '|' -path c:\data\scripts\ps1\TestFileLocation.csv
I am trying to create a CSV file for a file share that has deep folder structure.
I want the CSV to look like:
filename, filepath, file type, folderStructure
So far I have the following:
#Get-ChildItem -Path D:\ -Recurse
$directory="d:\"
gci $directory -recurse |
where {$_.psiscontainer} |
foreach {
get-childitem $_.fullname |
sort creationtime |
select -expand fullname -last 1
}
You don't need to recurse a recursive in Powershell. It will automatically go through all of the subdirectories of subdirectories.
I am also a little unsure of some of the information you wanted, but here is a script that does what you want mostly I believe and is IMO a little better to read.
Get-ChildItem -Path X:\Test -Recurse |`
foreach{
$Item = $_
$Type = $_.Extension
$Path = $_.FullName
$Folder = $_.PSIsContainer
$Age = $_.CreationTime
$Path | Select-Object `
#{n="Name";e={$Item}},`
#{n="Created";e={$Age}},`
#{n="filePath";e={$Path}},`
#{n="Extension";e={if($Folder){"Folder"}else{$Type}}}`
}| Export-Csv X:\Test\Results.csv -NoTypeInformation
You will need to change your path, as I created this for a test. My results look like this in Excel:
+-------------------------------+----------------+-------------------------------------+-----------+
| Name | Created | filePath | Extension |
+-------------------------------+----------------+-------------------------------------+-----------+
| test2 | 3/6/2013 19:21 | X:\Test\test2 | Folder |
| Results.csv | 3/6/2013 19:51 | X:\Test\Results.csv | .csv |
| test3 | 3/6/2013 19:21 | X:\Test\test2\test3 | Folder |
| New Text Document.txt | 3/6/2013 19:21 | X:\Test\test2\New Text Document.txt | .txt |
+-------------------------------+----------------+-------------------------------------+-----------+
Where it says "Folder" for the Extension just it returning that it is a directory instead of a blank (No extension).
OK, I changed the way it checks for the parent. It is no longer looking directly at the Parent attribute, and it should correct it now.
Get-ChildItem -Path D:\ -Recurse |`
foreach{
$Item = $_
$Type = $_.Extension
$Path = $_.FullName
$ParentS = ($_.Fullname).split("\")
$Parent = $ParentS[#($ParentS.Length - 2)]
$Folder = $_.PSIsContainer
$Age = $_.CreationTime
$Path | Select-Object `
#{n="Name";e={$Item}},`
#{n="Created";e={$Age}},`
#{n="Folder Name";e={if($Parent){$Parent}else{$Parent}}},`
#{n="filePath";e={$Path}},`
#{n="Extension";e={if($Folder){"Folder"}else{$Type}}}`
}| Export-Csv X:\Test\ResultsC.csv -NoTypeInformation
It is now taking the path to the current item, turning it into an array by splitting on the \, and then giving you the value at ArryLength - 2
I am not sure if this is the best approach but it works. I have a feeling code could be shorten to get the full path of a file.
$myDirectory = "D:\"
Get-ChildItem -Path $myDirectory -Recurse |`
foreach{
$Item = $_
$Type = $_.Extension
$Path = $_.FullName
$ParentS = ($_.Fullname).split("\")
$Parent = $ParentS[#($ParentS.Length - 2)]
$ParentPath = $_.PSParentPath
$ParentPathSplit = ($_.PSParentPath).split("::")
$ParentPathFinal = $ParentPathSplit[#($ParentPathSplit.Length -1)]
#$ParentPath = [io.path]::GetDirectoryName($myDirectory)
$Folder = $_.PSIsContainer
$Age = $_.CreationTime
$Path | Select-Object `
#{n="Name";e={$Item}},`
#{n="Created";e={$Age}},`
#{n="Folder Name";e={if($Parent){$Parent}else{$Parent}}},`
#{n="filePath";e={$Path}},`
#{n="Extension";e={if($Folder){"Folder"}else{$Type}}},`
#{n="Folder Name 2";e={if($Parent){$Parent}else{$Parent}}},`
##{n="Folder Path";e={$ParentPath}},`
#{n="Folder Path 2";e={$ParentPathFinal}}`
}| Export-Csv d:\ResultsC_2_F.csv -NoTypeInformation