I have a csv file that contains lines that I'd like to extract and add as columns to a separate csv file.
This is an example of the contents of the first csv file.
I want to take everything between the third comma and semi-colon for each one and add them as columns to this csv file.
My desired output is this. Because there may be multiple servers in the first csv file I'd like to export a csv for each server but I can figure that bit out.
I've been playing around with the following but I can't figure it out.
Import-CSV $CsvTargetPath | ForEach-Object {
$newData = $_
Import-CSV $CsvSourcePath | ForEach-Object {
$_.AdditionalDisks.split(";") | ForEach-Object {
$newRecordProperties = [ordered]#{
$DriveLetter = $_.split(",")[3]
$Label = $_.split(",")[4]
$Filesystem = $_.split(",")[5]
}
$newRecord = new-object psobject -Property $newRecordProperties
Write-Output $newRecord
}
$CsvTargetPath | Add-Member -MemberType NoteProperty -Name DriveLetter -Value $DriveLetter -PassThru
$CsvTargetPath | Add-Member -MemberType NoteProperty -Name Label -Value $Label -PassThru
$CsvTargetPath | Add-Member -MemberType NoteProperty -Name Filesystem -Value $Filesystem -PassThru
}
} | Export-CSV $CsvTargetPath -NoTypeInformation
Here is the contents of the first csv in plain text.
VmName AdditionalDisks
Server01 90,0,thick,b,sql backup,refs;110,1,thick,d,sql data,refs;60,2,thick,f,sql transaction log,refs;50,3,thin,l,sql audit,refs;30,0,thick,t,sql tempdb data,refs
Server02 90,0,thick,b,sql backup,refs;110,1,thick,d,sql data,refs;60,2,thick,f,sql transaction log,refs;50,3,thin,l,sql audit,refs;30,0,thick,t,sql tempdb data,refs
Assuming the second csv file can also hold info for multiple servers (how else would you know the Uuid for each disk?), you could do like below
$additionalDiskInfo = Import-Csv -Path 'D:\Test\file1.csv'
$serverInfo = Import-Csv -Path 'D:\Test\file2.csv'
$additionalDiskInfo | ForEach-Object {
$server = $_.VmName
$targetItems = $serverInfo | Where-Object { $_.VmName -eq $server }
$extraDisks = $_.AdditionalDisks -split ';'
# make sure you don't run into index errors
$maxItems = [math]::Min($targetItems.Count, $extraDisks.Count)
# loop through the arrays and output combined objects
$result = for ($i = 0; $i -lt $maxItems; $i++) {
$n1, $n2, $n3, $driveLetter, $label, $fileSystem = $extraDisks[$i] -split ','
$targetItems[$i] | Select-Object *,
#{Name = 'DriveLetter'; Expression = {$driveLetter}},
#{Name = 'Label'; Expression = {$label}},
#{Name = 'FileSystem'; Expression = {$fileSystem}}
}
# now write a new csv file for this server
$result | Export-Csv -Path ('D:\Test\DiskInfo-{0}.csv' -f $server) -NoTypeInformation
}
The resulting csv files would look like this
"VMName","Harddisk","Uuid","DriveLetter","Label","FileSystem"
"Server01","Hard disk 2","600C293-7e48-3a63-fb02-cd44df98fe79","b","sql backup","refs"
"Server01","Hard disk 3","600C293-7e48-3a63-fb02-cd44df98f454","d","sql data","refs"
"Server01","Hard disk 4","600C293-7e48-3a63-fb02-cd44df98f10a","f","sql transaction log","refs"
"Server01","Hard disk 5","600C293-7e48-3a63-fb02-cd44df98f483","l","sql audit","refs"
"Server01","Hard disk 6","600C293-7e48-3a63-fb02-cd44df98fced","t","sql tempdb data","refs"
Try like this:
$csv=Import-Csv $CsvSourcePath
$csv|%{
[pscustomobject][ordered]#{
DriveLetter=($_.AdditionalDisks.Split(","))[3]
Label=($_.AdditionalDisks.Split(","))[4]
FileSystem=($_.AdditionalDisks.Split(","))[5]
}}|export-csv $CsvTargetPath -append
I have a two csv files. Both have same number of columns but not in row. I would like to copy 3 columns from csv1 and merge into csv2 using powershell.
I am not good at scripting. Could someone help me?
Here is code I am using.
$csv1 = Import-Csv -LiteralPath C:\BackUp\PScripts\SDO\Report.csv
$csv2 = Import-Csv -LiteralPath C:\BackUp\PScripts\SDO\7Saturday9PM.csv
$Results = #()
foreach ($record in $csv2) {
Add-Member -InputObject $record -MemberType NoteProperty -Name FoundInAD -Value $($csv1 | ? { $_.InstalledOrNA -eq $record.InstalledOrNA } | select -Property FoundInAD)
$Results += $record
}
$Results | Select-Object -Property ComputerName,PatchingStyle,LastSyncTime,LastSyncResult,LastReportedStatusTime,PendingReboot,NeededCount,InstalledOrNA,FoundInAD
I need to import a CSV file and then replace full usernames domain\username with username.
The following lines work but I only receive the amended usernames as the output and not the full file.
Could you please advise?
$TestFile = Import-Csv .\file.csv
$NewFile = $TestFile | ForEach-Object {$_."Username" -replace 'domain\\',''}
When processing CSV input with a ForEach-Object loop you need to output the data back to the pipeline. Also, the -replace operator doesn't modify variables or properties in-place. It takes the value, does the work, and outputs the modified string to the success output stream. If you want to update a property you need to assign the modified value back to that property.
Change this:
$TestFile = Import-Csv .\file.csv
$NewFile = $TestFile | ForEach-Object {$_."Username" -replace 'domain\\',''}
into this:
$NewFile = Import-Csv .\file.csv | ForEach-Object {
$_.Username = $_.Username -replace 'domain\\', '' # update username
$_ # feed data back into the pipeline
}
and the code will do what you want.
You can perform the replace on the string data, then convert it into an object using ConvertFrom-Csv.
$TestFile = (Get-Content .\file.csv) -replace 'domain\\',''
$NewFile = ConvertFrom-Csv $TestFile
Here's one way - get the column names from the input table, iterate each row in the table, and output new custom objects with needed changes.
$table = Import-Csv "Test.csv"
# Get column names
$columnNames = ($table | Select-Object -First 1).PSObject.Properties |
Select-Object -ExpandProperty Name
# Iterate each row in the table
foreach ( $row in $table ) {
$outputObject = New-Object PSObject
foreach ( $columnName in $columnNames ) {
if ( $columnName -eq "Username" ) {
$outputObject | Add-Member NoteProperty "Username" ($row.Username.Split('\')[1])
}
else {
$outputObject | Add-Member NoteProperty $columnName $row.$columnName
}
}
$outputObject
}
To create a new CSV file as output, put the above code in a script and pipe to Export-Csv.
I have a well-formed CSV file that I want to read, reformatting some values while doing so. For some reason the code below does not work ($csv variable ends up being empty):
$csv = Import-Csv "result.log" -Delimiter "|" | ForEach-Object {
$_.PSObject.Properties.Value | ForEach-Object {
if ($_ -like '*---*') { $_ = "" }
}
}
What am I doing wrong?
Your result is blank because you aren't returning anything to the pipeline (you are just changing the value of one or more properties but then not outputting them).
There might be a simpler solution but I think this achieves what you want:
$CSV = Import-Csv "result.log" -Delimiter "|" | Foreach-Object {
$Output = New-Object -TypeName PSObject
$_.PSObject.Properties | Foreach-Object {
If ($_.Value -like '*---*') { $_.Value = '' }
$Output | Add-Member -Name $_.Name -Value $_.Value -MemberType NoteProperty
}
$Output
}
This performs the following for each row of the CSV:
creates a new empty PowerShell object
loops through the properties of the CSV, blanking ones that include --- and then uses Add-Member to add those properties to our object
outputs the new object to the standard pipeline
The result of this goes to the $CSV variable which you then could output as CSV via Export-CSV (or you could skip putting it in the variable and use Export-CSV on the end of the outer ForEach-Object).
I have written a script that tries to determine the max no. of character for each column. This is what I wrote:
$path = 'folder path'
$file = Get-ChildItem $path\*
$FileContent = foreach ($files in $file) {
$FileHeader = #( (Get-Content $files -First 1).Split($delimiter) )
$importcsv = #( Import-Csv $files -Delimiter "$delimiter" )
for ($i=0; $i -lt $FileHeader.Length; $i++) {
#select each column
$Column = #( $importcsv | select $FileHeader[$i] )
#find the max no. of character
$MaxChar = #(($Column[$i] |
Select -ExpandProperty $FileHeader[$i] |
Measure-Object -Maximum -Property Length).Maximum)
$output = New-Object PSObject
$output | Add-Member NoteProperty FullName ($files.FullName)
$output | Add-Member NoteProperty FileName ($files.Name)
$output | Add-Member NoteProperty Time (Get-Date -Format s)
$output | Add-Member NoteProperty FileHeader ($($FileHeader[$i]))
$output | Add-Member NoteProperty MaxCharacter ($($MaxChar[$i]))
Write-Output $output
}
}
The script above is just part of it, so $delimiter is already defined. And finally I will export the result as CSV.
The script runs without any error, but when I open the file it only gives me the first column/header the max no. of character, and the rest of column/header are missing.
The perfect result will be showing each column/header the max no. of character.
Is something wrong with my loop?
my boss is trying to create an automate process to finding all the information from the raw data and use those information to upload to the database, so part of the script that is missing is about determine the delimiter of the raw file, the $CleanHeader is clean version of $FileHeader (remove all special characters, turn capital letters to small letters), those cleanheaders will be use for headers in the table in the database. and he also want to know the maximum character in each column, so that info can use them in creating the size of the column in the table in the database (he knows this part can be done in sql), but he ask me whether it can be done in PowerShell or not.
This should work:
$ht = #{}
# import a CSV and iterate over its rows
Import-Csv $f.FullName -Delimiter "$delimiter" | ForEach-Object {
# iterate over the columns of each row
$_.PSObject.Properties | ForEach-Object {
# update the hashtable if the length of the current column value is greater
# than the value in the hashtable
if ($_.Value.Length -gt $ht[$_.Name]) {
$ht[$_.Name] = $_.Value.Length
}
}
}
# create an object for each key in the hashtable
$date = Get-Date -Format s
$ht.Keys | ForEach-Object {
New-Object -Type PSObject -Property #{
FullName = $f.FullName
Name = $f.Name
Time = $date
FileHeader = $_
MaxCharacter = $ht[$_]
}
}
FileHeader[$i] was returning the column name with quotes : "ColumnName" instead of ColumnName
To fix, just add a trim to the line where you pull the header :
$FileHeader = #( (Get-Content $files -First 1).Split($delimiter).trim('"') )