PowerShell and CSV files - powershell

I have a script, that runs with a for loop.
It outputs 3 variables: $A, $B and $C.
I would like at each iteration to outout those 3 variables onto the same line inside a file, instead of the standard output (each of them separaed by a comma).
I would like to add a header. I am cerating a CSV file.
I have seen several way to output variable inside a CSV file, but not inside a for loop.
Any way to concatenate those 3 variables and append them to a file?
Write-Host $A ',' $B ',' $C | Out-File -FilePath C:\temp\TEST.csv -Append

$A = 'foo'
$B = 'bar'
$C = 'baz'
"$A,$B,$C" # outputs Foo,Bar,Baz
However, if you're working with CSVs, the preferred method is to use Import-Csv and Export-Csv, which handles the formatting for you.

Write-Host output goes directly to the host console. It cannot be pipelined. Also, as has already been mentioned, you normally want to use the Import-Csv and Export-Csv cmdlets when dealing with CSVs (particularly since you want a CSV with headers).
To get a bunch of variables in a form that is exportable by Export-Csv construct a custom object like this:
New-Object -Type PSObject -Property #{
'X' = $A
'Y' = $B
'Z' = $C
} | Export-Csv 'C:\Temp\test.csv' -NoType
The keys of the property hashtable become the column titles of the CSV.
Since you say you want to export data from a for loop you'll need to add the parameter -Append to Export-Csv:
for (...) {
New-Object ... | Export-Csv 'C:\Temp\test.csv' -NoType -Append
}
That is because for loops don't write to the pipeline, meaning that something like this won't work:
for (...) {
New-Object ...
} | Export-Csv 'C:\Temp\test.csv' -NoType
However, depending on what your actual loop looks like you might be able to substitute it with a combination of the range operator (..) and a ForEach-Object loop:
1..5 | ForEach-Object {
New-Object ...
} | Export-Csv 'C:\Temp\test.csv' -NoType

Provided at each iteration to outout those 3 variables is not meant literal,
you could gather the for loops output in a variable via a PSCustomObject and then Export-Csv the result.
For demonstration a simple counting loop:
$Result = for ($i=1;$i -lt 5;$i++){[pscustomobject]#{X=$i;Y=$i+1;Z=$i+2}}
$Result
X Y Z
- - -
1 2 3
2 3 4
3 4 5
4 5 6
Then Export/ConvertTo-Csv:
$Result|ConvertTo-Csv -NoTypeInformation
"X","Y","Z"
"1","2","3"
"2","3","4"
"3","4","5"
"4","5","6"

Related

PowerShell output random numbers to csv. CSV full of empty lines

Actually 2 part question here. The code below outputs nothing but 1000 blank lines to the csv. I'm just trying to output a random range of numbers to a csv and I actually need to follow up with 4 more columns of randomly generated numbers like this first attempt so the second part of this is after getting this first issue resolved how would I direct the next ranges to the other columns?
Get-Random -Count 998 -InputObject (8000..8999) | Export-Csv -Path SingleColumn.csv -NoTypeInformation
Export-Csv same as ConvertTo-Csv is not designed to deal with array of values:
0..10 | ConvertTo-Csv # Outputs `AutomationNull.Value`
Both cmdlets require you to feed them objects:
0..10 | ForEach-Object { [pscustomobject]#{ foo = $_ } } | ConvertTo-Csv
You can create new objects easily with PSCustomObject.
As for the second question, you can dynamically create a dataset by tweaking this code:
$columnsCount = 5
$numberOfrows = 998
$min = 8000; $max = 9000
1..$numberOfrows | ForEach-Object {
$out = [ordered]#{}
foreach($column in 1..$columnsCount) {
$out["Column $column"] = Get-Random -Minimum $min -Maximum $max
}
[pscustomobject] $out
} | Export-Csv path/to/csv.csv -NoTypeInformation
Few lines of Csv output would look something like this:
"Column 1","Column 2","Column 3","Column 4","Column 5"
"8314","8937","8789","8946","8267"
"8902","8500","8107","8006","8287"
"8655","8204","8552","8681","8863"
"8643","8375","8891","8476","8475"
"8338","8243","8175","8568","8917"
"8747","8629","8054","8505","8351"
"8102","8859","8564","8018","8817"
"8810","8154","8845","8074","8436"
"8626","8731","8070","8156","8459"
....

Powershell - Using ConvertFrom-csv

I'm brand new to Powershell. I have a variable that contains comma separated values. What I want to do is read each entry in the csv string variable, and assign it to a variable. I am using ConvertFrom-csv to separate the data with headers.
How can I assign each value to a variable, or even better, use ConvertTo-csv to create a new csv string which only has, for example, columns 2/3/6/7 in it?
I would ultimately want to write that data out to a new csv file.
Here is my test code:
#Setup the variable
$Data = "test1,test2,test3,1234,5678,1/1/2021,12/31/2021"
$Data | ConvertFrom-csv -Header Header1,Header2, Header3, Header4, Header5, Header6, Header7
# Verify that an object has been created.
$Data |
ConvertFrom-csv -Header Header1,Header2, Header3, Header4, Header5, Header6, Header7 |
Get-Member
#Show header1
Write-Host "--------Value from $Data----------------------------------------"
$Data[0] #doesn't work, only displays the first character of the string
Write-Host "-----------------------------------------------------------------"
Let me suggest a different approach. If you use ConvertFrom-Csv and assign the result of a variable ($data), this will be an array of Custom Objects. You can run this through a loop that steps through the elements of the array , one at a time, and then through an inner loop that steps through the properties of each object one at a time, setting a variable with the same name as the field header and the same value as the current record's value.
I don't have code that does exactly what you want. But I'm including code that I wrote a few years back that does something similar only using Import-Csv instead of ConverFrom-Csv.
Import-Csv $driver | % {
$_.psobject.properties | % {Set-variable -name $_.name -value $_.value}
Get-Content $template | % {$ExecutionContext.InvokeCommand.ExpandString($_)}
}
Focus on the first inner loop. Each property of the current object will have a name that came from the header and a value that came from the current record of the Csv file. You can ignore the line that says ExpandString. That's just what I choose to do with the variables once they have been defined.
How can I assign each value to a variable, or even better, use ConvertTo-Csv to create a new csv string which only has, for example, columns 2/3/6/7 in it?
This is one way of automating this:
# Define the CSV without headers
$Data = "test1,test2,test3,1234,5678,1/1/2021,12/31/2021"
# Set the number of headers needed
$headers = $Data.Split(',') | ForEach-Object -Begin { $i = 1 } -Process {
"Header$i"; $i++
}
# Set the desired columns we want
$desiredColumns = 2,3,6,7 | ForEach-Object { $_ - 1 } | ForEach-Object {
$headers[$_]
}
# Convert to CSV and filter by Desired Columns
$Data | ConvertFrom-Csv -Header $headers | Select-Object $desiredColumns
Result
Header2 Header3 Header6 Header7
------- ------- ------- -------
test2 test3 1/1/2021 12/31/2021
Result as CSV
$Data | ConvertFrom-Csv -Header $headers |
Select-Object $desiredColumns | ConvertTo-Csv -NoTypeInformation
"Header2","Header3","Header6","Header7"
"test2","test3","1/1/2021","12/31/2021"

Powershell Remove spaces in the header only of a csv

First line of csv looks like this spaces are at after Path as well
author ,Revision ,Date ,SVNFolder ,Rev,Status,Path
I am trying to remove spaces only and rest of the content will be the same .
author,Revision,Date,SVNFolder,Rev,Status,Path
I tried below
Import-CSV .\script.csv | ForEach-Object {$_.Trimend()}
expanding on the comment with an example since it looks like you may be new:
$text = get-content .\script.csv
$text[0] = $text[0] -replace " ", ""
$csv = $text | ConvertFrom-CSV
Note: The solutions below avoid loading the entire CSV file into memory.
First, get the header row and fix it by removing all whitespace from it:
$header = (Get-Content -TotalCount 1 .\script.csv) -replace '\s+'
If you want to rewrite the CSV file to fix its header problem:
# Write the corrected header and the remaining lines to the output file.
# Note: I'm outputting to a *new* file, to be safe.
# If the file fits into memory as a whole, you can enclose
# Get-Content ... | Select-Object ... in (...) and write back to the
# input file, but note that there's a small risk of data loss, if
# writing back gets interrupted.
& { $header; Get-Content .\script.csv | Select-Object -Skip 1 } |
Set-content -Encoding utf8 .\fixed.csv
Note: I've chosen -Encoding utf8 as the example output character encoding; adjust as needed; note that the default is ASCII(!), which can result in data loss.
If you just want to import the CSV using the fixed headers:
& { $header; Get-Content .\script.csv | Select-Object -Skip 1 } | ConvertFrom-Csv
As for what you tried:
Import-Csv uses the column names in the header as property names of the custom objects it constructs from the input rows.
This property names are locked in at the time of reading the file, and cannot be changed later - unless you explicitly construct new custom objects from the old ones with the property names trimmed.
Import-Csv ... | ForEach-Object {$_.Trimend()}
Since Import-Csv outputs [pscustomobject] instances, reflected one by one in $_ in the ForEach-Object block, your code tries call .TrimEnd() directly on them, which will fail (because it is only [string] instances that have such a method).
Aside from that, as stated, your goal is to trim the property names of these objects, and that cannot be done without constructing new objects.
Read the whole file into an array:
$a = Get-Content test.txt
Replace the spaces in the first array element ([0]) with empty strings:
$a[0] = $a[0] -replace " ", ""
Write over the original file: (Don't forget backups!)
$a | Set-Content test.txt
$inFilePath = "C:\temp\headerwithspaces.csv"
$content = Get-Content $inFilePath
$csvColumnNames = ($content | Select-Object -First 1) -Replace '\s',''
$csvColumnNames = $csvColumnNames -Replace '\s',''
$remainingFile = ($content | Select-Object -Skip 1)

Powershell script to match string between 2 files and merge

I have 2 files that contain strings, each string in both files is delimited by a colon. Both files share a common string and I want to be able to merge both files (based on the common string) into 1 new file.
Examples:
File1.txt
tom:mioihsdihfsdkjhfsdkjf
dick:khsdkjfhlkjdhfsdfdklj
harry:lkjsdlfkjlksdjfsdlkjs
File2.txt
mioihsdihfsdkjhfsdkjf:test1
lkjsdlfkjlksdjfsdlkjs:test2
khsdkjfhlkjdhfsdfdklj:test3
File3.txt (results should look like this)
tom:mioihsdihfsdkjhfsdkjf:test1
dick:khsdkjfhlkjdhfsdfdklj:test3
harry:lkjsdlfkjlksdjfsdlkjs:test2
$File1 = #"
tom:mioihsdihfsdkjhfsdkjf
dick:khsdkjfhlkjdhfsdfdklj
harry:lkjsdlfkjlksdjfsdlkjs
"#
$File2 = #"
mioihsdihfsdkjhfsdkjf:test1
lkjsdlfkjlksdjfsdlkjs:test2
khsdkjfhlkjdhfsdfdklj:test3
"#
# You are probably going to want to use Import-Csv here
# I am using ConvertFrom-Csv as I have "inlined" the contents of the files in the variables above
$file1_contents = ConvertFrom-Csv -InputObject $File1 -Delimiter ":" -Header name, code # specifying a header as there isn't one provided
$file2_contents = ConvertFrom-Csv -InputObject $File2 -Delimiter ":" -Header code, test
# There are almost certainly better ways to do this... but this does work so... meh.
$results = #()
# Loop over one file finding the matches in the other file
foreach ($row in $file1_contents) {
$matched_row = $file2_contents | Where-Object code -eq $row.code
if ($matched_row) {
# Create a hashtable with the values you want from source and matched rows
$result = #{
name = $row.name
code = $row.code
test = $matched_row.test
}
# Append the matched up row to the final result set
$results += New-Object PSObject -Property $result
}
}
# Convert back to CSV format, with a _specific_ column ordering
# Although you'll probably want to use Export-Csv instead
$results |
Select-Object name, code, test |
ConvertTo-Csv -Delimiter ":"

Parse line of text and match with parse of CSV

As a continuation of a script I'm running, working on the following.
I have a CSV file that has formatted information, example as follows:
File named Import.csv:
Name,email,x,y,z
\I\RS\T\Name1\c\x,email#jksjks,d,f
\I\RS\T\Name2\d\f,email#jsshjs,d,f
...
This file is large.
I also have another file called Note.txt.
Name1
Name2
Name3
...
I'm trying to get the content of Import.csv and for each line in Note.txt if the line in Note.txt matches any line in Import.csv, then copy that line into a CSV with append. Continue adding every other line that is matched. Then this loops on each line of the CSV.
I need to find the best way to do it without having it import the CSV multiple times, since it is large.
What I got does the opposite though, I think:
$Dir = PathToFile
$import = Import-Csv $Dir\import.csv
$NoteFile = "$Dir\Note.txt"
$Note = GC $NoteFile
$Name = (($Import.Name).Split("\"))[4]
foreach ($j in $import) {
foreach ($i in $Note) {
$j | where {$Name -eq "$i"} | Export-Csv "$Dir\Result.csv" -NoTypeInfo -Append
}
}
This takes too long and I'm not getting the extraction I need.
This takes too long and I'm not getting the extraction I need.
That's because you only assign $name once, outside of the outer foreach loop, so you're basically performing the same X comparisons for each line in the CSV.
I would rewrite the nested loops as a single Where-Object filter, using the -contains operator:
$Import |Where-Object {$Note -contains $_.Name.Split('\')[4]} |Export-Csv "$Dir\Result.csv" -NoTypeInformation -Append
Group the imported data by your distinguishing feature, filter the groups by name, then expand the remaining groups and write the data to the output file:
Import-Csv "$Dir\import.csv" |
Group-Object { $_.Name.Split('\')[4] } |
Where-Object { $Note -contains $_.Name } |
Select-Object -Expand Group |
Export-Csv "$Dir\Result.csv" -NoType