I am using Import-CSV to get the data from a csv file that looks like:
P1,1,3,4
P2,4,5,6
P3,1,2,3
P4,8.7,6,3
I would like to keep the white-space in front of the text as it indicates the hierarchy. Import-CSV returns:
P1,1,3,4
P2,4,5,6
P3,1,2,3
P4,8.7,6,3
Is there a way to keep the white space?
Your CSV isn't correctly formatted, the items in each row should all be quoted to meet the file specification:
"P1,"1","3","4"
" P2,"4","5","6"
" P3,"1","2","3"
" P4,"8.7","6","3"
You can take a shortcut and only wrap the entries with leading spaces in quotes:
P1,1,3,4
" P2",4,5,6
" P3",1,2,3
" P4",8.7,6,3
Then Import-CSV will function as you're expecting, headers added for demonstration:
Import-CSV leading_spaces.csv -Header "Field1","Field2","Field3","Field4"
Gives you your desired output:
Field1 Field2 Field3 Field4
------ ------ ------ ------
P1 1 3 4
P2 4 5 6
P3 1 2 3
P4 8.7 6 3
As per James C's comment, you can do this with Get-Content:
$myData = Get-Content .\test2.txt
foreach($line in ($myData | Select-Object -Skip 1)){
[array]$results += [pscustomobject]#{
$myData[0].Split(",")[0] = $line.Split(",")[0]
$myData[0].Split(",")[1] = $line.Split(",")[1]
$myData[0].Split(",")[2] = $line.Split(",")[2]
$myData[0].Split(",")[3] = $line.Split(",")[3]
}
}
Maybe this will help, it will create a new object for each row:
get-content test.csv | % {
$row = New-Object PSObject
$i = 0
$_ -split "," | %{
$row | add-member Noteproperty "column$i" $_
$i++
}
$row
}
Output will look like this:
column0 column1 column2 column3
------- ------- ------- -------
P1 1 3 4
P2 4 5 6
P3 1 2 3
P4 8.7 6 3
Related
I have an issue I'm hoping someone might be able to provide some guidance on.
I have two csv files, File1 and File2.
File1:
value1,value2
A,10
B,30
C,45
D,39
File2:
value1,value2
A,10
B,32
C,44
E,7
F,3
What I am looking for is two things. I need to check if any items in value1 have been removed between the files, and/or I need to check if the corresponding number value2 has decreased.
First, to compare the files and find which lines have been added or removed in File1 vs File2. This is easy enough with Compare-Object if I compare only the value1 items.
So the result of my first compare, I'll see that from File1 to File2, I'll see that line E and F have been added, and line D has been removed. Perfect.
However, It's the next part I'm struggling with. I need to then compare value2 in each file, and determine if the number has decreased (or potentially increased or stayed the same).
The tricky part is I can't just compare line 4, value2 in File1 to line 4, value2 in File2, because one is for D and the other is for E. I don't know how to match the items first, then take essentially only the items that match and compare value2 for just those items? But what then happens there was no previous line to match? (because the line was newly added to file2 or the line was removed and only exists in file1)
In the end what I'm trying to come up with is a list of all the value1's that have been removed, and a list of all the value1's whose corresponding value2 has decreased since File1. I do not care about additions or increases.
Hope someone can provide some guidance.
Thank you!
To provide a single-pipeline alternative to Santiago's helpful answer:
It relies on Add-Member, Group-Object, and calculated properties via Select-Object:
(#'
value1,value2
A,10
B,30
C,45
D,39
'# | ConvertFrom-Csv | Add-Member -PassThru Source 1) +
(#'
value1,value2
A,10
B,32
C,44
E,7
F,3
'# | ConvertFrom-Csv | Add-Member -PassThru Source 2) |
Group-Object value1 | ForEach-Object {
if ($_.Count -eq 1) {
$_.Group[0] | Add-Member -PassThru Status #{ 1 = '<='; 2 = '=>' }[$_.Group[0].Source]
}
else {
$grp = $_
$change = ($_.Group[0].value2).CompareTo($_.Group[1].value2)
$_.Group[0] |
Select-Object value1,
#{ name = 'value2'; expression = { $grp.Group[0].value2, $grp.Group[1].value2 } },
#{ name = 'Status'; expression = { $change } }
}
}
The above yields the following:
value1 value2 Status
------ ------ ------
A {10, 10} 0
B {30, 32} -1
C {45, 44} 1
D 39 <=
E 7 =>
F 3 =>
<= and => have the same meaning as in the output of Compare-Object: <= indicates value1 values exclusive to the LHS, => those exclusive to the RHS.
-1 / 0 / 1 map onto an increase / equality / decrease in the value2 property value.
I decided to edit a bit the code, since I think you're doing some sort of data analysis, in my opinion I think the best way you could display the information is by merging both objects.
It may take a bit more processing time depending on how big the CSVs are but I think this is the cleaner approach of showing the data, specially if this is going to be exported to Excel, filtering the result object would be very easy.
Also, mklement0's answer is much more clever than mine. We could say this is maybe a more classic coding approach and he is using all the PowerShell toolset at his disposal. Big props to his answer too.
# Use Import-Csv here, this is just for testing
$csv1 = #'
value1,value2
A,10
B,30
C,45
D,39
'# | ConvertFrom-Csv
$csv2 = #'
value1,value2
A,10
B,32
C,44
E,7
F,3
'# | ConvertFrom-Csv
# Convert CSV1 to hashtable
$map = #{}
foreach($line in $csv1)
{
$map.Add($line.value1,$line.value2)
}
$result = [system.collections.generic.list[pscustomobject]]::new()
$map.Keys.ForEach({
if($_ -notin $csv2.value1)
{
$result.Add(
[pscustomobject]#{
value1 = $_
oldval2 = $map[$_]
newval2 = $null
status = 'REMOVED'
})
}
})
foreach($line in $csv2)
{
$out = [ordered]#{
value1 = $line.value1
newval2 = $line.value2
}
if(-not $map.ContainsKey($line.value1))
{
$out.oldval2 = $null
$out.status = 'ADDED'
$result.Add([pscustomobject]$out)
continue
}
$out.oldval2 = $map[$line.value1]
switch($line.value2)
{
{$_ -lt $map[$line.value1]}
{
$out.status = 'DECREASED'
continue
}
{$_ -gt $map[$line.value1]}
{
$out.status = 'INCREASED'
continue
}
Default
{
$out.status = 'EQUAL'
}
}
$result.Add([pscustomobject]$out)
}
Looking at $result:
PS /> $result
value1 oldval2 newval2 status
------ ------- ------- ------
D 39 REMOVED
A 10 10 EQUAL
B 30 32 INCREASED
C 45 44 DECREASED
E 7 ADDED
F 3 ADDED
If you wanted to see the values 'REMOVED' or 'DECREASED':
PS /> $result.where({$_.status -match 'REMOVED|DECREASED'})
value1 oldval2 newval2 status
------ ------- ------- ------
D 39 REMOVED
C 45 44 DECREASED
I have two CSV Files which look like this:
test.csv:
"Col1","Col2"
"1111","1"
"1122","2"
"1111","3"
"1121","2"
"1121","2"
"1133","2"
"1133","2"
The second looks like this:
test2.csv:
"Number","signs"
"1111","ABC"
"1122","DEF"
"1111","ABC"
"1121","ABC"
"1133","GHI"
Now the goal is to get a summary of all points from test.csv assigned to the "signs" of test2.csv. Reference are the numbers, as you may see.
Should be something like this:
ABC = 8
DEF = 2
GHI = 4
I have tried to test this out but cannot get the goal. What I have so far is:
$var = "C:\PathToCSV"
$csv1 = Import-Csv "$var\test.csv"
$csv2 = Import-Csv "$var\test2.csv"
# Process: group by 'Item' then sum 'Average' for each group
# and create output objects on the fly
$test1 = $csv1 | Group-Object Col1 | ForEach-Object {
New-Object psobject -Property #{
Col1 = $_.Name
Sum = ($_.Group | Measure-Object Col2 -Sum).Sum
}
}
But this gives me back the following output:
Ps> $test1
Sum Col1
--- ----
4 1111
2 1122
4 1121
4 1133
I am not able to get the summary and the mapping of the signs.
Not sure if I understand your question correctly, but I'm going to assume that for each value from the column "signs" you want to lookup the values from the column "Number" in the second CSV and then calculate the sum of the column "Col2" for all matches.
For that I'd build a hashtable with the pre-calculated sums for the unique values from "Col1":
$h1 = #{}
$csv1 | ForEach-Object {
$h1[$_.Col1] += [int]$_.Col2
}
and then build a second hashtable to sum up the lookup results for the values from the second CSV:
$h2 = #{}
$csv2 | ForEach-Object {
$h2[$_.signs] += $h1[$_.Number]
}
However, that produced a different value for "ABC" than what you stated as the desired result in your question when I processed your sample data:
Name Value
---- -----
ABC 12
GHI 4
DEF 2
Or did you mean you want to sum up the corresponding values for the unique numbers for each sign? For that you'd change the second code snippet to something like this:
$h2 = #{}
$csv2 | Group-Object signs | ForEach-Object {
$name = $_.Name
$_.Group | Select-Object -Unique -Expand Number | ForEach-Object {
$h2[$name] += $h1[$_]
}
}
That would produce the desired result from your question:
Name Value
---- -----
ABC 8
GHI 4
DEF 2
I have seen powershell script which also I have in mind. What I would like to add though is another column which would show the side indicator comparators ("==", "<=", "=>") and be named them as MATCH(if "==") and MISMATCH(if "<=" and "=>").
Any advise on how I would do this?
Here is the link of the script (Credits to Florent Courtay)
How can i reorganise powershell's compare-object output?
$a = Compare-Object (Import-Csv 'C:\temp\f1.csv') (Import-Csv 'C:\temp\f2.csv') -property Header,Value
$a | Group-Object -Property Header | % { New-Object -TypeName psobject -Property #{Header=$_.name;newValue=$_.group[0].Value;oldValue=$_.group[1].Value}}
========================================================================
The output I have in mind:
Header1 Old Value New Value STATUS
------ --------- --------- -----------
String1 Value 1 Value 2 MATCH
String2 Value 3 Value 4 MATCH
String3 NA Value 5 MISMATCH
String4 Value 6 NA MISMATCH
Here's a self-contained solution; simply replace the ConvertFrom-Csv calls with your Import-Csv calls:
# Sample CSV input.
$csv1 = #'
Header,Value
a,1
b,2
c,3
'#
$csv2 = #'
Header,Value
a,1a
b,2
d,4
'#
Compare-Object (ConvertFrom-Csv $csv1) (ConvertFrom-Csv $csv2) -Property Header, Value |
Group-Object Header | Sort-Object Name | ForEach-Object {
$newValIndex, $oldValIndex = ((1, 0), (0, 1))[$_.Group[0].SideIndicator -eq '=>']
[pscustomobject] #{
Header = $_.Name
OldValue = ('NA', $_.Group[$oldValIndex].Value)[$null -ne $_.Group[$oldValIndex].Value]
NewValue = ('NA', $_.Group[$newValIndex].Value)[$null -ne $_.Group[$newValIndex].Value]
Status = ('MISMATCH', 'MATCH')[$_.Group.Count -gt 1]
}
}
The above yields:
Header OldValue NewValue Status
------ -------- -------- ------
a 1 1a MATCH
c 3 NA MISMATCH
d NA 4 MISMATCH
Note:
The assumption is that a given Header column value appears at most once in each input file.
The Sort-Object Name call is needed to sort the output by Header valuesThanks, LotPings.
, because, due to how Compare-Object orders its output (right-side-only items first), the order of groups created by Group-Object would not automatically reflect the 1st CSV's order of header values (d would appear before c).
Given:
$column1 = #(1,2,3)
$column2 = #(4,5,6)
How can I combine them into an object $matrix which gets displayed as a matrix with the single arrays as columns:
column1 column2
------- -------
1 4
2 5
3 6
It seems that all of my solutions today requires calculated properties. Try:
$column1 = #(1,2,3)
$column2 = #(4,5,6)
0..($column1.Length-1) | Select-Object #{n="Id";e={$_}}, #{n="Column1";e={$column1[$_]}}, #{n="Column2";e={$column2[$_]}}
Id Column1 Column2
-- ------- -------
0 1 4
1 2 5
2 3 6
If the lengths of the arrays are not equal, you could use:
$column1 = #(1,2,3)
$column2 = #(4,5,6,1)
$max = ($column1, $column2 | Measure-Object -Maximum -Property Count).Maximum
0..$max | Select-Object #{n="Column1";e={$column1[$_]}}, #{n="Column2";e={$column2[$_]}}
I wasn't sure if you needed the Id, so I included it in the first sample to show how to include it.
Little better, maybe:
$column1 = #(1,2,3)
$column2 = #(4,5,6,7)
$i=0
($column1,$column2 | sort length)[1] |
foreach {
new-object psobject -property #{
loess = $Column1[$i]
lowess = $column2[$i++]
}
} | ft -auto
loess lowess
----- ------
1 4
2 5
3 6
7
Here's something I created today. It takes a range of 0 to one of the column lengths, then maps it to a list of hashes. Use the select to turn it into a proper table.
$table = 0..$ColA.Length | % { #{
ColA = $ColA[$_]
ColB = $ColB[$_]
}} | Select ColA, ColB
Using the following variables:
$ColA = #(1, 2, 3)
$ColB = #(4, 5, 6)
Results in
ColB ColA
---- ----
1 4
2 5
3 6
I came up with this.. but it seems too verbose. Anything shorter?
&{
for ($i=0; $i -lt $y.Length; $i++) {
New-Object PSObject -Property #{
y = $y[$i]
loess = $smooth_loess[$i]
lowess = $smooth_lowess[$i]
}
}
} | Format-Table -AutoSize
Here is a combination of mjolinor and Frode F. solutions. I ran into some problems using Frode's object construction trick using select-object. For some reason it would output hash values likely representing object references. I only code in PowerShell a few times a year, so I am just providing this in case anyone else finds it useful (perhaps even my future self).
$column1 = #(1,2,3)
$column2 = #(4,5,6,7)
$column3 = #(2,5,5,2,1,3);
$max = (
$column1,
$column2,
$column3 |
Measure-Object -Maximum -Property Count).Maximum;
$i=0
0..$max |
foreach {
new-object psobject -property #{
col1 = $Column1[$i]
col3 = $column3[$i]
col2 = $column2[$i++]
}
} | ft -auto
I have a file like this:
CONTOSO-A\AAA
CONTOSO-B\BBB
CONTOSO-B\CCC
CONTOSO-A\AAA
....
....
How can count each line to get:
CONTOSO-A\AAA - 2
CONTOSO-B\BBB - 1
CONTOSO-B\CCC - 1
Get-Content .\file.txt | Group-Object | Select-Object name, count
I'd use a hash table:
$counts = #{}
Get-Content c:\somedir\somefile.txt |
foreach { $counts[$_]++ }
$counts
Name Value
---- -----
CONTOSO-A\AAA 2
CONTOSO-B\CCC 1
CONTOSO-B\BBB 1
The simplest way to do this is probably:
PS C:\temp> #"
CONTOSO-A\AAA
CONTOSO-B\BBB
CONTOSO-B\CCC
CONTOSO-A\AAA
"# | set-content test.txt
get-content test.txt | group -NoElement
Count Name
----- ----
2 CONTOSO-A\AAA
1 CONTOSO-B\BBB
1 CONTOSO-B\CCC
Using the -NoElement option to group or Group-Object means you don't have to do a separate select to extract just name and count.
To get the exact format you asked for:
PS C:\temp> get-content test.txt | group -NoElement | % { $_.Name +" - "+$_.Count }
CONTOSO-A\AAA - 2
CONTOSO-B\BBB - 1
CONTOSO-B\CCC - 1
$stat = #{};
cat file.txt | % { $stat["$_"] = $stat["$_"] + 1; }
$stat;