Add a column to a csv file and fill up new column based on an existing column powershell - powershell

I have been trying to add a new column to a csv file and populating the new column based on value in an existing column.
I have a table like this:
|name | number | state | desc|
| ---- | ------ |-------|-----|
|a | 1 | n | i |
|b | 2 | n | j |
|c | 3 | l | j |
|d | 4 | m | k |
I want to add a new column data and populate it based on number column matching with an array.
This is my code so far:
$a=("a","b","c")
$b=("p","q","r")
.
.
.
$c= import-csv -Path "C:\..."
$b |where-object {filtered the file based on some criteria}| select-object number, state, desc, #{Name="data"; Expression={Foreach-object {if ($_.number in $a){$_data = "x"}
elseif($_.number in $b){$_.data = "y"}.......} | export-csv -notypeinformation -path "C:\...."
The script runs but do not populate the new column. Please help

You've got the right idea. Import-Csv will produce an array of objects and you can use Select-Object to add calculated properties, then pipe again to Export-Csv. However, it's not exactly clear from the description or the example code what the expression should be. How do you want to define the new "data" property?
For now I'll work with what we have. The array variables $a & $b will never match anything. Also you can't use ForEach-Object like that, nor will assigning to $data work. The returning value of the Expression script block gets assigned to the property you named data. The following example demonstrates the point:
$a = ( "1", "2", "3")
$b = ( "4", "5", "6")
Import-Csv -Path "C:\temp\12-22-20.csv"|
Select-Object number, state, desc,
#{Name = 'Data'; Expression = { If( $_.Number -in $a ){ 'x' } elseif( $_.Number -in $b ){ 'y' } Else { $null }}} |
Export-Csv -Path "C:\temp\12-22-20_New.csv" -NoTypeInformation
The resulting Csv file will look something like:
number state desc Data
------ ----- ---- ----
1 n i x
2 n j x
3 l j x
4 m k y
Update: Example Using Add-Member
You do not need to use a loop to add the property:
$a = ( "1", "2", "3")
$b = ( "4", "5", "6")
Import-Csv -Path "C:\temp\12-22-20.csv" |
Add-Member -MemberType ScriptProperty -Name "data" -Value { If( $this.Number -in $a ){ 'x' } elseif( $this.Number -in $b ){ 'y' } Else { $null }} -PassThru |
Export-Csv -Path C:\temp\12-22-20_New.csv -NoTypeInformation
By using a MemberType of ScriptProperty we can make a slight modification to script block, replacing $_ with $this The pipe is an implicit loop. I'm not sure if there are any detractions to using a ScriptProperty, but this exports as expected. This approach doesn't require storing the output in $c, but -PassThru would facilitate that if preferred.
99% of the time Select-Object is used for this. The only difference I'm aware of it Select-Object converts the objects to PSCustomObjects. Get-Member will preserve the underlying type, however Import-Csv only outputs PSCustomObjects in the first place, so there's no impact here.

Try iterating over the $c array of imported objects and add the new property to all objects. You want to make sure the new column exists in all of the objects. You can either use Select-Object as in your example, or you can use Add-Member to add it to the imported object.
$a=("a","b","c")
$b=("p","q","r")
...
$c = Import-Csv -Path "C:\..."
$c | ForEach-Object {
$value = ""
# custom logic for value of "data"
# if (...) { $value = ... }
$_ | Add-Member -MemberType NoteProperty -Name "data" -Value $value
}
$c | Export-Csv -NoTypeInformation -path "C:\...."

Related

Transform complex structure to CSV? [duplicate]

I have a JSON file that looks like this:
{
"id": 10011,
"title": "Test procedure",
"slug": "slug",
"url": "http://test.test",
"email": "test#test.com",
"link": "http://test.er",
"subject": "testing",
"level": 1,
"disciplines": [
"discipline_a",
"discipline_b",
"discipline_c"
],
"areas": [
"area_a",
"area_b"
]
},
I was trying to use the following command to convert that into the CSV file:
(Get-Content "PATH_TO\test.json" -Raw | ConvertFrom-Json)| Convertto-CSV -NoTypeInformation | Set-Content "PATH_TO\test.csv"
However, for disciplines and areas I am getting System.Object[] in the resulting CSV file.
Is there a way to put all those nested values as a separate columns in CSV file like area_1, area_2 etc. And the same for disciplines.
2017-11-20, Completely rewrote function to improve performance and add features as -ArrayBase and support for PSStandardMembers and grouped objects.
Flatten-Object
Recursively flattens objects containing arrays, hash tables and (custom) objects. All added properties of the supplied objects will be aligned with the rest of the objects.
Requires PowerShell version 2 or higher.
Cmdlet
Function Flatten-Object { # Version 00.02.12, by iRon
[CmdletBinding()]Param (
[Parameter(ValueFromPipeLine = $True)][Object[]]$Objects,
[String]$Separator = ".", [ValidateSet("", 0, 1)]$Base = 1, [Int]$Depth = 5, [Int]$Uncut = 1,
[String[]]$ToString = ([String], [DateTime], [TimeSpan]), [String[]]$Path = #()
)
$PipeLine = $Input | ForEach {$_}; If ($PipeLine) {$Objects = $PipeLine}
If (#(Get-PSCallStack)[1].Command -eq $MyInvocation.MyCommand.Name -or #(Get-PSCallStack)[1].Command -eq "<position>") {
$Object = #($Objects)[0]; $Iterate = New-Object System.Collections.Specialized.OrderedDictionary
If ($ToString | Where {$Object -is $_}) {$Object = $Object.ToString()}
ElseIf ($Depth) {$Depth--
If ($Object.GetEnumerator.OverloadDefinitions -match "[\W]IDictionaryEnumerator[\W]") {
$Iterate = $Object
} ElseIf ($Object.GetEnumerator.OverloadDefinitions -match "[\W]IEnumerator[\W]") {
$Object.GetEnumerator() | ForEach -Begin {$i = $Base} {$Iterate.($i) = $_; $i += 1}
} Else {
$Names = If ($Uncut) {$Uncut--} Else {$Object.PSStandardMembers.DefaultDisplayPropertySet.ReferencedPropertyNames}
If (!$Names) {$Names = $Object.PSObject.Properties | Where {$_.IsGettable} | Select -Expand Name}
If ($Names) {$Names | ForEach {$Iterate.$_ = $Object.$_}}
}
}
If (#($Iterate.Keys).Count) {
$Iterate.Keys | ForEach {
Flatten-Object #(,$Iterate.$_) $Separator $Base $Depth $Uncut $ToString ($Path + $_)
}
} Else {$Property.(($Path | Where {$_}) -Join $Separator) = $Object}
} ElseIf ($Objects -ne $Null) {
#($Objects) | ForEach -Begin {$Output = #(); $Names = #()} {
New-Variable -Force -Option AllScope -Name Property -Value (New-Object System.Collections.Specialized.OrderedDictionary)
Flatten-Object #(,$_) $Separator $Base $Depth $Uncut $ToString $Path
$Output += New-Object PSObject -Property $Property
$Names += $Output[-1].PSObject.Properties | Select -Expand Name
}
$Output | Select ([String[]]($Names | Select -Unique))
}
}; Set-Alias Flatten Flatten-Object
Syntax
<Object[]> Flatten-Object [-Separator <String>] [-Base "" | 0 | 1] [-Depth <Int>] [-Uncut<Int>] [ToString <Type[]>]
or:
Flatten-Object <Object[]> [[-Separator] <String>] [[-Base] "" | 0 | 1] [[-Depth] <Int>] [[-Uncut] <Int>] [[ToString] <Type[]>]
Parameters
-Object[] <Object[]>
The object (or objects) to be flatten.
-Separator <String> (Default: .)
The separator used between the recursive property names. .
-Depth <Int> (Default: 5)
The maximal depth of flattening a recursive property. Any negative value will result in an unlimited depth and could cause a infinitive loop.
-Uncut <Int> (Default: 1)
The number of object iterations that will left uncut further object properties will be limited to just the DefaultDisplayPropertySet. Any negative value will reveal all properties of all objects.
-Base "" | 0 | 1 (Default: 1)
The first index name of an embedded array:
1, arrays will be 1 based: <Parent>.1, <Parent>.2, <Parent>.3, ...
0, arrays will be 0 based: <Parent>.0, <Parent>.1, <Parent>.2, ...
"", the first item in an array will be unnamed and than followed with 1: <Parent>, <Parent>.1, <Parent>.2, ...
-ToString <Type[]= [String], [DateTime], [TimeSpan]>
A list of value types (default [String], [DateTime], [TimeSpan]) that will be converted to string rather the further flattened. E.g. a [DateTime] could be flattened with additional properties like Date, Day, DayOfWeek etc. but will be converted to a single (String) property instead.
Note:
The parameter -Path is for internal use but could but used to prefix property names.
Examples
Answering the specific question:
(Get-Content "PATH_TO\test.json" -Raw | ConvertFrom-Json) | Flatten-Object | Convertto-CSV -NoTypeInformation | Set-Content "PATH_TO\test.csv"
Result:
{
"url": "http://test.test",
"slug": "slug",
"id": 10011,
"link": "http://test.er",
"level": 1,
"areas.2": "area_b",
"areas.1": "area_a",
"disciplines.3": "discipline_c",
"disciplines.2": "discipline_b",
"disciplines.1": "discipline_a",
"subject": "testing",
"title": "Test procedure",
"email": "test#test.com"
}
Stress testing a more complex custom object:
New-Object PSObject #{
String = [String]"Text"
Char = [Char]65
Byte = [Byte]66
Int = [Int]67
Long = [Long]68
Null = $Null
Booleans = $False, $True
Decimal = [Decimal]69
Single = [Single]70
Double = [Double]71
Array = #("One", "Two", #("Three", "Four"), "Five")
HashTable = #{city="New York"; currency="Dollar"; postalCode=10021; Etc = #("Three", "Four", "Five")}
Object = New-Object PSObject -Property #{Name = "One"; Value = 1; Text = #("First", "1st")}
} | Flatten
Result:
Double : 71
Decimal : 69
Long : 68
Array.1 : One
Array.2 : Two
Array.3.1 : Three
Array.3.2 : Four
Array.4 : Five
Object.Name : One
Object.Value : 1
Object.Text.1 : First
Object.Text.2 : 1st
Int : 67
Byte : 66
HashTable.postalCode : 10021
HashTable.currency : Dollar
HashTable.Etc.1 : Three
HashTable.Etc.2 : Four
HashTable.Etc.3 : Five
HashTable.city : New York
Booleans.1 : False
Booleans.2 : True
String : Text
Char : A
Single : 70
Null :
Flatting grouped objects:
$csv | Group Name | Flatten | Format-Table # https://stackoverflow.com/a/47409634/1701026
Flatting common objects:
(Get-Process)[0] | Flatten-Object
Or a list (array) of objects:
Get-Service | Flatten-Object -Depth 3 | Export-CSV Service.csv
Note that a command as below takes hours to compute:
Get-Process | Flatten-Object | Export-CSV Process.csv
Why? because it results in a table with a few hundred rows and several thousand columns. So if you if would like to use this for flatting process, you beter limit the number of rows (using the Where-Object cmdlet) or the number of columns (using the Select-Object cmdlet).
For the latest Flatten-Object version, see: https://powersnippets.com/flatten-object/
The CSV conversion/export cmdlets have no way of "flattening" an object, and I may be missing something, but I know of no way to do this with a built-in cmdlet or feature.
If you can guarantee that disciplines and areas will always have the same number of elements, you can trivialize it by using Select-Object with derived properties to do this:
$properties=#('id','title','slug','url','email','link','subject','level',
#{Name='discipline_1';Expression={$_.disciplines[0]}}
#{Name='discipline_2';Expression={$_.disciplines[1]}}
#{Name='discipline_3';Expression={$_.disciplines[2]}}
#{Name='area_1';Expression={$_.areas[0]}}
#{Name='area_2';Expression={$_.areas[1]}}
)
(Get-Content 'PATH_TO\test.json' -Raw | ConvertFrom-Json)| Select-Object -Property $properties | Export-CSV -NoTypeInformation -Path 'PATH_TO\test.csv'
However, I am assuming that disciplines and areas will be variable length for each record. In that case, you will have to loop over the input and pull the highest count value for both disciplines and areas, then build the properties array dynamically:
$inputData = Get-Content 'PATH_TO\test.json' -Raw | ConvertFrom-Json
$counts = $inputData | Select-Object -Property #{Name='disciplineCount';Expression={$_.disciplines.Count}},#{Name='areaCount';Expression={$_.areas.count}}
$maxDisciplines = $counts | Measure-Object -Maximum -Property disciplineCount | Select-Object -ExpandProperty Maximum
$maxAreas = $counts | Measure-Object -Maximum -Property areaCount | Select-Object -ExpandProperty Maximum
$properties=#('id','title','slug','url','email','link','subject','level')
1..$maxDisciplines | % {
$properties += #{Name="discipline_$_";Expression=[scriptblock]::create("`$_.disciplines[$($_ - 1)]")}
}
1..$maxAreas | % {
$properties += #{Name="area_$_";Expression=[scriptblock]::create("`$_.areas[$($_ - 1)]")}
}
$inputData | Select-Object -Property $properties | Export-CSV -NoTypeInformation -Path 'PATH_TO\test.csv'
This code hasn't been fully tested, so it may need some tweaking to work 100%, but I believe the ideas are solid =)

Compare-Object - Separate side columns

Is it possible to display the results of a PowerShell Compare-Object in two columns showing the differences of reference vs difference objects?
For example using my current cmdline:
Compare-Object $Base $Test
Gives:
InputObject SideIndicator
987654 =>
555555 <=
123456 <=
In reality the list is rather long. For easier data reading is it possible to format the data like so:
Base Test
555555 987654
123456
So each column shows which elements exist in that object vs the other.
For bonus points it would be fantastic to have a count in the column header like so:
Base(2) Test(1)
555555 987654
123456
Possible? Sure. Feasible? Not so much. PowerShell wasn't really built for creating this kind of tabular output. What you can do is collect the differences in a hashtable as nested arrays by input file:
$ht = #{}
Compare-Object $Base $Test | ForEach-Object {
$value = $_.InputObject
switch ($_.SideIndicator) {
'=>' { $ht['Test'] += #($value) }
'<=' { $ht['Base'] += #($value) }
}
}
then transpose the hashtable:
$cnt = $ht.Values |
ForEach-Object { $_.Count } |
Sort-Object |
Select-Object -Last 1
$keys = $ht.Keys | Sort-Object
0..($cnt-1) | ForEach-Object {
$props = [ordered]#{}
foreach ($key in $keys) {
$props[$key] = $ht[$key][$_]
}
New-Object -Type PSObject -Property $props
} | Format-Table -AutoSize
To include the item count in the header name change $props[$key] to $props["$key($($ht[$key].Count))"].

Powershell - Insert column in between specific columns in csv file

I have 2 csv files
First file:
firstName,secondName
1234,Value1
2345,Value1
3456,Value1
4567,Value3
7645,Value3
Second file:
firstName,fileSplitter,Csv2ColumnOne,Csv2ColumnTwo,Csv2ColumnThree
1234,,1234,abc,Value1
1234,,1234,asd,Value1
3456,,3456,qwe,Value1
4567,,4567,mnb,Value1
I want to insert column secondName in the second file in between columns firstName and fileSplitter.
The result should look like this:
firstName,secondName,fileSplitter,Csv2ColumnOne,Csv2ColumnTwo,Csv2ColumnThree
1234,Value1,,1234,abc,Value1
1234,Value1,,1234,asd,Value1
3456,Value1,,3456,qwe,Value1
4567,Value3,,4567,mnb,Value1
I'm trying the following code:
Function InsertColumnInBetweenColumns
{
Param ($FirstFileFirstColumnTitle, $firstFile, [string]$1stColumnName, [string]$2ndColumnName, [string]$columnMergedFileBeforeInput)
Write-Host "Creating hash table with columns values `"$1stColumnName`" `"$2ndColumnName`" From $OimFileWithMatches"
$hashFirstFileTwoColumns = #{}
Import-Csv $firstFile | ForEach-Object {$hashFirstFileTwoColumns[$_.$1stColumnName] = $_.$2ndColumnName}
Write-Host "Complete."
Write-Host "Appending Merge file with column `"$2ndColumnName`" from file $secondCsvFileWithLocalPath"
Import-Csv $outputCsvFileWithLocalPath | Select-Object $columnMergedFileBeforeInput, #{n=$2ndColumnName; e={
if ($hashFirstFileTwoColumns.ContainsKey($_.$FirstFileFirstColumnTitle)) {
$hashFirstFileTwoColumns[$_.$FirstFileFirstColumnTitle]
} Else {
'Not Found'
}}}, * | Export-Csv "$outputCsvFileWithLocalPath-temp" -NoType -Force
Move-Item "$outputCsvFileWithLocalPath-temp" $outputCsvFileWithLocalPath -Force
Write-Host "Complete."
Write-Host ""
}
This function will be called in a for loop for each column found in the first file (can contain an indefinite number). For testing, I am only using 2 columns from the first file.
I'm getting an error output resulting the following:
Select : Property cannot be processed because property "firstName" already exists.
At C:\Scripts\Tests\Compare2CsvFilesOutput1WithMatchesOnly.ps1:490 char:43
+ Import-Csv $outputCsvFileWithLocalPath | Select $columnMergedFileBeforeInput, # ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (#{firstName=L...ntName=asdfas}:PSObject) [Select-Object], PSArgume
ntException
+ FullyQualifiedErrorId : AlreadyExistingUserSpecifiedPropertyNoExpand,Microsoft.PowerShell.Commands.SelectObjectC
ommand
I know the issue is where it says Select-Object $columnMergedFileBeforeInput,.
How can I get the loop statement to insert the column in between the before column (name is specified), and append the rest using *?
Update
Just an fyi, changing this line Select-Object $columnMergedFileBeforeInput, #{n=$2ndColumnName..... to this line Select-Object #{n=$2ndColumnName..... works, it just attaches the columns out of order. That is why I'm trying to insert the column in between. Maybe if i do it this way but insert the columns in backwards using the for loop, this would work...
Not sure if this is the most efficient way to do it, but it should do the trick. It just adds the property to the record from file2, then reorders the output so secondName is the second column. You can output results to csv where required too (ConvertTo-Csv).
$file1 = Import-Csv -Path file1.csv
$file2 = Import-Csv -Path file2.csv
$results = #()
ForEach ($record In $file2) {
Add-Member -InputObject $record -MemberType NoteProperty -Name secondName -Value $($file1 | ? { $_.firstName -eq $record.firstName } | Select -ExpandProperty secondName)
$results += $record
}
$results | Select-Object -Property firstName,secondName,fileSplitter,Csv2ColumnOne,Csv2ColumnTwo,Csv2ColumnThree
I've created the following function. What it does is find the match (in this case "firstname") and adds the matching columnname to the new array afther the columnname on which the match is made (little difficult to explain in my poor English).
function Add-ColumnAfterMatchingColumn{
[CmdletBinding()]
param(
[string]$MainFile,
[string]$MatchingFile,
[string]$MatchColumnName,
[string]$MatchingColumnName
)
# Import data from two files
$file1 = Import-Csv -Path $MainFile
$file2 = Import-Csv -Path $MatchingFile
# Find column names and order them
$columnnames = $file2 | gm | where {$_.MemberType -like "NoteProperty"} | Select Name | %{$_.Name}
[array]::Reverse($columnnames)
# Find $MatchColumnName index and put the $MatchingColumnName after it
$MatchColumnNameIndex = [array]::IndexOf($columnnames, $MatchColumnName)
if($MatchColumnNameIndex -eq -1){
$MatchColumnNameIndex = 0
}
$columnnames = $columnnames[0..$MatchColumnNameIndex] + $MatchingColumnName + $columnnames[($MatchColumnNameIndex+1)..($columnnames.Length -1)]
$returnObject = #()
foreach ($item in $file2){
# Find corresponding value MatchingColumnName in $file1 and add it to the current item
$item | Add-Member -Name "$MatchingColumnName" -Value ($file1 | ?{$_."$($MatchColumnName)" -eq $item."$($MatchColumnName)"})."$MatchingColumnName" -MemberType NoteProperty
# Add current item to the returnObject array, in the correct order
$newItem = New-Object psobject
foreach ($columnname in [string[]]$columnnames){
$newItem | Add-Member -Name $columnname -Value $item."$columnname" -MemberType NoteProperty
}
$returnObject += $newItem
}
return $returnObject
}
When you run this function you will have the following output:
Add-ColumnAfterMatchingColumn -MainFile C:\Temp\file1.csv -MatchingFile C:\Temp\file2.csv -MatchColumnName "firstname" -MatchingColumnName "secondname" | ft
firstName secondname fileSplitter Csv2ColumnTwo Csv2ColumnThree Csv2ColumnOne
--------- ---------- ------------ ------------- --------------- -------------
1234 Value1 abc Value1 1234
1234 Value1 asd Value1 1234
3456 Value1 qwe Value1 3456
4567 Value3 mnb Value1 4567

PowerShell Import-Csv Issue - Why is my output being treated as a single column and not a CSV?

So I have a CSV file which I need to manipulate a bit, select the data I need and export to another CSV file.
The code I have is:
$rawCSV = "C:\Files\raw.csv"
$outputCSV = "C:\Files\output.csv"
Import-Csv -Header #("a","b","c","d") -Path $rawCSV |
select -Skip 7 |
Where-Object { $_.b.length -gt 1 } |
ft b,a,c,d |
Out-File $outputCSV
So this code uses the Import-Csv command to allow me to select just the columns I need, add some headers in the order I want and then I am simply putting the output in to a CSV file called $outputCSV. The contents of this output file look something like this:
b a c d
- - - -
john smith 29 England
mary poopins 79 Walton
I am not sure what the delimiter is in this output and rather than these columns being treated as individuals, they are treated as just one column. I have gone on further to replace all the spaces with a comma using the code:
$b = foreach ($line in $a)
{
$fields = $line -split '`n'
foreach ($field in $fields)
{
$field -replace " +",","
}
}
Which produces a file that looks like this:
b,a,c,d
john,smith,29,England
mary,poppins,79,Walton
But these are all still treated as one column instead of four separate columns as I need.
* UPDATE *
Using the answer given by #, I now get a file looking like this:
Don't use ft to reorder your columns - it's intended to format output for the screen, not really suitable for CSV.
"Manual" solution:
$rawCSV = "C:\Files\raw.csv"
$outputCSV = "C:\Files\output.csv"
# Import and filter your raw data
$RawData = Import-Csv -Header #("a","b","c","d") -Path $rawCSV
$Data = $RawData | Select -Skip 7 | Where-Object { $_.b.length -gt 1 }
# Write your headers to the output file
"b","a","c","d" -join ',' | Out-File $outputCSV -Force
$ReorderedData = foreach($Row in $Data){
# Reorder the columns in each row
'{0},{1},{2},{3}' -f $Row.b , $Row.a , $Row.c, $Row.d
}
# Write the reordered rows to the output file
$ReorderedData | Out-File $outputCSV -Append
Using Export-Csv:
As of PowerShell 3.0, you could also push the rows into a [pscustomobject] and pipe that to Export-Csv (pscustomobject preserves the order in which you supply the properties):
$rawCSV = "C:\Files\raw.csv"
$outputCSV = "C:\Files\output.csv"
# Import and filter your raw data
$RawData = Import-Csv -Header #("a","b","c","d") -Path $rawCSV
$Data = $RawData | Select -Skip 7 | Where-Object { $_.b.length -gt 1 }
# Take the columns you're interested in, put them into new custom objects and export to CSV
$Data | ForEach-Object {
[pscustomobject]#{ "b" = $_.b; "a" = $_.a; "c" = $_.c; "d" = $_.d }
} | Export-Csv -NoTypeInformation $outputCSV
Export-Csv will take care of enclosing strings in quotes to escape ',' properly (one thing less for you to worry about)
First of all, what your raw CSV file looks like? If it's already like this
john,smith,29,England
mary,poppins,79,Walton
then import-csv will give you an array of objects which you can easily manipulate (and objects are the main reason to use PowerShell ;). For example, to check what you have after import:
$r = Import-Csv -Path $rawCSV -Header #("b","a","c","d")
$r.GetType()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True Object[] System.Array
$r[0] | get-member
TypeName: System.Management.Automation.PSCustomObject
Name MemberType Definition
---- ---------- ----------
Equals Method bool Equals(System.Object obj)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
ToString Method string ToString()
a NoteProperty System.String a=smith
b NoteProperty System.String b=john
c NoteProperty System.String c=29
d NoteProperty System.String d=England
For now you have array of objects with properties named "a","b","c","d". To manipulate objects you have select-object cmdlet:
$r | Select-Object a,b,c,d
a b c d
- - - -
smith john 29 England
poppins mary 79 Walton
And after all use export-csv to set the output file:
$r | where { $_.b.length -gt 1 } |
select a,b,c,d |
Export-Csv -NoTypeInformation -Encoding utf8 -path $outputCSV
I could think of two possible reasons why your data teated as one column:
consuming application expect different encoding and can't find
delimiters
delimiters are not commas but something else

powershell: Check if any of a bunch of properties is set

I'm importing a csv-file which looks like this:
id,value1.1,value1.2,value1.3,Value2.1,Value2.2,Value3.1,Value3.2
row1,v1.1,,v1.3
row2,,,,v2.1,v2.2
row3,,,,,,,v3.2
Now I want to check, if any of the value-properties in one group is set.
I can do
Import-Csv .\test.csv | where {$_.Value1.1 -or $_.Value1.2 -or $_.Value1.3}
or
Import-Csv .\test.csv | foreach {
if ($_.Value1 -or $_.Value2 -or $_.Value3) {
Write-Output $_
}
}
But my "real" csv-file contains about 200 columns and I have to check 31 properties x 5 different object types that are mixed up in this csv. So my code will be realy ugly.
Is there anything like
where {$_.Value1.*}
or
where {$ArrayWithPropertyNames}
?
You could easily use the Get-Member cmdlet to get the properties which have the correct prefix (just use * as a wildcard after the prefix).
So to achieve what you want you could just filter the data based on whether any of the properties with the correct prefix contains data.
The script below uses your sample data, with a row4 added, and filters the list to find all items which have a value in any property starting with value1.
$csv = #"
id,value1.1,value1.2,value1.3,Value2.1,Value2.2,Value3.1,Value3.2
row1,v1.1,,v1.3
row2,,,,v2.1,v2.2
row3,,,,,,,v3.2
row4,v1.1,,v1.3
"#
$data = ConvertFrom-csv $csv
$data | Where {
$currentDataItem = $_
$propertyValues = $currentDataItem |
# Get's all the properties with the correct prefix
Get-Member 'value1*' -MemberType NoteProperty |
# Gets the values for each of those properties
Foreach { $currentDataItem.($_.Name) } |
# Only keep the property value if it has a value
Where { $_ }
# Could just return $propertyValues, but this makes the intention clearer
$hasValueOnPrefixedProperty = $propertyValues.Length -gt 0
Write-Output $hasValueOnPrefixedProperty
}
Alternate solution:
$PropsToCheck = 'Value1*'
Import-csv .\test.csv |
Where {
(($_ | Select $PropsToCheck).psobject.properties.value) -contains ''
}