I have a data as follows :
Percent SiteName
-------------------------- --------
95.15 Walnu
88.15 Tucson
99.14 Tarrace
99.39 Tampa
94.73 walnu
92.85 Tarrace
I want to remove the duplicates in sitename and want the data as :
Percent SiteName
-------------------------- --------
95.15 Walnu
88.15 Tucson
99.14 Tarrace
99.39 Tampa
select-object -unique works only when i want a single parameter in the output. Is there any method to do this.
Please help
Thanks in advance
If you want to filter out extra objects that contain an already output SiteName value, you can use Group-Object.
# $data contains your collection of objects
$data | Group-Object SiteName | Foreach-Object { $_.Group[0] }
Group-Object creates a collection of items based on property values. Using SiteName will retrieve all duplicate SiteName valued objects into a GroupInfo object. The Group property contains the collection of all grouped objects. Using the index [0], the first one will be output, which works even for SiteName values that have no duplicates.
Related
Assuming a CSV File:
Name,group_name,group_id
foo,Best,1
bar,Worst,2
baz,Best,1
bob,Worst,2
What's the simplest form of Grouping by Powershell I can use to have output like:
Count group_id group_name Names
----- -------- ---------- -----
2 1 Best ["foo", "baz"]
2 2 Worst ["bar", "bob"]
Use the Group-Object cmdlet to group the rows together by name and id, then use Select-Object to extract the appropriate details from each group as individual properties:
# replace with `$Data = Import-Csv path\to\file.csv`
$Data = #'
Name,group_name,group_id
foo,Best,1
bar,Worst,2
baz,Best,1
bob,Worst,2
'#|ConvertFrom-Csv
# Group rows, then construct output record with `Select-Object`
$Data |Group-Object group_name,group_id |Select-Object Count,#{Name='group_id';Expression={$_.Group[0].group_id}},#{Name='group_name';Expression={$_.Group[0].group_name}},#{Name='Names';Expression={$_.Group.Name}}
I have a csv file, that contains the next data:
Pages,Pages BN,Pages Color,Customer
145,117,28,Report_Alexis
46,31,15,Report_Alexis
75,27,48,Report_Alexis
145,117,28,Report_Jack
46,31,15,Report_Jack
75,27,48,Report_Jack
145,117,28,Report_Amy
46,31,15,Report_Amy
75,27,48,Report_Amy
So what i need to do , is sum each column based on the report name and the export to another csv file like this
Pages,Pages BN,Pages Color,Customer
266,175,91,Report_Alexis
266,175,91,Report_Jack
266,175,91,Report_Amy
How can i do this?
I tried with this:
$coutnpages = Import-Csv "C:\temp\testcount\final file2.csv" |where {$_.Filename -eq 'Report_Jack'} | Measure-Object -Property Pages -Sum
then
$Countpages.Sum | Set-Content -Path "C:\temp\testcount\final file3.csv"
But this is just one, and then i dont know how to follow.
Can you please help me?
Working code
$IdentityColumns = #('Customer')
$ColumnsToSum = #('Pages', 'Pages BN', 'Pages Color')
$CSVFileInput = 'S:\SCRIPTS\1.csv'
Import-Csv -Path $CSVFileInput |
Group-Object -Property $IdentityColumns |
ForEach-Object {
$resultHT = #{ Customer = $_.Name } # This is result HashTable (Key-Value collection). We add here sum's next line.
#($_.Group | Measure-Object -Property $ColumnsToSum -Sum ) | # Run calculating of sum for all $ColumnsToSum`s in one line
ForEach-Object { $resultHT[$_.Property] = $_.Sum } # For each calculated property we set property in result HashTable
return [PSCustomObject]$resultHT # Convert HashTable to PSCustomObject. This better.
} | # End of ForEach-Object by groups
Select #($ColumnsToSum + $IdentityColumns) | # This sets order of columns. It may be important.
Out-GridView # Or replace with Export-Csv
#Export-Csv ...
Explanation:
Use Group-Object to make collection of groups. Groups have 4 properties:
Name - Name of group, equals to stingified values of property(-ies) you're grouping by
Values - Collection of values of properties you're grouping by (not stringified)
Count - Count of elements grouped into this group
Group - Values of elements grouped into this group
For grouping by single string properties (in this case it is ok), you can easily use Name of group, otherwise, always use Values.
So after Group-Object, you iterate not on collection-of-rows of CSV, but on collection-of-collections-of-rows grouped by some condition.
Measure-Object can process more than one propertiy for single pass (not mixing between values from different properties), we use this actively. This results in array of objects with attribute Property equal to passed to Measure-Object and value (Sum in our case). We move those Property=Sum pairs to hashtable.
[PSCustomObject] converts hashtable to object. Objects are always better for output.
I require an additional column added to this output called "Location" with each row of data containing the word "Varonis".
$fs | ForEach-Object {
$machine = $_.ServerName
$_.Volumes | Select-Object #{n='machine';e={$machine}}, Share, FileWalkMethod
} | Export-Csv D:\data\splunk\otl_varonis\otl_varonis_monitoring.csv -NoType
Current output:
"hmanas01n","E$","VaronisWindows"
Desired output:
"hmanas01n","E$","VaronisWindows", "Varonis"
Essentially this is a question of how to add custom fields to an existing object (i.e. data)...
In the code example you give, the columns of output are defined by the following command:
Select-Object #{n='machine';e={$machine}}, Share, FileWalkMethod
This effectively filters the data to just the Share and FileWalkMethod properties, as well as adding a new column called machine. The machine column is defined with two key-value pairs: n (or name/label) is the column title and e (or expression) is a bit of code that sets the value for that column; in this case whatever the $machine variable is set to.
You need to replicate the same mechanism that adds the machine column by adding this to the end of the Select-Object statement, where the expression is just a static value:
#{n="Location";e={"Varonis"}}
I am importing data from various csv files, usually with 4 or 5 fields.
e.g. one might look like:
id, name, surname, age
1,tom,smith,32
2,fred,bloggs,50
I have managed to grab the header row titles into and array that looks like:
id, name, surname, age
the first data row looks like:
#{ID=1; name=tom; surname=smith; age=32}
say I assign it to $myRow
what I want to be able to do is access the ID, name etc field in $myRow by index, 0, 1, 2 etc, not by the property name.
Is this possible?
Thanks
You can do something like this, but it may be slow for large sets of rows and/or properties:
$users =
import-csv myusers.csv |
foreach {
$i=0
foreach ($property in $_.psobject.properties.name)
{
$_ | Add-Member -MemberType AliasProperty -Name $i -Value $property -passthru
$i++
}
}
That just adds an Alias property for each property name in the object
When I wanted to do something similar, I went about it differently.
I used Import-Csv to get the contents into a table. Then I stepped through the table, row by row, and used an inner loop to retrieve the field values, one by one into variables with the same name as the column name.
This created a context where I could apply the values to variables embedded in some kind of template. Here is an edited version of the code.
foreach ($item in $list) {
$item | Get-Member -membertype properties | foreach {
Set-variable -name $_.name -value $item.$($_.name)
}
Invoke-expression($template) >> Outputfile.txt
}
I'm writing the expanded templates to an output file, but you get the idea. This end up working more or less the way mail merge applies a mailing list to a form letter.
I wouldn't use this approach for more than a few hundred rows and a dozen columns. It gets slow.
Explanation:
The inner loop needs more explanation. $list is a table that contains
the imported image of a csv file. $item is one row from this table.
Get-Member gets each field (called a property) from that row. Each
field has a name and a value. $_.name delivers the name of the
current field. $item.($_.name) delivers the value. Set-Variable
creates a variable. It's very inefficient to create the same
variables over and over again for each row in the table, but I don't
care.
This snippet was clipped from a larger snippet that imports a list and a template, produces an expansion of the template for each item in the list, and outputs the series of expansions into a text file. I didn't include the whole snippet because it's too far afield from the question that was asked.
You can actually index your array with ($MyRow[1]).age in order to get the age of the first row.
In PowerShell I want to pass the name of a header in a CSV file into another function in the PowerShell script.
How can I retrieve the value-text of a header name into a variable in CSV?
e.g. if I have the following CSV data:
ID Name Country
-- ---- -------
1 John United States
2 Beatrice Germany
3 Jouni Finland
4 Marcel France
In the above example how can I retrieve the Country column value text as "Country" text into a variable in my script?
(Note: I am familiar with the notation $_.Country to retrieve the value of, for example, "Germany" from a row by importing the CSV in Powershell)
My specific issue is that currently I have the following function in my script:
function GetItemIdFromTitle([string]$LookupTitle, [ref]$LookupId)
{
$LookupField = $LookupList.Fields["DEPTCATEGORY"]
$LookupItem = $LookupList.Items | where {$_['DEPTCATEGORY'] -like "*$LookupTitle*"}
$LookupId.Value = $LookupItem.ID
}
This currently takes a string value -> $LookupTitle and uses that to find an item in a SharePoint list. As you can see in the script I am hard-coding in the column name as "DEPTCATEGORY". This is the column name that will be looked up to in the SharePoint list.
Instead of hard-coding the column name I want to pass in the name of the column for the corresponding $LookupTitle value and replace the hard-coded "DEPTCATEGORY".
I am calling the above function as follows:
#GET THE LOOKUP COLUMN ID
GetItemIdFromTitle $_.DEPTCAT ([ref]$LookupIdValue)
( $_.DEPTCAT is the value from the row in the CSV column. )
Can I do something like
$myCountryColumnName = $_.Country.Property.Title
or
$myCategoryColumnName = $_.DEPTCAT.Property.Name
to get the column name from the CSV?
If you have an object in $obj, you could list all the property headers like this:
$obj | Get-member -MemberType 'NoteProperty' | Select-Object -ExpandProperty 'Name'
This is an array, so you can reference them individually like this:
($obj | Get-member -MemberType 'NoteProperty' | Select-Object -ExpandProperty 'Name')[0]
This would just give you the name of the first property for instance.
Assuming that you have already read in the CSV using Import-CSV you can examine the property of the resulting object. Similar to this answer but maintains column order in the resulting array
Examining the first row / element
$data = import-csv $path
$data[0].psobject.properties.name
So the second line will return a string array of the properties.
To get the column name from the csv, first, put the names of the column headers into an array (this will also allow you to loop through each column, if needed) ...
$inFilePath = "C:\path\to\file.csv"
$csvColumnNames = (Get-Content $inFilePath | Select-Object -First 1).Split(",")
... , secondly, index into the array by column position (index starts at 0). Given your original example it would be;
$myCountryColumnName = $csvColumnNames[2]
This is more of a general comment than an answer.
I needed to pull the first column header name from CSVs and I started with selecting the NoteProperty fields from Get-Member. This doesn't work because the order of the NoteProperty column header names might not match the order of the column headers in the CSV file.
futureSPQR's method will work every time because the text won't get reordered on you. Below is my one-liner version of his method to get the first column header name.
((Get-Content filename.csv)[0] -split(','))[0]
$f = Import-Csv "...csv"
Get-Member -InputObject $f[0] |
Where-Object {$_.MemberType -eq "NoteProperty"} |
select-object Name
If you're looking to check if a header name exists in the array, use below code
$Array | Get-member -MemberType 'NoteProperty'| where {$_.Name -eq "ColumnName"}