So at the momemt I'm searching for a way to merge 2 CSV files.
Here is an example for what I mean:
CSV1
"Name","Count"
"Klaus","3"
"Hans","2"
"Gerhard","1"
"Nina","6"
"Julia","10"
"Caro","19"
CSV2
"Name","Count"
"Klaus","2"
"Hans","1"
"Gerhard","1"
"Nina","1"
Now if I merge both, the output/result should be:
"Name","Count"
"Klaus","5"
"Hans","3"
"Gerhard","2"
"Nina","7"
"Julia","10"
"Caro","19"
I tried a lot, but I´ve never had suscess; I always had wrong results. Does anyone have an idea how to do this?
You can use Group-Object (alias group) to group everything by the Name property. Then you just need to sum up the Count property of each guy in the group. Measure-Object (alias measure) will do sums for you.
$grouped = Import-Csv .\csv1.csv, .\csv2.csv | group Name
$combined = $grouped |%{
New-Object PsObject -Prop #{ Name = $_.Name; Count = ($_.Group | measure -sum -prop Count).Sum }
}
$combined | Export-Csv .\combined.csv -NoType
Import the CSV files and convert each to a hash table, then find the common names:
$csv1 = Import-Csv -Path csv1.csv
$csv2 = Import-Csv -Path csv2.csv
$HashCSV1 = #{}
$HashCSV2 = #{}
$HashMerge = #{}
foreach($r in $csv1)
{
$HashCSV1[$r.Name] = $r.Count
}
foreach($r in $csv2)
{
$HashCSV2[$r.Name] = $r.Count
}
foreach ($key in $HashCSV1.Keys) {
if ($HashCSV2.ContainsKey($key)) {
$HashMerge[$key] = [int]$HashCSV1[$key] + [int]$HashCSV2[$key]
} else {
$HashMerge[$key] = $HashCSV1[$key]
}
}
foreach ($key in $HashCSV2.Keys) {
if (-not $HashCSV1.ContainsKey($key)) {
$HashMerge[$key] = $HashCSV2[$key]
}
}
&{$HashMerge.getenumerator() |
foreach {new-object psobject -Property #{Name = $_.name;Count=$_.value}}
} | export-csv merge.csv -notype
Related
I need to import a few groups from a CSV file and then export members - but I need it like one row each, something like:
"Group_name1", "member1, member2,member3"
"Group_name2", "member1,member2,member3"
"Group_name3", "member1,member2,member3"
And my script is working fine for a single group but I'm having troubles with adding a for-each loop - the result contains only last item from the CSV..
#$DL='plum'
$DL_List = "C:\ps1\shared_mailboxes\groups.csv"
$DL_array = (Import-Csv -Path $DL_List).name
foreach ($DL in $DL_array)
{
$DL_Membership = (Get-DistributionGroupMember -identity $DL).displayName
if([string]$DL_Membership -ne "")
{
$Members = ""
foreach($DL_Membership in $DL_Membership)
{
if($Members -ne "")
{
$Members=$Members +","
}
$Members = $Members + $DL_Membership
}
}
}
$ExportCSV=".\group_members_$((Get-Date -format yyyy-MMM-dd-ddd` hh-mm` tt).ToString()).csv"
$Result = #{'Group'=$DL;'Users'=$Members}
$Results = New-Object PSObject -Property $Result
$Results |select-object 'Group','Users' | Export-Csv -Path $ExportCSV -Notype -Append
I googled it but I'm not sure what I should change in my script..
The script has been re-writed using -join operator and now it's working as excepted.
$DL_List = Import-Csv "C:\ps1\shared_mailboxes\groups.csv"
$CsvOut = foreach ( $dl in $DL_List ) {
$Group = $dl.name
$MemberArray = (Get-DistributionGroupMember -identity $Group).displayName
$MemberList = $MemberArray -join ','
# generate the output object
# that goes in $CsvOut
[PSCustomObject] #{
Group = $Group
Members = $MemberList
}
}
$Timestamp = Get-Date -Format 'yyyy-MM-dd-HHmm' # this is a string
$CsvOutPath = "c:\ps1\shared_mailboxes\memberlist.$Timestamp.csv"
$CsvOut | Export-Csv $CsvOutPath -NoTypeInfo
I think you are over-complicating this.
If your desired result is a CSV file, then output objects straight away like below:
$DL_List = 'C:\ps1\shared_mailboxes\groups.csv'
$DL_array = (Import-Csv -Path $DL_List).name
$outFile = '.\group_members_{0:yyyy-MMM-dd-ddd hh-mm}.csv' -f (Get-Date)
$result = foreach ($DL in $DL_array) {
# output an object with the name of the DL and its members
[PsCustomObject]#{
Group = $DL
Users = (($DL | Get-DistributionGroupMember).displayName |
Where-Object {$_ -match '\S'}) -join ','
}
}
$result | Export-Csv -Path $outFile -NoTypeInformation
Using calculated properties, you can also do this without a foreach loop:
$DL_List = 'C:\ps1\shared_mailboxes\groups.csv'
$DL_array = (Import-Csv -Path $DL_List).name
$outFile = '.\group_members_{0:yyyy-MMM-dd-ddd hh-mm}.csv' -f (Get-Date)
$DL_array | Select-Object #{Name = 'Group'; Expresssion = {$_}},
#{Name = 'Users'; Expresssion = {
(($_ | Get-DistributionGroupMember).displayName |
Where-Object {$_ -match '\S'}) -join ','
}} |
Export-Csv -Path $outFile -NoTypeInformation
I'm trying to sort trough a recent report of PCs users don't seem to be using, for that ive got a CSV file called report, and a CVS file of all our PC's called data.
Report and Data only share 1 column which is the users full name, how can i get a result which provides both the users full name and the PC which is only from report?
So far i have the following code which works for getting the users full name, but I'm unsure how to get the device
$report = Import-Csv "C:\Temp\\report.CSV" -Delimiter ";"
$data = Import-Csv "C:\Temp\\data.CSV" -Delimiter ";"
$UserOutput = #()
ForEach ($name in $report)
{
$userMatch = $data | where {$_.FullName -like $name.FullName}
If($userMatch)
{
$UserOutput += New-Object PsObject -Property #{UserName =$name.FullName;Device=$userMatch.Device}
}
else
{
$UserOutput += New-Object PsObject -Property #{UserName =$name.FullName;Device ="NA"}
}
}
$UserOutput | ft
This gives a nice list, but i cant find the devices so it ends up looking like this
Device UserName
------ --------
NA Anders Aadal Jensen
NA Andr�s Kov�cs
NA Anette Dahnke
You can use a Group-Object -AsHashtable to correlate the FullName property on both arrays of objects. This assumes that the property values are an exact match when correlated.
$data = Import-Csv "C:\Temp\data.CSV" -Delimiter ";" |
Group-Object Device -AsHashTable -AsString
Import-Csv "C:\Temp\report.CSV" -Delimiter ";" | ForEach-Object {
$device = 'NA'
if($data.ContainsKey($_.FullName)) {
$device = $data[$_.FullName]
}
[pscustomobject]#{
UserName = $_.FullName
Device = $device
}
} | Format-Table
Assuming that the details provided by you available in both the csv file, here is a sample for you.
$CSV1 = Import-Csv "C:\Temp\\report.CSV" -Delimiter ";"
$CSV2 = Import-Csv "C:\Temp\\data.CSV" -Delimiter ";"
$Count = $CSV2.Count
$Results = For ($i = 0; $i -lt $Count; $i++) {
If ($CSV2[$i].FullName -eq $CSV1[$i].FullName) {
$Match = "Match"
}
Else {
$Match = "No Match found"
}
[PSCustomObject]#{
UserName = $CSV2[$i].FullName
Device = $CSV2[$i].Device
Results = $Match
}
}
$Results | Export-Csv -Path "C:\Temp\results.csv" -NoTypeInformation
The accepted answer looks to assume entry count on the CSVs are a one for one match. Since it has one loop comparing the same CSV array element numbers.
From your description, you mention Report.csv is a list of infrequently used PCs and the Data.csv is an "All PC" list. If your entry counts are not one-for-one, may need two loops like below.
$Results = 0..$($REPORT.Count - 1) | ForEach-Object {
$i = $_
$($DATA | ForEach-Object { if($_.Fullname -eq $REPORT[$i].Fullname) { <# DATA PSCUSOMOBJECT#> }})
}
Fairly new to PowerShell and having challenges with appending data in the first available row for each headered column in a csv file.
I would like to utilize foreach for each column's type of data that will be independent of another column's data. The column headers are $headers = "Scope", "Drawing", "Submittal", "Database", "Estimate", "Sequence" with a foreach to locate and append their individual items to each column. The current problem that is happening is that because each category/column with its respective foreach will add it on a separate row because the previous row already had data appended from another category's/column's data creating a diagonal appended data.
The reason that a separate foreach is being used is for each category/column is because the category's are looking for and filtering files independently for each category.
Below is what is happening in the CSV file:
| Scope | Drawing | Submittal | Database | Estimate | Sequence |
| ------| ------- |---------- |--------- |--------- |--------- |
| DATA01| empty | empty | empty | empty | empty |
| empty | DATA11 | empty | empty | empty | empty |
| empty | empty | DATA21 | empty | empty | empty |
| empty | empty | empty | DATA31 | empty | empty |
| empty | empty | empty | empty | DATA41 | empty |
| empty | empty | empty | empty | empty | DATA51 |
This is what would be the desired result be for the CSV file:
| Scope | Drawing | Submittal | Database | Estimate | Sequence |
| ------| ------- |---------- |--------- |--------- |--------- |
| DATA01| DATA11 | DATA21 | DATA31 | DATA41 | DATA51 |
Here is part of the code that is being worked on:
# Creates the CSV if it does not already exist
$headers = "Scope", "Mechanical Drawing", "Controls Submittal", "Database", "Estimate", "Sequence of Operations"
$psObject = New-Object psobject
foreach($header in $headers)
{
Add-Member -InputObject $psobject -MemberType noteproperty -Name $header -Value ""
}
$psObject | Export-Csv $CsvFile -NoTypeInformation
foreach ($file in $ScopeList)
{
$hash=#{
"Scope" = $file.Fullname
}
$NewItem = New-Object PSObject -Property $hash
Export-Csv $CsvFile -inputobject $NewItem -append -Force
}
foreach ($file in $DrawingList)
{
$hash=#{
"Drawing" = $file.Fullname
}
$NewItem = New-Object PSObject -Property $hash
Export-Csv $CsvFile -inputobject $NewItem -append -Force
}
foreach ($file in $SubtmittalList)
{
$hash=#{
"Submittal" = $file.Fullname
}
$NewItem = New-Object PSObject -Property $hash
Export-Csv $CsvFile -inputobject $NewItem -append -Force
}
foreach ($file in $DatabaseList)
{
$hash=#{
"Database" = $file.Fullname
}
$NewItem = New-Object PSObject -Property $hash
Export-Csv $CsvFile -inputobject $NewItem -append -Force
}
foreach ($file in $EstimateList)
{
$hash=#{
"Estimate" = $file.Fullname
}
$NewItem = New-Object PSObject -Property $hash
Export-Csv $CsvFile -inputobject $NewItem -append -Force
}
foreach ($file in $SequenceList)
{
$hash=#{
"Sequence" = $file.Fullname
}
$NewItem = New-Object PSObject -Property $hash
Export-Csv $CsvFile -inputobject $NewItem -append -Force
}
The PowerShell version being used is 5.1. Windows 10 OS.
Could someone help me understand how to append on the same row but a different column without erasing another column's existing row of data? Would this be something that could be done with splatting or looking at each variable ${named}List?
If I understand the question properly, you have 6 arrays of '$file' items that need to be combined into a CSV file.
Then instead of using 6 foreach loops, just use one indexed loop and create Objects from the various lists
If all lists have the same number of items:
$result = for ($i = 0; $i -lt $ScopeList.Count; $i++) {
[PsCustomObject]#{
Scope = $ScopeList[$i].FullName
Drawing = $DrawingList[$i].FullName
Submittal = $SubtmittalList[$i].FullName
Database = $DatabaseList[$i].FullName
Estimate = $EstimateList[$i].FullName
Sequence = $SequenceList[$i].FullName
}
}
# now save the result as CSV file
$result | Export-Csv -Path 'X:\Path\to\TheResult.csv' -NoTypeInformation
If the lists are not all of the same length, you need to do some extra work:
# get the maximum number of items of your lists
$maxItems = (($ScopeList, $DrawingList, $SubtmittalList, $DatabaseList, $EstimateList, $SequenceList) |
Measure-Object -Property Count -Maximum).Maximum
# loop over the maximum number of items and check each list
# if the index $i does not exceed the max number of items for that list
$result = for ($i = 0; $i -lt $maxItems; $i++) {
[PsCustomObject]#{
Scope = if ($i -lt $ScopeList.Count) {$ScopeList[$i].FullName} else { $null}
Drawing = if ($i -lt $DrawingList.Count) {$DrawingList[$i].FullName} else { $null}
Submittal = if ($i -lt $SubtmittalList.Count) {$SubtmittalList[$i].FullName} else { $null}
Database = if ($i -lt $DatabaseList.Count) {$DatabaseList[$i].FullName} else { $null}
Estimate = if ($i -lt $EstimateList.Count) {$EstimateList[$i].FullName} else { $null}
Sequence = if ($i -lt $SequenceList.Count) {$SequenceList[$i].FullName} else { $null}
}
}
# now save the result as CSV file
$result | Export-Csv -Path 'X:\Path\to\TheResult.csv' -NoTypeInformation
Like Theo posted, this mostly worked or brought the intentions closer to the goal.
within each foreach loop that created their independent lists, another list was created for filtering the list and used in the below section from Theo.
# loop over the maximum number of items and check each list
# if the index $i does not exceed the max number of items for that list
$result = for ($i = 0; $i -lt $maxItems; $i++) {
[PsCustomObject]#{
Scope = if ($i -lt $ScopeList.Count) {$ScopeList[$i].FullName} else { $null}
Drawing = if ($i -lt $DrawingList.Count) {$DrawingList[$i].FullName} else { $null}
Submittal = if ($i -lt $SubtmittalList.Count) {$SubtmittalList[$i].FullName} else { $null}
Database = if ($i -lt $DatabaseList.Count) {$DatabaseList[$i].FullName} else { $null}
Estimate = if ($i -lt $EstimateList.Count) {$EstimateList[$i].FullName} else { $null}
Sequence = if ($i -lt $SequenceList.Count) {$SequenceList[$i].FullName} else { $null}
}
}
# now save the result as CSV file
$result | Export-Csv -Path 'X:\Path\to\TheResult.csv' -NoTypeInformation
To get the $maxItems, an alternative way was to have a default value of 1 unless a user input has a keyword "list" when they are prompted for other info, which is another challenge on how to limit values to greater than or equal to 1...
I am currently using the code below that I found on here to concat 2 columns into a new column. The problem is that when I export the csv, the columns are not in the order that I list below. How can I sort the columns into the order I want?
Thanks
$objs =#();
$output = Import-csv -Path C:\Users\Dan\Desktop\External_ID_Test.csv | ForEach {
$Object = New-Object PSObject -Property #{
Employee_ID = $_.Employye_ID
Base_Cost_Center = $_.Base_Cost_Center
Entity = $_.Entity
ExternalID = [String]::Concat($_.Entity,$_.Base_Cost_Center)
}
$objs += $Object;
}
$objs
$objs | Export-CSv C:\Users\Dan\Desktop\Export_External_ID_Test.csv -NoTypeInformation
Your are assigning the output of the ForEach-Object to the variable $Output but aren't outputting anything.
Simplify your script by using an ordered [PSCustomObject] and avoiding the += which will rebuild the array $objs on every iteration.
$output = Import-csv -Path C:\Users\Dan\Desktop\External_ID_Test.csv | ForEach-Object {
[PSCustomObject]{
Employee_ID = $_.Employye_ID
Base_Cost_Center = $_.Base_Cost_Center
Entity = $_.Entity
ExternalID = [String]::Concat($_.Entity,$_.Base_Cost_Center)
}
}
$output
$output | Export-CSv C:\Users\Dan\Desktop\Export_External_ID_Test.csv -NoTypeInformation
Changed my code a little from part 2 example below but taking part 2 example which works great thanks to a member in stackoverflow.
[string] $Source = 'e:\Temp\DataFromSkywardEdited.csv';
[string] $Destination = 'e:\Temp\DataFromSkywardRD.csv';
[object] $SortByColNames = 'LastName','FirstName';
[string] $ShowColsByNumber = "{0},{1},{2},{3},{4},{5},{6},{7},{8}";
[object] $ColumnValueFormat = '$_.EmployeeID.Trim()', '$_.FirstName.Trim()', '$_.LastName.Trim()', '$_.Location.Trim()', '$_.Department.Trim()', '$_.TelephoneNo.Trim()', '$_.Email.Trim()', '$_.EmpTypeCode.Trim()', '$_.EmployeeTypeDescription.Trim()';
Get-Content $Source |
ConvertFrom-Csv -Delimiter $Delimiter |
Sort-Object -Property $SortByColNames -Unique |
ForEach-Object {
# Each of the values in $ColumnValueFormat must be executed to get the property from the loop variable ($_).
$values = foreach ($value in $ColumnValueFormat) {
Invoke-Expression $value
}
# Then the values can be passed in as an argument for the format operator.
$ShowColsByNumber -f $values
} |
Add-Content $Destination
How would I make $ColumnValueFormat to be represented by numbers instead of a column name?
So instead of:
[object] $ColumnValueFormat = '$_.EmployeeID.Trim()', '$_.FirstName.Trim()', '$_.LastName.Trim()', '$_.Location.Trim()', '$_.Department.Trim()', '$_.TelephoneNo.Trim()', '$_.Email.Trim()', '$_.EmpTypeCode.Trim()', '$_.EmployeeTypeDescription.Trim()';
To something like:
[object] $ColumnValueFormat = '$_.[0].Trim()', '$_.[1].Trim()', '$_.[2].Trim()', '$_.[3].Trim()', '$_.[4].Trim()', '$_.[5].Trim()', '$_.[6].Trim()', '$_.[7].Trim()', '$_.[8].Trim()';
This really looks like you're doing it the hard way, but to directly answer your question, you would simply remove the period that preceeds the bracketed numbers. So...
[object] $ColumnValueFormat = '$_[0].Trim()', '$_[1].Trim()', '$_[2].Trim()', '$_[3].Trim()', '$_[4].Trim()', '$_[5].Trim()', '$_[6].Trim()', '$_.[7]Trim()', '$_[8].Trim()';
How would I make $ColumnValueFormat to be represented by numbers instead of a column name?
You wouldn't. Just … no.
To trim the fields of a CSV you'd simply do something like this:
$csv = Import-Csv $Source
foreach ($record in $csv) {
foreach ($property in $record.PSObject.Properties) {
$property.Value = $property.Value.Trim()
}
}
$csv | Sort-Object $SortByColNames -Unique | Export-Csv $Destination
If you had to treat different fields in different ways you'd use a switch statement to distinguish between the values:
$property.Value = switch ($property.Name) {
'foo' { ... }
'bar' { ... }
...
}
modify the fields via calculated properties:
Import-Csv $Source |
Select-Object #{n='EmployeeID';e={$_.EmployeeID.Trim()},
#{n='FirstName';e={...},
... |
Sort-Object $SortByColNames -Unique |
Export-Csv $Destination
or re-create the records via New-Object:
Import-Csv $Source |
ForEach-Object {
New-Object -Type PSObject -Property #{
'EmployeeID' = $_.EmployeeID.Trim()
'FirstName' = ...
...
}
} |
Sort-Object $SortByColNames -Unique |
Export-Csv $Destination
In PowerShell v3 and newer you can even use the type accelerator [ordered] to have properties created in a particular order, like this:
$props = [ordered]#{
'EmployeeID' = $_.EmployeeID.Trim()
'FirstName' = ...
...
}
New-Object -Type PSObject -Property $props
or like this:
[PSObject][ordered]#{
'EmployeeID' = $_.EmployeeID.Trim()
'FirstName' = ...
...
}