Don't quite understand powershells "-Join" and "ConvertTo-Csv" - powershell

I have this code to create a csv and display it as a table:
$Keys = ("OrderDate","Region","Rep","Product","Units","Unit Cost","Total")
$Csv = #()
$Keys | ForEach-Object {
$Csv += $_ -Join ","
}
$Csv | ConvertFrom-Csv
My output is:
OrderDate
OrderDate,Region
OrderDate,Region,Rep
OrderDate,Region,Rep,Product
OrderDate,Region,Rep,Product,Units
OrderDate,Region,Rep,Product,Units,Unit Cost
OrderDate,Region,Rep,Product,Units,Unit Cost,Total
OrderDate
---------
Region
Rep
Product
Units
Unit Cost
Total
I would like these lines to be formatted as headers of a table, like this:
OrderDate Region Rep ProductUnits Unit Cost Total
----------- -------- ---- --------------- ----------- ------

What you're attempting can be done like this:
$Keys = "OrderDate","Region","Rep","Product","Units","Unit Cost","Total"
$Object = Select-Object -InputObject 0 -Property $Keys
That's typically most useful when you don't know what the properties are when writing the script. The input object doesn't matter as long as it's a scalar value (i.e., not a collection).
To get the exact output you've specified, you'd have to do $Object | Format-Table because PowerShell defaults to list output for custom objects, but that changes the object into a string so it's just for display.
A more generally useful pattern to create an object when you do know the properties is to do this:
$Object = [PSCustomObject]#{
"OrderDate" = # Some Value
"Region" = # Some Value
"Rep" = # Some Value
"Product" = # Some Value
"Units" = # Some Value
"Unit Cost" = # Some Value
"Total" = # Some Value
}
Here you instance the object and assign the values you want immediately.

Related

Convert multiple txt file to one single csv file [duplicate]

When we're trying to export data to other functions via the pipeline, we observe some strange behavior in PowerShell.
Example code:
$Array = #()
$Obj1 = [PSCustomObject]#{
Member1 = 'First'
Member2 = 'Second'
}
$Obj2 = [PSCustomObject]#{
Member1 = 'First'
Member2 = 'Second'
Member3 = 'Third'
}
$Array = $Obj1, $Obj2
$Array | Out-GridView -Title 'Not showing Member3'
$Array = $Obj2, $Obj1
$Array | Out-GridView -Title 'All members correctly displayed'
In the example above you can see that when the first object only contains 2 properties, the Out-GridView CmdLet (and others) only show 2 properties, even though the second object has 3 properties. However, when the first object in the array has 3 properties it does display them all correctly.
Is there a way around this? Because it's not possible to predict up front how many properties on an object there will be and if the object with the most properties will be the first one in the array.
I had the same experience once and created the following reusable 'Union' function:
# 2021-08-25 Removed Union function
Usage:
$Obj1, $Obj2 | Union | Out-GridView -Title 'Showing all members'
It is also supposed to work with complex objects. Some standard cmdlets output multiple object types at once and if you view them (e.g. Out-GridView) or dump them in a file (e.g. Export-Csv) you might miss a lot of properties. Take as another example:
Get-WmiObject -Namespace root/hp/instrumentedBIOS -Class hp_biosSetting | Union | Export-Csv ".\HPBIOS.csv"
Added 2014-09-19:
Maybe this is already between the lines in the comments $Array | Select * | … will not resolve the issue but specifically selecting the properties $Array | Select Member1, Member2, Member3 | … does.
Besides, although in most cases the Union function will work, there are some exceptions to that as it will only align the first object with the rest.
Consider the following object:
$List = #(
New-Object PSObject -Property #{Id = 2}
New-Object PSObject -Property #{Id = 1}
New-Object PSObject -Property #{Id = 3; Name = "Test"}
)
If you Union this object everything appears to be fine and if you e.g. ExportTo-CSV and work with the export .csv file from then on you will never have any issue.
$List | Union
Id Name
-- ----
2
1
3 Test
Still there is a catch as only the first object is aligned. If you e.g. sort the result on Id (Sort Id) or take just the last 2 (Select -Last 2) entries, the Name is not listed because the second object doesn’t contain the Name property:
$List | Union | Sort Id
Id
--
1
2
3
Therefor I have rewritten the Union-Object (Alias Union) function`):
Union-Object
# 2021-08-25 Removed Union-Object function
Syntax:
$Array | Union | Out-GridView -Title 'All members correctly displayed'
Update 2021-08-25
Based on az1d helpful feedback on an error caused by equal property names with different casing, I have created a new UnifyProperties function.
(I will no longer use the name UnionObject for his)
function UnifyProperties {
$Names = [System.Collections.Generic.HashSet[string]]::new([StringComparer]::OrdinalIgnoreCase)
$InputCollected = #($Input)
$InputCollected.ForEach({
foreach ($Name in $_.psobject.Properties.Name) { $Null = $Names.Add($Name) }
})
$inputCollected | Select-Object #($Names)
}
Usage:
[pscustomobject] #{ one = 1; two = 2; three = 3 },
[pscustomobject] #{ ONE = 10; THREE = 30; FOUR = 4 } |
UnifyProperties
one two three FOUR
--- --- ----- ----
1 2 3
10 30 4
See also: #13906 Add -UnifyProperties parameter to Select-Object

Changing Data in Columns in a CSV

I have a PowerShell script pulling data in from a CSV. What I am trying to do is "replace" the data in the Account column based on the value. For example, Account 001 = Hardware, Account 002 = Software, etc. The data in the CSV is being pulled from a SQL database so if it would be easier for me to change it in the SQL script, I can do that easily. The Account column in the CSV has 001, 002, etc. I want to change those values to Hardware, Software, etc. Thanks for the help.
$Results = import-csv Expenses.csv
$Array = #()
Foreach($R in $Results)
{
$Object = [pscustomobject][ordered] #{
Account = $R.Account
Vendor = $R.Desc1
Item = $R.Desc2
Amount = $R.Amount
}
$Array += $Object
}
$Array
If your CSV looks anything like this:
Account,Vendor,Item,Amount
001,Some Vendor,Something expensive, 1
002,Another Vendor,Something cheapish,26
you can update without a loop:
# create a lookup hashtable where you combine the account values with the wanted replacement
$lookup = #{
'001' = 'Hardware'
'002' = 'Software'
# etcetera
}
# import the csv and update the `Account` column
$Results = Import-Csv D:\Test\Expenses.csv | Select-Object #{Name = 'Account'; Expression = {$lookup[$_.Account]}}, * -ExcludeProperty Account
# display on screen
$Results
# output to (new) csv file
$Results | Export-Csv -Path D:\Test\Expenses_Updated.csv -NoTypeInformation
Result:
Account Vendor Item Amount
------- ------ ---- ------
Hardware Some Vendor Something expensive 1
Software Another Vendor Something cheapish 26
As per the comment of not2qubit some explanation about the Select-Object statement used.
Because the result should reflect all fields in the csv, where the existing field value named Account needs to be replaced, the code uses a Calculated property to set the Account field values using whatever was stored in the lookup Hashtable.
This is done with #{Name = 'Account'; Expression = {$lookup[$_.Account]}}
Next, all other fields contained in the csv are selected unchanged using the asteriks *.
Because we're overwriting the Accound field, but keep its name, the line ends with -ExcludeProperty Account in order to remove the original Account field in the output.
If we don't do that, PowerShell will show an error: Select-Object : The property cannot be processed because the property "Account" already exists.
If I have understood what you require correctly, you just want to change "001" to "Hardware" and so on in the object imported by the Import-Csv cmdlet. You can create a ScriptBlock with a switch that will return a value based off the value you have searched for. I could have recommended a Hashtable here too, but the benefit of a switch over a Hashtable, in this case, is that you can return the value using the default option if it is not specified. For example:
$Lookup = {
Param ([string]$Value)
switch ($Value) {
"001" { "Hardware" }
"002" { "Software" }
default { $Value }
}
}
$Results = Import-Csv Expenses.csv
foreach($R in $Results)
{
# Invoke the scriptblock with the named parameter.
$R.Account = & $Lookup -Value $R.Account
}
# Do stuff with $Results

Powershell Compare-object IF different then ONLY list items from one file, not both

I have deleted my original question because I believe I have a more efficient way to run my script, thus I'm changing my question.
$scrubFileOneDelim = "|"
$scrubFileTwoDelim = "|"
$scrubFileOneBal = 2
$scrubFileTwoBal = 56
$scrubFileOneAcctNum = 0
$scrubFileTwoAcctNum = 0
$ColumnsF1 = Get-Content $scrubFileOne | ForEach-Object{($_.split($scrubFileOneDelim)).Count} | Measure-Object -Maximum | Select-Object -ExpandProperty Maximum
$ColumnsF2 = Get-Content $scrubFileTwo | ForEach-Object{($_.split($scrubFileTwoDelim)).Count} | Measure-Object -Maximum | Select-Object -ExpandProperty Maximum
$useColumnsF1 = $ColumnsF1-1;
$useColumnsF2 = $ColumnsF2-1;
$fileOne = import-csv "$scrubFileOne" -Delimiter "$scrubFileOneDelim" -Header (0..$useColumnsF1) | select -Property #{label="BALANCE";expression={$($_.$scrubFileOneBal)}},#{label="ACCTNUM";expression={$($_.$scrubFileOneAcctNum)}}
$fileTwo = import-csv "$scrubFileTwo" -Delimiter "$scrubFileTwoDelim" -Header (0..$useColumnsF2) | select -Property #{label="BALANCE";expression={$($_.$scrubFileTwoBal)}},#{label="ACCTNUM";expression={$($_.$scrubFileTwoAcctNum)}}
$hash = #{}
$hashTwo = #{}
$fileOne | foreach { $hash.add($_.ACCTNUM, $_.BALANCE) }
$fileTwo | foreach { $hashTwo.add($_.ACCTNUM, $_.BALANCE) }
In this script I'm doing the following, counting header's to return the count and use it in a range operator in order to dynamically insert headers for later manipulation. Then I'm importing 2 CSV files. I'm taking those CSV files and pushing them into their own hashtable.
Just for an idea of what I'm trying to do from here...
CSV1 (as a hashtable) looks like this:
Name Value
---- -----
000000000001 000000285+
000000000002 000031000+
000000000003 000004685+
000000000004 000025877+
000000000005 000000001+
000000000006 000031000+
000000000007 000018137+
000000000008 000000000+
CSV2 (as a hashtable) looks like this:
Name Value
---- -----
000000000001 000008411+
000000000003 000018137+
000000000007 000042865+
000000000008 000009761+
I would like to create a third hash table. It will have all the "NAME" items from CSV2, but I don't want the "VALUE" from CSV2, I want it to have the "VALUE"s that CSV1 has. So in the end result would look like this.
Name Value
---- -----
000000000001 000000285+
000000000003 000004685+
000000000007 000018137+
000000000008 000000000+
Ultimately I want this to be exported as a csv.
I have tried this with just doing a compare-object, not doing the hashtables with the following code, but I abandoned trying to do it this way because file 1 may have 100,000 "accounts" where file 2 only has 200, and the result I was getting listed close to the 100,000 accounts that I didn't want to be in the result. They had the right balances but I want a file that only has those balances for the accounts listed in file 2. This code below isn't really a part of my question, just showing something I've tried. I just think this is much easier and faster with a hash table now so I would like to go that route.
#Find and Rename the BALANCE and ACCOUNT NUMBER columns in both files.
$fileOne = import-csv "$scrubFileOne" -Delimiter "$scrubFileOneDelim" -Header (0..$useColumnsF1) | select -Property #{label="BALANCE";expression={$($_.$scrubFileOneBal)}},#{label="ACCT-NUM";expression={$($_.$scrubFileOneAcctNum)}}
$fileTwo = import-csv "$scrubFileTwo" -Delimiter "$scrubFileTwoDelim" -Header (0..$useColumnsF2) | select -Property #{label="BALANCE";expression={$($_.$scrubFileTwoBal)}},#{label="ACCT-NUM";expression={$($_.$scrubFileTwoAcctNum)}}
Compare-Object $fileOne $fileTwo -Property 'BALANCE','ACCTNUM' -IncludeEqual -PassThru | Where-Object{$_.sideIndicator -eq "<="} | select * -Exclude SideIndicator | export-csv -notype "C:\test\f1.txt"
What you are after is filtering the Compare-Object function. This will show only one side of the result. YOu will need to place this before you exclude that property for it to work.
| Where-Object{$_.sideIndicator -eq "<="} |
Assuming that you have the following hash tables:
$hash = #{
'000000000001' = '000000285+';
'000000000002' = '000031000+';
'000000000003' = '000004685+';
'000000000004' = '000025877+';
'000000000005' = '000000001+';
'000000000006' = '000031000+';
'000000000007' = '000018137+';
'000000000008' = '000000000+';
}
$hashTwo = #{
'000000000001' = '000008411+';
'000000000003' = '000018137+';
'000000000007' = '000042865+';
'000000000008' = '000009761+';
}
you can create the third hash table by iterating over the keys from the second hash table and then assigning those keys to the value from the first hash table.
$hashThree = #{}
ForEach ($key In $hashTwo.Keys) {
$hashThree["$key"] = $hash["$key"]
}
$hashThree
The output of $hashThree is:
Name Value
---- -----
000000000007 000018137+
000000000001 000000285+
000000000008 000000000+
000000000003 000004685+
If you want the order of the data maintained (and you are using PowerShell 6 Core), you can use [ordered]#{} when creating the hash tables.

re-arrange and combine powershell custom objects

I have a system that currently reads data from a CSV file produced by a separate system that is going to be replaced.
The imported CSV file looks like this
PS> Import-Csv .\SalesValues.csv
Sale Values AA BB
----------- -- --
10 6 5
5 3 4
3 1 9
To replace this process I hope to produce an object that looks identical to the CSV above, but I do not want to continue to use a CSV file.
I already have a script that reads data in from our database and extracts the data that I need to use. I'll not detail the fairly long script that preceeds this point but in effect it looks like this:
$SQLData = Custom-SQLFunction "SELECT * FROM SALES_DATA WHERE LIST_ID = $LISTID"
$SQLData will contain ~5000+ DataRow objects that I need to query.
One of those DataRow object looks something like this:
lead_id : 123456789
entry_date : 26/10/2018 16:51:16
modify_date : 01/11/2018 01:00:02
status : WRONG
user : mrexample
vendor_lead_code : TH1S15L0NGC0D3
source_id : A543212
list_id : 333004
list_name : AA Some Text
gmt_offset_now : 0.00
SaleValue : 10
list_name is going to be prefixed with AA or BB.
SaleValue can be any integer 3 and up, however realistically extremely unlikely to be higher than 100 (as this is a monthly donation) and will be one of 3,5,10 in the vast majority of occurrences.
I already have script that takes the content of list_name, creates and populates the data I need to use into two separate psobjects ($AASalesValues and $BBSalesValues) that collates the total numbers of 'SaleValue' across the data set.
Because I cannot reliably anticipate the value of any SaleValue I have to dynamically create the psobjects properties like this
foreach ($record in $SQLData) {
if ($record.list_name -match "BB") {
if ($record.SaleValue -gt 0) {
if ($BBSalesValues | Get-Member -Name $($record.SaleValue) -MemberType Properties) {
$BBSalesValues.$($record.SaleValue) = $BBSalesValues.$($record.SaleValue)+1
} else {
$BBSalesValues | Add-Member -Name $($record.SaleValue) -MemberType NoteProperty -Value 1
}
}
}
}
The two resultant objects look like this:
PS> $AASalesValues
10 5 3 50
-- - - --
17 14 3 1
PS> $BBSalesvalues
3 10 5 4
- -- - -
36 12 11 1
I now have the data that I need, however I need to format it in a way that replicates the format of the CSV so I can pass it directly to another existing powershell script that is configured to expect the data in the format that the CSV is in, but I do not want to write the data to a file.
I'd prefer to pass this directly to the next part of the script.
Ultimately what I want to do is to produce a new object/some output that looks like the output from Import-Csv command at the top of this post.
I'd like a new object, say $OverallSalesValues, to look like this:
PS>$overallSalesValues
Sale Values AA BB
50 1 0
10 17 12
5 14 11
4 0 1
3 3 36
In the above example the values from $AASalesValues is listed under the AA column, the values from $BBSalesValues is listed under the BB column, with the rows matching the headers of the two original objects.
I did try this with hashtables but I was unable to work out how to both create them from dynamic values and format them to how I needed them to look.
Finally got there.
$TotalList = #()
foreach($n in 3..200){
if($AASalesValues.$n -or $BBSalesValues.$n){
$AACount = $AASalesValues.$n
$BBcount = $BBSalesValues.$n
$values = [PSCustomObject]#{
'Sale Value'= $n
AA = $AACount
BB = $BBcount
}
$TotalList += $values
}
}
$TotalList
produces an output of
Sale Value AA BB
---------- -- --
3 3 36
4 2
5 14 11
10 18 12
50 1
Just need to add a bit to include '0' values instead of $null.
I'm going to assume that $record contains a list of the database results for either $AASalesValues or $BBSalesValues, not both, otherwise you'd need some kind of selector to avoid counting records of one group with the other group.
Group the records by their SaleValue property as LotPings suggested:
$BBSalesValues = $record | Group-Object SaleValue -NoElement
That will give you a list of the SaleValue values with their respective count.
PS> $BBSalesValues
Count Name
----- ----
36 3
12 10
11 5
1 4
You can then update your CSV data with these values like this:
$file = 'C:\path\to\data.csv'
# read CSV into a hashtable mapping the sale value to the complete record
# (so that we can lookup the record by sale value)
$csv = #{}
Import-Csv $file | ForEach-Object {
$csv[$_.'Sale Values'] = $_
}
# Add records for missing sale values
$($AASalesValues; $BBSalesValues) | Select-Object -Expand Name -Unique | ForEach-Object {
if (-not $csv.ContainsKey($_)) {
$csv[$_] = New-Object -Type PSObject -Property #{
'Sale Values' = $_
'AA' = 0
'BB' = 0
}
}
}
# update records with values from $AASalesValues
$AASalesValues | ForEach-Object {
[int]$csv[$_.Name].AA += $_.Count
}
# update records with values from $BBSalesValues
$BBSalesValues | ForEach-Object {
[int]$csv[$_.Name].BB += $_.Count
}
# write updated records back to file
$csv.Values | Export-Csv $file -NoType
Even with your updated question the approach would be pretty much the same, you'd just add another level of grouping for collecting the sales numbers:
$sales = #{}
$record | Group-Object {$_.list_name.Split()[0]} | ForEach-Object {
$sales[$_.Name] = $_.Group | Group-Object SaleValue -NoElement
}
and then adjust the merging to something like this:
$file = 'C:\path\to\data.csv'
# read CSV into a hashtable mapping the sale value to the complete record
# (so that we can lookup the record by sale value)
$csv = #{}
Import-Csv $file | ForEach-Object {
$csv[$_.'Sale Values'] = $_
}
# Add records for missing sale values
$sales.Values | Select-Object -Expand Name -Unique | ForEach-Object {
if (-not $csv.ContainsKey($_)) {
$prop = #{'Sale Values' = $_}
$sales.Keys | ForEach-Object {
$prop[$_] = 0
}
$csv[$_] = New-Object -Type PSObject -Property $prop
}
}
# update records with values from $sales
$sales.GetEnumerator() | ForEach-Object {
$name = $_.Key
$_.Value | ForEach-Object {
[int]$csv[$_.Name].$name += $_.Count
}
}
# write updated records back to file
$csv.Values | Export-Csv $file -NoType

Find matches in two different Powershell objects based on one property

I am trying to find the matching names in two different types of Powershell objects
$Object1 has two properties - Name (string), ResourceID (uint32)
$object2 has one noteproperty - Name (system.string)
This gives me a list of the matching names but I also want the corresponding resourceID property from $object1.
$computers = Compare-Object $Object1.name $WSD_CM12 | where {$_.sideindicator -eq "=>"} | foreach {$_.inputobject}
These are big objects with over 10,000 items so I'm looking for the most efficient way to accomplish this.
If I'm understanding what you're after, I'd start by creating a hash table from your Object1 collection:
$object1_hash = #{}
Foreach ($object1 in $object1_coll)
{ $object1_hash[$object1.Name] = $object1.ResourceID }
Then you can find the ResourceID for any given Object2.name with:
$object1_hash[$Object2.Name]
Test bed for creating hash table:
$object1_coll = $(
New-Object PSObject -Property #{Name = 'Name1';ResourceID = 001}
New-Object PSObject -Property #{Name = 'Name2';ResourceID = 002}
)
$object1_hash = #{}
Foreach ($object1 in $object1_coll)
{ $object1_hash[$object1.Name] = $object1.ResourceID }
$object1_hash
Name Value
---- -----
Name2 2
Name1 1
Alternative method:
# Create sample list of objects with both Name and Serial
$obj1 = New-Object -Type PSCustomObject -Property:#{ Name = "Foo"; Serial = "1234" }
$obj2 = New-Object -Type PSCustomObject -Property:#{ Name = "Cow"; Serial = "4242" }
$collection1 = #($obj1, $obj2)
# Create subset of items with only Name
$objA = New-Object -Type PSCustomObject -Property:#{ Name = "Foo"; }
$collection2 = #($objA)
#Everything above this line is just to make sample data
# replace $collection1 and $collection2 with $Object1, $WSD_CM12
# Combine into one list
($collection1 + $collection2) |
# Group by name property
Group-Object -Property Name |
# I only want items that exist in both
Where { $_.Count -gt 1 } |
# Now give me the object
Select -Expand Group |
# And get the properties
Where { $_.Serial -ne $null }