I am not looking for a writing service but please can someone point me in the right direction as I am completely at a lost as to how to proceed.
Overview
I have a CSV which contains a lot of data, some of which comes from a script and some in manually imputed. I can run the script and get new data which is good. What I would like to do is find a way to compare the orginal CSV 1 to the new CSV 2 and update CSV 1.
Code I currently have
$Vips_to_check = #{}
Import-Csv 'C:\Users\user\Documents\20221201\Netscaler VIPs per Cluster_edited - Raw Data.csv' |
Where-Object {$_.PRD -match "No PRD code from VIP IP and VIP has no backend IPs" -or
$_.PRD -match "No PRD code found from VIP or backend IPs" -or
$_.PRD -match "No PRD code found from backend IPs" -and
$_.ipv46 -notcontains "0.0.0.0"} |
$Results_from_PIM = Import-Csv 'C:\Users\user\Documents\20221201\VIP-Owners_edited.csv'
Both of the CSV's have the same headers and layout which is good. I assume!
CSV 1
Name IPV46 Port Curstate Suggested PRD Display Name tech Owner Slack Channel Support Email
name 1 1.2.3.4 8080 Down No No No No No No No
CSV 2
Name IPV46 Port Curstate Suggested PRD Display Name tech Owner Slack Channel Support Email
name 1 1.2.3.4 8080 Down No PRD123 TMOL Gary TMOL Support Support#email.com nsr.sys
I would guess at creating a hashtable but I just can't seem to get my head around the format of them. I tried
$ht = $Results_from_pim #{}
$_.Name = (cant figure out how to reference the cell)
$_.PRD =
$_.("Display Name")
$_.("Tech Owner")
Once I have the data in the hash table how do I overwrite the CSV 1 data?
Any points or guides would be great. I have tried reading up on https://learn.microsoft.com/en-gb/powershell/scripting/learn/deep-dives/everything-about-hashtable?view=powershell-7.3 and https://learn.microsoft.com/en-us/powershell/scripting/learn/deep-dives/everything-about-pscustomobject?view=powershell-7.3
But that left me even more confused.
At the moment the difference is only 4 or 5 entries and it would of been quicker for me to manually edit in excel but as this script gets larger I can see it being more time consuming to do manually.
As always thank you.
UPDATE
$ht = #{}
foreach ($item in $Results_from_PIM) {
"name = $($item.name)"
"prd = $($item.PRD)"
"Display Name = $($item.'Display Name')"
"Tech Owner = $($item.'Tech Owner')"
"Slack Channel = $($item.'Slack Channel')"
"Support Email = $($Item.'Support Email')"
}
I have created the hash table that I wanted from the CSV 2. Just got to get it to compare to CSV 1.
Update 2
Further to #theo request I have adjusted the question. Also to clarify When I want to merge the CSV it is based on matching the Name, IPV46 and Port on both CSV and then moving the updated data from CSV2 into CSV1.
You can do that with the code below (no extra module needed):
$csv1 = 'C:\Users\user\Documents\20221201\Netscaler VIPs per Cluster_edited - Raw Data.csv'
$csv2 = 'C:\Users\user\Documents\20221201\VIP-Owners_edited.csv'
$Results_from_PIM = Import-Csv -Path $csv2
$newData = Import-Csv -Path $csv1 | ForEach-Object {
$search = $_.Name + $_.IPV46 + $_.Port # combine these fields into a single string
$compare = $Results_from_PIM | Where-Object { ($_.Name + $_.IPV46 + $_.Port) -eq $search }
if ($compare) {
# output the result from csv2
$compare
}
else {
# output the original row from csv1
$_
}
}
# now you can save the updated data to a new file or overwrite csv1 if you like
$csv3 = 'C:\Users\user\Documents\20221201\VIP-Owners_Updated.csv'
$newData | Export-Csv -Path $csv3 -NoTypeInformation
P.S. Please read about Formatting
After being direct to In PowerShell, what's the best way to join two tables into one? by #jdweng. I performed the following which seems to have meet my requirements
Install-Module -Name JoinModule -Scope CurrentUser
$Vips_to_check = Import-Csv 'C:\Users\user\Documents\20221201\Netscaler VIPs per Cluster - Raw Data.csv'
$Results_from_PIM = Import-Csv 'C:\Users\user\Documents\20221201\VIP-Owners.csv'
$Vips_to_check | Update-Object $Results_from_PIM -On name, Ipv46, port | Export-Csv 'C:\Users\user\Documents\20221201\Final_data1.csv'
Going to do further testing with larger data sets but appears to work as required.
Related
I have two files. One is called allper.csv
institutiongroup,studentid,iscomplete
institutionId=22343,123,FALSE
institutionId=22343,456,FALSE
institutionId=22343,789,FALSE
The other one is called actswithpersons.csv
abc,123;456
def,456
ghi,123
jkl,123;456
Note: The actswithpersons.csv does not have headers - they are going to be added in later via an excel power query so don't want them in there now. The actswithpersons csv columns are delimited with commas - there are only two columns, and the second one contains multiple personids - again Excel will deal with this later.
I want to remove all rows from allper.csv where the personid doesn't appear in actswithpersons.csv, and export them to another csv. So in the desired outcome, allper.csv would look like this
institutiongroup,studentid,iscomplete
institutionId=22343,123,FALSE
institutionId=22343,456,FALSE
and the export.csv would look like this
institutiongroup,studentid,iscomplete
institutionId=22343,789,FALSE
I've got as far as the below, which will put into the shell whether the personid is found in the actswithpersons.csv file.
$donestuff = (Get-Content .\ActsWithpersons.csv | ConvertFrom-Csv); $ids=(Import-Csv .\allper.csv);foreach($id in $ids.personid) {echo $id;if($donestuff -like "*$id*" )
{
echo 'Contains String'
}
else
{
echo 'Does not contain String'
}}
However, I'm not sure how to go the last step, and export & remove the unwanted rows from allper.csv
I've tried (among many things)
$donestuff = (Get-Content .\ActsWithpersons.csv | ConvertFrom-Csv);
Import-Csv .\allper.csv |
Where-Object {$donestuff -notlike $_.personid} |
Export-Csv -Path export.csv -NoTypeInformation
This took a really long time and left me with an empty csv. So, if you can give any guidance, please help.
Since your actswithpersons.csv doesn't have headers, in order for you to import as csv, you can specify the -Header parameter in either Import-Csv or ConvertFrom-Csv; with the former cmdlet being the better solution.
With that said, you can use any header name for those 2 columns then filter by the given column name (ID in this case) after your import of allper.csv using Where-Object:
$awp = (Import-Csv -Path '.\actswithpersons.csv' -Header 'blah','ID').ID.Split(';')
Import-Csv -Path '.\allper.csv' | Where-Object -Property 'Studentid' -notin $awp
This should give you:
institutiongroup studentid iscomplete
---------------- --------- ----------
institutionId=22343 789 FALSE
If you're looking to do it with Get-Content you can split by the delimiters of , and ;. This should give you just a single row of values which you can then compare the entirety of variable ($awp) using the same filter as above which will give you the same results:
$awp = (Get-Content -Path '.\actswithpersons.csv') -split ",|;"
Import-Csv -Path '.\allper.csv' | Where-Object -Property 'Studentid' -notin $awp
I've got this one that has been baffling me all day, and I can't seem to find any search results that match exactly what I am trying to do.
I have 2 CSV files, both of which have the same columns and headers. They look like this (shortened for the purpose of this post):
"plate","labid","well"
"1013740016604537004556","none46","F006"
"1013740016604537004556","none47","G006"
"1013740016604537004556","none48","H006"
"1013740016604537004556","3835265","A007"
"1013740016604537004556","3835269","B007"
"1013740016604537004556","3835271","C007"
Each of the 2 CSVs only have some actual Lab IDs, and the 'nonexx' are just fillers for the importing software. There is no duplication ie each 'well' is only referenced once across the 2 files.
What I need to do is merge the 2 CSVs, for example the second CSV might have a Lab ID for well H006 but the first will not. I need the lab ID from the second CSV imported into the first, overwriting the 'nonexx' currently in that column.
Here is my current code:
$CSVB = Import-CSV "$RootDir\SymphonyOutputPending\$plateID`A_Header.csv"
Import-CSV "$RootDir\SymphonyOutputPending\$plateID`_Header.csv" | ForEach-Object {
$CSVData = [PSCustomObject]#{
labid = $_.labid
well = $_.well
}
If ($CSVB.well -match $CSVData.wellID) {
write-host "I MATCH"
($CSVB | Where-Object {$_.well -eq $CSVData.well}).labid = $CSVData.labid
}
$CSVB | Export-CSV "$RootDir\SymphonyOutputPending\$plateID`_final.csv" -NoTypeInformation
}
The code runs but doesn't 'merge' the data, the final CSV output is just a replication of the first input file. I am definitely getting a match as the string "I MATCH" appears several times when debugging as expected.
Based on the responses in the comments of your question, I believe this is what you are looking for. This assumes that the both CSVs contain the exact same data with labid being the only difference.
There is no need to modify csv2 if we are just grabbing the labid to overwrite the row in csv1.
$csv1 = Import-Csv C:\temp\LabCSV1.csv
$csv2 = Import-Csv C:\temp\LabCSV2.csv
# Loop through csv1 rows
Foreach($line in $csv1) {
# If Labid contains "none"
If($line.labid -like "none*") {
# Set rows labid to the labid from csv2 row that matches plate/well
# May be able to remove the plate section if well is a unique value
$line.labid = ($csv2 | Where {$_.well -eq $line.well -and $_.plate -eq $line.plate}).labid
}
}
# Export to CSV - not overwrite - to confirm results
$csv1 | export-csv C:\Temp\LabCSV1Adjusted.csv -NoTypeInformation
Since you need to do a bi-directional comparison of the 2 Csvs you could create a new array of both and then group the objects by their well property, for this you can use Group-Object, then filter each group if their Count is equal to 2 where their labid property does not start with none else return the object as-is.
Using the following Csvs for demonstration purposes:
Csv1
"plate","labid","well"
"1013740016604537004556","none46","F006"
"1013740016604537004556","none47","G006"
"1013740016604537004556","3835265","A007"
"newrowuniquecsv1","none123","X001"
Csv2
"plate","labid","well"
"1013740016604537004556","none48","A007"
"1013740016604537004556","3835269","F006"
"1013740016604537004556","3835271","G006"
"newrowuniquecsv2","none123","X002"
Code
Note that this code assumes there will be a maximum of 2 objects with the same well property and, if there are 2 objects with the same well, one of them must have a value not starting with none.
$mergedCsv = #(
Import-Csv pathtocsv1.csv
Import-Csv pathtocsv2.csv
)
$mergedCsv | Group-Object well | ForEach-Object {
if($_.Count -eq 2) {
return $_.Group.Where{ -not $_.labid.StartsWith('none') }
}
$_.Group
} | Export-Csv pathtomerged.csv -NoTypeInformation
Output
plate labid well
----- ----- ----
1013740016604537004556 3835265 A007
1013740016604537004556 3835269 F006
1013740016604537004556 3835271 G006
newrowuniquecsv1 none123 X001
newrowuniquecsv2 none123 X002
If the lists are large, performance might be an issue as Where-Object (or any other where method) and Group-Object do not perform very well for embedded loops.
By indexing the second csv file (aka creating a hashtable), you have quicker access to the required objects. Indexing upon two (or more) items (plate and well) is issued here: Does there exist a designated (sub)index delimiter? and resolved by #mklement0 and zett42 with a nice CaseInsensitiveArrayEqualityComparer class.
To apply this class on Drew's helpful answer:
$csv1 = Import-Csv C:\temp\LabCSV1.csv
$csv2 = Import-Csv C:\temp\LabCSV2.csv
$dict = [hashtable]::new([CaseInsensitiveArrayEqualityComparer]::new())
$csv2.ForEach{ $dict.($_.plate, $_.well) = $_ }
Foreach($line in $csv1) {
If($line.labid -like "none*") {
$line.labid = $dict.($line.plate, $line.well).labid
}
}
$csv1 | export-csv C:\Temp\LabCSV1Adjusted.csv -NoTypeInformation
I am trying to input a list of users into PowerShell and get a specific security group attached to the user's account. At this current time, I have two pieces - an Excel sheet with multiple pieces of data, and a .txt with just the user's usernames. The script I have currently just inputs the user's usernames from the .txt and gets the security group from their account that matches a specific prefix, however I noticed doing it this way doesn't give any specific order. Even though the users are in a specific order (copied and pasted exactly from the excel document), the actual output doesn't come back well.
So, here's what I'd Like to do now, I just don't know how. I would like to get the content from the Excel document, take all of the usernames and do Get-ADPrincipalGroupMembership like I am now, and then write the security group Back to the line that matches the username. For example, if I looked up the SG for msnow, it would get the SG for msnow and then write the SG back to the row that has msnow, and continues through the list. Instead of just doing an Out-GridView, it would actually write this to the Excel document.
Any help on making this work?
Here is the code I have right now.
Import-Module ActiveDirectory
$Names = Get-Content C:\Temp\Users.txt
$Records = #()
Foreach ($ADUsers in $Names) {
Try {
$SG = Get-ADPrincipalGroupMembership -Identity $ADUsers | Select Name | Where {$_.Name -Like "SG - *"}
$SGName = $SG.Name
}
Catch [ADIdentityNotFoundException] {
$SGName = "User not found"
}
$Records += New-Object PSObject -Property #{"UserName" = $ADUsers;"Security Group" = $SGName}
}
Write-Host "Generating CSV File..."
$Records | Out-GridView
Thank you!
If you save the Excel as CSV, so it will look something like
"UserName","Security Group","InsideInfo"
"bloggsj","","tall guy"
"ftastic","","nothing worth mentioning"
things shouldn't be that hard to do.
$out = 'D:\Test\Updated_usersandgroups.csv'
$csv = Import-Csv -Path 'D:\Test\usersandgroups.csv'
Write-Host "Updating CSV File..."
foreach ($user in $csv) {
try {
$SG = Get-ADPrincipalGroupMembership -Identity $user.UserName -ErrorAction Stop
# if more groups are returned, combine them into a delimited string
# I'm using ', ' here, but you can change that to something else of course
$SGName = ($SG | Where-Object {$_.Name -Like "SG - *"}).Name -join ', '
}
catch [ADIdentityNotFoundException] {
$SGName = "User $($user.UserName) not found"
}
catch {
# something else went wrong?
$SGName = $_.Exception.Message
}
# update the 'Security Group' value
$user.'Security Group' = $SGName
}
Write-Host "Generating updated CSV File..."
$csv | Export-Csv -Path $out -UseCulture -NoTypeInformation
# show output on screen
$csv | Format-Table -AutoSize # or -Wrap if there is a lot of data
# show as GridView (sorts by column)
$csv | Out-GridView
Output in console would then look like
UserName Security Group InsideInfo
-------- -------------- ----------
bloggsj SG - Group1, SG - Group1 tall guy
ftastic SG - Group1 nothing worth mentioning
Note: I don't know what delimiter your Excel uses when saving to CSV file. On my Dutch machine, it uses the semi-colon ;, so if in your case this is not a comma, add the delimiter character as parameter to the Import-Csv cmdlet: -Delimiter ';'
Excel uses whatever is set in your locale as ListSeparator for the delimiter character. In PowerShell you can see what that is by doing (Get-Culture).TextInfo.ListSeparator. On output, the -UseCulture switch will make sure it uses that delimiter so Excel will understand
I have a csv file that may have unknown headers, one of the columns will contain email addresses for example.
Is there a way to select only the column that contains the email addresses and save it as a list to a variable?
One csv could have the header say email, another could say emailaddresses, another could say email addresses another file might not even have the word email in the header. As you can see, the headers are different. So I want to be able to detect the correct column first and use that data further in the script. Once the column is identified based on the data it contains, select that column only.
I've tried the where-object and select-string cmdlets. With both, the output is the entire array and not just the data in the column I am wanting.
$CSV = import-csv file.csv
$CSV | Where {$_ -like "*#domain.com"}
This outputs the entire array as all rows will contain this data.
Sample Data for visualization
id,first_name,bagel,last_name
1,Base,bcruikshank0#homestead.com,Cruikshank
2,Regan,rbriamo1#ebay.co.uk,Briamo
3,Ryley,rsacase2#mysql.com,Sacase
4,Siobhan,sdonnett3#is.gd,Donnett
5,Patty,pesmonde4#diigo.com,Esmonde
Bagel is obviously what we are trying to find. And we will play pretend in that we have no knowledge of the columns name or position ahead of time.
Find column dynamically
# Import the CSV
$data = Import-CSV $path
# Take the first row and get its columns
$columns = $data[0].psobject.properties.name
# Cycle the columns to find the one that has an email address for a row value
# Use a VERY crude regex to validate an email address.
$emailColumn = $columns | Where-Object{$data[0].$_ -match ".*#*.\..*"}
# Example of using the found column(s) to display data.
$data | Select-Object $emailColumn
Basically read in the CSV like normal and use the first columns data to try and figure out where the email address column is. There is a caveat that if there is more than one column that matches it will get returned.
To enforce only 1 result a simple pipe to Select-Object -First 1 will handle that. Then you just have to hope the first one is the "right" one.
If you're using Import-Csv, the result is a PSCustomObject.
$CsvObject = Import-Csv -Path 'C:\Temp\Example.csv'
$Header = ($CsvObject | Get-Member | Where-Object { $_.Name -like '*email*' }).Name
$CsvObject.$Header
This filters for the header containing email, then selects that column from the object.
Edit for requirement:
$Str = #((Get-Content -Path 'C:\Temp\Example.csv') -like '*#domain.com*')
$Headers = #((Get-Content -Path 'C:\Temp\Example.csv' -TotalCount 1) -split ',')
$Str | ConvertFrom-Csv -Delimiter ',' -Header $Headers
Other method:
$PathFile="c:\temp\test.csv"
$columnName=$null
$content=Get-Content $PathFile
foreach ($item in $content)
{
$SplitRow= $item -split ','
$Cpt=0..($SplitRow.Count - 1) | where {$SplitRow[$_] -match ".*#*.\..*"} | select -first 1
if ($Cpt)
{
$columnName=($content[0] -split ',')[$Cpt]
break
}
}
if ($columnName)
{
import-csv "c:\temp\test.csv" | select $columnName
}
else
{
"No Email column founded"
}
I have a csv that looks like
Name, email, address
Name, email, address
Name, email, address
I am wanting to run
New-Mailbox -Name "*Name*" -WindowsLiveID *email* -ImportLiveId
(where *x* is replaced by the value from the csv).
on each line in the csv file.
How can I do this?
$csv = Import-Csv c:\path\to\your.csv
foreach ($line in $csv) {
New-Mailbox -Name $line.Name -WindowsLiveID $line.Email -ImportLiveId
}
First line of csv has to be something like Name,Email,Address
If you cannot have the header in the CSV, you can also have:
$csv = Import-Csv c:\path\to\your.csv -Header #("Name","Email","Address")
-Header doesn't modify the csv file in any way.
import-csv .\file.csv -header ("first","second","third") | foreach{New-Mailbox -Name $_.first -WindowsLiveID $_.second -ImportLiveId}
This is some of the most useful information I have seen yet - it just made my job so much easier!!!
Combining Netapp commands:
get volumes from a controller, get snapshot schedule for said volumes, and export to a csv:
get-navol | Get-NaSnapshotSchedule | Export-Csv -path d:\something.csv
Import the csv reading in current values and assigning each column a label.
For each object, create a new schedule by RE-USING 4 of the 5 available columns/data fields
import-csv d:\something.csv -header ("label1","label2","label3","label4","label5") | foreach {Set-naSnapshotschedule $.label1 -Weeks $.label2 -Days $.label3 -Hours $.label4 -Whichhours "1,2,3,4,5"}
EXCELLENT STUFF!!!
Please note that the "Labels" should have an underscore - for whatever reason it isn't reflecting in the page so Dollar($)Underscore(_)Dot(.)Label