I have two .csv files, one is a dump from a third party application the other is a dump from Active Directory (using PowerShell).
The third party .csv looks something like this
e-mail address
Name
Title
Dept
john#work.com
John
Engineer
Office
mary#work.com
Mary
Supervisor
Factory
The AD .csv looks something like this
e-mail address
Name
Title
Dept
john#work.com
John
Engineer
Main Office
mary#work.com
Mary
Team Supervisor
Factory
Is there a way (ideally in PowerShell) that the two .csv files can be read and the differences highlighted and exported to third file.
e.g for the row containing john#work.con the Dept value is different, for the row containing mary#work.com the Title is different
The output would look something like this - where "Please update" has been entered into the required cell.
e-mail address
Name
Title
Dept
john#work.com
John
Engineer
Please update
mary#work.com
Mary
Please update
Factory
Regards
Philip
Here is one way you can do it, using Group-Object, using these CSVs as example:
$csv1 = #'
e-mail address,Name,Title,Dept
john#work.com,John,Engineer,Office
mary#work.com,Mary,Supervisor,Factory
guy#work.com,Guy,Supervisor,Factory
'# | ConvertFrom-Csv
$csv2 = #'
e-mail address,Name,Title,Dept
john#work.com,John,Engineer,Main Office
mary#work.com,Mary,Team Supervisor,Factory
otherguy#work.com,Other Guy,Team Supervisor,Factory
'# | ConvertFrom-Csv
Group both objects by their e-mail address property and then if the group count is equal to 1, skip that object since there is nothing to compare, else, compare each property against each other and update the property value of one of the objects (the object with index 0 in this case, which will be the one we return):
$csv1 + $csv2 | Group-Object 'e-mail address' | ForEach-Object {
if($_.Count -eq 1) { return }
# following assumes there will be no more than 2 elements!
# it also assumes both CSVs have the same column names!
foreach($prop in $_.Group[0].PSObject.Properties.Name) {
if($_.Group[0].$prop -ne $_.Group[1].$prop) {
$_.Group[0].$prop = 'Please Update'
}
}
$_.Group[0]
} | Format-Table
The result we can expect from above example, as you can see, guy and otherguy are not taken into consideration for the comparison:
e-mail address Name Title Dept
-------------- ---- ----- ----
john#work.com John Engineer Please Update
mary#work.com Mary Please Update Factory
Related
I have a csv file called newuser.csv that is generated daily and it's written on top of the old csv file, I created a separate csv called newuser2.csv that I could use to update only the new data from the newuser.csv file.
Here is an example
newuser.csv contains
givenName sn description title
Angel test US Empliyee IT Support Contractor
James test US Empliyee IT Support Contractor
John test US Empliyee IT Support Contractor
newuser2.csv contains
givenName sn description title
Angel test US Empliyee IT Support Contractor
newuser.csv over-written every day
newuser2.csv I created and I would like to pick up changes from the newuser.csv and update and enter those changes to newuser2.csv append.
I tried
#(Import-Csv C:\scripts\test\newusers.csv) + #(Import-Csv C:\scripts\test\newusers2.csv) | Export-Csv C:\scripts\test\newusers2.csv -Append
This did not work as it duplicates the data each time I run the script.
I would really applicate any help as this would save me so much time.
I think the gentle point #MathiasR.Jessen is making, is that we think you are using the wrong word :-). There is different ways to combine data, Overwriting is the -Force parameter e.g.
File1
A
B
File2
C
File1 Overwriting File 2
A
B
Append is "adding to the end" e.g.
File1
A
B
File2
C
File1 Appended to File 2
A
B
C
The word I believe you actually mean is Update. e.g. I want to update the list of users from file1 with the changes in file2. There is no "easy" parameter to do this because it's a more complicated process. For example, you have to choose your "unique identifier" as in, what uniquely identifies the user in order for you to update the right row. Typically this is something like usernames. In this case, without usernames, you can't simply use givenName as your only unique identifier because there may be multiple people with the name "John". Instead you have to (at the very least) see if the user matches by givenName and sn. (EDIT: Can use employeeID as unique identifier)
In this case, I read in the users, updated lists, and loop through both to find users who are in both lists and update the user information. I then export the list.
#Master list
$Users = Import-Csv C:\scripts\test\users.csv
#Update list
$NewUsers = Import-Csv C:\scripts\test\newusers.csv
#Loop through each user and make changes
foreach($User in $Users){
#Loop through new users
foreach($NewUser in $NewUsers){
#See if the user matches by employeeID
if($User.employeeID -eq $NewUser.employeeID)){
#Update User information
$User.description = $NewUser.description
$User.title = $NewUser.title
}
}
}
#Export the updated list
$Users | Export-Csv C:\scripts\test\updatedUsers.csv -Force
I have a CSV file that contains multiple vendors (Cisco, Redhat, vmware..etc), I need a PowerShell script to read this column (vendor) and add separate column "Multiple1 or Multiple2" (Remark) if the CSV contains multiple entries of same vendors.
I have attached screen shot of the sample file.
I tried from end to get this done, but it didn't work.
Okay, after Spikeys last comment I had a go at guessing what he might want to achieve.
I created a CSV file:
Product,Template
Microsoft Windows,
RedHat Enterprise,
Apple Safari,
Microsoft Windows,
RedHat Enterprise,
RedHat Enterprise,
and then wrote the following script. It's commented and produces the following output:
Product Template
------- --------
Microsoft Windows Multiple2
RedHat Enterprise Multiple3
Apple Safari Multiple1
Microsoft Windows Multiple2
RedHat Enterprise Multiple3
RedHat Enterprise Multiple3
Code:
$Csv = Import-Csv -Path "C:\Book1.csv"
#Hastables have key - value pairs. Example "Microsoft Windows" = 1. Here 'Microsoft Windows' is the key and '1' is the value
[hashtable]$ProductCount = #{}
#Go through each line in the CSV. This returns the product name e.g. Microsoft Windows
ForEach ($Product in $Csv.Product)
{
#If there us no key for the current product in hashtable $Productcount, then add it with value 1
If ($ProductCount.Keys -notcontains $Product)
{
$ProductCount.Add($Product, 1)
}
#If the above does not apply, then increase the value (effectively the count) by 1
Else
{
$ProductCount[$Product] = $ProductCount[$Product] + 1
}
}
#Go through each row in the CSV file. Each row is returned as it's own object with a 'Product' and 'Template' property
ForEach ($Row in $Csv)
{
#Extract the count for the current product from hastable $ProductCount
$Count = $ProductCount[$Row.Product]
#Set the 'Template' property for the current row object to multipile + the count we got earlier
$Row.Template = "Multiple$Count"
}
#Save the changes to the CSV file as a new CSV. You can also overwrite your old one if you like
$Csv | Export-Csv -Path "C:\Book2.csv"
I don't quite understand your question, but here are some techniques I find useful when working with CSV files.
Example CSV:
Name,City
Bob,BlackPool
Alice,Dover
Carl,Manchester
Presume you assign the CSV file to a variable like so
$CSV = Import-CSV -Path "C:\Stuff.csv"
1. You can access all rows in a column by typing the variable dot(.) column header, so
$CSV.Name
returns:
Bob
Alice
Carl
2. To access a row in a CSV file you need to use indexing, so
$CSV[1]
returns:
Name City
---- ----
Alice Dover
3. An easy way to replace a property of a specific row is to filter it using Where-Object. Say I want to change Carl's city to London.
$($CSV | Where-Object {$_.Name -like "Carl"}).City = "London"
Here is what happens:
What's in the parentheses is processed first, so we are selecting a row where the Name property is like "Carl" (You can use wilcard here, so "Ca*" would have worked too). Then, outside of the parentheses, we are setting the city property to "London".
Note: $_ represents the data currently in a pipeline, in this case that's the row containing Carl.
There is more stuff to know, but this might help you the most.
Don't forget to save your changes by using the Export-CSV cmdlet!
$CSV | Export-CSV -Path "C:\new.csv" -NoTypeInformation
I'm developing a script that connects to Azure AD and extracts failed login for any user so I'm probably going to get more than a row for user.
I have this code in a foreach (there is anything after part of code):
$ConvertedOutput | Select-Object #{Label="UserId"; Expression={$_.UserId}},
#{Label="CreationTime"; Expression={$_.CreationTime}},
#{Label="UserAgent"; Expression={$FinalOutput[0]."Value"}},
#{Label="Operation"; Expression={$_.Operation}},
#{Label="LogonError"; Expression={$_.LogonError}},
#{Label="ClientIP"; Expression={$_.ClientIP}} | Format-Table
How can I prevent from forming multiple tables? I only wanted the table for the first record, then additional records under the same table.
Thanks
here is the output
# Create an empty array to store your results in
[array]$results = #()
# This is your existing loop
foreach (...) {
...
$ConvertedOutput = <your existing code>
...
# Append your object to the results array
$results += $ConvertedOutput | Select-Object ... <your existing code>
}
# Now your results object contains all of the values from inside your loop
# So let's display that!
Write-Output $results
Welcome to stack overflow. I general, it is recommended to supply sample (fake) data in text format rather then pictures (of just headers), the makes life easier for us to answer your question.
Reading your code part, it doesn't add much value unless you planned to further extend it. All expressions generate the same keys and values as the original object, meaning that you can simplify this to just: Select-Object UserId, CreationTime, UserAgent, Operation, LogonError, LogonError, ClientIP or even: Select-Object * (or just omit the complete Select-Object), if you do not select a column subset.
With regards to your question,
By default PowerShell normally concatenates the output by it self, meaning that there is probably something else (that you are not sharing, e.g. a Write-Host command) that causes the data to be released preliminary from the pipeline.
Let me show this with fictive object lists created on the fly from three separate CSV lists:
$Result = &{
ConvertFrom-Csv #'
Id,FirstName,LastName
1,Nancy,Davolio
2,Andrew,Fuller
3,Janet,Leveling
'#
ConvertFrom-Csv #'
Id,FirstName,LastName
4,Margaret,Peacock
5,Steven,Buchanan
6,Michael,Suyama
'#
ConvertFrom-Csv #'
Id,FirstName,LastName
7,Robert,King
8,Laura,Callahan
9,Anne,Dodsworth
'#
}
With the above command, $Result contains the following data:
PS C:\> $Result
Id FirstName LastName
-- --------- --------
1 Nancy Davolio
2 Andrew Fuller
3 Janet Leveling
4 Margaret Peacock
5 Steven Buchanan
6 Michael Suyama
7 Robert King
8 Laura Callahan
9 Anne Dodsworth
One important thing to mention here, is that the columns of the three list should be have the same columns (or at least the first row should contain all expected columns), see: Not all properties displayed
If this doesn't help you further, I recommend you to add you more details to your question.
I have a CSV with the company contacts in.
I want to create a script to read the line of the CSV and chack against the current stored in AD.
When I read the contact from AD and then read the line from the CSV, in the screen they look the same but they are failing the does not equal to function.
My question is:
Is it best the read each part of the contact i.e.:
contact.firstname and compare to compare.firstname
contact.lastname and compare to compare.lastname
contact.emailaddress and compare to compare.emailaddress
using a group of If commands? Or is there an easier way to do it?
You could always start out somewhere, and then look for the perfect way. A slim example to help contributors get started on something that works for you could be:
$csv = import-csv C:\Temp\test.csv -Delimiter ";"
This will give you an object like this
firstname lastname email
--------- -------- -----
john doe john#doe.com
johnny doe johnny#doe.com
jim doe jim#doe.com
This object can now be parsed in several ways. Depending on the string variable you would like to compare, i suggest you use ForEach-Object(or it's %{} abbreviation)
Like so:
$csv | % {
if ($_.Firstname -eq $Contact.Firstname) {Write-host "Match!" -fo green}
else{Write-host "No match :(" -fo red}
}
You will be able to loop through all the objects of the csv file, one by one, and compare their note properties.
Hope it helps you get started.
I'm in the process of automating my new hire account creation in AD possess. I have created my scripts to import a csv file and that all works great. But now I am trying to automate it even further by just copying the contents of the email HR sends me into a txt file then have powershell format each line as a new column then export it to a csv file I can run my other script on (eventually combining them into just 1 script)
The email format comes in looking like this.
Username: Test User
Title: Sales
Department: Sales
Manager: Manager's Name
Similar User: CoWorker's Name`
I just want to copy and paste that into notepad and save it. then have powershell import it, format it like it is below and then export it.
Username Title Department Manager Similar User
Test User Sales Sales Manager's Name CoWorker's Name
I have been trying the import-csv and get-content commands but I am having no luck getting it formatted the way I need it. Any one have any suggestions?
A simple approach is to use ConvertFrom-StringData (the solution assumes that each line only contains exactly one :, and that the input file is named user.txt; PSv3+):
$userObj = [pscustomobject] ((Get-Content -Raw user.txt) -replace ':', '=' |
ConvertFrom-StringData)
Since ConvertFrom-StringData requires = as the separator between the key and its value, -replace is used to replace : with =.
Since ConvertFrom-StringData returns a hashtable, cast [pscustomobject] is used to convert it to a custom object.
Once the information has been parsed into custom object $userObj, it's easy to process it further:
Further processing as data:
To save the information to CSV file out.csv:
$userObj | Export-Csv -NoTypeInformation -Encoding Utf8 out.csv
This yields:
"Manager","Similar User","Department","Username","Title"
"Manager's Name","CoWorker's Name","Sales","Test User","Sales"
Further processing to reformat for display purposes:
The default formatting gives you this (which just happens to be very similar to the input:
Manager : Manager's Name
Similar User : CoWorker's Name
Department : Sales
Username : Test User
Title : Sales
$userObj | Format-Table yields:
Manager Similar User Department Username Title
------- ------------ ---------- -------- -----
Manager's Name CoWorker's Name Sales Test User Sales
To get the desired field order: $userObj | Format-Table 'Username', 'Title', 'Department', 'Manager', 'Similar User':
Username Title Department Manager Similar User
-------- ----- ---------- ------- ------------
Test User Sales Sales Manager's Name CoWorker's Name
If you want to get rid of the divider line (---- ...) and of leading and trailing empty lines:
((($userObj | Format-Table 'Username', 'Title', 'Department', 'Manager', 'Similar User') | Out-String -Stream) | ? { $_ -notmatch '^-|^$' })
Username Title Department Manager Similar User
Test User Sales Sales Manager's Name CoWorker's Name