I have two .csv files, one with a listing of employee ID's and a department identification number, and another with a listing of all equipment registered to them. The two files share the employee ID field, and I would like to take the department number from the first file and add it to each piece of the corresponding employee's equipment in the second file (or possibly output a third file with the joined information if that is the most expedient method). So far I have pulled the information I need from the first file and am storing it in a hash table, which I believe I should be able to use to compare to the other file, but I'm not sure exactly how to go about that. The other questions I have found on the site that may be related seem to be exclusively about checking for duplicates/changes between the two files. Any help would be much appreciated. Here is the code I have for creating the hashtable:
Import-Csv "filepath\filename.csv"|ForEach-Object -Begin{
$ids = #{}
} -Process {
$ids.Add($_.UserID,$_.'Cost Center')}
Edit:
Here is a sample of data:
First CSV:
UserID | Legal Name | Department
---------------------------------
XXX123| Namey Mcnamera | 1234
XXX321| Chet Manley | 4321
XXX000| Ron Burgundy | 9999
Second CSV:
Barcode | User ID | Department
--------------------------------
000000000000 | xxx123 | 0000
111111111111 | xxx123 | 0000
222222222222 | xxx123 | 0000
333333333333 | xxx321 | 0000
444444444444 | xxx321 | 0000
555555555555 | xxx000 | 0000
The second csv also has several more columns of data, but these three are the only ones I care about.
Edit 2:
Using this code from #wOxxOm (edited to add -force parameters as was receiving an error when attempting to write to department column due to an entry already existing):
$csv1 = Import-Csv "filename.csv"
$csv2 = Import-CSV "filename.csv"
$indexKey = 'UserID'
$index1 = #{}; foreach($row in $csv1){$index1[$row.$indexKey] = $row.'department'}
$copyfield = 'department'
foreach($row in $csv2){
if ($matched = $index1[$row.'User ID']){
Add-Member #{$copyField = $matched.$copyfield} -InputObject $row -Force
}
}
export-csv 'filepath.csv' -NoTypeInformation -Encoding UTF8 -InputObject $csv2 -Force
outputs the following information:
Count Length LongLength Rank SyncRoot IsReadOnly IsFixedSize IsSynchronized
48 48 48 1 System.Object[] FALSE TRUE FALSE
EDIT 3:
Got everything worked out with help from #Ross Lyons. Working code is as follows:
#First Spreadsheet
$users = Import-Csv "filepath.csv"
#Asset Listing
$assets = Import-Csv "filepath.csv"
[System.Array]$data = ""
#iterating through each row in first spreadsheet
foreach ($user in $users) {
#iterating through each row in the second spreadsheet
foreach ($asset in $assets) {
#compare user ID's in each spreadsheet
if ($user.UserID -eq $asset.'User ID'){
#if it matches up, copy the department data, user ID and barcode from appropriate spreadsheets
$data += $user.UserID + "," + $user."Department" + "," + $asset."Barcode" + ","
}
}
}
$data | Format-Table | Out-File "exportedData.csv" -encoding ascii -Force
Ok first, be gentle please, I'm still learning myself! Let me know if the following works or if anything is glaringly obviously wrong...
#this is your first spreadhseet with usernames & department numbers
$users = Import-Csv "spreadsheet1.csv"
#this is your second spreadsheet with equipment info & user ID's, but no department numbers
$assets = Import-Csv "spreadsheet2.csv"
#set a variable for your export data to null, so we can use it later
$export = ""
#iterating through each row in first spreadsheet
foreach ($user in $users) {
#iterating through each row in the second spreadsheet
foreach ($asset in $assets) {
#compare user ID's in each spreadsheet
if ($user.UserID -like $asset.'User ID')
#if it matches up, copy the department data, user ID and barcode from appropriate spreadsheets
$data = "$user.UserID" + "," + "$user.Department" + "," + "$asset.barcode" + "," + "~"
#splits the data based on the "~" that we stuck in at the end of the string
$export = $data -split "~" | Out-File "exportedData.csv" -Encoding ascii
}
}
Let me know what you think. Yes, I know this is probably not the best or most efficient way of doing it, but I think it will get the job done.
If this doesn't work, let me know and I'll have another crack at it.
The hashtable key should be the common field, its value should be the entire row which you can simply access later as $hashtable[$key]:
$csv1 = Import-Csv 'r:\1.csv'
$csv2 = Import-Csv 'r:\2.csv'
# build the index
$indexKey = 'employee ID'
$index1 = #{}; foreach ($row in $csv1) { $index1[$row.$indexKey] = $row }
# use the index
$copyField = 'department number'
foreach ($row in $csv2) {
if ($matched = $index1[$row.$indexKey]) {
Add-Member #{$copyField = $matched.$copyField} -InputObject $row
}
}
Export-Csv 'r:\merged.csv' -NoTypeInformation -Encoding UTF8 -InputObject $csv2
The code doesn't use pipelines for overall speedup.
Related
I have a table (CSV) which shows all the users that have ever logged on to a bunch of computers. Users can have 2 accounts, one username starts with "a" the other with "b", like a100 and b100 (the user behind is the same person).
Now I need to get the computers that have more then 2 accounts logged on which do not belong the same users. So A64 and B64 are not reported as separate users.
Here is the base list I have:
PC1,A64,B52,B64,A41
PC2,A51,B42,B51,A23
PC3,A42,B51
PC4,A5,B5
PC5,A1,B1,A14,A6
My plan was to split the "User"-column into more columns, so the table would look like this:
Computername,user1,user2,user3,user4,UserX
After this was done, I could iterate through the table and remove the leading letter in the Username, then I would try to get rid of doubles.
Do you think that makes sense?
Now I got stuck in the first task already. I know how to iterate though the second Column but how do I managed to get the result into another array so the output would be like:
Computername,user1,user2,user3,user4,UserX
Can you help me split?
$UserComputers = import-csv -Delimiter ";" "input.csv" -Header
'Computername','user1','user2','user3','user4'
$UserComputers | Select-Object *,
#{n='User1';e={$_.User1.Split(',')[0]}},
#{n='User2';e={$_.User1.Split(',')[1]}}
I get the error: Select-Object : The property cannot be processed because the property "User1" already exists.
It is useful to make "user" an array.
Get-Content "input.csv" | foreach {
$name, $users = $_.Split(",")
[pscustomobject]#{ Name = $name; Users = $users }
} | Where-Object { ($_.Users.Substring(1) | Select-Object -Unique).Count -gt 2 }
The output is below.
Name Users
---- -----
PC1 {A64, B52, B64, A41}
PC2 {A51, B42, B51, A23}
PC3 {A42, B51}
PC5 {A1, B1, A14, A6}
Input File (input.csv)
PC1,A64,B52,B64,A41
PC2,A51,B42,B51,A23
PC3,A42,B51
PC4,A5,B5
PC5,A1,B1,A14,A6
Powershell Script
Get-Content -Path .\input.csv |
Select-Object #{ Name = "Computer"; Expression = { $_.Split(',')[0] } },
#{ Name="Users"; Expression = { $_.Split(',')[1..($_.Split(',').Length-1)] |
Foreach-Object { $_.Substring(1) } | Select-Object -Unique } } |
Where-Object { $_.Users.Count -gt 2 }
Result:
Computer Users
------------- -----
PC1 {64, 52, 41}
PC2 {51, 42, 23}
PC5 {1, 14, 6}
P.S. Bonus: If you want to see more than 4 elements of the array on the screen change the variable
$FormatEnumerationLimit = 20
Explanation of the variable meaning
If your file is like your base list, you can do the following to build a new file with all the columns you need:
$maxColCount = 0
$data = get-content input.csv
foreach ($line in $data) {
$MaxColCount = [math]::Max($maxcolcount,($line -split ",").count)
}
$headers = #("ComputerName")
$MaxUserCount = $MaxColCount - 1
Foreach ($c in (1..$MaxUserCount)) {
$Headers += "User$c"
}
$Headers = $Headers -join ","
$Headers,$data | Set-Content "output.csv"
The code above assumes input.csv has the following format and each column after the first is a user:
PC1,A64,B52,B64,A41
PC2,A51,B42,B51,A23
PC3,A42,B51
PC4,A5,B5
PC5,A1,B1,A14,A6
I have 2 files that contain strings, each string in both files is delimited by a colon. Both files share a common string and I want to be able to merge both files (based on the common string) into 1 new file.
Examples:
File1.txt
tom:mioihsdihfsdkjhfsdkjf
dick:khsdkjfhlkjdhfsdfdklj
harry:lkjsdlfkjlksdjfsdlkjs
File2.txt
mioihsdihfsdkjhfsdkjf:test1
lkjsdlfkjlksdjfsdlkjs:test2
khsdkjfhlkjdhfsdfdklj:test3
File3.txt (results should look like this)
tom:mioihsdihfsdkjhfsdkjf:test1
dick:khsdkjfhlkjdhfsdfdklj:test3
harry:lkjsdlfkjlksdjfsdlkjs:test2
$File1 = #"
tom:mioihsdihfsdkjhfsdkjf
dick:khsdkjfhlkjdhfsdfdklj
harry:lkjsdlfkjlksdjfsdlkjs
"#
$File2 = #"
mioihsdihfsdkjhfsdkjf:test1
lkjsdlfkjlksdjfsdlkjs:test2
khsdkjfhlkjdhfsdfdklj:test3
"#
# You are probably going to want to use Import-Csv here
# I am using ConvertFrom-Csv as I have "inlined" the contents of the files in the variables above
$file1_contents = ConvertFrom-Csv -InputObject $File1 -Delimiter ":" -Header name, code # specifying a header as there isn't one provided
$file2_contents = ConvertFrom-Csv -InputObject $File2 -Delimiter ":" -Header code, test
# There are almost certainly better ways to do this... but this does work so... meh.
$results = #()
# Loop over one file finding the matches in the other file
foreach ($row in $file1_contents) {
$matched_row = $file2_contents | Where-Object code -eq $row.code
if ($matched_row) {
# Create a hashtable with the values you want from source and matched rows
$result = #{
name = $row.name
code = $row.code
test = $matched_row.test
}
# Append the matched up row to the final result set
$results += New-Object PSObject -Property $result
}
}
# Convert back to CSV format, with a _specific_ column ordering
# Although you'll probably want to use Export-Csv instead
$results |
Select-Object name, code, test |
ConvertTo-Csv -Delimiter ":"
I'm new to PS, so your patience is appreciated.
I'm trying to grab data from (2) separate CSV files and then dump them into a new CSV with (2) columns. Doing this for (1) is easy, but I don't know how to do it for more.
This works perfectly:
Import-CSV C:\File1.csv | Select "Employee" | Export-CSV -Path D:\Result.csv -NoTypeInformation
If I add another Import-CSV, then it simply overwrites the existing data:
Import-CSV C:\File2.csv | Select "Department" | Export-CSV -Path D:\Result.csv -NoTypeInformation
How can I get columns A and B populated with the info result from these two commands? Thanks for your help.
I would have choose this option:
$1 = Import-Csv -Path "C:\Users\user\Desktop\1.csv" | Select "Employee"
$2 = Import-Csv -Path "C:\Users\user\Desktop\2.csv" | Select "Department"
$marged = [pscustomobject]#()
$object = [pscustomobject]
for ($i=0 ; $i -lt $1.Count ; $i++){
$object = [pscustomobject]#{
Employees = $1[$i].Employee
Department = $2[$i].Department}
$marged += $object
}
$marged | ForEach-Object{ [pscustomobject]$_ } | Export-Csv -Path "C:\Users\user\Desktop\3.csv" -NoTypeInformation -Force
I'll explain how I would do this, but I do it this way because I'm more comfortable working with objects than with hastables. Someone else may offer an answer using hashtables which would probably work better.
First, I would define an array to hold your data, which can later be exported to CSV:
$report = #()
Then, I would import your CSV to an object that can be iterated through:
$firstSet = Import-CSV .\File1.csv
Then I would iterate through this, importing each row into an object that has the two properties I want. In your case these are Employee and Department (potentially more which you can add easily).
foreach($row in $firstSet)
{
$employeeName = $row.Employee
$employee = [PSCustomObject]#{
Employee = $employee
Department = ""
}
$report += $employee
}
And, as you can see in the example above, add this object to your report.
Then, import the second CSV file into a second object to iterate through (for good form I would actually do this at the begining of the script, when you import your first one):
$secondSet = Import-CSV .\File2.csv
Now here is where it gets interesting. Based on just the information you have provided, I am assuming that all employees in the one file are in the same order as the departments in the other files. So for example, if I work for the "Cake Tasting Department", and my name is on row 12 of File 1, row 12 of File 2 says "Cake Tasting Department".
In this case it's fairly easy. You would just roll through both lists and update the report:
$i = 0
foreach($row in $secondSet)
{
$dept = $row.Department
$report[i].Department = $dept
$i++
}
After this, your $report object will contain all of your employees in one row and departments in the other. Then you can export it to CSV:
$report | Export-CSV .\Result.csv -NoTypeInformation
This works if, as I said, your data aligns across both files. If not, then you need to get a little fancier:
foreach($row in $secondSet)
{
$emp = $row.Employee
$dept = $row.Department
$report | Where {$_.Employee -eq $emp} foreach {$_.Department = $dept
}
Technically you could just do it this way anyway, but it depends on a lot of things. First of all whether you have the data to match in that column across both files (which obviously in my example you don't otherwise you wouldn't need to do this in the first place, but you could match across other fields you may have, like EmployeeID or DoB). Second, on the sovereignty of individual records (e.g., if you have multiple matching records in your first file, you will have a problem; you would expect duplicates in the second as there are more than one person in each department).
Anyway, I hope this helps. As I said there is probably a 'better' way to do this, but this is how I would do it.
I have a csv file that may have unknown headers, one of the columns will contain email addresses for example.
Is there a way to select only the column that contains the email addresses and save it as a list to a variable?
One csv could have the header say email, another could say emailaddresses, another could say email addresses another file might not even have the word email in the header. As you can see, the headers are different. So I want to be able to detect the correct column first and use that data further in the script. Once the column is identified based on the data it contains, select that column only.
I've tried the where-object and select-string cmdlets. With both, the output is the entire array and not just the data in the column I am wanting.
$CSV = import-csv file.csv
$CSV | Where {$_ -like "*#domain.com"}
This outputs the entire array as all rows will contain this data.
Sample Data for visualization
id,first_name,bagel,last_name
1,Base,bcruikshank0#homestead.com,Cruikshank
2,Regan,rbriamo1#ebay.co.uk,Briamo
3,Ryley,rsacase2#mysql.com,Sacase
4,Siobhan,sdonnett3#is.gd,Donnett
5,Patty,pesmonde4#diigo.com,Esmonde
Bagel is obviously what we are trying to find. And we will play pretend in that we have no knowledge of the columns name or position ahead of time.
Find column dynamically
# Import the CSV
$data = Import-CSV $path
# Take the first row and get its columns
$columns = $data[0].psobject.properties.name
# Cycle the columns to find the one that has an email address for a row value
# Use a VERY crude regex to validate an email address.
$emailColumn = $columns | Where-Object{$data[0].$_ -match ".*#*.\..*"}
# Example of using the found column(s) to display data.
$data | Select-Object $emailColumn
Basically read in the CSV like normal and use the first columns data to try and figure out where the email address column is. There is a caveat that if there is more than one column that matches it will get returned.
To enforce only 1 result a simple pipe to Select-Object -First 1 will handle that. Then you just have to hope the first one is the "right" one.
If you're using Import-Csv, the result is a PSCustomObject.
$CsvObject = Import-Csv -Path 'C:\Temp\Example.csv'
$Header = ($CsvObject | Get-Member | Where-Object { $_.Name -like '*email*' }).Name
$CsvObject.$Header
This filters for the header containing email, then selects that column from the object.
Edit for requirement:
$Str = #((Get-Content -Path 'C:\Temp\Example.csv') -like '*#domain.com*')
$Headers = #((Get-Content -Path 'C:\Temp\Example.csv' -TotalCount 1) -split ',')
$Str | ConvertFrom-Csv -Delimiter ',' -Header $Headers
Other method:
$PathFile="c:\temp\test.csv"
$columnName=$null
$content=Get-Content $PathFile
foreach ($item in $content)
{
$SplitRow= $item -split ','
$Cpt=0..($SplitRow.Count - 1) | where {$SplitRow[$_] -match ".*#*.\..*"} | select -first 1
if ($Cpt)
{
$columnName=($content[0] -split ',')[$Cpt]
break
}
}
if ($columnName)
{
import-csv "c:\temp\test.csv" | select $columnName
}
else
{
"No Email column founded"
}
I don't know how to append a string to CSV. What am I doing:
I have two csv files. One with a list of host-names and id's and another one with a list of host-names and some numbers.
Example file 1:
Hostname | ID
IWBW140004 | 3673234
IWBW130023 | 2335934
IWBW120065 | 1350213
Example file 2:
ServiceCode | Hostname | ID
4 | IWBW120065 |
4 | IWBW140004 |
4 | IWBW130023 |
Now I read the content of file 1 in a two dimensional array:
$pcMatrix = #(,#())
Import-Csv $outputFile |ForEach-Object {
foreach($property in $_.PSObject.Properties){
$pcMatrix += ,($property.Value.Split(";")[1],$property.Value.Split(";")[2])
}
}
Then I read the content of file 2 and compare it with my array:
Import-Csv $Group".csv" | ForEach-Object {
foreach($property in $_.PSObject.Properties){
for($i = 0; $i -lt $pcMatrix.Length; $i++){
if($pcMatrix[$i][0] -eq $property.Value.Split('"')[1]){
#Add-Content here
}
}
}
}
What do I need to do, to append $pcMatrix[$i][1] to the active column in file 2 in the row ID?
Thanks for your suggestions.
Yanick
It seems like you are over-complicating this task.
If I understand you correctly, you want to populate the ID column in file two, with the ID that corresponds to the correct hostname from file 1. The easiest way to do that, is to fill all the values from the first file into a HashTable and use that to lookup the ID for each row in the second file:
# Read the first file and populate the HashTable:
$File1 = Import-Csv .\file1.txt -Delimiter "|"
$LookupTable = #{}
$File1 |ForEach-Object {
$LookupTable[$_.Hostname] = $_.ID
}
# Now read the second file and update the ID values:
$File2 = Import-Csv .\file2.txt -Delimiter "|"
$File2 |ForEach-Object {
$_.ID = $LookupTable[$_.Hostname]
}
# Then write the updated rows back to a new CSV file:
$File2 | Export-CSV -Path .\file3.txt -NoTypeInformation -Delimiter "|"