Merge 2 csv files using powershell append - powershell

I have a csv file called newuser.csv that is generated daily and it's written on top of the old csv file, I created a separate csv called newuser2.csv that I could use to update only the new data from the newuser.csv file.
Here is an example
newuser.csv contains
givenName sn description title
Angel test US Empliyee IT Support Contractor
James test US Empliyee IT Support Contractor
John test US Empliyee IT Support Contractor
newuser2.csv contains
givenName sn description title
Angel test US Empliyee IT Support Contractor
newuser.csv over-written every day
newuser2.csv I created and I would like to pick up changes from the newuser.csv and update and enter those changes to newuser2.csv append.
I tried
#(Import-Csv C:\scripts\test\newusers.csv) + #(Import-Csv C:\scripts\test\newusers2.csv) | Export-Csv C:\scripts\test\newusers2.csv -Append
This did not work as it duplicates the data each time I run the script.
I would really applicate any help as this would save me so much time.

I think the gentle point #MathiasR.Jessen is making, is that we think you are using the wrong word :-). There is different ways to combine data, Overwriting is the -Force parameter e.g.
File1
A
B
File2
C
File1 Overwriting File 2
A
B
Append is "adding to the end" e.g.
File1
A
B
File2
C
File1 Appended to File 2
A
B
C
The word I believe you actually mean is Update. e.g. I want to update the list of users from file1 with the changes in file2. There is no "easy" parameter to do this because it's a more complicated process. For example, you have to choose your "unique identifier" as in, what uniquely identifies the user in order for you to update the right row. Typically this is something like usernames. In this case, without usernames, you can't simply use givenName as your only unique identifier because there may be multiple people with the name "John". Instead you have to (at the very least) see if the user matches by givenName and sn. (EDIT: Can use employeeID as unique identifier)
In this case, I read in the users, updated lists, and loop through both to find users who are in both lists and update the user information. I then export the list.
#Master list
$Users = Import-Csv C:\scripts\test\users.csv
#Update list
$NewUsers = Import-Csv C:\scripts\test\newusers.csv
#Loop through each user and make changes
foreach($User in $Users){
#Loop through new users
foreach($NewUser in $NewUsers){
#See if the user matches by employeeID
if($User.employeeID -eq $NewUser.employeeID)){
#Update User information
$User.description = $NewUser.description
$User.title = $NewUser.title
}
}
}
#Export the updated list
$Users | Export-Csv C:\scripts\test\updatedUsers.csv -Force

Related

Reading Multiple Entry from CSV using PowerShell

I have a CSV file that contains multiple vendors (Cisco, Redhat, vmware..etc), I need a PowerShell script to read this column (vendor) and add separate column "Multiple1 or Multiple2" (Remark) if the CSV contains multiple entries of same vendors.
I have attached screen shot of the sample file.
I tried from end to get this done, but it didn't work.
Okay, after Spikeys last comment I had a go at guessing what he might want to achieve.
I created a CSV file:
Product,Template
Microsoft Windows,
RedHat Enterprise,
Apple Safari,
Microsoft Windows,
RedHat Enterprise,
RedHat Enterprise,
and then wrote the following script. It's commented and produces the following output:
Product Template
------- --------
Microsoft Windows Multiple2
RedHat Enterprise Multiple3
Apple Safari Multiple1
Microsoft Windows Multiple2
RedHat Enterprise Multiple3
RedHat Enterprise Multiple3
Code:
$Csv = Import-Csv -Path "C:\Book1.csv"
#Hastables have key - value pairs. Example "Microsoft Windows" = 1. Here 'Microsoft Windows' is the key and '1' is the value
[hashtable]$ProductCount = #{}
#Go through each line in the CSV. This returns the product name e.g. Microsoft Windows
ForEach ($Product in $Csv.Product)
{
#If there us no key for the current product in hashtable $Productcount, then add it with value 1
If ($ProductCount.Keys -notcontains $Product)
{
$ProductCount.Add($Product, 1)
}
#If the above does not apply, then increase the value (effectively the count) by 1
Else
{
$ProductCount[$Product] = $ProductCount[$Product] + 1
}
}
#Go through each row in the CSV file. Each row is returned as it's own object with a 'Product' and 'Template' property
ForEach ($Row in $Csv)
{
#Extract the count for the current product from hastable $ProductCount
$Count = $ProductCount[$Row.Product]
#Set the 'Template' property for the current row object to multipile + the count we got earlier
$Row.Template = "Multiple$Count"
}
#Save the changes to the CSV file as a new CSV. You can also overwrite your old one if you like
$Csv | Export-Csv -Path "C:\Book2.csv"
I don't quite understand your question, but here are some techniques I find useful when working with CSV files.
Example CSV:
Name,City
Bob,BlackPool
Alice,Dover
Carl,Manchester
Presume you assign the CSV file to a variable like so
$CSV = Import-CSV -Path "C:\Stuff.csv"
1. You can access all rows in a column by typing the variable dot(.) column header, so
$CSV.Name
returns:
Bob
Alice
Carl
2. To access a row in a CSV file you need to use indexing, so
$CSV[1]
returns:
Name City
---- ----
Alice Dover
3. An easy way to replace a property of a specific row is to filter it using Where-Object. Say I want to change Carl's city to London.
$($CSV | Where-Object {$_.Name -like "Carl"}).City = "London"
Here is what happens:
What's in the parentheses is processed first, so we are selecting a row where the Name property is like "Carl" (You can use wilcard here, so "Ca*" would have worked too). Then, outside of the parentheses, we are setting the city property to "London".
Note: $_ represents the data currently in a pipeline, in this case that's the row containing Carl.
There is more stuff to know, but this might help you the most.
Don't forget to save your changes by using the Export-CSV cmdlet!
$CSV | Export-CSV -Path "C:\new.csv" -NoTypeInformation

Updating Company contacts

I have a CSV with the company contacts in.
I want to create a script to read the line of the CSV and chack against the current stored in AD.
When I read the contact from AD and then read the line from the CSV, in the screen they look the same but they are failing the does not equal to function.
My question is:
Is it best the read each part of the contact i.e.:
contact.firstname and compare to compare.firstname
contact.lastname and compare to compare.lastname
contact.emailaddress and compare to compare.emailaddress
using a group of If commands? Or is there an easier way to do it?
You could always start out somewhere, and then look for the perfect way. A slim example to help contributors get started on something that works for you could be:
$csv = import-csv C:\Temp\test.csv -Delimiter ";"
This will give you an object like this
firstname lastname email
--------- -------- -----
john doe john#doe.com
johnny doe johnny#doe.com
jim doe jim#doe.com
This object can now be parsed in several ways. Depending on the string variable you would like to compare, i suggest you use ForEach-Object(or it's %{} abbreviation)
Like so:
$csv | % {
if ($_.Firstname -eq $Contact.Firstname) {Write-host "Match!" -fo green}
else{Write-host "No match :(" -fo red}
}
You will be able to loop through all the objects of the csv file, one by one, and compare their note properties.
Hope it helps you get started.

Powershell: Advanced file age searching

This is my first post within Stackoverflow, I have for many years just read many fantastic questions, answers and other various posts. I have learned a lot a lot from this fantastic community. I hope now that I have taken the brave step to really sink my teeth into powershell and join this community that I may be able to contribute in someway!
So I have started working on a project which at it's basic core level - list all files that are older than 7 years so they can then review and delete where possible.
I have however broken the whole entire script up into several stages. I am currently stuck at a step in stage 2.
I have been stuck for about 2 days on what to many of you powershell genius's out there may only take 10mins to figure out!
I must apologise for my stupidity and lack of knowledge, my experience with powershell scripting is limited to literally 5 working days, I am currently diving in and learning with books, but I also have a job to do so don't get to learn the easy way!
My script essentially has 3 steps;
Runs a Get-ACL of top Level DATA folders to create a listing of all Groups that have permissions on particular folder. I want to then either export this data or simply hold it for the next step.
Filter this gathered information based off a CSV which contains a Column labelled Role (Role will contain a group that the Folder Manager is exclusively in), and check the inherent member of this exlcusive group (maybe this last bit needs to be another step as well?)
Stores or Exports this list of exclusive members with their relevant folder to then later use as a variable for to send an email with a list of files that need to be deleted.
With the script below I am essentially stuck on Step 2 and how to create a filter from the CSV (or stored variables?) and apply it to the GET-ACL foreach loop. I may be going about this the whole wrong way using regex, and to be honest most of this is copy and paste and reading around the internet where people have done similar tasks. SO again I apologise if this is just a dumb way to go about it from the start!
I want to thank everyone in advance for all help, opinions and advice, I will listen to it all and I will try and take on-board as much as my brain can handle - I promise!
#$RoleList = import-csv "C:\DATA\scripts\GB_CMS\CSV\datafolders_rolelist.csv"
#foreach ($Manager in $RoleList) {
#$FolderManager = $RoleList.Role
$FolderManagers = Import-Csv C:\DATA\scripts\GB_CMS\CSV\datafolders_rolelist.csv | foreach {
New-Object PSObject -prop #{
Folder = $_.Folder;
Manager = $_.'Folder Manager';
Role = $_.Role
}
}
$Role = $FolderManagers.Role
$Role
gci "c:\DATA" | Where {$_.PSIsContainer} | get-acl |
ForEach $_.Name {
[regex]$regex="\w:\\\S+"
$path=$regex.match($_.Path).Value
$_ | select -expand access |
$
where {$_.identityreference -like "$Role"} |
Select #{Name="Path";Expression={$Path}},IdentityReference
}
Thanks,
Daniel.
Bit of a guess at what you want here. e.g. if you have folders
C:\Data\Accounts
C:\Data\Marketing
C:\Data\Sales
You might have permissions
C:\Data\Accounts {'FolderManagers-Accounts', 'Accounts', 'Directors'}
C:\Data\Marketing {'FolderManagers-Marketing', 'Marketing', 'Sales'}
C:\Data\Sales {'FolderManagers-Sales', 'Sales', 'Directors'}
and your CSV is
Name, Role, Email
Alice, FolderManagers-Accounts, alice#example.com
Bob, FolderManagers-Marketing, bob#example.com
And there will be a clear mapping of one (1) row in the CSV to one of the groups in the ACLs.
And you want, from your script:
Identify who to email about "C:\Data\Accounts"
How close am I?
# Import the managers. This will turn the CSV into an array of objects
# no need to do that explicitly
$FolderManagers = Import-Csv C:\DATA\scripts\GB_CMS\CSV\datafolders_rolelist.csv
# This will be a hashtable pairing up folder names with people
# e.g. 'C:\DATA\Accounts' -> Alice
$FolderMap = #{}
# Run through all the folders
GetChildItem -Path "C:\Data" -Directory | ForEach-Object {
# Run through the group/user ACL entries on the folder
foreach ($group in (Get-Acl $_.FullName).Access.IdentityReference)
{
# Look for a matching row in the CSV
$CsvRow = $FolderManagers | Where-Object {$_.Role -match $group}
if (-not $CsvRow)
{
Write-Error "No manager found for folder $_"
}
else
{
# Add to the map
# $_ converts to folder path, C:\DATA\Accounts
# $CsvRow is the person, #{Name=Alice, Role=..., Email=...}
$FolderMap[$_.FullName] = $CsvRow
}
}
}
Then it (the FolderMap) will be
Name Value
---- -----
C:\Data\Accounts {Name='Alice';Role=...
C:\Data\Marketing {Name='Bob';Role=...
you can query it with
$person = $FolderMap["c:\data\Accounts"]
$person.email
and if you really want to export it, maybe
$FolderMap | ConvertTo-Json | Set-Content foldermanagers.json
Nb. I wrote most of this off the top of my head, and it probably won't just run. And that's a problem with big, not very specific questions on StackOverflow.
Auto-generated PS help links from my codeblock (if available):
Import-Csv (in module Microsoft.PowerShell.Utility)
ForEach-Object
Get-Acl (in module Microsoft.PowerShell.Security)
Where-Object
Write-Error (in module Microsoft.PowerShell.Utility)

Rename Files with Index(Excel)

Anyone have any ideas on how to rename files by finding an association with an index file?
I have a file/folder structure like the following:
Folder name = "Doe, John EO11-123"
Several files under this folder
The index file(MS Excel) has several columns. It contains the names in 2 columns(First and Last). It also has a column containing the number EO11-123.
What I would like to do is write maybe a script to look at the folder names in a directory, compare/find an associated value in the index file(like that number EO11-123) and then rename all the files under the folder using a 4th column value in the index.
So,
Folder name = "Doe, John EO11-123", index column1 contains same value "EO11-123", use column2 value "111111_000000" and rename all the files under that directory folder to "111111_000000_0", "111111_000000_1", "111111_000000_2" and so on.
This possible with powershell or vbscript?
Ok, I'll answer your questions in your comment first. Importing the data into PowerShell allows you to make an array in powershell that you can match against, or better yet make a HashTable to reference for your renaming purposes. I'll get into that later, but it's way better than trying to have PowerShell talk to Excel and use Excel's search functions because this way it's all in PowerShell and there's no third party application dependencies. As for importing, that script is a function that you can load into your current session, so you run that function and it will automatically take care of the import for you (it opens Excel, then opens the XLS(x) file, saves it as a temp CSV file, closes Excel, imports that CSV file into PowerShell, and then deletes the temp file).
Now, you did not state what your XLS file looks like, so I'm going to assume it's got a header row, and looks something like this:
FirstName | Last Name | Identifier | FileCode
Joe | Shmoe | XA22-573 | JS573
John | Doe | EO11-123 | JD123
If that's not your format, you'll need to either adapt my code, or your file, or both.
So, how do we do this? First, download, save, and if needed unblock the script to Import-XLS. Then we will dot source that file to load the function into the current PowerShell session. Once we have the function we will run it and assign the results to a variable. Then we can make an empty hashtable, and for each record in the imported array create an entry in the hashtable where the 'Identifier' property (in your example above that would be the one that has the value "EO11-123" in it), make that the Key, then make the entire record the value. So, so far we have this:
#Load function into current session
. C:\Path\To\Import-XLS.ps1
$RefArray = Import-XLS C:\Path\To\file.xls
$RefHash = #{}
$RefArray | ForEach( $RefHash.Add($_.Identifier, $_)}
Now you should be able to reference the identifier to access any of the properties for the associated record such as:
PS C:\> $RefHash['EO11-123'].FileCode
JD123
Now, we just need to extract that name from the folder, and rename all the files in it. Pretty straight forward from here.
Get-ChildItem c:\Path\to\Folders -directory | Where{$_.Name -match "(?<= )(\S+)$"}|
ForEach{
$Files = Get-ChildItem $_.FullName
$NewName = $RefHash['$($Matches[1])'].FileCode
For($i = 1;$i -lt $files.count;$i++){
$Files[$i] | Rename-Item -New "$NewName_$i"
}
}
Edit: Ok, let's break down the rename process here. It is a lot of piping here, so I'll try and take it step by step. First off we have Get-ChildItem that gets a list of folders for the path you specify. That part's straight forward enough. Then it pipes to a Where statement, that filters the results checking each one's name to see if it matches the Regular Expression "(?<= )(\S+)$". If you are unfamiliar with how regular expressions work you can see a fairly good breakdown of it at https://regex101.com/r/zW8sW1/1. What that does is matches any folders that have more than one "word" in the name, and captures the last "word". It saves that in the automatic variable $Matches, and since it captured text, that gets assigned to $Matches[1]. Now the code breaks down here because your CSV isn't laid out like I had assumed, and you want the files named differently. We'll have to make some adjustments on the fly.
So, those folder that pass the filter will get piped into a ForEach loop (which I had a typo in previously and had a ( instead of {, that's fixed now). So for each of those folders it starts off by getting a list of files within that folder and assigning them to the variable $Files. It also sets up the $NewName variable, but since you don't have a column in your CSV named 'FileCode' that line won't work for you. It uses the $Matches automatic variable that I mentioned earlier to reference the hashtable that we setup with all of the Identifier codes, and then looks at a property of that specific record to setup the new name to assign to files. Since what you want and what I assumed are different, and your CSV has different properties we'll re-work both the previous Where statement, and this line a little bit. Here's how that bit of the script will now read:
Get-ChildItem c:\Path\to\Folders -directory | Where{$_.Name -match "^(.+?), .*? (\S+)$"}|
ForEach{
$Files = Get-ChildItem $_.FullName
$NewName = $Matches[2] + "_" + $Matches[1]
That now matches the folder name in the Where statement and captures 2 things. The first thing it grabs is everything at the beginning of the name before the comma. Then it skips everything until it gets tho the last piece of text at the end of the name and captures everything after the last space. New breakdown on RegEx101: https://regex101.com/r/zW8sW1/2
So you want the ID_LName, which can be gotten from the folder name, there's really no need to even use your CSV file at this point I don't think. We build the new name of the files based off the automatic $Matches variable using the second capture group and the first capture group and putting an underscore between them. Then we just iterate through the files with a For loop basing it off how many files were found. So we start with the first file in the array $Files (record 0), add that to the $NewName with an underscore, and use that to rename the file.

Simple PowerShell Script to loop through 1 CSV file to create a new CSV file from another

I know the title sounds confusing, but once I describe this, I'm certain there is a very easy way to perform what I need to do. I'm very new to PowerShell and am trying to perform a specific task that seems rather difficult to find a good answer for one the Web.
I have spent the past several days searching through methods of concatenating the data and joining the files together, but nothing that was specific enough to this task. All examples show how to display data, but nothing that loops through and adds data together to create a new csv file. If anything, I've over-researched this issue to the point of having to pose this message to see where I can get my brain de-cluttered with all of the useless options I've already tried...
I have two csv files. I call them csv's, but they are really just a single column of information each.
Files:
Users.csv
Offices.csv
The Users.csv file has a list of network user names. The Offices.csv file has a list of numbers that correspond to office locations.
What I want to have happen is to use a loop that will take each user from the users.csv file and create a new line in a separate csv file the adds each of the offices to it.
EXAMPLE:
Users.csv
NTNAME
domain\user1
domain\user2
domain\user3
Offices.csv
OFFICES
0001
0023
0043
0067
When combined, I would like csv file that looks like this:
NTNAME,OFFICES
domain\user1,0001
domain\user1,0023
domain\user1,0043
domain\user1,0067
domain\user2,0001
domain\user2,0023
domain\user2,0043
domain\user2,0067
domain\user3,0001
domain\user3,0023
domain\user3,0043
domain\user3,0067
Any help you can give would be greatly appreciated...
Borrowing Shay's awesome CSV field enumeration code:
$offices = Import-Csv 'C:\path\to\offices.csv'
Import-Csv 'C:\path\to\users.csv' | % {
foreach ($prop in $_.PSObject.Properties) {
$offices | select #{n=$prop.Name;e={$prop.Value}}, OFFICES
}
} | Export-Csv 'C:\path\to\combined.csv' -NoTypeInformation