Find and replace inside one csv column with powershell - powershell

I have a large CSV file that looks like this named student.export.text
Student Number,Last Name,Middle Name,First Name,Schoolid,Grade Level,Dob
I'm trying to build an automated task that will run nightly so that another piece of software can understand the CSV correctly.
Here is my code, but I'm missing something that is causing an error. I am new to Powershell and I am hoping for some advice.
Any help will be greatly appreciated!
$Replacements = #{
"5" = "AE";
"7" = "ER";
"10" = "FM";
"12" = "HC";
"14" = "JH";
"18" = "LE";
#...]
}
Import-Csv .\student.export.text | ForEach-Object {
$_.Schoolid = $Replacements[$_.Schoolid]
$_
} | Export-Csv -NoTypeInformation .\new.csv

Here's one approach that can work.
# declare hash table with School ID to School Name mapping
$schoolIdsToNames = #{
"3" = "SchoolA";
"4" = "SchoolB"
}
# import the CSV file
$csv = Import-Csv "C:\input.csv";
# for each row, replace the School ID field with the School Name
foreach($row in $csv)
{
$row.Schoolid = $schoolIdsToNames[$row.Schoolid];
}
# export the modified CSV
$csv | Export-Csv "C:\replaced.csv" -NoTypeInformation;
In the first step, we set up a PowerShell hashtable (a sort of key-value pair list), then import the CSV file using Import-Csv and store it in the $csv variable. This cmdlet will create an object from every row of the CSV that we can manipulate easily. For each row, we simply replace the Schoolid field with the value assigned to the ID key in the $schoolIdsToNames hashtable. Finally, we export the CSV to another file.
Another, more PowerShell-ly approach would be something like this:
Import-Csv "C:\test\school.csv" | Select-Object *, #{ Name = "SchoolName"; Expression = { $schoolIdsToNames[$_.Schoolid] } } | Export-Csv "C:\test\replaced2.csv" -NoTypeInformation
This one-liner imports the CSV and sends it down the pipeline. For each row, we select all properties of the row using Select-Object and add a new property called SchoolName, setting its value using the same hash table-based technique as above. Finally, we export the object list to CSV.

Related

Insert blank columns into csv with Powershell

In my script, I am building a custom Powershell object which will be sent to Export-Csv. The receiving party has required that I include some blank columns (no data and no header) and I have no idea how to do that.
If the object looks like this:
$obj = [PSCustomObject][ordered]#{
EMPLOYER_EIN = '123456'
ACTION_CODE = 1
LAST_NAME = Smith
FIRST_NAME = John
MIDDLE_INITIAL = $null
EMPLOYEE_SSN = '111-11-1111'
}
How can I have the resulting .csv file's first row look like this:
EMPLOYER_EIN,ACTION_CODE,,LAST_NAME,FIRST_NAME,MIDDLE_INITIAL,,EMPLOYEE_SSN
Put another way, after I run Export-Csv, I want the file to look like this when opened in Excel:
EMPLOYER_EIN
ACTION_CODE
LAST_NAME
FIRST_NAME
MIDDLE_INITIAL
EMPLOYEE_SSN
123456
1
Smith
John
111-11-1111
Note the extra columns between action_code/last_name and middle_initial/employee_ssn. I am using PS 5.1 but could use 7 if necessary.
As a test, I created a CSV test.csv with fields A,B, and C, and put a couple of lines of values:
"A","B","C"
1,2,3
4,5,6
I then executed the sequence of commands
Import-CSV -path Test.csv | Select-Object -Prop A," ",B,C | Export-CSV -Path test2.csv
and looked at the resultant test2.csv, which contained
#TYPE Selected.System.Management.Automation.PSCustomObject
"A"," ","B","C"
"1",,"2","3"
"4",,"5","6"
I believe that this is going to be the closest you'll get without manually processing the CSV as a text file.
This is essentially what Santiago Squarzon was suggesting in the comments.
If you need multiple "blank" columns, each one will have to have a header with a different non-zero number of spaces.
I suggest:
constructing the object with blank dummy properties with a shared name prefix, such as BLANK_, followed by a sequence number (the property names must be unique)
initially piping to ConvertTo-Csv, which allows use of a -replace operation to replace the dummy property names with empty strings in the first output line (the header line).
the result - which already is in CSV format - can then be saved to a CSV file with Set-Content.
$obj = [PSCustomObject] #{
EMPLOYER_EIN = '123456'
ACTION_CODE = 1
BLANK_1 = $null # first dummy property
LAST_NAME = 'Smith'
FIRST_NAME = 'John'
MIDDLE_INITIAL = $null
BLANK_2 = $null # second dummy property
EMPLOYEE_SSN = '111-11-1111'
}
$first = $true
$obj |
ConvertTo-Csv |
ForEach-Object {
if ($first) { # header row: replace dummy property names with empty string
$first = $false
$_ -replace '\bBLANK_\d+'
}
else { # data row: pass through
$_
}
} # pipe to Set-Content as needed.
Output (note the blank column names after ACTION CODE and MIDDLE_INITIAL):
"EMPLOYER_EIN","ACTION_CODE","","LAST_NAME","FIRST_NAME","MIDDLE_INITIAL","","EMPLOYEE_SSN"
"123456","1",,"Smith","John",,,"111-11-1111"

Changing Data in Columns in a CSV

I have a PowerShell script pulling data in from a CSV. What I am trying to do is "replace" the data in the Account column based on the value. For example, Account 001 = Hardware, Account 002 = Software, etc. The data in the CSV is being pulled from a SQL database so if it would be easier for me to change it in the SQL script, I can do that easily. The Account column in the CSV has 001, 002, etc. I want to change those values to Hardware, Software, etc. Thanks for the help.
$Results = import-csv Expenses.csv
$Array = #()
Foreach($R in $Results)
{
$Object = [pscustomobject][ordered] #{
Account = $R.Account
Vendor = $R.Desc1
Item = $R.Desc2
Amount = $R.Amount
}
$Array += $Object
}
$Array
If your CSV looks anything like this:
Account,Vendor,Item,Amount
001,Some Vendor,Something expensive, 1
002,Another Vendor,Something cheapish,26
you can update without a loop:
# create a lookup hashtable where you combine the account values with the wanted replacement
$lookup = #{
'001' = 'Hardware'
'002' = 'Software'
# etcetera
}
# import the csv and update the `Account` column
$Results = Import-Csv D:\Test\Expenses.csv | Select-Object #{Name = 'Account'; Expression = {$lookup[$_.Account]}}, * -ExcludeProperty Account
# display on screen
$Results
# output to (new) csv file
$Results | Export-Csv -Path D:\Test\Expenses_Updated.csv -NoTypeInformation
Result:
Account Vendor Item Amount
------- ------ ---- ------
Hardware Some Vendor Something expensive 1
Software Another Vendor Something cheapish 26
As per the comment of not2qubit some explanation about the Select-Object statement used.
Because the result should reflect all fields in the csv, where the existing field value named Account needs to be replaced, the code uses a Calculated property to set the Account field values using whatever was stored in the lookup Hashtable.
This is done with #{Name = 'Account'; Expression = {$lookup[$_.Account]}}
Next, all other fields contained in the csv are selected unchanged using the asteriks *.
Because we're overwriting the Accound field, but keep its name, the line ends with -ExcludeProperty Account in order to remove the original Account field in the output.
If we don't do that, PowerShell will show an error: Select-Object : The property cannot be processed because the property "Account" already exists.
If I have understood what you require correctly, you just want to change "001" to "Hardware" and so on in the object imported by the Import-Csv cmdlet. You can create a ScriptBlock with a switch that will return a value based off the value you have searched for. I could have recommended a Hashtable here too, but the benefit of a switch over a Hashtable, in this case, is that you can return the value using the default option if it is not specified. For example:
$Lookup = {
Param ([string]$Value)
switch ($Value) {
"001" { "Hardware" }
"002" { "Software" }
default { $Value }
}
}
$Results = Import-Csv Expenses.csv
foreach($R in $Results)
{
# Invoke the scriptblock with the named parameter.
$R.Account = & $Lookup -Value $R.Account
}
# Do stuff with $Results

Powershell: Create CSV entries

I have a CSV File and in Column 1 are words.
I want to modify the words, for example I want to add the String "cat" at the end and write it down in Column 2.
I've posted a Question days ago where #Theo archived this:
$CSV = Import-CSV -Path 'C:\path.csv' -Header Column1
$newCsv = foreach ($row in $CSV) {
# output an Object that gets collected in variable $newCsv
# Select-Object * takes everything already in $row,
# #{Name = 'Column2'; Expression = {$row.Column1 + 'cat'}} adds the extra column to it.
$row | Select-Object *, #{Name = 'Column2'; Expression = {$row.Column1 + '-cat'}}
}
# output on screen:
$newCsv
# output to new CSV file
$newCsv | Export-Csv -Path 'C:\path.csv' -NoTypeInformation
Output (on screen):
Column1 Column2
------- -------
Wild Wildcat
Copy Copycat
Hell Hellcat
Tom Tomcat
Snow Snowcat
As far as good, now I want to create and write down a Password in Column 3.
So I would create a variable with a randomized Password with some of the many PS generators already posted online.
And I also would like to declare Column1 and Column2 as a variable because I need those 3 Column entries further down the road to create a .txt File that include those.
Just if you curious why the hell do I create a .txt File AND a CSV:
Column1 is basicly a Systemname, Column2 Username and Column3 a Password.
I document access data in the CSV and create a Script in the .txt, so I can create the User in the Exchange by Script (for about 1000+ Systems).
I appreciate any hint!

How to export data into a specific column in a csv file in PowerShell

What I am trying to do is I import data from a csv file which has UserPrincipalnames and I am taking the names before the # symbol and then I want to export that data to a specific column in the same CSV file which in this case is o365Users.csv. I am able to write it out to a text file but I need to know how to export it out to Column G with the header name as SAM
This is my code:
$Addys = Import-Csv "C:\scripts\o365Users.csv"
$UPNs = $Addys.UserPrincipalName
foreach ($UPN in $UPNs) {
$Name = $UPN.Split("#")[0]
Write-Output $Name >> c:\scripts\o365Names.txt
}
To append a new column with the header SAM use Select-Object with a calculated property:
(Import-Csv 'C:\scripts\o365Users.csv') |
Select-Object -Property *,#{n='SAM';e={$_.UserPrincipalName.Split('#')[0]}}
If the new property has to be in a specific position you can't use the wildcard * but will have to enumerate all headers/columns/properties in the desired order, i.e.
(Import-Csv 'C:\scripts\o365Users.csv') |
Select-Object -Property ColA,ColB,ColC,ColD,ColE,ColF,#{n='SAM';e={$_.UserPrincipalName.Split('#')[0]}},ColH
replace Col_ with your real headers.
Due to enclosing the (Import-Csv) in parentheses you can export to the same file name (not recommended while still testing) - simply append
| Export-Csv 'C:\scripts\o365Users.csv' -NoTypeInformation
Here is a quick way to get just the output you are looking for. You would import the current CSV. Create an blank output array and in your loop add each name. Then export the CSV
$Addys = Import-Csv "C:\scripts\o365Users.csv"
$UPNs = $Addys.UserPrincipalName
[System.Collections.ArrayList]$Output = #()
foreach ($UPN in $UPNs) {
$Name = $UPN.Split("#")[0]
$Output.Add($Name) | Out-Null
}
$Output | Export-Csv -Path "C:\scripts\o365Users.csv" -NoTypeInformation

Export results of (2) cmdlets to separate columns in the same CSV

I'm new to PS, so your patience is appreciated.
I'm trying to grab data from (2) separate CSV files and then dump them into a new CSV with (2) columns. Doing this for (1) is easy, but I don't know how to do it for more.
This works perfectly:
Import-CSV C:\File1.csv | Select "Employee" | Export-CSV -Path D:\Result.csv -NoTypeInformation
If I add another Import-CSV, then it simply overwrites the existing data:
Import-CSV C:\File2.csv | Select "Department" | Export-CSV -Path D:\Result.csv -NoTypeInformation
How can I get columns A and B populated with the info result from these two commands? Thanks for your help.
I would have choose this option:
$1 = Import-Csv -Path "C:\Users\user\Desktop\1.csv" | Select "Employee"
$2 = Import-Csv -Path "C:\Users\user\Desktop\2.csv" | Select "Department"
$marged = [pscustomobject]#()
$object = [pscustomobject]
for ($i=0 ; $i -lt $1.Count ; $i++){
$object = [pscustomobject]#{
Employees = $1[$i].Employee
Department = $2[$i].Department}
$marged += $object
}
$marged | ForEach-Object{ [pscustomobject]$_ } | Export-Csv -Path "C:\Users\user\Desktop\3.csv" -NoTypeInformation -Force
I'll explain how I would do this, but I do it this way because I'm more comfortable working with objects than with hastables. Someone else may offer an answer using hashtables which would probably work better.
First, I would define an array to hold your data, which can later be exported to CSV:
$report = #()
Then, I would import your CSV to an object that can be iterated through:
$firstSet = Import-CSV .\File1.csv
Then I would iterate through this, importing each row into an object that has the two properties I want. In your case these are Employee and Department (potentially more which you can add easily).
foreach($row in $firstSet)
{
$employeeName = $row.Employee
$employee = [PSCustomObject]#{
Employee = $employee
Department = ""
}
$report += $employee
}
And, as you can see in the example above, add this object to your report.
Then, import the second CSV file into a second object to iterate through (for good form I would actually do this at the begining of the script, when you import your first one):
$secondSet = Import-CSV .\File2.csv
Now here is where it gets interesting. Based on just the information you have provided, I am assuming that all employees in the one file are in the same order as the departments in the other files. So for example, if I work for the "Cake Tasting Department", and my name is on row 12 of File 1, row 12 of File 2 says "Cake Tasting Department".
In this case it's fairly easy. You would just roll through both lists and update the report:
$i = 0
foreach($row in $secondSet)
{
$dept = $row.Department
$report[i].Department = $dept
$i++
}
After this, your $report object will contain all of your employees in one row and departments in the other. Then you can export it to CSV:
$report | Export-CSV .\Result.csv -NoTypeInformation
This works if, as I said, your data aligns across both files. If not, then you need to get a little fancier:
foreach($row in $secondSet)
{
$emp = $row.Employee
$dept = $row.Department
$report | Where {$_.Employee -eq $emp} foreach {$_.Department = $dept
}
Technically you could just do it this way anyway, but it depends on a lot of things. First of all whether you have the data to match in that column across both files (which obviously in my example you don't otherwise you wouldn't need to do this in the first place, but you could match across other fields you may have, like EmployeeID or DoB). Second, on the sovereignty of individual records (e.g., if you have multiple matching records in your first file, you will have a problem; you would expect duplicates in the second as there are more than one person in each department).
Anyway, I hope this helps. As I said there is probably a 'better' way to do this, but this is how I would do it.