Split Columns with delimiter and export to csv - powershell

I have a csv file that contains data in the below format:
Name Codes
------- ---------
John AJFKC,EFUY
Ben EFOID, EIUF
Alex OIPORE, OUOIJE
I would like to get the above in the below format and output it into a new csv file as below:
Name Codes
------- ---------
John AJFKC
John EFUY
Ben EFOID
Ben EIUF
Alex OIPORE
Alex OUOIJE

You can do the following if there will always be 2 codes per name:
Import-Csv file.csv | Foreach-Object {
$codes = ($_.Codes -split ',').Trim()
[pscustomobject]#{Name = $_.Name; Codes = $codes[0]}
[pscustomobject]#{Name = $_.Name; Codes = $codes[1]}
}
If the number of codes per name is unknown, you can do the following:
Import-Csv file.csv | Foreach-Object {
$codes = ($_.Codes -split ',').Trim()
foreach ($code in $codes) {
[pscustomobject]#{Name = $_.Name; Codes = $code}
}
}

If this is how this file really looks, it's not a Csv file. It's just a text file based copying and using what you posted. It's left-justified, vs columns, here the staggered look of it. The file has 5 spaces for that name column between it and that left justified Codes column with a code array.
So, though it looks like a table/tabular format, it's not really that. As in an import that name column would include the first of the code strings, thus cause some issues with your desired output. So, that text file would need to be manipulated a bit more, or who or whatever provides you the file, should make is a legitimate CSV.
So, here's my take on the assumptions in the aforementioned.
(Get-Content -Path 'D:\Temp\UserRecord.csv') -replace ', ',',' |
Select -Skip 2 |
ForEach {
$Name = (($PSItem) -split '\s+')[0]
($PSItem -split '\s+')[1] -split ',' |
ForEach {
[PSCustomObject]#{
Name = $Name
Codes = $PSItem
}
}
}
# Results
<#
Name Codes
---- -----
John AJFKC
John EFUY
Ben EFOID
Ben EIUF
Alex OIPORE
Alex OUOIJE
#>

try this:
Import-Csv "c:\temp\test.csv" -Delimiter "`t" | %{
$Name=$_.Name
$_.Codes -split "," | %{ [pscustomobject]#{Name=$Name;Code=$_.Trim()}}
}

Related

Convert txt to a object

I would like to convert the content of this file (the first two columns, BARCODE and LOCATION) to object, but I am stuck creating the object..
#BARCODE LOCATION LIBRARY STATUS COPY
#------- -------- ------- ------ ----
#L40001L8 slot 41 DRP_TAPE_DRPLTO Full Primary_Global
#L40002L8 slot 96 DRP_TAPE_DRPLTO Full Primary_Global
#L40034L8 IEPort2 DRP_TAPE_DRPLTO Full Primary_Global
$object= #{}
ForEach ($i in $(gc "C:\temp\tapes.txt"))
{
$object.BARCODE= ($i.ToString().Substring(0,8))
$object.LOCATION= ($i.ToString().Substring(0,19))
New-Object psobject -Property $object
}
Can anyone help me?
Use creating object from hashtable:
$ht = #{ 'key1' = 'value1'; key2 = 'value2' }
$ht['key3'] = 'value3'
$o = [PSCustomObject]$ht
When you have fixed-width columns:
$lines = #(#'
#BARCODE LOCATION LIBRARY STATUS COPY
#------- -------- ------- ------ ----
#L40001L8 slot 41 DRP_TAPE_DRPLTO Full Primary_Global
#L40002L8 slot 96 DRP_TAPE_DRPLTO Full Primary_Global
#L40034L8 IEPort2 DRP_TAPE_DRPLTO Full Primary_Global
'# -split "`r`n" )
# $lines = [System.IO.File]::ReadAllLines($filePath)
$lines = $lines | Select-Object -Skip 2
$objects = $lines | % {
return [PSCustomObject]#{
BARCODE = $_.Substring(1,8).Trim()
LOCATION = $_.Substring(13,8).Trim()
}
}
$objects
First of all, The content of the file looks very much like you have started off with an array of objects, then 'stringyfied' that using Format-Table and copy/pasted the output from the console to your textfile.
Why every line is now commented by placing # in front, I don't know. Perhaps you did this to format the question in SO?
What I'm trying to say is that if that is indeed what happened, then it would make much more sense to change the script you used to create the file by removing the Format-Table from the code and saving the object array using a cmdlet like Export-Csv, or by saving as JSON or XML..
Anyway, if you have no choice in the matter and need to convert that file back to objects, you can do this:
# read the file line-by-line and replace the whitespaces with a comma
$data = switch -Regex -File 'D:\Test\File1.txt' {
'^#[^-]' { $_.TrimStart("#") -replace '\s+', ',' }
# if the comment marks '#' are NOT in the file, use this instead
# '^[^-]' { $_ -replace '\s+', ',' }
}
$result = $data | ConvertFrom-Csv | Select-Object BARCODE, LOCATION
# output on screen
$result
# output to CSV file
$result | Export-Csv -Path 'D:\Test\Output1.csv' -UseCulture -NoTypeInformation
Output on screen will look like
BARCODE LOCATION
------- --------
L40001L8 slot
L40002L8 slot
L40034L8 IEPort2
You can simply double-click the Output1.csv file to open in Excel

How to fix broken lines records of a file using PowerShell?

In my csv file I'm getting data in incorrect format for a few rows, sometimes a line is broken into two lines as shown in below table. For EmpId 2, line is broken into two lines. How can I find such records and merge them into one line in correct format to fix the issue for such records using PowerShell. Expected output is shown in below table.
Input File data:
EmpId,EmpName,EmpLocation
1,"Jack","Austin"
2,"Pet
er","NYC"
3,"Raj","Delhi"
Expected Output:
EmpId,EmpName,EmpLocation
1,"Jack","Austin"
2,"Peter","NYC"
3,"Raj","Delhi"
My instinct was to do something similar to Karthick's answer, however I first took a look at the output of Import-Csv. Surprisingly it puts the line break in the individual property where it was found like:
Import-Csv C:\temp\Broken.csv | fl
EmpId : 1
EmpName : Jack
EmpLocation : Austin
EmpId : 2
EmpName : Pet
er
EmpLocation : NYC
EmpId : 3
EmpName : Raj
EmpLocation : Delhi
Notice "peter" is broken across 2 lines.
So I saw some potential to bring the objects in and modify the underlying property values instead of trying to fix up the string data. I cooked up the below:
$CSVData = Import-Csv C:\temp\Broken.csv
$CSVData |
ForEach-Object{
ForEach( $Property in $_.PSObject.Properties.Name )
{
$_.($Property) = $_.($Property) -replace "(`r|`n)"
}
}
$CSVData
# If you want to re-export:
$CSVData | Export-Csv -Path c:\temp\Fixed.csv -NoTypeInformation
This code should work regardless of which field has the line break. Give it a shot and let me know. Thanks!
You can try the below. This worked for me. I assumed the first line is the header.
$filepath = "D:\file.csv"
[string[]]$data = Get-Content $filepath
$data_Final = New-Object System.Collections.ArrayList
for($i = $j = 0; $i -lt $data.Count; $(if($i -eq $j){$i++}else{$i=$j+1}), ($j=$i)) {
While ( ($data[$i] -split ",").Count -ne 3 ) {
$j = $j+1
# Concatenate the target line ($i) with successive line(s) ($j) until the elements Count to 3
$data[$i] = $data[$i] + $data[$j]
}
$data_Final.Add($data[$i]) | Out-Null
}
$inputData = $data_Final | ConvertFrom-Csv
# Or, if you want to fix the csv uncomment the below
# $data_Final | ConvertFrom-Csv | Export-Csv $filepath -NoTypeInformation

Switch Names from one side to other

I have to set description to a list of sames provided in csv format.
I know I need samaccountnames so i am trying to pull up samaccount from named, unfortunately the names in csv are in reverse order with a header as name
example
Name
Snow, Jon
Starc,arya
lannister,jamie
In a nutshell, I tried
Import-Csv C:\list.csv |
foreach {
$_.Name = "{1}, {0}" -f ($_.Name -split ', ')
$_
No luck, any help is appreciated.
The names should come as -
Jon snow
Arya starc
Jamie lannister
so I can query AD for sam's
To have lastname and firstname you can do this:
$names = $string.split(",")
[array]::Reverse($names)
$names
So if I understood correctly, you want to skip the header (first line). Try changing the following:
Import-Csv C:\list.csv | Select-Object -Skip 1 | ConvertFrom-Csv -Header Name
(or increase '-Skip' amount according to how many lines you want to skip)
The question you have here to me reads a little vague, but I wanted to offer the below. If you specify the header of the CSV when you import it, you can output whatever Property you want, in whatever order:
$test = Import-Csv -Path "C:\temp\test.txt" -Header Last_Name,First_Name
$test | % {"$($_.First_Name) $($_.Last_Name)"}
It was
Import-Csv c:\list.csv |
foreach{
$last,$first = $_.name -split ","
new-object psobject -Property #{name = "$first,$last"}
}

Edit one .CSV using Information from Another

I have two .csv files, one with a listing of employee ID's and a department identification number, and another with a listing of all equipment registered to them. The two files share the employee ID field, and I would like to take the department number from the first file and add it to each piece of the corresponding employee's equipment in the second file (or possibly output a third file with the joined information if that is the most expedient method). So far I have pulled the information I need from the first file and am storing it in a hash table, which I believe I should be able to use to compare to the other file, but I'm not sure exactly how to go about that. The other questions I have found on the site that may be related seem to be exclusively about checking for duplicates/changes between the two files. Any help would be much appreciated. Here is the code I have for creating the hashtable:
Import-Csv "filepath\filename.csv"|ForEach-Object -Begin{
$ids = #{}
} -Process {
$ids.Add($_.UserID,$_.'Cost Center')}
Edit:
Here is a sample of data:
First CSV:
UserID | Legal Name | Department
---------------------------------
XXX123| Namey Mcnamera | 1234
XXX321| Chet Manley | 4321
XXX000| Ron Burgundy | 9999
Second CSV:
Barcode | User ID | Department
--------------------------------
000000000000 | xxx123 | 0000
111111111111 | xxx123 | 0000
222222222222 | xxx123 | 0000
333333333333 | xxx321 | 0000
444444444444 | xxx321 | 0000
555555555555 | xxx000 | 0000
The second csv also has several more columns of data, but these three are the only ones I care about.
Edit 2:
Using this code from #wOxxOm (edited to add -force parameters as was receiving an error when attempting to write to department column due to an entry already existing):
$csv1 = Import-Csv "filename.csv"
$csv2 = Import-CSV "filename.csv"
$indexKey = 'UserID'
$index1 = #{}; foreach($row in $csv1){$index1[$row.$indexKey] = $row.'department'}
$copyfield = 'department'
foreach($row in $csv2){
if ($matched = $index1[$row.'User ID']){
Add-Member #{$copyField = $matched.$copyfield} -InputObject $row -Force
}
}
export-csv 'filepath.csv' -NoTypeInformation -Encoding UTF8 -InputObject $csv2 -Force
outputs the following information:
Count Length LongLength Rank SyncRoot IsReadOnly IsFixedSize IsSynchronized
48 48 48 1 System.Object[] FALSE TRUE FALSE
EDIT 3:
Got everything worked out with help from #Ross Lyons. Working code is as follows:
#First Spreadsheet
$users = Import-Csv "filepath.csv"
#Asset Listing
$assets = Import-Csv "filepath.csv"
[System.Array]$data = ""
#iterating through each row in first spreadsheet
foreach ($user in $users) {
#iterating through each row in the second spreadsheet
foreach ($asset in $assets) {
#compare user ID's in each spreadsheet
if ($user.UserID -eq $asset.'User ID'){
#if it matches up, copy the department data, user ID and barcode from appropriate spreadsheets
$data += $user.UserID + "," + $user."Department" + "," + $asset."Barcode" + ","
}
}
}
$data | Format-Table | Out-File "exportedData.csv" -encoding ascii -Force
Ok first, be gentle please, I'm still learning myself! Let me know if the following works or if anything is glaringly obviously wrong...
#this is your first spreadhseet with usernames & department numbers
$users = Import-Csv "spreadsheet1.csv"
#this is your second spreadsheet with equipment info & user ID's, but no department numbers
$assets = Import-Csv "spreadsheet2.csv"
#set a variable for your export data to null, so we can use it later
$export = ""
#iterating through each row in first spreadsheet
foreach ($user in $users) {
#iterating through each row in the second spreadsheet
foreach ($asset in $assets) {
#compare user ID's in each spreadsheet
if ($user.UserID -like $asset.'User ID')
#if it matches up, copy the department data, user ID and barcode from appropriate spreadsheets
$data = "$user.UserID" + "," + "$user.Department" + "," + "$asset.barcode" + "," + "~"
#splits the data based on the "~" that we stuck in at the end of the string
$export = $data -split "~" | Out-File "exportedData.csv" -Encoding ascii
}
}
Let me know what you think. Yes, I know this is probably not the best or most efficient way of doing it, but I think it will get the job done.
If this doesn't work, let me know and I'll have another crack at it.
The hashtable key should be the common field, its value should be the entire row which you can simply access later as $hashtable[$key]:
$csv1 = Import-Csv 'r:\1.csv'
$csv2 = Import-Csv 'r:\2.csv'
# build the index
$indexKey = 'employee ID'
$index1 = #{}; foreach ($row in $csv1) { $index1[$row.$indexKey] = $row }
# use the index
$copyField = 'department number'
foreach ($row in $csv2) {
if ($matched = $index1[$row.$indexKey]) {
Add-Member #{$copyField = $matched.$copyField} -InputObject $row
}
}
Export-Csv 'r:\merged.csv' -NoTypeInformation -Encoding UTF8 -InputObject $csv2
The code doesn't use pipelines for overall speedup.

PowerShell Import-Csv Issue - Why is my output being treated as a single column and not a CSV?

So I have a CSV file which I need to manipulate a bit, select the data I need and export to another CSV file.
The code I have is:
$rawCSV = "C:\Files\raw.csv"
$outputCSV = "C:\Files\output.csv"
Import-Csv -Header #("a","b","c","d") -Path $rawCSV |
select -Skip 7 |
Where-Object { $_.b.length -gt 1 } |
ft b,a,c,d |
Out-File $outputCSV
So this code uses the Import-Csv command to allow me to select just the columns I need, add some headers in the order I want and then I am simply putting the output in to a CSV file called $outputCSV. The contents of this output file look something like this:
b a c d
- - - -
john smith 29 England
mary poopins 79 Walton
I am not sure what the delimiter is in this output and rather than these columns being treated as individuals, they are treated as just one column. I have gone on further to replace all the spaces with a comma using the code:
$b = foreach ($line in $a)
{
$fields = $line -split '`n'
foreach ($field in $fields)
{
$field -replace " +",","
}
}
Which produces a file that looks like this:
b,a,c,d
john,smith,29,England
mary,poppins,79,Walton
But these are all still treated as one column instead of four separate columns as I need.
* UPDATE *
Using the answer given by #, I now get a file looking like this:
Don't use ft to reorder your columns - it's intended to format output for the screen, not really suitable for CSV.
"Manual" solution:
$rawCSV = "C:\Files\raw.csv"
$outputCSV = "C:\Files\output.csv"
# Import and filter your raw data
$RawData = Import-Csv -Header #("a","b","c","d") -Path $rawCSV
$Data = $RawData | Select -Skip 7 | Where-Object { $_.b.length -gt 1 }
# Write your headers to the output file
"b","a","c","d" -join ',' | Out-File $outputCSV -Force
$ReorderedData = foreach($Row in $Data){
# Reorder the columns in each row
'{0},{1},{2},{3}' -f $Row.b , $Row.a , $Row.c, $Row.d
}
# Write the reordered rows to the output file
$ReorderedData | Out-File $outputCSV -Append
Using Export-Csv:
As of PowerShell 3.0, you could also push the rows into a [pscustomobject] and pipe that to Export-Csv (pscustomobject preserves the order in which you supply the properties):
$rawCSV = "C:\Files\raw.csv"
$outputCSV = "C:\Files\output.csv"
# Import and filter your raw data
$RawData = Import-Csv -Header #("a","b","c","d") -Path $rawCSV
$Data = $RawData | Select -Skip 7 | Where-Object { $_.b.length -gt 1 }
# Take the columns you're interested in, put them into new custom objects and export to CSV
$Data | ForEach-Object {
[pscustomobject]#{ "b" = $_.b; "a" = $_.a; "c" = $_.c; "d" = $_.d }
} | Export-Csv -NoTypeInformation $outputCSV
Export-Csv will take care of enclosing strings in quotes to escape ',' properly (one thing less for you to worry about)
First of all, what your raw CSV file looks like? If it's already like this
john,smith,29,England
mary,poppins,79,Walton
then import-csv will give you an array of objects which you can easily manipulate (and objects are the main reason to use PowerShell ;). For example, to check what you have after import:
$r = Import-Csv -Path $rawCSV -Header #("b","a","c","d")
$r.GetType()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True Object[] System.Array
$r[0] | get-member
TypeName: System.Management.Automation.PSCustomObject
Name MemberType Definition
---- ---------- ----------
Equals Method bool Equals(System.Object obj)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
ToString Method string ToString()
a NoteProperty System.String a=smith
b NoteProperty System.String b=john
c NoteProperty System.String c=29
d NoteProperty System.String d=England
For now you have array of objects with properties named "a","b","c","d". To manipulate objects you have select-object cmdlet:
$r | Select-Object a,b,c,d
a b c d
- - - -
smith john 29 England
poppins mary 79 Walton
And after all use export-csv to set the output file:
$r | where { $_.b.length -gt 1 } |
select a,b,c,d |
Export-Csv -NoTypeInformation -Encoding utf8 -path $outputCSV
I could think of two possible reasons why your data teated as one column:
consuming application expect different encoding and can't find
delimiters
delimiters are not commas but something else