Powershell - adding a string to each row - powershell

I have a csv with over 1,000 rows.
In powershell how do I add a fixed string (e.g. "File1")to each row of the csv file.
For example:
Cat,dog,mouse should become file1,cat,dog,mouse.
Hopefully this is clear. Thanks for assistance.

Try like this:
Import-Csv "CSV FILE Path" |
Select-Object #{n='YOUR NEW COLUMN HEADER NAME';e={"file1"}},* |
Export-Csv "CSV FILE Path"

Looking at your latest comment, I think the fastest way here is to read the file line-by-line and output that same line with the wanted prefix attached.
Something like:
$prefix = 'File1'
$result = switch -File 'D:\Test\TheInputFile.csv' {
default { "$prefix,$_" }
}
$result | Set-Content -Path 'D:\Test\TheOutputFile.csv'
If the input file is a csv with headers, you'll need to supply a new column header for the inserted field:
$prefix = 'File1'
$header = 'Path' # or some other not already used column name
$firstLine = $true
$result = switch -File 'D:\Test\TheInputFile.csv' {
default {
if ($firstLine) { "$header,$_"; $firstLine = $false }
else { "$prefix,$_" }
}
}
$result | Set-Content -Path 'D:\Test\TheOutputFile.csv'
A CSV file is a text file. The difference is that in a csv, data is presented in a structured way. Ideally it starts with a header line, defining the field names for the rest of the lines which contain the actual data.
Each field is separated from the next by a specific character, usually a comma or semi-colon or tab.
To make sure a field can also include such a character, these fields must be enclosed in double-quotes.
See here

Related

Save property with comma to CSV

I'm creating a CSV file in powershell.
Right now my code is:
Add-content -Path $filePath -Value "$($variable.Property)"
This works fine for the most part EXCEPT if the property contained a comma ie. "test, organization".
When I open up the CSV, the comma is taken with it (which is what i want) causing a extra separation. How do save "test, organization" to one column?
Referring to the documentation for Export-CSV, you will need to use a different delimiter, like a semi-colon.
When you read the CSV you should specify the delimiter as well: Import-CSV.
Try to quote your properties:
Add-content -Path $filePath -Value "'$($variable.Property)'"
Or use one of the built-in CSV commands, which automatically quote all values:
$foo.Bar | Export-Csv -Path $filePath
$foo.Bar| ConvertTo-Csv | Out-File -Path $filePath
If you just want to avoid issues with commas, you can change the delimiter between fields:
$foo | Export-Csv -Path $filePath -Delimiter '|'
Here is an article on how to use out-file or add-member with some cells of the row having commas in the variable values and some not.
https://imjustanengineer.blogspot.com/2022/01/so-youre-trying-to-use-powershell-out.html
Here is a code snippet, a more detailed explanation with a full working function is in the link. $outputArr is an array of all the lines of csv data you want to write to the csv. The loop checks each line to see if it contains commas inside of the individual cell entries and puts quotes around that entry if it does. If it does not, no adjustment is necessary and then a new array is appended to afterwards.
$index = 0;
foreach ($outputTemp in $outputArr)
{
if ($outputTemp.ToString().Contains(","))
{
$output += "`"$outputTemp`",";
}
else
{
$output += $outputTemp + ",";
}
$index++;
if ($index -eq $outputArr.Count)
{
if ($output.EndsWith(","))
{
$output = $output.Remove($output.Length - 1);
}
}
}
I found a diabolically simply answer after I opened a csv file in Excel and added text and commas to one column. When I saved, closed and reopened, the column still had all the words and commas properly formatted. So, then I opened the file in notepad++ and this is what I found:
column1text, column2text,"column3,text,with,commas"
In case it's not clear, and it took me a fair bit to recognize the little detail that makes all the difference, the opening double quote cannot have a space after the preceding comma.
column1text, column2text, "column3,text,with,commas"
splits all the words into separate columns because there is a space between
column2text, "column3,etc"
Take that space away
column2text,"column3,etc"
and everything within the double quotes stays in one column.
Example using active directory distinguishedName such as CN=somename,OU=Domain Controllers,DC=foo,DC=bar
$computers = get-adcomputer -filter *
foreach ($computer in $computers) {
$deviceName = $computer.Name
$dn = '"' + $computer.DistinguishedName + '"'
$guid = $computer.objectGUID
$lastLogon = $computer.LastLogonDate
$serialNumber = $computer.serialNumber
$whenCreated = $computer.whenCreated
"$guid, $lastLogon, $deviceName, $serialNumber, $whenCreated,$dn" | add-content "c:\temp\filename.csv"
}
It does not work if a space is added between $whenCreated, and $dn like so:
"$guid, $lastLogon, $deviceName, $serialNumber, $whenCreated, $dn" | add-content "c:\temp\filename.csv"
This took up an afternoon, so I hope this saves somebody some time and frustration.

How can I convert CSV files with a meta data header row into flat tables using Powershell?

I have a few thousand CSV files with a format similar to this (i.e. a table with a meta data row at the top):
dinosaur.csv,water,Benjamin.Field.12.Location53.Readings,
DATE,VALUE,QUALITY,STATE
2018-06-01,73.83,Good,0
2018-06-02,45.53,Good,0
2018-06-03,89.123,Good,0
Is it possible to use PowerShell to convert these CSV files into a simple table format such as this?
DATE,VALUE,QUALITY,STATE,FILENAME,PRODUCT,TAG
2018-06-01,73.83,Good,0,dinosaur.csv,water,Benjamin.Field.12.Location53.Readings
2018-06-02,45.53,Good,0,dinosaur.csv,water,Benjamin.Field.12.Location53.Readings
2018-06-03,89.123,Good,0,dinosaur.csv,water,Benjamin.Field.12.Location53.Readings
Or is there a better alternative to preparing these CSV's into a straight forward format to be ingested?
I have used PS to process simple CSV's before, but not with a meta data row that was important.
Thanks
Note: This is a faster alternative to thepip3r's helpful answer, and also covers the aspect of saving the modified content back to CSV files:
By using the switch statement to efficiently loop over the lines of the files as text, the costly calls to ConvertFrom-Csv, Select-Object and Export-Csv can be avoided.
Note that the switch statement is enclosed in $(), the subexpression operator, so as to enable writing back to the same file in a single pipeline; however, doing so requires keeping the entire (modified) file in memory; if that's not an option, enclose the switch statement in & { ... } and pipe it to Set-Content to a temporary file, which you can later use to replace the original file.
# Create a sample CSV file in the current dir.
#'
dinosaur.csv,water,Benjamin.Field.12.Location53.Readings,
DATE,VALUE,QUALITY,STATE
2018-06-01,73.83,Good,0
2018-06-02,45.53,Good,0
2018-06-03,89.123,Good,0
'# > sample.csv
# Loop over all *.csv files in the current dir.
foreach ($csvFile in Get-Item *.csv) {
$ndx = 0
$(
switch -File $csvFile.FullName {
default {
if ($ndx -eq 0) { # 1st line
$suffix = $_ -replace ',$' # save the suffix to append to data rows later
} elseif ($ndx -eq 1) { # header row
$_ + ',FILENAME,PRODUCT,TAG' # add additional column headers
} else { # data rows
$_ + ',' + $suffix # append suffix
}
++$ndx
}
}
) # | Set-Content $csvFile.FullName # <- activate this to write back to the same file.
# Use -Encoding as needed.
}
The above yields the following:
DATE,VALUE,QUALITY,STATE,FILENAME,PRODUCT,TAG
2018-06-01,73.83,Good,0,dinosaur.csv,water,Benjamin.Field.12.Location53.Readings
2018-06-02,45.53,Good,0,dinosaur.csv,water,Benjamin.Field.12.Location53.Readings
2018-06-03,89.123,Good,0,dinosaur.csv,water,Benjamin.Field.12.Location53.Readings
## If your inital block is an accurate representation
$s = get-content .\test.txt
## Get the 'metadata' line
$metaline = $s[0]
## Remove the metadata line from the original and turn it into a custom powershell object
$n = $s | where-object { $_ -ne $metaline } | ConvertFrom-Csv
## Split the metadata line by a comma to get the different parts for appending to the other content
$m = $metaline.Split(',')
## Loop through each item and append the metadata information to each entry
for ($i=0; $i -lt $n.Count; $i++) {
$n[$i] = $n[$i] | Select-Object -Property *,FILENAME,PRODUCT,TAG ## This is a cheap way to create new properties on an object
$n[$i].Filename = $m[0]
$n[$i].Product = $m[1]
$n[$i].Tag = $m[2]
}
## Display that the new objects reports as the desired output
$n | format-table

Powershell command required to strip ID/Passport No's from columns delimited text file

Have text File as per below;
New to Powershell so need guidance here.
INPUT TXT
membercode|id_number|passportnumber
BED_ODG0001|5007160081|
PF-000552516|7605430081|
PF-000704976|0385084|
PF-000678375||EP3800795
Need ONLY the ID_Number Column, but if there is no value move the Passportnumber value into the column.
REQUIRED OUTPUT TXT
5007160081
7605430081
0385084
EP3800795
I got a bit bored waiting on this, so here you go:
Have a look at the Import-Csv cmdlet.
Basically, your input txt file looks like a valid CSV file, including field headers, where the fields are separated using a pipe-symbol |.
Using Import-Csv in combination with parameter -Delimiter, it is really easy to load the data as an array of objects.
Every field in the file will be represented in these objects as properties that have names as defined in the file headers.
$inputFile = 'YOUR PATH AND FILENAME TO THE INPUT TXT FILE'
$outputFile = 'YOUR PATH AND FILENAME FOR THE OUTPUT TXT FILE'
# load the data from the CSV as an array of objects into a variable
$data = Import-Csv -Path $inputFile -Delimiter '|'
# loop through the elements of this array (the data rows in the file).
# inside the loop, the automatic variable '$_' is an object representing
# one element in the array (~~> row in the file)
$data | ForEach-Object {
# output the field 'id_number' or if that is not there, use the field 'passportnumber'
# and write it to the $outputFile file
if ($_.id_number) { $_.id_number } else { $_.passportnumber }
} | Set-Content -Path $outputFile
Result:
5007160081
7605430081
0385084
EP3800795

Commas in Powershell hash table key

Take this .txt file:
11111-2222, My, file, is, here.xml
22222-1111, My, filename.xml
22222-2323, My filename 2.xml
22222-2323, Myfilename3.xml
This text file represents a map linking ID's with names of files on a filesystem directory (each row is an entry, seperated by the first comma following the ID number). I have a powershell script that at a high-level will import this .txt file as CSV and put it in a map to where I can match a filename with it's corresponding ID number, which I need to append to a PUT request to an endpoint.
My script is working great for lines 3 and 4 above in the .txt... except for the filenames that have commas. Since we are delimiting with commas, Powershell is cutting off those field names short, causing for incorrect field values.
Code snippet:
$mapFile = "location\of\mytextfilehere.txt"
$contentObjects = "location\of\filesIwantToPost"
$map = #{}
Import-Csv $mapFile -Header ID,Filename | ForEach-Object { $map[$_.Filename] = $_.ID }
foreach ($file in $contentObjects) {
$content = Get-Content $object.PSPath
$putURI = "http://myendpoint:3000/" + $map[$file.Name]
$request = Invoke-WebRequest -Uri $putURI -Method PUT -Body $content
}
The above breaks when trying to PUT the file "My, file, is, here.xml" and "My, filename.xml". The map building saves only "My, " as the key value, since we are comma delimited.
Is there a way to help me deal with these commas and save these fields completely and correctly? Perhaps by delimiting my .txt with pipes instead of commas? Or is there a different alternate approach?
Your file is not CSV. Wrap names with quotes or parse it manually like below:
#test data
new-item sample.txt -ItemType File -Value "11111-2222, My, file, is, here.xml
22222-1111, My, filename.xml
22222-2323, My filename 2.xml
22222-2323, Myfilename3.xml"
#parse
cat sample.txt | %{
$ID, $File = $_ -split ',',2;
[PSCustomObject]#{ ID=$ID; File=$File.Trim() }
}

Append text to certain values in text file with PowerShell

I have a CSV text file separated with ; and it's in the format as:
USER_EMPLOYEE_ID;SYSTEM1;USERNAME1
The first column is an identity and the following pairs of columns are user's account on different active directories. I have placed garbage data but the idea is there.
ay7suve0001;ADDPWN;ay7suve0001
AAXMR3E0001;ADDPWN;AAXMR3E0001
ABABIL;ADDPWN;ABABIL
ABDF17;ADDPWN;ABDF17;
ABKMPPE0001;ADDPWN;ABKMPPE0001
ABL1FL;ADDPWN;ABL1FL
AB6JG8E0004;ADDPWN;AB6JG8E0004;
ACB4YB;ADDPWN;ACB4YB
ACK7J9;ADDPWN;ACK7J9
ACLZFS;ADDPWN;ACLZFS;
ACQXZ3;ADDPWN;ACQXZ3
Now there is a requirement that I have to append a fixed string like #ADDPWN.com to all the USERNAME1 values. Some records are having a ; and some don't.
Is there a quick way to append the #ADDPWN.com to each line taking care of:
any ;
any already #ADDPWN.com
From PowerShell?
Import-Csv is your friend. The following should get you on the right track.
Import-Csv "import.csv" -Delimiter ';' |
foreach {
if ($_.username1 -notlike '*#ADDPWN.com') { $_.username1 += '#ADDPWN.com' }
$_
} |
Export-Csv "export.csv" -Delimiter ';'
This assumes the first line of your csv file is your header line. If it's not, you can pass -Header 'USER_EMPLOYEE_ID','SYSTEM1','USERNAME1' as another parameter to Import-Csv.
Export-Csv adds some extra stuff like quotes around parameters, so you may need to play with the output format if you don't want that.
For another explanation how this works, check out Changes last name, first name to first name, last name in last column CSV powershell
This was a solution that worked for me.........
#opens list of file names
$file2 ="F:\OneDrive_Biz\PowerApps\SecurityCameraVideoApp\file_list_names.csv"
$x = Get-Content $file2
#appends URl to beginning of file name list
for($i=0; $i -lt $x.Count; $i++){
$x[$i] = "https://analytics-my.sharepoint.com/personal/gpowell_analytics_onmicrosoft_com/Documents/PowerApps/SecurityCameraVideoApp/Video_Files/" + $x[$i]
}
$x
#remove all files in target directory prior to saving new list
get-childitem -path C:\_TEMP\file_list_names.csv | remove-item
Add-Content -Path C:\_TEMP\file_list_names_url.csv -Value $x