I have a powershell script to parse a CSV file and add a new value to specific header. For Example Input file is a header and i want to replace the value under it. I have added the logic and it works for normal hearder but as this header has a space i am not able to manage it. Can you please help me how can i manage the space
Convert into CSV
Import-CSV $CSV_File | ForEach-Object {
{$_.Input file} = "$machine_path\file"
{$_.Output file} ="$machine_path\Output"
$_ | Export-Csv $machine_path\new.csv -NoTypeInformation
}
If you want to dereference a property with spaces in the name, use quotation marks:
Import-CSV $CSV_File | ForEach-Object {
$_.'Input file' = "$machine_path\file"
$_.'Output file' = "$machine_path\Output"
$_
}| Export-Csv $machine_path\new.csv -NoTypeInformation
Related
I have a CSV file named test.csv (C:\testing\test.csv) in this format:
File Name,Location,Added (GMT),Created (GMT),Last Modified (GMT),File Size (Bytes),File Size,Extension,Incident Type
10-MB-Test (1).docx,\\blah\Test 3,10/8/2020 21:13,10/8/2020 19:33,10/8/2020 16:26,10723331,10.23 (MB),docx,low_data_discover
10-MB-Test (1).xlsx,\\blah2\Test 3\,10/8/2020 21:14,10/8/2020 19:33,10/8/2020 16:25,9566567,9.12 (MB),xlsx,high_data_discover
1-MB-Test.docx,\\blah3\Test 3\,10/8/2020 21:13,10/8/2020 19:33,10/8/2020 16:37,1045970,1021.46 (KB),docx,medium_data_discover
I'm trying to replace trailing "\" characters (if they exist) for values in the Location column with nothing using this Powershell code:
$file1 = import-csv -path "C:\testing\test.csv" | % {$_."Location" -replace "\\$",""} | Select-Object * | export-csv -NoTypeInformation "C:\testing\blah.csv"
However, when I run the code, the only output I get is a column named "Length" with a numerical value. Can you assist?
You're only sending the new string (updated location) down the pipeline. You can update each location and then export it at the end.
$file1 = import-csv -path "C:\testing\test.csv"
$file1 | ForEach-Object {$_.location = $_.location -replace '\\$'}
$file1 | export-csv -NoTypeInformation "C:\testing\blah.csv"
What I am trying to do is I import data from a csv file which has UserPrincipalnames and I am taking the names before the # symbol and then I want to export that data to a specific column in the same CSV file which in this case is o365Users.csv. I am able to write it out to a text file but I need to know how to export it out to Column G with the header name as SAM
This is my code:
$Addys = Import-Csv "C:\scripts\o365Users.csv"
$UPNs = $Addys.UserPrincipalName
foreach ($UPN in $UPNs) {
$Name = $UPN.Split("#")[0]
Write-Output $Name >> c:\scripts\o365Names.txt
}
To append a new column with the header SAM use Select-Object with a calculated property:
(Import-Csv 'C:\scripts\o365Users.csv') |
Select-Object -Property *,#{n='SAM';e={$_.UserPrincipalName.Split('#')[0]}}
If the new property has to be in a specific position you can't use the wildcard * but will have to enumerate all headers/columns/properties in the desired order, i.e.
(Import-Csv 'C:\scripts\o365Users.csv') |
Select-Object -Property ColA,ColB,ColC,ColD,ColE,ColF,#{n='SAM';e={$_.UserPrincipalName.Split('#')[0]}},ColH
replace Col_ with your real headers.
Due to enclosing the (Import-Csv) in parentheses you can export to the same file name (not recommended while still testing) - simply append
| Export-Csv 'C:\scripts\o365Users.csv' -NoTypeInformation
Here is a quick way to get just the output you are looking for. You would import the current CSV. Create an blank output array and in your loop add each name. Then export the CSV
$Addys = Import-Csv "C:\scripts\o365Users.csv"
$UPNs = $Addys.UserPrincipalName
[System.Collections.ArrayList]$Output = #()
foreach ($UPN in $UPNs) {
$Name = $UPN.Split("#")[0]
$Output.Add($Name) | Out-Null
}
$Output | Export-Csv -Path "C:\scripts\o365Users.csv" -NoTypeInformation
I filtered by date this file data1.csv
2017.11.1,09:55,1.1,1.2,1.3,1.4,1
2017.11.2,09:55,1.5,1.6,1.7,1.8,2
I don't get a header with -NoTypeInformation:
$CutOff = (Get-Date).AddDays(-2)
$filePath = "data1.csv"
$Data = Import-Csv $filePath -Header Date,Time,A,B,C,D,E
$Data2 = $Data | Where-Object {$_.Date -as [datetime] -gt $Cutoff} | convertto-csv -NoTypeInformation -Delimiter "," | % {$_ -replace '"',''}
But when rewriting with Out-File
$Data2 | Out-File "data2.csv" -Encoding utf8 -Force
I get header back as data2.csv contains:
Date,Time,A,B,C,D,E
2017.11.2,09:55,1.5,1.6,1.7,1.8,2
Why do I have Date,Time,A,B,C,D,E ?
-NoTypeInformation is not about the header but the data type of the rows in the file. Remove it to see what shows up. From Microsoft
Omits the type information header from the output. By default, the string in the output contains #TYPE followed by the fully-qualified name of the object type.
Emphasis mine.
CSVs need headers. That is why it is making one. If you don't want to see the header in the output use Select-Object -Skip 1 to remove it.
$Data |
Where-Object {$_.Date -as [datetime] -gt $Cutoff} |
ConvertTo-CSV -NoTypeInformation -Delimiter "," |
Select-Object -Skip 1 |
% {$_ -replace '"'}
I would not pipe Out-File to itself. You could pipe to Set-Content here just as well.
I am guessing this whole process is to keep the source file in the same state just with some lines filtered out based on date. You could skip most of this just by parsing the date out in each line.
$threshold = (Get-Date).AddDays(-2)
$filePath = "c:\temp\bagel.txt"
(Get-Content $filePath) | Where-Object{
$date,$null=$_.Split(",",2)
[datetime]$date -gt $threshold
} | Set-Content $filePath
Now you don't have to worry about PowerShell CSV object structure or output since we act on the raw data of the file itself.
That will take each line of the input file and filter it out if the parsed date does not match the threshold. Change encoding on the input output cmdlets as you see necessary. What $date,$null=$_.Split(",",2) is doing is splitting the line
on the comma into 2 parts. First of which becomes $date and since this is just a filtering condition we dump the rest of the line into $null.
Properly-formed CSV files must have column headers. Your use of -NoTypeInformation in generating the CSV does not affect column headers; instead, it affects whether the PowerShell object type information is included. If you Export-CSV without -NoTypeInformation, the first line of your CSV file will have a line that looks like #TYPE System.PSCustomObject, which you don't want if you're going to open the CSV in a spreadsheet program.
If you subsequently Import-CSV, the headers (Date, Time, A, B, C) are used to create the fields of a PSObject, so that you can refer to them using the standard dot notation (e.g., $CSV[$line].Date).
The ability to specify -Header on Import-CSV is essentially a "hack" to allow the cmdlet to handle files that are comma-separated, but which did not include column headers.
I have a CSV file in which I want to change the headers names.
The current header is: name,id and I want to change it to company,transit
Following is what I wrote in script:
$a = import-csv .\finalexam\employees.csv -header name,id
foreach ($a in $as[1-$as.count-1]){
# I used 1 here because I want it to ignore the exiting headers.
$_.name -eq company, $_.id -eq transit
}
I don't think this is the correct way to do this.
You're over thinking this... All you want to do is replace the header row, so set the new header as the first item of an array, read in the file skipping the first line and add it to the array, output the array.
"Company,Transit"|Set-Content C:\Path\To\NewFile.csv
Get-Content C:\Path\To\Old.csv | Select -skip 1 | Add-Content C:\Path\To\NewFile.csv
Something very simple like this:
$file = Get-Content C:\temp\data.csv
"new,column,name" | Set-Content C:\temp\data.csv
$file | Select-Object -Skip 1 | Add-Content C:\temp\data.csv
Collect the complete file contents and then write a new header. Then restore the rest of the file content while -skiping the original header.
I'm using PowerShell To import a TAB separated file with headers. The generated file has a few empty strings "" on the end of first line of headers. PowerShell fails with an error:
"Cannot process argument because the
value of argument "name" is invalid.
Change the value of the "name"
argument and run the operation again"
because the header's require a name.
I'm wondering if anyone has any ideas on how to manipulate the file to either remove the double quotes or enumerate them with a "1" "2" "3" ... "10" etc.
Ideally I would like to not modify my original file. I was thinking something like this
$fileContents = Get-Content -Path = $tsvFileName
$firstLine = $fileContents[0].ToString().Replace('`t""',"")
$fileContents[0] = $firstLine
Import-Csv $fileContents -Delimiter "`t"
But Import-Csv is expecting $fileContents to be a path. Can I get it to use Content as a source?
You can either provide your own headers and ignore the first line of the csv, or you can use convertfrom-csv on the end like Keith says.
ps> import-csv -Header a,b,c,d,e foo.csv
Now the invalid headers in the file is just a row that you can skip.
-Oisin
If you want to work with strings instead use ConvertFrom-Csv e.g.:
PS> 'FName,LName','John,Doe','Jane,Doe' | ConvertFrom-Csv | Format-Table -Auto
FName LName
----- -----
John Doe
Jane Doe
I ended up needing to handle multiple instances of this issue. Rather than use the -Header and manually setting up each import instance I wrote a more generic method to apply to all of them. I cull out all of the `t"" instances of the first line and save the file to open as a $filename + _bak and import that one.
$fileContents = Get-Content -Path $tsvFileName
if( ([string]$fileContents[0]).ToString().Contains('""') )
{
[string]$fixedFirstLine = $fileContents[0].ToString().Replace('`t""',"")
$fileContents[0] = $fixedFirstLine
$tsvFileName = [string]::Format("{0}_bak",$tsvFileName
$fileContents | Out-File -FilePath $tsvFileName
}
Import-Csv $tsvFileName -Delimiter "`t"
My Solution if you have much columns :
$index=0
$ColumnsName=(Get-Content "C:\temp\yourCSCFile.csv" | select -First 1) -split ";" | %{
$index++
"Header_{0:d5}" -f $index
}
import-csv "C:\temp\yourCSCFile.csvv" -Header $ColumnsName