I have a CSV file containing two columns:server name with domain and date
servername.domain.domain.com,10/15/2018 6:28
servername1.domain.domain.com,10/13/2018 7:28
I need to remove the fully qualified name so it only has the shortname and I need to keep the second column so it looks as is like below either by sending to a new CSV or somehow removing the domain inplace somehow. Basically I want the second column untouched but I need it to be included when creating a new CSV with the altered column 1.
servername,10/15/2018 6:28
servername1,10/13/2018 7:28
I have this:
Import-Csv "filename.csv" -Header b1,b2 |
% {$_.b1.Split('.')[0]} |
Set-Content "filename1.csv"
This works great, but the problem is the new CSV is missing the 2nd column. I need to send the second column to the new CSV file as well.
Use a calculated property to replace the property you want changed, but leave everything else untouched:
Import-Csv 'input.csv' -Header 'b1', 'b2' |
Select-Object -Property #{n='b1';e={$_.b1.Split('.')[0]}}, * -Exclude b1 |
Export-Csv 'output.csv' -NoType
Note that you only need to use the parameter -Header if your CSV data doesn't already have a header line. Otherwise you should remove the parameter.
If your input file doesn't have headers and you want to create the output file also without headers you can't use Export-Csv, though. Use ConvertTo-Csv to create the CSV text output, then skip over the first line (to remove the headers) and write the rest to the output file with Set-Content.
Import-Csv 'input.csv' -Header 'b1', 'b2' |
Select-Object -Property #{n='b1';e={$_.b1.Split('.')[0]}}, * -Exclude b1 |
ConvertTo-Csv -NoType |
Select-Object -Skip 1 |
Set-Content 'output.csv'
Related
I feel like I'm going about this the wrong way, but I've gotten myself half way there... I have a log file called DataSet.log that is formatted as:
First Dataset;24226382;2020-10-01 00:00;Second.Data.Set 1.0;0;Third.DataSet 1.0;2;Fourth.DataSet 1.0;0;Fifth.Dataset 1.0;0
First Dataset;24421469;2020-10-01 01:00;Second.Data.Set 1.0;0;Third.DataSet 1.0;4;Fourth.DataSet 1.0;0;Fifth.Dataset 1.0;0
First Dataset;24667838;2020-10-01 02:00;Second.Data.Set 1.0;0;Third.DataSet 1.0;6;Fourth.DataSet 1.0;0;Fifth.Dataset 1.0;0
First Dataset;24667839;2020-10-01 02:00;Second.Data.Set 1.0;0;Third.DataSet 1.0;1;Fourth.DataSet 1.0;0;Fifth.Dataset 1.0;0
I'm trying to convert this to a new CSV file and get it to display only the third and seventh columns. I've gotten it to display in PowerShell properly with:
Import-Csv .\DataSet.log -Header A,B,C,D,E,F,G,H,I,J,K -Delimiter ';' | Format-Table C,G
I've tried to export it as:
$testPush = Import-Csv .\DataSet.log -Header A,B,C,D,E,F,G,H,I,J,K -Delimiter ';' | Format-Table C,G
$testPush | Out-File .\test.csv
And it does create a new csv file, but it's only displaying the third column with a header of "C G." Also, that header is in A2, where A1 is blank, and A3 is populated with "- -" before the datetime from the third column populates the remaining rows... What the heck am I doing wrong?
Try this:
$testPush = Import-Csv .\DataSet.log -Header A,B,C,D,E,F,G,H,I,J,K -Delimiter ';' | Select-Object C,G
$testPush | Export-CSv -Path .\test.csv -NoTypeInformation
I filtered by date this file data1.csv
2017.11.1,09:55,1.1,1.2,1.3,1.4,1
2017.11.2,09:55,1.5,1.6,1.7,1.8,2
I don't get a header with -NoTypeInformation:
$CutOff = (Get-Date).AddDays(-2)
$filePath = "data1.csv"
$Data = Import-Csv $filePath -Header Date,Time,A,B,C,D,E
$Data2 = $Data | Where-Object {$_.Date -as [datetime] -gt $Cutoff} | convertto-csv -NoTypeInformation -Delimiter "," | % {$_ -replace '"',''}
But when rewriting with Out-File
$Data2 | Out-File "data2.csv" -Encoding utf8 -Force
I get header back as data2.csv contains:
Date,Time,A,B,C,D,E
2017.11.2,09:55,1.5,1.6,1.7,1.8,2
Why do I have Date,Time,A,B,C,D,E ?
-NoTypeInformation is not about the header but the data type of the rows in the file. Remove it to see what shows up. From Microsoft
Omits the type information header from the output. By default, the string in the output contains #TYPE followed by the fully-qualified name of the object type.
Emphasis mine.
CSVs need headers. That is why it is making one. If you don't want to see the header in the output use Select-Object -Skip 1 to remove it.
$Data |
Where-Object {$_.Date -as [datetime] -gt $Cutoff} |
ConvertTo-CSV -NoTypeInformation -Delimiter "," |
Select-Object -Skip 1 |
% {$_ -replace '"'}
I would not pipe Out-File to itself. You could pipe to Set-Content here just as well.
I am guessing this whole process is to keep the source file in the same state just with some lines filtered out based on date. You could skip most of this just by parsing the date out in each line.
$threshold = (Get-Date).AddDays(-2)
$filePath = "c:\temp\bagel.txt"
(Get-Content $filePath) | Where-Object{
$date,$null=$_.Split(",",2)
[datetime]$date -gt $threshold
} | Set-Content $filePath
Now you don't have to worry about PowerShell CSV object structure or output since we act on the raw data of the file itself.
That will take each line of the input file and filter it out if the parsed date does not match the threshold. Change encoding on the input output cmdlets as you see necessary. What $date,$null=$_.Split(",",2) is doing is splitting the line
on the comma into 2 parts. First of which becomes $date and since this is just a filtering condition we dump the rest of the line into $null.
Properly-formed CSV files must have column headers. Your use of -NoTypeInformation in generating the CSV does not affect column headers; instead, it affects whether the PowerShell object type information is included. If you Export-CSV without -NoTypeInformation, the first line of your CSV file will have a line that looks like #TYPE System.PSCustomObject, which you don't want if you're going to open the CSV in a spreadsheet program.
If you subsequently Import-CSV, the headers (Date, Time, A, B, C) are used to create the fields of a PSObject, so that you can refer to them using the standard dot notation (e.g., $CSV[$line].Date).
The ability to specify -Header on Import-CSV is essentially a "hack" to allow the cmdlet to handle files that are comma-separated, but which did not include column headers.
I am new to power shell ,and I have the below format (pipe delimiter) with no column name:
01|1|06/28/2017 00:00:00|06/28/2017 00:00:00
I want to choose the third or any column from this format,I have tried the below code :
$columns=(Get-Content $filepath | Out-String | select -Skip 2 -First 1).Split("|")
but it is not working can any one help please.
Use Import-CSV with -Header and -Delimiter specified; that way, you get a structure (PSCustomObject[]) with attributes that you can reference directly and meaningfully. For example,
$EntryList = Import-CSV -Path $FilePath -Header ID,Type,StartTime,EndTime -Delimiter '|'
gets you an array of PSCustomObjects, where each object has the indicated fields. You can then (for example) refer to $EntryList[$n].ID, $EntryList[$n].StartTime, and so on.
I'm trying to see if there is a way to read the column values in a csv file based on the column location. The reason for this is the file I'm being handed always has it's titles being changed...
For example, lets say csv file column A (via excel) looks like the following:
ColumnOne
ValueOne
ValueTwo
ValueThree
Now the user changes the title:
Column 1
ValueOne
ValueTwo
ValueThree
Now I want to create an array of the first column. Normally what I do is the following:
$arrayFirstColumn = Import-Csv 'C:\test\test1.csv' | where-object {$_.ColumnOne} | select-object -expand 'ColumnOne'
However, as we can see if ColumnOne is changed to Column 1, it breaks this code. How can I create this array to allow an interchangeable column title, but the column location will always be the same?
You can specify headers of your own on import:
Import-Csv 'C:\path\to\your.csv' -Header 'MyHeaderA','MyHeaderB',...
As long as you don't export the data back to a CSV (or don't require the original headers to be in the output CSV as well) you can use whatever names you like. You can also specify as many header names as you like. If their number is less than the number of the columns in the CSV the additional columns will be omitted, if it's greater then the columns for the additional headers will be empty.
If you need to preserve the original headers you could get the header name(s) you need to work with in variable(s) like this:
$csv = Import-Csv 'C:\test\test1.csv'
$firstCol = $csv | Select-Object -First 1 | ForEach-Object {
$_.PSObject.Properties | Select-Object -First 1 -Expand Name
}
$arrayFirstColumn = $csv | Where-Object {$_.$firstCol} |
Select-Object -Expand $firstCol
Or you could simply read the first line from the CSV and split it to get an array with the headers:
$headers = (Get-Content 'C:\test\test1.csv' -TotalCount 1) -split ','
$firstCol = $headers[0]
One option:
$ImportFile = 'C:\test\test1.csv'
$FirstColumn = ((Get-Content $ImportFile -TotalCount 2 | ConvertFrom-Csv).psobject.properties.name)[0]
$FirstColumn
$arrayFirstColumn = Import-Csv $ImportFile | where-object {$_.$FirstColumn} | select-object -expand $FirstColumn
If you are using PowerShell v2.0 then the expression for $FirstColumn in $mjolinor's answer would be:
$FirstColumn = ((Get-Content $ImportFile -TotalCount 2 | ConvertFrom-Csv).psobject.properties | ForEach-Object {$_.name})[0]
(Apologies for starting a new answer; I do not yet have enough reputation to add a comment to mjolinor's post)
I have a CSV file with about 10 columns separated with a ; (semicolon). I would like to add another column which generates a hashkey for the first columns value.
Is there a possibility in Powershell to do this? Also are there short haskeys (up to 10 to 15 chars)?
Example:
Old:
10000;value2;value3....
New:
HashkeyOf10000;1000;value2;value3...
You can use a calculated property for adding a column to a CSV:
$csv = 'C:\path\to\your.csv'
(Import-Csv $csv -Delimiter ';') |
select -Property #{n='Hashkey';e={Calc-Hash $_.A}},* |
Export-Csv $csv -Delimiter ';' -NoType
Replace Calc-Hash with the actual name of your hash function and A with the actual name of the first column of your CSV.
The parentheses around Import-Csv are required to ensure that reading the file is completed before writing the output starts.