Export Log file to new CSV file in Powershell - powershell

I feel like I'm going about this the wrong way, but I've gotten myself half way there... I have a log file called DataSet.log that is formatted as:
First Dataset;24226382;2020-10-01 00:00;Second.Data.Set 1.0;0;Third.DataSet 1.0;2;Fourth.DataSet 1.0;0;Fifth.Dataset 1.0;0
First Dataset;24421469;2020-10-01 01:00;Second.Data.Set 1.0;0;Third.DataSet 1.0;4;Fourth.DataSet 1.0;0;Fifth.Dataset 1.0;0
First Dataset;24667838;2020-10-01 02:00;Second.Data.Set 1.0;0;Third.DataSet 1.0;6;Fourth.DataSet 1.0;0;Fifth.Dataset 1.0;0
First Dataset;24667839;2020-10-01 02:00;Second.Data.Set 1.0;0;Third.DataSet 1.0;1;Fourth.DataSet 1.0;0;Fifth.Dataset 1.0;0
I'm trying to convert this to a new CSV file and get it to display only the third and seventh columns. I've gotten it to display in PowerShell properly with:
Import-Csv .\DataSet.log -Header A,B,C,D,E,F,G,H,I,J,K -Delimiter ';' | Format-Table C,G
I've tried to export it as:
$testPush = Import-Csv .\DataSet.log -Header A,B,C,D,E,F,G,H,I,J,K -Delimiter ';' | Format-Table C,G
$testPush | Out-File .\test.csv
And it does create a new csv file, but it's only displaying the third column with a header of "C G." Also, that header is in A2, where A1 is blank, and A3 is populated with "- -" before the datetime from the third column populates the remaining rows... What the heck am I doing wrong?

Try this:
$testPush = Import-Csv .\DataSet.log -Header A,B,C,D,E,F,G,H,I,J,K -Delimiter ';' | Select-Object C,G
$testPush | Export-CSv -Path .\test.csv -NoTypeInformation

Related

How to handle dynamic columns for CSV file using Import-csv in PowerShell

CSV file does not have headers, Column may differ each time.
I want to compare with first column of csv file, but while exporting I would display the full row.
$getErrorCodes= {102,103}
CSV FILE( No headers)
101,101,101
102,102,102
Here is my code
Import-Csv $CSVFile -Header col1 | Where-Object {!($getErrorCodes.Contains($_.col1)) } | Export-Csv "resultfile.csv"
Current output
101
Expected output
101,101,101
For dynamic header, I used like this.
Dynamically will change $headers variable accordingly column count on csv.
$headers = 'Col1,Col2,Col3'
$headercolumns = $headers.ToString().Split(",")
Import-Csv $CSVFile -Header $headercolumns | Where-Object {!($getErrorCodes.Contains($_.col1)) } | Export-Csv "resultfile.csv"

Remove String from Character from column in CSV using Powershell

I have a CSV file containing two columns:server name with domain and date
servername.domain.domain.com,10/15/2018 6:28
servername1.domain.domain.com,10/13/2018 7:28
I need to remove the fully qualified name so it only has the shortname and I need to keep the second column so it looks as is like below either by sending to a new CSV or somehow removing the domain inplace somehow. Basically I want the second column untouched but I need it to be included when creating a new CSV with the altered column 1.
servername,10/15/2018 6:28
servername1,10/13/2018 7:28
I have this:
Import-Csv "filename.csv" -Header b1,b2 |
% {$_.b1.Split('.')[0]} |
Set-Content "filename1.csv"
This works great, but the problem is the new CSV is missing the 2nd column. I need to send the second column to the new CSV file as well.
Use a calculated property to replace the property you want changed, but leave everything else untouched:
Import-Csv 'input.csv' -Header 'b1', 'b2' |
Select-Object -Property #{n='b1';e={$_.b1.Split('.')[0]}}, * -Exclude b1 |
Export-Csv 'output.csv' -NoType
Note that you only need to use the parameter -Header if your CSV data doesn't already have a header line. Otherwise you should remove the parameter.
If your input file doesn't have headers and you want to create the output file also without headers you can't use Export-Csv, though. Use ConvertTo-Csv to create the CSV text output, then skip over the first line (to remove the headers) and write the rest to the output file with Set-Content.
Import-Csv 'input.csv' -Header 'b1', 'b2' |
Select-Object -Property #{n='b1';e={$_.b1.Split('.')[0]}}, * -Exclude b1 |
ConvertTo-Csv -NoType |
Select-Object -Skip 1 |
Set-Content 'output.csv'

Selecting columns from flat file in power shell with no column name

I am new to power shell ,and I have the below format (pipe delimiter) with no column name:
01|1|06/28/2017 00:00:00|06/28/2017 00:00:00
I want to choose the third or any column from this format,I have tried the below code :
$columns=(Get-Content $filepath | Out-String | select -Skip 2 -First 1).Split("|")
but it is not working can any one help please.
Use Import-CSV with -Header and -Delimiter specified; that way, you get a structure (PSCustomObject[]) with attributes that you can reference directly and meaningfully. For example,
$EntryList = Import-CSV -Path $FilePath -Header ID,Type,StartTime,EndTime -Delimiter '|'
gets you an array of PSCustomObjects, where each object has the indicated fields. You can then (for example) refer to $EntryList[$n].ID, $EntryList[$n].StartTime, and so on.

How to export to "non-standard" CSV with Powershell

I need to convert a file with this format:
2015.03.27,09:00,1.08764,1.08827,1.08535,1.08747,8941
2015.03.27,10:00,1.08745,1.08893,1.08604,1.08762,7558
to this format
2015.03.27,1.08764,1.08827,1.08535,1.08747,1
2015.03.27,1.08745,1.08893,1.08604,1.08762,1
I started with this code but can't see how to achieve the full transformation:
Import-Csv in.csv -Header Date,Time,O,H,L,C,V | Select-Object Date,O,H,L,C,V | Export-Csv -path out.csv -NoTypeInformation
(Get-Content out.csv) | % {$_ -replace '"', ""} | out-file -FilePath out.csv -Force -Encoding ascii
which outputs
Date,O,H,L,C,V
2015.03.27,1.08745,1.08893,1.08604,1.08762,8941
2015.03.27,1.08763,1.08911,1.08542,1.08901,7558
After that I need to
remove the header (I tried -NoHeader which is not recognized)
replace last column with 1.
How to do that as simply as possible (if possible without looping through each row)
Update : finally I have simplified requirement. I just need to replace last column with constant.
Ok, this could be one massive one-liner... I'm going to do line breaks at the pipes for sanity reasons though.
Import-Csv in.csv -header Date,Time,O,H,L,C,V|select * -ExcludeProperty time|
%{$_.date = [datetime]::ParseExact($_.date,"yyyy.MM.dd",$null).tostring("yyMMdd");$_}|
ConvertTo-Csv -NoTypeInformation|
select -skip 1|
%{$_ -replace '"'}|
Set-Content out.csv -encoding ascii
Basically I import the CSV, exclude the time column, convert the date column to an actual [datetime] object and then convert it back in the desired format. Then I pass the modified object (with the newly formatted date) down the pipe to ConvertTo-CSV, and skip the first line (your headers that you don't want), and then remove the quotes from it, and lastly output to file with Set-Content (faster than Out-File)
Edit: Just saw your update... to do that we'll just change the last column to 1 at the same time we modify the date column by adding $_.v=1;...
%{$_.date = [datetime]::ParseExact($_.date,"yyyy.MM.dd",$null).tostring("yyMMdd");$_.v=1;$_}|
Whole script modified:
Import-Csv in.csv -header Date,Time,O,H,L,C,V|select * -ExcludeProperty time|
%{$_.date = [datetime]::ParseExact($_.date,"yyyy.MM.dd",$null).tostring("yyMMdd");$_.v=1;$_}|
ConvertTo-Csv -NoTypeInformation|
select -skip 1|
%{$_ -replace '"'}|
Set-Content out.csv -encoding ascii
Oh, and this has the added benefit of not having to read the file in, write the file to the drive, read that file in, and then write the file to the drive again.

Powershell how to add data in the csv file to one column

I have a csv file with a few columns. For example
col1;col2;col3;col4
text1;text2;text3;text4
text5;text6;text7;text8
text9;text10;text11;text12
I want to add text to column col3, like this:
col1;col2;col3;col4
text1;text2;append\text3;text4
text5;text6;append\text7;text8
text9;text10;append\text11;text12
So question is:
How to do this?
(I'm stuck whit that how I add data to each column in col3.)
Give this a whirl:
Import-Csv -Delim ';' cols.csv |
ForEach-Object {$_.col3 = "prepend\$($_.col3)";$_} |
Export-Csv cols2.csv -Delim ';' -NoTypeInformation
Use the -NoTypeInformation parameter to avoid this comment getting put at the top of your CSV:
#TYPE System.Management.Automation.PSCustomObject
However, if you don't mind the comment then you can leave off the -NoTypeInformation parameter.
You probably know how to read/write data. To change the data, send them to Foreach-Object, alter the data and pass the object further to Export-Csv.
Import-Csv d:\temp\so\csv1.txt -Delimiter ';' |
ForEach-Object { $_.col3 = 'append\' +$_.col3; $_ } |
Export-Csv d:\temp\so\csv2.txt -Delimiter ';'