I am importing a CSV which has 5 "columns". One of them is a date and time. The 3rd party software that is reading this column, then does not sort the date well.
IE: (4/8/2022 1:24:08 PM) will sort above (4/13/2022 8:51:52 AM)
Even though 4/13 is after 4/8 it will not sort it properly. I would like to add a leading zero in front of the month and date with powershell. I did do some searching but nothing seems to make sense to me, I am not a HUGE programmer.
Thanks for any help!
This is what I am currently doing. I am using unique to remove duplicate rows (this is needed for what I am doing).
$FinalSessions = Import-Csv -Path "C:\Windows\AdminArsenal\UserSessions.csv" | Sort-Object * -Unique
$FinalSessions | Export-Csv -Path "C:\Windows\AdminArsenal\UserSessions.csv" -NoTypeInformation
$FinalSessions
You can use Get-Date to actually get a datetime object and then reformat it.
It would look something like this:
$FinalSessions = Import-Csv -Path "C:\Windows\AdminArsenal\UserSessions.csv"| Sort-Object * -Unique
$FinalSessions | % { $_.DateColumn = Get-Date $_.DateColumn -Format "MM/dd/yyyy hh:mm:ss tt" }
$FinalSessions | Export-Csv -Path
"C:\Windows\AdminArsenal\UserSessions.csv" -NoTypeInformation
$FinalSessions
Just replace "DateColumn" with the name of your column
Assuming that the column that contains the date-time string is named Date (adjust as needed):
Import-Csv -Path C:\Windows\AdminArsenal\UserSessions.csv |
ForEach-Object { $_.Date = ([datetime] $_.Date).ToString('s') } |
Sort-Object * -Unique -OutVariable finalSessions |
Export-Csv -Path C:\Windows\AdminArsenal\UserSessions.csv -NoTypeInformation
$finalSessions
Note that the s format specifier (in ISO 8601 format) is used to reformat the date-time strings, as that results in a string whose lexical sorting reliably indicates chronological order, across year boundaries; e.g. 2022-05-05T17:52:47
Related
I have a csv file with a timestamp column that has a custom number formatting in this format mm:ss.0.
I am trying to change that custom number formatting to h:mm:ss AM/PM, preferably during the Import-CSV command, but it doesn't have to be.
For example, 42:00.5 should be 12:42:00 AM.
Most of the examples I've seen had dates and times and could easily be converted using ToString and Get-Date but I couldn't get those examples working with just a time stamp.
Here's a few examples of what I've tried:
Import-Csv foo.csv |
Select *,#{label = 'timestamp';expression={get-date $_.Date -f 'h:mm:ss tt'}} -ExcludeProperty timestamp |
Export-Csv -NoTypeInformation foo2.cs
Import-Csv foo.csv |
Select-Object #{Name = 'timestamp'; Expression = { '{h:mm:ss tt}' -f [datetime]$_.'timestamp' }}, * -ExcludeProperty 'timestamp' |
Export-Csv -NoTypeInformation foo2.csv
What am I missing?
This is where I would go in Excel to change the custom number format. The selected is what the current number format is and the highlighted is what I need.
Everyday a file is being saved with the current date in format testDDMMYYY.csv for an example test24112017.csv.
I need to open the current day's file with Powershell, however, I haven't found a way for it to work with variables.
#Date (24112017)
[String]$a=(Get-Date).ToShortDateString() | Foreach-Object {$_ -replace "\.", ""}
#Open the file called test24112017
[int] $S1=Get-Content "D:\test$($a).csv" -Tail 1 | Foreach-Object {$_ -replace "`"", ""} | ForEach-Object{$_.Split(",")[1]} | write-host
How can I get that variable working inside that path?
Do not use (Get-Date).ToShortDateString() | Foreach-Object {$_ -replace "\.", ""}, just use Get-Date -Format 'ddMMyyyy' to format the date the way you want :
Get-Content "D:\test$(Get-Date -Format 'ddMMyyyy').csv" -Tail 1
Formatting Dates and Times
Hmm aren't you over-complicating things a bit?
First of all,
(Get-Date).ToShortDateString() doesn't give me 24112017 but 24-11-2017.
So i'm guessing we have different regional settings, what part of the world are you from?
Have you printed out $a? What does it give you?
I would go for a different approach. Since your filename is literally always newer than the previous one (because it holds a date).
I would just sort on the Name and select the first one (aka the newest file).
$newestfile = Get-ChildItem c:\temp | Sort-Object Name -Desc | select * -First 1
Now go do whatever you want with your latest file.
get-content $newestfile.fullname
I filtered by date this file data1.csv
2017.11.1,09:55,1.1,1.2,1.3,1.4,1
2017.11.2,09:55,1.5,1.6,1.7,1.8,2
I don't get a header with -NoTypeInformation:
$CutOff = (Get-Date).AddDays(-2)
$filePath = "data1.csv"
$Data = Import-Csv $filePath -Header Date,Time,A,B,C,D,E
$Data2 = $Data | Where-Object {$_.Date -as [datetime] -gt $Cutoff} | convertto-csv -NoTypeInformation -Delimiter "," | % {$_ -replace '"',''}
But when rewriting with Out-File
$Data2 | Out-File "data2.csv" -Encoding utf8 -Force
I get header back as data2.csv contains:
Date,Time,A,B,C,D,E
2017.11.2,09:55,1.5,1.6,1.7,1.8,2
Why do I have Date,Time,A,B,C,D,E ?
-NoTypeInformation is not about the header but the data type of the rows in the file. Remove it to see what shows up. From Microsoft
Omits the type information header from the output. By default, the string in the output contains #TYPE followed by the fully-qualified name of the object type.
Emphasis mine.
CSVs need headers. That is why it is making one. If you don't want to see the header in the output use Select-Object -Skip 1 to remove it.
$Data |
Where-Object {$_.Date -as [datetime] -gt $Cutoff} |
ConvertTo-CSV -NoTypeInformation -Delimiter "," |
Select-Object -Skip 1 |
% {$_ -replace '"'}
I would not pipe Out-File to itself. You could pipe to Set-Content here just as well.
I am guessing this whole process is to keep the source file in the same state just with some lines filtered out based on date. You could skip most of this just by parsing the date out in each line.
$threshold = (Get-Date).AddDays(-2)
$filePath = "c:\temp\bagel.txt"
(Get-Content $filePath) | Where-Object{
$date,$null=$_.Split(",",2)
[datetime]$date -gt $threshold
} | Set-Content $filePath
Now you don't have to worry about PowerShell CSV object structure or output since we act on the raw data of the file itself.
That will take each line of the input file and filter it out if the parsed date does not match the threshold. Change encoding on the input output cmdlets as you see necessary. What $date,$null=$_.Split(",",2) is doing is splitting the line
on the comma into 2 parts. First of which becomes $date and since this is just a filtering condition we dump the rest of the line into $null.
Properly-formed CSV files must have column headers. Your use of -NoTypeInformation in generating the CSV does not affect column headers; instead, it affects whether the PowerShell object type information is included. If you Export-CSV without -NoTypeInformation, the first line of your CSV file will have a line that looks like #TYPE System.PSCustomObject, which you don't want if you're going to open the CSV in a spreadsheet program.
If you subsequently Import-CSV, the headers (Date, Time, A, B, C) are used to create the fields of a PSObject, so that you can refer to them using the standard dot notation (e.g., $CSV[$line].Date).
The ability to specify -Header on Import-CSV is essentially a "hack" to allow the cmdlet to handle files that are comma-separated, but which did not include column headers.
I'm attempting to format some dates in the first column of a CSV. I would prefer to user something like powershell as I plan to automate this task. Does anyone have any advice on the best way to change the format of the date from something like MM/DD/YYY to YYYY-MM-DD? I've tried something like this:
$date = date -f ('yyyyMMdd')
$HMDA = Import-Csv "C:\HMDA\$date.YieldTableFixed.csv"
ForEach-Object {
$HMDA.Date = [datetime]::ParseExact($HMDA.Date).ToString('YYYY-MM-DD')
} |
Export-Csv -NoTypeInformation C:\HMDA\test.csv
Unfortunately, that didn't seem to do anything but give me a parse error and I can't seem to figure out why that is. Is there a way I can say something like:
ForEach-Object{
$HMDA.A2:$HMDA.A63 = HMDA.$AC.Date.Format('YYYY-MM-DD')
}
Ok, there's some basic errors here, but that's just a matter of not knowing better I think. Now this is hard to answer accurately because you did not give us an example of the incoming date field, so if it has some strange formatting this may throw errors as PowerShell fails to recognize that a string is in fact a date.
First off, if you pipe to a ForEach loop you reference the current object with $_. Such as:
Import-Csv "C:\HMDA\$date.YieldTableFixed.csv" | ForEach-Object {
$_.Date = get-date $_.Date -f 'yyyy-MM-dd'
} | Export-Csv -NoTypeInformation C:\HMDA\test.csv
What would probably be simpler, as I recently learned from somebody else here on SO, would be to use Select, create the updated property on the fly, and then exclude the original property, effectively replacing it with the new one. Something like this:
Import-Csv "C:\HMDA\$date.YieldTableFixed.csv" |
Select *,#{label = 'Date';expression={get-date $_.Date -f 'yyyy-MM-dd'}} -ExcludeProperty Date |
Export-Csv -NoTypeInformation C:\HMDA\test.csv
ParseExact() expects 3 parameters: the date string, a format string, and a format provider (which may be $null). Also, your output format string is incorrect (the year and day format specifiers need to be lowercase), and ForEach-Object reads from a pipeline.
Change this:
$HMDA = Import-Csv "C:\HMDA\$date.YieldTableFixed.csv"
ForEach-Object {
$HMDA.Date = [datetime]::ParseExact($HMDA.Date).ToString('YYYY-MM-DD')
} |
Export-Csv -NoTypeInformation C:\HMDA\test.csv
into this:
Import-Csv 'C:\HMDA\$date.YieldTableFixed.csv' | ForEach-Object {
$_.Date = [DateTime]::ParseExact($_.Date, '*M\/dd\/yyyy', $null).ToString('yyyy-MM-dd')
$_
} | Export-Csv -NoTypeInformation 'C:\HMDA\test.csv'
I have two CSV file with 50 column and more than 10K row. There is a column having date-time value. In some records it is 01/18/2013 18:16:32 and some records is 16/01/2014 17:32.
What I want to convert the column data like this: 01/18/2013 18:16. I want to remove seconds value. I want to do it by PowerShell script.
Sample data:
10/1/2014 13:18
10/1/2014 13:21
15/01/2014 12:03:19
15/01/2014 17:39:27
15/01/2014 18:29:44
17/01/2014 13:33:59
Since you're not going to convert to a sane date format anyway, you can just do a regex replace of that column:
Import-Csv foo.csv |
ForEach-Object {
$_.Date = $_.Date -replace '(\d+:\d+):\d+', '$1'
} |
Export-Csv -NoTypeInformation foo-new.csv
You could also go the route of using date parsing, but that's probably a bit slower:
Import-Csv foo.csv |
ForEach-Object {
$_.Date = [datetime]::Parse($_.Date).ToString('MM/dd/yyyy HH:mm')
} |
Export-Csv -NoTypeInformation foo-new.csv
If you're sure that there are no other timestamps or anything that could look like it elsewhere, you can also just replace everything that looks like it without even parsing as CSV:
$csv = Get-Content foo.csv -ReadCount 0
$csv = $csv -replace '(\d{2}/\d{2}/\d{4} \d{2}:\d{2}):\d{2}', '$1'
$csv | Out-File foo-new.csv