Edit csv column - powershell
I have a csv with 3 columns.
Date Time Event
12/19/2021 3:00pm Low
12/19/2021 1:30pm Low
12/20/2021 3:00pm Low
I'm trying to subtract 5 hours from each row.
How can I keep each object in a datetime format?
$time0 = Import-Csv $Tempdest | Select 'Time'
$timeEst = $time0 | ForEach-Object {
Write-Host $_.
([datetime]$_).AddHours(-5)
}
By looking at the provided screenshot this should work, it would be updating the imported CSV in memory. If it doesn't work it's best if you can share with us the CSV as plain text.
$csv = Import-Csv path/to/csv.csv
$csv | ForEach-Object {
$date = '{0} {1}' -f $_.Date, $_.Time -as [datetime]
$date = $date.AddHours(-5)
$_.Date = $date.ToShortDateString()
$_.Time = $date.ToShortTimeString()
}
$csv | Format-Table
Related
PowerShell ForEach-Object Only Matching first Value passed
I am writing a PowerShell script that reads a CSV file in the following format:
A,B,ProgrammeSeller,D,E,F,G,H,I,J,K,L,M,N,O,P,SellerCode,Date,Seller,Currency1,InvoiceAmmount,DiscountAmount,PurchasePrice,TotalsCurrency,TotalInvoiceAmmount,TotalDiscountAmmount,TotalPurchasePrice,Reviewed,AC,Signed
,,PROGRAMME,,,,,,,,,,,,,,380,09/12/2021,SELLER_BE,EUR,2813828.46,17594.22,2796234.24,EUR,0,0,0,Reviewed by:,,Signed:
,,PROGRAMME,,,,,,,,,,,,,,383,09/12/2021,SELLER_DE,EUR,3287812.58,17595.8,3270216.78,EUR,0,0,0,Reviewed by:,,Signed:
,,PROGRAMME,,,,,,,,,,,,,,383,09/12/2021,SELLER_DE,USD,1520725.4,11428.98,1509296.42,USD,0,0,0,Reviewed by:,,Signed:
,,PROGRAMME,,,,,,,,,,,,,,381,09/12/2021,SELLER_DK,DKK,6047281.25,26163.13,6021118.12,DKK,0,0,0,Reviewed by:,,Signed:
,,PROGRAMME,,,,,,,,,,,,,,381,09/12/2021,SELLER_DK,EUR,11376479.39,39580.28,11336899.11,EUR,0,0,0,Reviewed by:,,Signed:
,,PROGRAMME,,,,,,,,,,,,,,381,09/12/2021,SELLER_DK,USD,12571895.13,71198.51,12500696.62,USD,0,0,0,Reviewed by:,,Signed:
And I want to group and sum 3 of the columns (InvoiceAmmount,DiscountAmount,PurchasePrice), and update the columns (TotalInvoiceAmmount,TotalDiscountAmmount,TotalPurchasePrice) on a new file which is a copy of the original but with the columns (TotalInvoiceAmmount,TotalDiscountAmmount,TotalPurchasePrice) updated with the values of the sums whenever there is a match on Currency value.
I have written the following script:
$csvFile = Import-Csv "C:\OutputFiles\OTC_Purchase_Report_EUProgramme_20220420_TMP.csv"
$csvFinal = "C:\OutputFiles\OTC_Purchase_Report_EUProgramme_20220420.csv"
function WriteTotalsToCSV
{
$totals | ForEach-Object {
$totalCurrency = $_.Currency
$totalInvoiceAmmount = $_.TotalInvoiceAmmount
$totalDiscountAmmount = $_.TotalDiscountAmmount
$totalPurchasePrice = $_.TotalPurchasePrice
$csvFile | ForEach-Object {
if($_.Currency1 -eq $totalCurrency){
$_.TotalsCurrency = $totalCurrency
$_.TotalInvoiceAmmount = $totalInvoiceAmmount
$_.TotalDiscountAmmount = $totalDiscountAmmount
$_.TotalPurchasePrice = $totalPurchasePrice
}
$_ | Export-Csv $csvFinal -NoTypeInformation -Append
}
}
}
function InvoiceAmmountTotals
{
param ($file)
$headers = ($file | Get-Member -MemberType NoteProperty).Name
$totals = $file | Select-Object $headers | Group-Object Currency1
foreach ($total in $totals)
{
$showtotal = New-Object System.Management.Automation.PsObject
$showtotal | Add-Member NoteProperty Currency $total.Name
$showtotal | Add-Member NoteProperty TotalInvoiceAmmount ($total.Group | Measure-Object -Sum InvoiceAmmount).Sum
$showtotal | Add-Member NoteProperty TotalDiscountAmmount ($total.Group | Measure-Object -Sum DiscountAmount).Sum
$showtotal | Add-Member NoteProperty TotalPurchasePrice ($total.Group | Measure-Object -Sum PurchasePrice).Sum
$showtotal
}
}
#execution
$totals = InvoiceAmmountTotals $csvFile
WriteTotalsToCSV
The script is grouping and summing the totals as expected but when it writes the new CSV file it is supposed to update the columns based on a match of column Currency1, but it is only doing this for the first match (in this case EUR) and ignoring the remaining matches i.e.:
"A","B","ProgrammeSeller","D","E","F","G","H","I","J","K","L","M","N","O","P","SellerCode","Date","Seller","Currency1","InvoiceAmmount","DiscountAmount","PurchasePrice","TotalsCurrency","TotalInvoiceAmmount","TotalDiscountAmmount","TotalPurchasePrice","Reviewed","AC","Signed"
"","","PROGRAMME","","","","","","","","","","","","","","380","09/12/2021","SELLER_BE","EUR","2813828.46","17594.22","2796234.24","EUR","153995434.65","797318.07","153198116.58","Reviewed by:","","Signed:"
"","","PROGRAMME","","","","","","","","","","","","","","383","09/12/2021","SELLER_DE","EUR","3287812.58","17595.8","3270216.78","EUR","153995434.65","797318.07","153198116.58","Reviewed by:","","Signed:"
"","","PROGRAMME","","","","","","","","","","","","","","383","09/12/2021","SELLER_DE","USD","1520725.4","11428.98","1509296.42","USD","0","0","0","Reviewed by:","","Signed:"
"","","PROGRAMME","","","","","","","","","","","","","","381","09/12/2021","SELLER_DK","DKK","6047281.25","26163.13","6021118.12","DKK","0","0","0","Reviewed by:","","Signed:"
"","","PROGRAMME","","","","","","","","","","","","","","381","09/12/2021","SELLER_DK","EUR","11376479.39","39580.28","11336899.11","EUR","153995434.65","797318.07","153198116.58","Reviewed by:","","Signed:"
"","","PROGRAMME","","","","","","","","","","","","","","381","09/12/2021","SELLER_DK","USD","12571895.13","71198.51","12500696.62","USD","0","0","0","Reviewed by:","","Signed:"
Note when column Currency1 value is USD the columns (TotalInvoiceAmmount,TotalDiscountAmmount,TotalPurchasePrice) are not being updated.
Any suggestions on where I am going wrong here?
Note: The CSV files are much larger, I have added a small number of entries for example purpose
Thanks
It looks like you are looping through each line of the csv as many times as you have currencies because you are looping through $totals and inside of each of those loops you are looping through each line of the csv causing duplicates.
Loop only once through the csv lines and for each line find the matching currency in your $totals array to update each csv line with. Finally output all at once to Export-Csv
function WriteTotalsToCSV {
# only one loop through csv lines
$csvFile | ForEach-Object {
$line = $_
# find matching currency in your $totals array using Where()
$totalsMatch = $totals.Where({ $line.Currency1 -eq $_.Currency })
$line.TotalsCurrency = $totalsMatch.Currency
$line.TotalInvoiceAmmount = $totalsMatch.TotalInvoiceAmmount
$line.TotalDiscountAmmount = $totalsMatch.TotalDiscountAmmount
$line.TotalPurchasePrice = $totalsMatch.TotalPurchasePrice
$line
# collect all the lines first and then export to csv once at the end
} | Export-Csv -Path $csvFinal -NoTypeInformation
}
Get a logfile for a specific date
I want to save in my computer "C:\logFiles" a specific date for logfile generated by program in another PC,
path that i will get from it the log file is "C:\Sut\Stat\03-2021.log"
Example : this file "C:\Sut\Stat\03-2021.Sutwin.log" contenant all the log of Mars month but i just want to get the log of last 7 Days from 19-03-2021 to 26-03-2021
I found this script in the internet but i doesn't work for me i need some help:
Example of the file .log in the photo attached:
Rest of image for the first screenshot :
my PC name : c01234
name of PC contenant log file : c06789
file that i will get from it the infos : 03-2021.Sutwin.log (exist in pc c06789)
i want to transfer the contents of just last 7 days in a folder in my PC c01234 with name Week11_LogFile
$log = "2015-05-09T06:39:34 Some information here
2015-05-09T06:40:34 Some information here
" -split "`n" | Where {$_.trim()}
#using max and min value for the example so all correct dates will comply
$upperLimit = [datetime]::MaxValue #replace with your own date
$lowerLimit = [datetime]::MinValue #replace with your own date
$log | foreach {
$dateAsText = ($_ -split '\s',2)[0]
try
{
$date = [datetime]::Parse($dateAsText)
if (($lowerLimit -lt $date) -and ($date -lt $upperLimit))
{
$_ #output the current item because it belongs to the requested time frame
}
}
catch [InvalidOperationException]
{
#date is malformed (maybe the line is empty or there is a typo), skip it
}
}
Based on your images, your log files look like simple tab-delimited files.
Assuming that's the case, this should work:
# Import the data as a tab-delimited file and add a DateTime column with a parsed value
$LogData = Import-Csv $Log -Delimiter "`t" |
Select-Object -Property *, #{n='DateTime';e={[datetime]::ParseExact($_.Date + $_.Time, 'dd. MMM yyHH:mm:ss', $null)}}
# Filter the data, drop the DateTime column, and write the output to a new tab-delimited file
$LogData | Where-Object { ($lowerLimit -lt $_.DateTime) -and ($_.DateTime -lt $upperLimit) } |
Select-Object -ExcludeProperty DateTime |
Export-Csv $OutputFile -Delimiter "`t"
The primary drawback here is that on Windows Powershell (v5.1 and below) you can only export the data quoted. On Powershell 7 and higher you can use -UseQuotes Never to prevent the fields from being double quote identified if that's important.
The only other drawback is that if these log files are huge then it will take a long time to import and process them. You may be able to improve performance by making the above a one-liner like so:
Import-Csv $Log -Delimiter "`t" |
Select-Object -Property *, #{n='DateTime';e={[datetime]::ParseExact($_.Date + $_.Time, 'dd. MMM yyHH:mm:ss', $null)}} |
Where-Object { ($lowerLimit -lt $_.DateTime) -and ($_.DateTime -lt $upperLimit) } |
Select-Object -ExcludeProperty DateTime |
Export-Csv $OutputFile -Delimiter "`t"
But if the log files are extremely large then you may run into unavoidable performance problems.
It's a shame your example of a line in the log file does not reveal the exact date format.
2015-05-09 could be yyyy-MM-dd or yyyy-dd-MM, so I'm guessing it's yyyy-MM-dd in below code..
# this is the UNC path where the log file is to be found
# you need permissions of course to read that file from the remote computer
$remotePath = '\\c06789\C$\Sut\Stat\03-2021.log' # or use the computers IP address instead of its name
$localPath = 'C:\logFiles\Week11_LogFile.log' # the output file
# set the start date for the week you are interested in
$startDate = Get-Date -Year 2021 -Month 3 -Day 19
# build an array of formatted dates for an entire week
$dates = for ($i = 0; $i -lt 7; $i++) { '{0:yyyy-MM-dd}' -f $startDate.AddDays($i) }
# create a regex string from that using an anchor '^' and the dates joined with regex OR '|'
$regex = '^({0})' -f ($dates -join '|')
# read the log file and select all lines starting with any of the dates in the regex
((Get-Content -Path $remotePath) | Select-String -Pattern $regex).Line | Set-Content -Path $localPath
Datetime in PowerShell can't convert my Time
I'm writing a Script to compare two sets of timevalues and then calculate a exact time. My problem is the calculation with timestamps. I import the times from a .csv-file. The times look like this: 08:37; 11:47; 12:11; 17:34; etc. I made a variable for the times so i always have the correct time from the correct line from the csv file. My goal ist to calculate the time from one timestamp to another like this: 11:47 - 08:37 = 3:10 If i do this in my PowerShell Script an error occurs: The value "time=12:39" can not be converted to type "System.DateTime". Error: "The string was not recognized as a DateTime. An unknown word starts at index 1" Is datetime wrong in this case? How can i make this work? Thx for your help.
If this has to do with your previous question and the CSV actually looks like this:
name;prename;date;time
Gantz;Mario;09.02.;07:37
Gantz;Mario;09.02.;11:23
Gantz;Mario;09.02.;12:34
Gantz;Mario;09.02.;17:03
Then this should do it
# create two variables to hold the times parsed from the CSV, Initialize to $null
$current, $previous = $null
# load the csv and loop through the records
$result = Import-Csv -Path 'D:\Test\times.csv' -Delimiter ';' | ForEach-Object {
$current = [datetime]::ParseExact($_.time, 'HH:mm', $null)
if (!$previous) { $previous = $current }
# subtracting two DateTime objects results in a TimeStamp
$elapsed = $current - $previous
$previous = $current
# output the record with column 'elapsed' appended
$_ | Select-Object *, #{Name = 'elapsed'; Expression = {$elapsed.ToString()}}
}
# output on screen
$result | Format-Table -AutoSize
# output to new CSV file
$result | Export-Csv -Path 'D:\Test\times_and_intervals.csv' -Delimiter ';' -NoTypeInformation
Output on screen:
name prename date time elapsed
---- ------- ---- ---- -------
Gantz Mario 09.02. 07:37 00:00:00
Gantz Mario 09.02. 11:23 03:46:00
Gantz Mario 09.02. 12:34 01:11:00
Gantz Mario 09.02. 17:03 04:29:00
Now that I see you also have a 'date' column in there, you should include that in the conversion to [datetime] aswell:
# create two variables to hold the times parsed from the CSV, Initialize to $null
$current, $previous = $null
# load the csv and loop through the records
$result = Import-Csv -Path 'D:\Test\times.csv' -Delimiter ';' | ForEach-Object {
$completeDate = '{0}{1} {2}' -f $_.date, (Get-Date).Year, $_.time
$current = [datetime]::ParseExact($completeDate, 'dd.MM.yyyy HH:mm', $null)
if (!$previous) { $previous = $current }
# subtracting two DateTime objects results in a TimeStamp
$elapsed = $current - $previous
$previous = $current
# output the record with column 'elapsed' appended
$_ | Select-Object *, #{Name = 'elapsed'; Expression = {$elapsed.ToString()}}
}
# output on screen
$result | Format-Table -AutoSize
# output to new CSV file
$result | Export-Csv -Path 'D:\Test\times_and_intervals.csv' -Delimiter ';' -NoTypeInformation
You are getting the error because you are not specifying the values that you are importing as [datetime] I have replicated the error where I just specified 2 time values and subtracted them: $st = "08:37" $et = "11:47" $di = $st - $et Cannot convert value "08:37" to type "System.Int32". Error: "Input string was not in a correct format." Solution: Specify the values of each entry like so: [datetime]$starttime = "08:37" [datetime]$endtime = "11:47" $diff = $endtime - $starttime If you just want the time in minutes etc. you can enter $diff.Minutes respectively Hope this works for you.
Selecting items in CSV file using Date
I am trying to select items by date in a CSV file using PowerShell. The format in the CSV file for the date is 1/8/2018 10:04:00 AM. When I run this I get no data although I know that data exists.
$events = Import-Csv c:\normtest\server2_perf.csv | foreach {
New-Object PSObject -prop #{
Date = [DateTime]::Parse($_.Date);
CPULoad = $_.CPULoad;
MemLoad = $_.Memload
}
}
$events | Where { $_.Date -eq (Get-Date).AddDays(-4) }
As you have a time part to your date, this will only work for exactly 4 days from now (i.e. where time of day = right now).
Assuming this part works correctly: Date = [DateTime]::Parse($_.Date);, you can do this:
$start = (Get-Date).Date.AddDays(4)
$fin = $start.AddDays(1) # assuming 1 day window
$events |
Where {$_.Date -gt $start -and $_.Date -lt $fin}
Alternatively, you could treat the date field as string:
$events = Import-Csv c:\normtest\server2_perf.csv |
Where {$_.Date -like "$(Get-Date).AddDays(-4).ToString("M/d/yyyy"))*" }
Assuming your date format is "M/d/yyyy"
Convert date format in CSV using PowerShell
I have two CSV file with 50 column and more than 10K row. There is a column having date-time value. In some records it is 01/18/2013 18:16:32 and some records is 16/01/2014 17:32. What I want to convert the column data like this: 01/18/2013 18:16. I want to remove seconds value. I want to do it by PowerShell script. Sample data: 10/1/2014 13:18 10/1/2014 13:21 15/01/2014 12:03:19 15/01/2014 17:39:27 15/01/2014 18:29:44 17/01/2014 13:33:59
Since you're not going to convert to a sane date format anyway, you can just do a regex replace of that column:
Import-Csv foo.csv |
ForEach-Object {
$_.Date = $_.Date -replace '(\d+:\d+):\d+', '$1'
} |
Export-Csv -NoTypeInformation foo-new.csv
You could also go the route of using date parsing, but that's probably a bit slower:
Import-Csv foo.csv |
ForEach-Object {
$_.Date = [datetime]::Parse($_.Date).ToString('MM/dd/yyyy HH:mm')
} |
Export-Csv -NoTypeInformation foo-new.csv
If you're sure that there are no other timestamps or anything that could look like it elsewhere, you can also just replace everything that looks like it without even parsing as CSV:
$csv = Get-Content foo.csv -ReadCount 0
$csv = $csv -replace '(\d{2}/\d{2}/\d{4} \d{2}:\d{2}):\d{2}', '$1'
$csv | Out-File foo-new.csv