String manipulation while parsing CSV - powershell

The third column of a CSV file is as like this Every Monday 12am-2am of a CSV file.
ACB;Third Week;Every Monday 12am-2am
XYZ;Third Week;Every Tuesday 8pm-10pm
I would like to capture only 12am and 8pm and so on until the end of the csv file and write the details to a separate CSV file the complete output, like below.
ACB;12am
XYZ;8pm

Use calculated properties extract a modified part of the data. The idea is to use Select-Object to select and modify the data like so,
select code, #{name="day";expression={ if($_.day -match "\d{1,2}[ap]m") {$Matches[0]} }}
So what's going on? The select (shorthand for Select-Object) can use a script block as an expression for processing. The argument is a hash table that contains manipulated property's name as a data source and expression that does stuff to it. The part if($_.day -match "\d{1,2}[ap]m") {$Matches[0]} checks if the day column value matches a regex for 1 or 2 digits followed by am or pm.
As for more complete an example,
$data = #'
ACB;Third Week;Every Monday 12am-2am
XYZ;Third Week;Every Tuesday 8pm-10pm
'#
$c = ConvertFrom-Csv -InputObject $data -Delimiter ";" -Header #('code','week','day')
$c | select code, #{name="day";expression={ if($_.day -match "\d{1,2}[ap]m") {$Matches[0]} }}
# Output
code day
---- ---
ACB 12am
XYZ 8pm

Related

Is there a way to add 2 days to a date string of characters in a text file?

I have a file where the 2nd date field needs to be set to 2 days after the first date field in a text file for every record. For example, a record contains the following string:
11/24/2019 -----
and I want to replace the '-----' dashes with 11/26/2019 so that it shows as
11/24/2019 11/26/2019
Has anyone done this using a PowerShell script?
# Create sample input file
#'
11/24/2019 -----
01/01/2019 -----
'# > sample.txt
# Process each line of the file
Get-Content sample.txt | ForEach-Object {
'{0} {1}' -f ($dt = (-split $_)[0]),
([datetime] $dt).AddDays(2).ToString('MM/dd/yyyy')
}
The above yields (2 lines):
11/24/2019 11/26/2019
01/01/2019 01/03/2019
Explanation:
$dt = (-split $_)[0] uses the unary form of the -split operator to split the input line at hand ($_) by whitespace, extracts the first ([0]) token, and assigns it to variable $dt
([datetime] $dt).AddDays(2) casts the date string to a .NET System.DateTime instance and calls its .AddDays() method to add 2 days.
.ToString('MM/dd/yyyy') formats the resulting date the same way as the input date string.
'{0} {1}' -f ... uses the format operator to form a single output string from the two operands.

Converting single row to multiple rows in CSV using PowerShell

I have a CSV file with 2 columns and multiple rows. I have attached the contents of CSV file used for testing, with 2 columns and 30 rows having data from 2 person. But in real application the number of rows increases with the number of test made. So in practical application the dimension will be 2 columns and (15 times N) row where N is the number of person.
I need the output as N + 1 (header) rows and 15 columns (parameters). Can someone help me with a program to convert it using Powershell?
Gentle remainder that I took 2 readings for testing. In application the number of reading is not sure.
My input text file which is to be converted. The parameter and the corresponding values are separated by comma.
Name,Test
Age,18
Gender,Male
Time 1,379
Time 2,290
Time 3,305
Time 4,290
Time 5,319
Time 6,340
Time 7,436
Time 8,263
Time 9,290
Time 10,381
Responses,0
Average Reaction Time,329
Name,Test
Age,18
Gender,Male
Time 1,365
Time 2,340
Time 3,254
Time 4,270
Time 5,249
Time 6,350
Time 7,309
Time 8,527
Time 9,356
Time 10,407
Responses,1
Reaction Time,375
My code snippet for delimiting comma and transposing columns and rows
import-csv $file -delimiter "," | export-csv $outfile
(gc $outfile | select -Skip 1) | sc $outfile
$filedata = import-csv $outfile -Header Parameter , Value
$filedata | export-csv $outfile -NoTypeInformation
$Csv = import-csv $outfile
$Rows = #()
$Rows += $csv.Parameter -join ","
$Rows += $Csv.Value -join ","
Get-Process | Tee-Object -Variable ExportMe | Format-Table
$Rows | Set-Content $outfile
This is my current CSV file
Name,Age,Gender,Time 1,Time 2,Time 3,Time 4,Time 5,Time 6,Time 7,Time 8,Time 9,Time 10,Responses,Average Time,Name,Age,Gender,Time 1,Time 2,Time 3,Time 4,Time 5,Time 6,Time 7,Time 8,Time 9,Time 10,Responses,Average Time
Test,18,Male,379,290,305,290,319,340,436,263,290,381,0,329,Test,18,Male,365,340,254,270,249,350,309,527,356,407,1,375
I am expecting an output CSV like this
Name,Age,Gender,Time 1,Time 2,Time 3,Time 4,Time 5,Time 6,Time 7,Time 8,Time 9,Time 10,Responses,Average Time
Test,18,Male,379,290,305,290,319,340,436,263,290,381,0,329
Test,18,Male,365,340,254,270,249,350,309,527,356,407,1,375
I have also attached a snap of my actual and received output.
Thanks in Advance
I'd strongly suggest not trying to format the CSV by hand.
If you know that there are always exactly 15 rows of properties per person, you can do a nested loop to "chop up" your csv:
# import original csv
$rows = Import-Csv $file -Header Name,Value
# outer loop increments by 15 (span of one person) every time
$objects = for($i = 0;$i -lt $rows.Count;$i += 15){
# prepare an ordered dictionary to hold the properties
$props = [ordered]#{}
# generate an inner loop from the offset to offset+14
$i..($i+14)|%{
# copy each row to our dictionary
$props[$rows[$_].Name] = $rows[$_].Value
}
# cast our dictionary to an object
[pscustomobject]$props
}
# convert back to csv
$objects |Export-Csv $outfile -NoTypeInformation

Need 3 digits in column in a CSV-file

I have a script that extracts data from an XML file and put this into an CSV file with 2 columns.
The file looks like this:
Name, Count
1,34
3,55
15,66
103,99
etc.
So far so good...
My problem is that the program that reads the CSV-file always expect 3 digits in the column "Name".
So the CSV-file need to look like this:
Name, Count
001,34
003,55
015,66
103,99
etc.
How can I do this formatting using "Export-CSV"?
Please help I'm stuck here..
There are several ways to apply the changes to the csv file.
Read/Import to a variable, change name, Export-Csv variable
$Csv = Import-Csv .\Sample.csv
$Csv | ForEach-Object{ $_.Name = $_.Name.PadLeft(3,'0') }
$Csv | Export-Csv .\NewSample.csv -NoTypeInformation
Do the same on the fly with a calculated property reusing the same header/property name.
Import-Csv .\Sample.csv |
Select-Object #{n='Name';e={$_.Name.PadLeft(3,'0')}},Count|
Export-Csv .\NewSample2.csv -NoTypeInformation
Use the -f ( format ) operator with variable
i.e.
[int] $int = 25;
"{0:D3}" -f $int
Here 3 is a number of digits and Output will be :
025

PowerShell ForEach removes leading zeros

I am kind of new with PowerShell and programming in general, so I hope you have some patience while reading this. Before I explain my problem, I feel like I have to first tell you some background information:
I have all my transactions saved in $Transactions. Each transaction has Receiver, Date and Amount.
I have grouped the yearly transactions into $TransactionsPerYear the following way:
$TransactionsPerYear = $Transactions | Group-Object { [int]($_.date -replace '.*\.') }
(Btw. Could someone explain the regex in the end for me, what each character does?)
Next thing I am doing is grouping yearly income and expenses into separate variables. After this I am trying to extract the months from each year and save them into $Months. The date is in the following format dd.MM.yyyy
Question 1:
Here's how I can get all the dates, but how do I extract just the months?
$TransactionsPerYear | Select -ExpandProperty Group | Select -ExpandProperty date | Select -Unique
Question 2:
Because I don't know how to extract the months, I've tried it the following way:
[String[]]$Months = "01","02","03","04","05","06","07","08","09","10","11","12"
When I have each month in $Months I am trying to get monthly transactions and save them into new variables:
ForEach($Month in $Months){
New-Variable -Name "Transactions_$Month$Year" -Value ($Transactions | Where {$_.Date -like "*.$Month.$Year"} | Group-Object 'Receiver' | Select-Object Count, Name, #{L="Total";E={$_ | Select -ExpandProperty Group | Measure-Object Amount -Sum | Select -ExpandProperty Sum}} | Sort-Object {[double]$_.Total})
}
The problem that I am facing here is that ForEach removes the leading zero from each month, and when this happens, this part in ForEach doesn't match with anything, and the new variable is null:
Where {$_.Date -like "*.$Month.$Year"}
Let me know if you need more info. I'd be really thankful if anyone could help me.
The date looks like: 25.02.2016
From your post, it looks like you've jumped further down the rabbithole than necessary.
Instead of trying to do string manipulation every time you need to interact with the Date property, simply turn it into a DateTime object!
$Transactions = $Transactions |Select-Object *,#{Name='DateParsed';Expression={[datetime]::ParseExact($_.Date, 'dd.MM.yyyy', $null)}}
The DateTime.ParseExact() method allows us to specify the format (eg. dd.MM.yyyy), and parse a string representation of a date.
Now you can group on year simply by:
$TransactionsPerYear = $Transactions |Group-Object { $_.DateParsed.Year }
To group by both Year and then Month, I'd create a nested hashtable, like so:
# Create a hashtable, containing one key per year
$MonthlyTransactions = #{}
foreach($Year in $Transactions |Group {$_.DateParsed.Year})
{
# Create another hashtable, containing a key for each month in that year
$MonthlyTransactions[$Year.Name] = #{}
foreach($Month in $Year.Group |Group {$_.DateParsed.Month})
{
# Add the transactions to the Monthly hashtable
$MonthlyTransactions[$Year.Name][$Month.Name] = $Month.Group
}
}
Now you can calculate the transaction value for a specific month by doing:
$TotalValueMay2010 = ($MonthlyTransactions[2010][5] |Measure-Object Amount -Sum).Sum
(Btw. Could someone explain the regex in the end for me, what each character does?)
Sure:
. # match any character
* # zero of more times
\. # match a literal . (dot)
Taking your own example input string 25.02.2016, the first group (.*) will match on 25.02, and \. will match on the . right after, so the only thing left is 2016.
Do you mean this?
$dates = ([DateTime] "1/1/2016"),([DateTime] "1/2/2016"),
([DateTime] "2/1/2016"),([DateTime] "3/1/2016")
$uniqueMonths = $dates | ForEach-Object { $_.Month } | Sort-Object -Unique
# $uniqueMonths contains 1,2,3

Filtering a CSV where the system date format and file date formats differ

I have a large source CSV file with 3 example rows as below
0000000000254000,19/02/2014,"zzzzzzz","close","BIL","5378874","125.00",
0000000000254000,19/12/2013,"zzzzzzz","close","BIL","5378874","125.00",
0000000000254000,19/08/2013,"zzzzzzz","close","BIL","5378874","125.00",
I want to remove all lines that are < 10 days old i.e. where today() - above date column <10 and am close with the following script based on the kind donations of others on this site:
get-content C:\temp\date2.csv | where {
[date] ($_.split(','))[1] -lt (get-date).date.adddays(-10)
} | set-content "C:\temp\newdate.txt"
It works if I align the dates in the file with the powershell system date format but could anyone help with how I can do this within the above simple script please?
I've tried this at the start but failed miserably:
(Get-Culture).DateTimeFormat.ShortDatePattern="dd/MM/yyyy"
Try this:
Import-Csv C:\temp\date2.csv -Header c1,c2,c3,c4,c5,c6,c7 |
where { [datetime]::ParseExact($_.c2, "dd/MM/yyyy", $null) -lt (get-date).date.adddays(-10)} |
Export-Csv -NoTypeInformation "C:\temp\newdate.txt"
The -Header gives names to columns and you can change that to whatever you want as long as you amend $_.c2 on line 2 accordingly.