I'm still learning Powershell and in order to complete an assignment I need to convert written data in CSV files to a powershell accepted datetime format.
The dates in the CSV format are written as: 1-1-2014.
In order to convert them I found out (through a bit of brute force) that I can use the following code:
$Files = gci $Source\*.csv
$Data = Import-CSV $Files -Delimiter ";"
$Data | ForEach-Object { $_.Time = $_.Time -as [datetime]
$_
}
However, it would seem to me that this should work as well when written as follows:
$Data | ForEach-Object { $_.Time = $_.Time -as [datetime] $_ }
When I run it like that it returns a bunch of errors.
The $_ after [datetime] I also only added because a colleague used that in his functions, I don't really understand why it has to be put there, and why it needs to be put on a new line. Could anyone explain? If there is a different/better way to convert CSV data to a datetime format I'd be interested in hearing those as well, thanks in advance!
The first (multi-line) version works because PowerShell interprets a line-break after a complete statement as a terminator - it knows that $_ is a separate statement from $_.Time = $_.Time -as [datetime].
To place multiple separate statements on a single line, you'll have to use a semicolon ; to separate them:
$Data | ForEach-Object { $_.Time = $_.Time -as [datetime]; $_ }
Related
Forgive my beginner's attempt..
Any help would be greatly appreciated!
Here's the layout:
CSV Column Headers:
VENDOR,#PAGES,SENDER,RECIPIENT
DocInc,25,John,Mary
I would like to add a new row under that with:
AcmeInc,30,Bill,Paul
I am using this script (substituting actual data in param for testing):
param($Vendor)
param($Pages)
param($Sender)
param($Recipient)
$newRow = $Vendor, $Pages, $Sender, $Recipient
$newRow | Export-Csv -Append -path C:\Files\CSVFILE.CSV
Thanks in advance!
and welcome to StackOverflow! You have made a good attempt, and just some minor mistakes, so let's walk through this real quick. We'll start with the Param() block. This should be an array of parameters, not multiple Param() calls. So it should look like this:
param(
$Vendor,
$Pages,
$Sender,
$Recipient
)
That's simple enough, moving on. The Export-Csv cmdlet expects to get an object with properties to be able to export, not an array of strings. So we can do this one of two ways. We can either just do string manipulation and output the string, or we can make an object and export it to a csv. We'll go with the latter since that is what you were attempting I think.
First, we make an object:
$newRow = [PSCustomObject]#{
'Vendor'=$Vendor
'#Pages'=$Pages
'Sender'=$Sender
'Recipient'=$Recipient
}
Now that we have an object we can just pipe that to Export-Csv exactly how you were doing it before. So the completed code looks like:
param(
$Vendor,
$Pages,
$Sender,
$Recipient
)
$newRow = [PSCustomObject]#{
'Vendor'=$Vendor
'#Pages'=$Pages
'Sender'=$Sender
'Recipient'=$Recipient
}
$newRow | Export-Csv -Append -path C:\Files\CSVFILE.CSV
That's what a lot of people would call the "right way" to do this. You can do some string manipulation and just tack the resulting string to the end of your file, but it is unwise to do it this way since there's more to creating a CSV than just joining several strings with a comma. If you are determined to do string manipulation remember that you need to escape quotes before you enclose each value in quotes, and join them with commas.
param(
$Vendor,
$Pages,
$Sender,
$Recipient
)
$Line = '"{0}","{1}","{2}","{3}"' -f ($Vendor -replace '"','""'),($Pages -replace '"','""'),($Sender -replace '"','""'),($Recipient -replace '"','""')
$Line | Add-Content C:\Files\CSVFILE.CSV
I have a a CSV file that looks like this:
Host;Service;OK;WARN;CRIT;UNKNOWN;Flapping;H.Down;Downtime;N/A
Server1;DNS;0.00%;0.00%;100.00%;0.00%;0.00%;0.00%;0.00%;0.00%
Server2;Filesystem C:/;0.00%;0.00%;100.00%;0.00%;0.00%;0.00%;0.00%;0.00%
Server2;Filesystem D:/;28.82%;0.00%;70.68%;0.00%;0.00%;0.50%;0.00%;0.00%
Server3;Events;0.00%;0.00%;98.97%;0.00%;0.00%;1.03%;0.00%;0.00%
Server4;W32Time;0.00%;0.00%;100.00%;0.00%;0.00%;0.00%;0.00%;0.00%
I'm currently importing it with
$computers = Import-Csv .\foo.csv -Delimiter ";"
I've been trying a couple of different things to try and clear out the percent sign to no avail.
Ultimately, I'd like to get each of the values as numbers to which I can then format them so that any value above 0 (or 0.0%) gets formatted with a color. I figure that the CSV import is importing everything as a string, so it would be more difficult to math around a string, and even more so with a string with a special character in it.
One obvious thing to do is to pre-process your data file to remove the % signs. For example:
(Get-Content .\data.csv).Replace("%","") |
Out-File .\data2.csv -Append
Once this is done, it is easy to use maths on the 'numbers' by casting them to a numerical type, which you'd need to do for any CSV (since, as you say, PowerShell treats everything as a string in that case). For example:
Import-Csv .\data2.csv -Delimiter ";" |
ForEach-Object {
switch([double]$_.CRIT)
{
{$_ -lt 75.0} {"CRIT is under 75: $_"}
default {"CRIT is over 75: $_"}
}
}
EDIT: Compact version incorporating #LotPings suggestion:
(Get-Content .\data.csv).Replace("%","") |
ConvertFrom-Csv -Delimiter ";" |
ForEach-Object {
switch([double]$_.CRIT)
{
{$_ -lt 75.0} {"CRIT is under 75: $_"}
default {"CRIT is over 75: $_"}
}
}
I am trying to use a regex to find dates in a csv file and change the formatting because there are over 200 columns in this csv; manual column mapping for each date is not possible.
what I had previously was the following
$sf = '\\path\dept\Extracts\Date_Modified.csv'
$regex = "\d{1,2}/\d{1,2}/\d{4}"
(Get-Content $sf) |
Foreach-Object {$_ -replace $regex, (get-date -f "yyyy-MM-dd") } |
Set-Content $sf
that works fine if I want to replace all the dates with the current date, but that wasn't my goal. my goal is to recognize human entry type dates (mm/dd/yyyy) and change them to yyyy-mm-dd that the database table is expecting when I load the csv.
how can I modify this? or is there a better way to recognize date formats and change the format?
The answer: use capture groups. I don't know why you wouldn't be able to assign the regex to a variable before use (as I have done many times):
$sf = '\\path\dept\Extracts\Date_Modified.csv'
$regex = '(\d{1,2})\/(\d{1,2})\/(\d{4})'
#(Get-Content -Path $sf) |
ForEach-Object { $_ -replace $regex, '$3-$1-$2' } |
Set-Content -Path $sf
Of special note, use single-quotes in the replace statement so you don't end up trying to interpolate $1 into a (presumably null) variable.
A nice tough one for you all. I'm trying to find and replace a given string in a bunch of files. The files have a date stamp in the file name i.e. YYYY_MM_DD_file.txt
I wish to search and replace within a date range for these files and then replace a string I define, I cannot use date modified as the date range, I must rely on the stamp in the filename.
So far I set my date range in WPF text fields:
$Filename = $Filenamebox.text
$startdate = [datetime] $startdatetext.text
$enddate = [datetime] $enddatetext.Text
$NewFilenamereal = $Newfilename.Text
$array =
do {
$startdate.ToString('yyyy_MM_dd*')
$startdate = $startdate.AddDays(1)
}
until ($startdate -gt [datetime] $enddate)
$files1 = $array | foreach-object {"C:\Users\michael.lawton\Desktop\KGB\Test folder\$_"}
write-host $files1
I then get child items using the $files1 array I have created as a search mask for the files in the date range and find all matches. Store this in a variable and replace the string $filename with the new string $Newfilenamereal.
$Matches1 = get-childitem $files1 | select-string $Filename | foreach-object {$_ -replace $Filename,$Newfilenamereal} | out-string
write-host $Matches1
However I cannot work out how to overwrite what has been found and replaced in the $Matches1 variable to the original files. I have tried set-content, however this will simply either erase everything I have in the date stamped files or cannot understand the $files1 array as a file path.
So my question to you lovely people is how do I write what I have replaced in the environment to the actual files?
Just retrieve the file content using the Get-Content cmdlet and replace the string. Finally write it back using the Set-Content cmdlet:
Get-ChildItem $files1 | ForEach-Object {
($_ | Get-Content -Raw) -replace $Filename,$Newfilenamereal |
Set-Content -Path $_.FullName -Encoding UTF8
}
I'm working with some code that is going to take a series of performance counters, and then put the counters in a .csv file that rolls over every time it hits 1MB.
$Folder="C:\Perflogs\BBCRMLogs" # Change the bit in the quotation marks to whatever directory you want the log file stored in
$Computer = $env:COMPUTERNAME
$1GBInBytes = 1GB
$p = LOTS OF COUNTERS;
# If you want to change the performance counters, change the above list. However, these are the recommended counters for a client machine.
$num = 0
$file = "$Folder\SQL_log_${num}.csv"
if( !(test-path $folder)) {New-Item $Folder -type directory}
Get-Counter -counter $p -SampleInterval 2 -Continuous | Foreach {
if ((Get-Item $file -ErrorAction SilentlyContinue ).Length -gt 1mb)
{
$num +=1
$file = "$Folder\SQL_log_${num}.csv"
}
$_
} | Foreach-Object { $_ | Export-Csv $file -Force -Append}
Right now, it's working quite well. The iteration works fine, and it does create a new file each time the .csv reaches 1MB. However, each .CSV after the first is being created after 2 minutes already at 1MB, causing a new file to be created. I'm not quite sure why this is occurring, although I believe it's because Powershell is just rewriting the entirety of the .csv each time it creates it.
[I'm posting this as a new answer rather than editing the original because it's completely different. Replacing or appending to the original answer would make the ensuing discussion confusing.]
What you need to do is use a regex to extract the values from the Readings property of the output of Get-Counter, and manually construct CSV output from the timestamp and those values. Change the last line to this (format according to your preferred style):
| %{'"' + (Get-Date $_.Timestamp -f 's') + '","' + (([regex]::matches($_.Readings, '(?<=\\\\.+?:\n)(.+?)(?=\n)') | select -ExpandProperty Value) -join '","') + '"'} | Out-File $file -Append -Encoding ASCII
To break that down:
(Get-Date $_.Timestamp -f 's') This part is not strictly necessary, though I think it will make your results easier to follow. The 's' format puts the date in an ISO 8601 sortable pattern. You could substitute 'u' for another sortable format, or use your favorite custom format string. Or just replace it with $_.Timestamp to retain the original format.
[regex]::matches($_.Readings, '(?<=\\\\.+?:\n)(.+?)(?=\n)') The regex matches the contents of any line that is preceded by a line that begins with \\ and ends with : (those pesky counter names you wanted to get rid of). Note that I'm using [regex]::matches, which performs a global match, as opposed to [regex]::match or -match, which will just give you the first match for each string (the Readings property is a single string, so only the first counter reading would be returned).
| select -ExpandProperty Value Produces an array of all the matches, which you can then join with "," and surround with "'s to produce CSV output.
Since you're not using a conversion function, you also need to construct a header row. Add this line right above the pipeline:
`'"Timestamp","' + ($p -join '","') + '"' | Out-File $file -Append -Encoding ASCII`
That's assuming that $p is an array (which it should be). If it's a string, then depending on the format you can either use it as-is, or -split it and rejoin it in CSV format.
Change the last line to this, to convert each line to CSV format and then append it to the output file:
} | Foreach-Object {($_ | ConvertTo-Csv -NoTypeInformation)[1] | Out-File $file -Append -Encoding ASCII}.
A few notes:
The -Encoding ASCII is not strictly necessary, but you might have trouble with a Unicode CSV file in some applications (Excel, for example, won't open it as a CSV file by default, and everything will be in Column A)
The reason for the index in ($_ | ConvertTo-Csv -NoTypeInformation)[1] is that ConvertTo-Csv -NoTypeInformation still outputs the header row each time, so you want to grab the second line of the two-line output (($_ | ConvertTo-Csv -NoTypeInformation)[0] is the header row)
Since you're not outputting a header row, you'll need to output one to $file before the loop