Compare dates with different formats in csv file - powershell
Whats a good way to compare dates in a csv file that looks like this:
Date1,Date2,Date3
11/10/2016 9:45:00 PM,20161110,11/10/2016
11/15/2016 11:24:00 PM,20160924,11/10/2016
If a match is found, append a column like so...
Date1,Date2,Date3,MatchDates
11/10/2016 9:45:00 PM,20161110,11/10/2016,Match Found
11/15/2016 11:24:00 PM,20160924,11/10/2016,No Match Found
updated
Trying the code that is in the comments:
When comparing this with 2 of the columns...
$csvFile = 'C:\Scripts\Tests\test1.csv'
Import-Csv $csvFile | Select-Object *, #{n='MatchDates';e={
if(([datetime]$_.Date1).Date -eq $_.Date3){
'Match Found'
}Else{
'No Match Found'
}}} |
Export-Csv "$csvFile-results.csv" -NoTypeInformation -Force
output (isCorrect)...
Date1,Date2,Date3,MatchDates
11/10/2016 9:45:00 PM,20161110,11/10/2016,Match Found
11/15/2016 11:24:00 PM,20160924,11/10/2016,No Match Found
However, if I try to compare all 3 columns using the following code
Import-Csv $csvFile | Select-Object *, #{n='MatchDates';e={
if((([datetime]$_.Date1).Date -eq $_.Date3) -and (([datetime]$_.Date2).Date -eq $_.Date3) -and (([datetime]$_.Date1).Date -eq $_.Date2)){
'Match Found'
}Else{
'No Match Found'
}}} |
Export-Csv "$csvFile-results.csv" -NoTypeInformation -Force
output (isNotCorrect)...
Date1,Date2,Date3,MatchDates
11/10/2016 9:45:00 PM,20161110,11/10/2016,
11/15/2016 11:24:00 PM,20160924,11/10/2016,No Match Found
As you can see the value of row 1 and the last column is $null instead of showing Match Found
Maybe I'm not understanding something correctly?
You were on the right track with what we discussed in comments. Problem was with that middle date. It does not convert to a [datetime] without some help. That is where ParseExact comes in handy. Consider the following:
PS D:\temp> [datetime]"20160924"
Cannot convert value "20160924" to type "System.DateTime". Error: "String was not recognized as a valid DateTime."
At line:1 char:1
+ [datetime]"20160924"
+ ~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvalidCastParseTargetInvocationWithFormatProvider
PS D:\temp> [datetime]::parseexact("20160924","yyyyMMdd",[System.Globalization.CultureInfo]::CurrentCulture)
Saturday, September 24, 2016 12:00:00 AM
Note the second example returned a proper date object.
Do you remember the transitive property from math? We use that for simple comparison to see if all dates are the same. Not the only way by far but a simple one nonetheless. Building off your calculated property code
$csv | Select-Object *,#{Name='MatchDates';Expression={
$date1 = ([datetime]$_.Date1).Date
$date2 = ([datetime]::parseexact($_.Date2,"yyyyMMdd",[System.Globalization.CultureInfo]::CurrentCulture)).Date
$date3 = ([datetime]$_.Date1).Date
if($date1 -eq $date2 -and $date2 -eq $date3){
'Match Found'
} else {
'No Match Found'
}
}
}
Cleared up the if logic by saving the casted values in temproary variables.
Related
Where-Object Error When Passing Get-Content as Variable
First, my PS knowledge is very basic, so know that up front. I'm working on a basic script to search EventIDs in archived .evtx files and kick out "reports". The Where-Object queries are in .txt files stored in .\AuditEvents\ folder. I'm trying to do a ForEach on the .txt files and pass each query to Get-WinEvent. Here's an example of how the queries appear in the .txt files: {($_.ID -eq "11")} The script is: $ae = Get-ChildItem .\AuditEvents\ ForEach ($f in $ae) { $qs = Get-Content -Path .\AuditEvents\$f Get-WinEvent -Path .\AuditReview\*.evtx -MaxEvents 500 | Select-Object TimeCreated, ID, LogName, MachineName, ProviderName, LevelDisplayName, Message | Where-Object $qs | Out-GridView -Title $f.Name } This is the error: Where-Object : Cannot bind argument to parameter 'FilterScript' because it is null. At C:\Users\######\Desktop\PSAuditReduction\PSAuditReduction.ps1:6 char:177 + ... e, ProviderName, LevelDisplayName, Message | Where-Object $qs | Out-G ... + ~~~ + CategoryInfo : InvalidData: (:) [Where-Object], ParameterBindingValidationException + FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.WhereObjectCommand
Your symptom implies that $qs is $null, which in turn implies that file .\AuditEvents\$f is empty. However, even if it had content, you couldn't pass the resulting string as-is to the (positionally implied) -FilterScript parameter of Where-Object requires a script block ({ ... }). You must create a script block from the string explicitly, using [scriptblock]::Create(). A simplified example: # Simulated input using a literal string instead of file input via Get-Content $qs = '{ 0 -eq $_ % 2 }' # Sample filter: return $true for even numbers. # Remove the enclosing { and }, as they are NOT part of the code itself # (they are only needed to define script-block *literals* in source code). # NOTE: If you control the query files, you can simplify them # by omitting { and } to begin with, which makes this # -replace operation unnecessary. $qs = $qs.Trim() -replace '^\{(.+)\}$', '$1' # Construct a script block from the string and pass it to Where-Object 1..4 | Where-Object ([scriptblock]::Create($qs)) # -> 2, 4 Note: Your code assumes that each .\AuditEvents\$f file contains just one line, and that that line contains valid PowerShell source code suitable for use a Where-Object filter. Generally, be sure to only load strings that you'll execute as code from sources you trust. Taking a step back: As Abraham Zinala points out, a much faster way to filter event-log entries is by using Get-WinEvent's -FilterHashtable parameter. This allows you to save hastable literals in your query files, which you can read directly into a hashtable with Import-PowerShellDataFile: # Create a file with a sample filter. '#{Path=".\AuditEvents\.*evtx";ID=11}' > sample.txt # Read the file into a hashtable... $hash = Import-PowerShellDataFile sample.txt # ... and pass it to Get-WinEvent Get-WinEvent -MaxEvents 500 -FilterHashtable $hash | ...
I need to pass a line number from an object
>$search="<table id=" $linenumber= Get-Content ".\145039.html" | select-string $search | Select-Object LineNumber $search="</table>" $linenumber2= Get-Content ".\145039.html" | select-string $search | Select-Object LineNumber #$linenumber2 # the list of line numbers to fetch $linesToFetch = $linenumber[2]..$linenumber2[2] $currentLine = 1 $result = switch -File ".\145039.html" { default { if ($linesToFetch -contains $currentLine++) { $_ }} } # write to file and also display on screen by using -PassThru $result | Set-Content -Path ".\excerpt.html" -PassThru Cannot convert the "#{LineNumber=6189}" value of type "Selected.Microsoft.PowerShell.Commands.MatchInfo" to type "System.Int32". At line:10 char:1 + $linesToFetch = $linenumber[2]..$linenumber2[2] + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidArgument: (:) [], RuntimeException + FullyQualifiedErrorId : ConvertToFinalInvalidCastException $linenumber and $linenumber2 return values like below but I just need to get the number not the column header. LineNumber ---------- 6015 Also, the final version needs to loop through all the html files in a directory not just one static file. Sorry, there is probably a better way to do this but not sure how. Thanks in advance! Did a lot of googling but could not find the right solution. Updated code: $search1="disconnect-status" $linenumber1= Get-Content ".\145039.html" | select-string $search1 | Select-Object -ExpandProperty LineNumber $search2="</table>" $linenumber2= Get-Content ".\145039.html" | select-string $search2 | Select-Object -ExpandProperty LineNumber # the list of line numbers to fetch $linesToFetch = $linenumber1[3]..$linenumber2[1] $currentLine = 1 $result = switch -File ".\145039.html" { default { if ($linesToFetch -contains $currentLine++) { $_ }} } # write to file and also display on screen by using -PassThru $result | Set-Content -Path ".\excerpt.html" -PassThru _____________________________________________________________ Thank you # mklement0 This now works for one file at a time now I need it go select text from all the HTML files in the directory.
Your immediate problem is that you need to change Select-Object LineNumber (which, due to positional parameter binding, is equivalent to Select-Object -Property LineNumber) to Select-Object -ExpandProperty LineNumber. That is, you must use Select-Object's -ExpandProperty parameter in order to only get the values of the input objects' .LineNumber properties - see this post. That said, your approach can be optimized in a number of ways, allowing you to make do with only a switch statement: $output = $false; $openTagCount = 0 $result = switch -File .\145039.html { '<table id=' { if (++$openTagCount -eq 3) { $output = $true } # 3rd block found continue } '</table>' { if ($output) { break } # end of 3rd block -> exit continue } default { if ($output) { $_ } # inside 3rd block -> output line } } Note: This extracts the lines inside the third <table> element that has an id attribute, as implied by your original solution attempt; the solution you later edited into the question works differently. Taking a step back: It looks like your input is HTML, so you're usually better off using HTML parsing to handle your input: In Windows PowerShell you may be able to use Invoke-WebRequest relying on the Internet Explorer engine if present (it isn't anymore by default in recent Windows versions). In recent versions of Windows and in PowerShell (Core) (v6+), you'll either need New-Object -Com HTMLFile - see this answer - or a third-party solution such as such as the PowerHTML module that wraps the HTML Agility Pack - see this answer.
Append existing column in csv by matching values with array and with condition
I will do my best to break this down as simply as I can. what I have so far that is working: Currently I have two csv files... test1.csv test1ColumnN,test1ColumnI,test1ColumnD,selectDomainOne,selectDomainTwo,selectDomainThree asdf,asdf,asdf,,, nValue1,iValue1,dValue1,sValue1,, qwer,asdf,zxcv,,, nValue2,iValue2,dValue2,,, qwer,zxcv,asdf,lkjh,, nValue3,iValue3,dValue3,sValue3,, zxcv,qwer,asdf,,poiu, nValue1,iValue1,dValue1,,sValue1, nValue4,iValue4,dValue4,,sValue4, asdf,qwer,zxcv,fghj,mnbv, nValue5,iValue5,dValue5,,, asdf,cvbn,erty,,,uytr nValue7,iValue7,dValue7,,,sValue7 nValue8,iValue8,dValue8,,,sValue8 nValue9,iValue9,dValue9,,,sValue9 qwer,asdf,zxcv,poiu,lkjh,mnbv test2.csv DomainCatagories,test2ColumnS,test2ColumnA,test2ColumnN,test2ColumnI,test2ColumnD DomainOne,sValue1,aValue1,nValue1,,dValueN DomainOne,sValue2,aValue2,,iValue2,dValue2 DomainOne,sValue3,aValue2,nValue3,iValue3,dValue3 DomainTwo,sValue1,aValue2,,iValue1,dValueN DomainTwo,sValue4,aValue1,nValue4,,dValueN DomainTwo,sValue5,aValue1,nValue5,iValue5,dValue5 DomainThree,sValue7,aValue2,nValue7,iValue7,dValue7 DomainThree,sValue8,aValue1,nValue8,iValue8,dValue8 DomainThree,sValue9,aValue2,nValue9,iValue9,dValue9 Now I want to add a column (inside test2.csv) to match the sValue# from both test1.csv and test2.csv with the condition of ($_.DomainCatagories='DomainOne' from test2.csv) and ($_.selectDomainOne from test1.csv) To do this, I am using the following code... #Create Column $domainNameOne = #{} $domainNameOne = Import-Csv 'C:\Scripts\Tests\test1.csv' | Where-Object {$_.selectDomainOne} | Select-Object -Expand 'selectDomainOne' (Import-Csv 'C:\Scripts\Tests\test2.csv') | Select-Object -Property *, #{n='Test1sValues';e={ if($_.DomainCatagories -eq 'DomainOne'){ if(($domainNameOne -contains $_.test2ColumnS) -and ($_.test2ColumnS)){ $_.test2ColumnS } Else { 'Not found in test1' }}}} | Export-Csv "C:\Scripts\Tests\test2-Temp" -NoType Move-Item "C:\Scripts\Tests\test2-Temp" 'C:\Scripts\Tests\test2.csv' -Force After the code is run, I get the following test2.csv (isCorrect)... "DomainCatagories","test2ColumnS","test2ColumnA","test2ColumnN","test2ColumnI","test2ColumnD","Test1sValues" "DomainOne","sValue1","aValue1","nValue1","","dValueN","sValue1" "DomainOne","sValue2","aValue2","","iValue2","dValue2","Not found in test1" "DomainOne","sValue3","aValue2","nValue3","iValue3","dValue3","sValue3" "DomainTwo","sValue1","aValue2","","iValue1","dValueN","" "DomainTwo","sValue4","aValue1","nValue4","","dValueN","" "DomainTwo","sValue5","aValue1","nValue5","iValue5","dValue5","" "DomainThree","sValue7","aValue2","nValue7","iValue7","dValue7","" "DomainThree","sValue8","aValue1","nValue8","iValue8","dValue8","" "DomainThree","sValue9","aValue2","nValue9","iValue9","dValue9","" What I have that is not working: Next I run the following code... #Append Column $domainNameThree = #{} $domainNameThree = Import-Csv 'C:\Scripts\Tests\test1.csv' | Where-Object {$_.selectDomainThree} | Select-Object -Expand 'selectDomainThree' (Import-Csv 'C:\Scripts\Tests\test2.csv') | % { if($_.DomainCatagories -eq 'DomainThree'){ if(($domainNameThree -contains $_.test2ColumnS) -and ($_.test2ColumnS)){ $_.Test1sValues = $_.test2ColumnS } Else { $_.Test1sValues = 'Not found in test1' }}} | Export-Csv "C:\Scripts\Tests\test2-Temp" -NoType Move-Item "C:\Scripts\Tests\test2-Temp" 'C:\Scripts\Tests\test2.csv' -Force Instead of adding the values in the correct rows, it completely blanks out the whole file and saves it as an empty file. End Goal What I want the code to produce, is this (notice values filled in on last 3 rows in the last column)... "DomainCatagories","test2ColumnS","test2ColumnA","test2ColumnN","test2ColumnI","test2ColumnD","Test1sValues" "DomainOne","sValue1","aValue1","nValue1","","dValueN","sValue1" "DomainOne","sValue2","aValue2","","iValue2","dValue2","Not found in test1" "DomainOne","sValue3","aValue2","nValue3","iValue3","dValue3","sValue3" "DomainTwo","sValue1","aValue2","","iValue1","dValueN","" "DomainTwo","sValue4","aValue1","nValue4","","dValueN","" "DomainTwo","sValue5","aValue1","nValue5","iValue5","dValue5","" "DomainThree","sValue7","aValue2","nValue7","iValue7","dValue7","sValue7" "DomainThree","sValue8","aValue1","nValue8","iValue8","dValue8","sValue8" "DomainThree","sValue9","aValue2","nValue9","iValue9","dValue9","sValue9" What am I doing wrong in that 2nd code snippet?
The example you show from What I have that is not working: is missing a key portion. Export-Csv will take everything piped into it to populate the CSV but you are not providing any. Problem is that you are not passing anything through the pipe. Merely just updating one property. The simplest thing to do is add $_ after the if statement. Or you could just use a calculated property which you have done before in another one of your questions. The example below from Compare dates with different formats in csv file even uses an if statement. Import-Csv $csvFile | Select-Object *, #{n='MatchDates';e={ if((([datetime]$_.Date1).Date -eq $_.Date3) -and (([datetime]$_.Date2).Date -eq $_.Date3) -and (([datetime]$_.Date1).Date -eq $_.Date2)){ 'Match Found' }Else{ 'No Match Found' }}} | Export-Csv "$csvFile-results.csv" -NoTypeInformation -Force
Windows Powershell Export-CSV column/row value as filename
I'm having a problem with Powershell. I need to import a csv file, format it and export it. Now one of the fields is the same value on every line so I want to name the export file the same as a cell value. I am importing the csv with Import-Csv c:\tmp\200114.csv Formatting the output with Select-Object #{expression={$_.code}; label='ID NUMBER'} etc etc but one of the fields being DESCRIPTION Then exporting with: Export-Csv -NoTypeInformation c:\tmp\test1.csv So basically I want to name the file something like the following where DESCRIPTION is the Field name and the row is 1(as they are all the same): Export-Csv -NoTypeInformation c:\tmp\#{expression={$_.DESCRIPTION[1]};}.csv But I just get: Export-Csv : Cannot validate argument on parameter 'Delimiter'. The argument is null. Supply a non-null argument and tr y the command again. Ideally I could like the file name to be: Todays Date - DESCRIPTION Column ROW 1 Value.csv Many thanks for any input.... SAMPLE DATA: ID NUMBER,NAME,ADDRESS 1,ADDRESS 2,CITY,STATE,ZIP,Spare,PHONE #,DESCRIPTION,Spare,Spare,VALUE 1,Name 1,address 1,address 2,city 1,state 1,zip 1,,Phone 1,CC098-1,,,NCV 2,Name 2,address 2,address 3,city 2,state 2,zip 2,,Phone 2,CC098-1,,,NCV Thanks, I am very grateful for this. I am struggling to get this working though. It must be to do with: Select #{n='ID NUMBER';e={$_.code}}, #{n='DESCRIPTION';e={...}} I Don't know the Select command and can't find anything on it, I can't even see how the n= & e= etc are doing. I trimmed it down to: $csv = Import-Csv c:\tmp\200114.csv | Select-Object #{expression={$_.code}; label='ID NUMBER'} | $filename = "$(Get-Date -f 'yyyy-MM-dd') - $($csv[0].DESCRIPTION).csv" | $csv | Export-Csv $filename -NoTypeInformation but I just get errors: Expressions are only allowed as the first element of a pipeline. At line:1 char:108 + $csv = Import-Csv c:\tmp\200114.csv | Select-Object #{expression={$_.code}; label='ID NUMBER'} | $filename <<<< = "$ (Get-Date -f 'yyyy-MM-dd') - $($csv[0].DESCRIPTION).csv" | $csv | Export-Csv $filename -NoTypeInformation + CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException + FullyQualifiedErrorId : ExpressionsMustBeFirstInPipeline Thanks again.
Capture the processed CSV in a variable: $csv = Import-Csv c:\tmp\200114.csv | Select #{n='ID NUMBER';e={$_.code}}, #{n='DESCRIPTION';e={...}}, ... Construct your filename: $filename = "$(Get-Date -f 'yyyy-MM-dd') - $($csv[0].DESCRIPTION).csv" Then export the data: $csv | Export-Csv $filename -NoTypeInformation
Powershell: How to obtain the complete list of distribution groups
I am completely new to powershell, I have never touched this scripting language before. However, I have some backgrounds in perl and bash scripting. I am trying to implement a small script that will obtain the list of DG in Exchange server, filters the results to get only the groups that have a certain string, corresponding to the current year. Example: check the year, in this case 2011. Filter Name Contains 'P11' Return only the last DG name and parse the first 7 characters. How could I do this using powershell from an exchange server? Here is what I got: add-pssnapin Microsoft.Exchange.Management.PowerShell.E2010 # Retrieve all DGs $temp = Get-DistributionGroup -ResultSize Unlimited | foreach($group in $temp) { write-output "GroupName:$group " Write-output "GroupMembers:" Get-DistributionGroupMember $group |ft displayname,alias,primarysmtpaddress write-output ‘ ‘ } this results in the following error: Unexpected token 'in' in expression or statement. At C:\Users\jfb\Desktop\NewGroupProject.ps1:7 char:18 + foreach($group in <<<< $temp) + CategoryInfo : ParserError: (in:String) [], ParseException + FullyQualifiedErrorId : UnexpectedToken
Remove the trailing | in the line $temp = Get-DistributionGroup -ResultSize Unlimited | and it should work fine. What is happening is that since you had the | it is treating the foreach as a foreach-object
Try this (not tested). Create a date object,using Get-Date, and format the date to include the last two digits of the year enclosed in asterisks. This would be the wildcard for the Get-DistributionGroup cmdlet. Select the last DG object and expand its name. $name = Get-Date -Format *Pyy* $group = Get-DistributionGroup $name | Select-Object -Last 1 -ExpandProperty Name if($group) { $group.Substring(0,7) }