datetime comparison letting thread proceed when shouldn't - powershell

I have a backup script that puts files in a dated directory every night. I have another script, below, that goes through a list of dated directories and if it's within the range I want to delete, I will delete the directory, keeping the Saturday dated file. For some reason, for the dir dated Saturday, 1/12/2019, it's being deleted, even though the if statement should indicate it wouldn't be deleted.
This is my code:
function WeeklyCleanup($folderWeeklyCleanupDatedSubdirs) {
#find out date range to cleanup
$lastSaturday = GetLastSaturdayDate
$lastSaturday = [datetime]$lastSaturday
$weekAgoSunday = GetWeekAgoSundayDate
$weekAgoSunday = [datetime]$weekAgoSunday
#find out filename with Saturday date before current date but within week
Get-ChildItem $folderWeeklyCleanupDatedSubdirs | ForEach-Object {
write-output $_
#check to see if item is before day we want to remove
$temp = $_
$regex = [regex]::Matches($temp,'([0-9]+)-([0-9]+)-([0-9]+)')
if($regex.length -gt 0) #it matched
{
$year = $regex[0].Groups[1].Value
$month = $regex[0].Groups[2].Value
$day = $regex[0].Groups[3].Value
write-output $year
write-output $month
write-output $day
write-output "*************"
$dirDate = $regex[0].Groups[0].Value + " 12:00:00 PM"
write-output $dirDate
if($dirDate.length -gt 0) #it matched
{
$dateTimeObjectFromRegex = [datetime]$dirDate
##########this next statement is letting $dateTimeObjectFromRegex of 1/12/2019 through when it shouldn't. See time comparison below
if(([datetime]$dateTimeObjectFromRegex -lt [datetime]$lastSaturday) -and ([datetime]$dateTimeObjectFromRegex -ge [datetime]$weekAgoSunday)) #we're removing extra ones over last week, keep Saturday
{
$dirPathToRemove = Join-Path -path $folderWeeklyCleanupDatedSubdirs -ChildPath $temp.ToString()
Get-ChildItem -Path $dirPathToRemove #list the dir
#remove dir
if(-Not (Test-Path $dirPathToRemove )) #-PathType Container
{
$global:ErrorStrings.Add("Exception: No such path, $dirPathToRemove;; ")
write-output "++ Error: An error occured during copy operation. No such path, $dirPathToList ++"
}
else
{
#remove dir and subdirs
Remove-Item $dirPathToRemove -Force -Recurse
Get-ChildItem -Path $dirPathToRemove #list the dir
}
#Write-Output $_
#Write-Output " 1 "
} #if within last week
} #if dirDate length
} #if regex matched
} #get-childItem
}
function GetLastSaturdayDate()
{
$date = Get-Date #"$((Get-Date).ToString('yyyy-MM-dd'))"
#for($i=1; $i -le 7; $i++){
# if($date.AddDays(-$i).DayOfWeek -eq 'Saturday') #if found Saturday
# {
# $date.AddDays(-$i)
# $newDate = $date.AddDays(-$i)
# break
# }
#}
$newdate = $date.AddDays(-($date.DayOfWeek+1)%7)
return $newdate
}
function GetWeekAgoSundayDate()
{
$numberOfWeeks = 1; #week ago
$date = Get-Date #"$((Get-Date).ToString('yyyy-MM-dd'))"
#for($i=1; $i -le 7; $i++){
# if(($date.AddDays(-$i).DayOfWeek -eq 'Sunday') -and ($date.AddDays(-$i) -ne $date)) #if found a Sunday and it's not today
# {
# $date.AddDays(-$i)
# $newDate = $date.AddDays(-$i)
# break
# }
#}
#$newdate = $date.AddDays(-($date.DayOfWeek+1)%0)
$numDaysSincePreviousDate = $date.DayOfWeek.value__ + 0 #0 is Sunday
([System.DateTime] $previousDayOfWeek = $date.AddDays(- $numDaysSincePreviousDate)) | Out-Null
$previousDate = $previousDayOfWeek.AddDays(-($numberOfWeeks *7)).ToString("MM-dd-yyyy")
return $previousDate
}
The WeeklyCleanup script is called with this parameter:
$folderToCleanupDatedSubdirs = [System.IO.DirectoryInfo]"E:\Bak_TestDatedFolderCleanup"
For time comparison:
Directory item toLocRobo_2019-01-12 - Once I get regex with timestamp, I add the time in of 12:00:00 PM for the $dirDate variable. This becomes $dateTimeObjectFromRegex. Debugger shows it as Saturday, January 12, 2019 12:00:00 PM
When I run the program, I get $lastSaturday as Saturday, January 12, 2019 3:59:04 PM
When I run the program, debugger also shows $weekAgoSunday as Sunday, January 6, 2019 12:00:00 AM
From what I can see, it shouldn't be getting through that if statement to delete the dir for 1/12/2019.
I tried casting the dates to datetime to make sure it wasn't doing a string comparison in the if statement, even though I casted it above.
I've been looking at these links for more info on this, but it looks correct to me:
dateTimeComparisons not working expectedly
convert string to datetime
All I can think is that it's still treating the datetime in the first part of the if statement comparison like a string, so it's thinking 3:59 PM is greater than 12:00 PM. Any ideas how to get this date check to work? I don't care about times, just want to make sure it doesn't get rid of the Saturday-dated file from this past week, but only cleanup other dir's over that week.
Update
Made changes suggested by #brianist below, and it worked great. This is what it looks like so far. Saturday and Sunday functions haven't changed. He's asking me to post it. If he has any other suggestions how to get it to treat/compare what's returned from the last Saturday and weekAgoSunday functions as dates, without the cast, that'd make it look less clunky/easier to read. Thanks Brian!
#do weekly cleanup of DisasterBackup folder
function WeeklyCleanup($folderWeeklyCleanupDatedSubdirs) {
#find out current date
$currentDate = "$((Get-Date).ToString('yyyy-MM-dd'))" #example 2019-01-15
$currentDayOfWeek = (get-date).DayOfWeek.value__ #returns int value for day of week
#find out current day of week
$lastSaturday = GetLastSaturdayDate
$lastSaturday = [datetime]$lastSaturday #if we take away extra casts it won't do comparison line (-lt and -ge)
$weekAgoSunday = GetWeekAgoSundayDate
$weekAgoSunday = [datetime]$weekAgoSunday #if we take away extra casts it won't do comparison line (-lt and -ge), and can't move cast before function call because it isn't recognizing function anymore if I do
#find out filename with Saturday date before current date but within week
#get each dir item to check if we need to remove it
Get-ChildItem $folderWeeklyCleanupDatedSubdirs | ForEach-Object {
write-output $_
#check to see if item is before day we want to remove
$temp = $_
if($_.Name -match '(\d{4}-\d{2}-\d{2})$'){ #regex
write-output $Matches[1]
$dirDate = Get-Date $Matches[1] #turn regex into date
if(([datetime]$dirDate.Date -lt [datetime]$lastSaturday.Date) -and ([datetime]$dirDate.Date -ge [datetime]$weekAgoSunday.Date)) #we're removing extra ones over last week, keep Saturday
{
$dirPathToRemove = Join-Path -path $folderWeeklyCleanupDatedSubdirs -ChildPath $temp.ToString()
Get-ChildItem -Path $dirPathToRemove #list the dir
#remove dir
if(-Not (Test-Path $dirPathToRemove )) #-PathType Container
{
$global:ErrorStrings.Add("Exception: No such path, $dirPathToRemove;; ")
write-output "++ Error: An error occured during copy operation. No such path, $dirPathToList ++"
}
else
{
#remove dir and subdirs
Remove-Item $dirPathToRemove -Force -Recurse
Get-ChildItem -Path $dirPathToRemove #list the dir
}
} #if within last week
} #if regex matched
} #get-childItem
}

Re-reading your last sentence, I now see that your intention is to ignore times, not to include them in your comparison.
You are right to want to use [DateTime] objects when comparing, but do note that those objects always include a time.
Conveniently you can use the .Date property of such an object which returns a new one, with the time set to midnight. This is useful for comparing because the time would no longer be a factor.
Pulling out my modified if statement from below, you can do it like this:
if ($dirDate.Date -lt $lastSaturday.Date -and $dirDate.Date -ge $weekAgoSunday.Date) {
# do stuff
}
Now you're only comparing dates, and ignoring time!
Based on what you're showing it looks like the if statement is working as expected. To summarize, you say that:
$dateTimeObjectFromRegex is Saturday, January 12, 2019 12:00:00 PM
$lastSaturday is Saturday, January 12, 2019 3:59:04 PM
$weekAgoSunday is Sunday, January 6, 2019 12:00:00 AM
The conditional is:
if(([datetime]$dateTimeObjectFromRegex -lt [datetime]$lastSaturday)
-and ([datetime]$dateTimeObjectFromRegex -ge [datetime]$weekAgoSunday))
Therefore in pseudo-code it's:
if (
("January 12, 2019 12:00:00 PM" is earlier than "January 12, 2019 3:59:04 PM") # true
and
("January 12, 2019 12:00:00 PM" is later or the same as "January 6, 2019 12:00:00 AM") # true
) # true
I do want to point out that you are doing an awful lot of unnecessary datetime chopping and casting, and cleaning that up would help with readability and with debugging these types of issues.
$lastSaturday = GetLastSaturdayDate
$lastSaturday = [datetime]$lastSaturday
$weekAgoSunday = GetWeekAgoSundayDate
$weekAgoSunday = [datetime]$weekAgoSunday
Your functions already return [DateTime] objects, so there's no need for those casts.
$temp = $_
$regex = [regex]::Matches($temp,'([0-9]+)-([0-9]+)-([0-9]+)')
if($regex.length -gt 0) #it matched
{
$year = $regex[0].Groups[1].Value
$month = $regex[0].Groups[2].Value
$day = $regex[0].Groups[3].Value
write-output $year
write-output $month
write-output $day
write-output "*************"
$dirDate = $regex[0].Groups[0].Value + " 12:00:00 PM"
write-output $dirDate
This is quite complex, you could simplify it down to something like this:
if ($_.Name -match '(\d{4}-\d{2}-\d{2})$') {
# in here, you know the match was successful
$dirDate = Get-Date $Matches[1] # double check this, might need .Groups or something
if ($dirDate -lt $lastSaturday -and $dirDate -ge $weekAgoSunday) {
# do stuff
}
}
Probably some more optimizations etc. Hope this was helpful!

Related

how to read time from file and perform minus operation on hours using powershell

I have created a text file that contains id and time[24hr format] that is going to trigger one operation at the mentioned time.
I want to read both the values and minus 2 hrs from mentioned hr.
I tried, but I am not sure how to read input as time format(24hr notation HH:MM) and minus 2 hrs from that and also the scenario like 01:30 - 2hr = 23.30 (previous date)
$lines = Get-Content C:\wamp64\www\schedule_daily_task.txt | Where {$_ -notmatch '^\s+$'}
foreach ($line in $lines) {
$fields = $line -split '\s+'
$id = $fields[0]
$time = $fields[1]
#code here to change the format of $time to time format and perform minus oper
#01:30 - 2hr = 23.30 (previous date)
}
For date and time arithmetics, use .Net's DateTime class. It has method AddHours() which, when passed negative values, subtracts hours. Calculating new datetime to yesterday is quite straight-forward. Like so,
# First, let's get today's date with specific time and minute
$hh = ($fields[1] -split ':')[0]
$mm = ($fields[1] -split ':')[1]
$dd = get-date -date $(get-date).date -hour $hh -minute $mm
# Add -2 hours
$dd.AddHours(-2)
I would use switch -Regex -File for this to read the file line-by-line and get the values from the lines you are after using regex.
Advantage is that this is crazy fast aswell.
$result = switch -regex -File 'C:\wamp64\www\schedule_daily_task.txt' {
'^(\d+)\s+(\d{2}:\d{2})' {
$id = $Matches[1]
$h, $m = $Matches[2] -split ':'
$time = '{0:HH:mm}' -f (Get-Date -Hour $h -Minute $m -Second 0).AddHours(-2)
# output the id and updated time
"$id $time"
}
default {
# output any other line you may have in the file. You can of course
# also ignore these if they are unwanted in the output. In that case
# simply comment out this whole 'default' block
$_
}
}
# output to (new) file and because of '-PassThru' also on screen
$result | Set-Content -Path 'C:\wamp64\www\schedule_daily_task_minus_2.txt' -PassThru
Example input file:
2603961 01:30
1234567 11:59
9876543 23:30
Example output file:
2603961 23:30
1234567 09:59
9876543 21:30

PowerShell script efficiency advice

I have a telephony .csv with compiled data from January 2020 and some days of February, each row has the date and time spent on each status, since someone uses different status over the day the file has one row for each status, my script is supposed to go through the file, find the minimum date and then start saving on new files all the data for the same day, so I'll end with one file for 01-01-2020, 02-01-2020 and so on, but it has 15 hours running and it's still at 1/22.
The column I'm using for the dates is called "DateFull" and this is the script
write-host "opening file"
$AT= import-csv “C:\Users\xxxxxx\Desktop\SignOnOff_20200101_20200204.csv”
write-host "parsing and sorting file"
$go= $AT| ForEach-Object {
$_.DateFull= (Get-Date $_.DateFull).ToString("M/d/yyyy")
$_
}
Write-Host "prep day"
$min = $AT | Measure-Object -Property Datefull -Minimum
Write-Host $min
$dateString = [datetime] $min.Minimum
Write-host $datestring
write-host "Setup dates"
$start = $DateString - $today
$start = $start.Days
For ($i=$start; $i -lt 0; $i++) {
$date = get-date
$loaddate = $date.AddDays($i)
$DateStr = $loadDate.ToString("M/d/yyyy")
$now = Get-Date -Format HH:mm:ss
write-host $datestr " " $now
#Install-Module ImportExcel #optional import if you dont have the module already
$Check = $at | where {$_.'DateFull' -eq $datestr}
write-host $check.count
if ($check.count -eq 0 ){}
else {$AT | where {$_.'DateFull' -eq $datestr} | Export-Csv "C:\Users\xxxxx\Desktop\signonoff\SignOnOff_$(get-date (get-date).addDays($i) -f yyyyMMdd).csv" -NoTypeInformation}
}
$at = ''
The first loop doesn't make much sense. It loops through CSV contents and converts each row's date into different a format. Afterwards, $go is never used.
$go= $AT| ForEach-Object {
$_.DateFull= (Get-Date $_.DateFull).ToString("M/d/yyyy")
$_
}
Later, there is an attempt to calculate a value from uninitialized a variable. $today is never defined.
$start = $DateString - $today
It looks, however, like you'd like to calculate, in days, how old eldest record is.
Then there's a loop that counts from negative days to zero. During each iteration, the whole CSV is searched:
$Check = $at | where {$_.'DateFull' -eq $datestr}
If there are 30 days and 15 000 rows, there are 30*15000 = 450 000 iterations. This has complexity of O(n^2), which means runtime will go sky high for even relative small number of days and rows.
The next part is that the same array is processed again:
else {$AT | where {$_.'DateFull' -eq $datestr
Well, the search condition is exactly the same, but now results are sent to a file. This has a side effect of doubling your work. Still, O(2n^2) => O(n^2), so at least the runtime isn't growing in cubic or worse.
As for how to fix this, there are a few things. If you sort the CSV based on date, it can be processed afterwards in just a single run.
$at = $at | sort -Property datefull
Then, iterate each row. Since the rows are in ascending order, the first is the oldest. For each row, check if date has changed. If not, add it to buffer. If it has, save the old buffer and create a new one.
The sample doesn't convert file names in yyyyMMdd format, and it assumes there are only two columns foo and datefull like so,
$sb = new-object text.stringbuilder
# What's the first date?
$current = $at[0]
# Loop through sorted data
for($i = 0; $i -lt $at.Count; ++$i) {
# Are we on next date?
if ($at[$i].DateFull -gt $current.datefull) {
# Save the buffer
$file = $("c:\temp\OnOff_{0}.csv" -f ($current.datefull -replace '/', '.') )
set-content $file $sb.tostring()
# Pick the current date
$current = $at[$i]
# Create new buffer and save data there
$sb = new-object text.stringbuilder
[void]$sb.AppendLine(("{0},{1}" -f $at[$i].foo, $at[$i].datefull))
} else {
[void]$sb.AppendLine(("{0},{1}" -f $at[$i].foo, $at[$i].datefull))
}
}
# Save the final buffer
$file = $("c:\temp\OnOff_{0}.csv" -f ($current.datefull -replace '/', '.') )
set-content $file $sb.tostring()

Compare current date to date string in a file using powershell

I am writing some PS scripts to log times into a text file, login.txt, using the following code:
$logdir = "C:\FOLDER"
$logfile = "$logdir\LastLogin.txt"
$user = $env:USERNAME
$date = Get-Date -Format "dd-MM-yyyy"
if (!(Test-Path $logdir)){New-Item -ItemType Directory $logdir}else{}
if (!(Test-Path $logfile)){New-Item $logfile}else{}
if (Get-Content $logfile | Select-String $user -Quiet){write-host "exists"}else{"$user - $date" | Add-Content -path $logfile}
(Get-Content $logfile) | Foreach-Object {$_ -replace "$user.+$", "$user - $date"; } | Set-Content $logfile
This creates an entry in the text file like:
UserName - 01-01-1999
Using Powershell, I want to read the text file, compare the date, 01-01-1999, in the text file to the current date and if more than 30 days difference, extract the UserName to a variable to be used later in the script.
I would really appreciate any hints as to how I could do the following:
Compare the date in the text file to the current date.
If difference is more than 30 days, pick up UserName as a variable.
I would really appreciate any advice.
Checking all dates in the file with the help of a RegEx with named capture groups.
$logdir = "C:\FOLDER"
$logfile = Join-Path $logdir "LastLogin.txt"
$Days = -30
$Expires = (Get-Date).AddDays($Days)
Get-Content $logfile | ForEach-Object {
if ($_ -match "(?<User>[^ ]+) - (?<LastLogin>[0-9\-]+)") {
$LastLogin = [datetime]::ParseExact($Matches.LastLogin,"dd-MM-yyyy",$Null)
if ( $Expires -gt $LastLogin ) {
"{0} last login {1} is {2:0} days ago" -F $Matches.User, $Matches.LastLogin,
(New-TimeSpan -Start $LastLogin -End (Get-Date) ).TotalDays
}
}
}
Sample output
username last login 31-12-1999 is 6690 days ago
There is a way of doing that using regex (Regular Expressions). I will assume that the username which you get in your text file is .(dot) separated. For example, username looks like john.doe or jason.smith etc. And the entry in your text file looks like john.doe - 01-01-1999 or jason.smith - 02-02-1999. Keeping these things in mind our approach would be -
Using a regex we would get the username and date entry into a single variable.
Next up, we will split the pattern we have got in step 1 into two parts i.e. the username part and the date part.
Next we take the date part and if the difference is more than 30 days, we would take the other part (username) and store it in a variable.
So the code would look something like this -
$arr = #() #defining an array to store the username with date
$pattern = "[a-z]*[.][a-z]*\s[-]\s[\d]{2}[-][\d]{2}[-][\d]{4}" #Regex pattern to match entires like "john.doe - 01-01-1999"
Get-Content $logfile | Foreach {if ([Regex]::IsMatch($_, $pattern)) {
$arr += [Regex]::Match($_, $pattern)
}
}
$arr | Foreach {$_.Value} #Storing the matched pattern in $arr
$UserNamewithDate = $arr.value -split ('\s[-]\s') #step 2 - Storing the username and date into a variable.
$array = #() #Defining the array that would store the final usernames based on the time difference.
for($i = 1; $i -lt $UserNamewithDate.Length;)
{
$datepart = [Datetime]$UserNamewithDate[$i] #Casting the date part to [datetime] format
$CurrentDate = Get-Date
$diff = $CurrentDate - $datepart
if ($diff.Days -gt 30)
{
$array += $UserNamewithDate[$i -1] #If the difference between current date and the date received from the log is greater than 30 days, then store the corresponding username in $array
}
$i = $i + 2
}
Now you can access the usernames like $array[0], $array[1] and so on. Hope that helps!
NOTE - The regex pattern will change as per the format your usernames are defined. Here is a regex library which might turn out to be helpful.

powershell delete some files older than 90 days, with exceptions

$timeLimit1 = Get-Date.AddDays(-90)
Get-ChildItem <path> | ? {$_.LastWriteTime -le $timeLimit1 } | Remove-Item
So I have the easier part, but I am trying to figure out how to do something a little more complex.
I am setting up a backup deletion routine. The requirement is to keep the last 90 days of backups, then the final day of each month prior to that for the current year and finally a backup from December 31st for the prior 2 years.
I couldn't find any examples other than just a single check; is it possible to do several checks to automate that.
I suggest writing a custom filter to apply the logic e.g.
filter MyCustomFilter{
$File = $_
$FileDate = $File.LastWriteTime
$Now = (Get-Date)
$FileAgeDays = ($Now - $FileDate).TotalDays
if (
# Keep files for at least 90 days
( $FileAgeDays -lt 91
) -or
# Keep files from last day of the month this year
( $FileDate.Year -eq $Now.Year -and
$FileDate.Month -ne $FileDate.AddDays(1).Month
) -or
# Keep files from 31 Dec for up to 2 years
( $FileAgeDays -lt ( 2 * 365 ) -and
$FileDate.Month -eq 12 -and
$FileDate.Day -eq 31
)
) {
# File should be kept, so nothing is returned by the filter
} else {
# File should be deleted, so pass the file down the pipeline
write-output $File
}
}
Now your overall code would look something like this:
get-childItem -path <path> |
where-object { -not $_.PSIsContainer } |
MyCustomFilter |
Remove-Item
It goes without saying, that proper testing is required before letting this loose on production systems.
EDIT: A Simpler test for last day of month
I thought of a neater 'last day of the month' test, which is simply to compare the month property of the date under test, with the month property of the following day e.g.
$FileDate.Month -ne $FileDate.AddDays(1).Month
will return $true if $FileDate is last day of the month.
Just create an array of your "special" dates and add them to your filter.
$timeLimit1 = Get-Date
$timeLimit1.AddDays(-90)
$specialDates = (Get-Date "31-12-2014"), (Get-Date "31-05-2017") # continue the list
Get-ChildItem <path> | ? {$_.LastWriteTime -le $timeLimit1 -and $specialDates -notcontains $_.LastAccessTime.Date } | Remove-Item
I would also make a function to find the "special" dates.

get files/folders not written within a specific date range

Whats the powershell command for finding files in a folder that do not meet a criteria of date range for the Get-Childitem filter of LastWriteTime.
So, check to see if a directory has files that DO NOT contain any files that have LastWriteTime between 01/10/2012 (1st Oct) to 25/10/2012 (25th Oct).
I want to display the folders that DO NOT have any files that are in that range...that way I know they are old and the whole directory can be deleted.
example of this is:
Folder1 - some files written within October - ignore this whole folder and do not display these in the results
Folder2 - NO file has LastWriteTime written in the month of October and so this folder and files should be displayed.
I know this can be done with Get-ChildItem and I am stuck on the bit below..
Get-ChildItem E:\LogArchive -Recurse | Where-Object{$_.LastWriteTime.......?
Simple solution off top of the head is this:
Where-Object{ $_.LastWriteTime -lt $startDate -or $_.LastWriteTime -gt $endDate }
where $startDate is 01/10/2012 (1st Oct) and $endDate is 25/10/2012 (25th Oct).
Problem with this approach is that it does not account for time factor in the [datetime]. So in your example, if you have 25-Oct-2012 as an upper bound, it will return files created at 25-Oct-2012 9:00, which you may not want.
Below code calculates [datetime]'s with time part truncated, based on the input [datetime], and builds the month-to-date date range of ($dayStart, $dayEnd):
$entry_date = "20-Oct-2012 9:00" #to feed current date, use this: Get-Date
$dayEnd = Get-Date $entry_date -Minute 0 -Hour 0 -Second -0;
$dayStart = Get-Date $endDate -Day 1;
$dayStart, $dayEnd
For this sample code, here is the output:
Monday, October 01, 2012 12:00:00 AM
Saturday, October 20, 2012 12:00:00 AM
Notice this approach is flexible, because you can set $entry_date to either Date or String, and it may or may not have time part - it will work in all those cases. You can then have this code in Where-Object:
Where-Object{ $_.LastWriteTime -lt $dayStart -or $_.LastWriteTime -ge $dayEnd.AddDays(1) }
Notice how -ge $dayEnd.AddDays(1) fixes the issue when comparing 25-Oct-2012 to 25-Oct-2012 9:00.
Try this:
[datetime]$startDate = "10/01/2012" # or $startDate = get-date -Date 1/10/2012
[datetime]$endDate = "10/25/2012" # or $endDate = get-date -Date 25/10/2012
? { $_.LastWriteTime -lt $startDate -or $_.LastWriteTime -gt $endDate }