How to fix incorrect Date due to timezone in Core Data from using Calendar.current.startOfDay? - swift

I have erroneously used Calendar.current.startOfDay(for: Date()) to populate a date attribute in Core Data. This means that when users cross different timezones I may have different dates unintentially stored in the date attribute field e.g.
Timezone 1 - 25th 23:00
Timezone 2 - 25th 22:00
Timezone 3 - 26th 05:00
I need to update the Calendar to use UTC Timezone but I need to also perform a migration so that the existing entries in Core Data read like this…
Result:
Timezone 1 - 26th 00:00
Timezone 2 - 26th 00:00
Timezone 3 - 26th 00:00
What are the steps to perform this migration. If I do a UTC startOfDay on it Timezone 1 would get 25th 00:00 instead of 26th 0:00 which is what it should be. Is it possible to accurately update existing entries?
Edit:
For some context I need a reliable way to get all the entries for the 26th for example. I used startOfDay to store the date as it meant I could query by it too and have the relevant entry returned (at any moment in time get the startOfDay and it will give me the entries for the whole day). For historical dates I can do the same - let's say the user has navigated back 2 days I can take startOfDay and subtract 2 days using Calendar.current.date(byAdding: .day, value: -2, to: date) and query for that.
So now the timezone breaks the above logic but is there some way to fix this? If I loop through the entries I can figure out the date it was supposed to be for and perhaps change the attribute to a string - e.g. 26-05-2021 or start to store day, month, year instead and query that.
From reading your answer Duncan I don't think I want to use UTC calendar as it would start to store the entry against the incorrect date from the users perspective dependent on their timezone e.g. user moves to next day and utc is still on previous.
Edit 2:
In a migration I will take the date that is stored and map it to new day, month and year properties storing those instead by getting them from Calendar.current.dateComponents([.day, .month, .year], from: date). Then instead of query by date I will query by day month and year of the Calendar.current where the user is. The side effect here is there is potential the user adds something for today (27th) changes timezone and sees 26th data but I don't think it can be avoided and the old data will then show as intended.

If you took the current time and used Calendar.current.startOfDay(for: Date()) to calculate midnight in the user's local time zone, you have a loss of information. You don't know what time of day the operation was performed. If you saved the time of day in the local time zone in another field, you could reconstruct a Date in UTC.
It isn't clear that what you did was wrong. The day, month, and year is only meaningful in a specific time zone. I am in the Washington DC metro area. We are in daylight savings time (EDT). It is currently 20:56 on the 26th of May. However, it's 1:56 AM on the 27th of May in London, 2:57 AM in Munich, and 3:57 AM in Tel Aviv. All at the exact same instant in time. In UTC it is 0:57 AM on the 27th of May.
Most people think of the calendar date in their local time zone. That is their frame of reference. If you ask me the date right now, I'll tell you it's the evening of the 26th of May. Unless I know you are in a different time zone, that's the "right" answer to me.
If I start out at midnight on a given day in my time zone, calling Calendar.current.startOfDay(for: Date()) each hour, I'll get midnight that day for all 24 hours in my local time zone. For the first 20 hours of the day, that would be the same result I would get if I created a Calendar in UTC and made the same call. However, at 20:00 EDT, I would start getting the next calendar day if I made the same query in UTC.
If you don't know what time of day you made the call to Calendar.current.startOfDay(for: Date()), there is no foolproof to figure out the day/month year in UTC at the instant you made the call. It depends on the time of day in the local timezone, and that timezone's offset from UTC.
Consider this code:
var calendarUTC = Calendar(identifier: .gregorian)
if let utcTimeZone = TimeZone(identifier: "UTC") {
print("Valid time zone")
calendarUTC.timeZone = utcTimeZone
}
print ("Start of day in UTC is \(calendarUTC.startOfDay(for: Date()))")
print ("Start of day in local time zone is \(Calendar.current.startOfDay(for: Date()))")
That outputs:
Start of day in UTC is 2021-05-27 00:00:00 +0000
Start of day in local time zone is 2021-05-26 04:00:00 +0000
That's because right now, which is 20:56 on 26 May in my time zone, it's 0:56 on 27 May in UTC. So if I ask the UTC calendar for the start of day for now (Date()) I get midnight on 27 May, in UTC.
If I ask the same question of my local calendar, I get midnight on 26 may in my time zone, which is 4:00 AM on 26 May in UTC.
If I ran the same code this morning at 8:00 AM in my time zone, I would have gotten the output:
Start of day in UTC is 2021-05-26 00:00:00 +0000
Start of day in local time zone is 2021-05-26 04:00:00 +0000
(Since at 8:00 AM on 26 May in EDT is also 26 May in UTC.)

It's tricky and not 100% reliable and only works if you know that all days were created using startOfDay. But first you need to decide what you want. Say one date was created at 10pm in the New York, and one at exactly the same moment in London, at 4am the next day. What day do you want to be stored?
If your date stored is 25th, 10pm, then you know it was created in a timezone where the day started at 10pm UTC. You are lucky, there are only two time zones that would have created this, one without DST, one with DST. So you know it happened in one of these two time zones, within 24 hours.
Unfortunately, time zones cover 26 hours. Fortunately, only some islands in the Pacific Ocean have same time and different dates (+13 and -11 hours). For these places, you cannot possibly know which date is correct, but very few people would be affected.

Related

utcNow in Power Automate is using incorrect day (`dd`)

I'm running into a timezone issue when running my Flows on a schedule - and I can't seem to find a solution for it.
My timezone is currently +10:00 from UTC - Sydney, Australia.
For simplicities sake, I have a reoccurrence trigger running at 06:00 (6am local time) and 12:00 (12pm local time).
When I run my Actions - anything that needs a StartTime and an EndTime - I pass in the following parameters:
StartTime
Endtime
formatDateTime(utcNow(), 'yyyy-MM-dd"T"22:00:00.0000000Z')
formatDateTime( addDays(utcNow(), 1), 'yyyy-MM-dd"T"08:00:00.0000000Z')
The 22:00:00.0000000Z and 08:00:00.0000000Z above translate directly to 8am and 6pm respectively - essentially the entire work day hours.
Issue
When the trigger runs on the first occurrence, the StartTime and the EndTime are outputted to these values:
Tigger time - local
UTC date time
Start time UTC parameter
End time UTC parameter
2022-09-27 06:00
2022-09-26 20:00
2022-09-26T22:00:00.000000Z
2022-09-27T08:00:00.000000Z
When the trigger runs on the second occurrence, the StartTime and the EndTime are outputted to these values:
Tigger time - local
UTC date time
Start time UTC parameter
End time UTC parameter
2022-09-27 12:00
2022-09-27 02:00
2022-09-27T22:00:00.000000Z
2022-09-28T08:00:00.000000Z
Because the UTC date time has ticked over past midnight, the date value (dd) is now the next day due to the addDays(utcNow(), 1) expression.
This causes issues on the scripting of the Flow as when limiting the StartTime and EndTime of other actions, depending on when the Trigger runs is either looking at the local time's "Today" events or "Tomorrow" events.
For example, if I schedule an Out of Office / Automatic reply - and I schedule it to run on the 06:00 trigger, it will set my Automatic Reply to today - local time - 8am to 6pm. However, when it runs at the second trigger, 12:00 it will set my Automatic Reply to tomorrow - local time - 8am to 6pm.
Though this is not how I am using it, you can see it affects what actions do and perform.
Is there a way to ensure that it is always working of the current local date (dd) regardless if the UTC time has ticked over?
Have you tried using the addHours function instead of hard coding the hours in the formatdatetime function?
For the 6 am run I would try for the start time
formatDateTime(addHours(utcNow(), 2), 'yyyy-MM-ddTHH:mm:ss.0000000Z')
And for the end time
formatDateTime(addHours(utcNow(), 12), 'yyyy-MM-ddTHH:mm:ss.0000000Z')
You could also use a check to see which occurence run it is an combine that with the addhours approach.
equals(utcNow('HH'), '20')
It looks like, you want to do your Start time and End time calculations based on your LOCAL date, but you are doing the calculation based on the UTC date and thus get the wrong day, if UTC ticked over.
I guess the solution would be, to do your Start and End time calculations based on the LOCAL time instead of utcNow() and then transform the resulting LOCAL Start and End times into UTC times to be used as Start Time and End Time parameters.
Something to the effect of:
convertFromUtc(utcNow(),'+10:00','yyyy-MM-dd"T"06:00:00.000000')
to get the local start time of the current day and then from there
convertToUtc(localStartTime,'+10:00')
to get the UTC start time with the proper day.
And the same again for the End Time.
Though the other solutions seem logical in their workings, they dont account for daylight savings and other date time issues.
For the solution I ended up hard coding some of the data and calculations:
Action: "Convert time zone"
Base time: trigger time
Format string: Universal sortable date/time pattern [u]
Source: +10:00
Destination: UTC
Action: "Compose"
Expression: formatDateTime( body('Convert_time_zone'), 'dd')
Then for the start time UTC:
Action: Initialise variable
Name: Output Start Date Time
Type: String
Value: yyyy-MM-[Output from Compose*]"T"22:00:00.0000000Z')
* in that [Output from Compose] I made it an expression so I could subtract 1 day, using: sub( int(outputs('Compose')), 1)
This way I could always get my local date (dd) then manually shift the date, and set it as well as the time for start time.
Then I would use that local date for the End date time, and manually set the time.
Not the cleanest solution, but it works 100% of the time regardless of daylight savings or any time conversions. It is however not transferrable to other users without them editing the info.

Swift - Handling Firebase Timestamp

I am working on firebase project, I have Timestamp field in a Collection, which looks below image
Firebase screenshot
When I read that time stamp from Indian time zone, I am getting date string as Sun Jan 26, 2020 12:00 AM
But When I read it from Pago Pago - American Samoa(GMT-11) time Zone, I am getting date string as Sat Jan 25, 2020 07:30 AM
I am using below code
let formatter = DateFormatter()
formatter.dateFormat = "EEE MMM dd, yyyy hh:mm a"
let coll = Firestore.firestore().collection("Collection1")
coll.getDocuments { (snap, error) in
for doc in snap!.documents {
let date = doc["createdOn"] as! Timestamp
print(formatter.string(from: date.dateValue()))
}
}
In the firebase I configured that datefield value as Sun Jan 26, 2020 12:00 AM UTC +5:30.
My question is regardless of time zone, I want the date string result to be Sun Jan 26, 2020, how can I achieve this
Firestore timestamps don't have a sense of local timezone. What you see in the Firebase console is the timestamp interpreted in the timezone configured on the local machine. It will appear differently when viewed from different timezones, but the point in time represented by the timestamp always remains the same. Internally, timestamps use UTC. From the API documentation:
A Timestamp represents a point in time independent of any time zone or
calendar, represented as seconds and fractions of seconds at
nanosecond resolution in UTC Epoch time. It is encoded using the
Proleptic Gregorian Calendar which extends the Gregorian calendar
backwards to year one. It is encoded assuming all minutes are 60
seconds long, i.e. leap seconds are “smeared” so that no leap second
table is needed for interpretation. Range is from 0001-01-01T00:00:00Z
to 9999-12-31T23:59:59.999999999Z. By restricting to that range, we
ensure that we can convert to and from RFC 3339 date strings.
If you want to format the time according to a specific local timezone, you will have to make use of some library to help you with this. The timestamp object itself will not accomodate your local time formatting needs.

how to interpret postgres timezon

I am working on resolving a timezone issue and just want to understand how to interpret the timezone if I have got it right
2018-01-04 22:31:23.1+00
date time.1+00 ..any simple explanation on how can I know which time zone is this ?
2018-01-04 22:31:23.1+00
Year: 2018
Month: 01 (January)
Day: 04
Hour: 22 (10pm)
Minute: 31
Second: 23.1 (.1 is one-tenth of a second)
Offset: +00 (hours East of GMT)
Note, you cannot determine which time zone this is in - only the offset from UTC/GMT applicable at that timestamp.
Since multiple time zones can use UTC+00, you don't know which of those time zones it is in. It might just be in UTC, but it might be in Atlantic/Reykjavik, or Africa/Casablanca. One uses UTC year round, the other switches between UTC and UTC+1. There are others as well.

Handling Daylight Saving Time for scheduling purposes

I am working with an API that generates time periods based on a configured set of parameters.
So for example, I can specify I want 12 one month periods starting Jan 1st at midnight, and therefore the API will generate 12 monthly periods
01 Jan 2016 00:00:00 – 31 Jan 2016 23:59:59
01 Feb 2016 00:00:00 – 28 Feb 2016 23:59:59
Through to
01 Dec 2016 00:00:00 – 31 Dec 2016 23:59:59
Now the API expects a start date param supplied for the sequence of periods to be an ISO formatted string in UTC. So I’m currently in the UK, therefore if I choose to start the monthly periods from Jan 1st 2016 this would be 2016-01-01T00:00:00Z and is what I supply to the API call I am making to say when the first instance of my monthly periods should begin.
So now if I view the start and end dates of the generated monthly periods via the API, I will see them come back as
2016-01-01T00:00:00Z - 2016-01-31T23:59:59Z
2016-02-01T00:00:00Z - 2016-02-28T23:59:59Z
Etc to
2016-12-01T00:00:00Z - 2016-12-31T23:59:59Z
Something struck me about these generated periods and that is I want them to begin at midnight for where I currently am, but a period that is affected by GMT Daylight Time, so say my April period will look like so in the generated period from the API
2016-04-01T00:00:00Z - 2016-04-30T23:59:59Z
Parsing the start date for the above into a date object for viewing in the client (on my machine) will show up as
Fri Apr 01 2016 01:00:00 GMT+0100 (GMT Daylight Time)
So it’s saying that period starts at 1am and not midnight.
Now say if I wanted 12 months to be generated from 1st Jun when Daylight Savings is in effect.
My client side code currently will supply a start date to the API of 2016-05-31T23:00:00Z. This causes the API to generate start dates for each monthly period as being
2016-05-31T23:00:00Z - 2016-06-31T22:59:59Z (June Period)
2016-06-31T23:00:00Z - 2016-07-31T22:59:59Z (July)
Etc to
2017-04-30T23:00:00Z - 2017-05-31T22:59:59Z (May)
But now for a period that is not in GMT Daylight Time, so Jan for example it will show up as
2016-12-31T23:00:00Z - 2017-01-31T22:59:59Z
Meaning my client will see that start date as
Sat Dec 31 2016 23:00:00 GMT+0000 (GMT Standard Time)
So not Jan 1st at 00:00
Does this suggest that the API should know about the users timezone so the period generation logic in the API can factor this is when calculating the start and end dates?
Maybe I'm over thinking things here?!
The only real question I see here is:
Does this suggest that the API should know about the users timezone so the period generation logic in the API can factor this is when calculating the start and end dates?
In general, the answer is YES. Any time you are mapping calendar dates back to specific instants in time, then by definition you must involve time zones. That time zone can be that of the user's, or of some specific business location, or fixed to UTC, but they indeed part of the logic.
Consider that not every calendar date has a valid local "midnight" in every time zone. An example is 2015-10-18 in Sao Paulo Brazil, where the first moment of that day was 1:00 AM due to their spring-forward daylight saving time transition. There are many other examples of this around the world.
The only way to avoid the problem entirely is to not deal in specific times. If you are only working with whole dates, then your code can certainly compute the exact dates of each month without knowing anything about time zones. 2016-01-01 through 2016-01-31 has no awareness of time or time zone. It's only when you try to ask what the current date is, or if now is within that range, or if some other specific instant in time is within that range that you have to start thinking about time zones.

PostgreSQL date() with timezone

I'm having an issue selecting dates properly from Postgres - they are being stored in UTC, but
not converting with the Date() function properly.
Converting the timestamp to a date gives me the wrong date if it's past 4pm PST.
2012-06-21 should be 2012-06-20 in this case.
The starts_at column datatype is timestamp without time zone. Here are my queries:
Without converting to PST timezone:
Select starts_at from schedules where id = 40;
starts_at
---------------------
2012-06-21 01:00:00
Converting gives this:
Select (starts_at at time zone 'pst') from schedules where id = 40;
timezone
------------------------
2012-06-21 02:00:00-07
But neither convert to the correct date in the timezone.
Basically what you want is:
$ select starts_at AT TIME ZONE 'UTC' AT TIME ZONE 'US/Pacific' from schedules where id = 40
I got the solution from this article is below, which is straight GOLD!!! It explains this non-trivial issue very clearly, give it a read if you wish to understand pstgrsql TZ management better.
Expressing PostgreSQL timestamps without zones in local time
Here is what is going on. First you should know that 'PST timezone is 8 hours behind UTC timezone so for instance Jan 1st 2014, 4:30 PM PST (Wed, 01 Jan 2014 16:00:30 -0800) is equivalent to Jan 2nd 2014, 00:30 AM UTC (Thu, 02 Jan 2014 00:00:30 +0000). Any time after 4:00pm in PST slips over to the next day, interpreted as UTC.
Also, as Erwin Brandstetter mentioned above, postresql has two type of timestamps data type, one with a timezone and one without.
If your timestamps include a timezone, then a simple:
$ select starts_at AT TIME ZONE 'US/Pacific' from schedules where id = 40
will work. However if your timestamp is timezoneless, executing the above command will not work, and you must FIRST convert your timezoneless timestamp to a timestamp with a timezone, namely a UTC timezone, and ONLY THEN convert it to your desired 'PST' or 'US/Pacific' (which are the same up to some daylight saving time issues. I think you should be fine with either).
Let me demonstrate with an example where I create a timezoneless timestamp. Let's assume for convenience that our local timezone is indeed 'PST' (if it weren't then it gets a tiny bit more complicated which is unnecessary for the purpose of this explanation).
Say I have:
$ select timestamp '2014-01-2 00:30:00' AS a, timestamp '2014-01-2 00:30:00' AT TIME ZONE 'UTC' AS b, timestamp '2014-01-2 00:30:00' AT TIME ZONE 'UTC' AT TIME ZONE 'PST' AS c, timestamp '2014-01-2 00:30:00' AT TIME ZONE 'PST' AS d
This will yield:
"a"=>"2014-01-02 00:30:00" (This is the timezoneless timestamp)
"b"=>"2014-01-02 00:30:00+00" (This is the UTC TZ timestamp, note that up to a timezone, it is equivalent to the timezoneless one)
"c"=>"2014-01-01 16:30:00" (This is the correct 'PST' TZ conversion of the UTC timezone, if you read the documentation postgresql will not print the actual TZ for this conversion)
"d"=>"2014-01-02 08:30:00+00"
The last timestamp is the reason for all the confusion regarding converting timezoneless timestamp from UTC to 'PST' in postgresql. When we write:
timestamp '2014-01-2 00:30:00' AT TIME ZONE 'PST' AS d
We are taking a timezoneless timestamp and try to convert it to 'PST TZ (we indirectly assume that postgresql will understand that we want it to convert the timestamp from a UTC TZ, but postresql has plans of its own!). In practice, what postgresql does is it takes the timezoneless timestamp ('2014-01-2 00:30:00) and treats it as if it WERE ALREADY a 'PST' TZ timestamp (i.e: 2014-01-2 00:30:00 -0800) and converts that to UTC timezone!!! So it actually pushes it 8 hours ahead instead of back! Thus we get (2014-01-02 08:30:00+00).
Anyway, this last (un-intuitive) behavior is the cause of all confusion. Read the article if you want a more thorough explanation, I actually got results which are a bit different then their on this last part, but the general idea is the same.
I don't see the exact type of starts_at in your question. You really should include this information, it is the key to the solution. I'll have to guess.
PostgreSQL always stores UTC time for the type timestamp with time zone internally. Input and output (display) are adjusted to the current timezone setting or to the given time zone. The effect of AT TIME ZONE also changes with the underlying data type. See:
Ignoring time zones altogether in Rails and PostgreSQL
If you extract a date from type timestamp [without time zone], you get the date for the current time zone. The day in the output will be the same as in the display of the timestamp value.
If you extract a date from type timestamp with time zone (timestamptz for short), the time zone offset is "applied" first. You still get the date for the current time zone, which agrees with the display of the timestamp. The same point in time translates to the next day in parts of Europe, when it is past 4 p.m. in California for instance. To get the date for a certain time zone, apply AT TIME ZONE first.
Therefore, what you describe at the top of the question contradicts your example.
Given that starts_at is a timestamp [without time zone] and the time on your server is set to the local time. Test with:
SELECT now();
Does it display the same time as a clock on your wall? If yes (and the db server is running with correct time), the timezone setting of your current session agrees with your local time zone. If no, you may want to visit the setting of timezone in your postgresql.conf or your client for the session. Details in the manual.
Be aware that the timezone offset used the opposite sign of what's displayed in timestamp literals. See:
Peculiar time zone handling in a Postgres database
To get your local date from starts_at just
SELECT starts_at::date
Tantamount to:
SELECT date(starts_at)
BTW, your local time is at UTC-7 right now, not UTC-8, because daylight savings time is in effect (not among the brighter ideas of the human race).
Pacific Standard TIME (PST) is normally 8 hours "earlier" (bigger timestamp value) than UTC (Universal Time Zone), but during daylight saving periods (like now) it can be 7 hours. That's why timestamptz is displayed as 2012-06-21 02:00:00-07 in your example. The construct AT TIME ZONE 'PST' takes daylight saving time into account. These two expressions yield different results (one in winter, one in summer) and may result in different dates when cast:
SELECT '2012-06-21 01:00:00'::timestamp AT TIME ZONE 'PST'
, '2012-12-21 01:00:00'::timestamp AT TIME ZONE 'PST'
I know this is an old one but You may want to consider using AT TIME ZONE "US/Pacific" when casting to avoid any PST/PDT issues. So
SELECT starts_at::TIMESTAMPTZ AT TIME ZONE "US/Pacific"
FROM schedules
WHERE ID = '40';