I'm looking to convert Sharepoint TimeZone information into a Java TimeZone object for use with my application which is using the Sharepoint REST API. I understand that Sharepoint stores all it's timestamps in UTC but, when using the REST API, some of the timestamps are returned time zone adjusted and some are not. The format is the same for both except, as you would expect, the UTC values end with a "Z" and the time zone adjusted values do not. So, it's easy enough to convert those to Java DateTime objects but if I want to consistently return UTC values to my callers, I will have to adjust the "non-Z" values to UTC. Sharepoint allows me to get the configured TimeZone information for the Sharepoint server but what I need is a mapping from that information to the Java TimeZone ids. What Sharepoint provides through REST looks like this:
Description: (GMT-07:00) Mountain Time (US and Canada)
Bias: 420
Daylight Bias: -60
Standard Bias: 0
It doesn't provide any information that might indicate, say, when DST starts etc. but I figured I wouldn't need to worry about that as long as I can map the Sharepoint "Description" to a Java TimeZone id. So, I'm hoping that someone has run into this need before or perhaps has some other suggestion as to how I might get the UTC values I need from Sharepoint's REST API.
Java uses IANA time zones.
Mappings exist for Microsoft Windows to IANA time zones in the CLDR supplemental data.
However, Sharepoint time zones are not quite the same as regular Windows time zones. Instead of using string time zone keys like the ones found in the Windows registry (or via the Id property of .NET's TimeZoneInfo class), they use integer ids that are specific to Sharepoint. You can find a partial list here, and a more updated (unofficial) list here.
The Sharepoint time zone description loosely matches the DisplayName of the Windows time zone, but it's not a perfect match. It's entirely possible that some of the Sharepoint time zones are not defined as Windows time zones.
Assuming you mapped each Sharepoint SPTimeZone numeric id to a TimeZoneInfo id, then you could use the CLDR data to convert from there to IANA time zones. And if you can run .NET code, then you can do this quite easily with the Noda Time library, as described here.
UPDATE
I went ahead and mapped these to reasonable equivalents. These aren't guaranteed to be 100% accurate, but I think it will give you what you need.
CSV spreadsheet for the full mapping
JSON for SharePoint ID to IANA TZ name - ready for you to use in any language
These are mostly matched using the roughly equivalent Windows time zone, then to the IANA zone using the primary CLDR mapping. The only exceptions were:
SharePoint zones 33 and 81 are separate, but matched to the single "SA Western Standard Time" windows zone. They map to IANA zones America/La_Paz and America/Manaus respectively.
SharePoint zones 41 and 92 are separate, but matched to the single "Magadan Standard Time" windows zone. They map to IANA zones Asia/Magadan and Asia/Kamchatka respectively.
I've started using Matt's mappings and they look spot on. However, I noticed with Sharepoint 2013 that there are 10 more on the end of the list Matt provided. So, using the lovely list of IANA Time Zones here, http://en.wikipedia.org/wiki/List_of_tz_database_time_zones I have added these to Matt's list with my best guess from the IANA list (yep, just a best guess so if you have feedback, let me know).
"SharePoint ID","SharePoint Description","IANA Time Zone"
"95","(UTC-11:00) Coordinated Universal Time-11","Etc/GMT+11"
"96","(UTC-02:00) Coordinated Universal Time-02","Etc/GMT+2"
"97","(UTC+12:00) Coordinated Universal Time+12","Etc/GMT-12"
"98","(UTC+02:00) Damascus","Asia/Damascus"
"99","(UTC+12:00) Magadan","Asia/Magadan"
"100","(UTC+03:00) Kaliningrad, Minsk","Europe/Kaliningrad"
"101","(UTC+02:00) Istanbul","Europe/Istanbul"
"102","(UTC+06:00) Dhaka","Asia/Dhaka"
"103","(UTC-03:00) Salvador","America/Bahia"
"104","(UTC+02:00) E. Europe","Europe/Chisinau"
Related
I hate timezones, but this time I can't get by without them. I'm building an app with flutter that needs to preselect the user's timezone but still allow to change it. What timezones should I display?
After doing some research and looking at examples, I noticed 2 patterns
display timezones as city names. Ex "America/Denver"
display timezones as timezones. Ex "Mountain Standart Time"
I decided to go with the city name example because it was the easiest to code. This pattern is used in the "timezone" flutter package, iOS and Android devices, and returned when I query the user's timezone with the "flutter_native_timezone" package. This implementation, however, generated a lot of negative feedback from test users. Most people wanted to see Mountain Standard Time on the list. They also said if I'm using city names for timezones, I should pick the largest city if they share the same timezone. For example, America/Boise and America/Denver should not be displayed together because they're under Mountain Time and the largest should be included.
Can someone explain how I can solve this mess? Doing this manually is possible for 1 country, but doing this for all countries in the world is very tedious and error-prone. I would like to go with the timezone pattern, but this would require some sort of timezone resolution function that can take "America/Denver" and convert it to "Mountain Standart Time" and vice-versa.
I recommend using canonical TZ database names for timezones.
Have you checked timezone package?
This package provides the IANA time zone database and time zone aware DateTime class, TZDateTime.
They offer three different variants of the IANA database:
default: doesn't contain deprecated and historical zones with some
exceptions like "US/Eastern" and "Etc/UTC"; this is about 75% the
size of the all database.
all: contains all data from the IANA time zone database.
10y:default database truncated to contain historical data from 5
years ago until 5 years in the future; this database is about 25% the
size of the default database.
At 6:00am Sydney time, Rory Allan clicks the foofoo button in their browser.
This inserts into the foofoo table with an 8:00pm UTC timestamp in the database of my server.
A lot of people are excited about this foofoo button, and they wanted to see what time of the day Sydney time Rory clicked it.
As I stare at my screen in California and view the status of the click, I want to see that he clicked it at 6:00am, not 1:00pm, which is what I will see if I pull the UTC time from the server and let my browser convert it.
You see, a lot of different people click foofoo, such as Phillip Herman in Germany, or Ivan Efimiov in Russia. And all we care about is the relative time of the day they clicked it, in the location they clicked it, regardless of the viewing location.
I don't know the best way to do this. Do I take the local timestamp and convert it to a string, storing it in addition to the real UTC timestamp? Or is this a common problem with a common resolution that I haven't found? I'm guessing / hoping the latter.
This isn't Language specific. These dates have a long journey:
Unix timestamp > Python date > JSON > Node > Mongo > Node > Browser
Ok, clearly 8:00pm UTC is not enough information. But what do you really want to know?
Is 06:00 enough information?
Look for time-only data types, preferably those that reflect time-of-day, rather than elapsed-time. These are often called LocalTime, TimeOfDay, etc.
Or, use strings in HH:MM format (on a 24-hour clock)
Or record the total number of minutes as an integer (60 * HH) + MM.
Or if you need a higher precision, then use seconds, or milliseconds, or microseconds, etc...
Is 06:00 Australia/Sydney enough information?
Store the time as mentioned earlier and store the time zone name in a separate string.
But be careful, because without a date you don't know if UTC+10 or UTC+11 was in effect.
Is 06:00+10:00 enough information?
You could look for a time with time zone data type, such as the one that exists in PostgreSQL, though even the PostgreSQL docs strongly discourage using this type.
Instead, store the time and offset as separate components.
Be careful, because you'd not necessarily know if this data was from Sydney. There are other time zones with UTC+10 at parts of the year too.
Assuming you have the date, such as today, then:
Is 2017-05-31T06:00 enough information?
Store the date and time together in a component that is sometimes called DateTime or LocalDateTime in various languages - but be careful that it is not bound to any specific time zone. Use DateTimeKind.Unspecified in .NET, or "naive" DateTime's in Python, etc.
For MongoDB and others that don't have such a type, use a string in ISO8601 format.
Be careful, because this time could come from anywhere in the world. Nothing here relates it to a specific point in time.
Is 2017-05-31T06:00+10:00 enough information?
Some languages and databases have a DateTimeOffset or OffsetDateTime type for this purpose.
If not, you can store a "Unix Timestamp" and also store the offset from UTC separately.
Or you can just store a string in ISO8601 format with the offset included - just be very careful with regard to comparisons / sorting.
Is 2017-05-31T06:00 Australia/Sydney enough information?
Store separately the date+time component from the time zone.
Consider that a time may be ambiguous during a DST fall-back transition.
Is 2017-05-31T06:00+10:00 Australia/Sydney enough information?
Here we have everything we might possibly need. Use a ZonedDateTime in Java/Joda-Time/Noda-Time, or an "aware" DateTime in Python (pytz, dateutils, etc.), or similar types when they exist in your platform.
Watch out for timestamp with time zone, as you might expect it to store the time zone and it typically does not (despite the name).
If not available, then consider storing a Unix timestamp and separate the time zone name as a string.
As you can tell - there are a LOT of options, and it really depends on exact use case and features available in each language/platform. You'll find more details if you search/ask for each one separately.
tl;dr
Exchange date-time values in UTC as strings in standard ISO 8601 format.
Work in UTC
General rule in date-time handling is to think in UTC, work in UTC, log in UTC, share in UTC, and store in UTC. But present in zoned time for the user.
By think in UTC, I mean every programmer needs to learn to stop their parochial thinking about their own particular home time zone. Translating back-and-forth to your own zone to UTC to other zones will drive a person nuts. Think of UTC as The One True Time®. All other zones and offsets are mere variations.
This strategy is much like internationalization. The programmer uses key strings in her own human language to look up string values from the localization tool to present a value (piece of text) in the human language preferred by her user. In date-time handling, the programmer works in UTC but applies a zone preferred by her user for presenting text in the user-interface.
Moment in UTC
The basic Java class for this Instant. The Instant class represents a moment on the timeline in UTC with a resolution of nanoseconds (up to nine (9) digits of a decimal fraction).
So when Rory in Sydney clicks his button, we record an Instant object.
Instant instant = Instant.now() ;
instant.toString(): 2017-06-01T09:24:54.435Z
Zoned
To present that moment to Rory in his own time zone, we apply a ZoneId to get a ZonedDateTime object.
Specify a proper time zone name in the format of continent/region, such as America/Montreal, Africa/Casablanca, or Pacific/Auckland. Never use the 3-4 letter abbreviation such as EST or IST as they are not true time zones, not standardized, and not even unique(!).
ZoneId z = ZoneId.of( "Australia/Sydney" ) ;
ZonedDateTime zdt = instant.atZone( z );
zdt.toString(): 2017-06-01T19:24:54.435+10:00[Australia/Sydney]
Now we have two objects, instant & zdt, that both refer to the same simultaneous moment on the timeline. The only difference is wall-clock time.
If the user in California wants to see the moment of Rory's button click according to Sydney time, then we already have that solution seen above. If not, if the California user wants to see the moment of Rory's button click in her own California clock, then read on.
We can adjust into yet another region’s wall-clock time by applying another time zone.
ZoneId zAmericaLosAngeles = ZoneId.of( "America/Los_Angeles" );
ZonedDateTime zdtAmericaLosAngeles = instant.atZone( zAmericaLosAngeles );
zdtAmericaLosAngeles.toString(): 2017-06-01T02:24:54.435-07:00[America/Los_Angeles]
Now we have three objects that all represent the same simultaneous moment: instant, zdt, and zdtAmericaLosAngeles. One moment, three wall-clock times.
See this above code run live at IdeOne.com.
Time-of-day
If you literally meant you want the time-of-day only, without the date, you can extract a LocalTime object from those objects above.
But think twice about doing this, as presenting a time-only without date and zone can lead to ambiguity and misunderstanding.
LocalTime lt = zdt.toLocalTime();
all we care about is the relative time of the day they clicked it, in the location they clicked it, regardless of the viewing location
If you are really really really sure that is what you want, then combine my advice above. (But I doubt this is a wise way to go.)
LocalTime lt = LocalTime.now( ZoneId.of( "Australia/Sydney" ) ) ; // Current time-of-day in Sydney.
Library for date-time
We have been using the modern java.time classes in examples above. They are exceptional – I mean than literally. They are virtually unique. Virtually all other platforms have terrible support for date-time work. The predecessor to java.time was the Joda-Time project which was ported to .Net platform as Noda Time. Other than java.time & Noda Time, I know of no other decent library on other platforms.
ISO 8601
The ISO 8601 standard defines many sensible practical formats for textual representation of date-time values.
The java.time classes use the standard formats by default when generating & parsing strings. You have been viewing those ISO 8601 formats in examples above. Except for ZonedDateTime which wisely extends the standard by appending the name of the time zone in square brackets.
The T in the middle separates the date portion from the time-of-day portion.
For UTC, the Z on the end is short for Zulu and means UTC.
For offset-from-UTC, you see a plus/minus number of hours and minutes ahead of or behind UTC. A time zone is history of past, present, and future offsets in use by a particular region.
Database
How databases handle date-times varies widely, though poor support is most common. A database driver, such as JDBC drivers, add another layer of behavior. So no way to succinctly address that here. And this topic is already asked and answered in many other pages in Stack Overflow.
If your database lacks serious date-time support, you may be better off storing the ISO 8601 strings with the Z on the end. These values when sorted alphabetical are also in chronological order.
After allowing the user to select their timezone many applications ask if the DST adjustment should be made. Given resources like the tz database which contain past and present information on DST observances for each timezone, why do applications ask?
They shouldn't. Usually those that do are not using the tz database and have made invalid assumptions about how time zones work.
It is usually paired with a time zone selection dropdown that only lists numeric offsets, like this:
One should instead consider asking for time zone like this:
By asking for countries first, one can reduce the choice of time zones from the tz database to just a handful for the country. And since many countries only have a single time zone, sometimes the user will just need to select their country.
BTW - Both of the above graphics are from the Pluralsight course, Date and Time Fundamentals, of which I am the author. I cover this issue, and many other similar common mistakes.
You can also read more in the timezone tag wiki, in the section titled "TimeZone != Offset".
There is one common exception to this rule - Microsoft Windows. If the chosen time zone has DST, then Microsoft allows a user the option to disable it:
This is sometimes needed because there are places in the world that are not represented fully by the options Windows presents. Microsoft doesn't use the TZ database for this, but has their own time zones that they maintain.
For example, if you live in Atikokan, Ontario, Canda, the only valid selection in Windows is Eastern Time with DST disabled. Compare that with the TZ database, which has defined a zone specifically as "America/Atikokan".
This can create a problem for .NET developers, as TimeZoneInfo.Local.Id will return "Eastern Standard Time" regardless of whether the DST flag is turned on or off in the control panel. However, if it's disabled, then all of the adjustment rules will have been stripped away. In other words, TimeZoneInfo.Local != TimeZoneInfo.FindBySystemId(TimeZoneInfo.Local.Id). If the application just stores the ID, then it has no way to retrieve the time zone for somewhere like Atikokan.
I've read many a post here re: GWT date handling.
One in particular that struck a cord with me was this one
Sending a date and timezone from GAE server to GWT client
Anyhow, there's a need on a project I'm working on to be able to display days, hours, minute intervals as labels in a grid. My team has adopted an approach where all date/time instances are passed the client from the server in ISO8601 String format. The server time zone is to be respected by the client. The biz use case is that all date/time instances are in "market time", so that any browser that visits the app will see and work with dates in the "market time" timezone which happens to be GMT-05:00 (if Daylight Savings in effect) or GMT-06:00 (if Standard Time in effect).
I have posted some source on Github, here:
https://github.com/fastnsilver/gwt-datehandling-example
Particularly...
https://github.com/fastnsilver/gwt-datehandling-example/blob/master/src/main/java/me/fns/gwt/datehandling/client/util/CSTimeUtil.java
and the GWTTestCase
https://github.com/fastnsilver/gwt-datehandling-example/blob/master/src/test/java/me/fns/gwt/datehandling/client/util/CSTimeUtilTestGwt.java
in the hopes that someone can stare at the utility (and test) we're employing for date handling and help us see what we're not seeing.
EDIT
The basic problem is that CSTimeUtil#hoursInDay(Date) is not being calculated correctly in Production mode for "transition days" This method is used by other methods (like CSTimeUtil#dateToHour(Date) and CSTimeUtil#labelsForDay(Date)).
I have deployed our application with the current implementation of CSTimeUtil and it appears to work, but not quite. I'm really confused by alternate test results when e.g., mvn gwt:test is run in GWT Mode or Production Mode on Windows where the OS timezone is set to various timezones other than U.S. GMT-05:00 or GMT-06:00.
Based on some hints from Andrei and some serious blood, sweat and tears, I figured this out on my own. I have updated the code in Github, so if you're curious please go have a look there.
The basics:
Make sure all Strings are ISO8601 (no millis) compliant when sent from server to client and vice versa
Use DateTimeFormat.getFormat("yyyy-MM-ddTHH:mm:ss.SZZZZ") to format and parse dates
Retreive GMT-prefixed time zone info from java.util.Date in "Market time" using DateTimeFormat(Date, TimeZone), where TimeZone param is set as TimeZone.createTimeZone(TZ_CONSTANTS_INSTANCE.americaChicago()) and time zone String retrieved by TimeZone.getISOTimeZoneString(Date)
Generating days, see generateDay(Date, int) or hours generateHour(Date, int), from a source date had to take into consideration that an increment or decrement coudl trigger a change in time zone offset if occurring on a "transition day".
If you time zone is fixed, why would you use a string to represent date/time? You can send a standard Java Date object to the client. If you want, you can even store all dates and times as Longs and pass Longs only. You also send the GWT's TimeZone Json string for your time zone (once per session). You can find it in the GWT - there is a file with strings for all time zones.
On a client you use DateTimeFormat with many predefined formats to display whatever you need: full date, month and date, date and time, etc. Just remember to create TimeZone object from this Json string and use it in DateTimeFormat.getFormat(...).format(Date, TimeZone).
With this approach you don't have to worry about DST changes (they are encoded in that Json string) and locales. You only pass simple Date or Long objects.
I'm using GWT on the client (browser) and Joda Time on the server. I'd like to perform some DB lookups bounded by the day (i.e. 00:00:00 until 23:59:59) that a request comes in, with the time boundaries based on the user's (i.e. browser) timezone.
So I have the GWT code do a new java.util.Date() to get the time of the request, and send that to the server. Then I use Joda Time like so:
new DateTime(clientDate).toDateMidnight().toDateTime()
The trouble of course is that toDateMidnight(), in the absence of a specified TimeZone, will use the system's (i.e. the server's) TimeZone. I've been trying to find a simple way to pass the TimeZone from the browser to the server without much luck. In GWT I can get the GMT offset with:
DateTimeFormat.getFormat("Z").fmt(new Date())
which results in something like "-0400". But Joda Time's DateTimeZone.forID() wants strings formatted like "America/New_York", or an integer argument of hours and minutes. Of course I can parse "-0400" into -4 hours and 0 minutes, but I'm wondering if there is not a more straightforward way of doing this.
You could use java.util.Date's getTimezoneOffset() method. It's deprecated, but that's pretty usual for Date handling in GWT currently.
And AFAIR, you can specify something similar to "UTC+4" in Joda time.
Update: I looked it up, and it's "+04:00". Or use DateTimeZone.forOffsetHours() or even forOffsetMillis().
Gwittir (http://www.gwtsite.com) is a library for GWT that includes many cool utilities, like databinding, animation, reflection, and more. However, there are some other interesting goodies as well like the new Joda Time integration. If you have ever been frustrated by GWT’s lack of java.util.Calendar support, you’ll love this, as it makes it easy to do date manipulations in your applications.
otherwise, there are other ways to get timezone offset with + & -.
import java.util.TimeZone;
use: TimeZone.getDefault().getRawOffset()
this function will return the offset time in millisecond about your phone seeting. For Example, GMT-04:00 is equals to (-4)*60*60*1000 = -14400000.
After some operations to get the number which you want.
I have a similar but slightly different problem I think.
I actually need to store the clients timezone on the server, so that I can send out messages about dates stored in their calendar.
The dates are stored in UTC time in google app engine and of course I can store the current Timezone offset when creating the appointment. The problem comes when for instance I want to send out a summary email with a list of upcoming appointments in it. These appointments need to be offset with the correct Timezone adjustments for the client (Im happy to assume that they are still in the same timezone as when they created the appointment).
The real problem comes with Daylight Savings adjustments, so for instance I might have appointments stored for Saturday 30th October 2010 at 1pm (BST[GMT+60]) and Monday 1st November 2010 at 1pm (GMT).
So as you can imagine, I cant just use the current timezone offset (BST) as that would mean that the appointment on Monday 1st November would be listed as 2pm rather than 1pm (GMT+60)
It occurs to me that the best way to deal with this is just to store the timezone offset with each appointment individually, but I feel it would be much better to be able to determine the original timezone correctly in the first place, then just let java do the correct adjustments.