Shard Server crashed with invariant failure - commonPointOpTime.getTimestamp() >= lastCommittedOpTime.getTimestamp() - mongodb

We have deployed the community edition of MongoDB in a Kubernetes cluster. In this deployment, one of the shard DB pods is crashing with a failure error message,
2022-08-08T17:46:04.110+0000 F - [rsBackgroundSync] Invariant failure commonPointOpTime.getTimestamp() >= lastCommittedOpTime.getTimestamp() src/mongo/db/repl/rollback_impl.cpp 955
We want to understand in what circumstances such an error might arise.
Going through the logs, I understand that this specific server has a replication commit point greater than the rollback common point.
https://github.com/mongodb/mongo/blob/master/src/mongo/db/repl/rs_rollback.cpp#L1995
2022-08-08T17:43:22.737+0000 I REPL [rsBackgroundSync] Starting rollback due to OplogStartMissing: Our last optime fetched: { ts: Timestamp(1658645237, 1), t: 33 }. source's GTE: { ts: Timestamp(1658645239, 1), t: 34 }
2022-08-08T17:43:22.737+0000 I REPL [rsBackgroundSync] Replication commit point: { ts: Timestamp(1658645115, 8), t: 33 }
2022-08-08T17:43:22.737+0000 I REPL [rsBackgroundSync] Rollback using 'recoverToStableTimestamp' method.
2022-08-08T17:43:22.737+0000 I REPL [rsBackgroundSync] Scheduling rollback (sync source: mongo-shareddb-3.mongo-shareddb-service.avx.svc.cluster.local:27017)
2022-08-08T17:43:22.737+0000 I ROLLBACK [rsBackgroundSync] transition to ROLLBACK
2022-08-08T17:46:04.109+0000 I ROLLBACK [rsBackgroundSync] Rollback common point is { ts: Timestamp(1658645070, 13), t: 33 }
2022-08-08T17:46:04.110+0000 F - [rsBackgroundSync] Invariant failure commonPointOpTime.getTimestamp() >= lastCommittedOpTime.getTimestamp() src/mongo/db/repl/rollback_impl.cpp 955
2022-08-08T17:46:04.110+0000 F - [rsBackgroundSync] \n\n***aborting after invariant() failure\n\n
Timestamps in the log:
1658645105 - 24 July 2022 06:45:05 GMT
1658645237 - 24 July 2022 06:47:17 GMT
1658645239 - 24 July 2022 06:47:19 GMT
1658645115 - 24 July 2022 06:45:15 GMT
1658645070 - 24 July 2022 06:44:30 GMT
commonPointOpTime = 24 July 2022 06:44:30 GMT
lastCommittedOpTime = 24 July 2022 06:45:15 GMT
What could lead to such an invalid state?

Related

What is the effect of "#ignoredot#=true" in the dev.properties file?

The following is what is in the eclipse\workspace\.metadata\.plugins\org.eclipse.pde.core\Eclipse Application\dev.properties file:
#
#Mon Dec 05 21:33:21 CST 2022
a=bin
a;1.0.0.qualifier=bin
#ignoredot#=true
What is the effect of #ignoredot#=true ?

Wildlfy Schedule overlapping

I use a scheduler in WildFly 9, with this EJBs:
import javax.ejb.Singleton;
import javax.ejb.Startup;
import javax.ejb.Schedule;
I get loads of these warnings:
2020-01-21 12:35:59,000 WARN [org.jboss.as.ejb3] (EJB default - 6) WFLYEJB0043: A previous execution of timer [id=3e4ec2d2-cea9-43c2-8e80-e4e66593dc31 timedObjectId=FiloJobScheduler.FiloJobScheduler.FiskaldatenScheduler auto-timer?:true persistent?:false timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl#71518cd4 initialExpiration=null intervalDuration(in milli sec)=0 nextExpiration=Tue Jan 21 12:35:59 GMT+02:00 2020 timerState=IN_TIMEOUT info=null] is still in progress, skipping this overlapping scheduled execution at: Tue Jan 21 12:35:59 GMT+02:00 2020.
But when I measure the elapsed times, they are allways < 1 minute.
The Scheduling is:
#Schedule(second = "*", minute = "*/5", hour = "*", persistent = false)
Has anyone an idea what is going on?

A little logging would help you. This runs every second because that's what you're telling it to do with the second="*" section. If you want to only run every 5 minutes of every hour, change the schedule to:
#Schedule(minute = "*/5", hour="*", persistent = false)

Can only do 4 concurrent futures as maximum in Scala

I thought that using futures would easily allow me to to fire off one shot code blocks, however it seems I can only have 4 futures at a time.
Where does this restriction come from, or am I abusing Futures by using it like this?
import scala.concurrent._
import ExecutionContext.Implicits.global
import scala.util.{Failure, Success}
import java.util.Calendar
object Main extends App{
val rand = scala.util.Random
for (x <- 1 to 100) {
val f = Future {
//val sleepTime = rand.nextInt(1000)
val sleepTime = 2000
Thread.sleep(sleepTime)
val today = Calendar.getInstance().getTime()
println("Future: " + x + " - sleep was: " + sleepTime + " - " + today)
1;
}
}
Thread.sleep(10000)
}
Output:
Future: 3 - sleep was: 2000 - Mon Aug 31 10:02:44 CEST 2015
Future: 2 - sleep was: 2000 - Mon Aug 31 10:02:44 CEST 2015
Future: 4 - sleep was: 2000 - Mon Aug 31 10:02:44 CEST 2015
Future: 1 - sleep was: 2000 - Mon Aug 31 10:02:44 CEST 2015
Future: 7 - sleep was: 2000 - Mon Aug 31 10:02:46 CEST 2015
Future: 5 - sleep was: 2000 - Mon Aug 31 10:02:46 CEST 2015
Future: 6 - sleep was: 2000 - Mon Aug 31 10:02:46 CEST 2015
Future: 8 - sleep was: 2000 - Mon Aug 31 10:02:46 CEST 2015
Future: 9 - sleep was: 2000 - Mon Aug 31 10:02:48 CEST 2015
Future: 11 - sleep was: 2000 - Mon Aug 31 10:02:48 CEST 2015
Future: 10 - sleep was: 2000 - Mon Aug 31 10:02:48 CEST 2015
Future: 12 - sleep was: 2000 - Mon Aug 31 10:02:48 CEST 2015
Future: 16 - sleep was: 2000 - Mon Aug 31 10:02:50 CEST 2015
Future: 13 - sleep was: 2000 - Mon Aug 31 10:02:50 CEST 2015
Future: 15 - sleep was: 2000 - Mon Aug 31 10:02:50 CEST 2015
Future: 14 - sleep was: 2000 - Mon Aug 31 10:02:50 CEST 2015
I expected them to all show the same time.
To give some context, I thought I could use this construct and extend it by having a main loop, in which it sleeps every loop according to a value drawn from a exponential disitribution , to emulate user arrival/execution of a query. After each sleep I'd like to execute the query by sending it to the program's driver (in this case Spark, and the driver allows for multiple threads using it.) Is there a more obvious way than to use Futures?

When you are using using import ExecutionContext.Implicits.global,
It creates thread pool which has the same size of the number of CPUs.
From the source of the ExecutionContext.scala
The default ExecutionContext implementation is backed by a work-stealing thread pool. By default,
the thread pool uses a target number of worker threads equal to the number of [[https://docs.oracle.com/javase/8/docs/api/java/lang/Runtime.html#availableProcessors-- available processors]].
And there's good StackOverflow question: What is the behavior of scala.concurrent.ExecutionContext.Implicits.global?
Since the default size of the thread pool depends on number of CPUs, if you want to use larger thread pool, you have to write something like
import scala.concurrent.ExecutionContext
import java.util.concurrent.Executors
implicit val ec = ExecutionContext.fromExecutorService(Executors.newWorkStealingPool(8))
before executing the Future.
( In your code, you have to place it before for loop. )
Note that work stealing pool was added in java 8, scala has their own ForkJoinPool which does the work stealing: scala.concurrent.forkjoin.ForkJoinPool vs java.util.concurrent.ForkJoinPool
Also if you want one thread per Future, you can write something like
implicit val ec = ExecutionContext.fromExecutorService(Executors.newSingleThreadExecutor)
Therefore, the following code executes 100 threads in parallel
import scala.concurrent._
import java.util.concurrent.Executors
object Main extends App{
for (x <- 1 to 100) {
implicit val ec = ExecutionContext.fromExecutorService(Executors.newSingleThreadExecutor)
val f = Future {
val sleepTime = 2000
Thread.sleep(sleepTime)
val today = Calendar.getInstance().getTime()
println("Future: " + x + " - sleep was: " + sleepTime + " - " + today)
1;
}
}
Thread.sleep(10000)
}
In addition to work stealing thread pool and single thread executors, there's some other executors: http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Executors.html
Read the docs for detail:
http://docs.scala-lang.org/overviews/core/futures.html

The default pool when using import scala.concurrent.ExecutionContext.Implicits.global indeed has as many threads as you have cores on your machine. This is ideal for non-blocking code (no synchronous io/sleep/...) but can be problematic and even cause deadlocks when you use it for blocking code.
However, this pool can actually grow if you mark blocking code in a scala.concurrent.blocking block. The same marker is for example in use when you are using Await.result and Await.ready functions that block while waiting for a Future.
see the api docs for blocking
So all you have to do is update your example:
import scala.concurrent.blocking
...
val sleepTime = 2000
blocking{
Thread.sleep(sleepTime)
}
...
Now all futures will end after 2000 ms

you can also use
`implicit val ec = ExecutionContext.fromExecutorService(ExecutorService.newFixedThreadPool(NUMBEROFTHREADSYOUWANT))`
in NUMBEROFTHREADSYOUWANT you can give number of threads want to start.
This will use before Future .

Dealing with (birthday) dates and Timezone with Dojo

I have a simple widget that has:
<input class="input" id="${id}_dateOfBirth" name="dateOfBirth" data-dojo-type="dijit/form/DateTextBox" />
Note that it's a birthday. So, it's meant to stay the same, regardless of where you are when you are (timezone shouldn't happen). If you are born on the 10th of January at 3:00AM in England, and view your personal information from New York, you are meant to still see 10th of January, NOT the 9th!
I am in GMT+8 right now.
When I submit this form, this actually gets to the server when I put in 1/1/1970:
dateOfBirth: "1969-12-31T16:00:00.000Z"
Which is bad, because it's 8 hours short of the actual date.
Basically, I need a way for the DateTextBox to show the date as it came from the server, effectively ignoring the browser's timezone.

FWIW, here is my variant of UTCDateTextBox:
define([
"dojo/_base/declare",
"dijit/form/DateTextBox"
], function(declare, DateTextBox) {
function isValidDate(value) {
return value instanceof Date && isFinite(value.getTime());
}
function toUTCDate(value) {
if (isValidDate(value)) {
value = new Date(
Date.UTC(value.getFullYear(), value.getMonth(), value.getDate())
);
}
return value;
}
return declare(DateTextBox, {
_getValueAttr : function() {
return toUTCDate(this.inherited("_getValueAttr", arguments));
}
});
});
For my use case, I found that I didn't need to override _setValueAttr(). With the above implementation, when getUTCXXX(), toUTCString(), toISOString() or toJSON() are called on the date object returned from _getValueAttr(), then the correct UTC date with zeroed time elements is returned.
Hope this helps.

After much hacking and analysing Dojo's source code, I came up with this:
var UTCDateTextBox = declare( 'UTCDateTextBox', [ DateTextBox ], {
_getValueAttr: function(){
var ov = this.inherited(arguments);
if( ov ){
ov.setTime( ov.getTime() - ov.getTimezoneOffset() * 60 * 1000 );
}
return ov;
},
_setValueAttr: function( value, priorityChange, formattedValue){
var v = stamp.fromISOString( value );
if( v ){
v.setTime( v.getTime() + v.getTimezoneOffset() * 60 * 1000 );
value = v;
}
this.inherited(arguments);
}
});
Basically:
When the value is set, the timezone difference gets added. This means that if the server has 1979-12-25T00:00:00.000Z, rather than assigning Tue Dec 25 1979 08:00:00 GMT+0800 (WST), it will assign Tue Dec 25 1979 00:00:00 GMT+0800 (WST) . Basically, the date is converted locally to whatever it was in UTC.
When the value is parsed, it will be changed from Tue Dec 25 1979 00:00:00 GMT+0800 (WST) to Tue Dec 25 1979 08:00:00 GMT+0800 (WST)
The changed value is the one submitted to the server. So, it will be correct regardless of what timezone it will be edited at.
Since I only ever ever deal with dates, if the server has 1979-12-31T23:00:00Z (which is an error: for birthdays, the time is actually ignored and mustn't matter), this will happen:
When the value is set, ISO is 1979-12-31T23:00:00.000Z. So, Tue Jan 01 1980 07:00:00 GMT+0800 (WST) is changed into Mon Dec 31 1979 23:00:00 GMT+0800 (WST). This means that the right date is placed into the date textbox (31/12/1979).
When the value is parsed from the textbox, Mon Dec 31 1979 00:00:00 GMT+0800 (WST) becomes Mon Dec 31 1979 08:00:00 GMT+0800 (WST). So, the server will save 1979-12-31T00:00:00Z -- which is, again, the correct date!
If there are bettere solutions, please let me know. Frankly, I hope there are as this one feels like a bit of a cheat!

How do I set the timezone for Perl's localtime()?

In Perl, I'd like to look up the localtime in a specific timezone. I had been using this technique:
$ENV{TZ} = 'America/Los_Angeles';
my $now = scalar localtime;
print "It is now $now\n";
# WORKS: prints the current time in LA
However, this is not reliable -- notably, if I prepend another localtime() call before setting $ENV{TZ}, it breaks:
localtime();
$ENV{TZ} = 'America/Los_Angeles';
my $now = scalar localtime;
print "It is now $now\n";
# FAILS: prints the current time for here instead of LA
Is there a better way to do this?

Use POSIX::tzset.
use POSIX qw(tzset);
my $was = localtime;
print "It was $was\n";
$ENV{TZ} = 'America/Los_Angeles';
$was = localtime;
print "It is still $was\n";
tzset;
my $now = localtime;
print "It is now $now\n";
$ perl -v
This is perl, v5.8.8 built for x86_64-linux-thread-multi
Copyright 1987-2006, Larry Wall
Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.
Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.
$ perl tzset-test.pl
It was Wed Apr 15 15:58:10 2009
It is still Wed Apr 15 15:58:10 2009
It is now Wed Apr 15 12:58:10 2009

I'd strongly suggest using a module to do this. Specifically, I'd suggest using DateTime (see Perl DateTime Wiki or CPAN
Then you should be able to do something like the following:
use strict;
use warnings;
use DateTime;
my $dt = DateTime->now(); # *your* local time assuming your system knows it!
my $clone1 = $dt->clone; # taking a copy.
$clone1->set_time_zone('America/Los_Angeles');
print "$clone1\n"; # output using ISO 8601 format (there a lot of choices)
print "$dt\n";

Whilst your code works fine for me on both Linux (Perl 5.10.0) and MacOS X (5.8.9), there is a possible solution.
The underlying C functons used by Perl (ctime(), localtime(), etc) call tzset() the first time they're invoked, but not necessarily afterwards. By calling it yourself you should ensure that the timezone structures are correctly re-initialised after any change to $TZ.
Fortunately this is easy - the tzset() function is available in the POSIX module:
#!/usr/bin/perl -w
use POSIX qw[tzset];
$ENV{'TZ'} = 'Europe/London';
tzset();
print scalar localtime();
NB: some Google searches suggest that this is only necessary with Perl versions up to and including 5.8.8. Later versions always call tzset() automatically before each call to localtime().

use Time::Zone;
my $TZ = 'America/Los_Angeles';
my $now = scalar localtime time() + tz_offset($TZ);
print "It is now $now\n";
seems to work here. (The 'scalar' is redundant here since $now gives it scalar context, but it's also nice to be explicit.)
As per the comment, I got the original problem. This seems to fix it for me, but given that others aren't having the original problem, the "seems to work here" bit is intended as an invitation for those people to try this solution as well to ensure it doesn't break anything. (I have to wonder if alnitak noticed the difference between what I posted and the original post?)

Expanding on BrianP007 answer you use both TZ and _tzset
$was = localtime;
print "It was $was\n";
$ENV{TZ} = 'CST6CDT'; # America/Chicago
Time::Piece::_tzset(); # Local time is now Chicago Time
$was = localtime;
print "It is $was\n"; # Right now in Chicago
The trick is the TZ is set from your location to GMT. So normally you would think Chicago is UTC-6, but from Chicago it is 6 hours to UTC which = 'CST6'.
See http://science.ksc.nasa.gov/software/winvn/userguide/3_1_4.htm

Executive Summary:
Setting $ENV{TZ}='/*&+000000000005' and calling Time::Piece::_tzset() fixes localtime() to agree with the windoz system clock.
Sanguinarily gory details:
On Strawberry Perl, windoz 7/64, none of the "Standard" time zones works in the TZ environmental variable to localize localtime(). 'America/Chicago' gives exactly the same time as 'America/Los_Angeles' == 'CDT' == 'CST' == 'UTC' == '-01:00', etc. The list is infinite.
Every timezone on http://www.timeanddate.com/time/zones/ that I tried gives the right time if you are in Greenwich.
Every time from: http://en.wikipedia.org/wiki/List_of_tz_database_time_zones
also fails to change localtime() at all. And, there is no apparent indication. They do nothing and say nothing.
There is NO tzset() on windoz:
POSIX::tzset not implemented on this architecture
There is not even any concept of POSIX ???
C:\bin>cpan install POSIX
...
Warning: Cannot install POSIX, don't know what it is.
Try the command i /POSIX/
It appears to be baked into win8 and there are some dot NOT libraries for it.
For Austin, Texas, in the very Center of Central Intergalactic Time, thee correct $ENV{TZ} which gives me a scalar localtime() which ~agrees with the o/s level time function and the windoz clock is:
'/*&+5' !!! Yes Slash-Star-Ampersand-Plus-5 works!
P:\br1\sxsw.2015\sx-2015.0318\done>time
The current time is: 16:36:39.44
...
Time=Apr 14 16:36:42 2015, ENV->TZ=/*&+5
By running a for loop and trying random values from various posts, for Strawberry Perl uname='Win32 strawberry-perl 5.18.2.2...' with known timezone bugs, any 3 chars I tried (didn't try + or -) followed by +/- and a small number worked. Here is an array of text values and their output below:
use Time::Piece;
#tz = ('', 'CDT+5', 'CST+5', 'FKU+5', 'XYZ+5', '+5', '+05', '+05.00',
'America/Chicago', 'America/Los_Angeles', 'CDT',
'CST', 'UTC', 'PDT', 'PST', '-01:00', '+01:00', '-05:00'.
'ACDT', 'EASST', '5000', '+0500', '+5:00', '+05:00', 'SSS+1', 'SSS+0',
'zzz-1', 'ZZ1+5', '123+5', '___+5', '/*&+5', , '/*&+05', '/*&+005',
'/*&+000000000005');
foreach $tz (#tz) {
$ENV{TZ} = $tz if $tz;
Time::Piece::_tzset() if $tz;
printf("T%s, ENV->TZ=%s\n", scalar localtime, $ENV{TZ} || 'NoTZ');
}
Most every try with anything but XXX . +|- . integer gave UTC, but many were an hour off for no reason (America/Los_Angeles and America/Chicago gave the same value). I am almost sure I used to get away with just CDT and CST, possibly on Activestate (switched to Strawberry to compile my own Perl modules rather than rely on Activestate for everything). This is the first major snarl.
I rebuilt DateTime from scratch and it worked fine. DateTime::TimeZone::Local::Win32 "failed for 'Win32::TieRegistry'"
Here's the sorted result of the attempted zones above:
P:\br1\sxsw.2015\sx-2015.0318\done>bb | sort
Running c:/bin/bb.pl Tue Apr 14 21:43:56 2015
TTue Apr 14 16:43:56 2015, ENV->TZ=/*&+000000000005
TTue Apr 14 16:43:56 2015, ENV->TZ=/*&+005
TTue Apr 14 16:43:56 2015, ENV->TZ=/*&+05
TTue Apr 14 16:43:56 2015, ENV->TZ=/*&+5
TTue Apr 14 16:43:56 2015, ENV->TZ=___+5
TTue Apr 14 16:43:56 2015, ENV->TZ=123+5
TTue Apr 14 16:43:56 2015, ENV->TZ=CDT+5
TTue Apr 14 16:43:56 2015, ENV->TZ=CST+5
TTue Apr 14 16:43:56 2015, ENV->TZ=FKU+5
TTue Apr 14 16:43:56 2015, ENV->TZ=XYZ+5
TTue Apr 14 16:43:56 2015, ENV->TZ=ZZ1+5
ABOVE ALL WORKED Below most failed with UTC or +1 hour???
TTue Apr 14 20:43:56 2015, ENV->TZ=SSS+1
TTue Apr 14 21:43:56 2015, ENV->TZ=-01:00
TTue Apr 14 21:43:56 2015, ENV->TZ=+01:00
TTue Apr 14 21:43:56 2015, ENV->TZ=+05
TTue Apr 14 21:43:56 2015, ENV->TZ=+05:00
TTue Apr 14 21:43:56 2015, ENV->TZ=+0500
TTue Apr 14 21:43:56 2015, ENV->TZ=+5
TTue Apr 14 21:43:56 2015, ENV->TZ=+5:00
TTue Apr 14 21:43:56 2015, ENV->TZ=5000
TTue Apr 14 21:43:56 2015, ENV->TZ=CDT
TTue Apr 14 21:43:56 2015, ENV->TZ=CDT
TTue Apr 14 21:43:56 2015, ENV->TZ=CST
TTue Apr 14 21:43:56 2015, ENV->TZ=PDT
TTue Apr 14 21:43:56 2015, ENV->TZ=PST
TTue Apr 14 21:43:56 2015, ENV->TZ=SSS+0
TTue Apr 14 21:43:56 2015, ENV->TZ=UTC
TTue Apr 14 22:43:56 2015, ENV->TZ=-05:00ACDT
TTue Apr 14 22:43:56 2015, ENV->TZ=+05.00
TTue Apr 14 22:43:56 2015, ENV->TZ=America/Chicago
TTue Apr 14 22:43:56 2015, ENV->TZ=America/Los_Angeles
TTue Apr 14 22:43:56 2015, ENV->TZ=EASST
TTue Apr 14 22:43:56 2015, ENV->TZ=zzz-1
Even after finding and installing the Holy Grail, the TzFile module for the Olsen Database, it is still screwed, no difference!
Installing C:\bin\strawberry_perl_5_18\perl\site\lib\DateTime\TimeZone\Tzfile.pm
ZEFRAM/DateTime-TimeZone-Tzfile-0.010.tar.gz
C:\bin\strawberry_perl_5_18\perl\bin\perl.exe ./Build install --uninst 1 -- OK
Here are all of the alleged timezones which do nothing on this platform from:
#atz = DateTime::TimeZone->all_names();
printf("All tz names [%d] = %s\n", scalar #atz, join(", ", #atz));
All tz names [349] = Africa/Abidjan, Africa/Accra, Africa/Algiers, Africa/Bissau, Africa/Cairo, Africa/Casablanca, Africa/Ceuta, Africa/El_Aaiun, Africa/Johannesburg, Africa/Khartoum, Africa/Lagos, Africa/Maputo, Africa/Monrovia, Africa/Nairobi, Africa/Ndjamena, Africa/Tripoli, Africa/Tunis, Africa/Windhoek, America/Adak, America/Anchorage, America/Araguaina, America/Argentina/Buenos_Aires, America/Argentina/Catamarca, America/Argentina/Cordoba, America/Argentina/Jujuy, America/Argentina/La_Rioja, America/Argentina/Mendoza, America/Argentina/Rio_Gallegos, America/Argentina/Salta, America/Argentina/San_Juan, America/Argentina/San_Luis, America/Argentina/Tucuman, America/Argentina/Ushuaia, America/Asuncion, America/Atikokan, America/Bahia, America/Bahia_Banderas, America/Barbados, America/Belem, America/Belize, America/Blanc-Sablon, America/Boa_Vista, America/Bogota, America/Boise, America/Cambridge_Bay, America/Campo_Grande, America/Cancun, America/Caracas, America/Cayenne, America/Chicago, America/Chihuahua, America/Costa_Rica, America/Creston, America/Cuiaba, America/Curacao, America/Danmarkshavn, America/Dawson, America/Dawson_Creek, America/Denver, America/Detroit, America/Edmonton, America/Eirunepe, America/El_Salvador, America/Fortaleza, America/Glace_Bay, America/Godthab, America/Goose_Bay, America/Grand_Turk, America/Guatemala, America/Guayaquil, America/Guyana, America/Halifax, America/Havana, America/Hermosillo, America/Indiana/Indianapolis, America/Indiana/Knox, America/Indiana/Marengo, America/Indiana/Petersburg, America/Indiana/Tell_City, America/Indiana/Vevay, America/Indiana/Vincennes, America/Indiana/Winamac, America/Inuvik, America/Iqaluit, America/Jamaica, America/Juneau, America/Kentucky/Louisville, America/Kentucky/Monticello, America/La_Paz, America/Lima, America/Los_Angeles, America/Maceio, America/Managua, America/Manaus, America/Martinique, America/Matamoros, America/Mazatlan, America/Menominee, America/Merida, America/Metlakatla, America/Mexico_City, America/Miquelon, America/Moncton, America/Monterrey, America/Montevideo, America/Montreal, America/Nassau, America/New_York, America/Nipigon, America/Nome, America/Noronha, America/North_Dakota/Beulah, America/North_Dakota/Center, America/North_Dakota/New_Salem, America/Ojinaga, America/Panama, America/Pangnirtung, America/Paramaribo, America/Phoenix, America/Port-au-Prince, America/Port_of_Spain, America/Porto_Velho, America/Puerto_Rico, America/Rainy_River, America/Rankin_Inlet, America/Recife, America/Regina, America/Resolute, America/Rio_Branco, America/Santa_Isabel, America/Santarem, America/Santiago, America/Santo_Domingo, America/Sao_Paulo, America/Scoresbysund, America/Sitka, America/St_Johns, America/Swift_Current, America/Tegucigalpa, America/Thule, America/Thunder_Bay, America/Tijuana, America/Toronto, America/Vancouver, America/Whitehorse, America/Winnipeg, America/Yakutat, America/Yellowknife, Antarctica/Casey, Antarctica/Davis, Antarctica/DumontDUrville, Antarctica/Macquarie, Antarctica/Mawson, Antarctica/Palmer, Antarctica/Rothera, Antarctica/Syowa, Antarctica/Troll, Antarctica/Vostok, Asia/Almaty, Asia/Amman, Asia/Anadyr, Asia/Aqtau, Asia/Aqtobe, Asia/Ashgabat, Asia/Baghdad, Asia/Baku, Asia/Bangkok, Asia/Beirut, Asia/Bishkek, Asia/Brunei, Asia/Chita, Asia/Choibalsan, Asia/Colombo, Asia/Damascus, Asia/Dhaka, Asia/Dili, Asia/Dubai, Asia/Dushanbe, Asia/Gaza, Asia/Hebron, Asia/Ho_Chi_Minh, Asia/Hong_Kong, Asia/Hovd, Asia/Irkutsk, Asia/Jakarta, Asia/Jayapura, Asia/Jerusalem, Asia/Kabul, Asia/Kamchatka, Asia/Karachi, Asia/Kathmandu, Asia/Khandyga, Asia/Kolkata, Asia/Krasnoyarsk, Asia/Kuala_Lumpur, Asia/Kuching, Asia/Macau, Asia/Magadan, Asia/Makassar, Asia/Manila, Asia/Nicosia, Asia/Novokuznetsk, Asia/Novosibirsk, Asia/Omsk, Asia/Oral, Asia/Pontianak, Asia/Pyongyang, Asia/Qatar, Asia/Qyzylorda, Asia/Rangoon, Asia/Riyadh, Asia/Sakhalin, Asia/Samarkand, Asia/Seoul, Asia/Shanghai, Asia/Singapore, Asia/Srednekolymsk, Asia/Taipei, Asia/Tashkent, Asia/Tbilisi, Asia/Tehran, Asia/Thimphu, Asia/Tokyo, Asia/Ulaanbaatar, Asia/Urumqi, Asia/Ust-Nera, Asia/Vladivostok, Asia/Yakutsk, Asia/Yekaterinburg, Asia/Yerevan, Atlantic/Azores, Atlantic/Bermuda, Atlantic/Canary, Atlantic/Cape_Verde, Atlantic/Faroe, Atlantic/Madeira, Atlantic/Reykjavik, Atlantic/South_Georgia, Atlantic/Stanley, Australia/Adelaide, Australia/Brisbane, Australia/Broken_Hill, Australia/Currie, Australia/Darwin, Australia/Eucla, Australia/Hobart, Australia/Lindeman, Australia/Lord_Howe, Australia/Melbourne, Australia/Perth, Australia/Sydney, CET, CST6CDT, EET, EST, EST5EDT, Europe/Amsterdam, Europe/Andorra, Europe/Athens, Europe/Belgrade, Europe/Berlin, Europe/Brussels, Europe/Bucharest, Europe/Budapest, Europe/Chisinau, Europe/Copenhagen, Europe/Dublin, Europe/Gibraltar, Europe/Helsinki, Europe/Istanbul, Europe/Kaliningrad, Europe/Kiev, Europe/Lisbon, Europe/London, Europe/Luxembourg, Europe/Madrid, Europe/Malta, Europe/Minsk, Europe/Monaco, Europe/Moscow, Europe/Oslo, Europe/Paris, Europe/Prague, Europe/Riga, Europe/Rome, Europe/Samara, Europe/Simferopol, Europe/Sofia, Europe/Stockholm, Europe/Tallinn, Europe/Tirane, Europe/Uzhgorod, Europe/Vienna, Europe/Vilnius, Europe/Volgograd, Europe/Warsaw, Europe/Zaporozhye, Europe/Zurich, HST, Indian/Chagos, Indian/Christmas, Indian/Cocos, Indian/Kerguelen, Indian/Mahe, Indian/Maldives, Indian/Mauritius, Indian/Reunion, MET, MST, MST7MDT, PST8PDT, Pacific/Apia, Pacific/Auckland, Pacific/Bougainville, Pacific/Chatham, Pacific/Chuuk, Pacific/Easter, Pacific/Efate, Pacific/Enderbury, Pacific/Fakaofo, Pacific/Fiji, Pacific/Funafuti, Pacific/Galapagos, Pacific/Gambier, Pacific/Guadalcanal, Pacific/Guam, Pacific/Honolulu, Pacific/Kiritimati, Pacific/Kosrae, Pacific/Kwajalein, Pacific/Majuro, Pacific/Marquesas, Pacific/Nauru, Pacific/Niue, Pacific/Norfolk, Pacific/Noumea, Pacific/Pago_Pago, Pacific/Palau, Pacific/Pitcairn, Pacific/Pohnpei, Pacific/Port_Moresby, Pacific/Rarotonga, Pacific/Tahiti, Pacific/Tarawa, Pacific/Tongatapu, Pacific/Wake, Pacific/Wallis, UTC, WET