I'm losing the nanoseconds from the MongoDb interface for the ISODate object. All the nanoseconds are set to zero when I read them in perl.
First, my environment:
MongoDB version: 1.8.2
perl v5.12.4
MongoDB perl module version: 0.701.4
I have a Mongo DB that has rtcTime coded as an ISODate, as follows:
"rtcTime" : ISODate("2013-05-13T18:54:55.918Z")
The code to extract the rtcTime looks something like this:
my #results = $db->get_collection( 'timings' )->find( )->all();
foreach my $record ( #results )
{
print $record->{rtcTime}->nanoseconds()."\n";
}
Output is all 0's.
To fully reproduce the problem, create an ISODate object with arbitrary (non-zero) hires_epoch values in the MongoDB database. Then try to use the MongoDB / DateTime / DateTime::Format::ISO8061 modules to extract any kind of hires time data.
Q: Why can't I get my milliseconds, microseconds, or nanoseconds from the MongoDB ISODate data?
MongoDB stores documents in BSON format and its specification says:
BSON Date is a 64-bit integer that represents the number of milliseconds since the Unix epoch (Jan 1, 1970).
So, date precision is limited to miliseconds. If you really need to store nanoseconds, you shouldn't use date type. You have to use a long and store the timestamp, so that you'll no lose precision.
This is a bug in the way that the MongoDB Perl driver interacts with DateTime. When ISODate values are retrieved from the database, the driver initializes the DateTime object using its from_epoch constructor. But the nanosecond portion is not passed to the constructor.
DateTime does support passing the full value including nanoseconds, and the driver should be updated to fix that.
I've created a ticket for this bug and I will fix it. But maybe not until after the holiday weekend. :)
Related
It seems like all the date type in Mongo DB are saved as UTC time. Is there anyway to configure from the MongoDB server side that I can change the default date type to display as local time? For example, to display/save as CTC time on the DB? Thanks,
MongoDB stores datetime as the number of milliseconds since epoch. No timezone information is stored with the data.
The value returned by the database would look something like 1614847506286.
How that is displayed is completely up to the application.
I already have a SO question and answer on how to convert BSON Timestamp in a MongoDB aggregation, but now I have a situation where I would like to convert in node.js.
So just to repeat. My goal is to convert a "Timestamp" datatype to a javascript date, without doing it in an aggregation - is this possible?
If the BSON library you are using provides Timestamp type, you can use its getTime method to return the seconds, and create a date from that:
Date(object.clusterTime.getTime())
If you don't have that function available, the timestamp is a 64-bit value where the high 32-bits are the seconds since epoch, and the low 32-bits are a counter.
Bitshift the value 32 bits right or divide by 2^32 to get the seconds:
Date(object.clusterTime/Math.pow(2,32))
Spring Data Mongodb auto parser date object to '$date': millisecond, this query consume more than 1 min to return data, I find ISODate query is more quickly than $date, How I make Spring Data Mongodb parser date object to ISODate.
It is base on your DataType, As i use Instant, mongodb auto convert it to ISODate.
But as you said the date type as millisecond, that cost many time to return data, you should check your query first....
Please show the Object and query for more support
When storing a timestamp in mongodb should I store the 'unix-time' millis since some date or should I store an actual date/time?
What are the benefits of either?
Edit
To be more specific should I store a long that is seconds since Jan 1. 1970, or should I store a Javascript date object.
I recommend using the MongoDB native date format. The advantage of that is that you can then easily use the date operators:
http://docs.mongodb.org/manual/reference/operator/aggregation-date/
I have various timestamps stored in Mongo collections, some as floats and some as ints.
They are all stored in BST and the server will be switched to UTC soon. How do I convert them within Mongo to be UTC timestamps?
In MySQL I can do this:
UPDATE `table` SET `field` = CONVERT_TZ(`field`, 'Europe/London', 'UTC');
Is there a Mongo equivalent?
You'll have to use your language of choice and update them one at a time. It should be as simple as a for loop that loads the data and rewrites it.
Just double-check how your language of choice handles timestamps across timezones. Making such a data change can have all kinds of unexpected effects on production code.
Timestamps are generally in UTC and not in a specific timezone. All date/time libraries I've worked with return timestamps that are the number of seconds (or milliseconds) since Jan 1st 1970 UTC. Check the documentation for the library you used to create the timestamp to be sure.
This means that you should be ok, unless you have used a date/time library that does not follow this convention, or somehow calculated the timestamps yourself and accounted for the timezone.
For example in JavaScript if you store the value returned from new Date().getTime() and later pass that value to new Date(...) on a different system you will end up with the same absolute date/time, regardless of the timezones of the two systems. The same goes for Ruby, do Time.new.to_i on one machine, and run Time.at(...) on another and you will get the same absolute date/time. When I say "absolute date/time" I mean that it's UTC time will be the same, most likely the system will display it in the local time zone, but that is what you want.
Some points to consider about date in Mongo:
All dates are stored in UTC in MongoDB
MongoDB stores dates internally as a 64 bit integer representing
milliseconds since 1970-01-01T00:00:00Z
If the date value you provide is not already in UTC it
will be converted to UTC before being stored in MongoDB by the driver
The recommendation is not to use DateTime.Parse.
You will get into all sorts timezone issues regarding how DateTimes are formatted.
Instead just use one of the DateTime constructors with UTC flavor.
Example:
var dateTime = new DateTime(2011, 5, 5, 0, 0, 0, DateTimeKind.Utc);
Hope you find it useful.