saving a clientside date as UTC date object into Mongo - mongodb

I am trying to save a date into meteor mongodb my challenge is as follows:
1) if i use new Date() it creates a date object in mongo DB however it saves the time as local time as javascript Date() this always comes with a timezone +0x:hours based on browser local timezone. When i retrieve this it causes havoc as i am assuming everything in my db is UTC.
2) I want to use moment js library which is great because it can represent dates in UTC properly but my challenge is how do i get mongo db to accept a moment time? The minute i use moment.format() it saves it as a string!
So how can i send a date to a mongodb insert command with a date object that is in UTC? string just dont work :(
Any help would be appreciated.
Thanks

I think everything you need to know about both of these questions can be found here and here.
TLDR:
If you directly insert/update from the client you will store a timestamp based on the user's clock. It will still be stored as UTC, but you may or may not want to trust that the time is correct. I strongly suggest using a method for any db modifications which involve time so that the server's version of time will always be used.
Moment objects are not serializable to a format compatible with mongodb. Use a date object and format it on the client.

The problem with saving dates on the client is that each client can have a different time zone, or even wrong time set. Thus the only solution is to have the date set on the server. Using a method for each insert / update is not an elegant solution.
A common practice is to modify the document inside allow or deny callback:
Messages.allow({
insert: function(userId, doc) {
...
doc.timestamp = new Date();
return true;
},
});
That way you ensure all documents have a compatible timestamp, and you can use usual db methods on the client.

The Meteor community recently started an extensive document about how to use dates and times. You'll find a lot of useful information there, in addition to David Weldon's links:
https://meteor.hackpad.com/Meteor-Cookbook-Using-Dates-and-Times-qSQCGFc06gH
However, in particular I recommend using https://github.com/mizzao/meteor-timesync when security is not a concern. It allows you to client-locally obtain an accurate server time even if the client's clock is way off, without a round-trip to the server. This can be useful for all kinds of reasons - in my apps, I universally just use server-relative time and don't care about what the client's time is at all.

Related

Best practices for handling timezone on API filters

We save all our datetime data on database in UTC (a timestamp with time zone column in postgresql).
Assuming "America/Sao_Paulo" timezone, if a user saves an event "A" to the database at 2021-08-24 22:00:00 (local time) this will be converted to UTC and saved as 2021-08-25 01:00:00.
So, we are wondering what would be the best way (here "the best way" refers to the developer experience) to consume an API where is possible to filter events by start and end date.
Imagine the following situation: the user is on the website and needs to generate a report with all events that happened on 2021-08-24 (local time America/Sao_Paulo). For this, the user fills start and end date both with 2021-08-24.
If the website forwards this request directly to the API, the server will receive the same date provided by the user and some outcomes can happen:
If the server does not apply any transformation at all, the data returned will not contain the event "A" — by the user perspective, this is wrong.
The server can assume that the date is in UTC and transform start date to 2021-08-24 00:00:00 and end date to 2021-08-24 23:59:59. Then, apply the timezone of the user, generating: 2021-08-24 03:00:00 and 2021-08-25 02:59:59. Filtering the database now would bring the expected event "A".
The API itself could expected a start and end datetime in UTC. This way, the developer can apply the user timezone on client side and then forward to server (2021-08-24T03:00:00Z and 2021-08-25T02:59:59Z).
The API itself could expected a start and end datetime either in UTC or in with the supplied offset (2021-08-24T00:00:00-03:00 and 2021-08-24T23:59:59-03:00). Github does it this way.
What got us thinking was that a lot of APIs accept only a date part on a range filter (like the github API). So, are those APIs filtering the data in the client timezone or they assume the client knows the equivalent UTC date that they should filter by (we could not find any documentation that explains how github deals with an incoming date only filter)?
For us, makes more sense the date filter consider the timezone of the client and not leave to them the burden to know the equivalent UTC datetime of the saved event. But this complicates a bit the filtering logic.
To facilitate the filter logic, we thought that maybe have another column on database to also save the local datetime of the event (or only the local date) would be interesting. Is this a valid approach? Do you know any drawbacks?
*We know that on a database perspective, it is recommended to save datetime in UTC (not always, as showed here) but in our case this seems to only make things more difficult when handling API consumption.
*It is importante to know that, when saving an event, the user cannot provide when it happens, we always assume the event happens in the moment it is being saved.

ServiceNow Rest API Timefield

I am trying to write a Rest API client where ServiceNow Database will be polled every 10 minutes to get the Data.
Below is the Url that I built:
"https://servicenowinstance.com/api/now/table/employee_table?sysparm_limit=200&sysparm_offset=0&sysparm_query=sys_created_onBETWEENjavascript:gs.dateGenerate('2018-02-28','14:23:40')#javascript:gs.dateGenerate('2018-02-28','15:17:04')^ORDERBYsys_created_on".
After implementing Pagination I am starting the Incremental Load. Where I poll every 10 minutes to get the New Data. So in the above URL I get the BETWEEN . So I will get the Data which satisfies the Between Condition.
My Question is that the VM machine I use maintains UTC time. And I am not sure which Timezone does the ServiceNow Tables use to store the Data.
In short my question is what Timezone does ServiceNow use to store its Sys_created Field. Is it same as UTC or is it different?
The database stores dates and times as UTC (KB0534905), but depending on how you pull data via REST, it may return in the timezone of the user account being used for authentication.
Take a look at Table API GET, in particular the sysparm_display_value field.
Data retrieval operation for reference and choice fields. Based on
this value, retrieves the display value and/or the actual value from
the database.
true returns display values for all fields.
false returns actual values from the database. If a value is not specified, this parameter defaults to false.
all returns both actual and display values.
In your case since you're not setting that parameter, it should be in UTC.

Is that secure to compare mongodb dates on client-side?

I have a publication where I send a "record set" of items. Among these items, some have a field with a modification date (Type: Date).
I need to compare the date field with the current date in order to allow/forbid a user interface action. If my date is more than 24hours ago, the action is forbidden.
Initially, I wanted to create a dedicated publication in order to expose only the _id of the items with a Date field inferior to 24h from now.
When reading the excellent answer from #Dan Dascalescu here, I understood that I can't have different minimongo collections if the original Mongodb collection is the same: even if I use different subscriptions everything end up in the same minimongo collection/"record set".
I could just read and compare the Date field on client side and allow/forbid the action but is that secure? Can the client change the date manually? What would be the right way to achieve this?
Any checks that you do to forbid an action have a security implication. There are approaches that you can use here:
use Methods server side along with Meteor.call client side.
use deny rules if it's collection related. That way you get isomorphic behavior for free and instant feedback on client without sacrificing security.

date-time equality in breezejs

I am stuck with date-time equality in breezejs.
What I want to do is query data and return only the latest records that changed since my last sync.
...
query = query.where('ModifiedDateTime', '>=', lastSyncDate);
The ModifiedDateTime is a DateTime type on the dB side, and the lastSyncDate is the datetime of last sync from new Date(Date.now())
This works for the day but not for the time, what I wanted to do is call the getTime() function to get an integer to compare like described here: Beware equality tests but I can't figure out how to do it in the where clause?
Something like
query = query.where(ModifiedDateTime.getTime(), '>=', lastSyncDate.getTime());
But obviously this is not possible, is there another way to do it?
Browser DateTime is unlikely to be the same as server DateTime in the real world. I think you need a way to get the server sync DateTime from the server and use that in your request. A recommendation on how to proceed depends much on how you define lastSyncDateTime and perhaps your willingness to supplement the client/server protocol to include sync DateTime in the server response AND dig that out on the client so you can use it in a query. The tools are there but there is nothing native in Breeze to support this.
You might have a good feature request here for out-of-the-box breeze Web API protocol enhancement. You might suggest that we extend that protocol to include the server DateTime in the response (e.g. in a custom header) AND make that available to the breeze application developer as one of the properties of the query (and save) result.
But getting ahead of myself. First step is to explain what your need for this is and how you use it.

RESTful Webservice time value issues

I have created a RESTful webservice and this webservice uses a mysql database. This was done following a howto using the Netbeans IDE.
All is working fine except for one little thing.
There is one table that is set as a 'time' type (default values 00:00:00) but for some reason when i access the wadl i get to see:
<time>1970-01-01T17:00:00+01:00</time>
I am not a very good Java programmer but i saw in the source of the webservice that Netbeans made this:
public void setDate(Date time) {
this.time = time;
}
How do i change this to just the time value? Are there standard classes that i can use?
[edit]
I am running a glassfish server where i deployed a Netbeans generated war file.
The tutorial to generate a RESTful webservice using Netbeans and mysql
(netbeans.org/kb/docs/websvc/rest.html#entities-and-services)
In the database, the time value is usually just stored as a long value ignoring the date part, which results in that when it's converted to a date, the date value is the unix epoch value (i.e. 0).
So i'm not sure that this is an issue, just convert it back to a Date on the receiving end and you'll have a date with the time properly set.
EDIT:
I suppose you have a transfer object of some sort where you define this "time" parameter? Or are you using hibernate or similar objects as the output to your rest xml generator?
If so, have you tried changing the data type from Date to Time?
Another way would be to change the type to String and in the setDate method use SimpleDateFormat to get a string on the exact form you want.
The types exposed by the RESTful web service are defined in the class YourTableFacadeREST. Try to modify the returned type of the corresponding method in that class.
EDITED
The problem is that the above idea will not work when the exposed object is more complex and your date is only "a part" of that object. Probably the best solution is to handle the conversion with the code reading the object.
If your goal is just to display these data (i.e. for requests of type GET) you may try with a view. I have never done Jersey RESTful web services on a view, but it should work for the GET side. Your View should display the original table with the datetime field converted to date. See here the syntax for creating a view in MySql:
http://dev.mysql.com/doc/refman/5.0/en/create-view.html