we need sending some objects from database of various types within long-polling by rest. Data are sent and each record contains timestamp. When client receive new data from server he should create another poll request with record's timestamp as parameter which helps to specify following data records.
I consider about epoch unix time and store this value in each record in database to filtering and also this value will be sent with each poll requests.
What do you think about this solution? Is that usage fine or should I worry about something? Thanks.
EDIT:
I forget notice these data will be added by clients in different time-zones. This is also another reason why I consider use unix time.
Any format of storing the timestamp is fine, as long as users will be able to unambiguously interpret it. There is no reason for timestamp format in API to be the same as in database. Idea of API is to decouple model from database.
Personally I would choose one format from ISO 8601 Basic and Extended Notations. Example: 2008-09-15T15:53:00. In virtually all programing languages there are methods to handle this format (cast to unix timestamp or to internal date/time classes). For java you would use java.time.LocalDateTime#parse
Unix timestamp has some issues (they may be or not may be issues for you)
unable to represent dates before January 1st, 1970
unable to represent dates after January 19, 2038
not human-readable
does not contain timezone (timestamp itself does not have concept of timezone, but it may be useful to send client timezone along with timestamp. server may always normalise the value to UTC)
Related
I am a newbie to backend and creating a backend server that handles multiple countries' data.
Some countries have a daylight saving issue that requires us to correct timezones twice a year regarding their local data, which would make more work on the server, I think.
I know Unix timestamp is the same everywhere so
Is it easier for us to use the Unix timestamp for all data from anywhere rather than UTC+0 ?
I have a column in the DB that supposed to represent a deadline for ordering.
Let's assume the value is 11am EST.
I've saved in the DB as UTC during Winter so it was saved as 4PM UTC.
Problem is, when DST starts, the conversion back to EST was 12PM...
How can I solve this? on the DB level? if so how would I save a constant time and convert it to the correct value? If I use timestamp with TZ, will it then be read as 5PM UTC?
Thanks.
The scenario you described is caused by converting a future time to UTC and back again, using two different UTC offsets (UTC-5 and UTC-4). In general, one should not store future times (whether recurring or single-instance) in terms of UTC, unless the original reference point is already in terms of UTC. The advice "Always use UTC" applies only for past/present timestamps. It does not apply for future values.
PostgreSQL has several different date/time data types. The "with time zone" types will convert to/from UTC implicitly. The "without time zone" types will not.
In your case, it sounds like you would like to have a time-only value that describes the deadline that applies every day. Thus, you should use a time (aka time without time zone) data type for that field. You should then also store the IANA time zone ID in a separate character field (varchar(50) should be sufficient). For US Eastern Time, you would store 'America/New_York'. That assumes the cutoff is based on the same time zone for all users. If rather you are cutting it off at their time zone, then it could be different per-user, and you will need to determine the user's time zone.
When evaluating whether the deadline has passed, you would take the current UTC timestamp and convert it to the local date and time in the target time zone to have the current date and time in that zone. You'd then take a copy of that and replace the time part with your deadline time. Then compare those two values to see if the deadline has passed.
You should also be thinking about when does the deadline reset for the next day. Is it strictly based on the local date? If they try to order past that deadline, do you disallow it (and if so when do you allow it again), or does it apply to the next date? Only you can answer such questions, as it will vary based on your business needs.
The manipulations I described could be done directly in PostgreSQL, using functions like AT TIME ZONE and others, but generally you are better off doing them in your application layer. Most programming platforms have functions for manipulating dates and times, and for working with time zones.
We have a simple application in which we have all user in same timezone & therefore we are not interested to store timezone information in mongo date object.
Reason for such extreme step is we have multiple micro service using common database managed by different developers. Each of them requires to explicitly set timezone related stuff in query & forgetting same results in invalid dataset.
Since currently MongoDB folks Mongo Data Types
doesn't support storing dates without timezone.
Just eager to know that is their any alternative approach to represent date without timezone in mongo by which we can still able to take advantage of mongo database queries date based syntax like date ranges, date etc.
At the same time it would be convenient for DBA's to read and manage records.
Look at this answer: https://stackoverflow.com/a/6776273/6105830
You can use two types of long representation (milliseconds or format yyyyMMddHHmmss). These are the only ways to not store timezone and still be able to make range queries.
Unfortunately you lost some aggregation properties. But you can do something like keeping two representations and use them at opportune times.
UPDATE:
Do not store date as I said before. You will lost many and many features of MongoDB and also will be hard to perform major operators on date fields.
Newer versions of MongoDB has operators to deal with timezone, and it should be enough to work with ISOTime formats. My application was using my own suggestion to store date. Now I have to let my users select their TimeZone (company has grown and we need to expand to other countries). We are struggling to change all models to use timestamp instead of a normalized date format.
For further more explore the link: https://docs.mongodb.com/manual/reference/method/Date/
and you can also use MongoDB official community channel for questioning
Here is the link: https://developer.mongodb.com/community/forums/
You could consider storing all the dates in UTC and presenting to the users in UTC, so you don't have the problem of silent conversion by either client JavaScript, server or MongoDB and therefore confusion.
You can do it like this: new Date(Date.UTC(2000, 01, 28))
Here's MDN link on the subject.
This question already has answers here:
Recommended date format for REST GET API
(6 answers)
Closed 6 years ago.
Is there a standard or preferred way to format or enforce a format when I GET/PUT/POST dates/times using a RESTful service?
Is it best to be strict in validation? (e.g., YYYY-MM-DD HH:MM:SS GMT, or a Unix timestamp) Or as a general rule, should date/time formatting be more loosely defined, perhaps with multiple potential formats?
My main concern with Unix timestamps is that they are not very human readable, and I'll have to spend resources converting dates to and from timestamps.
ISO-8601 is generally a portable, readable timestamp format designed with data interchange in mind. It's supported by most languages either natively or through a 3rd-party library.
Unix timestamps, as you say, are not particularly readable and certainly don't make it easy for people to interact with the service - either for exploratory or debug purposes.
One thing to be aware of also is that database support for unix timestamps is not ideal. Natively, MySQL (for example) sees unix timestamps as (simply) unsigned integers. To make use of MySQL date/time manipulation you need to convert that to a MySQL timestamp type. Oracle also, iirc, doesn't really understand a unix timestamp as a date-time type.
For that kind of reason, I'd generally use an ISO-8601 format to transfer timestamp information rather than a unix timestamp.
I am just starting to learn about MongoDB and hoping to slowly migrate from MySQL.
In MySQL, there are two different data types - DATE ('0000-00-00') and DATETIME ('0000-00-00 00:00:00'). In my MySQL, I use the DATE type, but I am not sure how to transfer them into MongoDB. In MongoDB, there is a Date object, which is comparable to DATETIME. It seems it would be most appropriate to use Date objects, but that would be wasting space, since hours, min, sec are not utilized. On the other hand, storing dates as strings seems wrong.
Is there a golden standard on storing dates ('0000-00-00') in MongoDB?
I'm actually in the process of converting a MongoDB database where dates are stored as proper Date() types to instead store them as strings in the form yyyy-mm-dd. Why, considering that every other answerer says that this is a horrible idea? Simply put, because of the neverending pain I've been suffering trying to work with dates in JavaScript, which has no (real) concept of timezones. I had been storing UTC dates in MongoDB, i.e. a Date() object with my desired date and the time set as midnight UTC, but it's unexpectedly complicated and error-prone to get a user-submitted date correctly converted to that from whatever timezone they happen to be in. I've been struggling to get my JavaScript "whatever local timezone to UTC" code to work (and yes, I'm aware of Sugar.js and Moment.js) and I've decided that simple strings like the good old MySQL standard yyyy-mm-dd is the way to go, and I'll parse into Date() objects as needed at runtime on the client side.
Incidentally, I'm also trying to sync this MongoDB database with a FileMaker database, which also has no concept of timezones. For me the simplicity of simply not storing time data, especially when it's meaningless like UTC midnight, helps ensure less-buggy code even if I have to parse to and from the string dates now and then.
BSON (the storage data format used by mongo natively) has a dedicated date type UTC datetime which is a 64 bit (so, 8 byte) signed integer denoting milliseconds since Unix time epoch. There are very few valid reasons why you would use any other type for storing dates and timestamps.
If you're desperate to save a few bytes per date (again, with mongo's padding and minimum block size and everything this is only worth the trouble in very rare cases) you can store dates as a 3 byte binary blob by storing it as an unsigned integer in YYYYMMDD format, or a 2 byte binary blob denoting "days since January 1st of year X" where X must be chosen appropriately since that only supports a date range spanning 179 years.
EDIT: As the discussion below demonstrates this is only a viable approach in very rare circumstances. Basically; use mongo's native date type ;)
If you really care about saving 4 bytes per field (in case you have many DATE fields per document) you can store dates as int32 fields in form 20110720 (note MySQL DATE occupies 3 bytes, so the storage will be greater in any case). Otherwise I'd better stick to standard datetime type.