I have a sails api with the created date formatted like
"createdAt": "2018-11-01T11:49:53.700Z",
i can get the contains filtering on a field working e.g
api2/items?status=IN_PROGRESS
but can't get the date range working, have tried the following
api2/items?createdAt={'>=":2018-11-01T11:49:53.700Z, '<=":2018-11-01T11:49:53.700Z}
/api2/items?where={createdAt: { '>=': 2018-11-01T11:49:53.700Z, '<=': 2018-11-01T11:49:53.700Z }}
any ideas?
I don't believe waterline supports this type of query on a datetime field. I would urge you to instead store these as a number (the unix time), which you will more easily be able to do such queries. When you want to format these items for display, you can use moment.js to help out.
Related
Context
I have an index with a field called "date" which contains dates. I need an elasticsearch query that returns records where date is greater than a specific date value.
Issue
Running the following query with range filter returns does not work. Records with earlier dates are returned in the result set.
{
"size": 1000,
"query": {
"filtered": {
"filter": {
"range": {
"date": {
"gt": "2014-02-23T00:00:00"
}
}
}
}
}
}
Questions
What is the correct query to pull data where date is greater than a
specific value?
If my query is syntactically correct, is there
something else I can go check (e.g. datatype of field is actually
date)?
How should I go about root causing this?
etc.
Solution
In lieu of implementing mapping, I came up with a partial solution. I used Chrome to analyze some of the Kibana traffic. I noticed Kibana is passing date filters as int values. So, I converted the dates to ints using Unix timestamp conversion and things are working now.
(Reference http://www.epochconverter.com/)
What about mapping?
I looked at the mappings earlier. On my index they don't exist. I seem to recall reading that mappings will be inferred for known types that have strong consistency.
My date data is consistent:
- no nulls
- dates are getting flipped from SQL, to C#, to Elastic
I guess I could implement a mapping, but I'm going with the Epoch conversion for now until I have a true need to map this for some other compelling reason.
Your query is syntactically correct.
Use get mapping API to see the document mapping:
curl -XGET 'http://localhost:9200/twitter/_mapping/tweet'
It's hard to say where goes wrong. Probably the mapping of date field is not date type actually.
I am trying to post a query to Sails.js via HTTP POST in order to find records created after a certain point in time.
Using the following syntax works perfectly and gives me back one record with the correct time:
localhost:1337/object?where={"createdAt":"2014-08-19T14:36:16.047Z"}
When trying to add an operator the query does not work any longer giving me no records in the object collection :
localhost:1337/object?where={"createdAt":{">":"2014-08-19T14:36:16.047Z"}}
Due to the fact that the date conversion from ISODate to dateTime works for the exact query I suppose it should also work for the ">" query.
Highly appreciate some help on this one.
I'm inserting a Mongo doc with the following time-stamp:
val format = new java.text.SimpleDateFormat("yyyyMMddHHmmss")
format.format(new Date()).toLong
Here's what the section looks like from Mongo's shell:
"{Timestamp" : NumberLong("20130919161948")}"
Based on a few tests, it appears to me that I can simply compare 2 documents by Timestamp by simply checking > or < for the yyyyMMddHHmmss format.
Please let me know if this time-stamp is OK for Mongo. Will I be able to query with it?
Mongo will not understand this as a timestamp, but as a number. As you set your date with a format going from year to seconds, you will be able to query mongo using > or < to know if it is before or after.
However if you want to mongo to treat the data as a date, you will need to use the appropriate bson date format. By having mongo treat it as a date, you will have all Mongo date operations available, like extracting year, day of week, etc.. read more
If you are using casbah, and Joda, you can enable serialization and deserialization by an explicit call:
import com.mongodb.casbah.conversions.scala._
RegisterJodaTimeConversionHelpers()
Read more here.
#Kevin, I think you are right. java.util.Date is supported in BSON object.
Using NumberLong to represent timestamp allows you to do range queries, but with BSON date type, date operation in aggregation framework becomes possible, which is more powerful.
I have activities that I should add to mongodb, activities have, dates and times, I should sort them, What I want, is not to have another field for timestamp and index that field also, I don't want the time when the document is created, I want to embed that activitie's timestamp to mongoid, I searched and tried this one but did not work,
"_id" => new MongoId($stamp)
Is there a chance, that I can embed future timestamp to mongoid ? thank you :)
It is possible, but not directly through the driver. I have written an example at Create MongoDB ObjectID from date in the past using PHP driver that shows you how to do this.
I am using the Rally bulk query API to pull data from multiple tables. My issue happens when I try to use a placeholder for the Iteration's StartDate and pass it along to a following query same bulk request. i.e.
"iteration": "/Iteration?fetch=ObjectID,StartDate&query=(Name = \"Sprint 1\")",
"started": "${iteration.StartDate}",
"other_queries": "...?query=(CreatedDate > $(iteration.StartDate))"
The bulk service seems to convert this field to a formatted string. Is there a way to prevent this from happening? I am attempting to use the placeholder to limit other queries by date without making several requests.
It looks like the iteration object comes back with the date correctly, but when it is used as a placeholder it is automatically converted to a string.
"started": ["Wed Jan 16 22:00:00 MST 2013"],
"iteration": {
"Results": [
....
"StartDate": "2013-01-17T05:00:00.000Z",
]}
Unfortunately no, as this functionality is currently implemented, this is expected behavior. The placeholder is converted to a formatted String server-side, so it will be necessary to formulate a similar followup request if the same data is needed in another query.