As far as I can tell there seems to be no way to perform a Find lookup in MongoDB that performs a Date lte without using either the Date or ISODate JavaScript constructor methods.
I am using a technology stack that uses LUA scripts that are invoked by a secondary Perl system. The issue is that I can only pass pure JSON between these systems and subsequently on to the MongoDB instance itself.
This presents a number off issues, the major one being that any JavaScript functions supported by Mongo are not an option. (Assuming there's no way to get Mongo to interpret functions stored within string values?)
Currently I can perform a lte lookup using the Aggregate method with something like:
{
"$match": {
"$expr": {
"$and": {
"$lte": {
"start_date": {
"$dateFromString": {
"dateString": "2020-06-07T04:44:39.993Z"
}
}
}
}
}
}
}
While this works, this is just for a single date comparison, it would be nice to be able to reduce the amount of code since I expect to be creating some fairly complex queries.
Ideally I would be able to use Find and do the following:
{
"start_date": {
"$lte": "2020-06-07T04:44:39.993Z"
}
}
As far as I know Find doesn't seem to support lte date lookups using a string based Date value.
Things I've tried:
Using various ISO and JavaScript formatted date strings.
Using Epoch seconds and millisecond string values.
Attempting to escape the JavaScript function calls within the string value.
Is what I am attempting to do at all possible? Is using pure JSON an option? Or do I just have to concede defeat and use the Aggregate function to perform the String to Date conversion and just deal with the additional boiler plate code or is there a fundamental thing I'm missing with Mongo that would let me do this?
Related
I'm using MongoDB Stitch to create a data enabled API, but when I make a GET request, the data is returned where numbers are displayed as:
"firstHit": {
"$numberInt": "3"
Where I would like them to be return just as:
"firstHit": 3
I have a lot of objects within objects, and I am inserting the data through the mongo shell, I'm not sure of that is of any importance.
Anyone have any experience with this? Thank you!
By default, the result format returned by MongoDB Stitch webhooks is in MongoDB Extended JSON format, or EJSON for short. This is useful to define data types that would otherwise be lost in normal JSON. There are some object types that have no equivalent in JSON, for example ObjectId() and Date().
If you would like to return as a normal JSON, you could set the response object as an example below:
exports = function(payload, response) {
result = {"firsthit": 10};
response.setStatusCode(200);
response.setHeader("Content-Type", "application/json");
response.setBody(JSON.stringify(result));
}
You may also find EJSON library and Stitch Utility Packages as useful additional information.
I'm a complete beginer in MongoDB, so excuse me if this is trivial.
I have a collection of labels (strings) and a function that maps a char to its width in a given font.
I would like to retrieve the labels ordered by their width (the result of the function).
Can someone tell me if it's possible, and if yes, how to do it?
Thanks
In general it is possible to derive some value from existing fields using a combination of the available server-side functions and then sort by that value. Here is an example:
Given a document
{
"input" : 1
}
you could do this:
db.collection.aggregate({
$addFields: {
"result": {
$sum: [ "$input", 42 ] // function that calculates "result = input + 42"
}
}
}, {
$sort: {
"result": 1 // sort by calculated field in ascending order
}
})
You could try to write your mapping function using MongoDB functions. I would not advise doing that, however, since you'd be modelling parts of your business logic inside a database call which shouldn't be done really.
This is not possible.
On the other hand document oriented design is not application agnostic. That means you structure your data according to the way you want to access your data. In this case you would index the text width as a field ad sort by that field. Any text change would mean, that the text width has to be recalculated.
This is not a bad thing, because you would optimize the (hopefully many) read operations and have a bit of code and a performance loss during (the hopefully few) write operations.
Using adminMongo as web-ui for MongoDB, I want to filter and order the documents. Filtering works simple:
{
"status": 4
}
But additionally, all documents should be sorted by date field, which is a unix timestamp. In MySQL I would simply add ORDER BY date DESC but as a newbie, It's not clear for me how to do this in adminMongo.
I took a look in the documentation. They execute js code with methods. To get in this, I used the shell and could get my expected data result using the following code:
db.getCollection('my-collection').find({"status": 4}).sort({date: -1})
However this works on the shell, it creates an error in adminMongo. Seems that adminMongo expects a json document. So we've 2 syntax variants? It's confusing me, since on SQL we've SQL and don't care if we write SQL in any programming language, admin ui or directly on the shell.
I think I've to convert the find method to json, but don't know why. According to the docs, we can use $sort as $sort: { 'date': -1}. Calling aggreagate() method on the collection, this works. On adminMongo, I got an error when trying to send something like
{
{ "$sort": { 'date': -1} }
}
What is the right syntax which I need to use on adminMongo? Where is documented, how I can convert methods like aggregate which are used for sorting, to this syntax?
Context
I have an index with a field called "date" which contains dates. I need an elasticsearch query that returns records where date is greater than a specific date value.
Issue
Running the following query with range filter returns does not work. Records with earlier dates are returned in the result set.
{
"size": 1000,
"query": {
"filtered": {
"filter": {
"range": {
"date": {
"gt": "2014-02-23T00:00:00"
}
}
}
}
}
}
Questions
What is the correct query to pull data where date is greater than a
specific value?
If my query is syntactically correct, is there
something else I can go check (e.g. datatype of field is actually
date)?
How should I go about root causing this?
etc.
Solution
In lieu of implementing mapping, I came up with a partial solution. I used Chrome to analyze some of the Kibana traffic. I noticed Kibana is passing date filters as int values. So, I converted the dates to ints using Unix timestamp conversion and things are working now.
(Reference http://www.epochconverter.com/)
What about mapping?
I looked at the mappings earlier. On my index they don't exist. I seem to recall reading that mappings will be inferred for known types that have strong consistency.
My date data is consistent:
- no nulls
- dates are getting flipped from SQL, to C#, to Elastic
I guess I could implement a mapping, but I'm going with the Epoch conversion for now until I have a true need to map this for some other compelling reason.
Your query is syntactically correct.
Use get mapping API to see the document mapping:
curl -XGET 'http://localhost:9200/twitter/_mapping/tweet'
It's hard to say where goes wrong. Probably the mapping of date field is not date type actually.
I'm using PyMongo to fetch data from MongoDB. All documents in the collection look like the structure below:
{
"_id" : ObjectId("50755d055a953d6e7b1699b6"),
"actor":
{
"languages": ["nl"]
},
"language":
{
"value": "nl"
}
}
I'm trying to fetch all the conversations where the property language.value is inside the property actor.languages.
At the moment I know how to look for all conversations with a constant value inside actor.languages (eg. all conversations with en inside actor.languages).
But I'm stuck on how to do the same comparison with a variable value (language.value) inside the current document.
Any help is welcome, thanks in advance!
db.testcoll.find({$where:"this.actor.languages.indexOf(this.language.value) >= 0"})
You could use a $where provided your query set is small, but any real size and you could start seeing problems, especially since this query seems like one that needs to be run in realtime on a page and the JS engine is single threaded among other problems.
I would actually consider a better way in this case is through the client side, it is quite straight forward, pull out records based on one of the values, iterate and test their conditional double value (i.e. pull out based on language.value being nl and test actor.languages value for that previous value).
I would imagine you might be able to do this with the aggregation framework however, at the min you cannot use computed fields within $match. I would imagine it would look like this:
{$project:
{languages_value: "$language.value", languages: "$actor.languages"}
}, {$match: {"$languages": {$in:"$languages_values"}}
If you could. But there might be a way.