I'm using PyMongo to fetch data from MongoDB. All documents in the collection look like the structure below:
{
"_id" : ObjectId("50755d055a953d6e7b1699b6"),
"actor":
{
"languages": ["nl"]
},
"language":
{
"value": "nl"
}
}
I'm trying to fetch all the conversations where the property language.value is inside the property actor.languages.
At the moment I know how to look for all conversations with a constant value inside actor.languages (eg. all conversations with en inside actor.languages).
But I'm stuck on how to do the same comparison with a variable value (language.value) inside the current document.
Any help is welcome, thanks in advance!
db.testcoll.find({$where:"this.actor.languages.indexOf(this.language.value) >= 0"})
You could use a $where provided your query set is small, but any real size and you could start seeing problems, especially since this query seems like one that needs to be run in realtime on a page and the JS engine is single threaded among other problems.
I would actually consider a better way in this case is through the client side, it is quite straight forward, pull out records based on one of the values, iterate and test their conditional double value (i.e. pull out based on language.value being nl and test actor.languages value for that previous value).
I would imagine you might be able to do this with the aggregation framework however, at the min you cannot use computed fields within $match. I would imagine it would look like this:
{$project:
{languages_value: "$language.value", languages: "$actor.languages"}
}, {$match: {"$languages": {$in:"$languages_values"}}
If you could. But there might be a way.
Related
I'm a complete beginer in MongoDB, so excuse me if this is trivial.
I have a collection of labels (strings) and a function that maps a char to its width in a given font.
I would like to retrieve the labels ordered by their width (the result of the function).
Can someone tell me if it's possible, and if yes, how to do it?
Thanks
In general it is possible to derive some value from existing fields using a combination of the available server-side functions and then sort by that value. Here is an example:
Given a document
{
"input" : 1
}
you could do this:
db.collection.aggregate({
$addFields: {
"result": {
$sum: [ "$input", 42 ] // function that calculates "result = input + 42"
}
}
}, {
$sort: {
"result": 1 // sort by calculated field in ascending order
}
})
You could try to write your mapping function using MongoDB functions. I would not advise doing that, however, since you'd be modelling parts of your business logic inside a database call which shouldn't be done really.
This is not possible.
On the other hand document oriented design is not application agnostic. That means you structure your data according to the way you want to access your data. In this case you would index the text width as a field ad sort by that field. Any text change would mean, that the text width has to be recalculated.
This is not a bad thing, because you would optimize the (hopefully many) read operations and have a bit of code and a performance loss during (the hopefully few) write operations.
My requirement is to get json pair from mqtt subscriber at different timings under single_id in cloudant, but I'm facing error while trying to insert new json pair in existing _id, it simply replace old one. I need at least 10 json pair under one _id. Injecting at different timings.
First, you should make sure about your architectural decision to update a particular document multiple times. In general, this is discouraged, though it depends on your application. Instead, you could consider a way to insert each new piece of information as a separate document and then use a map-reduce view to reflect the state of your application.
For example (I'm going to assume that you have multiple "devices", each with some kind of unique identifier, that need to add data to a cloudant DB)
PUT
{
"info_a":"data a",
"device_id":123
}
{
"info_b":"data b",
"device_id":123
}
{
"info_a":"message a"
"device_id":1234
}
Then you'll need a map function like
_design/device/_view/state
{
function (doc) {
emit(doc.device_id, 1);
}
Then you can GET the results of that view to see all of the "info_X" data that is associated with the particular device.
GET account.cloudant.com/databasename/_design/device/_view/state
{"total_rows":3,"offset":0,"rows":[
{"id":"28324b34907981ba972937f53113ac3f","key":123,"value":1},
{"id":"d50553d206d722b960fb176f11841974","key":123,"value":1},
{"id":"eaa710a5fa1ff4ba6156c997ddf6099b","key":1234,"value":1}
]}
Then you can use the query parameters to control the output, for example
GET account.cloudant.com/databasename/_design/device/_view/state?key=123&include_docs=true
{"total_rows":3,"offset":0,"rows":[
{"id":"28324b34907981ba972937f53113ac3f","key":123,"value":1,"doc":
{"_id":"28324b34907981ba972937f53113ac3f",
"_rev":"1-bac5dd92a502cb984ea4db65eb41feec",
"info_b":"data b",
"device_id":123}
},
{"id":"d50553d206d722b960fb176f11841974","key":123,"value":1,"doc":
{"_id":"d50553d206d722b960fb176f11841974",
"_rev":"1-a2a6fea8704dfc0a0d26c3a7500ccc10",
"info_a":"data a",
"device_id":123}}
]}
And now you have the complete state for device_id:123.
Timing
Another issue is the rate at which you're updating your documents.
Bottom line recommendation is that if you are only updating the document once per ~minute or less frequently, then it could be reasonable for your application to update a single document. That is, you'd add new key-value pairs to the same document with the same _id value. In order to do that, however, you'll need to GET the full doc, add the new key-value pair, and then PUT that document back to the database. You must make sure that your are providing the most recent _rev of that document and you should also check for conflicts that could occur if the document is being updated by multiple devices.
If you are acquiring new data for a particular device at a high rate, you'll likely run into conflicts very frequently -- because cloudant is a distributed document store. In this case, you should follow something like the example I gave above.
Example flow for the second approach outlined by #gadamcox for use cases where document updates are not required very frequently:
[...] you'd add new key-value pairs to the same document with the same _id value. In order to do that, however, you'll need to GET the full doc, add the new key-value pair, and then PUT that document back to the database.
Your application first fetches the existing document by id: (https://docs.cloudant.com/document.html#read)
GET /$DATABASE/100
{
"_id": "100",
"_rev": "1-2902191555...",
"No": ["1"]
}
Then your application updates the document in memory
{
"_id": "100",
"_rev": "1-2902191555...",
"No": ["1","2"]
}
and saves it in the database by specifying the _id and _rev (https://docs.cloudant.com/document.html#update)
PUT /$DATABASE/100
{
"_id": "100",
"_rev": "1-2902191555...",
"No":["1","2"]
}
I have this collection of products and i want to display a top 10 products based on a custom sort function
[{ _id: 1, title, tags:['a'], createdAt:ISODate("2016-01-28T00:00:00Z") } ,
{ _id: 2, title, tags:['d','a','e'], createdAt:ISODate("2016-01-24T00:00:00Z") }]
What i want to do is to sort it based on a "magic score" that can be calculated. For example, based on this formula: tag_count*5 - number_of_days_since_it_was_created.
If the first one is 1 day old, this makes the score:
[{_id:1 , score: 4}, {_id:2, score: 10}]
I have a few ideas on how i can achieve this, but i'm not sure how good they are, especially since i'm new to both mongo and meteor:
start an observer (Meteor.observe) and every time a document is
modified (or a new one created), recalculate the score and update it
on the collection itself. If i do this, i could just use $orderBy
where i need it.
after some reading i discovered that mongo aggregate or map_reduce
could help me achieve the same result, but as far as i found out,
meteor doesn't support it directly
sort the collection on the client side as an array, but using this
method i'm not sure how it will behave with pagination (considering that i subscribe to a limited number of documents)
Thank you for any information you can share with me!
Literal function sorting is just being implemented in meteor, so you should be able to do something like
Products.find({}, {sort: scoreComparator});
in an upcoming release.
You can use the transform property when creating collection. In this transform, store the magic operation as a function.
score=function(){
// return some score
};
transformer=function(product){
product.score=score;
// one could also use prototypal inheritance
};
Products=new Meteor.Collection('products',{transform:transformer});
Unfortunately, you cannot yet use the sort operator on virtual fields, because minimongo does not support it.
So the ultimate fall-back as you mentioned while nor the virtual field sorting nor the literate function sorting are supported in minimongo is client side sorting :
// Later, within some template
scoreComparator=function(prd1,prd2){
return prd1.score()-prd2.score();
}
Template.myTemplate.helpers({
products:function(){
return Products.find().fetch().sort(scoreComparator);
}
});
i'm not sure how it will behave with pagination (considering that i subscribe to a limited number of documents)
EDIT : the score will be computed among the subscribed documents, indeed.
I have some mongodb object let's call it place which contains geo information, look at the example:
{
"_id": "234235425e3g33424".
"geo": {
"lon": 12.23456,
"lat": 34.23322
}
"some_field": "value"
}
With every place, a list of features is associated with:
{
"_id": "2334sgfgsr435d",
"place_id": "234235425e3g33424",
"feature_field" : "some_value"
}
As you see features are linked to places thanks to place_id field. Now I would like to find: list of features connected with nearest places. But I would like also add search contition on place.some_field and feature.feature_field. And what is important I would like to limit results.
Now I am using such approach:
I query on places with condition on geo and some_filed
I query on features with condition on feature_field and place_id (limit only to ones found in 1.)
I limit results in my application code
My question is: is there better approach to such task? Now I cannot use mongo limit() function, as when I do it on places I can end with too few results as I need to make second query. I cannot limit() on second query as results will come up with random order, and I would like to sort it by distance.
I know I can put data into one document, but I presume that list of features will be long and I can exceed BSON size limit.
Running out of 16mb for just the features seems unlikely... but it's possible. I don't think you realize how much 16mb is, so do the maths before assuming anything!
In any case, with MongoDB you can not do a query with fields from two collections. A query always deals with one specific collection only. I have done a very similar thing than what you have here though, which i've described in an article: http://derickrethans.nl/indexing-free-tags.html — have a look at that for some more inspiration.
in SQL world I could do something to the effect of:
SELECT name FROM table WHERE UPPER(name) = UPPER('Smith');
and this would match a search for "Smith", "SMITH", "SmiTH", etc... because it forces the query and the value to be the same case.
However, MongoDB doesn't seem to have this capability without using a RegEx, which won't use indexes and would be slow for a large amount of data.
Is there a way to convert a stored value to a particular case before doing a search against it in MongoDB?
I've come across the $toUpper aggregate, but I can't figure out how that would be used in this particular case.
If there's not way to convert stored values before searching, is it possible to have MongoDB convert a value when it's created in Mongo? So when I add a document to the collection it would force the "name" attribute to a particular case? Something like a callback in the Rails world.
It looks like there's the ability to create stored JS for MongoDB as well, similar to a Stored Procedure. Would that be a feasible solution as well?
Mostly looking for a push in the right direction; I can figure out the particular code once I know what I'm looking for, but so far I'm not even sure if my desired functionality is doable.
You have to normalize your data before storing them. There is no support for performing normalization as part of a query at runtime.
The simplest thing to do is probably to save both a case-normalized (i.e. all-uppercase) and display version of the field you want to search by. Suppose you are storing users and want to do a case-insensitive search on last name. You might store:
{
_id: ObjectId(...),
first_name: "Dan",
last_name: "Crosta",
last_name_upper: "CROSTA"
}
You can then create an index on last_name_upper, and query like:
> db.users.find({last_name_upper: "CROSTA"})