I'm not sure this is possible, but i'd like to create a single view or at least a single query that looks in different collections based on what's being queried.
for example, if the first character is an "A" look in the "Aresults" collection, if it's a "B" look in the "Bresults" collection, etc.
I could potentially create a "A-Z" collection with just those letters, and do a $lookup from there based on a condition, but i'm not sure how to do that either.
I am aware that i could create a view with a $unionWith having all the "*results" collections, but that seems very inefficient.
Any other ideas? Is there perhaps some type of dynamic query structure within mongodb like in MySQL (couldn't find any)?
Thanks
Something like this?
const prefix = db.meta_data.findOne({field: condition}).prefix ;
db.createView('view_name', prefix + 'results', [<your aggregation pipeline>]);
or this?
const pipeline = [];
db.meta_data.find({ field: condition }).forEach(x => {
pipeline.push({ $unionWith: { coll: prefix + 'results' } });
});
db.collection.aggregate([pipeline]);
Related
Is there a way to push all the documents of a given collection in a array?
I did this but is there any quicker way?
var ops = [];
db.getCollection('stock').find({}).forEach(function (stock) {
ops.push(stock);
})
PS: I use Mongo 3.4
You can just use the toArray function on the cursor that's returned from find, like this:
var ops = db.getCollection('stock').find({}).toArray();
Note: As with your original solution, this might suffer with performance if the stock collection contains millions of documents.
As an aside, you can use db.stock directly to shorten the query a little bit:
var ops = db.stock.find({}).toArray();
Try using lean query option. in your case:
db.getCollection('stock').find({}).lean()
You could as well use $facet which will allow you to create the array on the server side - provided the resulting document array is no bigger than 16MB in which case you'll get an exception:
db.stock.aggregate({
$facet: {
ops: [ { $match: {} } ]
}
})
In order to reduce the amount of data returned you could limit the number of returned fields in the above pipeline (instead of an empty $match stage - which is a hack anyway - you would then use $project).
I need to search for the existence of a property that is within another object.
the collection contains documents that look like:
"properties": {
"source": {
"a/name": 12837,
"a/different/name": 76129
}
}
As you can see below, part of the query string is from a variable.
With some help from JohnnyHK (see mongo query - does property exist? for more info), I've got a query that works by doing the following:
var name = 'a/name';
var query = {};
query['properties.source.' + name] = {$exists: true};
collection.find(query).toArray(function...
Now I need to see if I can index the collection to improve the performance of this query.
I don't have a clue how to do this or if it is even possible to index for this.
Suggestions?
2 things happening in here.
First probably you are looking for sparse indexes.
http://docs.mongodb.org/manual/core/index-sparse/
In your case it could be a sparse index on "properties.source.a/name" field. Making indexes on field will dramatically improve your query lookup time.
db.yourCollectionName.createIndex( { "properties.source.a/name": 1 }, { sparse: true } )
Second thing. Always when you want to know whether your query is fast/slow, use mongo console, run your query and on its result call explain method.
db.yourCollectionName.find(query).explain();
Thanks to it you will know whether your query uses indexes or not, how many documents it had to check in order to complete query and some others useful information.
I have a mongoDB document that has the following structure:
{
user:user_name,
streams:[
{user:user_a, name:name_a},
{user:user_b, name:name_b},
{user:user_c, name:name_c}
]
}
I want to use $pullAll to remove from the streams array, passing it an array of streams (the size of the array varies from 1 to N):
var streamsA = [{user:"user_a", name:"name_a"},{user:"user_b", name:"name_b"}]
var streamsB = [{name:"name_a", user:"user_a"},{name:"name_b", user:"user_b"}]
I use the following mongoDB command to perform the update operation:
db.streams.update({name:"user_name", {"$pullAll:{streams:streamsA}})
db.streams.update({name:"user_name", {"$pullAll:{streams:streamsB}})
Removing streamsA succeeds, whereas removing streamsB fails. After digging through the mongoDB manuals, I saw that the order of fields in streamsA and streamsB records has to match the order of fields in the database. For streamsB the order does not match, that's why it was not removed.
I can reorder the streams to the database document order prior to performing an update operation, but is there an easier and cleaner way to do this? Is there some flag that can be set to update and/or pullAll to ignore the order?
Thank You,
Gary
The $pullAll operator is really a "special case" that was mostly intended for single "scalar" array elements and not for sub-documents in the way you are using it.
Instead use $pull which will inspect each element and use an $or condition for the document lists:
db.streams.update(
{ "user": "user_name" },
{ "$pull": { "streams": { "$or": streamsB } }}
)
That way it does not matter which order the fields are in or indeed look for an "exact match" as the current $pullAll operation is actually doing.
conceptually what I am trying to figure out is if there is an alternative to accessing nested docs with mongo other than dot notation.
What I am trying to accomplish:
I have a user collection, and each user has a nested songVotes collection where the keys for this nested songVotes collection are the songIds and the value is their vote form the user -1,0, or 1.
I have a "room collection" where many users go and their collective votes for each song influence the room. A single room also has a nested songVotes collection with keys as songIds, however the value is the total number of accumulated votes for all the users in the room added up. For purposes of Meteor.js, its more efficient as users enter the room to add their votes to this nested cumulative vote collection.
Again because reactive joins in Meteor.js arent supported in any kind of efficient way, it also doesnt make sense to break out these nested collections to solve my problem.
So what I am having trouble with is this update operation when a user first enters the room where I take a single users nested songVotes collection and use the mongo $inc operator to apply it to the nested cumulative songVotes collection of the entire room.
The problem is that if you want to use the $inc operator with nested fields, you must use dot notation to access them. So what I am asking on a broad sense is if there is a nice way to apply updates like this to a nested object. Or perhaps specify a global dot notation prefix for $inc something like:
var userVotes = db.collection.users.findOne('user_id').songVotes
// userVotes --> { 'song1': 1, 'song2': -1 ... }
db.rooms.update({ _id: 'blah' }, { $set: { roomSongVotes: { $inc: userVotes } } })
You do need to use dot notation, but you can still do that in your case by programmatically building up the update object:
var userVotes = {'song1': 1, 'song2': -1};
var update = {$inc: {}};
for (var songId in userVotes) {
update.$inc['roomSongVotes.' + songId] = userVotes[songId];
}
db.rooms.update({ _id: 'blah' }, update);
This way, update gets built up as:
{ '$inc': { 'roomSongVotes.song1': 1, 'roomSongVotes.song2': -1 } }
I would like to speed up an query on my mongoDB which uses $where to compare two fields in the document, which seems to be really slow.
My query look like this:
db.mycollection.find({ $where : "this.lastCheckDate < this.modificationDate})
What I would like to do is add a field to my document, i.e. isCheckDateLowerThenModDate, on which I could execute a probably much faster query:
db.mycollection.find({"isCheckDateLowerThenModDate":true})
I quite new to mongoDB an have no idea how to do this. I would appreciate if someone could give me some hints or examples on
How to initialize such a field on an existing collection
How to maintain this field. Which means how to update this field when lastCheckDate or modificationDate changes.
Thanks in advance for your help!
You are thinking in a right way!
1.How to initialize such a field on an existing collection.
Most simple way is to load each document (from your language), calculate this field, update and save.
Or you could perform an update via mongo shell:
db.mycollection.find().forEach(function(doc) {
if(doc.lastCheckDate < doc.modificationDate)
{
doc.isCheckDateLowerThenModDate = true;
}
else
{
doc.isCheckDateLowerThenModDate = false;
}
db.mycollection.save(doc);
});
2.How to maintain this field. Which means how to update this field when
lastCheckDate or modificationDate changes.
You have to do it yourself from your client code. Make some wrapper for update, save operations and recalculate this value each time there. To be absolutely sure that this update works -- write unit tests.
The $where clause is slow because it is evaluating each document using the JavaScript interpreter.
There are a few alternatives:
1) Assuming your use case is "look for records that need updating", take advantage of a sparse index:
add a boolean field like needsChecking and $set this whenever the modificationDate is updated
in your "check" procedure, find the documents that have this field set (should be fast due to the sparse index)
db.mycollection.find({'needsChecking':true});
after you've done whatever check is needed, $unset the needsChecking field.
2) A new (and faster) feature in MongoDB 2.2 is the Aggregation Framework.
Here is an example of adding a "isUpdated" field based on the date comparison, and then filtering the matching documents:
db.mycollection.aggregate(
{ $project: {
_id: 1,
name: 1,
type: 1,
modificationDate: 1,
lastCheckDate: 1,
isUpdated: { $gt:["$modificationDate","$lastCheckDate"] }
}},
{ $match : {
isUpdated : true,
}}
)
Some current caveats of using the Aggregation Framework are:
you have to specify fields to include aside from _id
the result is limited to the current maximum BSON document size (16Mb in MongoDB 2.2)