Resolving a ref inside of a MongoDB aggregate - mongodb

I have a Product model object that has the following field in its schema:
category : { type: ObjectId, turnOn: false, ref: "category" }
It references a category model that has a title field in it:
var categorySchema = Schema({
title : { type: String }
});
I'm using the product.category property (which is of type ObjectId as shown above) in a MongoDB aggregate but really want the category.title property from the category model rather than _id in the final resultset.
The following code gets the job done, but you'll see that I'm having to do some looping at the end to "resolve" the title field for the given product.category (ObjectId). Is there anyway to do all of that within the aggregate? In other words, is there a way to get the category model object's title field in the groups that are returned rather than having to do the extra looping work? Based on posts I've researched I don't see a built-in way but wanted to double-check.
getProductsGroupedByCategory = function(callback) {
Category.find(function(err, cats) {
var aggregate = [
{
$group: {
_id: "$category",
products: {
$push: {
title: "$title",
authors: "$authors",
publishDate: "$publishDate",
description: "$description"
}
}
}
},
{
$sort: {
"_id": 1
}
}
];
Product.aggregate(aggregate, function(err, catProducts) {
//Grab name of category and associate with each group
//since we only have the category_id at this point
for (var i = 0; i<catProducts.length;i++) {
var catProduct = catProducts[i];
for (var j=0;j<cats.length;j++) {
if (catProduct._id.toString() === cats[j]._id.toString()) {
catProduct.category = cats[j].title;
}
}
};
callback(err, catProducts);
});
});
}, //more code follows

An example datum would be helpful along with what you need out of it. From What I understand you are looking to get the title in to the grouping criteria and that should be doing by having a compound grouping criteria i.e.
_id: {category: "$category", title: "$title"}
If the title is within an array, you should do unwind, group and then wind again to achieve the result.

Related

Update a mongo field based on another collections data

I'm looking for a way to update a field based on the sum of the data of another collection.
I tried to bring all the meals and use forEach to call the Products collection for each meal, tested if it was working, but I got a time out.
meals.find().forEach(meal => {
var products = db.Products.find(
{ sku: { $in: meal.products } },
{ _id: 1, name: 1, sku: 1, nutritional_facts: 1 }
)
printjson(products)
})
My goal was to execute something like this below to get the desired result, but I got "SyntaxError: invalid for/in left-hand side". Is not possible to use for in inside a mongo query?
db.Meals.find({}).forEach(meal => {
const nutri_facts = {};
db.Products.find({ sku: { $in: meal.products } },
{ _id: 1, name: 1, sku: 1, nutri_facts: 1 }).forEach(product => {
for (let nutriFact in product.nutri_facts) {
nutri_facts[nutriFact] =
parseFloat(nutri_facts[nutriFact]) +
parseFloat(product.nutri_facts[nutriFact]);
}
}
});
for (let nutriFact in nutri_facts) {
meal.nutri_facts[nutriFact] =
nutri_facts[nutriFact];
}
}
db.Meals.updateOne({ _id: meal._id }, meal)
});
I also had a hard time trying to figure out how to use aggregate and lookup in this case but was not successful.
Is it possible to do that?
Example - Meals Document
{
_id: ObjectId("..."),
products : ["P068","L021","L026"], //these SKUs are part of this meal
nutri_facts: {
total_fat: 5g,
calories: 100kcal
(...other properties)
}
}
For each meal I need to look for its products on 'Products' collections using 'sku' field.
Then I will sum the nutritional facts of all products to get the meal nutritional facts.
Example Products Document
{
_id: ObjectId("..."),
sku: 'A010'
nutri_facts: {
total_fat: 2g,
calories: 40kcal
(...other properties)
}
}
I know that mongo might not be the best option in this case, but the entire application is already built using it.
For each meal I need to look for its products on 'Products'
collections using 'sku' field. Then I will sum the nutritional facts
of all products to get the meal nutritional facts.
db.Meals.find( { } ).forEach( meal => {
// print(meal._id);
const nutri_facts_var = { };
db.Products.find( { sku: { $in: meal.products } }, { nutri_facts: 1 }.forEach( product => {
// printjson(product.nutri_facts);
for ( let nutriFact in product.nutri_facts ) {
let units = (product.nutri_facts[nutriFact].split(/\d+/)).pop();
// print(units)
// Checks for the existence of the field and then adds or assigns
if ( nutri_facts_var[nutriFact] ) {
nutri_facts_var[nutriFact] = parseFloat( nutri_facts_var[nutriFact] ) + parseFloat( product.nutri_facts[nutriFact] );
}
else {
nutri_facts_var[nutriFact] = parseFloat( product.nutri_facts[nutriFact] );
}
nutri_facts_var[nutriFact] = nutri_facts_var[nutriFact] + units;
}
} );
// printjson(nutri_facts_var);
db.Meals.updateOne( { _id: meal._id }, { $set: { nutri_facts: nutri_facts_var } } );
} );
NOTES:
I use the variable nutri_facts_var name ( the var suffixed) so
that we can distinguish the user defined variable names easily from
the document fields names.
{ _id: 1, name: 1, sku: 1, nutri_facts: 1 } changed to {
nutri_facts: 1 }. The _id is included by default in a
projection. The fields name and sku are not needed.
db.Meals.updateOne({ _id: meal._id }, meal) is not a correct
syntax. The update operations use Update
Operators.
The corrected code: db.Meals.updateOne( { _id: meal._id }, { $set:
{ nutri_facts: nutri_facts_v } } ). Note we are updating the
nutifacts only, not all details.
Since the individual nutrifacts are stored as strings (e.g.,
"100kcal"), during arithmetic the string parts are stripped. So, we
capture the string units (e.g., "kcal") for each nutrifact and
append it later after the arithmetic. The following code strips and
stores the units part: let units =
(product.nutri_facts[nutriFact].split(/\d+/)).pop().
Use the mongo shell methods print and printjson to print the
contents of a variable or an object respectively - for debugging
purposes.
Note the query updates the Meal collection even if the nutri_facts field is not defined in it; the $set update operator creates new fields and sets the values in case the fields do not exist.

Pulling/deleting an item from a nested array

Note: it's a Meteor project.
My schema looks like that:
{
_id: 'someid'
nlu: {
data: {
synonyms:[
{_id:'abc', value:'car', synonyms:['automobile']}
]
}
}
}
The schema is defined with simple-schema. Relevant parts:
'nlu.data.synonyms.$': Object,
'nlu.data.synonyms.$._id': {type: String, autoValue: ()=> uuidv4()},
'nlu.data.synonyms.$.value': {type:String, regEx:/.*\S.*/},
'nlu.data.synonyms.$.synonyms': {type: Array, minCount:1},
'nlu.data.synonyms.$.synonyms.$': {type:String, regEx:/.*\S.*/},
I am trying to remove {_id:'abc'}:
Projects.update({_id: 'someid'},
{$pull: {'nlu.data.synonyms' : {_id: 'abc'}}});
The query returns 1 (one doc was updated) but the item was not removed from the array. Any idea?
This is my insert query
db.test.insert({
"_id": "someid",
"nlu": {
"data": {
"synonyms": [
{
"_id": "abc"
},
{
"_id": "def"
},
10,
[ 5, { "_id": 5 } ]
]
}
}
})
And here is my update
db.test.update(
{
"_id": "someid",
"nlu.data.synonyms._id": "abc"
},
{
"$pull": {
"nlu.data.synonyms": {
"_id": "abc"
}
}
}
)
The problem broke down to the autoValue parameter on your _id property.
This is a very powerful feature to manipulate automatic values on your schema. However, it prevented from pulling as it had always returned a value, indicating that this field should be set.
In order to make it aware of the pulling, you can make it aware of an operator being present (as in cases of mongo updates).
Your autoValue would then look like:
'nlu.data.synonyms.$._id': {type: String, autoValue: function(){
if (this.operator) {
this.unset();
return;
}
return uuidv4();
}},
Edit: Note the function here being not an arrow function, otherwise it losses the context that is bound on it by SimpleSchema.
It basically only returns a new uuid4 when there is no operator present (as in insert operations). You can extend this further by the provided functionality (see the documentation) to your needs.
I just summarized my code to a reproducable example:
import uuidv4 from 'uuid/v4';
const Projects = new Mongo.Collection('PROJECTS')
const ProjectSchema ={
nlu: Object,
'nlu.data': Object,
'nlu.data.synonyms': {
type: Array,
},
'nlu.data.synonyms.$': {
type: Object,
},
'nlu.data.synonyms.$._id': {type: String, autoValue: function(){
if (this.operator) {
this.unset();
return;
}
return uuidv4();
}},
'nlu.data.synonyms.$.value': {type:String, regEx:/.*\S.*/},
'nlu.data.synonyms.$.synonyms': {type: Array, minCount:1},
'nlu.data.synonyms.$.synonyms.$': {type:String, regEx:/.*\S.*/},
};
Projects.attachSchema(ProjectSchema);
Meteor.startup(() => {
const insertId = Projects.insert({
nlu: {
data: {
synonyms:[
{value:'car', synonyms:['automobile']},
]
}
}
});
Projects.update({_id: insertId}, {$pull: {'nlu.data.synonyms' : {value: 'car'}}});
const afterUpdate = Projects.findOne(insertId);
console.log(afterUpdate, afterUpdate.nlu.data.synonyms.length); // 0
});
Optional Alternative: Normalizing Collections
However there is one additional note for optimization.
You can work around this auto-id generation issue by normalizing synonyms into an own collection, where the mongo insert provides you an id. I am not sure how unique this id will be compared to uuidv4 but i never faced id issues with that.
A setup could look like this:
const Synonyms = new Mongo.Collection('SYNONYMS');
const SynonymsSchema = {
value: {type:String, regEx:/.*\S.*/},
synonyms: {type: Array, minCount:1},
'synonyms.$': {type:String, regEx:/.*\S.*/},
};
Synonyms.attachSchema(SynonymsSchema);
const Projects = new Mongo.Collection('PROJECTS')
const ProjectSchema ={
nlu: Object,
'nlu.data': Object,
'nlu.data.synonyms': {
type: Array,
},
'nlu.data.synonyms.$': {
type: String,
},
};
Projects.attachSchema(ProjectSchema);
Meteor.startup(() => {
// just add this entry once
if (Synonyms.find().count() === 0) {
Synonyms.insert({
value: 'car',
synonyms: ['automobile']
})
}
// get the id
const carId = Synonyms.findOne()._id;
const insertId = Projects.insert({
nlu: {
data: {
synonyms:[carId] // push the _id as reference
}
}
});
// ['MG464i9PgyniuGHpn'] => reference to Synonyms document
console.log(Projects.findOne(insertId).nlu.data.synonyms);
Projects.update({_id: insertId}, {$pull: {'nlu.data.synonyms' : carId }}); // pull the reference
const afterUpdate = Projects.findOne(insertId);
console.log(afterUpdate, afterUpdate.nlu.data.synonyms.length);
});
I know this was not part of the question but I just wanted to point out that there are many benefits of normalizing complex document structures into separate collections:
no duplicate data
decouple data that is not intended to be bound (here: Synonyms could be also used independently from Projects)
update referred documents once, all Projects will point to the very actual version (since it's a reference)
finer publication/subscription handling => more control about what data flows over the wire
reduces complex auto and default value generation
changes in the referred collection's schema may have only few consequences for UI and functions that make use of the referrer's schema.
Of course this has also disadvantages:
more collections to handle
more code to write (more code = more potential errors)
more tests to write (much more time to invest)
sometimes you need to denormalize back for this one case out of 100
you have to invest a lot of time in data schema design before starting to code

fetching documents based on nested subdoc array value

I'm trying to get all documents in a collection based on a subdocument array values. This is my data structure in the collection i'm seeking:
{
_id: ObjectId('...'),
name: "my event",
members:
[
{
_id: ObjectId('...'),
name: "family",
users: [ObjectId('...'),ObjectId('...'),ObjectId('...')]
},
{
_id: ObjectId('...'),
name: "work",
users: [ObjectId('...'),ObjectId('...'),ObjectId('...')]
}
]
}
I should point out that the schema of these objects are defined like so:
Events:
{
name: { type: String },
members: {type: [{ type: ObjectId, ref: 'MemberGroup' }], default:[] }
}
MemberGroup:
{
name: { type: String },
users: [{type: mongoose.Schema.Types.ObjectId, ref: 'User'}]
}
and of course User is just an arbitrary object with an id.
What i'm trying to fetch: i want to retrieve all events which has a specific user id in its member.users field.
i'm not sure if its event possible in a single call but here is what i've tried:
var userId = new mongoose.Schema.ObjectId('57d9503ef10d5ffc08d6b8cc');
events.find({members: { $elemMatch : { users: { $in: [userId]} } }})
this syntax work but return no elements even though i know there are matching elements (using robomongo to visualize the db)
So after a long search in stackoverflow i came across the good answare here:
MongoDB: How to find out if an array field contains an element?
my query changed to the following which gave me my desired result:
var userId = new mongoose.Schema.ObjectId('57d9503ef10d5ffc08d6b8cc');
events.find({}, {members: { $elemMatch : { users: { $eq: userId} } }})
notice the use of the second find parameter to specify the query limit instead of the first one (which is odd)

Mongoose Update Instance from Nested Array

In Mongoose, lets say I have a User object pulled from MongoDB and that user has an array of Interests. Now I get an instance of one of that user's Interests.
var user = ...
var interest = ...
... //Make some changes to interest.
How do I update that Interest object (after making some changes to it) within the User array in the DB?
Edit
Here is my current code. It doesn't work and doesn't give an error.
User.update(
{
'_id': user._id,
'interests._id': interest._id
},
{
'$set': {
'interests.$.xyzProperty': interest.xyzProperty
}
},
function(err,obj){//some error checking}
);
If you set an if for each interest, you can access the interest by the $ operator.
user document
{
_id: ObectId('54b568531ef35a7c348f21f2'),
interests: [
{
_id: 12345,
title: 'Tacos',
description: 'I Love tacos'
},
{...},
{...},
]
}
If I know which interest sub document I want to update, I simply query it like so:
UserModel.find({_id: ObectId('54b568531ef35a7c348f21f2'), 'interests.i_d': 12345}).lean().exec(function (err, user) {
var interest = ... //find specific interest
interest.description = 'I love tacos... Like, a lot'.
UserModel.update(
{
_id: user._id,
'interests._id': interest._id
},
{
$set: {
'interests.$.description': interest.description
}
},
function (err, update) {
console.log(err, update);
}
);
});
This uses the $ positional operator and updates the specific sub document(or item in an array).

Meteor publish: secret field of subdocument

I have the following collection with an array of subdocument:
{
_id:
players: [
{
_id: 1
answer
score:
},
{
_id: 2
answer:
score:
}]
}
I want to perform a publish function in order that it excludes the field answer of the other player. I.e player 1 should have on his local minimongo this doc:
{
_id:
players: [
{
_id: 1
answer
score:
},
{
_id: 2
score:
}]
}
I tried something like this:
Meteor.publish('game', function (id) {
return Game.find({_id: id}, {players.player_id: 0});
});
But I don't know how to only remove the field answer for the specific player.
I hate working with arrays like this in MongoDB. Personally I would use another collection GamePlayers with a document for each player in each game e.g.
Game ({ _id: g1 })
GamePlayers ({ _id: 0, gameId: g1, playerId: p1, answer: x, score: 0 });
GamePlayers ({ _id: 1, gameId: g1, playerId: p2, answer: y, score: 5 });
This would make things a lot easier.
But to actually answer your question here is a way to do it. I'm sure there is a more elegant way to do it but again I struggle using arrays in MongoDB so I can't think of it.
Since meteor publishes are effectively observeChanges functions we can do this:
note: this assumes that the _id of each player in the players array is equal to the user's Meteor.userId(), if it's not then you will need to provide the playerId as another argument to the publish along with gameId and change as appropriate.
I also assume your Games collection is called "games"
Games = new Meteor.Collection("games")
Meteor.publish('game', function(gameId) {
var self = this;
var handle = Games.find(gameId).observeChanges({
added: function(id, fields) {
self.added("games", id, removeSecretPlayerInfo(fields, self.userId));
},
changed: function(id, fields) {
self.changed("games", id, removeSecretPlayerInfo(fields, self.userId));
},
removed: function(id) {
self.removed("games", id);
}
});
self.ready();
self.onStop(function() {
handle.stop();
});
});
//this function takes all the fields that would be sent to the client,
//goes through the player array and if the player's id _id is not equal to
//the id of the user making the subscription we remove the answer from the array
//before sending it to them
var removeSecretPlayerInfo = function(fields, playerId) {
if (fields.players) {
for (var i = 0; i < fields.players.length; i++) {
if (fields.players[i]._id !== playerId)
delete fields.players[i].answer;
}
}
return fields;
}
You need to use quotations when querying a subfield. Also note that the second parameter to find is on options object that should have a property fields.
return Game.find({_id: id}, {fields: {'players.answer': 0}});