Below I have a structure for supporting custom picklist fields (in this example) within my sails.js application. The general idea is we support a collection of custom picklist values on any model within the app and the customer can have total control of the configuration of the custom field.
I went with this relationship model as using a simple json field lacks robustness when it comes to updating each individual custom picklist value. If I allow a customer to change "Internal" to "External" I need to update all records that have the value "Internal" recorded against that custom picklist with the new value.
This way - when I update the "value" field of CustomPicklistValue wherever that record is referenced via ID it will use the new value.
Now the problem comes when I need to integrate this model into my existing report engine...
rawCollection
.aggregate(
[
{
$match: {
createdAt: {
$gte: rangeEndDate,
$lte: rangeStartDate
},
...$match
}
},
{
$project: {
...$project,
total: $projectAggregation
}
},
{
$group: {
_id: {
...$groupKey
},
total: {
[`$${aggrAttrFunc}`]: "$total"
}
}
}
],
{
cursor: {
batchSize: 100
}
}
)
Here is the main part of a method for retrieving and aggregating any models stored in my mongodb instance. A user can specify a whole range of things including but not limited to the model, field specific date ranges and filters such as "where certificate status equals expired" etc.
So I'm now presented with this data structure:
{
id: '5e5fb732a9422a001146509f',
customPicklistValues: [
{
id: '5e4e904f16ab94bff1a324a0',
value: 'Internal',
fieldName: 'Business Group',
customPicklist: '109c7a1a9d00b664f2ee7827'
},
{
id: '5e4e904f16ab94bff1a324a4',
value: 'Slack',
fieldName: 'Application',
customPicklist: '109c5a1a9d00b664f2ee7827'
}
],
}
And for the life of me can't work out if there's any way I can essentially pull out fieldName and value for each of the populated records as key-value pairs and add each to the parent record before running my match clause...
I think I need to use lookup to initially populate the customPicklistValues and then merge them somehow?
Any help appreciated.
EDIT:
#whoami has suggested I use $addFields. There was a fair amount I needed to do before $addFields to populate the linked records (due to how Waterline via sails.js handles saving Mongo ObjectIDs in related collections as strings), you can see my steps in compass here:
Final step would be to edit this or add a stage to it to actually be able to support a key:value pair like Business Group: "Finance" in this example.
You can try these stages after your $lookup stage :
db.collection.aggregate([
{
$addFields: {
customPicklistValues:
{
$arrayToObject: {
$map: {
input: '$customPicklistValues',
in: { k: '$$this.fieldName', v: '$$this.value' }
}
}
}
}
},
{ $replaceRoot: { newRoot: { $mergeObjects: ['$customPicklistValues', '$$ROOT'] } } },
{ $project: { customPicklistValues: 0 } }
])
Test : MongoDB-Playground
Related
I have query in MongoDB for which I'm trying to upsert an inner nested attribute that contains a dot(.) in the key. E.g. a document might look something like: (below is a fictitious example just to highlight the constraint I'm facing.)
const person = {
name: 'Peter',
address: {
'NY.postalCode': 12345,
'CA.postalCode': 23456,
}
}
However, when I try to update one of the nested attribute in address with the below $set operation, I get an additional object NY under address and its subKey postalCode as a result, instead of the flattened attribute within address.
await Person.findByIdAndUpdate(id, {
$set: {
'address.NY.postalCode': 98765,
}
}, { new: true });
// Output
{
name: 'Peter',
address: {
'NY.postalCode': 12345,
'CA.postalCode': 23456,
NY: {
postalCode: 98765,
}
}
}
I've tried using the escape character for dot (\u002e), but get the same output. Also, I have seen some new features for setting fields but only in Mongo v5: https://www.mongodb.com/docs/manual/reference/operator/aggregation/setField/#mongodb-expression-exp.-setField. However, we're using Mongo v4.2 and would not be able to upgrade until a while later.
Would like to check if there are any means to $set nested attributes in address that don't completely replace the address object? i.e. to be able to specifically upsert an inner attribute within address object?
For example, is it possible to use the aggregation framework to workaround this?
I've tried something like the below, but it didn't work - not sure if its something that I did wrong or its probably not possible to use the aggregation pipeline..
Person.aggregate([
{
$match: {
_id: id,
},
},
{
$replaceRoot: { newRoot: {
$mergeObjects: [
'$$ROOT.address',
{
'NY.postalCode': 98765,
},
],
} },
},
]);
For your scenario, you need to achieve the update with aggregation pipeline.
Use $literal to escape the field name with dot.
Via $mergeObjects to merge current address object with { NY.postalCode': 98765 }.
db.collection.update({
"_id": ObjectId("5a934e000102030405000000")
},
[
{
$set: {
"address": {
$mergeObjects: [
"$address",
{
$literal: {
"NY.postalCode": 98765
}
}
]
}
}
}
],
{
new: true
})
Demo # Mongo Playground
I've two mongo collections - File and Folder.
Both have some common fields like name, createdAt etc. I've a resources API that returns a response having items from both collections, with a type property added. type can be file or folder
I want to support pagination and sorting in this list, for example sort by createdAt. Is it possible with aggregation, and how?
Moving them to a container collection is not a preferred option, as then I have to maintain the container collection on each create/update/delete on either of the collection.
I'm using mongoose too, if it has got any utility function for this, or a plugin.
In this case, you can use $unionWith. Something like:
Folder.aggregate([
{ $project: { name: 1, createdAt: 1 } },
{
$unionWith: {
coll: "files", pipeline: [ { $project: { name: 1, createdAt: 1 } } ]
}
},
... // your sorting go here
])
I have a collection Users:
{
_id: "5cds8f8rfdshfd"
name: "Ted"
attending: [ObjectId("2cd9fjdkfsld")]
}
I have another collection Events:
{
_id: "2cd9fjdkfsld"
title: "Some Event Attended"
},
{
_id: "34dshfj29jg"
title: "Some Event NOT Attended"
}
I would like to return a list of all events being attended by a given user. However, I need to do this query from the Events collection as this is part of a larger query.
I have gone through the following questions:
$lookup on ObjectId's in an array - This question has the array as a local field; mine is foreign
MongoDB lookup when foreign field is an array of objects - The array is of objects themselves
MongoDB lookup when foreign field is an array
I have tried various ways of modifying the above answers to fit my situation but have been unsuccessful. The second answer from the third question gets me closest but I would like to filter out unmatching results rather than have them returned with a value of 0.
My desired output:
[
{
_id: "2cd9fjdkfsld"
title: "Some Event Attended"
},
]
One option would be like this:
db.getCollection('Events').aggregate({
$lookup: // join
{
from: "Users", // on Users collection
let: { eId: "$_id" }, // keep a local variable "eId" that points to the currently looked at event's "_id"
pipeline: [{
$match: { // filter where
"_id": ObjectId("5c6efc937ef75175b2b8e7a4"), // a specific user
$expr: { $in: [ "$$eId", "$attending" ] } // attends the event we're looking at
}
}],
as: "users" // push all matched users into the "users" array
}
}, {
$match: { // remove events that the user does not attend
"users": { $ne: [] }
}
})
You could obviously get rid of the users field by adding another projection if needed.
I have Item schema in which I have item details with respective restaurant. I have to find all items of particular restaurant and group by them with 'type' and 'category' (type and category are fields in Item schema), I am able to group items as I want but I wont be able to get complete item object.
My query:
db.items.aggregate([{
'$match': {
'restaurant': ObjectId("551111450712235c81620a57")
}
}, {
'$group': {
id: {
'$push': '$_id'
}
, _id: {
type: '$type'
, category: '$category'
}
}
}, {
$project: {
id: '$id'
}
}])
I have seen one method by adding each field value to group then project it. As I have many fields in my Item schema I don't feel this will good solution for me, Can I get complete object instead of Ids only.
Well you can always use $$ROOT providing that your server is MongoDB 2.6 or greater:
db.items.aggregate([
{ '$match': {'restaurant': ObjectId("551111450712235c81620a57")}},
{ '$group':{
_id : {
type : '$type',
category : '$category'
},
id: { '$push': '$$ROOT' },
}}
])
Which is going to place every whole object into the members of the array.
You need to be careful when doing this as with larger results you are certain to break BSON limits.
I would suggest that you are trying to contruct some kind of "search results", with "facet counts" or similar. For that you are better off running a separate query for the "aggregation" part and one for the actual document results.
That is a much safer and flexible approach than trying to group everything together.
Given a MongoDB collection with the following document structure:
{
array_of_subdocs:
[
{
animal: "cat",
count: 10
},
{
animal: "dog",
count: 20
},
...
]
}
where each document contains an array of sub-documents, I want transform the collection into documents of the structure:
{
cat: { count: 10 },
dog: { count: 20 },
...
}
where each sub-document is now the value of a new field in the main document named after one of the values within the sub-document (in the example, the values of the animal field is used to create the name of the new fields, i.e. cat and dog).
I know how to do this with eval with a Javascript snippet. It's slow. My question is: how can this be done using the aggregation pipeline?
According to this feature request and its solution, the new feature will be added for this functionality - Function called arrayToObject.
db.foo.aggregate([
{$project: {
array_of_subdocs: {
$arrayToObject: {
$map: {
input: "$array_of_subdocs",
as: "pair",
in: ["$$pair.animal", "$$pair.count"]
}
}
}
}}
])
But at the moment, no solution. I suggest you to change your data structure. As far as I see there are many feature requests labeled as Major but not done for years.