I'm trying to build a map as part of a group operation in MongoDB but am having no luck.
My documents are structured as follows :
type : artist,
artist : David Bowie,
song : "Starman"
For example as part of my aggregation I want to group all "types" and have a map containing the artists name as the key and the songs as a value. For an output like :
[_id : artist, myMap : ["David Bowie" : "Starman"]]
But I'm not sure if this is even possible. I've tried using Mongos $push operation but it doesn't like it when I try to push a map. Has anyone attempted this before?
Heres the query I tried :
db.music.aggregate(
{
$group: {
_id: "$type",
myMap : { $push: {"$artist" : "$song" }}
}
}
)
This may be close to what you are looking for:
db.music.aggregate(
{$project :
{
"type" : 1,
foo : { "artist" : "$artist" , "song" : "$song"}
}
},
{$group :
{ "_id" : "$type", "map" : {$push: "$foo" }}
}
)
I think you can not do that with MongoDB. But there is a Jira Ticket asking for that feature.
Related
I did this in my mongodb:
db.teams.insert({name:"Alpha team",employees:[{name:"john"},{name:"david"}]});
db.teams.insert({name:"True team",employees:[{name:"oliver"},{name:"sam"}]});
db.teams.insert({name:"Blue team",employees:[{name:"jane"},{name:"raji"}]});
db.teams.find({"employees.name":/.*o.*/});
But what I got was:
{ "_id" : ObjectId("5ddf3ca83c182cc5354a15dd"), "name" : "Alpha team", "employees" : [ { "name" : "john" }, { "name" : "david" } ] }
{ "_id" : ObjectId("5ddf3ca93c182cc5354a15de"), "name" : "True team", "employees" : [ { "name" : "oliver" }, { "name" : "sam" } ] }
But what I really want is
[{"name":"john"},{"name":"oliver"}]
I'm having a hard time finding examples of this without using some kind of programmatic iterator/loop. Or examples I find return the parent document, which means I'd have to parse out the embedded array employees and do some kind of UNION statement?
Eg.
How to get embedded document in mongodb?
Retrieve only the queried element in an object array in MongoDB collection
Can someone point me in the right direction?
Please add projections to filter out the fields you don't need. Please refer the project link mongodb projections
Your find query should be constructed with the projection parameters like below:
db.teams.find({"employees.name":/.*o.*/}, {_id:0, "employees.name": 1});
This will return you:
[{"name":"john"},{"name":"oliver"}]
Can be solved with a simple aggregation pipeline.
db.teams.aggregate([
{$unwind : "$employees"},
{$match : {"employees.name":/.*o.*/}},
])
EDIT:
OP Wants to skip the parent fields. Modified query:
db.teams.aggregate([
{$unwind : "$employees"},
{$match : {"employees.name":/.*o.*/}},
{$project : {"name":"$employees.name",_id:0}}
])
Output:
{ "name" : "john" }
{ "name" : "oliver" }
I have the following documents,
{
"_id" : ObjectId("5b85312981c1634f59751604"),
"date" : "0"
},
{
"_id" : ObjectId("5b85312981c1634f59751604"),
"date" : "20180330"
},
{
"_id" : ObjectId("5b85312981c1634f59751604"),
"date" : "20180402"
},
{
"_id" : ObjectId("5b85312981c1634f59751604"),
"date" : "20180323"
},
I tried to convert date to ISODate using $toDate in aggregation,
db.documents.aggregate( [ { "$addFields": { "received_date": { "$cond": [ {"$ne": ["$date", "0"] }, {"$toDate": "$date"}, new Date("1970-01-01") ] } } } ] )
the query executed fine, but when I
db.documents.find({})
to examine all the documents, nothing changed, I am wondering how to fix it. I am using MongoDB 4.0.6 on Linux Mint 19.1 X64.
As they mentioned in the comments, aggregate doesn't update documents in the database directly (just an output of them).
If you'd like to permanently add a new field to documents via aggregation (aka update the documents in the database), use the following .forEach/.updateOne method:
Your example:
db.documents
.aggregate([{"$addFields":{"received_date":{"$cond":[{"$ne":["$date","0"]}, {"$toDate": "$date"}, new Date("1970-01-01")]}}}])
.forEach(function (x){db.documents.updateOne({_id: x._id}, {$set: {"received_date": x.received_date}})})
Since _id's value is an ObjectID(), there may be a slight modification you need to do to {_id:x._id}. If there is, let me know and I'll update it!
Another example:
db.users.find().pretty()
{ "_id" : ObjectId("5acb81b53306361018814849"), "name" : "A", "age" : 1 }
{ "_id" : ObjectId("5acb81b5330636101881484a"), "name" : "B", "age" : 2 }
{ "_id" : ObjectId("5acb81b5330636101881484b"), "name" : "C", "age" : 3 }
db.users
.aggregate([{$addFields:{totalAge:{$sum:"$age"}}}])
.forEach(function (x){db.users.updateOne({name: x.name}, {$set: {totalAge: x.totalAge}})})
Being able to update collections via the aggregation pipeline seems to be quite valuable because of what you have the power to do with aggregation (e.g. what you did in your question, doing calculations based on other fields within the document, etc.). I'm newer to MongoDB so maybe updating collections via aggregation pipeline is "bad practice", but it works and it's been quite valuable for me. I wonder why it isn't more straight-forward to do?
Note: I came up with this method after discovering Nazo's now-deprecated .save() method. Shoutout to Nazo!
I am performing joins in mongodb using $lookup, now i am facing a problem here.
I have two collections first one to contains users all bookmarks brands and the second one contains all details about the brands.Now i am trying to return all brands details bookmarked by the user.
user_bookmarked Collection
{"mobile_no": "8971740148", "brands": ["5829c1df334d40e20e1d1c19", "5829c1df334d40e20e1d1c20", "5829c1df334d40e20e1d1c21"]}
brands Collection
{"_id": ObjectId("5829c1df334d40e20e1d1c19"), "brand_name": "Versace"}
{"_id": ObjectId("5829c1df334d40e20e1d1c20"), "brand_name": "Lee Cooper"}
{"_id": ObjectId("5829c1df334d40e20e1d1c21"), "brand_name": "Levis"}
My aggregation pipeline code is given below
{ $match: { mobile_no: mobile_no }},
{ $unwind: { path: "$brands", includeArrayIndex: "brandsposition"}},
{ $lookup: {from: "brands",localField: "brands",foreignField: "_id",as: "user_bookmarks"}},
Now the problem i am facing is that above code doesn't return anything as i am storing brands ids as string in my user_bookmarked collection but not as ObjectId so nothing is being returned. Now can anyone please tell me how can i change field type inside aggregation query.
Second thing i want to ask please tell me when using $lookup then does mongodb utilises index on foreign_field or not. Because i ran above aggregation pipeline with explain: true but i don't found any index that was utilised by above query i got this returned by the output.
db.user_bookmarked.runCommand('aggregate', {pipeline: [{ $match: { mobile_no: mobile_no }},
{ $unwind: { path: "$brands", includeArrayIndex: "brandsposition"}},
{ $lookup: {from: "brands",localField: "brands",foreignField: "_id",as: "user_bookmarks"}}], explain: true})
{
"waitedMS" : NumberLong(0),
"stages" : [
{
"$cursor" : {
"query" : {
"mobile_no" : "8971740148"
},
"queryPlanner" : {
"plannerVersion" : 1,
"namespace" : "test.restaurants",
"indexFilterSet" : false,
"parsedQuery" : {
"mobile_no" : {
"$eq" : "8971740148"
}
},
"winningPlan" : {
"stage" : "COLLSCAN",
"filter" : {
"mobile_no" : {
"$eq" : "8971740148"
}
},
"direction" : "forward"
},
"rejectedPlans" : [ ]
}
}
},
{
"$unwind" : {
"path" : "$brands",
"includeArrayIndex" : "brandsposition"
}
},
{
"$lookup" : {
"from" : "brands",
"as" : "user_bookmarks",
"localField" : "brands",
"foreignField" : "_id"
}
}
],
"ok" : 1
}
Now can anyone please help me out of here i have searched about both of this thing which is how to change type of field inside aggregation and does $lookup utilises indexes but Haven't found anything useful please help me out of here guys it would be really appreciable.
You cannot convert the string to a object Id within the pipeline, you'll have to go though each document and convert it manually, using something like (you shouldnt be storing a mix match of types anyway, so it's probably worth updating in the long run):
how to convert string to numerical values in mongodb
as for does $lookup use index, If you look at the stats from this blog you'll see that indexes are used -
http://guyharrison.squarespace.com/blog/2016/7/4/join-performance-in-mongodb-32-using-lookup.html
Try casting your brands to ObjectIds before your population:
user_bookmarked.brands.map((brand) => return mongoose.Types.ObjectId(brand) )
But you really should consider storing them as refs instead, your model should look something like:
const user_bookmarked = new mongoose.Schema({
...
brands: [{type: mongoose.Schema.Types.ObjectId, ref: 'Brands'}],
...
})
This way they will be ObjectIds from the start.!
Regarding the second question this post explains it I think: join-performance-in-mongodb-32-using-lookup
I have a collection of Projects in where projects are like this:
{
"_id" : ObjectId("57e3e55c638cb8b971"),
"allocInvestor" : "Example Investor",
"fieldFoo" : "foo bar",
"name" : "GTP 3 (Roof Lease)"
}
I want to receive a list of projects grouped by allocInvestor field and only show fields: name and id
If I use aggregate and $group like this:
db.getCollection('projects').aggregate([
{"$group" : {
_id:"$allocInvestor", count:{$sum:1}
}
}
])
I receive a count of project per allocInvestor but I need is to receive the list of allocInvestor with subarray of projects per allocInvestor.
I'm using meteor by the way in case that helps. But I first want to get the query right on mongodb then try for meteor.
You can use $push or $addToSet to create a list of name and _id per every group.
$push allows duplicates and $addToSet does not add an element to the list again, if it is already there.
db.getCollection('projects').aggregate([
{ "$group" : { _id : "$allocInvestor",
count : {$sum : 1},
"idList" : {"$addToSet" : "$_id"},
"nameList" : {"$addToSet":"$name"}
}
}
]);
To get the name and _id data in a single list:
db.getCollection('projects').aggregate([
{ "$group" : { _id : "$allocInvestor", "projects" : {"$addToSet" : {id : "$_id", name: "$name"}}}},
{"$project" : {"_id" : 0, allocInvestor : "$_id", "projects" : 1 }}
]);
Use the $$ROOT operator to reference the entire document and then use project to eliminate the fields that you do not require.
db.projects.aggregate([
{"$group" : {
"_id":"$allocInvestor",
"projects" : {"$addToSet" : "$$ROOT"}
}
},
{"$project" : {
"_id":0,
"allocInvestor":"$_id",
"projects._id":1
"projects.name":1
}
}
])
I have a MongoDB document like this example doc:
{
"_id" : "A",
"articleNumber" : "0123456",
"shopDependentProperties" :
{
"shop" : "DE",
"foo" : "foo",
"bar" : "bar"
}
}
and want to pull out the properties of shopDependentProperties, to have the following result
{
"_id" : "A",
"articleNumber" : "0123456",
"foo" : "foo",
"bar" : "bar"
}
In MongoDB Shell i can solve it this way:
db.test.aggregate(
[
{
$project:
{
_id : "$_id",
articleNumber : "$articleNumber",
foo:"$shopDependentProperties.foo",
bar:"$shopDependentProperties.bar"
}
}
]
)
But: In Spring Data MongoDB i can't extract the embedded document contents.
I tried many combinations, nothing worked. For example:
ProjectionOperation projection = Aggregation.project("_id");
projection.andExpression("shopDependentProperties.foo").as("foo");
projection.andExpression("shopDependentProperties.bar").as("bar");
System.out.println(projection.toDBObject(Aggregation.DEFAULT_CONTEXT));
will ignore the shopDependentProperties.shop stuff and just print out
{ "$project" : { "_id" : 1}}
Any suggestions?
Thx
Haven't tested this, but as of
http://docs.mongodb.org/manual/reference/operator/aggregation/project/
you specify included / excluded fields like this:
db.test.aggregate(
[
{
$project:
{
_id : "$_id",
articleNumber : 1,
"shopDependentProperties.foo": 1,
"shopDependentProperties.bar": 1
}
}
]
)
Further down they explain, how to include embedded documents in the projection result.
I know how to do it in MongoDB, the problem was about doing the same thing in Spring Data.
But it works the same way, why didn't I try that before?
Solution:
ProjectionOperation projection = Aggregation.project(
"brandName",
"$shopDependentProperties.foo",
"$shopDependentProperties.bar" );