Supposing I have the following situation on mongodb:
{
"_id" : 654321,
"first_name" : "John",
"demand" : [1, 20, 4, 10 ],
"group" : [1, 2]
}
{
"_id" : 654321,
"first_name" : "Bert",
"demand" : [4, 10 ],
"group" : [1, 3]
}
1 - Is it possible to groupby based on the first index of "group" array ([1]) ?
2- Is it possible to reverse the index order, and sum those demand arrays vertically ?
Desired output:
1 - Select only group.0 : 1
2 - reverse the array order $reverseArray
[1, 20, 4, 10 ] -> [10, 4, 20, 1] (reversed)
[4, 10] -> [10, 4] (reversed)
3 - Sum (vertical axis)
[20, 8, 20, 1]
Finally, return the normal order:
[1, 20, 8, 20]
1 - Is it possible to groupby based on the first index of "group"
array ([1]) ?
To get the first index position (i.e., 0; array indexes start from 0) use the $arrayElemAt aggregation operator:
db.collection.aggregate([ { $group: { _id: { $arrayElemAt: [ "$group", 0 ] } } }, ] )
2- Is it possible to reverse the index order, and sum those demand
arrays vertically ?
You can reverse an array using the $reverseArray aggregation array operator.
To get the sum of values of each array's element position, (i) get the index of each value with unwind, and finally (ii) group by the index and sum the values.
db.collection.aggregate( [
{
$addFields: {
demand: { $reverseArray: "$demand" }
}
},
{
$unwind: { path: "$demand", includeArrayIndex: "ix" }
},
{
$group: {
_id: "$ix",
sum: { $sum: "$demand" }
}
},
{
$sort: { _id: 1 } // this is optional; sorts by index position
}
] )
Related
My MongoDB version is 3.4. I have documents similar to these:
{
id: 1,
refs: [2, 3, 4, 5, 6]
},
{
id: 3,
refs: [2, 5, 8]
}
Integers in refs array indicate other documents' ids in the same collection. What I want to achieve is to retrieve all documents in all refs combined in a result set. Suppose my imaginary query matches documents with ids 1 and 3 above, I am trying to get all the documents with 1's refs + 3's refs (2, 3, 4, 5, 6, 8)
I used $unwind + $lookup to first deconstruct refs array to individual documents and then look up in the same collection to retrieve entire documents, but I think this performs a separate query for each id in refs.
Is there a more efficient method?
I don't know a way of querying match {id:1} and {id:3} and combining them in a single line. But you can add a document that references 1 and 3 id's like below:
{
"_id" : ObjectId("60dd201bdb16e85753c4eef0"),
"id" : 1,
"refs" : [
2,
3,
4,
5,
6
]
}
{
"_id" : ObjectId("60dd201bdb16e85753c4eef1"),
"id" : 3,
"refs" : [
2,
5,
8
]
}
{
"_id" : ObjectId("60dd27acdb16e85753c4eef2"),
"id" : 10,
"refs" : [
1,
3
]
}
And then you can use match aggregate like {$match: {id: 10}} This makes a join with $lookup and brings what you want in a single line.
db.test.aggregate([
{$match: {id: 10}},
{$lookup: {
from: "test",
localField: "refs",
foreignField: "id",
as: "result"
}},
{$set: {result: {$reduce: {
input: "$result.refs",
initialValue: [],
in: {$setUnion: ["$$value", "$$this"]},
}}}}
]);
The result is:
{ "_id" : ObjectId("60dd27acdb16e85753c4eef2"), "id" : 10, "refs" : [ 1, 3 ], "result" : [ 2, 3, 4, 5, 6, 8 ] }
We used at first pipeline stage $lookup and thus made a left join on same collection. And then used $setUnion opearator in a $reduce aggregate function.
Not: We didn't use $concatArrays for duplicate items.
I got a problem when I use db.collection.aggregate in MongoDB.
I have a data structure like:
_id:...
Segment:{
"S1":1,
"S2":5,
...
"Sn":10
}
It means the following in Segment: I might have several sub attributes with numeric values. I'd like to sum them up as 1 + 5 + .. + 10
The problem is: I'm not sure about the sub attributes names since for each document the segment numbers are different. So I cannot list each segment name. I just want to use something like a for loop to sum all values together.
I tried queries like:
db.collection.aggregate([
{$group:{
_id:"$Account",
total:{$sum:"$Segment.$"}
])
but it doesn't work.
You have made the classical mistake to have arbitrary field names. MongoDB is "schema-free", but it doesn't mean you don't need to think about your schema. Key names should be descriptive, and in your case, f.e. "S2" does not really mean anything. In order to do most kinds of queries and operations, you will need to redesign you schema to store your data like this:
_id:...
Segment:[
{ field: "S1", value: 1 },
{ field: "S2", value: 5 },
{ field: "Sn", value: 10 },
]
You can then run your query like:
db.collection.aggregate( [
{ $unwind: "$Segment" },
{ $group: {
_id: '$_id',
sum: { $sum: '$Segment.value' }
} }
] );
Which then results into something like this (with the only document from your question):
{
"result" : [
{
"_id" : ObjectId("51e4772e13573be11ac2ca6f"),
"sum" : 16
}
],
"ok" : 1
}
Starting Mongo 3.4, this can be achieved by applying inline operations and thus avoid expensive operations such as $group:
// { _id: "xx", segments: { s1: 1, s2: 3, s3: 18, s4: 20 } }
db.collection.aggregate([
{ $addFields: {
total: { $sum: {
$map: { input: { $objectToArray: "$segments" }, as: "kv", in: "$$kv.v" }
}}
}}
])
// { _id: "xx", total: 42, segments: { s1: 1, s2: 3, s3: 18, s4: 20 } }
The idea is to transform the object (containing the numbers to sum) as an array. This is the role of $objectToArray, which starting Mongo 3.4.4, transforms { s1: 1, s2: 3, ... } into [ { k: "s1", v: 1 }, { k: "s2", v: 3 }, ... ]. This way, we don't need to care about the field names since we can access values through their "v" fields.
Having an array rather than an object is a first step towards being able to sum its elements. But the elements obtained with $objectToArray are objects and not simple integers. We can get passed this by mapping (the $map operation) these array elements to extract the value of their "v" field. Which in our case results in creating this kind of array: [1, 3, 18, 42].
Finally, it's a simple matter of summing elements within this array, using the $sum operation.
Segment: {s1: 10, s2: 4, s3: 12}
{$set: {"new_array":{$objectToArray: "$Segment"}}}, //makes field names all "k" or "v"
{$project: {_id:0, total:{$sum: "$new_array.v"}}}
"total" will be 26.
$set replaces $addFields in newer versions of mongo. (I'm using 4.2.)
"new_array": [
{
"k": "s1",
"v": 10
},
{
"k": "s2",
"v": 4
},
{
"k": "s3",
"v": 12
}
]
You can also use regular expressions. Eg. /^s/i for words starting with "s".
Suppose I have this document i mongodb.
{ "_id" : 1, "seats" : [ 80, 85, 90, 95 ] }
{ "_id" : 2, "seats" : [ 88, 90, 92, 97 ] },
{ "_id" : 3, "seats" : [ 85, 100, 90, 85 ] },
I want to increment the seats array from position 1 to position 3 of the "_id": 1 and make it to "seats": [80,86, 91, 96]. How to achieve this ?
You can try a combination of both the $inc update operator and the dot notation to access the elements of the seats array by the zero-based index position and increment them by one. The following update() operation uses the $inc operator to increase the seats array from position 1 to position 3 of the "_id": 1 field by 1:
db.collection.update(
{ "_id": 1 },
{
"$inc": {
"seats.1": 1,
"seats.2": 1,
"seats.3": 1
}
}
);
To update each element in the seats array from given index you need to loop over each element for that index and use the $inc operator to increment the value with "bulk" operation.
var bulkOp = db.collection.initializeOrderedBulkOp();
var fromIndex = 1
var toIndex = 3
db.collection.find({ '_id': 1 }).forEach(function(doc) {
var seats = doc.seats;
for(var index = fromIndex; index <= toIndex; index++) {
bulkOp.find({ '_id': doc._id, 'seats': seats[index] })
.updateOne({ '$inc': { 'seats.$':1 } });
bulkOp.execute();
Then db.collection.find({ '_id': 1 }) yields
{ "_id" : 1, "seats" : [ 80, 86, 91, 96 ] }
I got a problem when I use db.collection.aggregate in MongoDB.
I have a data structure like:
_id:...
Segment:{
"S1":1,
"S2":5,
...
"Sn":10
}
It means the following in Segment: I might have several sub attributes with numeric values. I'd like to sum them up as 1 + 5 + .. + 10
The problem is: I'm not sure about the sub attributes names since for each document the segment numbers are different. So I cannot list each segment name. I just want to use something like a for loop to sum all values together.
I tried queries like:
db.collection.aggregate([
{$group:{
_id:"$Account",
total:{$sum:"$Segment.$"}
])
but it doesn't work.
You have made the classical mistake to have arbitrary field names. MongoDB is "schema-free", but it doesn't mean you don't need to think about your schema. Key names should be descriptive, and in your case, f.e. "S2" does not really mean anything. In order to do most kinds of queries and operations, you will need to redesign you schema to store your data like this:
_id:...
Segment:[
{ field: "S1", value: 1 },
{ field: "S2", value: 5 },
{ field: "Sn", value: 10 },
]
You can then run your query like:
db.collection.aggregate( [
{ $unwind: "$Segment" },
{ $group: {
_id: '$_id',
sum: { $sum: '$Segment.value' }
} }
] );
Which then results into something like this (with the only document from your question):
{
"result" : [
{
"_id" : ObjectId("51e4772e13573be11ac2ca6f"),
"sum" : 16
}
],
"ok" : 1
}
Starting Mongo 3.4, this can be achieved by applying inline operations and thus avoid expensive operations such as $group:
// { _id: "xx", segments: { s1: 1, s2: 3, s3: 18, s4: 20 } }
db.collection.aggregate([
{ $addFields: {
total: { $sum: {
$map: { input: { $objectToArray: "$segments" }, as: "kv", in: "$$kv.v" }
}}
}}
])
// { _id: "xx", total: 42, segments: { s1: 1, s2: 3, s3: 18, s4: 20 } }
The idea is to transform the object (containing the numbers to sum) as an array. This is the role of $objectToArray, which starting Mongo 3.4.4, transforms { s1: 1, s2: 3, ... } into [ { k: "s1", v: 1 }, { k: "s2", v: 3 }, ... ]. This way, we don't need to care about the field names since we can access values through their "v" fields.
Having an array rather than an object is a first step towards being able to sum its elements. But the elements obtained with $objectToArray are objects and not simple integers. We can get passed this by mapping (the $map operation) these array elements to extract the value of their "v" field. Which in our case results in creating this kind of array: [1, 3, 18, 42].
Finally, it's a simple matter of summing elements within this array, using the $sum operation.
Segment: {s1: 10, s2: 4, s3: 12}
{$set: {"new_array":{$objectToArray: "$Segment"}}}, //makes field names all "k" or "v"
{$project: {_id:0, total:{$sum: "$new_array.v"}}}
"total" will be 26.
$set replaces $addFields in newer versions of mongo. (I'm using 4.2.)
"new_array": [
{
"k": "s1",
"v": 10
},
{
"k": "s2",
"v": 4
},
{
"k": "s3",
"v": 12
}
]
You can also use regular expressions. Eg. /^s/i for words starting with "s".
I want to be able to find multiple documents that have three or more matching values in an array. Let's say we the following documents:
[{
name: 'John',
cars: [1, 2, 3, 4]
},
{
name: 'Jane',
cars: [1, 2, 3, 8]
},
{
name: 'Smith',
cars: [1, 8, 10]
}]
And we want to find documents that have at least three of the values (in cars) in the following array:
[1, 2, 3, 4, 5, 6, 7]
The results would then be:
[{
name: 'John',
cars: [1, 2, 3, 4]
},
{
name: 'Jane',
cars: [1, 2, 3, 8]
}]
Anyone know how to achieve this?
You can have a $in query issued and then by code filter the record having 3 or more entries in the desired array. (Here is some samle python code)
def dennisQuestion():
permissibleCars = [1,2,3,4,5,6,7]
cursor = db.collection.find({"cars": {"$in": permissibleCars}})
for record in cursor:
if len(set(permissible) & set(record["cars"]))) >= 3
yield record
This is a good question, and I don't think there's a simple way to do it with the usual operators that MongoDB gives you. However I can think of the following methods to achieve this:
1. New Field
Calculate this in app code and maintain the result in a new field on the document.
2. Brute Force
db.Collection.find( { $or: [
{ cars: $all [ 1, 2, 3 ] },
{ cars: $all [ 2, 3, 4 ] },
... list out all 35 combinations
] } )
3. Use $where
db.Collection.find( { cars: { $in: [1,2,3,4,5,6,7] }, $where: function() {
var numMatches = 0;
for (var i = 1; i <= 7; i++)
if (this.cars.indexOf(i) > -1) numMatches++;
return numMatches >= 3;
} } );
I had to slightly modify #Zaid Masud option 3 when the values where strings in Mongo 4.0.3:
db.Collection.find( { cars: { $in: ["s1", "s2", "s3" , "s4", "s5" , "s6" , "s7"] },
$where: function() {
var options = ["s1", "s2", "s3" , "s4", "s5" , "s6" , "s7"];
var numMatches = 0;
for (var i = 0; i < 7; i++)
if (this.cars.indexOf(options[i]) > -1)
numMatches++;
return numMatches >= 3;
}
} );
(This seemed a bit large for a comment)
For Mongo v4.4.1 this query works
[
{
$project: {
name: 1,
cars: 1,
show: {
$let: {
vars: {
"b": {
$gte: [{$size: {$setIntersection: [ [1,2,3,4,5,6,7],"$cars"]}},3]
}
},
in: "$$b"
}
}
}
},
{
$match: {
show: true,
}
},
{
$project: {
show: 0
}
}
]