I have these array of date ranges. It's for a chart feature in my wep app.
[{
"start": "7/01/2016",
"end": "7/31/2016"
},{
"start": "8/01/2016",
"end": "8/31/2016"
},{
"start": "9/01/2016",
"end": "9/30/2016"
}]
This is my sample data.
{
"_id": 68,
"charges": [
{
"id": "ch_1AD2wYHDsLEzoG2tjPo7uGnq",
"amount": 1200,
"created": "7/13/2016"
},{
"id": "ch_1ADPRPHDsLEzoG2t1k3o0qCz",
"amount": 2000,
"created": "8/1/2016"
},{
"id": "ch_1ADluFHDsLEzoG2t608Bppzn",
"amount": 900,
"created": "8/2/2016"
},{
"id": "ch_1AE8OWHDsLEzoG2tBmlm1A22",
"amount": 1800,
"created": "9/14/2016"
}
]
}
This is the result that I'm trying to achieve.
[
{
"created": "9/13/2016",
"amount": 1200
},{
"created": "9/14/2016",
"amount": 2900
},{
"created": "9/15/2016",
"amount": 1800
},
]
Can I achieve that without looping the date range and querying inside? I only manage to get this far
[
{
$match: { _id: 68 }
},{
$unwind: "$charges"
},{
I don't know what to do here
}
]
NOTE: Nevermind the date formatting
you can achieve this using the new $bucket operator introduced in mongodb 3.4 liek this :
db.collection.aggregate([
{
$match:{
_id:68
}
},
{
$unwind:"$charges"
},
{
$bucket:{
groupBy:"$charges.created",
boundaries:[
"7/01/2016",
"8/01/2016",
"9/01/2016",
"9/30/2016"
],
default:"others",
output:{
amount:{
$sum:"$charges.amount"
}
}
}
}
])
explaination:
match a specific document using $match
unwind charges array
group by range ( range is provided in boundaries)
this output:
{ "_id" : "7/01/2016", "amount" : 1200 }
{ "_id" : "8/01/2016", "amount" : 2900 }
{ "_id" : "9/01/2016", "amount" : 1800 }
Related
I want generate report from mongodb. I have a aggregation which generated:
[{"_id": {
"m": 1,
"y": 2020
},
"meals": [
{
"name": "Sandwich",
"servings": 2
},
{
"name": "Fish",
"servings": 7
},
{
"name": "Pizza",
"servings": 3
},
{
"name": "Beef",
"servings": 3
},
{
"name": "Soup",
"servings": 3
}]},
{"_id": {
"m": 12,
"y": 2018
},
"meals": [
{
"name": "Beef",
"servings": 1
},
{
"name": "Spaghetti",
"servings": 2
}]}]
I need to get 3 elements where servings the largest and sort them. If elements are not three, just sort. For example:
[{"_id": {
"m": 1,
"y": 2020
},
"meals": [
{
"name": "Fish",
"servings": 7
},
{
"name": "Pizza",
"servings": 3
},
{
"name": "Beef",
"servings": 3
}]},
{"_id": {
"m": 12,
"y": 2018
},
"meals": [
{
"name": "Spaghetti",
"servings": 2
},
{
"name": "Beef",
"servings": 1
}
]}]
I can't use find(), because I want to do this in aggregation. I tried to use $filter, but I am doing something wrong.
$unwind deconstruct meals array
$sort by servings in descending order
$group by _id and reconstruct meals array
$slice to get first 3 elements from meals
db.collection.aggregate([
{ $unwind: "$meals" },
{ $sort: { "meals.servings": -1 } },
{
$group: {
_id: "$_id",
meals: { $push: "$meals" }
}
},
{
$project: {
meals: { $slice: ["$meals", 3] }
}
}
])
Playground
It is very important to sort and find data from the servers by mongodb.
So I think that you should use the some commands as cooperating.
I mean in your case, you use that
find().sort().limit(3)
I hope that it would help you.
$unwind and $group are expensive.
Starting form version >= 4.4, MongoDB supports functions. You can use it in combination with slice to get the desired result.
db.collection.aggregate([
{
"$project": {
"meals": {
"$slice": [
{
"$function": {
"body": "function(updates) { updates.sort((a, b) => b.servings - a.servings); return updates;}",
"args": [
"$meals"
],
"lang": "js"
}
},
3
],
},
},
},
])
Mongo Playground Sample Execution
What I'm trying to do is add up the sum of the scores for any German Restaurant in Manhattan. I want to return the top 5 restaurants with their name and total score.
Here is the setup of the json data I'm working with:
{
"address": {
"building": "1007",
"coord": [ -73.856077, 40.848447 ],
"street": "Morris Park Ave",
"zipcode": "10462"
},
"borough": "Bronx",
"cuisine": "Bakery",
"grades": [
{ "date": { "$date": 1393804800000 }, "grade": "A", "score": 2 },
{ "date": { "$date": 1378857600000 }, "grade": "A", "score": 6 },
{ "date": { "$date": 1358985600000 }, "grade": "A", "score": 10 },
{ "date": { "$date": 1322006400000 }, "grade": "A", "score": 9 },
{ "date": { "$date": 1299715200000 }, "grade": "B", "score": 14 }
],
"name": "Morris Park Bake Shop",
"restaurant_id": "30075445"
}
I've tried multiple variations of this:
db.restaurants.aggregate([{$unwind: "$grades"}, {$group: {_id: {borough: "Manhattan", cuisine:"German"}, total:{$sum:"scores"}}}])
And this is what keeps getting returned. I'm new to MongoDb and I'm just not sure what I'm doing wrong.
{ "_id" : { "borough" : "Manhattan", "cuisine" : "German" }, "total" : 0 }
So here is an aggregation that might work assuming the average score is what is measured...
Query:
db.restaurants.aggregate([
{ $match: { "borough": "Manhattan", "cuisine": "German" } },
{ $unwind: "$grades" },
{ $group: {
"_id": "$name",
"avg_score": { $avg: "$grades.score" }
}
},
{ $sort: { "avg_score": -1 } },
{ $limit: 5 }
]).pretty()
Pipeline Explaination:
match on borough and cuisine to filter to only desirable conditions
Unwind the grades array to flatten structure for review
group on the restaurant name and get the average score
Sort on the average score descending to get the top scores at the
top of the list
Limit output to the top 5 restaurants.
EDIT:
Sort by sum instead of average...
db.restaurants.aggregate([
{ $match: { "borough": "Manhattan", "cuisine": "German" } },
{ $unwind: "$grades" },
{ $group: {
"_id": "$name",
"avg_score": { $avg: "$grades.score" },
"sum_score": { $sum: "$grades.score" }
}
},
{ $sort: { "sum_score": -1 } },
{ $limit: 5 }
]).pretty()
I have a below structure maintained in a sample collection.
{
"_id": "1",
"name": "Stock1",
"description": "Test Stock",
"lines": [
{
"lineNumber": "1",
"priceInfo": {
"buyprice": 10,
"sellprice": 15
},
"item": {
"id": "BAT10001",
"name": "CricketBat",
"description": "Cricket bat"
},
"quantity": 10
},
{
"lineNumber": "2",
"priceInfo": {
"buyprice": 10,
"sellprice": 15
},
"item": {
"id": "BAT10002",
"name": "CricketBall",
"description": "Cricket ball"
},
"quantity": 10
},
{
"lineNumber": "3",
"priceInfo": {
"buyprice": 10,
"sellprice": 15
},
"item": {
"id": "BAT10003",
"name": "CricketStumps",
"description": "Cricket stumps"
},
"quantity": 10
}
]
}
I have a scenario where i will be given lineNumber and item.id, i need to filter the above collection based on lineNumber and item.id and i need to project only selected fields.
Expected output below:
{
"_id": "1",
"lines": [
{
"lineNumber": "1",
"item": {
"id": "BAT10001",
"name": "CricketBat",
"description": "Cricket bat"
},
"quantity": 10
}
]
}
Note: I may not get lineNumber all the times, if lineNumber is null then i should filter for item.id alone and get the above mentioned output.The main purpose is to reduce the number of fields in the output, as the collection is expected to hold huge number of fields.
I tried the below query,
db.sample.aggregate([
{ "$match" : { "_id" : "1"} ,
{ "$project" : { "lines" : { "$filter" : { "input" : "$lines" , "as" : "line" , "cond" :
{ "$and" : [ { "$eq" : [ "$$line.lineNumber" , "3"]} , { "$eq" : [ "$$line.item.id" , "BAT10001"]}]}}}}}
])
But i got all the fields, i'm not able to exclude or include the required fields.
I tried the below query and it worked for me,
db.Collection.aggregate([
{ $match: { _id: '1' } },
{
$project: {
lines: {
$map: {
input: {
$filter: {
input: '$lines',
as: 'line',
cond: {
$and: [
{ $eq: ['$$line.lineNumber', '3'] },
{ $eq: ['$$line.item.id', 'BAT10001'] },
],
},
},
},
as: 'line',
in: {
lineNumber: '$$line.lineNumber',
item: '$$line.item',
quantity: '$$line.quantity',
},
},
},
},
},
])
You can achieve it with $unwind and $group aggregation stages:
db.collection.aggregate([
{$match: {"_id": "1"}},
{$unwind: "$lines"},
{$match: {
$or: [
{"lines.lineNumber":{$exists: true, $eq: "1"}},
{"item.id": "BAT10001"}
]
}},
{$group: {
_id: "$_id",
lines: { $push: {
"lineNumber": "$lines.lineNumber",
"item": "$lines.item",
"quantity": "$lines.quantity"
}}
}}
])
$match - sets the criterias for the documents filter. The first stage is takes document with _id = "1", the second takes only documents which have lines.lineNumber equal to "1" or item.id equal to "BAT10001".
$unwind - splits the lines array into seperated documents.
$group - merges the documents by the _id element and puts the generated object with lineNumber, item and quantity elements into the lines array.
How to get education data (include the parent) for each family member in families that has year 2006? Supose I have data in mongodb:
{
"name": "Families",
"size": 3,
"families": [
{
"name": "Johny's Family",
"size": 2,
"family_member": [
{
"name": "Ruben",
"age": 22,
"education": [
{
"school": "Edward Academy",
"year": 2003
}
]
},
{
"name": "Hana",
"age": 20,
"education": [
{
"school": "Edward Academy",
"year": 2006
},
{
"school": "Nanyang University",
"year": 2012
}
]
}
]
},
{
"name": "Boy's Family",
"size": 1,
"family_member": [
{
"name": "Boy",
"age": 23,
"education": [
{
"school": "Broklyn Academy",
"year": 2003
},
{
"school": "Home School",
"year": 2006
}
]
}
]
}
]
}
I have try to get it using plain find function in mongodb but the result is not realy I wanted. this is my mongo script:
db.getCollection('tester').find(
{
"name":"Families",
"families.family_member.education.year":2006
},
{
"families.$.family_member.education.year":1
}
)
Can anyone suggest the best method to get the data become something like this:
{
"name": "Families",
"size": 3,
"families": [
{
"name": "Johny's Family",
"size": 2,
"family_member": [
{
"name": "Hana",
"age": 20,
"education": [
{
"school": "Edward Academy",
"year": 2006
}
]
}
]
},
{
"name": "Boy's Family",
"size": 1,
"family_member": [
{
"name": "Boy",
"age": 23,
"education": [
{
"school": "Home School",
"year": 2006
}
]
}
]
}
]
}
You may use JavaScript code to process your data because mongodb return data in unit of Document.
process "families" field like this:
doc.families.map((family)=>{
family.family_member = family.family_member.map((member)=>{
member.education = member.education.filter((edu)=>edu.year==2006);
return member;
});
return family;
})
You can try this one. You can get expected result using aggregation and you have to use $unwind then $match and $group
db.CollectionName.aggregate([
{$unwind: "$families"},
{$unwind: "$families.family_member"},
{$unwind: "$families.family_member.education"},
{$match: {"families.family_member.education.year": 2006}},
{
$group: {
_id: "$_id",
name: {$first: "$name"},
size: {$first: "$size"},
families: {$push: "$families"}
}
}
])
Using aggregation pipeline we can get this result. To achieve this use $unwind and $match. $unwind need to be used twice since we have deeply nested arrays
db.tester.aggregate([
{$unwind:"$families"},
{$unwind:"$families.family_member"},
{$unwind:"$families.family_member.education"},
{$match:{"families.family_member.education.year":2006}}
])
Change your query like below, It should work:-
db.getCollection('tester').find({"families.family_member.education.year": 2006},{"families.family_member.education.year":1})
OR
If you want to print whole things, use only this:-
db.getCollection('tester').find({"families.family_member.education.year": 2006}
Thanks.
I have the following data schema:
{
"Address" : "Test1",
"City" : "London",
"Country" : "UK",
"Currency" : "",
"Price_History" : {
"2014-07-04T02:42:58" : [
{
"value1" : 98,
"value2" : 98,
"value3" : 98
}
],
"2014-07-04T03:50:50" : [
{
"value1" : 91,
"value2" : 92,
"value3" : 93
}
]
},
"Location" : [
9.3435,
52.1014
],
"Postal_code" : "xxx"
}
how could generate a query in mongodb to search for all results between "2014-07-04T02:42:58" and "2014-07-04T03:50:50" or how could generate a query to select only results with values from 91 till 93 without to know the date ?
thanks
Not a really good way to model this. A better example would be as follows:
{
"Address" : "Test1",
"City" : "London",
"Country" : "UK",
"Currency" : "",
"Price_History" : [
{ "dateEnrty": 1, "date": ISODate("2014-07-04T02:42:58Z"), "value": 98 },
{ "dateEntry": 2, "date": ISODate("2014-07-04T02:42:58Z"), "value": 98 },
{ "dateEntry": 3, "date": ISODate("2014-07-04T02:42:58Z"), "value": 98 },
{ "dateEntry": 1, "date": ISODate("2014-07-04T03:50:50Z"), "value": 91 },
{ "dateEntry": 2, "date": ISODate("2014-07-04T03:50:50Z"), "value": 92 },
{ "dateEntry": 3, "date": ISODate("2014-07-04T03:50:50Z"), "value": 93 },
],
"Location" : [
9.3435,
52.1014
],
"Postal_code" : "xxx"
}
Or something along those lines that does not utilize the path dependency. Your query here would be relatively simple, but also considering that MongodDB searches documents and not arrays for something like this. But you can dissect with the aggregation framework:
db.collection.aggregate([
// Still match first to reduce the possible documents
{ "$match": {
"Price_History": {
"$elemMatch": {
"date": {
"$gte": ISODate("2014-07-04T02:42:58Z"),
"$lte": ISODate("2014-07-04T03:50:50Z")
},
"value": 98
}
}
}},
// Unwind to "de-normalize"
{ "$unwind": "$Price_History" },
// Match this time to "filter" the array which is now documents
{ "$match": {
"Price_History.date": {
"$gte": ISODate("2014-07-04T02:42:58Z"),
"$lte": ISODate("2014-07-04T03:50:50Z")
},
"Price_Hisotry.value": 98
}},
// Now group back each document with the matches
{ "$group": {
"_id": "$_id",
"Address": { "$first": "$Address" },
"City": { "$first": "$City" },
"Country": { "$first": "$Country" },
"Currency": { "$first": "$Currency" },
"Price_History": { "$push": "$Price_History" },
"Location": { "$first": "$Location" },
"Postal_Code": { "$first": "$Postal_Code" }
}}
])
Or otherwise better off hanging the "normalization" and just go for discrete documents that you can simply process via a standard .find(). Must faster and simpler.
{
"Address" : "Test1",
"City" : "London",
"Country" : "UK",
"Currency" : "",
"date": ISODate("2014-07-04T02:42:58Z"),
"value": 98
}
Etc. So then just query:
db.collection.find({
"date": {
"$gte": ISODate("2014-07-04T02:42:58Z"),
"$lte": ISODate("2014-07-04T03:50:50Z")
},
"value": 98
})
I would really go with that as a "de-normalized" "Price History" collection as it is much more efficient and basically what the aggregation statement is emulating.
The query you ask for is possible using something that evaluates JavaScript like MongoDB mapReduce, but as I have already said, it will need to scan the entire collection without any index assistance, and that is bad.
Take your case to the boss to re-model and earn your bonus now.