Single array of objects sort and slice not working - mongodb

I have a single entry on a collection like this:
{
"_id" : ObjectId("60c6f7a5ef86bd1a5402e928"),
"cid" : 1,
"array1" : [
{ "type": "car", value: 20 },
{ "type": "bike", value: 50 },
{ "type": "bus", value: 5 },
{ "type": "cycle", value: 100 },
...... 9000 more entry something like this
],
"array2" : [
{ "type": "laptop", value: 200 },
{ "type": "desktop", value: 15 },
{ "type": "tablet", value: 55 },
{ "type": "mobile", value: 90 },
...... 9000 more entry something like this
]
}
Now I want to sort and slice the data for the pagination purpose.
For that I wrote the query which works well on slice case but not on sort case.
This is my query which works for slice case
let val = await SomeCollectionName.findOne(
{ cid: 1 },
{ _id: 1 , array1: { $slice: [0, 10] } } ---> its return the 10 data. Initially it return from 0 to 10, then next call $slice: [10, 10]
).exec();
if (val) {
//console.log('Got the value')
}
console.log(error)
This is my query When I add sort with slice
let val = await SomeCollectionName.findOne(
{ cid: 1 },
{ _id: 1 , array1: { $sort: { value: -1 }, $slice: [0, 10] } }
).exec();
if (val) {
//console.log('Got the value')
}
console.log(error)
Is there anyone who guide me where I'm wrong or suggest me what is the efficient way for getting the data.
UPDATE
I am getting the answer from the above question and looking for the same implementation for two array.
Everything is same. Earlier I was dealing with 1 array now this time I have to deal with two array.
Just curious to know that how these things happen
I wrote the aggregation query but one array results is fine but others are returning the same data throughout the array.
This is my query as per the suggestion of dealing with single array with sort and slice
db.collection.aggregate([
{
"$match": {
"cid": 1
}
},
{
$unwind: "$array1"
},
{
$unwind: "$array2"
},
{
"$sort": {
"array1.value": -1,
"array2.value": -1,
}
},
{
$skip: 0
},
{
$limit: 3
},
{
$group:{
"_id":"$_id",
"array1":{$push:"$array1"},
"array2":{$push:"$array2"}
}
}
])

The issue is that $sort is not supported by findOne() in its projection parameter.
You can instead use aggregation to achieve the expected result,
db.collection.aggregate([
{
"$match": {
"cid": 1
}
},
{
$unwind: "$array1"
},
{
"$sort": {
"array1.value": -1
}
},
{
$skip: 0
},
{
$limit: 3
},
{
$group: {
"_id": "$_id",
"array1": {
$push: {
"type": "$array1.type",
"value": "$array1.value"
}
},
"array2": {
"$first": "$array2"
}
},
},
{
$unwind: "$array2"
},
{
"$sort": {
"array2.value": -1
}
},
{
$skip: 0
},
{
$limit: 3
},
{
$group: {
"_id": "$_id",
"array2": {
$push: {
"type": "$array2.type",
"value": "$array2.value"
}
},
"array1": {
"$first": "$array1"
}
},
}
])
Aggregation
$unwind

Related

How to delete ($pull) nested array elements with property value null in arrays with more than one element

I would like to delete ($pull) nested array elements where one of the element's properties is null and where the array has more than one element.
Here is an example. In the following collection, I would like to delete those elements of the Orders array that have Amount = null and where the Orders array has more than one element. That is, I would like to delete only the element with OrderId = 12, but no other elements.
db.TestProducts.insertMany([
{
ProductDetails: { "ProductId": 1, Language: "fr" },
Orders: [
{ "OrderId": 11, "Amount": 200 },
{ "OrderId": 12, "Amount": null }
]
},
{
ProductDetails: { "ProductId": 2, Language: "es" },
Orders: [
{ "OrderId": 13, "Amount": 300 },
{ "OrderId": 14, "Amount": 400 }
]
},
{
ProductDetails: { "ProductId": 3, Language: "en" },
Orders: [
{ "OrderId": 15, "Amount": null }
]
}
]);
The following attempt is based on googling and a combination of a few other StackOverflow answers, e.g. Aggregate and update MongoDB
db.TestProducts.aggregate(
[
{ $match: { "Orders.Amount": { "$eq": null } } },
{ $unwind: "$Orders" },
{
"$group": {
"_id": {
ProductId: "$ProductDetails.ProductId",
Language: "$ProductDetails.Language"
},"count": { "$sum": 1 }
}
},
{ "$match": { "count": { "$gt": 1 } } },
{ "$out": "temp_results" }
],
{ allowDiskUse: true}
);
db.temp_results.find().forEach((result) => {
db.TestProducts.updateMany({"ProductDetails.ProductId": result._id.ProductId, "ProductDetails.Language": result._id.Language },
{ $pull: { "Orders": {"Amount": null } }})
});
This works, but I am wondering if it can be done in a simpler way, especially if it is possible to delete the array elements within the aggregation pipeline and avoid the additional iteration (forEach).
You can check these conditions in the update query, check 2 conditions
Amount is null
check the expression $expr condition for the size of the Orders array is greater than 1
db.TestProducts.updateMany({
"Orders.Amount": null,
"$expr": {
"$gt": [{ "$size": "$Orders" }, 1]
}
},
{
"$pull": {
"Orders": { "Amount": null }
}
})
Playground
an example
an example might help:
let feed = await Feed.findOneAndUpdate(
{
_id: req.params.id,
feeds: {
$elemMatch: {
type: FeedType.Location,
locations: {
$size: 0,
},
},
},
},
{
$pull: {
feeds: { locations: { $size: 0 }, type: FeedType.Location },
},
},
{ new: true, multi: true }
);

Paginate MongoDB aggregation response

I have a database of users that have skills. I have set up a way to find users in the database using am aggregation method included in mongoose. Depending on the search criteria I input into the aggregation, the results may be too big to actually display on my front end app. I am curious how I can paginate an aggregation query with the typical limit, page, and skip variables like you would do in a typical GET request.
Here is my aggregation query:
const foundUsers = await User.aggregate([
{
$addFields: {
matchingSkills: {
$filter: {
input: '$skills',
cond: {
$or: test,
},
},
},
requiredSkills,
},
},
{
$addFields: {
// matchingSkills: '$$REMOVE',
percentageMatch: {
$multiply: [
{ $divide: [{ $size: '$matchingSkills' }, skillSearch.length] }, // yu already know how many values you need to pass, thats' why `2`
100,
],
},
},
},
{
$addFields: {
matchingSkillsNames: {
$map: {
input: '$matchingSkills',
as: 'matchingSkill',
in: '$$matchingSkill.skill',
},
},
},
},
{
$addFields: {
missingSkills: {
$filter: {
input: '$requiredSkills',
cond: {
$not: {
$in: ['$$this', '$matchingSkillsNames'],
},
},
},
},
},
},
{
$match: { percentageMatch: { $gte: 25 } },
},
]);
Passing these skills to this aggregate function:
{
"skillSearch": [
{
"class": "skills",
"skill": "SQL",
"operator": "GT",
"yearsExperience": 6
},
{
"class": "skills",
"skill": "C",
"operator": "GT",
"yearsExperience": 1
}
]
}
Will result in a response similar to this:
{
"_id": "60184ce81e65633873d709aa",
"name": "Brad",
"email": "brad#gmail.com",
"password": "$2a$12$37v2RwaO5LhSMT8GJQSZyel.Aawn6AmlqqSOkZtopqIIXyJ0LRBfu",
"__v": 0,
"skills": [
{
"_id": "60a306ce819cde701c1934a8",
"skill": "SQL",
"yearsExperience": 8
},
{
"_id": "60a306ce819cde701c1934a9",
"skill": "C",
"yearsExperience": 5
},
{
"_id": "60a306ce819cde701c1934aa",
"skill": "PL/I",
"yearsExperience": 2
},
{
"_id": "60a306ce819cde701c1934ab",
"skill": "Awk",
"yearsExperience": 9
}
],
"matchingSkills": [
{
"_id": "60a306ce819cde701c1934a8",
"skill": "SQL",
"yearsExperience": 8
},
{
"_id": "60a306ce819cde701c1934a9",
"skill": "C",
"yearsExperience": 5
}
],
"requiredSkills": [
"SQL",
"C"
],
"percentageMatch": 100,
"matchingSkillsNames": [
"SQL",
"C"
],
"missingSkills": []
},
For the pagination, you need to pass the page and size form the front end
$sort to sort the documents,
$skip skip the documents. For eg : if you are in page two and u need 10 rows , u need to skip first 10 documents
$limit to how many documents you need to show after skip
here is the code
db.collection.aggregate([
{
$sort: {
_id: 1
}
},
{
$skip: 0 // page*size
},
{
$limit: 10 // size
}
])
Working Mongo playground
More than, the pagination requires total elements too, for that
db.collection.aggregate([
{
"$facet": {
"elements": [
{
"$group": {
"_id": null,
"count": { "$sum": 1 }
}
}
],
"data": [
{ $sort: { _id: 1 } },
{ $skip: 0 }, // page*size
{ $limit: 10 } // size
]
}
},
{ "$unwind": "$elements" },
{
"$addFields": {
"elements": "$$REMOVE",
"totalRecords": "$elements.count"
}
}
])
Working Mongo playground

Add sequence number to items from mongodb query

I have MongoDB collection items with following documents:
{ "name": "First", "value": 8 },
{ "name": "Second", "value": 2 },
{ "name": "Third", "value": 5 }
I would like to sort them by name or value and then add sequence number (index property) to each of them (first item will have "index": 1, second "index": 2, etc.). So for this query:
db.items.aggregate([{ $sort: { "value": 1 } }])
result should be:
{ "name": "Second", "value": 2, "index": 1 },
{ "name": "Third", "value": 5, "index": 2 },
{ "name": "First", "value": 8, "index": 3 }
This should also works with $skip. With { "$skip": 1 }, result should be items with indexes 2, 3, not 1, 2. Is there any way to do this using pure MongoDB, without any application logic?
Note: Inspired by #turvishal's answer to meet the below things
Index starting from 1
Skip the way OP wants
Mongo-play
db.collection.aggregate([
{
$sort: {
value: 1
}
},
{
$group: {
_id: 1,
items: {
$push: {
name: "$name",
value: "$value"
}
}
}
},
{
$unwind: {
path: "$items",
includeArrayIndex: "index"
}
},
{
$project: {
_id: 0,
name: "$items.name",
value: "$items.value",
index: { //Incrementing the index by 1
$add: [
"$index",
1
]
}
}
},
{ //Addition of skip
"$skip": 1
}
])
Might be this is one option you can do with includeArrayIndex in $unwind,
$sort by value that you already did
$group by name and value and push in items
$unwind deconstruct items and set field index in includeArrayIndex
$project required fields and increment index with because it start with 0 zero
db.collection.aggregate([
{ $sort: { value: 1 } },
{
$group: {
_id: 1,
items: {
$push: {
name: "$name",
value: "$value"
}
}
}
},
{
$unwind: {
path: "$items",
includeArrayIndex: "index"
}
},
{
$project: {
_id: 0,
name: "$items.name",
value: "$items.value",
index: { $add ["$index", 1] }
}
}
])
Playground: https://mongoplayground.net/p/572ReIpRcl6

mongoose aggregate sort one array by field

I would like to fetch only one element from each group of fields:
here is my result
[{
"_id":{"device_id":"9"},
"x":["2019-11-17T18:03:33.000Z","2019-11-17T12:02:35.000Z"],
"y":["93","3"]
},
{
"_id":{"device_id":"8"},
"x":["2019-11-16T12:05:33.000Z","2019-11-16T12:02:35.000Z"],
"y":["33","3"]
}]
I would like to get only the last inserted data, means only one value from x and one value from y.
This is the code :
aggregate([
{ "$sort": { "timeStamp": -1 } },
{
$group:
{
_id: { device_id: '$device_id' },
x: {
$push: "$timeStamp"
},
y: {
$push: "$value"
},
device_name: {
$push: "$device_name"
},
}
}
}]
I have tried to add project exp:
{
$project: {
time: "$x",
value:"$y",
integerValues:{
$map:{
input: "x",
as: "integerValue",
in: { $trunc: "$$integerValue" }
}
}
}
}
This is the result I expect to have it:
{
"_id":{"device_id":"8"},
"x":["2019-11-16T12:05:33.000Z"],
"y":["33"]
},
{
"_id":{"device_id":"9"},
"x":["2019-11-17T18:03:33.000Z"],
"y":["93"]
}
I was searching so much about this problem without any solution
All you need is a single project stage
aggregate([{
$project: {
x: {
$max: "$x"
},
y: {
$max: "$y"
}
}
}])
The result you get will be something like this,
[
{
"_id": {
"device_id": "9"
},
"x": "2019-11-17T18:03:33.000Z",
"y": "93"
},
{
"_id": {
"device_id": "8"
},
"x": "2019-11-16T12:05:33.000Z",
"y": "33"
}
]

MongoDB Get average of group considering rank of document

I have documents getting in order like:
{
"_id": "abcde1",
"value" : 300
},
{
"_id": "abcde2",
"value" : 200
},
{
"_id": "abcde3",
"value" : 400
},
{
"_id": "abcde4",
"value" : 500
},
{
"_id": "abcde5",
"value" : 600
}
i.e,
I want average of "_id" of first 2, first 4 and all 5 documents matching like in single query:
{
"value_2" : 250, // Average of first 2 documents
"value_4" : 350, // Average of first four documents
"value_5" : 400 // Average of all 5 documents
}
Is it possible to Group documents based on rank of document.
I can do 3 results in 3 separate queries. Is it possible in single query?
You could try running the following pipeline:
db.collection.aggregate([
// previous pipeline here
{
"$group": {
"_id": null,
"values": { "$push": "$value" }
}
},
{ "$unwind": { "path": "$values", "includeArrayIndex": "rank" } },
{
"$group": {
"_id": null,
"value_2_sum": {
"$sum": {
"$cond": [
{ "$lt": ["$rank", 2] },
"$values",
0
]
}
},
"value_2_count": {
"$sum": {
"$cond": [
{ "$lt": ["$rank", 2] },
1,
0
]
}
},
"value_4_sum": {
"$sum": {
"$cond": [
{ "$lt": ["$rank", 4] },
"$values",
0
]
}
},
"value_4_count": {
"$sum": {
"$cond": [
{ "$lt": ["$rank", 4] },
1,
0
]
}
},
"value_5": { "$avg": "$values" }
}
},
{
"$project": {
"value_2" : { "$divide": ["$value_2_sum", "$value_2_count"] }, // Average of first 2 documents
"value_4" : { "$divide": ["$value_4_sum", "$value_4_count"] }, // Average of first four documents
"value_5" : 1
}
}
])
You could use a $facet aggregation stage:
// { _id: "abcde1", value: 300 }
// { _id: "abcde2", value: 200 }
// { _id: "abcde3", value: 400 }
// { _id: "abcde4", value: 500 }
// { _id: "abcde5", value: 600 }
db.collection.aggregate([
{ $facet: {
value_2: [ { $limit: 2 }, { $group: { _id: null, value_2: { $avg: "$value" } } } ],
value_4: [ { $limit: 4 }, { $group: { _id: null, value_4: { $avg: "$value" } } } ],
value_5: [ { $limit: 5 }, { $group: { _id: null, value_5: { $avg: "$value" } } } ]
}},
// {
// value_2: [ { _id: null, value_2: 250 } ],
// value_4: [ { _id: null, value_4: 350 } ],
// value_5: [ { _id: null, value_5: 400 } ]
// }
{ $set: {
value_2: { $first: "$value_2.value_2" },
value_4: { $first: "$value_4.value_4" },
value_5: { $first: "$value_5.value_5" }
}}
])
// { "value_2" : 250, "value_4" : 350, "value_5" : 400 }
The $facet stage allows us to run multiple aggregation pipelines within a single stage on the same set of input documents. Each sub-pipeline has its own field in the output document where its results are stored as an array of documents.
Each field is thus produced by its own aggregation pipeline whose first stage is a simple $limit, followed by a $group stage that'll produce the $avg (average) of all considered documents.
The second part of the pipeline (the $set stage) is just there to clean-up the $facet output to the format you wished for.