MongoDB $limit inside an $and - mongodb

I'm looking to implement a query that will return me a single JSON with 2 sets of data together.
What I look for is to retrieve 40 orders which offerDate is still in the future ( $gte: today ) and 8 orders which offerDate is in the past.
Currently I only have the 40 in the future like so:
const orders = await db.Order.aggregate([
{
$match: {
offerDate: {
$gte: moment().startOf('day').toDate(),
},
},
},
{
$limit: 40,
},
How could I achieve to add the limit inside an $and to get both together?
I tried with:
$match: {
$and: [
{
offerDate: {
$gte: moment().startOf('day').toDate(),
},
$limit: 40
},
{
offerDate: {
$lte: moment().startOf('day').toDate(),
},
$limit: 8
},
]
}
},
But that didn't work at all and I cannot find the right way to do it.
Thanks a lot!

$limit is a pipeline stage which cannot be used within any other stage so what you're trying is not the way to go. You simply need two separate $matches followed by $limit and the $facet operator allows you to do that:
{
$facet: {
greater: [
{ $match: { offerDate: { $gte: moment().startOf('day').toDate() } } },
{ $limit: 40 }
],
lower: [
{ $match: { offerDate: { $lte: moment().startOf('day').toDate() } } },
{ $limit: 8 }
]
}
},
{
$project: {
all: { $concatArrays: [ "$greater", "$lower" ] }
}
}

Related

MongdDB: Combining query results of two collections as one

There are two collections (view and click) like following:
# View collection
_id publisher_id created_at
617f8ea98e0f54f05e10e796 1 2021-11-01T00:00:00.000Z
617f8eab8e0f54f05e10e798 1 2021-11-01T00:00:00.000Z
617f8eac8e0f54f05e10e79a 1 2021-11-01T00:00:00.000Z
617f90cea187d30ebbecdee9 2 2021-11-01T00:00:00.000Z
# Click collection
_id publisher_id created_at
617f8ea98e0f54f05e10e796 1 2021-11-01T00:00:00.000Z
617f8eab8e0f54f05e10e798 2 2021-11-01T00:00:00.000Z
How can I get the following expected results with one query?
(or)
What is the best way for the following expected results?
# Expected For Publisher ID(1)
_id view_count click_count
2021/11/1 3 1
# Expected For Publisher ID(2)
_id view_count click_count
2021/11/1 1 1
Currently, I am using 2 queries for both collections and combining results as one in code.
For View
db.view.aggregate([
/*FirstStage*/
{
$match:
{
"$and":
[
{
"publisher_id": 1
},
{
"created_at": {$gte: new ISODate("2021-11-01"), $lt: new ISODate("2021-11-28")}
}
]
}
},
/*SecondStage*/
{
$group:
{
_id: {$dateToString: {format: '%Y/%m/%d', date: "$created_at"}},
count: {
$sum: 1
}
}
}
])
For Click
db.click.aggregate([
/*FirstStage*/
{
$match:
{
"$and":
[
{
"publisher_id": 1
},
{
"created_at": {$gte: new ISODate("2021-11-01"), $lt: new ISODate("2021-11-28")}
}
]
}
},
/*SecondStage*/
{
$group:
{
_id: {$dateToString: {format: '%Y/%m/%d', date: "$created_at"}},
count: {
$sum: 1
}
}
}
])
Because you are querying two different collections there is no "good" way to merge this into one query, the only way I can think of is using $facet, where the first stage is the "normal" one, and the other stage starts with a $lookup from the other collection.
This approach does add overhead, which is why I recommend to just keep doing the merge in code, however for the sake of answering here is a sample:
db.view.aggregate([
{
$facet: {
views: [
{
$match: {
"$and": [
{
"publisher_id": 1
},
{
"created_at": {
$gte: ISODate("2021-11-01"),
$lt: ISODate("2021-11-28")
}
}
]
}
},
],
clicks: [
{
$limit: 1
},
{
$lookup: {
from: "click",
let: {},
pipeline: [
{
$match: {
"$and": [
{
"publisher_id": 1
},
{
"created_at": {
$gte: ISODate("2021-11-01"),
$lt: ISODate("2021-11-28")
}
}
]
}
},
],
as: "clicks"
}
},
{
$unwind: "$clicks"
},
{
$replaceRoot: {
newRoot: "$clicks"
}
}
]
}
},
{
$project: {
merged: {
"$concatArrays": [
"$views",
"$clicks"
]
}
}
},
{
$unwind: "$merged"
},
{
$group: {
_id: {
$dateToString: {
format: "%Y/%m/%d",
date: "$merged.created_at"
}
},
count: {
$sum: 1
}
}
}
])
Mongo Playground

How to query an array and retrieve it from MongoDB

Updated:
I have a document on the database that looks like this:
My question is the following:
How can I retrieve the first 10 elements from the friendsArray from database and sort it descending or ascending based on the lastTimestamp value.
I don't want to download all values to my API and then sort them in Python because that is wasting my resources.
I have tried it using this code (Python):
listOfUsers = db.user_relations.find_one({'userId': '123'}, {'friendsArray' : {'$orderBy': {'lastTimestamp': 1}}}).limit(10)
but it just gives me this error pymongo.errors.OperationFailure: Unknown expression $orderBy
Any answer at this point would be really helpful! Thank You!
use aggregate
first unwind
then sort according timestap
group by _id to create sorted array
use addfields and filter for getting first 10 item of array
db.collection.aggregate([
{ $match:{userId:"123"}},
{
"$unwind": "$friendsArray"
},
{
$sort: {
"friendsArray.lastTimeStamp": 1
}
},
{
$group: {
_id: "$_id",
friendsArray: {
$push: "$friendsArray"
}
},
},
{
$addFields: {
friendsArray: {
$filter: {
input: "$friendsArray",
as: "z",
cond: {
$lt: [
{
$indexOfArray: [
"$friendsArray",
"$$z"
]
},
10
]
}// 10 is n first item
}
}
},
}
])
https://mongoplayground.net/p/2Usk5sRY2L2
and for pagination use this
db.collection.aggregate([
{ $match:{userId:"123"}},
{
"$unwind": "$friendsArray"
},
{
$sort: {
"friendsArray.lastTimeStamp": 1
}
},
{
$group: {
_id: "$_id",
friendsArray: {
$push: "$friendsArray"
}
},
},
{
$addFields: {
friendsArray: {
$filter: {
input: "$friendsArray",
as: "z",
cond: {
$and: [
{
$gt: [
{
$indexOfArray: [
"$friendsArray",
"$$z"
]
},
10
]
},
{
$lt: [
{
$indexOfArray: [
"$friendsArray",
"$$z"
]
},
20
]
},
]
}// 10 is n first item
}
}
},
}
])
The translation of your find to aggregation(we need unwind that why aggregation is used) would be like the bellow query.
Test code here
Query (for descending replace 1 with -1)
db.collection.aggregate([
{
"$match": {
"userId": "123"
}
},
{
"$unwind": {
"path": "$friendsArray"
}
},
{
"$sort": {
"friendsArray.lastTimeStamp": 1
}
},
{
"$limit": 10
},
{
"$replaceRoot": {
"newRoot": "$friendsArray"
}
}
])
If you want to skip some before limit add one stage also
{
"$skip" : 10
}
To take the 10-20 messages for example.

How to find prev/next document after sort in MongoDB

I want to find prev/next blog documents whose publish date is closest to the input document.
Below is the document structure.
Collection Examples (blog)
{
blogCode: "B0001",
publishDate: "2020-09-21"
},
{
blogCode: "B0002",
publishDate: "2020-09-22"
},
{
blogCode: "B0003",
publishDate: "2020-09-13"
},
{
blogCode: "B0004",
publishDate: "2020-09-24"
},
{
blogCode: "B0005",
publishDate: "2020-09-05"
}
If the input is blogCode = B0003
Expected output
{
blogCode: "B0005",
publishDate: "2020-09-05"
},
{
blogCode: "B0001",
publishDate: "2020-09-21"
}
How could I get the output result? In sql, it seems using ROW_NUMBER can solve my problem, however I can't find a solution to achieve the feature in MongoDB. The alternate solution may be reference to this answer (But, it seems inefficient). Maybe using mapReduce is another better solutions? I'm confused at the moment, please give me some help.
You can go like following.
We need to compare existing date with given date. So I used $facet to categorize both dates
The original data should be one Eg : B0003. So that I just get the first element of the origin[] array to compare with rest[] array
used $unwind to flat the rest[]
Substract to get the different between both dates
Again used $facet to find previous and next dates.
Then combined both to get your expected result
NOTE : The final array may have 0<elements<=2. The expected result given by you will not find out whether its a prev or next date if there is a one element. So my suggestion is add another field to say which date it is as the mongo playground shows
[{
$facet: {
origin: [{
$match: { blogCode: 'B0001' }
}],
rest: [{
$match: {
$expr: {
$ne: ['$blogCode','B0001']
}
}
}]
}
}, {
$project: {
origin: {
$arrayElemAt: ['$origin',0]
},
rest: 1
}
}, {
$unwind: {path: '$rest'}
}, {
$project: {
diff: {
$subtract: [{ $toDate: '$rest.publishDate' },{ $toDate: '$origin.publishDate'}]
},
rest: 1,
origin: 1
}
}, {
$facet: {
prev: [{
$sort: {diff: -1}
},
{
$match: {
diff: {$lt: 0 }
}
},
{
$limit: 1
},
{
$addFields:{"rest.type":"PREV"}
}
],
next: [{
$sort: { diff: 1 }
},
{
$match: {
diff: { $gt: 0 }
}
},
{
$limit: 1
},
{
$addFields:{"rest.type":"NEXT"}
}
]
}
}, {
$project: {
combined: {
$concatArrays: ["$prev", "$next"]
}
}
}, {
$unwind: {
path: "$combined"
}
}, {
$replaceRoot: {
newRoot: "$combined.rest"
}
}]
Working Mongo playground
Inspire for the solution of varman proposed. I also find another way to solve my problem by using includeArrayIndex.
[
{
$sort: {
"publishDate": 1
},
},
{
$group: {
_id: 1,
root: {
$push: "$$ROOT"
}
},
},
{
$unwind: {
path: "$root",
includeArrayIndex: "rownum"
}
},
{
$replaceRoot: {
newRoot: {
$mergeObjects: [
"$root",
{
rownum: "$rownum"
}
]
}
}
},
{
$facet: {
currRow: [
{
$match: {
blogCode: "B0004"
},
},
{
$project: {
rownum: 1
}
}
],
root: [
{
$match: {
blogCode: {
$exists: true
}
}
},
]
}
},
{
$project: {
currRow: {
$arrayElemAt: [
"$currRow",
0
]
},
root: 1
}
},
{
$project: {
rownum: {
prev: {
$add: [
"$currRow.rownum",
-1
]
},
next: {
$add: [
"$currRow.rownum",
1
]
}
},
root: 1
}
},
{
$unwind: "$root"
},
{
$facet: {
prev: [
{
$match: {
$expr: {
$eq: [
"$root.rownum",
"$rownum.prev"
]
}
}
},
{
$replaceRoot: {
newRoot: "$root"
}
}
],
next: [
{
$match: {
$expr: {
$eq: [
"$root.rownum",
"$rownum.next"
]
}
}
},
{
$replaceRoot: {
newRoot: "$root"
}
}
],
}
},
{
$project: {
prev: {
$arrayElemAt: [
"$prev",
0
]
},
next: {
$arrayElemAt: [
"$next",
0
]
},
}
},
]
Working Mongo playground

How to know that aggregated group has previous/next values?

Suppose I have the following aggregation pipeline:
db.getCollection('posts').aggregate([
{ $match: { _id: { $gt: "some id" }, tag: 'some tag' } },
{ $limit: 5 },
{ $group: { _id: null, hasNextPage: {??}, hasPreviousPage: {??} } }
])
As a result $match and $limit stages would result in a subset of all the posts with a tag some tag. How can I know that there're posts before and after my subSet?
One of the possible ways, I guess, is to have expression (with $let) inside hasPreviousPage and hasNextPage that would search for one post with _id less than "some id" and greater than $last: "$_id"respectively. But I'm not sure how I can reference my group as a variable in $let. Also, maybe there're some other more effective ways.
You can use below aggregation:
db.posts.aggregate([
{ $match: { tag: 'some tag' } },
{ $sort: { _id: 1 } },
{
$facet: {
data: [
{ $match: { _id: { $gt: 'some id' } } },
{ $limit: 5 }
],
hasPreviousPage: [
{ $match: { _id: { $lte: 'some id' } } },
{ $count: "totalPrev" }
],
hasNextPage: [
{ $match: { _id: { $gt: 'some id' } } },
{ $skip: 5 },
{ $limit: 1 }, // just to check if there's any element
{ $count: "totalNext" }
]
}
},
{
$unwind: { path: "$hasPreviousPage", preserveNullAndEmptyArrays: true }
},
{
$unwind: { path: "$hasNextPage", preserveNullAndEmptyArrays: true }
},
{
$project: {
data: 1,
hasPreviousPage: { $gt: [ "$hasPreviousPage.totalPrev", 0 ] },
hasNextPage: { $gt: [ "$hasNextPage.totalNext", 0 ] }
}
}
])
To apply any paging you have to $sort your collection to get results in deterministic order. On a set that's sorted and filtered by tag you can run $facet which allows you to apply multiple subaggregations. Pipelines that are representing previous and nextPage can be ended with $count. Every subaggregation in $facet will return an array so we can run $unwind to get nested document instead of array for hasPreviousPage and hasNextPage. Option preserveNullAndEmptyArrays is required here cause otherwise MongoDB will remove whole document from aggregation pipeline if there are no prev / next documents. In the last step we can just convert subaggregations to boolean values.

how to query mongo a value with $lte and $in?

I have a document like:
{
_id:"1",
archive:[{mark:10},{mark:20},{mark:30},{mark:40}]
},
{
_id:"2",
archive:[{mark:12},{mark:25},{mark:30}]
},
{
_id:"3",
archive:[{mark:15},{mark:18}]
}
....
I want to find document where archive have {mark:30} but less then mark:30.
If I use query:
{archive:{$elemMatch:{ mark:{$lte:30}}}
will get document with 2,3
If I user query:
{archive:{$elemMatch:{ mark:{$lte:30,$in:[30]}}}
will get 1,2.
how to only get document 2?
Thank you!
You could try a query which would satisfy the logic
match documents where archive have { "mark": 30 } AND mark is NOT
greater than 30 (same as less than or equal to 30):
db.collection.find({
"archive.mark": {
"$eq": 30,
"$not": { "$gt": 30 }
}
})
or using the $and operator explicitly as
db.collection.find({
"$and": [
{ "archive.mark": 30 },
{ "archive.mark": { "$not": {
"$gt": 30
} } }
]
})
With $elemMatch
db.getCollection('yourCollection').find(
{
archive: {
$elemMatch: {
$and: [
{ mark: { $eq: 30 }},
{ mark: { $lte: 30 }},
{ mark: { $not: { $gt: 30 } } }
]
}
}
})