I am in the process of moving pagination / filtering away from the client and onto the server.
Data is presented in a table, each column header has a text input where you can type and filter the dataset by what is typed in. This uses a simple indexOf check on the inputted text and the dataset to allow partial matches.
Example table column
{ name: "test 1" }, { name: "test 2" }
The image / data above shows a column in the table. If I were to type in "tes" both results would appear.
let filteredResults = data.filter(row => row.name.toLowerCase().indexOf(filterValue) > -1)
I have now moved this filtering onto the server but I am struggling to work out how to do a similar partial match when querying my data.
This is my query:
aggregate([
{
$facet: {
results: [
{
$match: {
"name": req.body.name
}
},
{
$skip: pageOptions?.pageNo ? (pageOptions.pageNo - 1) * 10 : 0
},
{
$limit: 10
}
],
totalCount: [
{
$match: {
"name": req.body.name
}
},
{ $count: 'totalCount' }
]
}
},
{
$addFields:
{
"total": { $arrayElemAt: ["$totalCount.totalCount", 0] }
}
},
{
$project: {
"totalCount": 0
}
}
]
Each of the fields in the $match stage are possible columns from the table, in this example just the name field. You could filter by more then 1. The above works with exact matches so if we were to search the name column with "test 1" then that record would be returned but if we search for "tes" nothing would be returned.
Any help with this would be great!
You can use a $regex match to perform your case-insensitive, partial string match.
db.collection.aggregate([
{
$match: {
"name": {
// put your query in $regex option
$regex: "tes",
$options: "i"
}
}
},
{
$facet: {
results: [
{
$skip: 0
},
{
$limit: 10
}
],
totalCount: [
{
$count: "totalCount"
}
]
}
},
{
$addFields: {
"total": {
$arrayElemAt: [
"$totalCount.totalCount",
0
]
}
}
},
{
$project: {
"totalCount": 0
}
}
])
Here is the Mongo playground for your reference.
I was able to solve this by using text indexes and the $text operator:
[
{
$match: { $text: { $search: "asdfadsf" } }
}
]
Related
I have two collections - "users" and "follows". "Follows" simply contains documents with a "follower" field and a "followee" field that represent when a user follows another user. What I want to do is to be able to query the users but display the users that I (or whatever user is making the request) follow first. For example if I follow users "5" and "14", when I search the list of users, I want users "5" and "14" to be at the top of the list, followed by the rest of the users in the database.
If I were to first query all the users that I follow from the "Follows" collection and get an array of those userIDs, is there a way that I can sort by using something like {$in: [userIDs]}? I don't want to filter out the users that I do not follow, I simply want to sort the list by showing the users that I do follow first.
I am using nodejs and mongoose for this.
Any help would be greatly appreciated. Thank you!
Answer
db.users.aggregate([
{
$addFields: {
sortBy: {
$cond: {
if: {
$in: [ "$_id", [ 5, 14 ] ]
},
then: 0,
else: 1
}
}
}
},
{
$sort: {
sortBy: 1
}
},
{
$unset: "sortBy"
}
])
Test Here
If you don't want you on the list, then
db.users.aggregate([
{
$addFields: {
sortBy: {
$cond: {
if: {
$in: [ "$_id", [ 5, 14 ] ]
},
then: 0,
else: 1
}
}
}
},
{
$sort: {
sortBy: 1
}
},
{
$unset: "sortBy"
},
{
$match: {
"_id": { $ne: 1 }
}
}
])
Test Here
If you want to sort users first
db.users.aggregate([
{
$sort: {
_id: 1
}
},
{
$addFields: {
sortBy: {
$cond: {
if: {
$in: [
"$_id",
[
5,
14
]
]
},
then: 0,
else: 1
}
}
}
},
{
$sort: {
sortBy: 1,
}
},
{
$unset: "sortBy"
},
{
$match: {
"_id": {
$ne: 1
}
}
}
])
Test Here
Given this aggregation pipeline:
[
{
$addFields: {
_myVar: "x"
}
},
{
$match: {
array: "x"
}
}
]
How can the field with value x only be set once?
For example, this does not work, it times out:
[
{
$addFields: {
_myVar: "x"
}
},
{
$match: {
$expr: {
$in: [
"$_myVar", "$array"
]
}
}
}
]
The variable needs to be available throughout the pipeline, so only using the value in the $match stage as condition is not a solution.
What is the solution?
You can do something like this here i added two fields and checking if _myArray has _myVar, this is just to explain how can you check... in your case you have to replace _myArray with your actual array against which you want t to match
[{
$addFields: {
_myVar: "x",
_myArray: ['X', 'Y', 'x']
}
}, {
$addFields: {
has: {
$in: ["$_myVar", "$_myArray"]
}
}
}, {
$match: {
has: true
}
}]
I am trying to aggregate poems-collection. Each poem has "lines" files which is array of lines like
lines: [
{
id: '123'
text: 'ABC'
},
{
id: '567'
text: 'AKA'
},
{
id: '890'
text: 'ZXZ'
}
...
]
db.getCollection('poems').aggregate([
{
$match: {
"languageId": "en",
"published": { $exists: true, $ne: false }
}
},
{
$group: {
_id: {
"userId": "$userId"
},
"lastPoem": {
$last: "$$ROOT" // take just last document alternatives $first or $push (all)
},
"count": {
$sum: 1
}
}
},
{ "$sort": { 'lastPoem.publishedDate': -1 } },
{ "$skip": 0 },
{ "$limit": 10 }
])
I need to slice number of "lines" to 5 for example.
How do I use slice in this case with aggregation?
I tried to put different places, but did not get it to work.
{ "lastPoem.lines": { "$slice": [ "$lines", 10 ] } }
Thank you!
The lines field is inside lastPoem it should $lastPoem.lines and you have used just $lines in $slice,
$addFields after $group stage and before $sort stage
{
$addFields: {
"lastPoem.lines": {
$slice: ["$lastPoem.lines", 5]
}
}
}
Playground
I'm doing paginated search with mongoDb in my Springboot API.
For a customer search path, I'm building a query with a bunch of criteria depending on the user input.
I then do a count to display the total number of results (and the computed number of page associated)
Long total = mongoTemplate.count(query, MyEntity.class);
I then do the paginated query to return only current page results
query.with(PageRequest.of(pagination.getPage(), pagination.getPageSize()));
query.with(Sort.by(Sort.Direction.DESC, "creationDate"));
List<MyEntity> listResults = mongoTemplate.find(query, MyEntity.class);
It all works well.
Now on my total results, i often have multiple result for the same users, I want to display those in the paginated list, but I also want to display a new counter with the total distinct user that are in that search.
I saw the findDistinct parameter
mongoTemplate.findDistinct(query, "userId", OnboardingItineraryEntity.class, String.class);
But I do not want to retrieve a huge list and do a count on it. Is there a way to easily do:
mongoTemplate.countDistinct(query, "userId", OnboardingItineraryEntity.class, String.class);
Cause I've a huge number of criteria, so i find it sad to have to rebuild an Aggregate object from scratch ?
Bonus question, sometime userId will be null, Is there an easy way do count number of distinct (not null) + number of null in one query?
Or do I need to do a query, when i add an extra criteira on userId being null, do a count on that, and then do the count distinct on all and add them up manualy in my code (minus one).
MongoDB aggregation solves this problem in several ways.
Aggregate with $type operator:
db.myEntity.aggregate([
{$match:...}, //add here MatchOperation
{
"$group": {
"_id": {
"$type": "$userId"
},
"count": {
"$sum": 1
}
}
}
])
MongoPlayground
---Ouput---
[
{
"_id": "null", //null values
"count": 2
},
{
"_id": "missing", // if userId doesn't exists at all
"count": 1
},
{
"_id": "string", //not null values
"count": 4
}
]
Single document with null and NonNull fields
db.myEntity.aggregate([
{$match:...}, //add here MatchOperation
{
"$group": {
"_id": "",
"null": {
$sum: {
$cond: [
{
$ne: [{ "$type": "$userId"}, "string"]
},
1,
0
]
}
},
"nonNull": {
"$sum": {
$cond: [
{
$eq: [{ "$type": "$userId" }, "string"]
},
1,
0
]
}
}
}
}
])
MongoPlayground
---Output---
[
{
"_id": "",
"nonNull": 4,
"null": 3
}
]
Performing $facet operator
db.myEntity.aggregate([
{$match:...}, //add here MatchOperation
{
$facet: {
"null": [
{
$match: {
$or: [
{
userId: {
$exists: false
}
},
{
userId: null
}
]
}
},
{
$count: "count"
}
],
"nonNull": [
{
$match: {
$and: [
{
userId: {
$exists: true
}
},
{
userId: {
$ne: null
}
}
]
}
},
{
$count: "count"
}
]
}
},
{
$project: {
"null": {
$ifNull: [
{
$arrayElemAt: [
"$null.count",
0
]
},
0
]
},
"nonNull": {
$ifNull: [
{
$arrayElemAt: [
"$nonNull.count",
0
]
},
0
]
}
}
}
])
MongoPlayground
Note: Try any of these solutions and let me know if you have any problem creating the MongoDB aggregation.
Suppose I have the following aggregation pipeline:
db.getCollection('posts').aggregate([
{ $match: { _id: { $gt: "some id" }, tag: 'some tag' } },
{ $limit: 5 },
{ $group: { _id: null, hasNextPage: {??}, hasPreviousPage: {??} } }
])
As a result $match and $limit stages would result in a subset of all the posts with a tag some tag. How can I know that there're posts before and after my subSet?
One of the possible ways, I guess, is to have expression (with $let) inside hasPreviousPage and hasNextPage that would search for one post with _id less than "some id" and greater than $last: "$_id"respectively. But I'm not sure how I can reference my group as a variable in $let. Also, maybe there're some other more effective ways.
You can use below aggregation:
db.posts.aggregate([
{ $match: { tag: 'some tag' } },
{ $sort: { _id: 1 } },
{
$facet: {
data: [
{ $match: { _id: { $gt: 'some id' } } },
{ $limit: 5 }
],
hasPreviousPage: [
{ $match: { _id: { $lte: 'some id' } } },
{ $count: "totalPrev" }
],
hasNextPage: [
{ $match: { _id: { $gt: 'some id' } } },
{ $skip: 5 },
{ $limit: 1 }, // just to check if there's any element
{ $count: "totalNext" }
]
}
},
{
$unwind: { path: "$hasPreviousPage", preserveNullAndEmptyArrays: true }
},
{
$unwind: { path: "$hasNextPage", preserveNullAndEmptyArrays: true }
},
{
$project: {
data: 1,
hasPreviousPage: { $gt: [ "$hasPreviousPage.totalPrev", 0 ] },
hasNextPage: { $gt: [ "$hasNextPage.totalNext", 0 ] }
}
}
])
To apply any paging you have to $sort your collection to get results in deterministic order. On a set that's sorted and filtered by tag you can run $facet which allows you to apply multiple subaggregations. Pipelines that are representing previous and nextPage can be ended with $count. Every subaggregation in $facet will return an array so we can run $unwind to get nested document instead of array for hasPreviousPage and hasNextPage. Option preserveNullAndEmptyArrays is required here cause otherwise MongoDB will remove whole document from aggregation pipeline if there are no prev / next documents. In the last step we can just convert subaggregations to boolean values.