regex as $filter in projection - mongodb

I'm trying to find (using a regexp) an array field and returns that element only
This is my data
[
{
"_id": "56d6e8bbf7404bd80a017edb",
"name": "document1",
"tags": [
"A area1",
"B area2",
"C area3"
]
},
{
"_id": "56d6e8bbf7404bd82d017ede",
"name": "document2",
"tags": [
"b_area3",
"b_area4",
"b_area5"
]
}
]
My query
var query=new RegExp('^'+string, "i");
Model.find({tags:query},{'tags.$': 1}, function(err,data){
if(err) console.log(err);
res.json(data);
});
This query selects only the tags field (as I want), but selects the first element. I need the element(s) that matches the query.
EDIT: I tried the mongodb aggregate too, the $filter cond is wrong. I get the error "MongoError: invalid operator $regex"
caseNote.aggregate([
{ $match: {tags:query}},
{ $project: {
tags: {$filter: {
input: 'tags',
as: 'item',
cond: {$regex: ['$$item', query]}
}}
}}
], function (err, result) {
if (err) {
console.log(err);
} else {
res.json(result);
}
});
EDIT2: on #zangw suggestion, this is the mongoose version, but it's incomplete: the tags fields is fine (needs test), but the query still returns the whole document.
caseNote
.aggregate({ $match: {tags: {$in:['area3']}}})
.unwind('tags')
.exec(function(err,d){
res.json(d);
});

According this issue Use $regex as the expression in a $cond, the $regex could NOT be used with cond for current mongo version.
Maybe you can try this one, filter the area3 through $match, then get all matched tags through $group, then remove the _id through $project.
caseNote.aggregate([{$unwind: '$tags'},
{$match: {tags: /area3/}},
{$group: {_id: null, tags: {$push: '$tags'}}},
{$project: {tags: 1, _id: 0}}])
.exec(function(err, tags) {
if (err)
console.log(err);
else
console.log(tags);
});
Results:
{ "tags" : [ "C area3", "b_area3" ] }

As per #zangw's answer,
this ISSUE-SERVER-8892, According this issue Use $regex as the expression in a $cond, the $regex could NOT be used with cond for current mongo version.
MongoDB v4.1.11 has launched new features in ISSUE-SERVER-11947, This features adds three new expressions $regexFind, $regexFindAll and $regexMatch to the aggregation language.
In your example, You can use $regexMatch expression,
Model.aggregate([
{
$project: {
tags: {
$filter: {
input: "$tags",
cond: {
$regexMatch: {
input: "$$this",
regex: query
}
}
}
}
}
}
])
Playground

This is how I solved it. If the query can be parsed to regex the projection is not added to aggregation, but happening after the db request. If the query string is normal string, the projection added.
const { query } = req; // /rea/ or 'C area3'
const convertIfRegex = string => {
const parts = string.split('/')
let regex = string;
let options = '';
if (parts.length > 1) {
regex = parts[1];
options = parts[2];
} else {
return false
}
try {
return new RegExp(regex, options);
} catch (e) {
return null
}
};
const regex = convertIfRegex(queryString);
const aggregations = [{ $match: { tags:query } }]
if (!regex) {
aggregations.push({
$project: {
tags: {$filter: {
input: 'tags',
as: 'item',
cond: {$eq: ['$$item', query]}
}}
}
})
}
let result = await caseNote.aggregate(aggregations);
if (regex) {
result = result.reduce((acc, entry) => {
const tags = entry.tags.filter(({ tag }) => (
tag.match(regex)
))
if (tags.length) return [...acc, { ...entry, tags }];
return acc;
})
}
res.json(result)

Related

Mongodb find document in collection from field in another collection

I have two collections: Sharing and Material.
Sharing:
{
from_id: 2
to_id: 1
material_id: material1
}
Material:
{
_id: material1
organization_id: 2
},
{
_id: material2
organization_id: 1
},
{
_id: material3
organization_id: 1
},
--Edit:
There are three materials, 2 belong to organization_id(1) and 1 belongs to organization_id(2). The organization_id does not match 1 in material1 (and instead belongs to material2), but in the Sharing collection, the to_id does match 1. If the match exists, I'd like to find the Material document _id which is equal to the material_id of Sharing AND find the Material documents where the organization_id is equal to 1.
I'd like to check if a field in Sharing (to_id) has a value that is equal to a field in Material (organization_id) AND check if organization_id is equal to 1. If there is a document that exists from this, do another check to find whether the _id of Material is equal to the material_id of Sharing and return all documents & the total count.
If there is no equal value, I'd like to omit that result and send the object with only organization_id equal to 1 and get the total count of this result.
Right now, I do it in a very inefficient way using .map() to find this. Below is my code:
export const getMaterials = async (req, res) => {
const sharing = await Sharing.find({to_id: 1});
let doneLoad;
try {
if (sharing && sharing.length>0) {
const sharingTotal = await Material.find( {$or: [ {organization_id: 1}, {_id: sharing.map((item) => item.material_id)} ] } ).countDocuments();
const sharingMats = await Material.find( {$or: [ {organization_id: 1}, {_id: sharing.map((item) => item.material_id)} ] } );
res.status(200).json({data: sharingMats});
doneLoad= true;
}
else if (!doneLoad) {
const materialTotal = await Material.find({organization_id: 1}).countDocuments();
const materials = await Material.find({organization_id: 1});
res.status(200).json({data: materials});
}
} catch (error) {
res.status(404).json({ message: error.message });
}
}
I have tried using aggregation to get my desired result but I cannot find any solution that fits my requirements. Any help would be great as I am quite new to using mongodb. Thanks.
Edit (desired result):
Materials: [
{
_id: material1,
organization_id: 1
},
{
_id: material2,
organization_id: 1
},
{
_id: material3,
organization_id: 1
}
]
You can use sub-pipeline in a $lookup to perform the filtering. $addFields the count using $size later.
db.Sharing.aggregate([
{
"$match": {
to_id: 1
}
},
{
"$lookup": {
"from": "Material",
"let": {
to_id: "$to_id",
material_id: "$material_id"
},
"pipeline": [
{
"$match": {
$expr: {
$or: [
{
$eq: [
"$$to_id",
"$organization_id"
]
},
{
$eq: [
"$$material_id",
"$_id"
]
}
]
}
}
},
{
"$addFields": {
"organization_id": 1
}
}
],
"as": "materialLookup"
}
},
{
"$addFields": {
"materialCount": {
$size: "$materialLookup"
}
}
}
])
Here is the Mongo playground for your reference.

MongoDB aggregation - operator to read in documents

Since Mongo only supports one $text field per aggregation pipeline (inside the first $match stage), that means you can't perform a logical AND, since you can't $and the results of multiple $text searches.
// Fails due to "too many text expressions"
db.Employees.aggregate([
{$match: {$and: [
{$text: {$search: "senior"}},
{$text: {$search: "manager"}}
]}}
])
Therefore I need to perform multiple separate $text searches, combine the results in my NodeJS code, and pass that result set back into an aggregation pipeline for further processing (e.g. $addFields, $match, $sort).
Is there a way to do something like...
let results1 = db.Employees.find({"$text":{"$search":"senior"}}, {"score":{"$meta":"textScore"}})
let results2 = db.Employees.find({"$text":{"$search":"manager"}}, {"score":{"$meta":"textScore"}})
let combinedResults = _.intersectionWith(results1, results2, _.isEqual)
let finalResults = /* pass combinedResults into aggregation pipeline and execute it */
Something like the opposite of the $out operator, where I'm reading in a result set instead.
I'm using NestJS and Mongoose if that helps.
There are restrictions in $text, that you already know,
There is a option if you have limited fields, using $regexMatch, I am not sure, in how many fields you have text index, but with this you can combine match conditions with $and operator for multiple fields,
Example Data:
[
{ _id: 1, f1: "senior", f2: "manager" },
{ _id: 2, f1: "junior", f2: "manager" },
{ _id: 3, f1: "fresher", f2: "developer" },
{ _id: 4, f1: "manager", f2: "senior" }
]
Aggregation Query 1:
$addFields to add new field matchResult for matching status in boolean
db.collection.aggregate([
{
$addFields: {
matchResult: {
$and: [
first $or condition match if f1 or f2 fields match senior then return true other wise return false
{
$or: [
{ $regexMatch: { input: "$f1", regex: "senior", options: "x" } },
{ $regexMatch: { input: "$f2", regex: "senior", options: "x" } }
]
},
second $or condition match if f1 or f2 fields match manager then return true other wise return false
{
$or: [
{ $regexMatch: { input: "$f1", regex: "manager", options: "x" } },
{ $regexMatch: { input: "$f2", regex: "manager", options: "x" } }
]
}
]
}
}
},
$match condition return result that have matchResult is equal to true
{ $match: { matchResult: true } }
])
Playground
Aggregation Query 2:
if you are not using array fields then this is sort way, directly you can concat all fields on one field, here i have merged f1 and f2 with space in allField
db.collection.aggregate([
{
$addFields: {
allField: { $concat: ["$f1", " ", "$f2"] }
}
},
this will match $and condition on both word match if both true then return true otherwise false
{
$addFields: {
matchResult: {
$and: [
{ $regexMatch: { input: "$allField", regex: "senior", options: "x" } },
{ $regexMatch: { input: "$allField", regex: "manager", options: "x" } }
]
}
}
},
$match condition return result that have matchResult is equal to true
{ $match: { matchResult: true } }
])
Playground
Note: This is alternate approach for limited fields, but imaging if more then 5 fields then it affects speed and performance of the query.

Aggregate and reduce a nested array based upon an ObjectId

I have an Event document structured like so and I'm trying to query against the employeeResponses array to gather all responses (which may or may not exist) for a single employee:
[
{
...
eventDate: 2019-10-08T03:30:15.000+00:00,
employeeResponses: [
{
_id:"5d978d372f263f41cc624727",
response: "Available to work.",
notes: ""
},
...etc
];
}
];
My current mongoose aggregation is:
const eventResponses = await Event.aggregate([
{
// find all events for a selected month
$match: {
eventDate: {
$gte: startOfMonth,
$lte: endOfMonth,
},
},
},
{
// unwind the employeeResponses array
$unwind: {
path: "$employeeResponses",
preserveNullAndEmptyArrays: true,
},
},
{
$group: {
_id: null,
responses: {
$push: {
// if a response id matches the employee's id, then
// include their response; otherwise, it's a "No response."
$cond: [
{ $eq: ["$employeeResponses._id", existingMember._id] },
"$employeeResponses.response",
"No response.",
],
},
},
},
},
{ $project: { _id: 0, responses: 1 } },
]);
As you'll no doubt notice, the query above won't work after more than 1 employee records a response because it treats each individual response as a T/F condition, instead of all of the responses within the employeeResponses array as a single T/F condition.
As a result, I had remove all subsequent queries after the initial $match and do a manual reduce:
const responses = eventResponses.reduce((acc, { employeeResponses }) => {
const foundResponse = employeeResponses.find(response => response._id.equals(existingMember._id));
return [...acc, foundResponse ? foundResponse.response : "No response."];
}, []);
I was wondering if it's possible to achieve the same reduce result above, but perhaps using mongo's $reduce function? Or refactor the aggregation query above to treat all responses within the employeeResponses as a single T/F condition?
The ultimate goal of this aggregation is extract any previously recorded employee's responses and/or lack of a response from each found Event within a current month and place their responses into a single array:
["I want to work.", "Available to work.", "Not available to work.", "No response.", "No response." ...etc]
You can use $filter with $map to reshape your data and filter by _id. Then you can keep using $push with $ifNull to provide default value if an array is empty:
db.collection.aggregate([
{
$addFields: {
employeeResponses: {
$map: {
input: {
$filter: {
input: "$employeeResponses",
cond: {
$eq: [ "$$this._id", "5d978d372f263f41cc624727"]
}
}
},
in: "$$this.response"
}
}
}
},
{
$group: {
_id: null,
responses: { $push: { $ifNull: [ { $arrayElemAt: [ "$employeeResponses", 0 ] }, "No response" ] } }
}
}
])
Mongo Playground

$inc not working mongoDB

I have a MongoDB data like this
{
_id: "5aa8f087e1eee70004a99e1d"
users: [{facebookId: "-1", unread: 0},{facebookId: "323232321", unread: 1}]
}
I want to increment users.unread where facebookId in not "-1"
I tried this query
chat.update({ "_id": { "$in": chatId }, "users.facebookId": { "$ne": "-1" } },{"$inc": { "users.$.unread": 1 } }, { multi: true, upsert: true }, function(err, data) {
if (err) throw err;
console.log(data);
});
From MongoDB docs:
If the query matches the array using a negation operator, such as $ne, $not, or $nin, then you cannot use the positional operator to update values from this array.
However, if the negated portion of the query is inside of an $elemMatch expression, then you can use the positional operator to update this field
Below code should work if you want to increment first matching element:
db.chat.update({ "_id":"5aa8f087e1eee70004a99e1d", "users": { $elemMatch: { "facebookId": { "$ne": "-1" } }}},{"$inc": { "users.$.unread": 1 }})
Otherwise you can use array filters in MongoDB 3.6
db.col.update({ "_id":"5aa8f087e1eee70004a99e1d" }, { "$inc" : { "users.$[cond].unread" : 1 } }, { arrayFilters: [ { "cond.facebookId": { $ne: "-1" } } ] })

mongoDb $in with aggregate query

How do I write an $in query along with aggregate in mongoDB? I want to write an equivalent mongoDB query for the SQL below
SELECT name,count(something) from collection1
where name in (<<list of Array>>) and cond1 = 'false'
group by name
The equivalent mongo query follows:
db.collection1.aggregate([
{ "$match": {
"name": { "$in": arrayList },
"cond1": "false"
} },
{ "$group": {
"_id": "$name",
"count": { "$sum": "$something" }
} }
])
Suppose you have an Schema with field tags
{
tags: ['banana', 'apple', 'orange']
}
and you want find out apple inside tags with aggregate function then
const keywords = "apple"; // req.body.keywords
const filter = { $match : { tags: {$in : [keywords] } }}
Schema.aggregate(filter).then(result=>{
// You can do what you want with result
})