Aggregate documents where objects in array matches multiple conditions - mongodb

I have a collection with documents similar to such:
{
"_id": ObjectId("xxxxx"),
"item": [
{ "property": ["attr1", "+1"] },
{ "property": ["attr2", "-1"] }
]
}
{
"_id": ObjectId("xxxxy"),
"item": [
{ "property": ["attr1", "-1"] },
{ "property": ["attr2", "0"] }
]
}
{
"_id": ObjectId("xxxxz"),
"item": [
{ "property": ["attr1", "0"] },
{ "property": ["attr2", "+1"] }
]
}
Preferably using an aggregation pipeline, is there any way to match the document if and only if any one of the properties match more than one condition?
For example, I want a query where one object in the array matches both of these conditions:
("item.property": "attr1") AND ("item.property": /^\+/)
That is, a single property where it contains "attr1" and an element that starts with "+".
However, using my current query that looks like this:
collection.aggregate(
{ $match:
{ $and:
[
{ "item.property": "attr1" },
{ "item.property": /^\+/ }
]
}
}
This would match both the first and last document because both contain a property with "attr1" and an element that stats with "+". However, I do not want this query to match the last document, since the element that starts with "+" does not belong to the same object in the array.
Is there any way to achieve this behavior using the aggregation framework?

You can use the below query with $elemMatch to match the array's both values.
Something like
db.collection_name.aggregate({
"$match": {
"item": {
"$elemMatch": {
"property.0": "attr1",
"property.1": /^\+/
}
}
}
});
Also, you can use $all operator if you don't want to reference array index.
db.collection_name.aggregate({
"$match": {
"item": {
"$elemMatch": {
"property": {
"$all": [
"attr1",
/^\+/
]
}
}
}
}
});

Related

MongoDB Rust Driver weird behavior

There is this weird thing,
I have installed the MongoDB Compass and made a aggregation query that works in the Aggregation tab but now when I use the same query in my rust web server it behaves very weirdly
Original message:
{"_id":{"$oid":"61efd41c56ffe6b1b4a15c7a"},"time":{"$date":"2022-01-25T10:42:36.175Z"},"edited_time":{"$date":"2022-01-30T14:29:54.361Z"},"changes":[],"content":"LORA","author":{"$oid":"61df3cab3087579f8767a38d"}}
Message in MongoDB compass after the query:
{
"_id": {
"$oid": "61efd41c56ffe6b1b4a15c7a"
},
"time": {
"$date": "2022-01-25T10:42:36.175Z"
},
"edited_time": {
"$date": "2021-12-17T09:55:45.856Z"
},
"changes": [{
"time": {
"$date": "2021-12-17T09:55:45.856Z"
},
"change": {
"ChangedContent": "LORA"
}
}],
"content": "LMAO",
"author": {
"$oid": "61df3cab3087579f8767a38d"
}
}
Message after the Web Servers query:
{
"_id": {
"$oid": "61efd41c56ffe6b1b4a15c7a"
},
"time": {
"$date": "2022-01-25T10:42:36.175Z"
},
"edited_time": {
"$date": "2022-01-30T14:40:57.152Z"
},
"changes": {
"$concatArrays": ["$changes", [{
"time": {
"$date": "2022-01-30T14:40:57.152Z"
},
"change": {
"ChangedContent": "$content"
}
}]]
},
"content": "LMAO",
"author": {
"$oid": "61df3cab3087579f8767a38d"
}
}
Pure query in MongoDB Compass:
$set stage
{
"changes": { $concatArrays: [ "$changes", [ { "time": ISODate('2021-12-17T09:55:45.856+00:00'), "change": { "ChangedContent": "$content" } } ] ] },
"edited_time": ISODate('2021-12-17T09:55:45.856+00:00'),
"content": "LMAO",
}
Pure query in Web Server:
let update_doc = doc! {
"$set": {
"changes": {
"$concatArrays": [
"$changes", [
{
"time": now,
"change": {
"ChangedContent": "$content"
}
}
]
]
},
"edited_time": now,
"content": content
}
};
I am using update_one method,
like this
messages.update_one(message_filter, update_doc, None).await?;
I don't really understand, and this happens often, sometimes it fixes it self when I add somewhere randomly some scope in the doc eg.: { } but this time I couldn't figure it out,
I had version of the query with $push but that didn't work too
Is there some fault in the rust driver or am I doing something wrong, are there some rules about formatting when using rust driver that I am missing?
The $set aggregation pipeline stage is different from the $set update operator. And the only difference that I can tell, is the pipeline stage handles $concatArrays while the update operator does not.
$set Aggregation Pipeline Stage
$set appends new fields to existing documents. You can include one or more $set stages in an aggregation operation.
To add field or fields to embedded documents (including documents in arrays) use the dot notation.
To add an element to an existing array field with $set, use with $concatArrays.
$set Update Operator
Starting in MongoDB 5.0, update operators process document fields with
string-based names in lexicographic order. Fields with numeric names
are processed in numeric order.
If the field does not exist, $set will add a new field with the
specified value, provided that the new field does not violate a type
constraint. If you specify a dotted path for a non-existent field,
$set will create the embedded documents as needed to fulfill the
dotted path to the field.
If you specify multiple field-value pairs, $set will update or create
each field.
So if you want to update an existing document by inserting elements into an array field, use the $push update operator (potentially with $each if you're inserting multiple elements):
let update_doc = doc! {
"$set": {
"edited_time": now,
"content": content
},
"$push": {
"changes": {
"time": now,
"change": {
"ChangedContent": "$content"
}
}
}
};
Edit: I missed that $content was supposed to be mapped from the existing field as well. That is not supported by an update document, however MongoDB has support for using an aggregation pipeline to update the document. See: Update MongoDB field using value of another field So you can use the original $set just in a different way:
let update_pipeline = vec![
doc! {
"$set": {
"changes": {
"$concatArrays": [
"$changes", [
{
"time": now,
"change": {
"ChangedContent": "$content"
}
}
]
]
},
"edited_time": now,
"content": content
}
}
];
messages.update_one(message_filter, update_pipeline, None).await?;

don't allow partially matching document to add to array with $push using updateOne function

check a document whether it is partially matching (2 or more fields) with existing documents in array('claims') with $push / $addToSet using updateOne and shouldn't allow to be pushed to array
structure is as follows:
{
"_id": ObjectId("5f50c93dda654e891c903efb"),
"claims":[
{
"username": "Foo",
"claim_id":"12345",
"claimed_at":"2020-08-31T17:18:52.818737",
},
{
"username": "Bar",
"claim_id":"54321",
"claimed_at":"2020-08-31T17:18:52.818737",
}
]
}
couldn't use $addToSet since other field value like claimed_at is not known,
if it was to match single field only like user_id it was working,
I was missing something here
db.claims.updateOne(
{
"_id": ObjectId("5f50c93dda654e891c903efb"),
"claims":{"$ne": {"username":"Foo"}}
},
{
"$push":
{
"claims":
{
"username": "Foo",
"claim_id":"12345",
"claimed_at":"2020-09-02T17:08:23.514342",
}
}
})
You can try using $elemMatch and $not,
db.claims.updateOne({
"_id": ObjectId("5f50c93dda654e891c903efb"),
claims: {
$not: {
$elemMatch: {
username: "Foo",
claim_id: "12345"
}
}
}
}, {
"$push": {
"claims": {
"username": "Foo",
"claim_id": "12345",
"claimed_at": "2020-09-02T17:08:23.514342",
}
}
})

MongoDb aggregation project onto collection

I've a problem with a huge MongoDb aggregation pipeline. I've many constraint and I've simplified the problem a lot. Hence, don't discuss the goal for this query.
I've a mongo aggregation that gives something similar to this:
[
{
"content": {
"processes": [
{
"id": "101a",
"title": "delivery"
},
{
"id": "101b",
"title": "feedback"
}
]
}
}
]
To this intermediate result I'm forced to apply a project operation in order to obtain something similar to this:
[
{
"results":
{
"titles": [
{
"id": "101a",
"value": "delivery"
},
{
"id": "101b",
"value": "feedback"
}
]
}
}
]
enter code here
But applying this projections:
"results.titles.id": "$content.processes.id",
"results.titles.value": "$content.processes.title"
I obtain this:
[
{
"results":
{
"titles": {
"id": ["101a", "101b"]
"value": ["delivery", "feedback"]
}
}
}
}
]
Collection are created but not in the proper position.
Is it possible to exploit some operator inside the project operation in order to tell mongo to create an array in a parent position?
Something like this:
"results.titles.$[x].value" : "$content.processes.value"
You can use the dot notation to project entire array:
db.col.aggregate([
{
$project: {
"results.titles": "$content.processes"
}
}
])
and if you need to rename title to value then you have to apply $map operator:
db.col.aggregate([
{
$project: {
"results.titles": {
$map: {
input: "$content.processes",
as: "process",
in: {
id: "$$process.id",
value: "$$process.title"
}
}
}
}
}
])

Match multiple values in array

I have the following data structure:
[{"url":"http://guyanachronicle.com/","originalUrl":"http://www.guyanachronicle.com","applications":[]}
,{"url":"http://www.lightbot.com","originalUrl":"http://www.lightbot.com","applications":[{"name":"Apache","confidence":"100","version":"","icon":"Apache.svg","website":"http://apache.org","categories":["Web Servers"]},{"name":"Facebook","confidence":"100","version":"","icon":"Facebook.svg","website":"http://facebook.com","categories":["Widgets"]},{"name":"Google Analytics","confidence":"100","version":"","icon":"Google Analytics.svg","website":"http://google.com/analytics","categories":["Analytics"]},{"name":"Twitter","confidence":"100","version":"","icon":"Twitter.svg","website":"http://twitter.com","categories":["Widgets"]},{"name":"jQuery","confidence":"100","version":"1.4","icon":"jQuery.svg","website":"http://jquery.com","categories":["JavaScript Frameworks"]}]}
,{"url":"http://www.fitnessfirst.com.au","originalUrl":"http://www.fitnessfirst.com.au","applications":[{"name":"AdInfinity","confidence":"100","version":"","icon":"AdInfinity.png","website":"http://adinfinity.com.au","categories":["Advertising Networks"]},{"name":"CloudFlare","confidence":"100","version":"","icon":"CloudFlare.svg","website":"http://www.cloudflare.com","categories":["CDN"]},{"name":"Facebook","confidence":"100","version":"","icon":"Facebook.svg","website":"http://facebook.com","categories":["Widgets"]},{"name":"Google Analytics","confidence":"100","version":"UA","icon":"Google Analytics.svg","website":"http://google.com/analytics","categories":["Analytics"]},{"name":"Google Tag Manager","confidence":"100","version":"","icon":"Google Tag Manager.png","website":"http://www.google.com/tagmanager","categories":["Tag Managers"]},{"name":"Microsoft ASP.NET","confidence":"100","version":"4.0.30319","icon":"Microsoft ASP.NET.png","website":"http://www.asp.net","categories":["Web Frameworks"]},{"name":"Modernizr","confidence":"100","version":"","icon":"Modernizr.png","website":"http://www.modernizr.com","categories":["JavaScript Frameworks"]},{"name":"New Relic","confidence":"100","version":"","icon":"New Relic.png","website":"http://newrelic.com","categories":["Analytics"]},{"name":"Nginx","confidence":"100","version":"","icon":"Nginx.svg","website":"http://nginx.org/en","categories":["Web Servers"]},{"name":"OWL Carousel","confidence":"100","version":"","icon":"OWL Carousel.png","website":"https://owlcarousel2.github.io/OwlCarousel2/","categories":["Widgets"]},{"name":"Optimizely","confidence":"100","version":"","icon":"Optimizely.png","website":"http://optimizely.com","categories":["Analytics"]},{"name":"YouTube","confidence":"100","version":"","icon":"YouTube.png","website":"http://www.youtube.com","categories":["Video Players"]},{"name":"jQuery","confidence":"100","version":"1.11.1","icon":"jQuery.svg","website":"http://jquery.com","categories":["JavaScript Frameworks"]},{"name":"IIS","confidence":"100","version":"","icon":"IIS.png","website":"http://www.iis.net","categories":["Web Servers"]},{"name":"Windows Server","confidence":"100","version":"","icon":"Microsoft.svg","website":"http://microsoft.com/windowsserver","categories":["Operating Systems"]}]}
,{"url":"https://www.totaltools.com.au/","originalUrl":"http://www.totaltools.com.au","applications":[{"name":"Facebook","confidence":"100","version":"","icon":"Facebook.svg","website":"http://facebook.com","categories":["Widgets"]},{"name":"Google Analytics","confidence":"100","version":"UA","icon":"Google Analytics.svg","website":"http://google.com/analytics","categories":["Analytics"]},{"name":"Google Tag Manager","confidence":"100","version":"","icon":"Google Tag Manager.png","website":"http://www.google.com/tagmanager","categories":["Tag Managers"]},{"name":"Hotjar","confidence":"100","version":"","icon":"Hotjar.png","website":"https://www.hotjar.com","categories":["Analytics"]},{"name":"Magento","confidence":"100","version":"2","icon":"Magento.png","website":"http://www.magentocommerce.com","categories":["Ecommerce"]},{"name":"Prototype","confidence":"100","version":"","icon":"Prototype.png","website":"http://www.prototypejs.org","categories":["JavaScript Frameworks"]},{"name":"RequireJS","confidence":"100","version":"","icon":"RequireJS.png","website":"http://requirejs.org","categories":["JavaScript Frameworks"]},{"name":"Twitter Bootstrap","confidence":"100","version":"","icon":"Twitter Bootstrap.png","website":"http://getbootstrap.com","categories":["Web Frameworks"]},{"name":"Underscore.js","confidence":"100","version":"","icon":"Underscore.js.png","website":"http://underscorejs.org","categories":["JavaScript Frameworks"]},{"name":"jQuery","confidence":"100","version":"","icon":"jQuery.svg","website":"http://jquery.com","categories":["JavaScript Frameworks"]},{"name":"jQuery Mobile","confidence":"100","version":"","icon":"jQuery Mobile.svg","website":"http://jquerymobile.com","categories":["Mobile Frameworks"]},{"name":"jQuery UI","confidence":"100","version":"","icon":"jQuery UI.svg","website":"http://jqueryui.com","categories":["JavaScript Frameworks"]},{"name":"PHP","confidence":"100","version":"","icon":"PHP.svg","website":"http://php.net","categories":["Programming Languages"]}]}
,{"url":"http://www.bestandless.com.au","originalUrl":"http://www.bestandless.com.au","applications":[]}
,{"url":"http://www.flightsimstore.com","originalUrl":"http://www.flightsimstore.com","applications":[{"name":"Google Analytics","confidence":"100","version":"","icon":"Google Analytics.svg","website":"http://google.com/analytics","categories":["Analytics"]},{"name":"HeadJS","confidence":"50","version":"","icon":"HeadJS.png","website":"http://headjs.com","categories":["JavaScript Frameworks"]},{"name":"Nginx","confidence":"100","version":"1.12.0","icon":"Nginx.svg","website":"http://nginx.org/en","categories":["Web Servers"]},{"name":"PHP","confidence":"100","version":"5.3.29","icon":"PHP.svg","website":"http://php.net","categories":["Programming Languages"]},{"name":"SWFObject","confidence":"100","version":"","icon":"SWFObject.png","website":"http://github.com/swfobject/swfobject","categories":["Miscellaneous"]},{"name":"StatCounter","confidence":"100","version":"","icon":"StatCounter.png","website":"http://www.statcounter.com","categories":["Analytics"]},{"name":"SumoMe","confidence":"100","version":"","icon":"SumoMe.png","website":"http://sumome.com","categories":["Widgets","Marketing Automation"]},{"name":"osCommerce","confidence":"100","version":"","icon":"osCommerce.png","website":"http://www.oscommerce.com","categories":["Ecommerce"]},{"name":"MySQL","confidence":"100","version":"","icon":"MySQL.svg","website":"http://mysql.com","categories":["Databases"]}]}]);
That I uploaded to a collection in MongoDB called websites (it essentially is just a list of URLs and the applications that are running on them)
I use the following query to return all websites who have a certain application installed:
db.websites.find({"applications.name":"osCommerce"}).pretty();
This works and returns a list of websites using "osCommerce" however, when trying to use $and to search for websites that are using multiple applications my query doesn't work.
I'm trying to use:
db.inventory.find( { $and: [ {applications.name:{"Apache"}},
{applications.name: {"osCommerce"}} ] } ).pretty();
With no luck, what is the correct way to do this?
You want $all
db.inventory.find({
"applications.name": { "$all": [ "Apache", "osCommerce" ] }
})
Or combined with with $elemMatch if you want more conditions than just "name" inside the array:
db.inventory.find({
"applications": {
"$all": ]
{ "$elemMatch": { "name": "Apache" } },
{ "$elemMatch": { "name": "osCommerce" } }
]
}
})
Noting though that the data you present in the question has no document that matches this condition.
If you want "either" value then that is an $in query:
db.inventory.find({
"applications.name": { "$in": [ "Apache", "osCommerce" ] }
})
or in fact represented with $or if you wanted "multiple properties" just like above:
db.inventory.find({
"$or": [
{ "applications": { "$elemMatch": { "name": "Apache" } } }
{ "applications": { "$elemMatch": { "name": "osCommerce" } } }
]
})

Are there restrictions on MongoDB collections property names?

I've a document structure wich contains a property named shares which is an array of objects.
Now I tried to match all documents where shared contains the matching _account string with dot notation (shares._account).
It's not working but it seems it's because of the _ char in front of property _account.
So if I put the string to search for inside the name property in that object everything works fine with dot notation.
Are there any limitations on property names?
Thought an _ is allowed because the id has it also in mongodb and for me it's a kind of convention to daclare bindings.
Example:
// Collection Item example
{
"_account": { "$oid" : "526fd2a571e1e13b4100000c" },
"_id": { "$oid" : "5279456932db6adb60000003" },
"name": "shared.jpg",
"path": "/upload/24795-4ui95s.jpg",
"preview": "/img/thumbs/24795-4ui95s.jpg",
"shared": false,
"shares": [
{
"name": "526fcb177675f27140000001",
"_account": "526fcb177675f27140000001"
},
{
"name": "tim",
"_account": "526fd29871e1e13b4100000b"
}
],
"tags": [
"grüngelb",
"farbe"
],
"type": "image/jpeg"
},
I tried to get the item with following query:
// Query example
{
"$or": [
{
"$and": [
{
"type": {
"$in": ["image/jpeg"]
}
}, {
"shares._account": "526fcb177675f27140000001" // Not working
//"shares.name": "526fcb177675f27140000001" // Working
}
]
}
]
}
Apart from the fact that $and can be omitted and $or is pointless "image/jpeg" != "image/jpg":
db.foo.find({
"type": {"$in": ["image/jpeg"]},
"shares._account": "526fcb177675f27140000002"
})
Or if you really want old one:
db.foo.find({
"$or": [
{
"$and": [
{
"type": {
"$in": ["image/jpeg"]
}
}, {
"shares._account": "526fcb177675f27140000002"
}
]
}
]
}
Both will return example document.
Your current query has some unnecessarily complicated constructs:
you don't need the $or and $and clauses ("and" is the default behaviour)
you are matching a single value using $in
The query won't find match the sample document because your query doesn't match the data:
the type field you are looking for is "image/jpg" but your sample document has "image/jpeg"
the shares._account value you are looking for is "526fcb177675f27140000001" but your sample document doesn't include this.
A simplified query that should work is:
db.shares.find(
{
"type": "image/jpeg",
"shares._account": "526fcb177675f27140000002"
}
)