After spending several hours trying to solve this, and not finding my answer in the docs of on StackOverflow, I'm opeing a question here.
I have a large collection (3.5M docucuments) and want to filter out those that match on a specific combination of sub fields.
E.g. the documents look like:
{
_id:...,
...<a number of fields>
"ML":[
{
"_id": ...,
... <more fields>
"Op": [
"_id": ...,
"Pr": {
"P94": <number>,
"P95" : ...,
...,
"P145": <optional and number>
}
{...},
...
],
{...},
...
],
...
}
So P145 is sometimes there, sometimes not.
I want to find al documents that have a "ML.Op.Pr" with both "P94":8 and P145 exists.
I've tried and failed (as I get no/0 results):
.find({"ML.Op.Pr":{"P94":8,"P145":1})
.find({"ML.Op.Pr":{$and[{"P94":8},{"P145":1}]}})
I've also tried $and as a first step,
.find({$and[{"ML.Op.Pr.P94":8},{"ML.Op.Pr.P145":1}]})
but since both ML and Op are an array with multiple entries, it returns too many results. I need both Pr's to be set in the same array element.
As you can see I'm first trying to find where P145 = 1, because when I replace it with $exists it doesn't parse at all.
How should I do this?
You have to use nested $elemMatch for each sub-array divisions in order to get the desired result.
db.collection.find({
"ML": {
"$elemMatch": {
"Op": {
"$elemMatch": {
"Pr.P94": 8,
"Pr.P145": {
"$exists": true
},
}
}
}
}
})
Mongo Playground sample execution
Related
I am fairly new to MongoDB and cant seem to find a solution to this problem.
I have a database of documents that has this structure:
{
id: 1
elements: [ {elementId: 1, nr1: 1, nr2: 3}, {elementId:2, nr1:5, nr2: 10} ]
}
I am looking for a query that can add a value nr3 which is for example nr2/nr1 to all the objects in the elements array, so that the resulting document would look like this:
{
id: 1
elements: [ {elementId: 1, nr1: 1, nr2: 3, nr3:3}, {elementId:2, nr1:5, nr2: 10, nr3: 2} ]
}
So I imagine a query along the lines of this:
db.collection.updateOne({id:1}, {$set:{"elements.$[].nr3": nr2/nr1}})
But I cant find how to get the value of nr2 and nr1 of the same object in the array.
I found some similar questions on stackoverflow stating this is not possible, but they were 5+ years old, so I thought maybe they have added support for something like this.
I realize I can achieve this with first querying the document and iterate over the elements-array doing updates along the way, but for the purpose of learning I would love to see if its possible to do this in one query.
You can use update with aggregation pipeline starting from MongoDB v4.2,
$map to iterate loop of elements
divide nr2 with nr1 using $divide
merge current object and new field nr3 using $mergeObjects
db.collection.updateOne(
{ id: 1 },
[{
$set: {
elements: {
$map: {
input: "$elements",
in: {
$mergeObjects: [
"$$this",
{ nr3: { $divide: ["$$this.nr2", "$$this.nr1"] } }
]
}
}
}
}
}]
)
Playground
db.collection.update(
{ id:1},
{ "$set": { "elements.$[elem].nr3":elements.$[elem].nr2/elements.$[elem].nr1} },
{ "multi": true }
);
I guess this should work
I'm not quite understanding how fuzzy works in the $searchBeta stage of aggregation. I'm not getting the desired result that I want when I'm trying to implement full-text search on my backend. Full text search for MongoDB was released last year (2019), so there really aren't many tutorials and/or references to go by besides the documentation. I've read the documentation, but I'm still confused, so I would like some clarification.
Let's say I have these 5 documents in my db:
{
"name": "Lightning Bolt",
"set_name": "Masters 25"
},
{
"name": "Snapcaster Mage",
"set_name": "Modern Masters 2017"
},
{
"name": "Verdant Catacombs",
"set_name": "Modern Masters 2017"
},
{
"name": "Chain Lightning",
"set_name": "Battlebond"
},
{
"name": "Battle of Wits",
"set_name": "Magic 2013"
}
And this is my aggregation in MongoDB Compass:
db.cards.aggregate([
{
$searchBeta: {
search: { //search has been deprecated, but it works in MongoDB Compass; replace with 'text'
query: 'lightn',
path: ["name", "set_name"],
fuzzy: {
maxEdits: 1,
prefixLength: 2,
maxExpansion: 100
}
}
}
}
]);
What I'm expecting my result to be:
[
{
"name": "Lightning Bolt", //lightn is in 'Lightning'
"set_name": "Masters 25"
},
{
"name": "Chain Lightning", //lightn is in 'Lightning'
"set_name": "Battlebond"
}
]
What I actually get:
[] //empty array
I don't really understand why my result is empty, so it would be much appreciated if someone explained what I'm doing wrong.
What I think is happening:
db.cards.aggregate... is looking for documents in the "name" and "set_name" fields for words that have a max edit of one character variation from the "lightn" query. The documents that are in the cards collection contain edits that are greater than 2, and therefor your expected result is an empty array. "Fuzzy is used to find strings which are similar to the search term or terms"; used with maxEdits and prefixLength.
Have you tried the term operator with the wildcard option? I think the below aggregation would get you the results you were actually expecting.
e.g.
db.cards.aggregate([
{$searchBeta:
{"term":
{"path":
["name","set_name"],
"query": "l*h*",
"wildcard":true}
}}]).pretty()
You need to provide an index to use with your search query.
The index is basically the analyzer that your query will use to process your results regarding if you want to a full match of the text, or you want a partial match etc.
You can read more about Analyzers from here
In your case, an index based on STANDARD analyzer will help.
After you create your index your code, modified below, will work:
db.cards.aggregate([
{
$search:{
text: { //search has been deprecated, but it works in MongoDB Compass; replace with 'text'
index: 'index_name_for_analyzer (STANDARD in your case)'
query: 'lightn',
path: ["name"] //since you only want to search in one field
fuzzy: {
maxEdits: 1,
prefixLength: 2,
maxExpansion: 100
}
}
}
}
]);
I'm trying to make an aggregation query to find all documents that do not contain a certain element. It needs to be an aggregation because I want to be able to edit the returned documents. Eg. I only want to return some fields and I also want to be able to do a group on eg. the "producer" element.
I already tried practically all I can think of. I tried unwinding the arrays, but then I created even more documents where the element packagingInformation was mission than originally. I tried using $ne, $eq, $gt, $lte,.. to find the documents needed,... but they always return all documents because of the nested array structure.
$ArrayToObject didn't do the trick either for me.
I'm clueless on how to achieve this. The tripple nested array structure beats my imagination.
The only thing that returns me the wanted result is the following query:
db.product.find({
"json.productData.productInformation.details.packagingInformation": { $exists: false }
})
But this doesn't suffice since it's not an aggregate, thus it doens't allow me to continue to do queries with the results. And the $exists doesn't work in aggregates.
This is the JSON structure which I'm struggling with (dummy data).
{
_id: 5ckflsmdk543klmf543klmtrkmgdfm,
productNumber: 001,
json: {
productData: {
productNumber: 001,
producer: coca-cola,
productInformation: [
{
trackingInformation: {
lastUpdate: 01-01-12,
creationDate: 01-01-11
},
details: [
packagingInformation: [
quantity: 5,
size: 20cm
],
productType: drinks,
otherMeaningLessInformation: whatever,
andEvenMoreInformationInArrays: [
andTheInformationGoesOn: wow,
andOn: nastyArrayStructures
]
]
]
}
}
}
}
The wanted result would be to return all the documents that do not contain the packagingInformation array or the packagingInformation.quantity element.
or even better, to return all documents but with an extra field:
containsPackagingInformation: true/false. With false being the result of all documents that do not contain packagingInformation or packagingInformation.quantity.
$exists DOES WORK in a aggregation.
$exists works the same way it works in .find
you can form a query like:
db.collection.aggregate({ $match: {
$or: [
{
"json.productData.productInformation.details.packagingInformation.quantity": {
$exists: false
}
},
{
"json.productData.productInformation.details.packagingInformation": {
$exists: false
}
}
] } })
Try this query here with dummy data
given a document structure as shown, where the trades array can have thousands of items... how on earth could one do a query that would verify that the sequence always has 'startTradeId' one number higher than the previous items 'endTradeId', all the way through the array? is this even possible?
{
"name": "STOCK",
"trades": [{
"endTradeId": 41306,
"startTradeId": 41302,
...
},
{
"endTradeId": 41301,
"startTradeId": 41297,
...
},
{
"endTradeId": 41296,
"startTradeId": 41240,
...
},
...
]
}
You can use $where operator like below :
db.your_collection.find( { $where : function(){ return "this.trades.startTradeId > this.trades.endTradeId" }});
I have a collection which elements can be simplified to this:
{tags : [1, 5, 8]}
where there would be at least one element in array and all of them should be different. I want to substitute one tag for another and I thought that there would not be a problem. So I came up with the following query:
db.colll.update({
tags : 1
},{
$pull: { tags: 1 },
$addToSet: { tags: 2 }
}, {
multi: true
})
Cool, so it will find all elements which has a tag that I do not need (1), remove it and add another (2) if it is not there already. The problem is that I get an error:
"Cannot update 'tags' and 'tags' at the same time"
Which basically means that I can not do pull and addtoset at the same time. Is there any other way I can do this?
Of course I can memorize all the IDs of the elements and then remove tag and add in separate queries, but this does not sound nice.
The error is pretty much what it means as you cannot act on two things of the same "path" in the same update operation. The two operators you are using do not process sequentially as you might think they do.
You can do this with as "sequential" as you can possibly get with the "bulk" operations API or other form of "bulk" update though. Within reason of course, and also in reverse:
var bulk = db.coll.initializeOrderedBulkOp();
bulk.find({ "tags": 1 }).updateOne({ "$addToSet": { "tags": 2 } });
bulk.find({ "tags": 1 }).updateOne({ "$pull": { "tags": 1 } });
bulk.execute();
Not a guarantee that nothing else will try to modify,but it is as close as you will currently get.
Also see the raw "update" command with multiple documents.
If you're removing and adding at the same time, you may be modeling a 'map', instead of a 'set'. If so, an object may be less work than an array.
Instead of data as an array:
{ _id: 'myobjectwithdata',
data: [{ id: 'data1', important: 'stuff'},
{ id: 'data2', important: 'more'}]
}
Use data as an object:
{ _id: 'myobjectwithdata',
data: { data1: { important: 'stuff'},
data2: { important: 'more'} }
}
The one-command update is then:
db.coll.update(
'myobjectwithdata',
{ $set: { 'data.data1': { important: 'treasure' } }
);
Hard brain working for this answer done here and here.
Starting in Mongo 4.4, the $function aggregation operator allows applying a custom javascript function to implement behaviour not supported by the MongoDB Query Language.
And coupled with improvements made to db.collection.update() in Mongo 4.2 that can accept an aggregation pipeline, allowing the update of a field based on its own value,
We can manipulate and update an array in ways the language doesn't easily permit:
// { "tags" : [ 1, 5, 8 ] }
db.collection.updateMany(
{ tags: 1 },
[{ $set:
{ "tags":
{ $function: {
body: function(tags) { tags.push(2); return tags.filter(x => x != 1); },
args: ["$tags"],
lang: "js"
}}
}
}]
)
// { "tags" : [ 5, 8, 2 ] }
$function takes 3 parameters:
body, which is the function to apply, whose parameter is the array to modify. The function here simply consists in pushing 2 to the array and filtering out 1.
args, which contains the fields from the record that the body function takes as parameter. In our case, "$tag".
lang, which is the language in which the body function is written. Only js is currently available.
In case you need replace one value in an array to another check this answer:
Replace array value using arrayFilters