I have mongoDB collection items with following document structure:
{ name: string, values: string[] }
Then I have large amount of documents outside of database, which I want add to the db.
If document with same name already exists in database, push its value to values of db item,
If document with same name doesn't exist, create new document.
For example, let's have these records in the database:
[
{ "name": "A", "values": ["Alaska"] },
{ "name": "B", "values": [] }
]
Now add these records:
[
{ "name": "A", "value": "Australia" },
{ "name": "C", "value": "Canada" }
]
Result should be:
[
{ "name": "A", "values": ["Alaska", "Australia"] },
{ "name": "B", "values": [] },
{ "name": "C", "values": ["Canada"] }
]
However the number of documents can be hundreds of thousands. Is there any better way to upsert array of records than one by one?
db.items.update({ "name": "A" }, { "$set": { "name": "A" }, "$push": { "values": "Australia" } }, { "upsert": true })
db.items.update({ "name": "C" }, { "$set": { "name": "C" }, "$push": { "values": "Canada" } }, { "upsert": true })
...
You can use bulk writes to aggregate multiple updates into a single network request, but if the changes are individually determined per document and cannot be expressed as a query applying to multiple documents, each change must be its own insert/update command.
Related
I was grouping documents based on object, it works perfect but I have no idea whether mongo group (object) based on reference or value. I mean, mongo works on reference based or values based while we group.
Anybody knows, what's the behind the hook working of $group? Is mongo group based on reference or value?
Here are dummy documents,
[
{
name: "a",
title: {
title: "b",
order: 1
},
group: "B"
},
{
name: "b",
title: {
title: "b",
order: 1
},
group: "Bs"
},
{
name: "c",
title: {
title: "c",
order: 2
},
group: "B"
}
]
aggregation query,
db.collection.aggregate([
{
"$group": {
"_id": "$title",
"items": {
"$addToSet": {
"name": "$name",
"group": "$group"
}
}
}
}
])
it returns,
[
{
"_id": {
"order": 1,
"title": "b"
},
"items": [
{
"group": "B",
"name": "a"
},
{
"group": "Bs",
"name": "b"
}
]
},
{
"_id": {
"order": 2,
"title": "c"
},
"items": [
{
"group": "B",
"name": "c"
}
]
}
]
MongoDB uses this comparison rules
See how it compares objects, it goes field by field, checks name and then value etc. It goes inside and its recursive.
My collection has array "name" with objects inside. I need to remove only those objects inside array where "name.x" is blank.
"name": [
{
"name.x": [
{
"_id": "607e7fcca57aa56e2a06b57b",
"name": "abc",
"type": "123"
}
],
"_id": {
"$oid": "62232cd70ce38c5007de31e6"
},
"qty": "1.0",
"Unit": "pound,lbs"
},
{
"name.x": [
{
"_id": "607e7fcca57aa56e2a06b430",
"name": "xyz",
"type": "123"
}
],
"_id": {
"$oid": "62232cd70ce38c5007de31e7"
},
"qty": "1.0",
"Unit": "pound,lbs"
},{
"name.x": []
,
"_id": {
"$oid": "62232cd70ce38c5007de31e7"
},
"qty": "1.0",
"Unit": "pound,lbs"
}
I tried to get all the ids where name.x is blank using python and used $pull to remove objects base on those ids.But the complete array got deleted.How can I remove the objects that meet the condition.
Think MongoDB update with aggregation pipeline meets your requirement especially to deal with the field name with ..
$set - Update the name array field by $filter name.x field is not an empty array.
db.collection.update({},
[
{
$set: {
name: {
$filter: {
input: "$name",
cond: {
$ne: [
{
$getField: {
field: "name.x",
input: "$$this"
}
},
[]
]
}
}
}
}
}
],
{
multi: true
})
Sample Mongo Playground
I have the following collection:
{
"invoice": {
"data": [{
"name": "VOUCHERNUMBER",
"value": "59302311"
}, {
"name": "VOUCHERDATE",
"value": "2020-02-20"
}
]
}
},
{
"invoice": {
"data": [{
"name": "VOUCHERNUMBER",
"value": "59112389"
}, {
"name": "VOUCHERDATE",
"value": "2020-02-20"
}
]
}
},
{
"invoice": {
"data": [{
"name": "VOUCHERNUMBER",
"value": "59302378"
}, {
"name": "VOUCHERDATE",
"value": "2020-02-11"
}
]
}
}
My task is to build a query that find all invoices which invoicenumbers includes "11" (or any other substring).
So I built the following statement:
{"invoice.data.name": "VOUCHERNUMBER", "invoice.data.value": {$regex : "11"} }
I'm expecting a result of the first two objects, but because of the second value in the third object, mongodb returns me all three objects. Then I tried
{$and : [{"invoice.data.name": "VOUCHERNUMBER"}, {"invoice.data.value": {$regex : "11"}}]}
with the same result ...
So I'm running out of ideas. Is there a solution to search for the string only in the value field where the corresponding "name" field contains "VOUCHERNUMBER"?
You need $elemMatch.
The $elemMatch operator matches documents that contain an array field with at least one element that matches all the specified query criteria.
db.collection.find({
"invoice.data": {
"$elemMatch": {
"name": "VOUCHERNUMBER",
"value": {
$regex: "11"
}
}
}
})
Sample Mongo Playground
Apologies for the confusing title, I am not sure how to summarize this.
Suppose I have the following list of documents in a collection:
{ "name": "Lorem", "source": "A" }
{ "name": "Lorem", "source": "B" }
{ "name": "Ipsum", "source": "A" }
{ "name": "Ipsum", "source": "B" }
{ "name": "Ipsum", "source": "C" }
{ "name": "Foo", "source": "B" }
as well an ordered list of accepted sources, where lower indexes signify higher priority
sources = ["A", "B"]
My query should:
Take a list of available sources and a list of wanted names
Return a maximum of one document per name.
In case of multiple matches, the document with the most prioritized source should be chosen.
Example:
wanted_names = ['Lorem', 'Ipsum', 'Foo', 'NotThere']
Result:
{ "name": "Lorem", "source": "A" }
{ "name": "Ipsum", "source": "A" }
{ "name": "Foo", "source": "B" }
The results don't necessarily have to be ordered.
Is it possible to do this with a Mongo query alone? If so could someone point me towards a resource detailing how to accomplish it?
My current solution doesn't support a list of names, and instead relies on a Python script to execute multiple queries:
db.collection.aggregate([
{$match: {
"name": "Lorem",
"source": {
$in: sources
}}},
{$addFields: {
"order": {
$indexOfArray: [sources, "$source"]
}}},
{$sort: {
"order": 1
}},
{$limit: 1}
]);
Note: _id fields are omitted in this question for the sake of brevity
How about this: With $group we have $min operator which takes lower source
Note: If you prioritize as ['B', 'A'], use $max then
db.collection.aggregate([
{
$match: {
"name": {
$in: [
"Lorem",
"Ipsum",
"Foo",
"NotThere"
]
},
"source": {
$in: [
"A",
"B"
]
}
}
},
{
$group: {
_id: "$name",
source: {
$min: "$source"
}
}
},
{
$project: {
_id: 0,
name: "$_id",
source: 1
}
}
])
MongoPlayground
I have a document of the following form:
{
"name": "root1",
"children": [{
"name": "A",
"children": [{
"name": "A1",
"items": 20
}, {
"name": "A2",
"items": 19
}],
"items": 8
}, {
"name": "B",
"items": 12
}],
"items": 1
}
That is, each level has a "name" field, an "items" field, and optionally a children field. I would like to run a query which returns the total number of items for each root. In this example, it should return (since 20+19+8+12+1=60)
{ "_id" : "root1", "items" : 60 }
However, each document can have arbitrarily many levels. That is, this example has two to three children below the root, but other documents may have more. That is, I cannot do something like
db.myCollection.aggregate( { $unwind : "$children" },
{ $group : { _id : "$name", items: { $sum : "$items" } } } )
What sort of query will work?
There really is no way to descend arrays to arbitrary depths using the aggregation framework. For this sort of structure you need to use mapReduce where you can programatically do this:
db.collection.mapReduce(
function () {
var items = 0;
var action = function(current) {
items += current.items;
if ( current.hasOwnProperty("children") ) {
current.children.forEach(function(child) {
action( child );
});
}
};
action( this );
emit( this.name, items );
},
function(){},
{ "out": { "inline": 1 } }
)
If you do not want mapReduce then consider another structure for your data and do things differently:
{ "name": "root1", "items": 1, "path": [], "root": null },
{ "name": "A", "items": 8, "path": ["root1"], "root": "root1" },
{ "name": "A1", "items": 20, "path": ["root1", "A"], "root": "root1" },
{ "name": "A2", "items": 19, "path": ["root1", "A"], "root": "root1" },
{ "name": "B", "items": 12, "path": ["root1"], "root": "root1" }
Then you just have a simple aggregate:
db.collection.aggregate([
{ "$group": {
"_id": {
"$cond": [
"$root",
"$root",
"$name"
]
},
"items": { "$sum": "$items" }
}}
])
So if you take a different approach to mapping a hierarchy then doing things such as aggregating totals for paths is much easier without the recursive inspection that would otherwise be required.
The approach that you need depends on your actual usage requirements.