I have some data as below:
{MS: 'MS1', fileName: 'file1', RSCP: 75, EcNo: 10, ...}
{MS: 'MS2', fileName: 'file1', RSCP: 76, EcNo: 11, ...}
{MS: 'MS3', fileName: 'file1', RSCP: 77, EcNo: 12, ...}
{MS: 'MS4', fileName: 'file1', RSCP: 78, EcNo: 13, ...}
I need to query data like that:
find all documents that fileName = file1
if MS = MS1 return RSCP
if MS = MS2 return EcNo
how can I query mongo like this in one query?
You can try below aggregation query :
db.collection.aggregate([
/** filter docs with condition */
{
$match: {
fileName: "file1"
}
},
/** Re-create `EcNo` & `RSCP` fields based on conditions
* `$$REMOVE` completely removes the field
*/
{
$addFields: {
EcNo: { $cond: [ { $eq: [ "$MS", "MS1" ] }, "$$REMOVE", "$EcNo" ] },
RSCP: { $cond: [ { $eq: [ "$MS", "MS2" ] }, "$$REMOVE", "$RSCP" ] }
}
}
])
Test : mongoplayground
Related
I have following documents in a MongoDb:
from pymongo import MongoClient
client = MongoClient(host='my_host', port=27017)
database = client.forecast
collection = database.regions
collection.delete_many({})
regions = [
{
'id': 'DE',
'sites': [
{
'name': 'paper_factory',
'energy_consumption': 1000
},
{
'name': 'chair_factory',
'energy_consumption': 2000
},
]
},
{
'id': 'FR',
'sites': [
{
'name': 'pizza_factory',
'energy_consumption': 3000
},
{
'name': 'foo_factory',
'energy_consumption': 4000
},
]
}
]
collection.insert_many(regions)
Now I would like to copy the property sites.energy_consumption to a new field sites.new_field for each site:
set_stage = {
"$set": {
"sites.new_field": "$sites.energy_consumption"
}
}
pipeline = [set_stage]
collection.aggregate(pipeline)
However, instead of copying the individual value per site, all site values are collected and added as an array. Intead of 'new_field': [1000, 2000] I would like to get 'new_field': 1000 for the first site:
{
"_id": ObjectId("61600c11732a5d6b103ba6be"),
"id": "DE",
"sites": [
{
"name": "paper_factory",
"energy_consumption": 1000,
"new_field": [
1000,
2000
]
},
{
"name": "chair_factory",
"energy_consumption": 2000,
"new_field": [
1000,
2000
]
}
]
},
{
"_id": ObjectId("61600c11732a5d6b103ba6bf"),
"id": "FR",
"sites": [
{
"name": "pizza_factory",
"energy_consumption": 3000,
"new_field": [
3000,
4000
]
},
{
"name": "foo_factory",
"energy_consumption": 4000,
"new_field": [
3000,
4000
]
}
]
}
=> What expression can I use to only use the corresponding entry of the array?
Is there some sort of current-index operator:
$sites[<current_index>].energy_consumption
or an alternative dot operator (would remind me on difference between * multiplication and .* element wise matrix multiplication)?
$sites:energy_consumption
Or is this a bug?
Edit
I also tried to use the "$" positional operator, e.g. with
sites.$.new_field
or
$sites.$.energy_consumption
but then I get the error
FieldPath field names may not start with '$'
Related:
https://docs.mongodb.com/manual/reference/operator/aggregation/set/#std-label-set-add-field-to-embedded
In MongoDB how do you use $set to update a nested value/embedded document?
If the field is member of an array by selecting it you are selecting all of them.
{ar :[{"a" : 1}, {"a" : 2}]}
"$ar.a" = [1 ,2]
Also you cant mix update operators with aggregation, you cant use things like
$sites.$.energy_consumption, if you are doing aggregation you have to use aggregate operators, with only exception the $match stage where you can use query operators.
Query
alternative slightly different solution from yours using $setField
i guess it will be faster, but probably little difference
no need to use javascript it will be slower
this is >= MongoDB 5 solution, $setField is new operator
Test code here
aggregate(
[{"$set":
{"sites":
{"$map":
{"input":"$sites",
"in":
{"$setField":
{"field":"new_field",
"input":"$$this",
"value":"$$this.energy_consumption"}}}}}}]
)
use $addFields
db.collection.update({},
[
{
"$addFields": {
"sites": {
$map: {
input: "$sites",
as: "s",
in: {
name: "$$s.name",
energy_consumption: "$$s.energy_consumption",
new_field: {
$map: {
input: "$sites",
as: "value",
in: "$$value.energy_consumption"
}
}
}
}
}
}
}
])
mongoplayground
I found following ugly workarounds that set the complete sites instead of only specifying a new field with dot notation:
a) based on javascript function
set_stage = {
"$set": {
"sites": {
"$function": {
"body": "function(sites) {return sites.map(site => {site.new_field = site.energy_consumption_in_mwh; return site})}",
"args": ["$sites"],
"lang": "js"
}
}
}
}
b) based on map and mergeObjects
set_stage = {
"$set": {
"sites": {
"$map": {
"input": "$sites",
"in": {
"$mergeObjects": ["$$this", {
"new_field": "$$this.energy_consumption_in_mwh"
}]
}
}
}
}
}
If there is some kind of $$this context for the dot operator expression, allowing a more elegant solution, please let me know.
I am trying to write a mongoDB aggregation query in Scala.
How do I write Scala code to use "$let" in '$project' stage?
I am wondering if Variable should be used. Not sure how?
'$project': {
'myprojitem' :{
'$let': {
'vars' : { 'myVariable1': { '$or': [...] } }
'in' : {
'$cond': [
'$$myVariable1',
{ ... },
{ ... },
]
}
}
I figured out the answer. Hopefully it helps someone.
val doc : Document = Document("{
'$let': {
'vars' : { 'myVariable1': { '$or': [...] } },
'in' : { '$cond': ['$$myVariable1',{ ... },{ ... } ]
}
}")
var pipeline = mutable.Buffer[Bson]()
pipeline += Aggregates.project(Projections.fields(
Projections.computed("myprojitem",doc)
))
Basically, every { name : expression } can be written as :
Document("name" -> expression)
Or
Document( "{name : expression}")
$let is used to bind variables together to a results obj. The syntax follows the rule:
{
$let:
{
vars: { <var1>: <expression>},
in: <expression>
}
}
for mere details you should take a look at $let (aggregation) definition from mongodb manual
Here is a text book example just to make more sense:
Consider the following data:
{ _id: 1, price: 10, tax: 0.50, applyDiscount: true }
{ _id: 2, price: 10, tax: 0.25, applyDiscount: false }
And imagine that we want to generate a result for the finalTotal in a way that:
Where Disc = 10% if applyDiscount: true and 0 otherwise.
So we need now to create the aggregation on the data to construct this equation. So we can get a results like:
{ _id: 1, finalTotal: 9.45 }
{ _id: 2, finalTotal: 10.25 }
We can do this by doing:
$project: {
finalTotal: {
$let: {
vars: {
total: { $add: [ '$price', '$tax' ] },
discounted: { $cond: { if: '$applyDiscount', then: (0.9, else: 1 } }
},
in: { $multiply: [ "$$total", "$$discounted" ] }
}
}
}
We can break this down:
Step 1. adding price to tax together to a variable called total
total: { $add: [ '$price', '$tax' ] },
Step 2. transforming the condition in numbers (variable discounted)
discounted: { $cond: { if: '$applyDiscount', then: 0.9, else: 1 } }
Step 3. performing the operation $multiply operation between the constructed $$total and $$discounted
in: { $multiply: [ "$$total", "$$discounted" ] }
I am new to NoSQL databases and MongoDb. I've got the following question:
I have a collection of such documents:
{
vals: [
{
value: 111,
timestamp: 1563454669669
},
{
value: 222,
timestamp: 1563454689665
},
{
value: 333,
timestamp: 1563454669658
}
.......
]
}
I would like to convert it into the following documents using aggregation pipeline:
{
vals: [
[
111,
1563454669669
],
[
222,
1563454689665
],
[
333,
1563454669658
]
.......
]
}
After years of work with relational databases, it's quite hard to understand..
You can use $map operator to transform one array into another array:
db.collection.aggregate([
{
$project: {
vals: {
$map: {
input: "$vals",
in: [ "$$this.value", "$$this.timestamp" ]
}
}
}
}
])
Mongo Playground
Document Structure
{
_id: 5,
grades: [
{ grade_ : 80, mean: 75, std: 8 },
{ mean: 90, std: 5 },
{ mean: 85, std: 3 }
]
}
As per above document structure in mongodb i want rename key grade_ to grade
db.collection.update({"_id":5},{"$rename":{"grades.grade_":"grades.grade"}},{"upsert":false,"multi":true})
which gives below error
"writeError" : {
"code" : 28,
"errmsg" : "cannot use the part (grades of grades.grade_) to traverse the element ({grades: [ { grade_: 80.0, mean: 75.0, std: 8.0 }, { mean: 90.0, std: 5.0 }, { mean: 85.0, std: 3.0 } ]})"
}
I want to rename key grade_ to grade, expected output
{
_id: 5,
grades: [
{ grade : 80, mean: 75, std: 8 },
{ mean: 90, std: 5 },
{ mean: 85, std: 3 }
]
}
As per MongoDB documentation: ($rename does not work if these fields are in array elements.)
For fields in embedded documents, the $rename operator can rename these fields as well as move the fields in and out of embedded documents. $rename does not work if these fields are in array elements.
So, you need to write your custom logic to update.
db.collection.find({
"grades.grade_": { $exists : 1 }
}).forEach( function( doc ) {
for( i=0; i < doc.grades.length; i++ ) {
if(doc.grades[i].grade_ != undefined) {
doc.grades[i].grade = doc.grades[i].grade_;
delete doc.grades[i].grade_;
}
}
db.collection.update({ _id : doc._id }, doc);
});
$rename do not works in an array. So,you can use Aggregate framework's $addField to rename fields in an array.
db.collection.aggregate([
{
$addFields: {
grades: {
$map: {
input: "$grades",
as: "grade",
in: {
grade: "$$grade.grade_",
mean: "$$grade.mean",
std: "$$grade.std"
}
}
}
}
}
])
Output:
[
{
"_id": 5,
"grades": [
{"grade": 80,"mean": 75,"std": 8},
{"mean": 90,"std": 5},
{"mean": 85,"std": 3}
]
}
]
Let's say my collection has documents with ExpName field and Rname field. Expname are all of the type - exp_1, exp_2 etc. Rname is a character string with 4 dashes for example. "As-34rt-d3r5-4453f-er4"
I need to aggregate based on experiment name and removing the text between the last two dashes. In the example I gave above that would be "As-34rt-d3r5"
question 1) how do i incorporate this in one table?
question 2) i solved this in a dirty fashion for one exp, because it seemed like the number of characters was almost the same, so I could just take the first 13 characters which seemed like it was the the substring omitting the last two dashes. Is there a correct way to do this if the text was not so uniform?
db.getCollection('rest01').aggregate(
{$match : {ExpName : "exp_1"}},
{$group: {_id :"$ExpName",_id : {$substr : ["$RName", 0,13]}, total: { $sum:1 }}
})
Ideally I would like to have a result that says Expname, Rnamesubstring, count. This code snippet was for exp_1 one alone. Is it even possible to get it all in one result?
Here is how you could do that:
db.getCollection('rest01').aggregate({
$project: {
"ExpName": 1,
"splitRName": { $split: [ "$RName", "-" ] } // add an array with the constituents of your dash-delimited string id as a new field "splitRName"
}
}, {
$group: {
_id: { // our group id shall be made up of both...
"ExpName": "$ExpName", // ...the "ExpName" field...
"Rnamesubstring": { // and some parts of the "RName" field
$concat:
[
{ $arrayElemAt: [ "$splitRName", 0 ] },
"-",
{ $arrayElemAt: [ "$splitRName", 1 ] },
"-",
{ $arrayElemAt: [ "$splitRName", 2 ] }
]
}
},
total: { $sum: 1 }
}
})
In case you want to do it in MongoDB v3.2 (as stated in your comment), here is something that is not exactly pretty but works:
db.getCollection('rest01').aggregate({
$group: {
_id: { // our group id shall be made up of both...
"ExpName": "$ExpName", // ...the "ExpName" field...
"Rnamesubstring": {
$substr:
[
"$RName",
0,
{
$ifNull:
[
{
$arrayElemAt:
[{
$filter: {
input: {
$map: {
input: [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30 /* add numbers as required */ ],
as: "index",
in: {
$cond: {
if: { $eq: [ "-", { $substr: [ "$RName", "$$index", 1 ] } ] }, // if the string we look at is a dash...
then: "$$index", // ...then let's remember it
else: null // ...otherwise ignore it
}
}
}
},
as: "item",
cond: { $ne: [ null, "$$item" ] } // get rid of all null values
}
},
2 ] // we want the position of the third dash in the string (only)
},
1000 // in case of a malformed RName (wrong number of dashes or completely missing) we want the entire substring
]
}
]
}
},
total: { $sum: 1 }
}
})
Update 2: You seem to be having some data related issues as per your comments (so either missing RName values or improperly structured ones, i.e. without the required number of sections with dashes in between). I have updated the above statement for v3.2 to deal with these rows. You may want to find out, though, which rows actually cause this behaviour. They can be easily identified using the following statement:
db.getCollection('rest01').aggregate({
$project: {
_id: 1,
RName: 1,
"Rnamesubstring": {
$arrayElemAt:
[{
$filter: {
input: {
$map: {
input: [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30 /* add numbers as required */ ],
as: "index",
in: {
$cond: {
if: { $eq: [ "-", { $substr: [ "$RName", "$$index", 1 ] } ] }, // if the string we look at is a dash...
then: "$$index", // ...then let's remember it
else: null // ...otherwise ignore it
}
}
}
},
as: "item",
cond: { $ne: [ null, "$$item" ] } // get rid of all null values
}
},
2 ] // we want the position of the third dash in the string (only)
}
}
}, {
$match: { "Rnamesubstring": { $exists:false } }
})