document:
{"_id":"5cb0dfe234a8a30c9c0af127",
"sensors":
[{"value0":0.153,
"value1":-0.306,
"value2":9.807}],
"timestamp":1555095522489,"__v":0}
I want to get 4 field (timestamp and value 0..2) without any array / object.
unwind work only against array but not objects. What should I do?
desired output :
{timestamp":1555095522489,
value0":0.153,
value1":-0.306,
value2":9.807}
Use $unwind and $replaceRoot aggregation operators
db.collection.aggregate([
{ "$unwind": "$sensors" },
{ "$replaceRoot": { "newRoot": { "$mergeObjects": ["$sensors", { "timestamp": "$timestamp" }] }}}
])
Related
I have a large collection of documents with datetime fields in them, and I need to retrieve the most recent document for any given queried list.
Sample data:
[
{"_id": "42.abc",
"ts_utc": "2019-05-27T23:43:16.963Z"},
{"_id": "42.def",
"ts_utc": "2019-05-27T23:43:17.055Z"},
{"_id": "69.abc",
"ts_utc": "2019-05-27T23:43:17.147Z"},
{"_id": "69.def",
"ts_utc": "2019-05-27T23:44:02.427Z"}
]
Essentially, I need to get the most recent record for the "42" group as well as the most recent record for the "69" group. Using the sample data above, the desired result for the "42" group would be document "42.def".
My current solution is to query each group one at a time (looping with PyMongo), sort by the ts_utc field, and limit it to one, but this is really slow.
// Requires official MongoShell 3.6+
db = db.getSiblingDB("someDB");
db.getCollection("collectionName").find(
{
"_id" : /^42\..*/
}
).sort(
{
"ts_utc" : -1.0
}
).limit(1);
Is there a faster way to get the results I'm after?
Assuming all your documents have the format displayed above, you can split the id into two parts (using the dot character) and use aggregation to find the max element per each first array (numeric) element.
That way you can do it in a one shot, instead of iterating per each group.
db.foo.aggregate([
{ $project: { id_parts : { $split: ["$_id", "."] }, ts_utc : 1 }},
{ $group: {"_id" : { $arrayElemAt: [ "$id_parts", 0 ] }, max : {$max: "$ts_utc"}}}
])
As #danh mentioned in the comment, the best way you can do is probably adding an auxiliary field to indicate the grouping. You may further index the auxiliary field to boost the performance.
Here is an ad-hoc way to derive the field and get the latest result per grouping:
db.collection.aggregate([
{
"$addFields": {
"group": {
"$arrayElemAt": [
{
"$split": [
"$_id",
"."
]
},
0
]
}
}
},
{
$sort: {
ts_utc: -1
}
},
{
"$group": {
"_id": "$group",
"doc": {
"$first": "$$ROOT"
}
}
},
{
"$replaceRoot": {
"newRoot": "$doc"
}
}
])
Here is the Mongo playground for your reference.
I need a mongodb query to get the list or map of values with unique value of the field(f) as the key in the collection and count of documents having the same value in the field(f) as the mapped value. How can I achieve this ?
Example:
Document1: {"id":"1","name":"n1","city":"c1"}
Document2: {"id":"2","name":"n2","city":"c2"}
Document3: {"id":"3","name":"n1","city":"c3"}
Document4: {"id":"4","name":"n1","city":"c5"}
Document5: {"id":"5","name":"n2","city":"c2"}
Document6: {"id":"6,""name":"n1","city":"c8"}
Document7: {"id":"7","name":"n3","city":"c9"}
Document8: {"id":"8","name":"n2","city":"c6"}
Query result should be something like this if group by field is "name":
{"n1":"4",
"n2":"3",
"n3":"1"}
It would be nice if the list is also sorted in the descending order.
It's worth noting, using data points as field names (keys) is somewhat considered an anti-pattern and makes tooling difficult. Nonetheless if you insist on having data points as field names you can use this complicated aggregation to perform the query output you desire...
Aggregation
db.collection.aggregate([
{
$group: { _id: "$name", "count": { "$sum": 1} }
},
{
$sort: { "count": -1 }
},
{
$group: { _id: null, "values": { "$push": { "name": "$_id", "count": "$count" } } }
},
{
$project:
{
_id: 0,
results:
{
$arrayToObject:
{
$map:
{
input: "$values",
as: "pair",
in: ["$$pair.name", "$$pair.count"]
}
}
}
}
},
{
$replaceRoot: { newRoot: "$results" }
}
])
Aggregation Explanation
This is a 5 stage aggregation consisting of the following...
$group - get the count of the data as required by name.
$sort - sort the results with count descending.
$group - place results into an array for the next stage.
$project - use the $arrayToObject and $map to pivot the data such
that a data point can be a field name.
$replaceRoot - make results the top level fields.
Sample Results
{ "n1" : 4, "n2" : 3, "n3" : 1 }
For whatever reason, you show desired results having count as a string, but my results show the count as an integer. I assume that is not an issue, and may actually be preferred.
I need to create some plots from single documents existing in mongodb. I can only use the mongodb aggregation framework (so for example I cannot just pull the documents into python and work with it there). I am using the query builder of metabase, so I am limited from this regard.
In order to do this, I am first using some $match queries in order to identify the documents that I need to look at (these are predefined and static). After the $match stage, I am left with one document (this is ok) with the following structure.
{
"id": 1,
"locs": {
"a":1,
"b":2,
"c":3
}
}
I need to change this structure to something like this:
[{"a":1}, {"b":2}, {"c":3"}]
or any other form that would allow me to create pie charts out of the structure.
Thanks!
You can convert locs object to array using $objectToArray. Now $unwind the locs array to split into multiple documents. Use $group with $push accumulator to make the split data again into k and v format. And finally use $replaceRoot with the final data field to move it to $$ROOT position.
db.collection.aggregate([
{ "$project": { "data": { "$objectToArray": "$locs" }}},
{ "$unwind": "$data" },
{ "$group": {
"_id": "$data",
"data": { "$push": { "k": "$data.k", "v": "$data.v" }}
}},
{ "$project": {
"data": { "$arrayToObject": "$data" }
}},
{ "$replaceRoot": { "newRoot": "$data" }}
])
I am currently in the process of modifying a schema and I need to do a relatively trivial transform using the aggregation framework and a bulkWrite.
I want to be able to take this array:
{
...,
"images" : [
"http://example.com/...",
"http://example.com/...",
"http://example.com/..."
]
}
and aggregate to a similar array where the original value is encapsulated:
{
...,
"images" : [
{url: "http://example.com/..."},
{url: "http://example.com/..."},
{url: "http://example.com/..."}
]
}
This slow query works, but it is ridiculously expensive to unwind an entire collection.
[
{
$match: {}
},
{
$unwind: {
path : "$images",
}
},
{
$group: {
_id: "$_id",
images_2: {$addToSet: {url: "$images"}}
}
},
]
How can this be achieved with project or some other cheaper aggregation?
$map expression should do the job, try this:
db.col.aggregate([
{
$project: {
images: {
$map: {
input: '$images',
as: 'url',
in: {
url: '$$url'
}
}
}
}
}
]);
You don't need to use the bulkWrite() method for this.
You can use the $map aggregation array operator to apply an expression to each element element in your array.
Here, the expression simply create a new object where the value is the item in the array.
let mapExpr = {
"$map": {
"input": "$images",
"as": "imageUrl",
"in": { "url": "$$imageUrl }
}
};
Finally you can use the $out aggregation pipeline operator to overwrite your collection or write the result into a different collection.
Of course $map is not an aggregation pipeline operator so which means that the $map expression must be use in a pipeline stage.
The way you do this depends on your MongoDB version.
The best way is in MongoDB 3.4 using $addFields to change the value of the "images" field in your document.
db.collection.aggregate([
{ "$addFields": { "images": mapExpr }},
{ "$out": "collection }
])
From MongoDB 3.2 backwards, you need to use the $project pipeline stage but you also need to include all the other fields manually in your document
db.collection.aggregate([
{ "$project": { "images": mapExpr } },
{ "$out": "collection }
])
I want to get all matching values, using $elemMatch.
// create test data
db.foo.insert({values:[0,1,2,3,4,5,6,7,8,9]})
db.foo.find({},{
'values':{
'$elemMatch':{
'$gt':3
}
}
}) ;
My expecected result is {values:[3,4,5,6,7,8,9]} . but , really result is {values:[4]}.
I read mongo document , I understand this is specification.
How do I search for multi values ?
And more, I use 'skip' and 'limit'.
Any idea ?
Using Aggregation:
db.foo.aggregate([
{$unwind:"$values"},
{$match:{"values":{$gt:3}}},
{$group:{"_id":"$_id","values":{$push:"$values"}}}
])
You can add further filter condition in the $match, if you would like to.
You can't achieve this using an $elemMatch operator since, mongoDB doc says:
The $elemMatch projection operator limits the contents of an array
field that is included in the query results to contain only the array
element that matches the $elemMatch condition.
Note
The elements of the array are documents.
If you look carefully at the documentation on $elemMatch or the counterpart to query of the positional $ operator then you would see that only the "first" matched element is returned by this type of "projection".
What you are looking for is actually "manipulation" of the document contents where you want to "filter" the content of the array in the document rather than return the original or "matched" element, as there can be only one match.
For true "filtering" you need the aggregation framework, as there is more support there for document manipulation:
db.foo.aggregate([
// No point selecting documents that do not match your condition
{ "$match": { "values": { "$gt": 3 } } },
// Unwind the array to de-normalize as documents
{ "$unwind": "$values },
// Match to "filter" the array
{ "$match": { "values": { "$gt": 3 } } },
// Group by to the array form
{ "$group": {
"_id": "$_id",
"values": { "$push": "$values" }
}}
])
Or with modern versions of MongoDB from 2.6 and onwards, where the array values are "unique" you could do this:
db.foo.aggregate([
{ "$project": {
"values": {
"$setDifference": [
{ "$map": {
"input": "$values",
"as": "el",
"in": {
"$cond": [
{ "$gt": [ "$$el", 3 ] },
"$$el",
false
]
}
}},
[false]
]
}
}}
])