I have below 2 colection
**aggregate collection**
{
_id: '1',
test_name: 'test1',
status: "flaked"
test_ids: [1, 2]
}
{
_id: '2',
test_name: 'test1',
status: "Flaked"
test_ids: [3, 4]
}
**test collection**
{
_id: 1,
run_id: 1,
test_name: 'test1',
status: "fail"
pipeline: "A"
}
{
_id: 2,
run_id: 2
test_name: 'test1',
status: "pass",
pipeline: "A"
}
{
_id: 3,
run_id: 3
test_name: 'test1',
status: "Infra",
pipeline: "B"
}
{
_id: 4,
run_id: 4
status: "pass",
test_name: 'test1',
pipeline: "B"
}
I am trying to % of how many times the test flaked like No of Flakes/ no of runs. Above should give me (1/4).
Test is considered flake only when aggregate result is flaked and one of test is failed. Can this be built with one query.
Related
Data:
_id: ObjectId(''),
restaurantId: ObjectId(''),
orderId: ObjectId(''),
reviews: [
{
type: "food",
tags: ["good food", "nice food"]
},
{
type: "pricing",
tags: ["best price", "good price"]
}
]
Group by restaurant id
Get total reviews by type: Ex: food: 3, pricing: 2, ambience: 1
Group by type and tags and get counts:
Ex: food-superb: 1, food-loudt:2, pricing-superb: 2, ambience-superb: 1
Expected result:
{
_id: restaurantId,
types: {
food: 4,
....
},
tags: {
nicePrice: 20
}
}
If a have a following structure :
{
_id: 1,
name: 'a',
info: []
},
{
_id: 2,
name: 'b',
info: [
{
infoID: 100,
infoData: 'my info'
}
]
},
{
_id: 3,
name: 'c',
info: [
{
infoID: 200,
infoData: 'some info 200'
},
{
infoID: 300,
infoData: 'some info 300'
}
]
}
I need to query in such a way to obtain the documents where infoID is 100 showing the infoData, or nothing if info is empty, or contains subdocuments with infoID different from 100.
That is, I would want the following output:
{
_id: 1,
name: 'a',
infoData100: null
},
{
_id: 2,
name: 'b',
infoData100: 'my info'
},
{
_id: 3,
name: 'c',
infoData100: null
}
If I $unwind by info and $match by infoID: 100, I lose records 1 and 3.
Thanks for your responses.
Try below query :
Query :
db.collection.aggregate([
/** Adding a new field or you can use $project instead of addFields */
{
$addFields: {
infoData100: {
$cond: [
{
$in: [100, "$info.infoID"] // Check if any of objects 'info.infoID' has value 100
},
{
// If any of those has get that object & get infoData & assign it to 'infoData100' field
$let: {
vars: {
data: {
$arrayElemAt: [
{
$filter: {
input: "$info",
cond: { $eq: ["$$this.infoID", 100] }
}
},
0
]
}
},
in: "$$data.infoData"
}
},
null // If none has return NULL
]
}
}
}
]);
Test : MongoDB-Playground
I have a dataset with metrics collected from a group of sensors.
My dataset looks like this:
{type: 1, display: 'foo', value: 'A'}
{type: 2, display: 'bar', value: 'B'}
{type: 2, display: 'foo', value: 'B'}
I am trying to aggregate the results and get some meaning insights via a REST API. I am somehow trying to produce aggregated results as:
[{
type: 1,
displays: [
{
name: 'foo',
count: 1
}
],
values: [
{
name: 'A',
count: 1
}
],
total_count: 1
},{
type: 2,
displays: [
{
name: 'foo',
count: 1
} , {
name: 'bar',
count: 1
}
],
values: [
{
name: 'B',
count: 2
}
],
total_count: 2
}]
Summarizing the aggregated results and producing shallow results is straight forward, I am struggling though as I can't created the nested counters for types and displays all together.
I have tried to use various aggregation operators with no luck.
Basically I can get one group by types or displays as:
db.logs.aggregate([
{
$group: {
_id: {
type: '$type',
display: '$display'
},
count: { $sum: 1 }
}
}, {
$group: {
_id: '$_id.type',
displays: {
$push: {
name: "$_id.display",
count: "$count"
}
}
}
}
]);
Any help will be highly appreciated.
I am new to MongoDB queries, and I understand the basics of find. However I haven't yet come to grips with $lookup, $project, $aggregate, $match, etc, which is pretty much needed to do anything fancy. And I need a fancy query 😃 See tricky issue below:
Collections:
schema
{ _id: 1, name: “Schema1”, }
{ _id: 2, name: “Schema2”, }
device:
{ _id: 1, schema: 1, organisation: 2 }
{ _id: 2, schema: 1, organisation: 2 }
{ _id: 3, schema: 2, organisation: 2 }
field:
{ _id: 1, organisation: 2, name: “cost”, displayType: “number” }
{ _id: 2, organisation: 2, name: “retail”, displayType: “number” }
{ _id: 3, organisation: 2, name: “project”, displayType: “string” }
fieldvalue
{ _id: 1, device: 1, field: 1, organisation: 2, value: 2000 }
{ _id: 2, device: 1, field: 2, organisation: 2, value: 3000 }
{ _id: 3, device: 2, field: 1, organisation: 2, value: 1000 }
{ _id: 4, device: 2, field: 2, organisation: 2, value: 1000 }
{ _id: 5, device: 3, field: 1, organisation: 2, value: 500 }
{ _id: 6, device: 1, field: 3, organisation: 2, value: “Project1” }
{ _id: 7, device: 2, field: 3, organisation: 2, value: “Project2” }
{ _id: 8, device: 3, field: 3, organisation: 2, value: “Project2” }
I want to query FieldValue.
I want to $sum all “value” where field is 1
BUT only those which “shares” device with field 3 with value “Project2”.
As parameters I have:
The id of the field I want to sum, e.g. 1 (cost)
The id of the "project” field, e.g. 3 (project)
The value of the “project” field (id 3) which also must be met in order to qualify for summation.
So my query should only $sum the “value” of id’s: 3 and 5
How can I do that in Mongo Query?
And would it be possible to add more contraints? E.g. the “schema” of the “device” must be 3, which should then result in a $sum of just id: 5
Is it possible to perform both a map reduce with a lookup in the same query pipeline efficiently?
Let's say I've two collections:
items: { _id, group_id, createdAt }
purchases: { _id, item_id }
I want to get the top n item groups, based on the number of purchases on the most recent x items per group.
If I had the number of purchases available in the item documents, then I could aggregate and sort, but this is not the case.
I can get the most recent x items per group as so:
let x = 3;
let map = function () {
emit(this.group_id, { items: [this] });
};
let reduce = function (key, values) {
return { items: getLastXItems(x, values.map(v => v.items[0])) };
};
let scope = { x };
db.items.mapReduce(map, reduce, { out: { inline: 1 }, scope }, function(err, res) {
if (err) {
...
} else {
// res is an array of { group_id, items } where items is the last x items of the group
}
});
But I'm missing purchase count so I can't use it to sort groups, and output the top n groups (which btw I'm not even sure I can do)
I'm using this on a web server, and running the query with scope variable depending on the user context, so I don't want to output the result to another collection and have to do everything inline.
=== edit 1 === add data example:
Sample data could be:
// items
{ _id: '1, group_id: 'a', createdAt: 0 }
{ _id: '2, group_id: 'a', createdAt: 2 }
{ _id: '3, group_id: 'a', createdAt: 4 }
{ _id: '4, group_id: 'b', createdAt: 1 }
{ _id: '5, group_id: 'b', createdAt: 3 }
{ _id: '6, group_id: 'b', createdAt: 5 }
{ _id: '7, group_id: 'b', createdAt: 7 }
{ _id: '8, group_id: 'c', createdAt: 5 }
{ _id: '9, group_id: 'd', createdAt: 5 }
// purchases
{ _id: '1', item_id: '1' }
{ _id: '2', item_id: '1' }
{ _id: '3', item_id: '3' }
{ _id: '4', item_id: '5' }
{ _id: '5', item_id: '5' }
{ _id: '6', item_id: '6' }
{ _id: '7', item_id: '7' }
{ _id: '8', item_id: '7' }
{ _id: '9', item_id: '7' }
{ _id: '10', item_id: '3' }
{ _id: '11', item_id: '9' }
and sample result with n = 3 and x = 2 would be:
[
group_id: 'a', numberOfPurchasesOnLastXItems: 4,
group_id: 'b', numberOfPurchasesOnLastXItems: 3,
group_id: 'c', numberOfPurchasesOnLastXItems: 1,
]
I think this can be solved with the aggregation pipeline, but I've no idea on how bad this is, especially performance wise.
Concerns I have are:
will the aggregation pipeline be able to benefits from indexes, on lookup and sort?
can the lookup + projection that's only used to count matching items be simplified
Anyway, I think one solution I could be:
x = 2;
n = 3;
items.aggregate([
{
$lookup: {
from: 'purchases',
localField: '_id',
foreignField: 'item_id',
as: 'purchases',
},
},
/*
after the join, the data is like {
_id: <itemId>,
group_id: <itemGroupId>,
createdAt: <itemCreationDate>,
purchases: <arrayOfPurchases>,
}
*/
{
$project: {
group_id: 1,
createdAt: 1,
pruchasesCount: { $size: '$purchases' },
}
}
/*
after the projection, the data is like {
_id: <itemId>,
group_id: <itemGroupId>,
createdAt: <itemCreationDate>,
purchasesCount: <numberOfPurchases>,
}
*/
{
$sort: { createdAt: 1 }
},
{
$group: {
_id: '$group_id',
items: {
$push: '$purchasesCount',
}
}
}
/*
after the group, the data is like {
_id: <groupId>,
items: <array of number of purchases per item, sorted per item creation date>,
}
*/
{
$project: {
numberOfPurchasesOnMostRecentItems: { $sum: { $slice: ['$purchasesCount', x] } },
}
}
/*
after the projection, the data is like {
_id: <groupId>,
numberOfPurchasesOnMostRecentItems: <number of purchases on the last x items>,
}
*/
{
$sort: { numberOfPurchasesOnMostRecentItems: 1 }
},
{ $limit : n }
]);