Overview
A few devices are collecting data and sending it to a Node/MongoDb endpoint. Then, the user would use an endpoint to get all that data into a json.
Models
Device Model
const deviceSchema = new Schema({
group: { type: Schema.Types.ObjectId, ref: 'Group' },
deviceId: { type: String, unique: true },
name: String,
notes: String,
pac: String,
endCertificate: String,
lat: Number,
lng: Number
});
Message Model
const messageSchema = new Schema({
deviceId: { type: String, required: true },
raw: { type: String, required: true },
receivedAt: { type: Date, default: Date.now() }
});
One device can have N messages
Problem to solve
I want to get a json that has all the devices and have an array containing
all the messages that belongs to that device.
[
{
"id":"5b86c323e95759603ad7ea54",
"deviceId":"Device 01",
"name":"Device bla",
"notes":"...",
"pac":"pac",
"lat":-20.817396,
"endCertificate":"cert",
"lng":-27.031321,
"messages":[
{
"id":"5b869a42e0b94041b5f21eed",
"deviceId":"Device 01",
"raw":"1111111",
"receivedAt":"2018-08-29T13:04:43.641Z",
"__v":0
},
{
"id":"5b8c782fef4f8e98783f6f35",
"deviceId":"Device 01",
"raw":"2222222",
"receivedAt":"2018-09-01T09:04:43.641Z",
"__v":0
},
{
"id":"5b8c7840ef4f8e98783f6f3e",
"deviceId":"Device 01",
"raw":"3333333",
"receivedAt":"2018-09-02T09:04:43.641Z",
"__v":0
}
]
},
{
"id":"5b8c28ec38c51813cd159bac",
"deviceId":"Device 02",
"name":"Device ...",
"notes":"...",
"lat":-27.812296,
"lng":-27.073314,
"__v":0,
"messages":[
{
"id":"5b8c784cef4f8e98783f6f43",
"deviceId":"Device 02",
"raw":"1111111",
"receivedAt":"2018-09-01T09:04:43.641Z",
"__v":0
}
]
}
]
My solution
To get a json as the one above I have:
const [results, itemCount] = await Promise.all([
Device.aggregate([
{ $match: {} },
{
$lookup: {
from: 'messageschemas',
localField: 'deviceId',
foreignField: 'deviceId',
as: 'messages'
}
}
]).limit(req.query.limit).skip(req.skip)
.exec(),
Device.countDocuments(match)
]);
res.setHeader('X-Total-Count', itemCount);
res.send(results);
My question
How can I order the messages I get from the $lookup into messages[] by
'receivedAt'?
You need to $unwind the messages array and can apply $sort with receivedAt field and finally $group to rollback again into the array.
Device.aggregate([
{ "$match": { }},
{ "$lookup": {
"from": "messageschemas",
"localField": "deviceId",
"foreignField": "deviceId",
"as": "messages"
}},
{ "$unwind": "$messages" },
{ "$sort": { "messages.receivedAt": 1 }},
{ "$group": {
"_id": "$_id",
"messages": { "$push": "$messages" }
}}
])
Which can be simply done with the 3.6 $lookup syntax
Device.aggregate([
{ "$match": { }},
{ "$lookup": {
"from": "messageschemas",
"let": { "deviceId": "$deviceId" },
"pipeline": [
{ "$match": { "$expr": { "$eq": [ "$deviceId", "$$deviceId" ] } } },
{ "$sort": { "receivedAt": 1 }}
],
"as": "messages"
}}
])
Related
I have two models
export const StorySchema = new Schema({
type: { type: String, required: true },
texts: { type: Array, require: true },
});
export const TextSchema = new Schema({
textKey: { type: String, required: true },
text: { type: String, required: true },
});
My collections
// stories
[
{
"type": "export",
"texts": ["export", "download"]
},
...
]
// Text
[
{
"textKey": "export",
"text": "Export ....."
},
{
"textKey": "download",
"text": "Download ....."
},
...
]
I want to combine a field textKey from collection text with array texts from collection story and write field text from collection text into result query. I have to get a array of object
[
{
"type": "export",
"texts": ["Export .....", "Download ....."]
},
...
]
I tried to create a aggregate multiple collections
public async getStoriesList(): Promise<Story[]> {
return await this.storyModel.aggregate([{
$lookup: {
from: 'texts',
localField: 'texts',
foreignField: 'textKey',
as: 'texts',
},
}]);
}
But i got an empty array. Where I made an error? How I can create a aggregate?
You can't lookup on an array, you could achieve what you want using this aggregation but it may be slow if you have big collections:
db.stories.aggregate([
{
"$unwind": "$texts"
},
{
"$lookup": {
"from": "texts",
"localField": "texts",
"foreignField": "textKey",
"as": "data"
}
},
{
"$unwind": "$data"
},
{
"$group": {
"_id": "type",
"texts": {
"$push": "$data.text"
}
}
},
{
"$project": {
"type": "$_id",
"_id": 0,
"texts": 1
}
}
])
I have a collection of news articles in mongodb and another collection that maps a user's ID to an article's ID and has a "like" state, which can be either "like" or "dislike" or "none" if no entry with the user and article exists.
Here are both schemas:
// news collection
const articleSchema = new Schema({
title: String,
content: String,
})
// newslikes collection
const articleLikeSchema = new Schema({
user: { type: Schema.Types.ObjectId, ref: 'Client' },
article: { type: Schema.Types.ObjectId, ref: 'News' },
state: { type: String, enum: ['like', 'dislike'] }
})
I'm trying to write an aggregation query which joins these two collection using $lookup and then finds the state of a specific user's like on all articles. This is what I have so far:
const results = await News.aggregate([
{ $match: query },
{ $sort: { date: -1 } },
{ $skip: page * pageLength },
{ $limit: pageLength },
{ $lookup: {
from: 'newslikes',
localField: '_id',
foreignField: 'article',
as: 'likes'
} },
{ $project: {
title: 1,
likes: 1,
content: 1,
// numLikes: { $size: '$likes' }
userLikeStatus: {
$filter: {
input: '$likes',
as: 'like',
cond: {
$eq: ['$user._id', '5ccf13adcec5e6d84f940417']
}
}
}
} }
])
However this is not working. Is what I'm doing even the correct approach or is there a better way to do this rather than $filter?
You can use below aggregation with mongodb 3.6 and above
News.aggregate([
{ "$match": query },
{ "$sort": { "date": -1 } },
{ "$skip": page * pageLength },
{ "$limit": pageLength },
{ "$lookup": {
"from": "newslikes",
"let": { "articleId": "$_id" },
"pipeline": [
{ "$match": {
"$expr": { "$eq": [ "$article", "$$articleId" ] },
"user": mongoose.Types.ObjectId("5ccf13adcec5e6d84f940417")
}}
],
"as": "likes"
}},
{ "$addFields": {
"userLikeStatus": { "$ifNull": [{ "$arrayElemAt": ["$likes.state", 0] }, "none"] }
}}
])
Or the way you are trying
Basically here you need to put $cond for the field userLikeStatus i.e if the $size of the array after $filter is $gte 1 then user likes it else does not.
News.aggregate([
{ "$match": query },
{ "$sort": { "date": -1 } },
{ "$skip": page * pageLength },
{ "$limit": pageLength },
{ "$lookup": {
"from": "newslikes",
"localField": "_id",
"foreignField": "article",
"as": "likes"
}},
{ "$project": {
"title": 1,
"likes": 1,
"content": 1,
// numLikes: { $size: '$likes' }
"userLikeStatus": {
"$let": {
"vars": {
"array": {
"$filter": {
"input": "$likes",
"as": "like",
"cond": { "$eq": ["$$like.user", mongoose.Types.ObjectId("5ccf13adcec5e6d84f940417")] }
}
}
},
"in": {
"$ifNull": [{ "$arrayElemAt": ["$$array.state", 0] }, "none"]
}
}
}
}}
])
I have 2 collections, the first storing the animes watched by each user, their status etc:
const listSchema = mongoose.Schema({
user: {
type: Schema.Types.ObjectId,
ref: 'user'
},
animes: [{
_id: false,
anime: {
type: Schema.Types.ObjectId,
ref: 'anime',
unique: true
},
status: String,
episodes: Number,
rating: Number
}]
});
and the second storing all animes and the relevant information.
I want to show that second collection but adding status and episodes fields that are filled from the list of the logged in user.
I tried the following:
Anime.aggregate([
{
$lookup: {
'from': 'lists',
localField: '_id',
foreignField: 'animes.anime',
'as': 'name'
},
},
{
$unwind: '$animes'
},
{
$project:{
status:'$lists.animes.status'
}
}
]
)
but it returns an empty array.
I also don't know how to filter by userId seeing how the _id is in the foreign collection.
You can use below aggregation
{ "$lookup": {
"from": "lists",
"let": { "id": "$_id" },
"pipeline": [
{ "$match": { "$expr": { "$in": ["$$id", "$animes.anime"] }}},
{ "$unwind": "$animes" },
{ "$match": { "$expr": { "$eq": ["$animes.anime", "$$id"] }}}
],
"as": "name"
}}
For some reason, I can't retrieve the author name from another collection on my aggregate query.
db.getCollection('books').aggregate([
{
$match: {
authorId: { $nin: [ObjectId('5b9a008575c50f1e6b02b27b'), ObjectId('5ba0fb3275c50f1e6b02b2f5'), ObjectId('5bc058b6ae9a2a4d6df330b1')]},
isBorrowed: { $in: [null, false] },
status: 'ACTIVE',
},
},
{
$lookup: {
from: "authors",
localField: "authorId", // key of author id in "books" collection
foreignField: "_id", // key of author id in "authors" collection
as: "bookAuthor",
}
},
{
$group: {
_id: {
author: '$authorId',
},
totalSalePrice: {
$sum: '$sale.amount',
},
},
},
{
$project: {
author: '$_id.author',
totalSalePrice: '$totalSalePrice',
authorName: '$bookAuthor.name', // I can't make this appear
_id: 0,
},
},
{ $sort: { totalSalePrice: -1 } },
])
Any advice on where I had it wrong? Thanks for the help.
Two things that are missing here: you need $unwind to convert bookAuthor from an array into single object and then you need to add that object to your $group stage (so that it will be available in next stages), try:
db.getCollection('books').aggregate([
{
$match: {
authorId: { $nin: [ObjectId('5b9a008575c50f1e6b02b27b'), ObjectId('5ba0fb3275c50f1e6b02b2f5'), ObjectId('5bc058b6ae9a2a4d6df330b1')]},
isBorrowed: { $in: [null, false] },
status: 'ACTIVE',
},
},
{
$lookup: {
from: "authors",
localField: "authorId",
foreignField: "_id",
as: "bookAuthor", // this will be an array
}
},
{
$unwind: "$bookAuthor"
},
{
$group: {
_id: {
author: '$authorId',
},
bookAuthor: { $first: "$bookAuthor" },
totalSalePrice: {
$sum: '$sale.amount',
},
},
},
{
$project: {
author: '$_id.author',
totalSalePrice: '$totalSalePrice',
authorName: '$bookAuthor.name',
_id: 0,
},
},
{ $sort: { totalSalePrice: -1 } },
])
Actually you have lost the bookAuthor field in the $group stage. You have to use $first accumulator to get it in the next $project stage.
{ "$group": {
"_id": { "author": "$authorId" },
"totalSalePrice": { "$sum": "$sale.amount" },
"authorName": { "$first": "$bookAuthor" }
}},
{ "$project": {
"author": "$_id.author",
"totalSalePrice": "$totalSalePrice",
"authorName": { "$arrayElemAt": ["$bookAuthor.name", 0] }
"_id": 0,
}}
I am new in using MongoDB's aggregation framework and here I have below a schema:
var bookSoldSchema = new Schema({
buyer:{
type: Number
}
book: {
type: Schema.Types.ObjectId, ref: 'Book'
}
});
var bookSchema = new Schema({
bookName: { type: String },
categories: [{ type: Schema.Types.ObjectId, ref: 'BookCategory'}],
})
Each book has multiple categories, I would like to show the top 5 best seller categories and each category I need show top 3 books which where sold most, and I also need show the category name, the results which I need is:
[
{
categoryId: xxx,
categoryName:xxx,
top3books:[
{bookId:xxx, bookName:xxx},
{bookId:xxx, bookName:xxx},
{bookId:xxx, bookName:xxx},
]
},
{
categoryId: xxx,
categoryName:xxx,
top3books:[
{bookId:xxx, bookName:xxx},
{bookId:xxx, bookName:xxx},
{bookId:xxx, bookName:xxx},
]
}
]
How can I go about this in MongoDB?
You can run the following aggregation pipeline which assumes you have a model BookSold that uses the bookSoldSchema above, books and bookcategories underlying collections:
BookSold.aggregate([
{
"$lookup": {
"from": "books",
"localField": "book",
"foreignField": "_id",
"as": "book_join"
}
},
{ "$unwind": "$book_join" },
{ "$unwind": "$book_join.categories" },
{
"$lookup": {
"from": "bookcategories",
"localField": "book_join.categories",
"foreignField": "_id",
"as": "categories"
}
},
{ "$unwind": "$categories" },
{ "$sort": { "categories.name": 1, "buyer": -1 } },
{
"$group": {
"_id": "$categories.name",
"categoryId": { "$first": "$categories._id" },
"buyer": { "$first": "$buyer" },
"books": {
"$push": {
"bookId": "$book_join._id",
"bookName": "$book_join.bookName"
}
}
}
},
{ "$sort": { "buyer": -1 } },
{ "$limit": 5 },
{
"$project": {
"_id": 0,
"categoryId": 1,
"categoryName": "$_id",
"top3books": { "$slice": [ "$books", 3 ] }
}
}
], function(err, result) {
if (err) handleError(err);
console.log(JSON.stringify(result, null, 4));
})