How to query to get specific objects from array of objects? - mongodb

Currently I am learning mongodb. Suppose I have one collection named post in mongodb which data is :
[{
id: 123,
uId: 111,
msg: 'test 1',
attachments: [
{name: 'attach1', url: 'https://example.com', isDeleted: false},
{name: 'attach2', url: 'https://example.com', isDeleted: true}
]
},
{
id: 456,
uId: 222,
msg: 'test 2',
attachments: [
{name: 'attach1', url: 'https://example.com', isDeleted: true}
]
},
{
id: 789,
uId: 333,
msg: 'test 3',
attachments: [
{name: 'attach1', url: 'https://example.com', isDeleted: false}
]
}]
Now i want the result of all post where attachments.isDeleted is false which look like :
[{
id: 123,
uId: 111,
msg: 'test 1',
attachments: [
{name: 'attach1', url: 'https://example.com', isDeleted: false}
]
},
{
id: 456,
uId: 222,
msg: 'test 2',
attachments: []
},
{
id: 789,
uId: 333,
msg: 'test 3',
attachments: [
{name: 'attach1', url: 'https://example.com', isDeleted: false}
]
}]
I tried this db.post.find({attachments: {$elemMatch: {isDeleted: false}}}) but it is not working. I have taken help from [https://stackoverflow.com/questions/62953855/how-get-query-for-array-of-objects-in-mongodb]

I think this is what you are looking for. It uses the Mongodb aggregation framework. You can take a look the documentation to see details. In brief, the $project stage allow us select fields to show or to hide, and it admits calculated fields. We calculated a new attachments field using $filter stage with a the given condition (isDeleted equals to false).
db.collection.aggregate([
{
"$project": {
_id: 1,
uId: 1,
msg: 1,
attachments: {
"$filter": {
"input": "$attachments",
"as": "attachment",
"cond": {
"$eq": [
"$$attachment.isDeleted",
false
]
}
}
}
}
}
])
You can try it in: https://mongoplayground.net/p/YDvpkbZIZ2H
Note that I changed id by _id, but it was only for style purpose.
Hope I helped.

Related

mongodb count the number of consecutive matching docs and merge them into one

Consider that I have a mongodb collection called chatMessages with these properties (using mongoose on nodejs):
const schema = {
_id: ObjectID,
chatID: String,
type: String,
message: String,
senderID: String,
date: Date
}
Imagine that I have 5 documents in the collection:
[
{
_id: 'id1',
chatID: 'alternateChat',
type: 'txt',
message: 'first message',
senderID: 'first_sender_id',
date: new Date('0001-01-02T01:01:11.001Z')
},
{
_id: 'id2',
chatID: 'alternateChat',
type: 'groupedMsgs',
message: 'second message',
senderID: 'first_sender_id',
date: new Date('0001-01-03T01:01:11.001Z')
},
{
_id: 'id3',
chatID: 'alternateChat',
type: 'groupedMsgs',
message: 'third message',
senderID: 'first_sender_id',
date: new Date('0001-01-04T01:01:11.001Z')
},
{
_id: 'id4',
chatID: 'alternateChat',
type: 'txt',
message: 'fourth message',
senderID: 'first_sender_id',
date: new Date('0001-01-05T01:01:11.001Z')
},
{
_id: 'id5',
chatID: 'alternateChat',
type: 'groupedMsgs',
message: 'fifth message',
senderID: 'first_sender_id',
date: new Date('0001-01-06T01:01:11.001Z')
}
]
I want to query these documents such that consecutive rows (when ranked by date) with type of groupedMsgs are represented as one, as well as the number of consecutive rows that are present for each unique groupedMsgs in the final output. Concretely, I would like output as shown below:
[
{
_id: 'id1',
chatID: 'alternateChat',
type: 'txt',
message: 'first message',
senderID: 'first_sender_id',
date: new Date('0001-01-02T01:01:11.001Z')
},
{
_id: 'id2',
chatID: 'alternateChat',
type: 'groupedMsgs',
message: 'second message',
senderID: 'first_sender_id',
date: new Date('0001-01-03T01:01:11.001Z'),
numConsecutiveItems: 2
},
{
_id: 'id4',
chatID: 'alternateChat',
type: 'txt',
message: 'fourth message',
senderID: 'first_sender_id',
date: new Date('0001-01-05T01:01:11.001Z')
},
{
_id: 'id5',
chatID: 'alternateChat',
type: 'groupedMsgs',
message: 'fifth message',
senderID: 'first_sender_id',
date: new Date('0001-01-06T01:01:11.001Z'),
numConsecutiveItems: 1
}
]
Notice that the third message is not in the final output because it has type of groupedMsgs and consecutively follows another message with type groupedMsgs, and the second message has numConsecutiveItems of 2 for the same reason. More so, the fifth message is present because it doesn't immediately follow another groupedMsgs message, and its value of numConsecutiveItems is 1 for the same reason. What is an aggregation pipeline that can do this for me? My preference would be to avoid using $accumulator, $function, $where, and $accumulator to avoid running javascript during the query as that can slow down the query operation, but I'm open to all answers nevertheless.
It can be done, but it is not recommended.
The method I used here (avoiding using "accumulator" per your request) is not a good practice and should not be used for large datasets, since it first groups all your data to one document and then multiply it by the number of documents. If you do choose to use something like that, I strongly recommend adding a $match phase before it in order to reduce the number of calculated documents.
In your case, you can run it for short periods of time (chunks), noticing that a message with "type" different from "groupedMsgs" is a good place to cut between them.
db.collection.aggregate([
{"$sort": { date: 1}},
{"$group": {"_id": 0,
data: {"$push": {type: "$type", chatID: "$chatID", message: "$message",
senderID: "$senderID", date: "$date", id: "$_id"}},
docs: {"$push": {type: "$type", id: "$_id"}}}},
{"$unwind": {path: "$data", includeArrayIndex: "data.inx"}},
{"$addFields": {"nextType": {$cond: [
{$lt: [{$add: ["$data.inx", 1]},{$size: "$docs"}]},
{"$arrayElemAt": ["$docs",{$add: ["$data.inx", 1]}]},
"NA"]},
"prevType": {$cond: [
{$gte: [{$add: ["$data.inx", -1]}, 0]},
{"$arrayElemAt": ["$docs", {$add: ["$data.inx", -1]}]},
"NA"]}}},
{"$project": {data: 1, nextType: "$all.nextType.type", prevType: "$all.prevType.type",
isfirstGroup: {$cond: [{$and: [{$ne: ["$data.type", "$prevType.type"]},
{$eq: ["$data.type", "groupedMsgs"]}]}, true, false]},
isLastGroup: {$cond: [{$and: [{$ne: ["$data.type", "$nextType.type"]},
{$eq: ["$data.type", "groupedMsgs"]}]}, true, false]}}},
{"$group": {_id: 0, starts: {$push: {isfirstGroup: "$isfirstGroup", origInx: "$data.inx"}},
ends: {$push: {isLastGroup: "$isLastGroup", origInx: "$data.inx",
data: "$data"}}}},
{$facet: {
groupedMsgs: [
{$project: {startItems: {$filter: {input: "$starts", as: "item",
cond: {$eq: ["$$item.isfirstGroup", true]}}},
endItems: {$filter: {input: "$ends", as: "item",
cond: {$eq: ["$$item.isLastGroup", true]}}}}},
{"$unwind": {path: "$endItems", includeArrayIndex: "endItems.inx"}},
{"$project": {data: "$endItems.data", maxInx: "$endItems.origInx",
minObj: {"$arrayElemAt": ["$startItems", "$endItems.inx"]}}},
{"$addFields": {"data.numConsecutiveItems": {"$subtract": [{$add: ["$maxInx", 1]},
"$minObj.origInx"]}}},
{"$replaceRoot": {newRoot: "$data"}}
],
others: [
{"$project": {items: {$filter: {input: "$ends", as: "item",
cond: {$ne: ["$$item.data.type", "groupedMsgs"]}}}}},
{"$unwind": "$items"},
{"$replaceRoot": {newRoot: "$items.data"}}
]
}
},
{"$project": {results: {$setUnion: ["$groupedMsgs", "$others"]}}},
{"$unwind": "$results"},
{"$project": {type: "$results.type", chatID: "$results.chatID", message: "$results.message",
senderID: "$results.senderID", date: "$results.date", id: "$results._id",
numConsecutiveItems: "$results.numConsecutiveItems"}},
{"$sort": {date: 1}},
])
You can check it on the playground: https://mongoplayground.net/p/1C5apUd4e5p .
Please notice I added one document to the playground to make things more clear

How to get this parent child query data in Mongo DB

I am building a simple messaging system and I have a messageSchema which is setup as below:
const messageSchema = new mongoose.Schema({
replyTo: {
type: mongoose.Schema.ObjectId,
ref: 'Message',
default: null,
},
name: {
type: Object,
required: [true, 'Message has a name'],
},
user: {
type: mongoose.Schema.ObjectId,
ref: 'User',
},
email: {
type: Object,
required: [true, 'Message has a email'],
},
subject: {
type: String,
required: [true, 'Message has a subject'],
},
content: {
type: String,
required: [true, 'Message has content'],
},
createdAt: {
type: Date,
default: Date.now(),
},
status: {
type: String,
default: 'unread',
required: [true, 'Message has a status'],
},});
Sample data below:
[
{
_id: ObjectId('53ed7efca75ca1a5248a281a'),
name: 'Person 1',
createdAt: ISODate('2021-01-01T01:00:00.000Z'),
subject: 'M1',
content: 'M1 content',
replyTo: null,
},
{
_id: ObjectId('53ed80bba75ca1a5248a281b'),
name: 'Person 2',
subject: 'M2 - Reply 1 to M1',
content: 'M2 content',
createdAt: ISODate('2021-01-01T02:00:00.000Z'),
replyTo: ObjectId('53ed7efca75ca1a5248a281a'),
},
{
_id: ObjectId('53ed80bba75ca1a5248a281c'),
name: 'Person 3',
subject: 'M3 - Reply 2 to M1',
content: 'M3 content',
createdAt: ISODate('2021-01-01T03:00:00.000Z'),
replyTo: ObjectId('53ed7efca75ca1a5248a281a'),
},
{
_id: ObjectId('53ed80bba75ca1a5248a281d'),
name: 'Person 4',
subject: 'M4',
content: 'M4 content',
createdAt: ISODate('2021-01-01T02:30:00.000Z'),
replyTo: null,
},
];
I am now trying to query the sample above to produce an inbox style response, so that the root message (no replyTo) is the top-level message, has a latest node with latest message info, and has children(if any) in a children node. See below for desired output.
[
{
_id: ObjectId('53ed7efca75ca1a5248a281a'),
name: 'Person 1',
createdAt: ISODate('2021-01-01T01:00:00.000Z'),
subject: 'M1',
content: 'M1 content',
replyTo: null,
latest: {
_id: ObjectId('53ed80bba75ca1a5248a281c'),
name: 'Person 3',
subject: 'M3 - Reply 2 to M1',
content: 'M3 content',
createdAt: ISODate('2021-01-01T03:00:00.000Z'),
replyTo: ObjectId('53ed7efca75ca1a5248a281a'),
},
children: [
{
_id: ObjectId('53ed80bba75ca1a5248a281b'),
name: 'Person 2',
subject: 'M2 - Reply 1 to M1',
content: 'M2 content',
createdAt: ISODate('2021-01-01T02:00:00.000Z'),
replyTo: ObjectId('53ed7efca75ca1a5248a281a'),
},
{
_id: ObjectId('53ed80bba75ca1a5248a281c'),
name: 'Person 3',
subject: 'M3 - Reply 2 to M1',
content: 'M3 content',
createdAt: ISODate('2021-01-01T03:00:00.000Z'),
replyTo: ObjectId('53ed7efca75ca1a5248a281a'),
},
]
},
{
_id: ObjectId('53ed80bba75ca1a5248a281d'),
name: 'Person 4',
subject: 'M4',
content: 'M4 content',
createdAt: ISODate('2021-01-01T02:30:00.000Z'),
replyTo: null,
latest: {
_id: ObjectId('53ed80bba75ca1a5248a281d'),
name: 'Person 4',
subject: 'M4',
content: 'M4 content',
createdAt: ISODate('2021-01-01T02:30:00.000Z'),
replyTo: null,
},
children: []
},];
Appreciate any help in getting this query sorted. Thanks.
You can use aggregation pipeline stages,
$match replyTo us null condition
$graphLookup to join same collection to get reply messages in children
$addFields to check condition is children is empty then return root document otherwise return last element from children
db.collection.aggregate([
{ $match: { replyTo: null } },
{
"$graphLookup": {
"from": "collection",
"startWith": "$_id",
"connectFromField": "_id",
"connectToField": "replyTo",
"as": "children"
}
},
{
$addFields: {
latest: {
$cond: [
{ $eq: [{ $size: "$children" }, 0] },
"$$ROOT",
{ $arrayElemAt: [{$slice: ["$children", -1]}, 0] }
]
}
}
}
])
Playground
For query optimization you can do last stage $addFields process in your client side.

Populate data in mongoose on collections(generated by strapi.io)

I have 3 collections in mongodb generated by strapi.io, I want customize query populate data by mongoose in my project. But i cannot populate data like strapi result.
Category collection:
{
_id: "5d10a731c5077836540bebf0",
posts: [
"5d10a6fbc5077836540bebed",
"5d10af45c5077836540bebf1"
],
name: "Support",
ids: "support",
keywords: "support",
des: "support des",
createdAt: "2019-06-24T10:34:25.383Z",
updatedAt: "2019-06-24T11:08:53.475Z",
__v: 0,
id: "5d10a731c5077836540bebf0"
}
Post collection:
{
_id: "5d10af45c5077836540bebf1",
ids: "test",
title: "this is test",
des: "test",
keywords: "test",
body: "testtesttesttesttest",
createdAt: "2019-06-24T11:08:53.467Z",
updatedAt: "2019-06-24T11:08:53.477Z",
__v: 0,
id: "5d10af45c5077836540bebf1"
}
FileUpload collection:
{
_id: "5d10af45c5077836540bebf2",
name: "THUMBNAIL.png",
sha256: "0bP9PI3R_ygF07cLhg5U-syLeVCz4ZbBrwZZi2OtPL4",
hash: "9f4e8bbd13b94f2baa3d26b335124717",
ext: ".png",
mime: "image/png",
size: "297.33",
url: "/uploads/9f4e8bbd13b94f2baa3d26b335124717.png",
provider: "local",
related:
[
{
_id: "5d10af45c5077836540bebf3",
ref: "5d10af45c5077836540bebf1",
kind: "Post",
field: "pic"
}
],
createdAt: "2019-06-24T11:08:53.501Z",
updatedAt: "2019-06-24T11:08:53.505Z",
__v: 0,
id: "5d10af45c5077836540bebf2"
}
I can populate "posts" on Category model with
Cat.find().populate({path: 'posts',model: 'Post'})
But I can not populate "categories" and "pic" on Post model.
How can I populate "categories" and "pic" like this result of strapi:
{
_id: "5d10af45c5077836540bebf1",
ids: "test",
title: "this is test",
des: "test",
keywords: "test",
body: "testtesttesttesttest",
createdAt: "2019-06-24T11:08:53.467Z",
updatedAt: "2019-06-24T11:08:53.477Z",
__v: 0,
id: "5d10af45c5077836540bebf1",
pic: {
_id: "5d10af45c5077836540bebf2",
name: "THUMBNAIL.png",
sha256: "0bP9PI3R_ygF07cLhg5U-syLeVCz4ZbBrwZZi2OtPL4",
hash: "9f4e8bbd13b94f2baa3d26b335124717",
ext: ".png",
mime: "image/png",
size: "297.33",
url: "/uploads/9f4e8bbd13b94f2baa3d26b335124717.png",
provider: "local",
related: [
"5d10af45c5077836540bebf1"
],
createdAt: "2019-06-24T11:08:53.501Z",
updatedAt: "2019-06-24T11:08:53.505Z",
__v: 0,
id: "5d10af45c5077836540bebf2"
},
categories:
[
{
posts: [
"5d10a6fbc5077836540bebed",
"5d10af45c5077836540bebf1"
],
_id: "5d10a731c5077836540bebf0",
name: "Support",
ids: "support",
keywords: "support",
des: "support des",
createdAt: "2019-06-24T10:34:25.383Z",
updatedAt: "2019-06-24T11:08:53.475Z",
__v: 0,
id: "5d10a731c5077836540bebf0"
}
]
}
you have to create another collection inside the post. as like you have an independent collection then you can populate. for example :
Post: {body: ,title: ,Image : { image: {name : , etc }}}
this can be a collection inside collection. the above code is not correct you have to mention your own fields.

Mongoose findbyid / find({id: "..."}) returning Null

I have seen this asked several times, but I haven't found a solution that has worked. I am trying to query a MongoDB using Mongoose .findById and am not getting the expected results. find({id: "..."}) is also not returning anything.
Using find without any parameters displays all of the expected results including id key-value pairs, but I cannot use the id value in the query.
Using mongoose: 5.4.9
const mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/mongo-exercises', { useNewUrlParser: true });
const courseSchema = new mongoose.Schema({
name: String,
author: String,
tags: [String],
date: { type: Date, default: Date.now },
isPublished: Boolean,
price: Number
});
const Course = mongoose.model('Course', courseSchema);
async function getCourses() {
return await Course
.find()
}
async function run() {
const result = await getCourses();
console.log(result);
}
run();
//Return
[ { tags: [ 'react', 'frontend' ],
_id: 5a68fdf95db93f6477053ddd,
date: 2018-01-24T21:43:21.589Z,
name: 'React Course',
author: 'Mosh',
isPublished: false,
__v: 0 },
{ tags: [ 'aspnet', 'backend' ],
_id: 5a68fde3f09ad7646ddec17e,
date: 2018-01-24T21:42:59.605Z,
name: 'ASP.NET MVC Course',
author: 'Mosh',
isPublished: true,
price: 15,
__v: 0 },
{ tags: [ 'node', 'backend' ],
_id: 5a68fe2142ae6a6482c4c9cb,
date: 2018-01-24T21:44:01.075Z,
name: 'Node.js Course by Jack',
author: 'Jack',
isPublished: true,
price: 12,
__v: 0 },
{ tags: [ 'node', 'backend' ],
_id: 5a68fdd7bee8ea64649c2777,
date: 2018-01-24T21:42:47.912Z,
name: 'Node.js Course',
author: 'Mosh',
isPublished: true,
price: 20,
__v: 0 },
{ tags: [ 'node', 'backend' ],
_id: 5a68ff090c553064a218a547,
date: 2018-01-24T21:47:53.128Z,
name: 'Node.js Course by Mary',
author: 'Mary',
isPublished: false,
price: 12,
__v: 0 },
{ tags: [ 'angular', 'frontend' ],
_id: 5a6900fff467be65019a9001,
date: 2018-01-24T21:56:15.353Z,
name: 'Angular Course',
author: 'Mosh',
isPublished: true,
price: 15,
__v: 0 },
{ tags: [ 'express', 'backend' ],
_id: 5a68fdc3615eda645bc6bdec,
date: 2018-01-24T21:42:27.388Z,
name: 'Express.js Course',
author: 'Mosh',
isPublished: true,
price: 10,
__v: 0 } ]
That code verifies I'm connected to the correct database and retrieving real ids. When I modify the getCourses function as shown below, I get null or an empty array depending on whether I use findById or find({id: "..."}).
async function getCourses() {
return await Course
.findById('5a68fdf95db93f6477053ddd')
}
//null
async function getCourses() {
return await Course
.find({ id: '5a68fdf95db93f6477053ddd' })
}
// []
async function getCourses() {
return await Course
.find({ _id: '5a68fdf95db93f6477053ddd' })
}
// []
Any help would be greatly appreciated. Thank you!
EDIT: Showing full find() response.
Upon our discussion as your importing the data from JSON file, So its inserting the _id as string, While finding the document by _id, moognoose automatically converting string to Object.
// Original JSON
{
"_id": "5a68fde3f09ad7646ddec17e",
"tags": [
"aspnet",
"backend"
],
"date": "2018-01-24T21:42:59.605Z",
"name": "ASP.NET MVC Course",
"author": "Mosh",
"isPublished": true,
"price": 15,
"__v": 0
}
Solution 1 :
To insert the _id as in the JSON, change your _id field with $oid as below,
{
"_id": { "$oid": "5a68fde3f09ad7646ddec17e" },
"tags": [
"aspnet",
"backend"
],
"date": "2018-01-24T21:42:59.605Z",
"name": "ASP.NET MVC Course",
"author": "Mosh",
"isPublished": true,
"price": 15,
"__v": 0
}
Solution 2 :
Remove _id from your JSON, MongoDB will generate a _id automatically
{
"tags": [
"aspnet",
"backend"
],
"date": "2018-01-24T21:42:59.605Z",
"name": "ASP.NET MVC Course",
"author": "Mosh",
"isPublished": true,
"price": 15,
"__v": 0
}

MongoDB: multiple $elemMatch

I have MongoDB documents structured like this:
{_id: ObjectId("53d760721423030c7e14266f"),
fruit: 'apple',
vitamins: [
{
_id: 1,
name: 'B7',
usefulness: 'useful',
state: 'free',
}
{
_id: 2,
name: 'A1',
usefulness: 'useful',
state: 'free',
}
{
_id: 3,
name: 'A1',
usefulness: 'useful',
state: 'non_free',
}
]
}
{_id: ObjectId("53d760721423030c7e142670"),
fruit: 'grape',
vitamins: [
{
_id: 4,
name: 'B6',
usefulness: 'useful',
state: 'free',
}
{
_id: 5,
name: 'A1',
usefulness: 'useful',
state: 'non_free',
}
{
_id: 6,
name: 'Q5',
usefulness: 'non_useful',
state: 'non_free',
}
]
}
I want to query and get all the fruits which have both {name: 'A1', state: 'non_free'} and {name: 'B7', state: 'free'}.
In the worst case I want at least to count these entries if getting them is not possible and if the equivalent code exists for pymongo, to know how to write it.
For the given example I want to retrieve only the apple (first) document.
If I use $elemMatch it works only for one condition, but not for both. E.g. if I query find({'vitamins': {'$elemMatch': {'name': 'A1', 'state': 'non_free'}, '$elemMatch': {'name': 'B7', 'state': 'free'}}}) it will retrieve all the fruits with {'name': 'B7', 'state': 'free'}.
In this case you can use the $and-operator .
Try this query:
find({
$and: [
{'vitamins': {'$elemMatch': {'name': 'A1', 'state': 'non_free'} } },
{'vitamins': {'$elemMatch': {'name': 'B7', 'state': 'free'} } }
]
});
To explain why you received only the result matching the second criteria: The objects inside each {} you pass to MongoDB are key/value pairs. Each key can only exist once per object. When you try to assign a value to the same key twice, the second assignment will override the first. In your case you assigned two different values to the key $elemMatch in the same object, so the first one was ignored. The query which actually arrived in MongoDB was just find({'vitamins': {'$elemMatch': {'name': 'B7', 'state': 'free'}}}).
Whenever you need to apply the same operator to the same key twice, you need to use $or or $and.
var fruits = db.fruits.find({
"vitamins": {
$all: [{
$elemMatch: {
"name": "A1",
"state": "non_free"
}
}, {
$elemMatch: {
"name": "B7",
"state": "free"
}
}]
}
})
let query = [];
query.push({
id: product.id,
});
query.push({ date });
for (const slot of slots) {
query.push({
slots: {
$elemMatch: {
id: slot.id,
spots: { $gte: slot.spots },
},
},
});
}
const cal = await Product.findOne({ $and: query });