I'm new to MongoDB and probably this is a common use case, but I didn't find an answer to this specific case.
I would like to filter out those MongoDB documents where a timeslot in booking is still free, so there is no overlapping with existing bookings.
My collection entries:
{
"_id": "5efc9f89749c983ffd58c55f",
"training": "football",
"bookings": [
{
"from": 1593607419,
"to": 1593622800
}
]
},
{
"_id": "5efc9f89749c983ffd58c55f",
"training": "baseball",
"bookings": [
{
"from": 1593607419,
"to": 1593622800
},
{
"from": 1593687419,
"to": 1593722800
}
]
}
Expected result with input "From: 1593500000" and "To: 1593600000".
{
"_id": "5efc9f89749c983ffd58c55f",
"training": "football"
},
{
"_id": "5efc9f89749c983ffd58c55f",
"training": "baseball"
}
Both documents have free timeslots (no overlapping with existing bookings).
Expected result with input "From: 1593670000" and "To: 1593690000".
{
"_id": "5efc9f89749c983ffd58c55f",
"training": "football"
}
Only football is returned, because the queried values overlaps with the second booking entry of baseball.
Is it possible to do this with a single query? For example using between or something like that?
Or is there a better approach/best practice?
I implement the query with Spring Data REST, but just some help with the mongodb query would already be very helpful.
This is more of a logical problem, I'm not familiar with Spring Data, your query should look like this.
{
bookings: {
$not: {
$elemMatch: {
$or: [{
from: { $gte: start, $lte: end }
}, {
to: { $gte: start, $lte: end }
}]
}
}
}
}
Mongo Playground
Related
I'm trying to build a complex, nested aggregation pipeline in MongoDB (4.4.9 Community Edition, using the pymongo driver for Python 3.10).
There are relevant data points in different collections which I want to aggregate into one, NEW (ideally) view (or, if that doesn't work) collection.
The collections, and the relevant fields therein follow a hierarchy. There is members, which contains the top-level key on which other data is to be merged,
membershipNumber.
> members.find_one()
{'_id': ObjectId('61153299af6122XXXXXXXXXXXXX'), 'membershipNumber': 'N03XXXXXX'}
Then, there's a different collection, which contains membershipNumber, but also a different, linked field, an_user_id. an_user_id is used in other collections to denote records/fields in arrays that pertain to that particular user.
I 'join' members and an_users like so:
result = members.aggregate([
{
'$lookup': {
'from': 'an_users',
'localField': 'membershipNumber',
'foreignField': 'memref',
'as': 'an_users'
}
},
{ '$unwind' : '$an_users' },
{
'$project' : {
'_id' : 1,
'membershipNumber' : 1,
'an_user_id' : '$an_users.user_id'
}
}
]);
So far so good, this returns the desired, aggregated record:
{'_id': ObjectId('61153253aBBBBBBBBBBBB'),
'membershipNumber': 'N0XXXXXXXX',
'an_user_id': '48XXXXXX'}
Now, I have a third collection, which contains the an_user_id as a string in arrays, denoting wherever that user clicked a given email, whereby a record is an email (and the an_user_ids in the clicks array are users that clicked a link in that email.
{'_id': ObjectId('blah'),
'email_id': '407XXX',
'actions_count': 17,
'administrative_title': 'test',
'bounce': ['3440XXXX'],
'click': ['38294CCC',
'418FFFF',
'48XXXXXX',
'38eGGGG'}
I want to count the number occurences of a given an_user_id (which I've attained from aggregating) in arrays (e.g. clicks, bounces, opens) in the emails collection, and include it in the .aggregate call, to retrieve something like this:
{'_id': ObjectId('61153253aBBBBBBBBBBBB'),
'membershipNumber': 'N0XXXXXXXX',
'an_user_id': '48XXXXXX',
'n_email_clicks' : 412,
'n_email_bounces' : 12
}
Further, I might want to also attach counts of an_user_id in other collections in my DB.
Consider, e.g., this collection called events:
{
"_id": "617ffa96ee11844e143a63dd",
"id": "12345",
"administrative_title": "my_event",
"created_at": {
"$date": "2020-01-15T16:28:50.000Z"
},
"event_creator_id": "123456",
"event_title": "my_event",
"group_id": "123456",
"permalink": "event_id",
"rsvp_count": 54,
"rsvps": [{
"rsvp_id": "56789",
"display_name": "John Doe",
"rsvp_user_id": "48XXXXXX",
"rsvp_created_at": {
"$date": "2020-01-28T15:38:50.000Z"
},
"rsvp_updated_at": {
"$date": "2020-01-28T15:38:50.000Z"
},
"first_name": "John",
"last_name": "Doe",
}, {
"rsvp_id": "543895",
"display_name": "James Appleslice",
"rsvp_user_id": "N03XXXXXX",
"rsvp_created_at": {
"$date": "2020-02-05T13:15:14.000Z"
},
"rsvp_updated_at": {
"$date": "2020-02-05T13:15:14.000Z"
},
"first_name": "James",
"last_name": "Appleslice"}
]
}
So, the end-product would look something like this:
{'_id': ObjectId('61153253aBBBBBBBBBBBB'),
'membershipNumber': 'N0XXXXXXXX',
'an_user_id': '48XXXXXX',
'n_email_clicks' : 412,
'n_email_bounces' : 12,
'n_rsvps' : 12
}
My idea was to use the $lookup parameter -- however, I only know how to use this for matching on fields that I have in the parent collection that I'm performing the aggregation on, but not on fields that have been generated in the process of the aggregation.
Any help would be hugely appreciated!
You could use $lookup pipeline. First you would $lookup the user id followed by another $lookup to verify if the user id exists in email. Lastly few more stages to collect the results and format per your need. Furthermore, you can add $out stage if you would like to write the results into another collection.
db.members.aggregate([{
$lookup: {
from: "an_users",
let: {
membershipNumber: "$membershipNumber"
},
pipeline: [
{
$match: {
$expr: {
$eq: [
"$memref",
"$$membershipNumber"
]
},
}
},
{
"$lookup": {
"from": "emails",
"localField": "user_id",
"foreignField": "click",
"as": "clicks"
}
},
{
"$project": {
"_id": 1,
"membershipNumber": 1,
"an_user_id": "$user_id",
"n_email_clicks": {
$size: "$clicks"
}
}
}
],
as: "details"
}
},
{
$replaceRoot: {
newRoot: {
$mergeObjects: [
{
$arrayElemAt: [
"$details",
0
]
},
"$$ROOT"
]
}
}
},
{
$project: {
details: 0
}
}])
Working example - https://mongoplayground.net/p/yrFsNp44hpi
Stack Overflow!
I have a very strange problem with using $elemMatch in MongoDB. I added multiple documents to a collection. Some of these documents were added using import feature in MongoDB Compass (Add Data -> Import File -> JSON) and some of them were added using insertMany().
Here is an example structure of a single document:
{
"id": "1234567890",
"date": "YYYY-MM-DD",
"contents": {
"0": {
"content": {
"id": "1111111111",
"name": "Name 1"
}
},
"1": {
"content": {
"id": "2222222222",
"name": "Name 2"
}
},
"2": {
"content": {
"id": "3333333333",
"name": "Name 3"
}
}
}
}
The thing is, when I use the following find query using this filter:
{date: "<some_date_here>", "contents": {
$elemMatch: {
"content.id": <some_id_here>
}
}}
ONLY documents that were imported from MongoDB Compass are showing up. Documents that were added by Mongosh or by NodeJS driver (doesn't matter), do NOT show up.
Am I missing something obvious here? What should I do in order to make all documents in a collection (that matches filter) to show up?
Simple filters that do not include $elemMatch work well and all documents that match the filtering rules show up. Problem seems to be with $elemMatch.
I tried adding the same batch of documents using different methods but only direct importing a JSON file in MongoDB Compass make them appear using a filter mentioned above.
Thank you for your help!
$elemMatch if for matching array , and in this case you don't have array
first you should convert contents object to array and then check the query for example id with filter and use match to find all doc that have specific data and size of new filters array
db.collection.aggregate([
{
"$addFields": {
"newField": {
"$objectToArray": "$contents"
}
}
},
{
"$addFields": {
"newField": {
"$filter": {
"input": "$newField",
"as": "z",
"cond": {
$eq: [
"$$z.v.content.id",
"1111111111"
]
}
}
}
}
},
{
"$addFields": {
"newField": {
$size: "$newField"
}
}
},
{
$match: {$and:[ {newField: {
$gt: 0
}},{date:{$gt:Date}}]}
},
{$project:{
contents:1,
date:1,
id:1,
}}
])
https://mongoplayground.net/p/pue4QPp1dYR
in mongoplayground I don't add filter of date
I have two collections userProfile and skills,
Eg:userProfile
{
"_id": "5f72c6d4e23732390c96b031",
"name":"name"
"other_skills": [
"1","2"
],
"primary_skills": [
{
"_id": "607ffd1549e13876fef7f2c5",
"years": 4.5,
"skill_id": "1"
},
{
"_id": "607ffd1549e13876fef7f2c6",
"years": 2,
"skill_id": "2"
},
{
"_id": "607ffd1549e13876fef7f2c7",
"years": 1,
"skill_id": "3"
}
]
}
Eg:Skills
{
"_id":1,
"name": "Ruby on Rails",
}
{
"_id":2,
"name": "PHP",
}
{
"_id":3,
"name": "php",
}
I want to retrieve the userprofile based on the skills
eg: input of skill php i want to retrieve the userprofiles that matches either in primary_skills or other_skills
But I got confused about the implementation, I think it can do with pipeline in lookup and the elemMatch. This is the query I tried so far
const skills = ['php','PHP']
userProfile.aggrigate([{
$lookup:{
from:'skills',
let:{'primary_skills':'$primary_skills'},
pipeline:[
{
$match:{
primary_skills:{
$elemMatch:{
name:'' //not sure how to write match
}
}
}
}
]
}
}])
Can somebody help me with this, Thanks in advance
I'll first show you how to correct your pipeline to work, however this approach is very inefficient as you will have to $lookup on every single user in your db which is obviously a lot of overhead.
Here is how to properly match your condition:
const skills = ['php','PHP']
db.userProfile.aggregate([
{
$lookup: {
from: "skills",
let: {
"primary_skills": {
$map: {
input: "$primary_skills",
as: "skill",
in: "$$skill.skill_id"
}
},
"other_skills": "$other_skills"
},
pipeline: [
{
$match: {
$expr: {
"$in": [
"$_id",
{
"$concatArrays": [
"$$other_skills",
"$$primary_skills"
]
}
]
}
}
}
],
as: "skills"
}
},
{
$match: {
'skills.name': {$in: skills}
}
}
])
Mongo Playground
As I've said I recommend you do not do this. what I suggest you do is split it into 2 calls, first fetch the relevant skill ids. and then query on users.
By doing this you can also utilize indexes for much faster queries, like so:
const skills = ['php', 'PHP'];
const matchedSkillIds = await skills.distinct('_id', {name: {$in: skills}});
const users = await userProfile.find({
$or: [
{
'primary_skills.skill_id': {$in: matchedSkillIds}
},
{
'other_skills': {$in: matchedSkillIds}
}
]
})
Finally if you do insist on doing it in one query at the very least start the pipeline from the skill collection.
I have a collection that consists of objects like this:
{
"name": "Event One",
"user_id": "1"
"rsvp": [
{"id": "1","name":"John Doe","industry":"Software"},
{"id": "2","name":"John Doe II","industry":"Software"},
]
}
I am trying to query all events that a user has rsvp too and that he did not create. Then match the rsvp with the industry the are in. Is it possible to query in mongoDB and have it returned like this:
[
{"id": "1","name":"John Doe"},
{"id": "2","name":"John Doe II"},
]
The query I have is below:
events.find({"$and":[{"rsvp.contact.indusrty":{"$in":["Software","Information Systems"]}},{"user_id":{"$ne":"5d335704802df000076bad97"}}]},{"projection":{"rsvp.contact.name":true},"typeMap":{"root":"array","document":"array"}})
I am new to MONGODB and can't seem to find anything that is helping me figure this out.
You need $match to define filtering criteria, $unwind to get single document per rsvp and $replaceRoot to promote rsvp to root model:
db.events.aggregate([
{
$match: {
"$and":[
{
"rsvp.industry":{
"$in":["Software","Information Systems"]
}
},
{ "user_id":{"$ne":"5d335704802df000076bad97"}}
]
}
},
{
$unwind: "$rsvp"
},
{
$replaceRoot: {
newRoot: "$rsvp"
}
},
{
$project: {
industry: 0
}
}
])
Mongo Playground
I'm very new to MongoDb so I'm used to SQL.
Right now I have two collections in my database:
1) Series (which has nested subdocuments)
2) Review (decided to reference to episode subdocument because there will be a lot of reviews)
See this picture for a better understanding.
Now I want to achieve te following. For every review (two in this case), I want to get the episode name.
I tried the following:
db.review.aggregate([
{
$lookup:{
from:"series",
localField:"episode",
foreignField:"seasons.episodes._id",
as:"episode_entry"
}
}
]).pretty()
The problem is that this returns (ofcourse) not only the title of the referenced episode, but it returns the whole season document.
See the picture below for my current output.
I don't know how to achieve it. Please help me.
I'm using Mongo 3.4.9
I would recommend the following series structure which unwinds the season array into multiple documents one for each season.
This will help you with inserting/updating the episodes directly.
Something like
db.series.insertMany([
{
"title": "Sherlock Holmes",
"nr": 1,
"episodes": [
{
"title": "A Study in Pink",
"nr": 1
},
{
"title": "The Blind Banker",
"nr": 2
}
]
},
{
"title": "Sherlock Holmes",
"nr": 2,
"episodes": [
{
"title": "A Scandal in Belgravia",
"nr": 1
},
{
"title": "The Hounds of Baskerville",
"nr": 2
}
]
}
])
The lookup query will do something like this
episode: { $in: [ episodes._id1, episodes._id2, ... ] }
From the docs
If the field holds an array, then the $in operator selects the
documents whose field holds an array that contains at least one
element that matches a value in the specified array (e.g. , , etc.)
So lookup will return all episodes when there is a match. You can then filter to keep only the one matching your review's episode.
So the query will look like
db.review.aggregate([
{
"$lookup": {
"from": "series",
"localField": "episode",
"foreignField": "episodes._id",
"as": "episode_entry"
}
},
{
"$addFields": {
"episode_entry": [
{
"$arrayElemAt": {
"$filter": {
"input": {
"$let": {
"vars": {
"season": {
"$arrayElemAt": [
"$episode_entry",
0
]
}
},
"in": "$$season.episodes"
}
},
"as": "result",
"cond": {
"$eq": [
"$$result._id",
"$episode"
]
}
}
}
},
0
]
}
}
])