I want to find products and for each product attach deals to it. A deal is a product from same collection, yet based on some common properties.
So as per my requirement pipeline should return documents, for each document find other products those aren't same as current, but have equal detail.duration. But even though I've many docs with same duration, deals are always []. Could you please figure out the issue with my pipeline?
Following is the aggregation pipeline I'm running:
I've added filter _id $in just for clarity based on shown documents below. This isn't a part of real pipeline $match query.
db.products
.aggregate([
{
$match: {
_id: {
$in: [
ObjectId("6210fa8746bee3fcbd0ad062"),
ObjectId("6210fa7c46bee3fcbd0acc21"),
],
},
"detail.duration": { $gt: 0 },
},
},
{
$lookup: {
from: "products",
let: { id: "$_id", duration: "$detail.duration" },
as: "deals",
pipeline: [
{
$match: {
_id: { $ne: "$id" },
"detail.duration": "$duration",
},
},
{ $project: { detail: 1 } },
{ $limit: 1 },
],
},
},
{ $limit: 2 },
{ $project: { deals: 1 } },
])
.pretty();
This was the result:
{ "_id" : ObjectId("6210fa7c46bee3fcbd0acc21"), "deals" : [ ] }
{ "_id" : ObjectId("6210fa8746bee3fcbd0ad062"), "deals" : [ ] }
Following are two example documents in the collection:
{
"_id" : ObjectId("6210fa8746bee3fcbd0ad062"),
"book" : "https://wegotrip.com/en/paris-d3/muse-d-orsay-and-musee-de-l-orangerie-combined-tour-ticket-p1117/?SUB_ID=336264",
"address" : "Rue de Lille, 62bis",
"countryName" : "France",
"cityName" : "Paris",
"location" : {
"lang" : 48.859886,
"lat" : 2.3254821,
"country" : ObjectId("6210fa7746bee3fcbd0aca20"),
"city" : ObjectId("6210fa7746bee3fcbd0aca1c"),
"location" : "Rue de Lille, 62bis",
"_id" : ObjectId("6210fa8746bee3fcbd0ad063")
},
"includes" : [
{
"value" : "Skip-the-line ticket to Orsay Museum",
"included" : true
},
{
"value" : "Skip-the-line ticket to the Musée de l'Orangerie",
"included" : true
},
{
"value" : "Detailed description of the Nymphéas from Claude Monet",
"included" : true
},
{
"value" : "Interesting stories of many great artists and their lives",
"included" : true
},
{
"value" : "An easy walkthrough of the Musée d'Orsay and the Musée de l'Orangerie and their great collection",
"included" : true
},
{
"value" : "Headphones — you should bring your own",
"included" : false
}
],
"price" : {
"priceConcession" : null,
"priceChild" : null,
"price" : 57,
"currency" : ObjectId("6210fa7746bee3fcbd0aca2f"),
"_id" : ObjectId("6210fa8746bee3fcbd0ad064")
},
"detail" : {
"isPass" : false,
"features" : [
{
"key" : "audio_guide",
"value" : "Audio Guide"
}
],
"highlights" : [
"Admire the masterpieces by Monet, Renoir, Degas, Cézanne, and many more",
"Discover one of the finest collections of Impressionist art in the world",
"Visit the Nymphéas by Monet, one of the greatest pieces of Impressionism",
"Explore the Guillaume and Walter collection and find out what makes it unique"
],
"details" : [ ],
"images" : [
{
"id" : 7270,
"description" : "",
"cover" : false,
"preview" : "https://app.wegotrip.com/media/CACHE/images/store/1117/dsc04800/01d0770dcc0cac4c6de0f6eae70742f6.jpg",
"full" : "https://app.wegotrip.com/media/store/1117/dsc04800.jpg"
},
{
"id" : 7269,
"description" : "",
"cover" : false,
"preview" : "https://app.wegotrip.com/media/CACHE/images/store/1117/nympheasannees30salle1parisiennephotorogerviolet/e1270aef1c01391290df71d1f83c8abc.jpg",
"full" : "https://app.wegotrip.com/media/store/1117/nympheasannees30salle1parisiennephotorogerviolet.jpg"
},
{
"id" : 7268,
"description" : "",
"cover" : false,
"preview" : "https://app.wegotrip.com/media/CACHE/images/store/1117/ob1f7c80dsc02414-large/7712cb29e133ee3acb4b2bffbc2ac654.jpg",
"full" : "https://app.wegotrip.com/media/store/1117/ob1f7c80dsc02414-large.jpg"
},
{
"id" : 7267,
"description" : "",
"cover" : false,
"preview" : "https://app.wegotrip.com/media/CACHE/images/store/1117/tuileriesgardensb16dsc00678talrg/47430ab8a257e3ccd2337d7a0d750c57.jpg",
"full" : "https://app.wegotrip.com/media/store/1117/tuileriesgardensb16dsc00678talrg.jpg"
},
{
"id" : 7266,
"description" : "",
"cover" : false,
"preview" : "https://app.wegotrip.com/media/CACHE/images/store/1117/009/54223ef27aac5cd94fe5c20893abf2de.jpg",
"full" : "https://app.wegotrip.com/media/store/1117/009.jpg"
},
{
"id" : 7264,
"description" : "",
"cover" : false,
"preview" : "https://app.wegotrip.com/media/CACHE/images/store/1117/monet-morning-with-weeping-willow/09bf842cc9a9d7eade8d0739f704699f.jpg",
"full" : "https://app.wegotrip.com/media/store/1117/monet-morning-with-weeping-willow.jpg"
}
],
"duration" : 2,
"_id" : ObjectId("6210fa8746bee3fcbd0ad065")
},
"availability" : null,
"subcategory" : [
{
"id" : 6,
"title" : "Sightseeing Tickets & Passes",
"slug" : "sightseeing-tickets-passes"
}
],
"category" : [
{
"id" : 6,
"title" : "Sightseeing Tickets & Passes",
"slug" : "sightseeing-tickets-passes"
}
],
"type" : "Audio Guide",
"description" : "Visit the famous Musee d'Orsay and Musée de l'Orangerie in Paris with this combined self-guided tour! \r\n\r\nNavigate through the maze of exhibition rooms with mobile app and see a collection of works by the Impressionists and Expressionists – Seurat, Cezanne, Gaugin, Monet, Renoir, Manet, Van Gogh, Degas; sculptors like Rodin, Pompon and others. Check out a mini-version of the Statue of Liberty! \r\n\r\nExplore the Nymphéas paintings by Claude Monet, that is called \"the Sistine chapel of Impressionism\". Admire the great works of Picasso, Soutine, Rousseau, Matisse and many others part of the Paul Guillaume and Jean Walter collection. Learn about the style and private life of the artists.\r\n\r\nThe audio-guide will provide you with all the information on the cultural significance of these paintings. Walking through rooms you will understand how revolutionary for those times Manet’s, Cezanne’s and Degas’ creation really was casting doubts on conservative, academic conceptions of 'true art' and offering new techniques and ideas.",
"thumbnail" : "https://app.wegotrip.com/media/CACHE/images/store/1117/013/c0b8cce52cb61ab1f30872e6e93385b4.jpg",
"name" : "Musée d'Orsay/Musée de l'Orangerie Combined Admission Ticket & Audio Tour",
"attractionDescription" : "",
"attractionName" : "Musée d'Orsay & Musée de l'Orangerie",
"attraction" : ObjectId("6210fa8746bee3fcbd0ad056"),
"provider" : {
"rating" : {
"count" : 0,
"average" : null,
"_id" : ObjectId("6210fa8746bee3fcbd0ad067")
},
"preview" : "https://app.wegotrip.com/media/CACHE/images/store/1117/013/c0b8cce52cb61ab1f30872e6e93385b4.jpg",
"slug" : "muse-d-orsay-and-musee-de-l-orangerie-combined-tour-ticket",
"id" : "1117",
"key" : "1",
"_id" : ObjectId("6210fa8746bee3fcbd0ad066")
},
"__v" : 0
}
{
"_id" : ObjectId("6210fa7c46bee3fcbd0acc21"),
"book" : "https://wegotrip.com/en/barcelona-d1/the-dali-museum-in-figueres-p3/?SUB_ID=336264",
"address" : "Pujada del Castell, 43",
"countryName" : "Spain",
"cityName" : "Barcelona",
"location" : {
"lang" : 42.26829425831263,
"lat" : 2.95884132385254,
"country" : ObjectId("6210fa7746bee3fcbd0aca3e"),
"city" : ObjectId("6210fa7746bee3fcbd0aca3a"),
"location" : "Pujada del Castell, 43",
"_id" : ObjectId("6210fa7c46bee3fcbd0acc22")
},
"includes" : [
{
"value" : "Recommendations of places to visit to understand the life of Dali better",
"included" : true
},
{
"value" : "Skip-the-line ticket to Dali Theatre-Museum",
"included" : true
},
{
"value" : "Headphones — you should bring your own",
"included" : false
}
],
"price" : {
"priceConcession" : null,
"priceChild" : null,
"price" : 33,
"currency" : ObjectId("6210fa7746bee3fcbd0aca2f"),
"_id" : ObjectId("6210fa7c46bee3fcbd0acc23")
},
"detail" : {
"isPass" : false,
"features" : [
{
"key" : "audio_guide",
"value" : "Audio Guide"
}
],
"highlights" : [
"Discover Dali's surrealism starting with the building of the museum — it's definitely one of a kind",
"Inside the museum you'll find the most famous and controversial works of the artist",
"Our tour will provide you with insights and exiting facts about Dali's works"
],
"details" : [ ],
"images" : [
{
"id" : 6916,
"description" : "",
"cover" : false,
"preview" : "https://app.wegotrip.com/media/CACHE/images/store/3/figueres-oleguer2/032b55c27bb2cd119bdc7fe6c4b86491.jpeg",
"full" : "https://app.wegotrip.com/media/store/3/figueres-oleguer2.jpeg"
},
{
"id" : 6915,
"description" : "",
"cover" : false,
"preview" : "https://app.wegotrip.com/media/CACHE/images/store/3/sky-monument-statue-golden-museum-yellow-1156442-pxherecom/28c645449a9f45ec1e8ede7b7ffbe30f.jpg",
"full" : "https://app.wegotrip.com/media/store/3/sky-monument-statue-golden-museum-yellow-1156442-pxherecom.jpg"
},
{
"id" : 6914,
"description" : "",
"cover" : false,
"preview" : "https://app.wegotrip.com/media/CACHE/images/store/3/architecture-window-museum-landmark-surrealism-catalonia-800928-pxherecom/43691ba6aecc2ee084c300c150e32a03.jpg",
"full" : "https://app.wegotrip.com/media/store/3/architecture-window-museum-landmark-surrealism-catalonia-800928-pxherecom.jpg"
},
{
"id" : 831,
"description" : "",
"cover" : false,
"preview" : "https://app.wegotrip.com/media/CACHE/images/store/3/figueres-oleguers3k6yoz/b9c3093c79cf50e621e022706af59ad6.jpg",
"full" : "https://app.wegotrip.com/media/store/3/figueres-oleguers3k6yoz.jpg"
},
{
"id" : 832,
"description" : "",
"cover" : false,
"preview" : "https://app.wegotrip.com/media/CACHE/images/store/3/shutterstock82210018/2a2450d4f75edf4549d36f2286b6f19b.jpg",
"full" : "https://app.wegotrip.com/media/store/3/shutterstock82210018.jpg"
},
{
"id" : 833,
"description" : "",
"cover" : false,
"preview" : "https://app.wegotrip.com/media/CACHE/images/store/3/dali-museum-8983261920/aa0d93e475c7b7388bee88ff14f8d795.jpg",
"full" : "https://app.wegotrip.com/media/store/3/dali-museum-8983261920.jpg"
},
{
"id" : 834,
"description" : "",
"cover" : false,
"preview" : "https://app.wegotrip.com/media/CACHE/images/store/3/shutterstock196896461/74fc427d0a27f0aa199ed24f4c51bcc5.jpg",
"full" : "https://app.wegotrip.com/media/store/3/shutterstock196896461.jpg"
}
],
"duration" : 2,
"_id" : ObjectId("6210fa7c46bee3fcbd0acc24")
},
"availability" : null,
"subcategory" : [
{
"id" : 3,
"title" : "Theme Tours",
"slug" : "theme-tours"
},
{
"id" : 1,
"title" : "Culture & History",
"slug" : "culture-and-history"
},
{
"id" : 6,
"title" : "Sightseeing Tickets & Passes",
"slug" : "sightseeing-tickets-passes"
}
],
"category" : [
{
"id" : 3,
"title" : "Theme Tours",
"slug" : "theme-tours"
},
{
"id" : 1,
"title" : "Culture & History",
"slug" : "culture-and-history"
},
{
"id" : 6,
"title" : "Sightseeing Tickets & Passes",
"slug" : "sightseeing-tickets-passes"
}
],
"type" : "Audio Guide",
"description" : "The Dalí Theatre and Museum is a museum of the artist Salvador Dalí in his home town of Figueres, in Catalonia, Spain. Dalí is buried there in a crypt below the stage. \r\n\r\nImmerse yourself in an exciting journey through the world of the genius of surrealism. Reveal the meaning of his ambiguous creations and learn the history of the artist's life. Enjoy the unique world of Dali in this excursion.",
"thumbnail" : "https://app.wegotrip.com/media/CACHE/images/store/001_Ispaniya_Figeras_Teatr-01/783c3a10c34eb40c29f14f704cd9c8d1.jpeg",
"name" : "The Dali Theatre-Museum: Skip-the-Line & Audio Tour",
"attractionDescription" : "",
"attractionName" : "Dali Theatre and Museum",
"attraction" : ObjectId("6210fa7c46bee3fcbd0acc15"),
"provider" : {
"rating" : {
"count" : 0,
"average" : null,
"_id" : ObjectId("6210fa7c46bee3fcbd0acc26")
},
"preview" : "https://app.wegotrip.com/media/CACHE/images/store/001_Ispaniya_Figeras_Teatr-01/783c3a10c34eb40c29f14f704cd9c8d1.jpeg",
"slug" : "the-dali-museum-in-figueres",
"id" : "3",
"key" : "1",
"_id" : ObjectId("6210fa7c46bee3fcbd0acc25")
},
"__v" : 0
}
Both of the above have detail.duration set to 2 and as per query, these 2 should have each other considered as a deal and found in result docs, but query returns deals: [], an empty array. I'm unable to figure out the problem.
From $match (Restrictions)
The $match query syntax is identical to the read operation query syntax; i.e. $match does not accept raw aggregation expressions. To include aggregation expression in $match, use a $expr query expression.
And you need to use $$ to get the variable value.
let
To reference variables in pipeline stages, use the "$$" syntax.
Change the $match stage in the pipeline as:
{
$match: {
_id: {
$ne: "$$id"
},
$expr: {
$eq: [
"$detail.duration",
"$$duration"
]
}
}
}
Sample Mongo Playground
I need to execute the following query:
db.S12_RU.find({"venue.raw":a,"title":/b|c|d|e/}).sort({"year":-1}).skip(X).limit(Y);
where X and Y are numbers.
The number of documents in my collection is:
208915369
Currently, this sort of query takes about 6 minutes to execute.
I have the following indexes:
[
{
"v" : 2,
"key" : {
"_id" : 1
},
"name" : "_id_"
},
{
"v" : 2,
"key" : {
"venue.raw" : 1
},
"name" : "venue.raw_1"
},
{
"v" : 2,
"key" : {
"venue.raw" : 1,
"title" : 1,
"year" : -1
},
"name" : "venue.raw_1_title_1_year_-1"
}
]
A standard document looks like this:
{ "_id" : ObjectId("5fc25fc091e3146fb10484af"), "id" : "1967181478", "title" : "Quality of Life of Swedish Women with Fibromyalgia Syndrome, Rheumatoid Arthritis or Systemic Lupus Erythematosus", "authors" : [ { "name" : "Carol S. Burckhardt", "id" : "2052326732" }, { "name" : "Birgitha Archenholtz", "id" : "2800742121" }, { "name" : "Kaisa Mannerkorpi", "id" : "240289002" }, { "name" : "Anders Bjelle", "id" : "2419758571" } ], "venue" : { "raw" : "Journal of Musculoskeletal Pain", "id" : "49327845" }, "year" : 1993, "n_citation" : 31, "page_start" : "199", "page_end" : "207", "doc_type" : "Journal", "publisher" : "Taylor & Francis", "volume" : "1", "issue" : "", "doi" : "10.1300/J094v01n03_20" }
Is there any way to make this query execute in a few seconds?
I have 2 collections structured as below. I have tried $lookup to get the result but I am not getting any result because of my local and foreign fields are in array of object.
Below is my structure:
{
"_id" : ObjectId("5795a3531d3f3afc19caefef"),
"name" : "category1",
"updatedAt" : "1469431592786",
"resources" : [
{
"_id" : ObjectId("5791be003fa3bedc15d3adde"),
"title" : "resource1",
"availability" : false
},
{
"_id" : ObjectId("5795a3771d3f3afc19caeff0"),
"title" : "resource2",
"availability" : true
}
]
}
Above "categories" schema have resources array of object. this resource _id is stored in bookings collection in following way:
"booking":
{
"_id" : ObjectId("57960aa8000ca7a46b7ef683"),
"name" : "1469491200000",
"__v" : 0,
"schedule" : [
{
"resourceId" : ObjectId("5791be003fa3bedc15d3adde"),
"userId" : ObjectId("5791be003fa3bedc15d3adcve"),
"title" : "grofep",
"_id" : ObjectId("57960aa8f9f9951c1fc923b1")
},
{
"resourceId" : ObjectId("5791be003fa3bedc15d3bddz"),
"userId" : ObjectId("5791be003fa3bedc15d3adcve"),
"title" : "mr3",
"_id" : ObjectId("57960aa8f9f9951c1fc923b2")
},
{
"resourceId" : ObjectId("5791be003fa3bedc15d3adde"),
"userId" : ObjectId("5791be003fa3bedc15d3adcve"),
"title" : "grofep23",
"_id" : ObjectId("57960aa8f9f9951c1fc923b3")
}
]
}
Now I want to get all the schedule of booking collection with their resource information.I want to fetch resources from categories table on the basis of booking schedule.
Desired output:
[
{
"name" : "1469491200000",
"resourceId" : ObjectId("5791be003fa3bedc15d3adde"),
"resourceTitle":"title",
"availability":false,
"bookings": [
{
"userId" : ObjectId("5791be003fa3bedc15d3adcve"),
"title" : "grofep",
"_id" : ObjectId("57960aa8f9f9951c1fc923b1")
},
{
"userId" : ObjectId("5791be003fa3bedc15d3adcve"),
"title" : "grofep23",
"_id" : ObjectId("57960aa8f9f9951c1fc923b3")
}
]
},
{
"name" : "1469491200000",
"resourceId" : ObjectId("5791be003fa3bedc15d3bddz"),
"resourceTitle":"mr3",
"availability":false,
"bookings": [
{
"userId" : ObjectId("5791be003fa3bedc15d3adcve"),
"title" : "mr3",
"_id" : ObjectId("57960aa8f9f9951c1fc923b2")
}
]
}
]
Help me to get this desired result.
Thanks.
I'm having trouble figuring out the right aggregation pipe operations to return the results I need.
I have a collection similar to the following :-
{
"_id" : "writer1",
"Name" : "writer1",
"Website" : "website1",
"Reviews" : [
{
"Film" : {
"Name" : "Jurassic Park",
"Genre" : "Action"
},
"Score" : 4
},
{
"Technology" : {
"Name" : "Mad Max",
"Genre" : "Action"
},
"Score" : 5
}
]
}
{
"_id" : "writer2",
"Name" : "writer2",
"Website" : "website1",
"Reviews" : [
{
"Technology" : {
"Name" : "Mad Max",
"Genre" : "Action"
},
"Score" : 5
}
]
}
And this is my aggregation so far : -
db.writers.aggregate([
{ "$unwind" : "$Reviews" },
{ "$match" : { "Reviews.Film.Name" : "Jurassic Park" } },
{ "$group" : { "_id" : "$Website" , "score" : { "$avg" : "$Reviews.Score" },
writers :{ $push: { name:"$Name", score:"$Reviews.Score" } }
}}
])
This returns only writers who have a review of the matching film and also only websites that have at least 1 writer who has reviewed the film,
however, I need to return all websites containing a list of their all writers, with a score of 0 if they haven't written a review for the specified film.
so, I am currently getting : -
{ "_id" : "website1", "score" : 4, "writers" : [ { "name" : "writer1", "score" : 4 } ] }
When I actually need : -
{ "_id" : "website1", "score" : 2, "writers" : [ { "name" : "writer1", "score" : 4 },{ "name" :"writer2", "score" : 0 } ] }
Can anyone point me in the right direction?
Cheers
I have a nested embedded document CompanyProduct below is structure
{
"_id" : ObjectId("53d213c5ddbb1912343a8ca3"),
"CompanyID" : 90449,
"Name" : Company1,
"CompanyDepartment" : [
{
"_id" : ObjectId("53d213c5ddbb1912343a8ca4")
"DepartmentID" : 287,
"DepartmentName" : "Stores",
"DepartmentInventory" : [
{
"_id" : ObjectId("53b7b92eecdd765430d763bd"),
"ProductID" : 1,
"ProductName" : "abc",
"Quantity" : 100
},
{
"_id" : ObjectId("53b7b92eecdd765430d763bd"),
"ProductID" : 2,
"ProductName" : "xyz",
"Quantity" : 1
}
],
}
],
}
There can be N no of companies and each company can have N number of departments and each department can have N number of products.
I want to do a search to find out a particular product quantity under a particular company
I tried below query but it does not work. It returns all the products for the specific company, the less than 20 condition doesn't work.
db.CompanyProduct.find({$and : [{"CompanyDepartment.DepartmentInventory.Quantity":{$lt :20}},{"CompanyID":90449}]})
How should the query be?
You are searching from companyProduct's sub documents. So it will return you companyProduct whole document, it is NoSQL database , some how we do not need to normalize the collection , but your case it has to be normalize , like if you want to EDIT/DELETE any sub document and if there are thousand or millions of sub document then what will you do ... You need to make other collection with the name on CompanyDepartment and companyProduct collection should be
productCompany
{
"_id" : ObjectId("53d213c5ddbb1912343a8ca3"),
"CompanyID" : 90449,
"Name" : Company1,
"CompanyDepartment" : ['53d213c5ddbb1912343a8ca4'],
}
and other collection companyDepartment
{
"_id" : ObjectId("53d213c5ddbb1912343a8ca4")
"DepartmentID" : 287,
"DepartmentName" : "Stores",
"DepartmentInventory" : [
{
"_id" : ObjectId("53b7b92eecdd765430d763bd"),
"ProductID" : 1,
"ProductName" : "abc",
"Quantity" : 100
},
{
"_id" : ObjectId("53b7b92eecdd765430d763bd"),
"ProductID" : 2,
"ProductName" : "xyz",
"Quantity" : 1
}
],
}
after this you got array of companyDeparment' ID and only push and pull query will be used on productCompany
A Solution can be
db.YourCollection.aggregate([
{
$project:{
"CompanyDepartment.DepartmentInventory":1,
"CompanyID" : 1
}
},{
$unwind: "$CompanyDepartment"
},{
$unwind: "$CompanyDepartment.DepartmentInventory"
},{
$match:{$and : [{"CompanyDepartment.DepartmentInventory.Quantity":{$lt :20}},{"CompanyID":90449}]}
}
])
the result is
{
"result" : [
{
"_id" : ObjectId("53d213c5ddbb1912343a8ca3"),
"CompanyID" : 90449,
"CompanyDepartment" : {
"DepartmentInventory" : {
"_id" : ObjectId("53b7b92eecdd765430d763bd"),
"ProductID" : 2,
"ProductName" : "xyz",
"Quantity" : 1
}
}
}
],
"ok" : 1
}