mongoDB distict problems - mongodb

It's one of my data as JSON format:
{
"_id" : ObjectId("5bfdb412a80939b6ed682090"),
"accounts" : [
{
"_id" : ObjectId("5bf106eee639bd0df4bd8e05"),
"accountType" : "DDA",
"productName" : "DDA1"
},
{
"_id" : ObjectId("5bf106eee639bd0df4bd8df8"),
"accountType" : "VSA",
"productName" : "VSA1"
},
{
"_id" : ObjectId("5bf106eee639bd0df4bd8df9"),
"accountType" : "VSA",
"productName" : "VSA2"
}
]
}
I want to make a query to get all productName(no duplicate) of accountType = VSA.
I write a mongo query:
db.Collection.distinct("accounts.productName", {"accounts.accountType": "VSA" })
I expect: ['VSA1', 'VSA2']
I get: ['DDA','VSA1', 'VSA2']
Anybody knows why the query doesn't work in distinct?

Second parameter of distinct method represents:
A query that specifies the documents from which to retrieve the distinct values.
But the thing is that you showed only one document with nested array of elements so whole document will be returned for your condition "accounts.accountType": "VSA".
To fix that you have to use Aggregation Framework and $unwind nested array before you apply the filtering and then you can use $group with $addToSet to get unique values. Try:
db.col.aggregate([
{
$unwind: "$accounts"
},
{
$match: {
"accounts.accountType": "VSA"
}
},
{
$group: {
_id: null,
uniqueProductNames: { $addToSet: "$accounts.productName" }
}
}
])
which prints:
{ "_id" : null, "uniqueProductNames" : [ "VSA2", "VSA1" ] }

Related

Individual search result in multiple values in arrays

I have following model:
{
"_id" : ObjectId("5d61aaf8108e185191552bbb"),
"serials" : [
"e127av48-0697-4977-b096-5ce79c89a414",
"d163f80a-55ff-40fe-90b4-331ece5bebd5",
"4740021f-e9b5-4ca5-bf0e-8554c123bb94",
"320ffd42-f101-4b1d-8ff4-80bc693a29e6",
"fef5e68b-aed0-4a96-9488-7941c41d1c1f",
"2c0752ba-bf7a-4a3b-bd9f-14db4b2f8bae",
"6c5ff44d-5979-4bff-af12-9e6d282c3789",
"9c91bf91-72d7-4b71-827b-924947d6e93d",
"fb34b28e-afb1-4b6a-a3c1-5a1fe44246ee",
"91ab22ef-702f-4cbd-8919-a67a2b9a684c",
"ee1a7cb2-e088-47e6-a824-c8697df7d94c",
"0dc4c687-4db2-481e-a1a6-491320dede11",
"34612148-3e01-44ee-b262-de2035e63691",
"5ba85baf-e48a-40af-8578-55ff1a873c76",
"19fe3672-b6cb-4bb6-8d21-93412b938584",
"1d0d6f6d-1b49-461b-8661-ecbf43a6595e",
"d9a5455c-65ee-45e1-ae49-33cc15dec841",
"4a690a00-a76c-4d3e-aee3-78b2bb731b0c",
"ae331830-40b4-457c-8cc4-5d548f769c3e",
"fe3e460b-c89d-4ace-8a36-5ba2b53bf4d0",
"2cc6a2a0-e029-475f-a7fc-a46a79afb605",
"a7d07767-eada-4ce3-b083-9b048e9ae9f4"
],
"name" : "ApiCard",
"producer" : "Farmina",
"form" : "syrop",
"__v" : 0
}
I would like to retrive documents (multiple) from collection based on this serial numbers ("serials" field). For example i am finding:
[
"e127av48-0697-4977-b096-5ce79c89a414",
"d163f80a-55ff-40fe-90b4-331ece5bebd5",
"4740021f-e9b5-4ca5-bf0e-8554c123bb94",
"key that doesn't exist",
]
We have to assume that one of the serial number doesn't exist, so would like to get information for individual serial, expected output:
[
{
"serial":"e127av48-0697-4977-b096-5ce79c89a414",
"doc":{
....whole document where above serial is in array field "serials"
}
},
{
"serial":"e127av48-0697-4977-b096-5ce79c89a414",
"doc":{
....whole document where above serial is in array field "serials"
}
},
{
"serial":"e127av48-0697-4977-b096-5ce79c89a414",
"doc":{
....whole document where above serial is in array field "serials"
}
},
{
"serial":"key that doesn't exist",
"doc": null
}
]
I was trying the simplest solution - mongodb find by multiple array items, but unfortunately it'doesn't return info for individual serial number. I'am not sure it's possible to prepare this kind of query. I think some complex aggregation could perform it, but i don't even know this kind of pipelines.
Of course, i can get simple solution by using multiple aggregate or even find, but it could impact on performance, when application will be looking for 10000 records per request.
The following query can do the trick:
db.collection.aggregate([
{
$limit:1
},
{
$project:{
"_id":0,
"serialsToSearch":[
"e127av48-0697-4977-b096-5ce79c89a414",
"d163f80a-55ff-40fe-90b4-331ece5bebd5",
"4740021f-e9b5-4ca5-bf0e-8554c123bb94",
"key that doesn't exist",
]
}
},
{
$unwind:"$serialsToSearch"
},
{
$lookup:{
"from":"collection",
"let":{
"serial":"$serialsToSearch"
},
"pipeline":[
{
$match:{
$expr:{
$in:["$$serial","$serials"]
}
}
},
{
$project:{
"serials":0
}
}
],
"as":"searialsLookup"
}
},
{
$unwind:{
"path":"$searialsLookup",
"preserveNullAndEmptyArrays":true
}
},
{
$project:{
"serial":"$serialsToSearch",
"doc":{
$ifNull:["$searialsLookup",null]
}
}
}
]).pretty()
Data Set:
{
"_id" : ObjectId("5d61aaf8108e185191552bbb"),
"serials" : [
"e127av48-0697-4977-b096-5ce79c89a414",
"d163f80a-55ff-40fe-90b4-331ece5bebd5",
"4740021f-e9b5-4ca5-bf0e-8554c123bb94",
"320ffd42-f101-4b1d-8ff4-80bc693a29e6",
"fef5e68b-aed0-4a96-9488-7941c41d1c1f",
"2c0752ba-bf7a-4a3b-bd9f-14db4b2f8bae",
"6c5ff44d-5979-4bff-af12-9e6d282c3789",
"9c91bf91-72d7-4b71-827b-924947d6e93d",
"fb34b28e-afb1-4b6a-a3c1-5a1fe44246ee",
"91ab22ef-702f-4cbd-8919-a67a2b9a684c",
"ee1a7cb2-e088-47e6-a824-c8697df7d94c",
"0dc4c687-4db2-481e-a1a6-491320dede11",
"34612148-3e01-44ee-b262-de2035e63691",
"5ba85baf-e48a-40af-8578-55ff1a873c76",
"19fe3672-b6cb-4bb6-8d21-93412b938584",
"1d0d6f6d-1b49-461b-8661-ecbf43a6595e",
"d9a5455c-65ee-45e1-ae49-33cc15dec841",
"4a690a00-a76c-4d3e-aee3-78b2bb731b0c",
"ae331830-40b4-457c-8cc4-5d548f769c3e",
"fe3e460b-c89d-4ace-8a36-5ba2b53bf4d0",
"2cc6a2a0-e029-475f-a7fc-a46a79afb605",
"a7d07767-eada-4ce3-b083-9b048e9ae9f4"
],
"name" : "ApiCard",
"producer" : "Farmina",
"form" : "syrop",
"__v" : 0
}
Output:
{
"serial" : "e127av48-0697-4977-b096-5ce79c89a414",
"doc" : {
"_id" : ObjectId("5d61aaf8108e185191552bbb"),
"name" : "ApiCard",
"producer" : "Farmina",
"form" : "syrop",
"__v" : 0
}
}
{
"serial" : "d163f80a-55ff-40fe-90b4-331ece5bebd5",
"doc" : {
"_id" : ObjectId("5d61aaf8108e185191552bbb"),
"name" : "ApiCard",
"producer" : "Farmina",
"form" : "syrop",
"__v" : 0
}
}
{
"serial" : "4740021f-e9b5-4ca5-bf0e-8554c123bb94",
"doc" : {
"_id" : ObjectId("5d61aaf8108e185191552bbb"),
"name" : "ApiCard",
"producer" : "Farmina",
"form" : "syrop",
"__v" : 0
}
}
{ "serial" : "key that doesn't exist", "doc" : null }
Note: The query won't give expected output if the collection would be empty.
Aggregation stages details:
STAGE I: Limiting the records to 1, as initially, our motive is to inject the input array in aggregation. The injection would be done in no time.
STAGE II: Projecting the input array as serialsToSearch
STAGE III: Now we have the input array as a field, we can unwind it
STAGE IV: Lookup in the same collection with each field of the input array and check if the searched serial is present in serials array
STAGE V: unwinding the lookup output
STAGE VI: Projecting fields as per the response required.

How to remove an element from inner array of nested array pymongo using $ pull

Here is my news document structure
{
"_id" : ObjectId("5bff0903bd9a221229c7c9b2"),
"title" : "Test Page",
"desc" : "efdfr",
"mediaset_list" : [
{
"_id" : ObjectId("5bfeff94bd9a221229c7c9ae"),
"medias" : [
{
"_id" : ObjectId("5bfeff83bd9a221229c7c9ac"),
"file_type" : "png",
"file" : "https://aws.com/gg.jpg",
"file_name" : "edf.jpg"
},
{
"_id" : ObjectId("5bfeff83bd9a221229c7c9ad"),
"file_type" : "mov",
"file" : "https://aws.com/gg.mov",
"file_name" : "abcd.mov"
}
]
}
]}
The queries that i've tried are given below
Approach 1
db.news.find_and_modify({},{'$pull': {"mediaset_list": {"medias": {"$elemMatch" : {"_id": ObjectId('5bfeff83bd9a221229c7c9ac')}} }}})
Approach 2
db.news.update({},{'$pull': {"mediaset_list.$.medias": {"_id": ObjectId('5bfeff83bd9a221229c7c9ac')}} })
Issue we are facing
The above queries are removing entire elements inside 'mediaset_list' . But i only want to remove the element inside 'medias' matching object ID.
Since you have two nested arrays you have to use arrayFilters to indicate which element of outer array should be modified, try:
db.news.update({ _id: ObjectId("5bff0903bd9a221229c7c9b2") },
{ $pull: { "mediaset_list.$[item].medias": { _id: ObjectId("5bfeff83bd9a221229c7c9ad") } } },
{ arrayFilters: [ { "item._id": ObjectId("5bfeff94bd9a221229c7c9ae") } ] })
So item is used here as a placeholder which will be used by MongoDB to determine which element of mediaset_list needs to be modified and the condition for this placeholder is defined inside arrayFilters. Then you can use $pull and specify another condition for inner array to determine which element should be removed.
From #micki's mongo shell query (Answer above) , This is the pymongo syntax which will update all news document with that media id .
db.news.update_many({},
{
"$pull":
{ "mediaset_list.$[item].medias": { "_id": ObjectId("5bfeff83bd9a221229c7c9ad") } } ,
},
array_filters=[{ "item._id": ObjectId("5bfeff94bd9a221229c7c9ae")}],
upsert=True)

Mongo DB - how to query for id dependent on oldest date in array of a field

Lets say I have a collection called phone_audit with document entries of the following form - _id which is the phone number, and value containing items that always contains 2 entries (id, and a date).
Please see below:
{
"_id" : {
"phone_number" : "+012345678"
},
"value" : {
"items" : [
{
"_id" : "c14b4ac1db691680a3fb65320fba7261",
"updated_at" : ISODate("2016-03-14T12:35:06.533Z")
},
{
"_id" : "986b58e55f8606270f8a43cd7f32392b",
"updated_at" : ISODate("2016-07-23T11:17:53.552Z")
}
]
}
},
......
I need to get a list of _id values for every entry in that collection representing the older of the two items in each document.
So in the above - result would be [c14b4ac1db691680a3fb65320fba7261,...]
Any pointers at the type of query to execute would be v.helpful even if the exact syntax is not correct.
With aggregate(), you can $unwind value.items, $sort by update_at, then use $first to get the oldest:
[
{
"$unwind": "$value.items"
},
{
"$sort": { "value.items.updated_at": 1 }
},
{
"$group":{
_id: "$_id.phone_number",
oldest:{$first:"$value.items"}
}
},
{
"$project":{
value_id: "$oldest._id"
}
}
]

How to Avoid Duplicate Entries in MongoDb Meteor App

How to avoid duplicate entries in mongoDb in Meteor application.
On the command: db.products.find({},{"TEMPLATE_NAME": 1},{unique : true})
{ "_id" : ObjectId("5555d0a16ce3b01bb759a771"), "TEMPLATE_NAME" : "B" }
{ "_id" : ObjectId("5555d0b46ce3b01bb759a772"), "TEMPLATE_NAME" : "A" }
{ "_id" : ObjectId("5555d0c86ce3b01bb759a773"), "TEMPLATE_NAME" : "C" }
{ "_id" : ObjectId("5555d0f86ce3b01bb759a774"), "TEMPLATE_NAME" : "C" }
{ "_id" : ObjectId("5555d1026ce3b01bb759a775"), "TEMPLATE_NAME" : "A" }
{ "_id" : ObjectId("5555d1086ce3b01bb759a776"), "TEMPLATE_NAME" : "B" }
I want to retrieve only the unique template names and show them on HTML page.
Use the aggregation framework where your pipeline stages consist of the $group and $project operators respectively. The $group operator step groups the input documents by the given key and thus will return distinct documents in the result. The $project operator then reshapes each document in the stream, such as by adding new fields or removing existing fields:
db.products.aggregate([
{
"$group": {
"_id": "$TEMPLATE_NAME"
}
},
{
"$project": {
"_id": 0,
"TEMPLATE_NAME": "$_id"
}
}
])
Result:
/* 0 */
{
"result" : [
{
"TEMPLATE_NAME" : "C"
},
{
"TEMPLATE_NAME" : "A"
},
{
"TEMPLATE_NAME" : "B"
}
],
"ok" : 1
}
You could then use the meteorhacks:aggregate package to implement the aggregation in Meteor:
Add to your app with
meteor add meteorhacks:aggregate
Then simply use .aggregate function like below.
var products = new Mongo.Collection('products');
var pipeline = [
{
"$group": {
"_id": "$TEMPLATE_NAME"
}
},
{
"$project": {
"_id": 0,
"TEMPLATE_NAME": "$_id"
}
}
];
var result = products.aggregate(pipeline);
-- UPDATE --
An alternative that doesn't use aggregation is using underscore's methods to return distinct field values from the collection's find method as follows:
var distinctTemplateNames = _.uniq(Collection.find({}, {
sort: {"TEMPLATE_NAME": 1}, fields: {"TEMPLATE_NAME": true}
}).fetch().map(function(x) {
return x.TEMPLATE_NAME;
}), true)
;
This will return an array with distinct product template names ["A", "B", "C"]
You can check out some tutorials which explain the above approach in detail: Get unique values from a collection in Meteor and METEOR – DISTINCT MONGODB QUERY.
You can use distinct of mongodb like :
db.collectionName.distinct("TEMPLATE_NAME")
This query will return you array of distinct TEMPLATE_NAME

In a Mongo collection, how do you query for a specific object in an array?

I'm trying to retrieve an object from an array in mongodb. Below is my document:
{
"_id" : ObjectId("53e9b43968425b29ecc87ffd"),
"firstname" : "john",
"lastname" : "smith",
"trips" : [
{
"submitted" : 1407824585356,
"tripCategory" : "staff",
"tripID" : "1"
},
{
"tripID" : "2",
"tripCategory" : "volunteer"
},
{
"tripID" : "3",
"tripCategory" : "individual"
}
]
}
My ultimate goal is to update only when trips.submitted is absent so I thought I could query and determine what the mongo find behavior would look like
if I used the $and query operator. So I try this:
db.users.find({
$and: [
{ "trips.tripID": "1" },
{ "trips": { $elemMatch: { submitted: { $exists: true } } } }
]
},
{ "trips.$" : 1 } //projection limits to the FIRST matching element
)
and I get this back:
{
"_id" : ObjectId("53e9b43968425b29ecc87ffd"),
"trips" : [
{
"submitted" : 1407824585356,
"tripCategory" : "staff",
"tripID" : "1"
}
]
}
Great. This is what I want. However, when I run this query:
db.users.find({
$and: [
{ "trips.tripID": "2" },
{ "trips": { $elemMatch: { submitted: { $exists: true } } } }
]
},
{ "trips.$" : 1 } //projection limits to the FIRST matching element
)
I get the same result as the first! So I know there's something odd about my query that isn't correct. But I dont know what. The only thing I've changed between the queries is "trips.tripID" : "2", which in my head, should have prompted mongo to return no results. What is wrong with my query?
If you know the array is in a specific order you can refer to a specific index in the array like this:-
db.trips.find({"trips.0.submitted" : {$exists:true}})
Or you could simply element match on both values:
db.trips.find({"trips" : {$elemMatch : {"tripID" : "1",
"submitted" : {$exists:true}
}}})
Your query, by contrast, is looking for a document where both are true, not an element within the trips field that holds for both.
The output for your query is correct. Your query asks mongo to return a document which has the given tripId and the field submitted within its trips array. The document you have provided in your question satisfies both conditions for both tripIds. You are getting the first element in the array trips because of your projection.
I have assumed you will be filtering records by the person's name and then retrieving the elements inside trips based on the field-exists criteria. The output you are expecting can be obtained using the following:
db.users.aggregate(
[
{$match:
{
"firstname" : "john",
"lastname" : "smith"
}
},
{$unwind: "$trips"},
{$match:
{
"trips.tripID": "1" ,
"trips.submitted": { $exists: true }
}
}
]
)
The aggregation pipeline works as follows. The first $match operator filters one document (in this case the document for john smith) The $unwind operator in mongodb aggregation unwinds the specified array (trips in this case), in effect denormalizing the sub-records associated with the parent records. The second $match operator filters the denormalized/unwound documents further to obtain the one required as per your query.