How to avoid duplicate entries in mongoDb in Meteor application.
On the command: db.products.find({},{"TEMPLATE_NAME": 1},{unique : true})
{ "_id" : ObjectId("5555d0a16ce3b01bb759a771"), "TEMPLATE_NAME" : "B" }
{ "_id" : ObjectId("5555d0b46ce3b01bb759a772"), "TEMPLATE_NAME" : "A" }
{ "_id" : ObjectId("5555d0c86ce3b01bb759a773"), "TEMPLATE_NAME" : "C" }
{ "_id" : ObjectId("5555d0f86ce3b01bb759a774"), "TEMPLATE_NAME" : "C" }
{ "_id" : ObjectId("5555d1026ce3b01bb759a775"), "TEMPLATE_NAME" : "A" }
{ "_id" : ObjectId("5555d1086ce3b01bb759a776"), "TEMPLATE_NAME" : "B" }
I want to retrieve only the unique template names and show them on HTML page.
Use the aggregation framework where your pipeline stages consist of the $group and $project operators respectively. The $group operator step groups the input documents by the given key and thus will return distinct documents in the result. The $project operator then reshapes each document in the stream, such as by adding new fields or removing existing fields:
db.products.aggregate([
{
"$group": {
"_id": "$TEMPLATE_NAME"
}
},
{
"$project": {
"_id": 0,
"TEMPLATE_NAME": "$_id"
}
}
])
Result:
/* 0 */
{
"result" : [
{
"TEMPLATE_NAME" : "C"
},
{
"TEMPLATE_NAME" : "A"
},
{
"TEMPLATE_NAME" : "B"
}
],
"ok" : 1
}
You could then use the meteorhacks:aggregate package to implement the aggregation in Meteor:
Add to your app with
meteor add meteorhacks:aggregate
Then simply use .aggregate function like below.
var products = new Mongo.Collection('products');
var pipeline = [
{
"$group": {
"_id": "$TEMPLATE_NAME"
}
},
{
"$project": {
"_id": 0,
"TEMPLATE_NAME": "$_id"
}
}
];
var result = products.aggregate(pipeline);
-- UPDATE --
An alternative that doesn't use aggregation is using underscore's methods to return distinct field values from the collection's find method as follows:
var distinctTemplateNames = _.uniq(Collection.find({}, {
sort: {"TEMPLATE_NAME": 1}, fields: {"TEMPLATE_NAME": true}
}).fetch().map(function(x) {
return x.TEMPLATE_NAME;
}), true)
;
This will return an array with distinct product template names ["A", "B", "C"]
You can check out some tutorials which explain the above approach in detail: Get unique values from a collection in Meteor and METEOR – DISTINCT MONGODB QUERY.
You can use distinct of mongodb like :
db.collectionName.distinct("TEMPLATE_NAME")
This query will return you array of distinct TEMPLATE_NAME
Related
I need to count the recurrences of a value in the collection A, so I do
db.collectionA.aggregate( [ { $group : { name : "$name", count :{$sum: 1 } } } ] )
And have something like
{
"name": "Bruce",
"count": 2
},
{
"_id": "Alfred",
"count": 3
}
Then I need to get this result and populate a field of the collection B, I imagine something like a forEach but don't know how to implement
db.collectionB.findAndModify({query: {"name":forEach of the pervious result},
update:{"nameRecurrences": value of the count}})
Looking at your aggregation pipeline :
db.collectionA.aggregate( [
{ $group : { _id : '$name', name : {$first : "$name"}, nameRecurrences :{$sum: 1 } } }, // renamed field `nameRecurrences` to match with field name in `collection-B`
{$project : {_id : 0}} ] ) // Removing _id to avoid conflicts on merge
On MongoDB version >= 4.2 you can use $merge aggregation operator to merge result of aggregation pipeline on one collection to another collection :
Just add below stage as last stage of aggregation pipeline :
{$merge : { into: { db: "dbName", coll: "collectionB" }, on: "name", whenNotMatched: "discard"}} // Remember to create unique index on `name` field on `collectionB`
Since you're using MongoDB version 3.6.16 :
If the collection has to be created now then you can use $out, but if it's an existing collection with lot of fields in each document apart from just name & nameRecurrences then you can try this in code :
Since you've different filters and their respective update part, then you can take advantage of .bulkWrite() to update multiple documents :
let bulkArr = []
for (const i of aggregationResult){
bulkArr.push( { updateOne : {
"filter" : { "name" : i.name },
"update" : { $set : { "nameRecurrences" : i.nameRecurrences } }
} })
}
db.collectionB.bulkWrite(bulkArr)
Here is my news document structure
{
"_id" : ObjectId("5bff0903bd9a221229c7c9b2"),
"title" : "Test Page",
"desc" : "efdfr",
"mediaset_list" : [
{
"_id" : ObjectId("5bfeff94bd9a221229c7c9ae"),
"medias" : [
{
"_id" : ObjectId("5bfeff83bd9a221229c7c9ac"),
"file_type" : "png",
"file" : "https://aws.com/gg.jpg",
"file_name" : "edf.jpg"
},
{
"_id" : ObjectId("5bfeff83bd9a221229c7c9ad"),
"file_type" : "mov",
"file" : "https://aws.com/gg.mov",
"file_name" : "abcd.mov"
}
]
}
]}
The queries that i've tried are given below
Approach 1
db.news.find_and_modify({},{'$pull': {"mediaset_list": {"medias": {"$elemMatch" : {"_id": ObjectId('5bfeff83bd9a221229c7c9ac')}} }}})
Approach 2
db.news.update({},{'$pull': {"mediaset_list.$.medias": {"_id": ObjectId('5bfeff83bd9a221229c7c9ac')}} })
Issue we are facing
The above queries are removing entire elements inside 'mediaset_list' . But i only want to remove the element inside 'medias' matching object ID.
Since you have two nested arrays you have to use arrayFilters to indicate which element of outer array should be modified, try:
db.news.update({ _id: ObjectId("5bff0903bd9a221229c7c9b2") },
{ $pull: { "mediaset_list.$[item].medias": { _id: ObjectId("5bfeff83bd9a221229c7c9ad") } } },
{ arrayFilters: [ { "item._id": ObjectId("5bfeff94bd9a221229c7c9ae") } ] })
So item is used here as a placeholder which will be used by MongoDB to determine which element of mediaset_list needs to be modified and the condition for this placeholder is defined inside arrayFilters. Then you can use $pull and specify another condition for inner array to determine which element should be removed.
From #micki's mongo shell query (Answer above) , This is the pymongo syntax which will update all news document with that media id .
db.news.update_many({},
{
"$pull":
{ "mediaset_list.$[item].medias": { "_id": ObjectId("5bfeff83bd9a221229c7c9ad") } } ,
},
array_filters=[{ "item._id": ObjectId("5bfeff94bd9a221229c7c9ae")}],
upsert=True)
It's one of my data as JSON format:
{
"_id" : ObjectId("5bfdb412a80939b6ed682090"),
"accounts" : [
{
"_id" : ObjectId("5bf106eee639bd0df4bd8e05"),
"accountType" : "DDA",
"productName" : "DDA1"
},
{
"_id" : ObjectId("5bf106eee639bd0df4bd8df8"),
"accountType" : "VSA",
"productName" : "VSA1"
},
{
"_id" : ObjectId("5bf106eee639bd0df4bd8df9"),
"accountType" : "VSA",
"productName" : "VSA2"
}
]
}
I want to make a query to get all productName(no duplicate) of accountType = VSA.
I write a mongo query:
db.Collection.distinct("accounts.productName", {"accounts.accountType": "VSA" })
I expect: ['VSA1', 'VSA2']
I get: ['DDA','VSA1', 'VSA2']
Anybody knows why the query doesn't work in distinct?
Second parameter of distinct method represents:
A query that specifies the documents from which to retrieve the distinct values.
But the thing is that you showed only one document with nested array of elements so whole document will be returned for your condition "accounts.accountType": "VSA".
To fix that you have to use Aggregation Framework and $unwind nested array before you apply the filtering and then you can use $group with $addToSet to get unique values. Try:
db.col.aggregate([
{
$unwind: "$accounts"
},
{
$match: {
"accounts.accountType": "VSA"
}
},
{
$group: {
_id: null,
uniqueProductNames: { $addToSet: "$accounts.productName" }
}
}
])
which prints:
{ "_id" : null, "uniqueProductNames" : [ "VSA2", "VSA1" ] }
Lets say I have a collection called phone_audit with document entries of the following form - _id which is the phone number, and value containing items that always contains 2 entries (id, and a date).
Please see below:
{
"_id" : {
"phone_number" : "+012345678"
},
"value" : {
"items" : [
{
"_id" : "c14b4ac1db691680a3fb65320fba7261",
"updated_at" : ISODate("2016-03-14T12:35:06.533Z")
},
{
"_id" : "986b58e55f8606270f8a43cd7f32392b",
"updated_at" : ISODate("2016-07-23T11:17:53.552Z")
}
]
}
},
......
I need to get a list of _id values for every entry in that collection representing the older of the two items in each document.
So in the above - result would be [c14b4ac1db691680a3fb65320fba7261,...]
Any pointers at the type of query to execute would be v.helpful even if the exact syntax is not correct.
With aggregate(), you can $unwind value.items, $sort by update_at, then use $first to get the oldest:
[
{
"$unwind": "$value.items"
},
{
"$sort": { "value.items.updated_at": 1 }
},
{
"$group":{
_id: "$_id.phone_number",
oldest:{$first:"$value.items"}
}
},
{
"$project":{
value_id: "$oldest._id"
}
}
]
I have a collection of documents which contain unique id field. Now I have a list of ids which may contain some ids that do not exist in the collection. What's the best way to find out those ids from the list?
I know I can use $in operator to get the documents which have ids contained in the list then compare with the given id list, but is there better way to do it?
I suppose you have the following documents in your collection:
{ "_id" : ObjectId("55b725fd7279ca22edb618bb"), "id" : 1 }
{ "_id" : ObjectId("55b725fd7279ca22edb618bc"), "id" : 2 }
{ "_id" : ObjectId("55b725fd7279ca22edb618bd"), "id" : 3 }
{ "_id" : ObjectId("55b725fd7279ca22edb618be"), "id" : 4 }
{ "_id" : ObjectId("55b725fd7279ca22edb618bf"), "id" : 5 }
{ "_id" : ObjectId("55b725fd7279ca22edb618c0"), "id" : 6 }
and the following list of id
var listId = [ 1, 3, 7, 9, 8, 35 ];
We can use the .filter method to return the array of ids that is not in your collection.
var result = listId.filter(function(el){
return db.collection.distinct('id').indexOf(el) == -1; });
This yields
[ 7, 9, 8, 35 ]
Now you can also use the aggregation frameworks and the $setDifference operator.
db.collection.aggregate([
{ "$group": { "_id": null, "ids": { "$addToSet": "$id" }}},
{ "$project" : { "missingIds": { "$setDifference": [ listId, "$ids" ]}, "_id": 0 }}
])
This yields:
{ "missingIds" : [ 7, 9, 8, 35 ] }
Unfortunately MongoDB can only use built in functions (otherwise I'd recommend using a set) but you could try and find all distinct id's in your list then just manually pull them out.
Something like (untested):
var your_unique_ids = ["present", "not_present"];
var present_ids = db.getCollection('your_col').distinct('unique_field', {unique_field: {$in: your_unique_ids}});
for (var i=0; i < your_unique_ids.length; i++) {
var some_id = your_unique_ids[i];
if (present_ids.indexOf(some_id) < 0) {
print(some_id);
}
}
Below query will fetch you the result :
var listid = [1,2,3,4];
db.collection.aggregate([
{$project: { uniqueId :
{
"$setDifference":
[ listid , db.collection.distinct( "unique_field" )]} , _id : 0 }
},
{$limit:1}
]);