Finding a distinct set of fields in MongoDB - mongodb

I have the following JSON collection in MongoDB.
{
"_id": ObjectId("57529381551673386c9150a6"),
"team_code": 3,
"team_id": 2
},
{
"_id": ObjectId("57529381551673386c91514a"),
"team_code": 4,
"team_id": 5
},
{
"_id": ObjectId("57529381551673386c91514b"),
"team_code": 3,
"team_id": 2
},
{
"_id": ObjectId("57529381551673386c91514c"),
"team_code": 4,
"team_id": 5,
}
As it can be seen from the data , there a 2 records each with (team_code=3, team_id =2) and (team_code =4, team_id=5).
Is it possible to get the distinct set of team codes and team ids.
Some thing like ,
{
"team_code" : 3, "team_id" : 2
},
{
"team_code" : 4, "team_id" : 5,
}

You can do this using the following Aggregation Pipeline:
var distinctIdCode = { $group: { _id: { team_code: "$team_code", team_id: "$team_id" } } }
db.foo.aggregate([distinctIdCode])
This will give you:
{ "_id" : { "team_code" : 4, "team_id" : 5 } }
{ "_id" : { "team_code" : 3, "team_id" : 2 } }
This query returns new documents created from the documents in your collection. The documents returned have an _id which is the result of grouping the collection documents on the team_code and team_id fields.

For Java code:
Bson field1 = new Document("team_code", "$team_code").append("team_id", "$team_id");Bson groupField = new Document("_id", field1 );
Bson group = new Document("$group", groupField );List<Bson> bsonList = new ArrayList<Bson>();
bsonList.add(group);
com.mongodb.client.AggregateIterable<Document> res=collection.aggregate( bsonList );
for (Document doc : res) {
Document subDoc=(Document)doc.get("_id");
System.out.println(subDoc.get("team_code")+" : "+subDoc.get("team_id"));
//deptEnvs.put(subDoc.getInteger("dept"), subDoc.getString("smEnv"));
}

Related

Display field document in mongoDB after execute Query aggregate

This is an example of a data document
{
"_id" : ObjectId("5f437e7846103b2ad0fc5d7d"),
"order_no" : "O-200824-AGFJDQW",
"shipment_no" : "S-200824-AGWCRRM",
"member_id" : 2200140,
"ponta_id" : "9990010100280214",
"plu" : 14723,
"product_name" : "AQUA Air Mineral Botol Air Pet 600ml",
"qty" : 2,
"store_id" : "TD46",
"stock_on_hand" : 0,
"transaction_date" : ISODate("2020-08-24T08:28:29.931Z"),
"created_date" : ISODate("2020-08-24T08:46:48.441Z")
}
this is the data query that I run
var bulan = 12 //month is written with number. example: August = 8
db.log_stock_oos.aggregate([
{
$project: {
month: {
$month: '$transaction_date'
}
}
},
{
$match: {month: bulan}
}
]);
but the result is like this after I run the query
{
"_id" : ObjectId("5f44689607fe453fbfba433e"),
"month" : 12
}
how to make the output exactly like the document display that I attached above??
this is my reference
When you use the projection, its kind of if your value 1 then include the field, if your value 0 then exclude the field from the whole documents. Projection
You can do two things
Use the projection
db.collection.aggregate([
{
$project: {
month: {
$month: "$transaction_date"
},
order_no: 1,
shipment_no: 1,
member_id: 1,
//other fields like above with the value 1
}
},
// match stages
])
Use $addFields
use $addFields incited of $project in your code. If will create a filed if not exists in your document, else it will overwrite the field

Update existing mongodb data into an embedded document

I am new to MongoDB so this is probably a basic question (hopefully). I currently have 10 million records with 410 fields loaded in a mongodb collection like so:
{
"_id" : ObjectId("........"),
"AddressID" : 123455,
"IndividualId" : 1,
"personfirstname" : "FirstName",
"personmiddleinitial" : "M",
"personlastname" : "LastName",
"etc": "....."
}
I need to wrap all of this data into an embedded document like so:
{
"_id" : ObjectId("........"),
"data" : {
"AddressID" : 123455,
"IndividualId" : 1,
"personfirstname" : "FirstName",
"personmiddleinitial" : "M",
"personlastname" : "LastName",
"etc": "....."
}
I don't necessarily need to update this data in-place but that would be nice. If I need to export this data somehow specifying the new format and then re-import the new, updated data that is fine. Performing this via the MongoDB shell would be ideal.
As suggested by chridam within comments you can execute the following aggregation pipeline:
db.collectionName.aggregate([
{ $project: { _id: "$_id", data: "$$ROOT" } },
{ $out: "newCollectionName" }
]);
This way you have the _id field both at root level and in the data object. Thus, you can execute a massive update to unset the second one:
db.newCollectionName.updateMany(
{},
{ $unset: { "data._id": "" } }
);
Finally, you can drop the first collection and rename the second to restore the original name on the updated collection:
db.collectionName.drop();
db.newCollectionName.rename("collectionName");
This approach fully works within the database, avoiding fetching any of your 10 million documents.
You can simply do this in the shell with the following
db.test.find().forEach(function(doc){
doc = { _id: doc._id, data: doc };
delete doc.data._id;
db.test.save(doc);
});
For example, if we insert the following documents:
> db.test.insertMany([
... {
... _id: ObjectId("5a91af8908e17c5997e03b7e"),
... field1: false,
... field2: 0,
... field3: "No"
... },
... {
... _id: ObjectId("5a91afbc08e17c5997e03b7f"),
... field1: true,
... field2: 1,
... field3: "Yes"
... }])
{
"acknowledged" : true,
"insertedIds" : [
ObjectId("5a91af8908e17c5997e03b7e"),
ObjectId("5a91afbc08e17c5997e03b7f")
]
}
Then run:
db.test.find().forEach(function(doc){
doc = { _id: doc._id, data: doc };
delete doc.data._id;
db.test.save(doc);
});
Our documents now look like this:
> db.test.find().pretty()
{
"_id" : ObjectId("5a91af8908e17c5997e03b7e"),
"data" : {
"field1" : false,
"field2" : 0,
"field3" : "No"
}
}
{
"_id" : ObjectId("5a91afbc08e17c5997e03b7f"),
"data" : {
"field1" : true,
"field2" : 1,
"field3" : "Yes"
}
}

How to add key-value pair to object in MongoDB

If I have a document with the following basic structure:
{
...
Monday: { a:1, b:2 },
Tuesday: { c:3, d:4 }
...
}
Am I able to 'push' an additional key:value pair to Monday's value? Result would be:
{
Monday: { a:1, b:2, z:8 },
Tuesday: { c:3, d:4 }
...
}
The $push operator seems to only work for arrays.
Just do something like that
db.foo.update({"_id" :ObjectId("...") },{$set : {"Monday.z":8}})
How to add a new key:value pair to all existing objects of a mongoDB documents
Old Key and Value Pairs
> db.students.find().pretty();
{ "_id" : ObjectId("601594f5a22527655335415c"), "name" : "Doddanna" }
{ "_id" : ObjectId("601594f5a22527655335415d"), "name" : "Chawan" }
Update New Key and Value Pairs Using updateMany() and $set
> db.students.updateMany({},{$set:{newKey1:"newValue1", newKey2:"newValue2", newKeyN:"newValueN"}});
{ "acknowledged" : true, "matchedCount" : 2, "modifiedCount" : 2 }
Have a look on Updated pretty result
> db.students.find().pretty();
{
"_id" : ObjectId("601594f5a22527655335415c"),
"name" : "Doddanna",
"newKey1" : "newValue1",
"newKey2" : "newValue2",
"newKeyN" : "newValueN"
}
{
"_id" : ObjectId("601594f5a22527655335415d"),
"name" : "Chawan",
"newKey1" : "newValue1",
"newKey2" : "newValue2",
"newKeyN" : "newValueN"
}
I know this might be irrelevant to the question but as a matter of fact, I opened this page because I was looking for an exact query with mongoose. here is my answer with mongoose.
If we have an abstract model (mongoose schema) named week in our javascript application then the code will be:
// javascript with mongoose
...
const key = "z";
const KeyValue = 8;
await week.updateOne({
_id, // mongoDb document id
},
{
$set:{
[`Monday.${key}`]: KeyValue,
},
},
{
upsert: true // options
},
);
...
var json = {
Monday: { a:1, b:2 },
Tuesday: { c:3, d:4 } }
json['Monday']['z'] = 8;
console.log(json);

get from array what's not in mongo [duplicate]

I have a collection of documents which contain unique id field. Now I have a list of ids which may contain some ids that do not exist in the collection. What's the best way to find out those ids from the list?
I know I can use $in operator to get the documents which have ids contained in the list then compare with the given id list, but is there better way to do it?
I suppose you have the following documents in your collection:
{ "_id" : ObjectId("55b725fd7279ca22edb618bb"), "id" : 1 }
{ "_id" : ObjectId("55b725fd7279ca22edb618bc"), "id" : 2 }
{ "_id" : ObjectId("55b725fd7279ca22edb618bd"), "id" : 3 }
{ "_id" : ObjectId("55b725fd7279ca22edb618be"), "id" : 4 }
{ "_id" : ObjectId("55b725fd7279ca22edb618bf"), "id" : 5 }
{ "_id" : ObjectId("55b725fd7279ca22edb618c0"), "id" : 6 }
and the following list of id
var listId = [ 1, 3, 7, 9, 8, 35 ];
We can use the .filter method to return the array of ids that is not in your collection.
var result = listId.filter(function(el){
return db.collection.distinct('id').indexOf(el) == -1; });
This yields
[ 7, 9, 8, 35 ]
Now you can also use the aggregation frameworks and the $setDifference operator.
db.collection.aggregate([
{ "$group": { "_id": null, "ids": { "$addToSet": "$id" }}},
{ "$project" : { "missingIds": { "$setDifference": [ listId, "$ids" ]}, "_id": 0 }}
])
This yields:
{ "missingIds" : [ 7, 9, 8, 35 ] }
Unfortunately MongoDB can only use built in functions (otherwise I'd recommend using a set) but you could try and find all distinct id's in your list then just manually pull them out.
Something like (untested):
var your_unique_ids = ["present", "not_present"];
var present_ids = db.getCollection('your_col').distinct('unique_field', {unique_field: {$in: your_unique_ids}});
for (var i=0; i < your_unique_ids.length; i++) {
var some_id = your_unique_ids[i];
if (present_ids.indexOf(some_id) < 0) {
print(some_id);
}
}
Below query will fetch you the result :
var listid = [1,2,3,4];
db.collection.aggregate([
{$project: { uniqueId :
{
"$setDifference":
[ listid , db.collection.distinct( "unique_field" )]} , _id : 0 }
},
{$limit:1}
]);

How to Avoid Duplicate Entries in MongoDb Meteor App

How to avoid duplicate entries in mongoDb in Meteor application.
On the command: db.products.find({},{"TEMPLATE_NAME": 1},{unique : true})
{ "_id" : ObjectId("5555d0a16ce3b01bb759a771"), "TEMPLATE_NAME" : "B" }
{ "_id" : ObjectId("5555d0b46ce3b01bb759a772"), "TEMPLATE_NAME" : "A" }
{ "_id" : ObjectId("5555d0c86ce3b01bb759a773"), "TEMPLATE_NAME" : "C" }
{ "_id" : ObjectId("5555d0f86ce3b01bb759a774"), "TEMPLATE_NAME" : "C" }
{ "_id" : ObjectId("5555d1026ce3b01bb759a775"), "TEMPLATE_NAME" : "A" }
{ "_id" : ObjectId("5555d1086ce3b01bb759a776"), "TEMPLATE_NAME" : "B" }
I want to retrieve only the unique template names and show them on HTML page.
Use the aggregation framework where your pipeline stages consist of the $group and $project operators respectively. The $group operator step groups the input documents by the given key and thus will return distinct documents in the result. The $project operator then reshapes each document in the stream, such as by adding new fields or removing existing fields:
db.products.aggregate([
{
"$group": {
"_id": "$TEMPLATE_NAME"
}
},
{
"$project": {
"_id": 0,
"TEMPLATE_NAME": "$_id"
}
}
])
Result:
/* 0 */
{
"result" : [
{
"TEMPLATE_NAME" : "C"
},
{
"TEMPLATE_NAME" : "A"
},
{
"TEMPLATE_NAME" : "B"
}
],
"ok" : 1
}
You could then use the meteorhacks:aggregate package to implement the aggregation in Meteor:
Add to your app with
meteor add meteorhacks:aggregate
Then simply use .aggregate function like below.
var products = new Mongo.Collection('products');
var pipeline = [
{
"$group": {
"_id": "$TEMPLATE_NAME"
}
},
{
"$project": {
"_id": 0,
"TEMPLATE_NAME": "$_id"
}
}
];
var result = products.aggregate(pipeline);
-- UPDATE --
An alternative that doesn't use aggregation is using underscore's methods to return distinct field values from the collection's find method as follows:
var distinctTemplateNames = _.uniq(Collection.find({}, {
sort: {"TEMPLATE_NAME": 1}, fields: {"TEMPLATE_NAME": true}
}).fetch().map(function(x) {
return x.TEMPLATE_NAME;
}), true)
;
This will return an array with distinct product template names ["A", "B", "C"]
You can check out some tutorials which explain the above approach in detail: Get unique values from a collection in Meteor and METEOR – DISTINCT MONGODB QUERY.
You can use distinct of mongodb like :
db.collectionName.distinct("TEMPLATE_NAME")
This query will return you array of distinct TEMPLATE_NAME