I have following document structure (This is dummy document for understanding purpose)
{
"id" : "p1245",
"Info" : [
{
"cloth_name" : "ABC",
"cloth_type" : "C"
},
{
"cloth_name" : "PQR",
"cloth_type" : "J"
},
{
"cloth_name" : "SAM",
"cloth_type" : "T"
}
]
},
{
"id" : "p124576",
"Info" : [
{
"cloth_name" : "HTC",
"cloth_type" : "C"
}
]
}
From these document I want to project the "cloth_type", so I tried following java code
DBObject fields = new BasicDBObject("id", 1);
fields.put("ClothType","$Info.cloth_type");
DBObject project = new BasicDBObject("$project", fields);
List<DBObject> pipeline = Arrays.asList(project);
AggregationOptions aggregationOptions = AggregationOptions.builder().batchSize(100).outputMode(AggregationOptions.OutputMode.CURSOR).allowDiskUse(true).build();
Cursor cursor = collection.aggregate(pipeline, aggregationOptions);
while (cursor.hasNext())
{
System.out.println(cursor.next());
}
(I don't want to use "$unwind" here)
and get following output:
{ "id" : "p1245" , "ClothType" : [ "C" , "J" , "T"]}
{ "id" : "p124576" , "ClothType" : [ "C"]}
If there are multiple "cloth_type" for single id, then I want only the last cloth_type from this array.
I want something like, e.g. if there is array of "ClothType" [ "C", "J", "T"] then I want to project only [ "T"] i.e last element of array.
Is there any ways to achive this without using "$unwind".
Related
I want to find one document and clone/copy that document and create 100 new documents with new value for few fields using shell script in mongodb.
Below is my document
{
"_id" : ObjectId("5ef59bde562c9824176e9f20"),
"productDefinition" : {
"product" : {
"companies" : {
"company" : {
"productionformation" : {
"productNumber" : "E128",
"venderNumber" : "0470",
"venderName" : "ALPHA SERVICES LLC"
}
}
}
}
},
"executionId" : "123456"
}
After executing the shell script, i want to have new 100 collection with new values for the below fields
"executionId" : "NewExecutionId" // This value will be Fixed for all new 100 documents
"productNumber" : "1" //This value will be increasing.. for first document 1, for second document 2, etc..
"venderNumber" : "1" //This value will be increasing.. for first document 1, for second document 2, etc..
My new collection will be looking like this.
First new document
{
"_id" : ObjectId("5ef59bde562c9824176e9f20"),
"productDefinition" : {
"product" : {
"companies" : {
"company" : {
"productionformation" : {
"productNumber" : "1",
"venderNumber" : "1",
"venderName" : "ALPHA SERVICES LLC"
}
}
}
}
},
"executionId" : "newExecutionId"
}
Second new document
{
"_id" : ObjectId("5ef59bde562c9824176e9f20"),
"productDefinition" : {
"product" : {
"companies" : {
"company" : {
"productionformation" : {
"productNumber" : "2",
"venderNumber" : "2",
"venderName" : "ALPHA SERVICES LLC"
}
}
}
}
},
"executionId" : "newExecutionId"
}
Third new document
{
"_id" : ObjectId("5ef59bde562c9824176e9f20"),
"productDefinition" : {
"product" : {
"companies" : {
"company" : {
"productionformation" : {
"productNumber" : "3",
"venderNumber" : "3",
"venderName" : "ALPHA SERVICES LLC"
}
}
}
}
},
"executionId" : "newExecutionId"
}
Like this fourth document , fifth document, etc... till 100th document...
I tried with this script. but its not working.
copy = db.myCollection.find({"executionId" : "123456",
"productDefinition.product.companies.company.productionformation.productNumber" : "E128" ,
"productDefinition.product.companies.company.productionformation.venderNumber" :"0470" })
for (var i = 1; i< 101; i++){
copy.executionId = "newExecutionId";
copy.productDefinition.product.companies.company.productionformation.productNumber = i;
copy.productDefinition.product.companies.company.productionformation.venderNumber" = i;
db.myCollection.insert(copy);
}
You will be needing to fix following things:
Use findOne instead of find as it will return single matching document.
Use let (instead of var) while running the loop because there are asynchronous DB operations in loop body.
Similarly, create a Deep copy of matchedDoc result / (copy variable) inside for loop body, to avoid updating same object's reference value.
Hope it helps !
I have two collections.
1.Equipment
db.getCollection("Equipment").find({
$and: [
{ $where: 'this._id.length <= 7' },
{ "model": "A505"}
]})
{
"_id" : "1234567",
"locationId" : "DATALOAD",
"model" : "A505",
"subscriberId" : "",
"status" : "Stock",
"headendNumber" : "4"
}
{
"_id" : "P13050I",
"locationId" : "1423110302801",
"model" : "A505",
"subscriberId" : "37",
"status" : "Stock",
"headendNumber" : "4"
}
I will get more than 100 documents (rows) Equipment collection.
2.Subscriber
db.getCollection('Subscriber').find({})
{
"_id" : "5622351",
"equipment" : [
"0018015094E6",
"1234567",
"ADFB70878422",
"M10610TCB052",
"MA1113FHQ151"
]
}
{
"_id" : "490001508063",
"equipment" : [
"17616644510288",
"P13050I",
"M91416EA4251",
"128552270280560"
]
}
In the Subscriber collection, I need to remove (get all the id from Equipment collection loop it) only the matches equipment field.
Forex from the above result, I need to remove only "1234567", and "P13050I"
Expected output.
db.getCollection('Subscriber').find({})
{
"_id" : "5622351",
"equipment" : [
"0018015094E6",
"ADFB70878422",
"M10610TCB052",
"MA1113FHQ151"
]
}
{
"_id" : "490001508063",
"equipment" : [
"17616644510288",
"M91416EA4251",
"128552270280560"
]
}
Please help me, anyone.
You can use the following to update records.
Let's find records which need to deleted and store them in array
var equipments = [];
db.getCollection("Equipment").find({ $and: [
{ $where: 'this._id.length <= 7' },
{ "model": "A505"}
]}).forEach(function(item) => {
equipments.push(item._id)
})
Now, iterate over records of the second collection and update if required.
db.getCollection('Subscriber').find({}).forEach(function(document) => {
var filtered = document.equiment.filter(id => equipments.indexOf(id) < 0);
if(filtered.length < document.equipment.length){
db.getCollection('Subscriber').update({"_id": document.id }, { $set: {'equipment': filtered}})
}
})
.filter(id => equipments.indexOf(id) < 0) will keep entries which is not present in initially populated array equipments and it will persist if there is any change.
I have below 3 documents. Each represents a contact for a user :
{
"_id" : ObjectId("57f9f9f3b91d070315273d0d"),
"profileId" : "test",
"displayName" : "duplicateTest",
"email" : [
{
"emailId" : "a#a.com"
},
{
"emailId" : "b#b.com"
},
{
"emailId" : "c#c.com"
}
]
}
{
"_id" : ObjectId("57f9fab2b91d070315273d11"),
"profileId" : "test",
"displayName" : "duplicateTest2",
"email" : [
{
"emailId" : "a#a.com"
}
]
}
{
"_id" : ObjectId("57f9fcefb91d070315273d15"),
"profileId" : "test",
"displayName" : "duplicateTest2",
"email" : [
{
"emailId" : "b#b.com"
}
]
}
I need to aggregate/group them by array elements so that I can identify the duplicate contact ( based on email id). Since there is a common email id between doc (1 & 2) and doc( 1 & 3) these 3 represent one contact and should be merged into one as one contact.
I tried doing this using $unwind and $group in java as below:
List<DBObject> aggList = new ArrayList<DBObject>();
BasicDBObject dbo = new BasicDBObject("$match", new BasicDBObject("profileId", "0fb72dcf-292b-4343-a0e7-1d613a803b1e"));
aggList.add(dbo);
BasicDBObject dboUnwind = new BasicDBObject("$unwind", "$email");
aggList.add(dboUnwind);
BasicDBObject dboGroup = new BasicDBObject("$group",
new BasicDBObject().append("_id", new BasicDBObject("name", "$email.emailId"))
.append("uniqueIds", new BasicDBObject("$addToSet", "$_id"))
.append("count", new BasicDBObject("$sum", 1)));
aggList.add(dboGroup);
BasicDBObject dboCount = new BasicDBObject("$match", new BasicDBObject("count", new BasicDBObject("$gte", 2)));
aggList.add(dboCount);
BasicDBObject dboSort = new BasicDBObject("$sort", new BasicDBObject("count",-1));
aggList.add(dboSort);
BasicDBObject dboLimit = new BasicDBObject("$limit", 10);
aggList.add(dboLimit);
AggregationOutput output = collection.aggregate(aggList);
System.out.println(output.results());
This groups docs by email id (and rightly so) but doesn't serves the purpose.
Any help would be highly appreciated.
I need to implement the feature where user can be prompted about the possible duplicate contacts in his repository. I need aggregation result to be something like:
[
{
"_id":{
"name":[
{
"emailId" : "a#a.com"
},
{
"emailId" : "b#b.com"
},
{
"emailId" : "c#c.com"
}
]
},
"uniqueIds":[
{
"$oid":"57f9fcefb91d070315273d15"
},
{
"$oid":"57f9fcefb91d070315273d11"
},
{
"$oid":"57f9fcefb91d070315273d15"
}
],
"count":3
},
So basically, I need _id for all possible duplicate contacts (there could be another group of duplicates with _ids list as above) so that I can prompt it to user and user can merge them at his will.
Hope its more clear now. Thanks!
Well your question differs a bit from the result you are seeking. Your inital question pointed me to the following aggregation:
db.table.aggregate(
[
{
$unwind: "$email"
},
{
$group: {
_id : "$email.emailId",
duplicates : { $addToSet : "$_id"}
}
}
]
);
This results in:
{
"_id" : "c#c.com",
"duplicates" : [
ObjectId("57f9f9f3b91d070315273d0d")
]
}
{
"_id" : "b#b.com",
"duplicates" : [
ObjectId("57f9fcefb91d070315273d15"),
ObjectId("57f9f9f3b91d070315273d0d")
]
}
{
"_id" : "a#a.com",
"duplicates" : [
ObjectId("57f9fab2b91d070315273d11"),
ObjectId("57f9f9f3b91d070315273d0d")
]
}
Grouped by EMail.
But the sample output you added to your question made this aggregation:
db.table.aggregate(
[
{
$unwind: "$email"
},
{
$group: {
_id : "$profileId",
emails : { $addToSet : "$email.emailId"},
duplicates : { $addToSet : "$_id"}
}
}
]
);
Which results in:
{
"_id" : "test",
"emails" : [
"c#c.com",
"b#b.com",
"a#a.com"
],
"duplicates" : [
ObjectId("57f9fcefb91d070315273d15"),
ObjectId("57f9fab2b91d070315273d11"),
ObjectId("57f9f9f3b91d070315273d0d")
]
}
This is the first of 7 test/example documents, in collection "SoManySins."
{
"_id" : ObjectId("51671bb6a6a02d7812000018"),
"Treats" : "Sin1 = Gluttony",
"Sin1" : "Gluttony",
"Favourited" : "YES",
"RecentActivity" : "YES",
"GoAgain?" : "YeaSure."
}
I would like to be able to query to retrieve any info in any position,
just by referring to the position. The following document,
{
"_id" : ObjectId("51671bb6a6a02d7812000018"),
"Sin1" : "Gluttony",
"?????????" : "??????",
"RecentActivity" : "YES",
"GoAgain?" : "YeaSure."
}
One could retrieve whatever might be in the 3rd key~value
pair. Why should one have to know ahead of time what the
data is, in the key? If one has the same structure for the
collection, who needs to know? This way, you can get
double the efficiency? Like having a whole lot of mailboxes,
and your app's users supply the key and the value; your app
just queries the dbs' documents' arrays' positions.
Clara? finally? I hope?
The sample document you've provided is not saved as an array in BSON:
{
"_id" : ObjectId("51671bb6a6a02d7812000018"),
"Sin1" : "Gluttony",
"?????????" : "??????",
"RecentActivity" : "YES",
"GoAgain?" : "YeaSure."
}
Depending on the MongoDB driver you are using, the fields here are typically represented in your application code as an associative array or hash. These data structures are not order-preserving so you cannot assume that the 3rd field in a given document will correspond to the same field in another document (or even that the same field ordering will be consistent on multiple fetches). You need to reference the field by name.
If you instead use an array for your fields, you can refer by position or select a subset of the array using the $slice projection.
Example document with an array of fields:
{
"_id" : ObjectId("51671bb6a6a02d7812000018"),
"fields": [
{ "Sin1" : "Gluttony" },
{ "?????????" : "??????" },
{ "RecentActivity" : "YES" },
{ "GoAgain?" : "YeaSure." }
]
}
.. and query to find the second element of the fields array (a $slice with skip 1, limit 1):
db.SoManySins.find({}, { fields: { $slice: [1,1]} })
{
"_id" : ObjectId("51671bb6a6a02d7812000018"),
"fields" : [
{
"?????????" : "??????"
}
]
}
This is one way to Query and get back data when you may not
know what the data is, but you know the structure of the data:
examples in Mongo Shell, and in PHP
// the basics, setup:
$dbhost = 'localhost'; $dbname = 'test';
$m = new Mongo("mongodb://$dbhost");
$db = $m->$dbname;
$CursorFerWrites = $db->NEWthang;
// defining a set of data, creating a document with PHP:
$TheFieldGenerator = array( 'FieldxExp' => array(
array('Doc1 K1'=>'Val A1','Doc1 K2'=>'ValA2','Doc1 K3'=>'Val A3'),
array('Doc2 K1'=>'V1','Doc2 K2'=>'V2','Doc2 K3'=>'V3' ) ) ) ;
// then write it to MongoDB:
$CursorFerWrites->save($TheFieldGenerator);
NOTE : In the Shell : This produces the same Document:
> db.NEWthang.insert({"FieldxExp" : [
{"Doc1 K1":"Val A1","Doc1 K2":"Val A2","Doc1 K3":"Val A3"},
{"Doc2 K1":"V1", "Doc2 K2":"V2","Doc2 K3":"V3"}
]
})
#
Now, some mongodb Shell syntax:
> db.NEWthang.find().pretty()
{
"_id" : ObjectId("516c4053baa133464d36e836"),
"FieldxExp" : [
{
"Doc1 K1" : "Val A1",
"Doc1 K2" : "Val A2",
"Doc1 K3" : "Val A3"
},
{
"Doc2 K1" : "V1",
"Doc2 K2" : "V2",
"Doc2 K3" : "V3"
}
]
}
> db.NEWthang.find({}, { "FieldxExp" : { $slice: [1,1]} } ).pretty()
{
"_id" : ObjectId("516c4053baa133464d36e836"),
"FieldxExp" : [
{
"Doc2 K1" : "V1",
"Doc2 K2" : "V2",
"Doc2 K3" : "V3"
}
]
}
> db.NEWthang.find({}, { "FieldxExp" : { $slice: [0,1]} } ).pretty()
{
"_id" : ObjectId("516c4053baa133464d36e836"),
"FieldxExp" : [
{
"Doc1 K1" : "Val A1",
"Doc1 K2" : "Val A2",
"Doc1 K3" : "Val A3"
}
]
}
Finally, how about write the Query in some PHP ::
// these will be for building the MongoCursor:
$myEmptyArray = array();
$TheProjectionCriteria = array('FieldxExp'=> array('$slice' => array(1,1)));
// which gets set up here:
$CursorNEWthang1 = new MongoCollection($db, 'NEWthang');
// and now ready to make the Query/read:
$ReadomgomgPls=$CursorNEWthang1->find($myEmptyArray,$TheProjectionCriteria);
and the second document will be printed out:
foreach ($ReadomgomgPls as $somekey=>$AxMongoDBxDocFromCollection) {
var_dump($AxMongoDBxDocFromCollection);echo '<br />';
}
Hope this is helpful for a few folks.
Similar to this question
Barrowing the data set, I have something similar to this:
{
'user_id':'{1231mjnD-32JIjn-3213}',
'name':'John',
'campaigns':
[
{
'campaign_id':3221,
'start_date':'12-01-2012',
},
{
'campaign_id':3222,
'start_date':'13-01-2012',
}
]
}
And I want to add a new key in the campaigns like so:
{
'user_id':'{1231mjnD-32JIjn-3213}',
'name':'John',
'campaigns':
[
{
'campaign_id':3221,
'start_date':'12-01-2012',
'worker_id': '00000'
},
{
'campaign_id':3222,
'start_date':'13-01-2012',
'worker_id': '00000'
}
]
}
How to insert/update a new key into an array of objects?
I want to add a new key into every object inside the array with a default value of 00000.
I have tried:
db.test.update({}, {$set: {'campaigns.worker_id': 00000}}, true, true)
db.test.update({}, {$set: {campaigns: {worker_id': 00000}}}, true, true)
Any suggestions?
I'm supposing that this operation will occur once, so you can use a script to handle it:
var docs = db.test.find();
for(var i in docs) {
var document = docs[i];
for(var j in document.campaigns) {
var campaign = document.campaigns[j];
campaign.worker_id = '00000';
}
db.test.save(document);
}
The script will iterate over all documents in your collection then over all campaigns in each document, setting the *worker_id* property.
At the end, each document is persisted.
db.test.update({}, {$set: {'campaigns.0.worker_id': 00000}}, true, true
this will update 0 element.
if you want to add a new key into every object inside the array you should use:
$unwind
example:
{
title : "this is my title" ,
author : "bob" ,
posted : new Date() ,
pageViews : 5 ,
tags : [ "fun" , "good" , "fun" ] ,
comments : [
{ author :"joe" , text : "this is cool" } ,
{ author :"sam" , text : "this is bad" }
],
other : { foo : 5 }
}
unwinding tags
db.article.aggregate(
{ $project : {
author : 1 ,
title : 1 ,
tags : 1
}},
{ $unwind : "$tags" }
);
result:
{
"result" : [
{
"_id" : ObjectId("4e6e4ef557b77501a49233f6"),
"title" : "this is my title",
"author" : "bob",
"tags" : "fun"
},
{
"_id" : ObjectId("4e6e4ef557b77501a49233f6"),
"title" : "this is my title",
"author" : "bob",
"tags" : "good"
},
{
"_id" : ObjectId("4e6e4ef557b77501a49233f6"),
"title" : "this is my title",
"author" : "bob",
"tags" : "fun"
}
],
"OK" : 1
}
After you could write simple updaiting query.