I would like to update a collection by transforming all documents from this form:
{
"_id" : "somestring i made",
"value" : {
"a" : 0.42361499999999996,
"b" : 3,
"c" : "foo",
"d" : "bar"
}
}
To this form (with new id's):
{
"_id" : ObjectId("77d987f6dsf6f76sa7676df"),
"a" : 0.42361499999999996,
"b" : 3,
"c" : "foo",
"d" : "bar"
}
So essentially take the fields out of the object "value" and reset the id to a real document id.
First get the document , convert to required format , remove the old doc and again insert the modified one .
Something like
db.collection.find({}).forEach(function(doc){
var obj = { a : doc.value.a,
b : doc.value.b,
c : doc.value.c,
d : doc.value.d};
db.collection.remove(doc);
db.collection.insert(obj);
});
Related
I am trying to write a query in mongo that will create a new table, loop through my data set, and insert the TopExecutiveTitle into the new table. I also would like it to keep count of each position and only insert a position into the table when it is new.
This is what I have so far. This code loops through my table and inserts the TopExectuiveTitle into a new table. However, it does not group them together and keep count. How do I write my query so that it will?
db.car.find().forEach( function (x) {
db.TopExecutiveTable.insert({Topexecutivetitle: x.Topexecutivetitle})
});
Here is a sample of a document in my database.
{
"_id" : ObjectId("5a22c8e562c2e489c5df70fa"),
"2016rank" : 1,
"Dealershipgroupname" : "AutoNation Inc.?",
"Address" : "200 S.W. 1st Ave.",
"City/State/Zip" : "Fort Lauderdale, FL 33301",
"Phone" : "(954) 769-7000",
"Companywebsite" : "www.autonation.com",
"Topexecutive" : "Mike Jackson",
"Topexecutivetitle" : "chairman & CEO",
"Totalnewretailunits" : "337,622",
"Totalusedunits" : "225,713",
"Totalfleetunits" : 3,
"Totalwholesaleunits" : "82,342",
"Total_units" : "649,415",
"Total_number_of _dealerships" : 260,
"Grouprevenuealldepartments*" : "$21,609,000,000",
"2015rank" : 1
}
The result I would like is something like this
"Topexecutivetitle" : "chairman & CEO"
"Count" : 3
"Topexecutivetitle" : "president"
"Count" : 7
}
To do this you need to use the aggregate function of mongo, something like this:
db.car.aggregate([
{
$group:{
_id:"$Topexecutivetitle",
count:{$sum:1}
}
},
{
$project:{
Topexecutivetitle:"$_id",
count:1,
_id:0
}
},
{
$out:"result"
}])
This will give you your desired output and store it into a new collection "result":
{
"_id" : "president",
"count" : 1.0
},
{
"_id" : "chairman & CEO",
"count" : 3.0
}
I am newbie in MongoDB. I have stored data inside mongoDB in below format
"_id" : ObjectId("51d5725c7be2c20819ac8a22"),
"chrom" : "chr22",
"pos" : 17060409,
"information" : [
{
"name" : "Category",
"value" : "3"
},
{
"name" : "INDEL",
"value" : "INDEL"
},
{
"name" : "DP",
"value" : "31"
},
{
"name" : "FORMAT",
"value" : "GT:PL:GQ"
},
{
"name" : "PV4",
"value" : "1,0.21,0.00096,1"
}
],
"sampleID" : "Job1373964150558382243283"
I want to update the value to 11 which has the name as Category.
I have tried below query:
db.VariantEntries.update({$and:[ { "pos" : 117199533} , { "sampleID" : "Job1373964150558382243283"},{"information.name":"Category"}]},{$set:{'information.value':'11'}})
but Mongo replies
can't append to array using string field name [value]
How one can form a query which will update the particular value?
You can use the $ positional operator to identify the first array element to match the query in the update like this:
db.VariantEntries.update({
"pos": 17060409,
"sampleID": "Job1373964150558382243283",
"information.name":"Category"
},{
$set:{'information.$.value':'11'}
})
In MongoDB you can't adress array values this way. So you should change your schema design to:
"information" : {
'category' : 3,
'INDEL' : INDEL
...
}
Then you can adress the single fields in your query:
db.VariantEntries.update(
{
{"pos" : 117199533} ,
{"sampleID" : "Job1373964150558382243283"},
{"information.category":3}
},
{
$set:{'information.category':'11'}
}
)
I have a below map:
var mapFunction = function() {
if(this.url.match(/http:\/\/test.com\/category\/.*?\/checkout/)) {
var key=this.em;
var value = {
url : 'checkout',
count : 1,
account_id:this.accId
}emit(key,value); };
if(this.url.match(/http:\/\/test.com\/landing/)) {
var key=this.em;
var value = {
url : 'landing',
count : 1,
account_id:this.accId
}emit(key,value); };
}
Then I have defined reduce something like below:
var reduceFunction = function (keys, values) {
var reducedValue = {count_checkout:0, count_landing:0};
for (var idx = 0; idx < values.length; idx++) {
if(values[idx].url=='checkout'){
reducedValue.count_checkout++;
}
else {
reducedValue.count_landing++;
}
}
return reducedValue;
}
Now, lets say I have only 1 record:
{
"_id" : ObjectId("516a7cff6dad5949ddf3f7b6"),
"ip" : "1.2.3.4",
"accId" : 123,
"em" : "testing#test.com",
"pgLdTs" : ISODate("2013-04-11T18:30:00Z"),
"url" : "http://test.com/category/prr/checkout",
"domain" : "www.test.com",
"pgUdTs" : ISODate("2013-04-14T09:55:11.682Z"),
"title" : "Test",
"ua" : "Mozilla",
"res" : "1024*768",
"rfr" : "www.google.com"
}
Now if I fire my map reduce like below:
db.test_views.mapReduce(mapFunction,reduceFunction,{out:{inline:1}})
The I get below result returned:
{
"_id" : "testing#test.com",
"value" : {
"url" : "checkout",
"count" : 1,
"account_id" : 123
}
}
So, its basically returning me the map. Now, if I go a add another document for this email id. Finally it becomes something like below.
{
"_id" : ObjectId("516a7cff6dad5949ddf3f7b6"),
"ip" : "1.2.3.4",
"accId" : 123,
"em" : "testing#test.com",
"pgLdTs" : ISODate("2013-04-11T18:30:00Z"),
"url" : "http://test.com/category/prr/checkout",
"domain" : "www.test.com",
"pgUdTs" : ISODate("2013-04-14T09:55:11.682Z"),
"title" : "Test",
"ua" : "Mozilla",
"res" : "1024*768",
"rfr" : "www.google.com"
}
{
"_id" : ObjectId("516a7e1b6dad5949ddf3f7b7"),
"ip" : "1.2.3.4",
"accId" : 123,
"em" : "testing#test.com",
"pgLdTs" : ISODate("2013-04-11T18:30:00Z"),
"url" : "http://test.com/category/prr/checkout",
"domain" : "www.test.com",
"pgUdTs" : ISODate("2013-04-14T09:59:55.326Z"),
"title" : "Test",
"ua" : "Mozilla",
"res" : "1024*768",
"rfr" : "www.google.com"
}
Then, I go again and fire the map reduce, it gives me proper results
{
"_id" : "testing#test.com",
"value" : {
"count_checkout" : 2,
"count_landing" : 0
}
}
Can anyone please help me out in understanding why it returns me a map for single document and doesn't do the counting in reduce.
Thanks for help.
-Lalit
Can anyone please help me out in understanding why it returns me a map for single document and doesn't do the counting in reduce.
The Reduce step combines documents with the same key into a single result document. If you only have one key in the data emitted by your Map function, the data is already "reduced" and the reduce() will not be called.
This is the expected behaviour of the MapReduce algorithm.
The reduce function should return the same type of value objects as the map function emits.
Like you've experienced, when there's a single value associated with a key - the reduce function will not be called at all .
From the MongoDB MapReduce Documentation:
Requirements for the reduce Function:
...
the type of the return object must be identical to the type of the value emitted by the map function to ensure that the following operations is true:
reduce(key, [ C, reduce(key, [ A, B ]) ] ) == reduce( key, [ C, A, B ] )
I have some data that looks like this:
[
{
"_id" : ObjectId("4e2f2af16f1e7e4c2000000a"),
"advertisers" : [
{
"created_at" : ISODate("2011-07-26T21:02:19Z"),
"category" : "Infinity Pro Spin Air Brush",
"updated_at" : ISODate("2011-07-26T21:02:19Z"),
"lowered_name" : "conair",
"twitter_name" : "",
"facebook_page_url" : "",
"website_url" : "",
"user_ids" : [ ],
"blog_url" : "",
},
and I was thinking that a query like this would give the id of the advertiser:
var start = new Date(2011, 1, 1);
> var end = new Date(2011, 12, 12);
> db.agencies.find( { "created_at" : {$gte : start , $lt : end} } , { _id : 1 , program_ids : 1 , advertisers { name : 1 } } ).limit(1).toArray();
But my query didn't work. Any idea how I can add the fields inside the nested elements to my list of fields I want to get?
Thanks!
Use dot notation (e.g. advertisers.name) to query and retrieve fields from nested objects:
db.agencies.find({
"advertisers.created_at": {
$gte: start,
$lt: end
}
},
{
_id: 1,
program_ids: 1,
"advertisers.name": 1
}
}).limit(1).toArray();
Reference: Retrieving a Subset of Fields
and Dot Notation
db.agencies.find(
{ "advertisers.created_at" : {$gte : start , $lt : end} } ,
{ program_ids : 1 , advertisers.name : 1 }
).limit(1).pretty();
There is one thing called dot notation that MongoDB provides that allows you to look inside arrays of elements. Using it is as simple as adding a dot for each array you want to enter.
In your case
"_id" : ObjectId("4e2f2af16f1e7e4c2000000a"),
"advertisers" : [
{
"created_at" : ISODate("2011-07-26T21:02:19Z"),
"category" : "Infinity Pro Spin Air Brush",
"updated_at" : ISODate("2011-07-26T21:02:19Z"),
"lowered_name" : "conair",
"twitter_name" : "",
"facebook_page_url" : "",
"website_url" : "",
"user_ids" : [ ],
"blog_url" : "",
},
{ ... }
If you want to go inside the array of advertisers to look for the property created_at inside each one of them, you can simply write the query with the property {'advertisers.created_at': query} like follows
db.agencies.find( { 'advertisers.created_at' : { {$gte : start , $lt : end} ... }
When i don't use pagination, everything works fine (i have only 3 records in this collection, so all of them are listed here):
db.suppliers.find({location: {$near: [-23.5968323, -46.6782386]}},{name:1,badge:1}).sort({badge:-1})
{ "_id" : ObjectId("4f33ff549112b9b84f000070"), "badge" : 3, "name" : "Dedetizadora Alvorada" }
{ "_id" : ObjectId("4f33ff019112b9b84f00005b"), "badge" : 2, "name" : "Sampex Desentupidora e Dedetizadora" }
{ "_id" : ObjectId("4f33feae9112b9b84f000046"), "badge" : 1, "name" : "Higitec Desentupimento e Dedetização" }
But when i try to paginate from the first to the second page, one record doesn't show up and one is repeated:
db.suppliers.find({location: {$near: [-23.5968323, -46.6782386]}},{name:1,badge:1}).sort({badge:-1}).skip(0).limit(2)
{ "_id" : ObjectId("4f33ff549112b9b84f000070"), "badge" : 3, "name" : "Dedetizadora Alvorada" }
{ "_id" : ObjectId("4f33feae9112b9b84f000046"), "badge" : 1, "name" : "Higitec Desentupimento e Dedetização" }
db.suppliers.find({location: {$near: [-23.5968323, -46.6782386]}},{name:1,badge:1}).sort({badge:-1}).skip(2).limit(2)
{ "_id" : ObjectId("4f33feae9112b9b84f000046"), "badge" : 1, "name" : "Higitec Desentupimento e Dedetização" }
Am i doing something wrong or is this some kind of bug?
edit:
Here is a workaround for this. Basically you shouldn't mix $near queries with sorting; use $within instead.
There is an open issue regarding the same problem. Please have a look & vote Geospatial result paging fails when sorting with additional keys