Geonear sort by distance and time - mongodb

I have the following data:
{
"_id" : ObjectId("55a8c1ba3996c909184d7a22"),
"uid" : "1db82e8a-2038-4818-b805-76a46ba62639",
"createdate" : ISODate("2015-07-17T08:50:02.892Z"),
"palce" : "aa",
"sex" : 1,
"longdis" : 1,
"location" : [ 106.607312, 29.575281 ]
}
{
"_id" : ObjectId("55a8c1ba3996c909184d7a24"),
"uid" : "1db82e8a-2038-4818-b805-76a46ba62639",
"createdate" : ISODate("2015-07-17T08:50:02.920Z"),
"palce" : "bbb",
"sex" : 1,
"longdis" : 1,
"location" : [ 106.589896, 29.545098 ]
}
{
"_id" : ObjectId("55a8c1ba3996c909184d7a25"),
"uid" : "1db82e8a-2038-4818-b805-76a46ba62639",
"createdate" : ISODate("2015-07-17T08:50:02.922Z"),
"palce" : "ccc",
"sex" : 1,
"longdis" : 1,
"location" : [ 106.590758, 29.566713 ]
}
{
"_id" : ObjectId("55a8c1ba3996c909184d7a26"),
"uid" : "1db82e8a-2038-4818-b805-76a46ba62639",
"createdate" : ISODate("2015-07-17T08:50:02.923Z"),
"palce" : "ddd",
"sex" : 1,
"longdis" : 1,
"location" : [ 106.637039, 29.561436 ]
}
{
"_id" : ObjectId("55a8c1bc3996c909184d7a27"),
"uid" : "1db82e8a-2038-4818-b805-76a46ba62639",
"createdate" : ISODate("2015-07-17T08:50:04.499Z"),
"palce" : "eee",
"sex" : 1,
"longdis" : 1,
"location" : [ 106.539522, 29.57929 ]
}
{
"_id" : ObjectId("55a8d12e78292fa3837ebae4"),
"uid" : "1db82e8a-2038-4818-b805-76a46ba62639",
"createdate" : ISODate("2015-07-17T09:55:58.947Z"),
"palce" : "fff",
"sex" : 1,
"longdis" : 1,
"location" : [ 106.637039, 29.561436 ]
}
I want to first of all, sort by the distance, if the distance is the same, sort by the time.
my command :
db.runCommand( {
geoNear: "paging",
near: [106.606033,29.575897 ],
spherical : true,
maxDistance : 1/6371,
minDistance:0/6371,
distanceMultiplier: 6371,
num:2,
query: {'_id': {'$nin': []}}
})
or
db.paging.find({
'location':{
$nearSphere: [106.606033,29.575897],
$maxDistance:1
}
}).limit(5).skip((2 - 1) * 2).sort({createdate:-1})
How can I sort on both "nearest" and "createddate"?

The correct query to use here uses the aggregation framework which has the $geoNear pipeline stage to assist with this. It's also the only place you get to "sort" by multiple keys, as unforntunately the "geospatial" $nearSphere does not have a "meta" projection for "distance" like $text has a "score".
Also the geoNear database command you are using can also not be used with "cursor" .sort() in that way either.
db.paging.aggregate([
{ "$geoNear": {
"near": [106.606033,29.575897 ],
"spherical": true,
"distanceField": "distance",
"distanceMuliplier": 6371,
"maxDistance": 1/6371
}},
{ "$sort": { "distance": 1, "createdate": -1 } },
{ "$skip": ( 2-1 ) * 2 },
{ "$limit": 5 }
])
That is the equivalent of what you are trying to do.
With the aggregation framework you use the "pipeline operators" instead of "cursor modifiers" to do things like $sort, $skip and $limit. Also these must be in a Logical order, whereas the cursor modifiers generally work it out.
It's a "pipeline", just like "Unix pipe". |
Also, be careful with "maxDistance" and "distanceMuliplier". Since your co-ordinates are in "legacy co-ordinate pairs" and not GeoJSON format, then the distances are measured in "radians". If you have GeoJSON stored location data then the result is returned in "meters".

Related

Get matched embedded document(s) from array

I've got a lot of documents using the following structure in MongoDB:
{
"_id" : ObjectId("..."),
"plant" : "XY_4711",
"hour" : 1473321600,
"units" : [
{
"_id" : ObjectId("..."),
"unit_id" : 10951,
"values" : [
{
"quarter" : 1473321600,
"value" : 395,
},
{
"quarter" : 1473322500,
"value" : 402,
},
{
"quarter" : 1473323400,
"value" : 406,
},
{
"quarter" : 1473324300,
"value" : 410,
}
]
}
]
}
Now I need to find all embedded document values where the quarter is between some given timestamps (eg: { $gte: 1473324300, $lte: 1473328800 }).
I've only got the unit_id and the quarter timestamp from/to for filtering the documents. And I only need the quarter and value grouped and ordered by unit.
I'm new in MongoDB and read something about find() and aggregate(). But I don't know how to do it. MongoDB 3.0 is installed on the server.
Finally I've got it:
I simply have to take apart each array, filtering out the things I don't need and put it back together:
db.collection.aggregate([
{$match : {$and : [{"units.values.quarter" : {$gte : 1473324300}}, {"units.values.quarter" : {$lte : 1473328800 }}]}},
{$unwind: "$units"},
{$unwind: "$units.values"},
{$match : {$and : [{"units.values.quarter" : {$gte : 1473324300}}, {"units.values.quarter" : {$lte : 1473328800 }}]}},
{$project: {"units": {values: {quarter: 1, "value": 1}, unit_id: 1}}},
{$group: {"_id": "$units.unit_id", "quarter_values": {$push: "$units.values"}}} ,
{$sort: {"_id": 1}}
])
Will give:
{
"_id" : 10951,
"quarter_values" : [
{
"quarter" : 1473324300,
"value" : 410
},
{
"quarter" : 1473325200,
"value" : 412
},
{
"quarter" : 1473326100,
"value" : 412
},
{
"quarter" : 1473327000,
"value" : 411
},
{
"quarter" : 1473327900,
"value" : 408
},
{
"quarter" : 1473328800,
"value" : 403
}
]
}
See: Return only matched sub-document elements within a nested array for a detailed description!
I think I have to switch to $map or $filter in the future. Thanks to notionquest for supporting my questions :)
Please see the sample query below. I didn't exactly get your grouping requirement. However, with this sample query you should be able to change and get your desired output.
db.collection.aggregate([
{$unwind : {path : "$units"}},
{$match : {$and : [{"units.values.quarter" : {$gte : 1473324300}}, {"units.values.quarter" : {$lte : 1473328800 }}]}},
{$project : {"units" : {values : {quarter : 1, "value" : 1}, unit_id : 1}}},
{$group : { _id : "$units.unit_id", quarter_values : { $push :{ quarter : "$units.values.quarter", value : "$units.values.value"}}}},
{$sort : {_id : 1 }}
]);
Sample output:-
{
"_id" : 10951,
"quarter_values" : [
{
"quarter" : [
1473321600,
1473322500,
1473323400,
1473324300
],
"value" : [
395,
402,
406,
410
]
}
]
}

mongodb $near query is slow

One mongodb collection
{
"_id" : ObjectId("574bbae4d009b5364abaebe5"),
"cityid" : 406,
"location" : {
"type" : "Point",
"coordinates" : [
118.602355,
24.89083
]
},
"shopid" : "a"
}
with about 50, 000 rows;
and indexes:
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "pingan-test.shop_actinfo_collection_0530"
},
{
"v" : 1,
"key" : {
"location" : "2dsphere"
},
"name" : "location_2dsphere",
"ns" : "pingan-test.shop_actinfo_collection_0530",
"2dsphereIndexVersion" : 3
},
{
"v" : 1,
"key" : {
"shopid" : 1,
"cityid" : 1
},
"name" : "shopid_1_cityid_1",
"ns" : "pingan-test.shop_actinfo_collection_0530"
}
]
I query this collection like:
body = {'cityid': 2, 'location': {'$near': {'$geometry': {'type': 'Point', 'coordinates': [122.0, 31.0]}}}, 'shopid': {'$in': ['a','b']}}
results = collection.find(body, {'shopid': 1, '_id':0},).batch_size(20).limit(20)
shops = list(results)
The question is that it run about 400ms. But it just take 30ms if we don't care about location.
why and how to fix? please.
You have an index on shopid and cityid, but you search for cityid. Since the index is ordered by shopid first it cannot be used to search by cityid. If you change the index to cityid: 1, shopid: 1, then you will see a performance improvement because your query will be able to search using the index.
after all, i got it.
I just create a index to cityid: 1, shopid: 1, "location" : "2dsphere"
, and then, world peace。
and thanks #tiramisu again.

Mongodb aggregation: how to use unwind->group->project multiple times

I have an orders collection where I need to calculate some sums from multiple sub-arrays arrays but I can't figure out how to loose the multiplied items that the double unwind creates.
db.Orders.aggregate(
{$unwind: "$items"},
{$unwind: "$shipping"},
{$group: {
_id: {
year: { '$year': '$createdAt' },
month: { '$month': '$createdAt' },
day: { '$dayOfMonth': '$createdAt' }
},
mainItems: { $addToSet: '$items' },
totalSales: {$sum: {
$multiply: ["$items.quantity", "$items.variants.price"]
}},
averageSales: {$avg: {$multiply: ["$items.quantity", "$items.variants.price"]}},
/* this will not sum the individial orders because the unwind
* created multiple document per order*/
ordersPlaced: {$sum: 1},
itemsPurchased: {$sum: "$items.quantity"},
totalRefundAmount: {$sum: 0},
chargedForShipping: {$sum: "$shipping.shipmentMethod.rate"}
}}
)
If I take out the shipping from the unwind and the group the query will return the correct values except for the chargedForShipping (0 since it's unwinded) and ordersPlaces which will still be more than expected (but I also need the shipping information and even more additional ones that I took out for easier understanding).
Sample data:
[{
"_id" : "xK29ZHxGcYvgWgx5p",
"sessionId" : "yw7e9G7uBzYTy9Grq",
"userId" : "fZREMm2DmsnMosMKj",
"shopId" : "oiqQDnuBwabj44q2o",
"billing" : [
{
"shopId" : "oiqQDnuBwabj44q2o",
"_id" : "9TMJj9w65MmAkgg27",
"paymentMethod" : {
"amount" : 22.45,
"status" : "settled",
"mode" : "capture",
"transactionId" : "AP",
"createdAt" : ISODate("2016-02-15T13:44:35.116Z"),
"transactions" : [
{ type:"refund", amount:5}
]
}
},
{
"shopId" : "9YfkXWyCci8fN43Pj",
"_id" : "RwW8xMnFzQqdTpqtg",
"paymentMethod" : {
"createdAt" : ISODate("2016-02-15T13:44:35.116Z")
}
},
{
"shopId" : "SgXWPKGJkxBw6qsbT",
"_id" : "ASizt6BtkxpCxgEJn",
"paymentMethod" : {
"createdAt" : ISODate("2016-02-15T13:44:35.116Z")
}
}
],
"shipping" : [
{
"_id" : "yXb5T5zLuxPYmgoT5",
"shipmentMethod" : {
"name" : "Continental US",
"_id" : "womiJX2QZBFQQWFur",
"rate" : 9.949999999999999,
"shopId" : "9YfkXWyCci8fN43Pj",
},
"items" : [
{
"_id" : "48s9bmDfrRMqnkije",
"productId" : "KXtF5xqERWJsXk2yP",
"shopId" : "SgXWPKGJkxBw6qsbT",
"variantId" : "YQDHuyPHbhx4wruZx",
"quantity" : 1
}
],
"packed" : false,
"shipped" : false,
}
],
"items" : [
{
"_id" : "hhuiGFTBkLACLpPjQ",
"shopId" : "9YfkXWyCci8fN43Pj",
"productId" : "sDYNXMrnRJiyQ8gex",
"quantity" : 1,
"variants" : {
"_id" : "muJi6Bqnq2CD8B7AR",
"price" : 2.5,
"title" : "egy",
"weight" : 23,
},
"type" : "simple",
},
{
"_id" : "48s9bmDfrRMqnkije",
"shopId" : "SgXWPKGJkxBw6qsbT",
"productId" : "KXtF5xqERWJsXk2yP",
"quantity" : 1,
"variants" : {
"_id" : "YQDHuyPHbhx4wruZx",
"title" : "Bogi varinat title",
"price" : 10,
"type" : "variant",
"compareAtPrice" : 100000,
"weight" : 100,
},
"type" : "simple",
}
],
"email" : "test#user.com",
"createdAt" : ISODate("2016-02-15T13:44:35.091Z"),
"updatedAt" : ISODate("2016-02-15T14:24:55.174Z")
}]
What I would need is orderTotal per month, shippingTotal per month, totalRefunded per month, average sales per month. The issue is one I need from the items sub-array the other from the shipping sub-array and the third from the billing sub-array that is why I have issues with the unwind.

MongoDB aggregate $match and $group with $sum

i have a collection with documents like this:
{
"Company" : "4433",
"Descripcion" : "trabajo",
"Referencia" : "11817",
"HoraImportado" : "15:54",
"ImportedOd" : "2014-05-20T13:54:28.493Z",
"Items" : [],
"Notes" : [
{
"_id" : ObjectId("537b5ea4c61b1d1743f43420"),
"NoteDateTime" : "2014-05-20T13:54:44.418Z",
"Description" : "nota",
"IsForTechnician" : true,
"Username" : "admin"
},
{
"_id" : ObjectId("537c4a549e956f77ab8c7c38"),
"NoteDateTime" : ISODate("2014-05-21T06:40:20.299Z"),
"Description" : "ok",
"IsForTechnician" : true,
"Username" : "admin"
}
],
"OrderState" : "Review",
"SiniestroDe" : "Emergencia",
"Technicians" : [
{
"TechnicianId" : ObjectId("53465f9d519c94680327965d"),
"Name" : "Administrator",
"AssignedOn" : ISODate("2014-05-20T13:54:44.373Z"),
"RemovedOn" : null
}
],
"TechniciansHistory" : [
{
"TechnicianId" : ObjectId("53465f9d519c94680327965d"),
"Name" : "Administrator",
"AssignedOn" : ISODate("2014-05-20T13:54:44.373Z"),
"RemovedOn" : null
},
{
"Name" : "Nuevo",
"AssignedOn" : ISODate("2014-05-20T13:54:44.373Z"),
"RemovedOn" : null,
"TechnicianId" : ObjectId("5383577a994be8b9a9e3f01e")
}
],
"Telefonos" : "615554006",
"_id" : ObjectId("537b5ea4c61b1d1743f4341f"),
"works" : [
{
"code" : "A001",
"name" : "Cambiar bombilla",
"orderId" : "537b5ea4c61b1d1743f4341f",
"price" : "11",
"ID" : 33,
"lazyLoaded" : true,
"status" : 0,
"Date" : ISODate("2014-05-21T06:40:20.299Z"),
"TechnicianId" : "53465f9d519c94680327965d",
"_id" : ObjectId("537c4a549e956f77ab8c7c39")
},
{
"code" : "A001",
"name" : "Cambiar bombilla",
"orderId" : "537b5ea4c61b1d1743f4341f",
"price" : "11",
"ID" : 34,
"lazyLoaded" : true,
"status" : 0,
"Date" : ISODate("2014-05-21T06:40:20.299Z"),
"TechnicianId" : "53465f9d519c94680327965d",
"_id" : ObjectId("537c4a549e956f77ab8c7c3a")
}
]
}
Now i want to get the works for a selected TechnicianId array, group by TechnicianId and get the sum of the works.price for each technician.+
I try with this:
db.orders.aggregate([
{ $match: { 'works.TechnicianId': {$in:['53465f9d519c94680327965d']}}},
{ $group: { _id: "$works.TechnicianId",total:{$sum:'$works.price'}}},
])
And this is the result:
{
"result" : [
{
"_id" : [
"53465f9d519c94680327965d",
"53465f9d519c94680327965d"
],
"total" : 0
}
],
"ok" : 1
}
The total its the $sum but its 0 but should be 44.
Try adding unwind,
db.orders.aggregate([
{ $match: { 'works.TechnicianId': {$in:['53465f9d519c94680327965d']}}},
{ $unwind: "$works" },
{ $group: { _id: "$works.TechnicianId",total:{$sum:'$works.price'}}},
])
Look here for more info : http://docs.mongodb.org/manual/reference/operator/aggregation/unwind/
The price value is a string. $sum only operates on Numbers.
I've checked this by running the following:
db.foo.insert({"cost": "1"})
db.foo.insert({"cost": "2"})
db.foo.insert({"cost": "3"})
db.foo.insert({"cost": 4})
db.foo.insert({"cost": 5})
db.foo.aggregate([{$group: {_id: null, cost: {$sum: "$cost"}}}])
{ "result" : [ { "_id" : null, "cost" : 9 } ], "ok" : 1 }
According to this answer, you can't cast values in normal Mongo queries, so you can't change the string to a number inline.
You should either update all values to a Number datatype or use map-reduce. I'd go for the former.
If the value is a string to prevent floating point errors, consider multiplying by 100 to store the value in cents: "10.50" --> 1050
As Lalit Agarwal indicated, you'll also need to unwind the array of works. Example of what happens if you don't:
db.bar.insert({"works": [{price: 10}]})
db.bar.insert({"works": [{price: 20}, {price: 30}]})
db.bar.insert({"works": [{price: 40}, {price: 50}]})
db.bar.aggregate([
{$group: {_id: null, total: {$sum: "$works.price"} }}
])
{ "result" : [ { "_id" : null, "total" : 0 } ], "ok" : 1 }
db.bar.aggregate([
{$unwind: "$works"},
{$group: {_id: null, total: {$sum: "$works.price"} }}
])
{ "result" : [ { "_id" : null, "total" : 150 } ], "ok" : 1 }
What $unwind does is make 5 documents out of the initial 3, all with a single value in the works field. It then groups and sums them.
db.inventory.insert(
{
item: “ABC1”,
details: {
model: “14Q3”,
manufacturer: “XYZ Company”
},
stock: [ { size: “S”, qty: 25 }, { size: “M”, qty: 50 } ],
category: “clothing”
}
)

Exact Reduce function

I have a collection of the type :
{
"_id" : ObjectId("51f1fcc08188d3117c6da351"),
"cust_id" : "abc123",
"ord_date" : ISODate("2012-10-03T18:30:00Z"),
"status" : "A",
"price" : 25,
"items" : [{
"sku" : "ggg",
"qty" : 7,
"price" : 2.5
}, {
"sku" : "ppp",
"qty" : 5,
"price" : 2.5
}]
}
I want to fetch only the "items" object whose "items.qty">5 and and"items.sku"=="ggg".
I applied Map reduce:
cmd { "mapreduce" : "orders" , "map" : "function map(){var items_out={items:[]};for(i in this.items){items_out.items.push(this.items[i].sku);};emit(this._id,[items_out]);}" , "reduce" : "function reduce(key,values){return {'result':values};}" , "verbose" : true , "out" : { "replace" : "map_reduce"} , "query" : { "$where" : "return this.items.some(function(entry){return entry.qty>5})&&this.items.some(function(entry){return entry.sku=='ggg'})"}},
but I am getting all the sku values something like this:
{ "data": [ { "items": [ "ggg", "ppp" ] } ]}
Whereas it should give only ggg as this is the only value matching criteria.
Use the following command:
db.orders.aggregate(
{$unwind : "$items"},
{$match : {"items.qty": {$gt: 5 }}},
{$match : {"items.sku" : "ggg"}},
{$project : {_id:0, items:1}}
)