I was happy to see that $near support for geospatial indexes was recently added to minimongo in Meteor 0.6.6. However, it doesn't appear that the sorting behavior of $near (it should sort in order of distance) is reactive. That is, when an document is added to the collection, the client loads it, but always at the end of the result list, even if it is closer to the $near coordinate than other documents. When I refresh the page, the order is corrected.
For example:
Server:
Meteor.publish('events', function(currentLocation) {
return Events.find({loc: {$near:{$geometry:{ type:"Point", coordinates:currentLocation}}, $maxDistance: 2000}});
});
Client:
Template.eventsList.helpers({
events: function() {
return Events.find({loc: {$near:{$geometry:{ type:"Point", coordinates:[-122.3943391, 37.7935434]}},
$maxDistance: 2000}});
}
});
Is there a way to get it to sort reactively?
There is nothing special about sorting reactivity for $near queries as supposed to any other query in minimongo: minimongo uses some sorting function either based on your sort specifier passed in query or a default sorting for queries containing $near operator.
Minimongo would sort everything and compare previous order with the new order every time something updates.
From your original question, it's unclear what behavior you expect and what do you see instead. Just to prove mentioned sorting works reactively, I wrote a mini-app to show it:
html templates:
<body>
{{> hello}}
</body>
<template name="hello">
Something will go here:
{{#each things}}
<p>{{name}}
{{/each}}
</template>
and JS file:
C = new Meteor.Collection('things');
if (Meteor.isClient) {
Template.hello.things = function () {
return C.find({location:{$near:{$geometry:{type: "Point",coordinates:[0, 0]}, $maxDistance:50000}}});
};
}
if (Meteor.isServer) {
Meteor.startup(function () {
C.remove({});
var j = 0;
var x = [10, 2, 4, 3, 9, 1, 5, 4, 3, 1, 9, 11];
// every 3 seconds insert a point at [i, i] for every i in x.
var handle = Meteor.setInterval(function() {
var i = x[j++];
if (!i) {
console.log('Done');
clearInterval(handle);
return;
}
C.insert({
name: i.toString(),
location: {
type: "Point",
coordinates: [i/1000, i/1000]
}
});
}, 3000);
});
}
What I see right after starting application and opening the browser: Numbers appear on the screen one by one from the x array. Every time new number arrives, it appears on the correct spot, keeping the sequence sorted all the time.
Did you mean something else by '$near reactive sorting'?
Related
On my server side I have for various objects a publication which basically returns the count. Every different object has a different publication name like this:
Meteor.publish('object1Count', function(...
Meteor.publish('object2Count', function(...
Which are something like this:
Meteor.publish('object1Count', function(arg) {
var self = this;
var count = 0;
var initializing = true;
var query = arg?{arg:arg}:{};
var projection = !arg?{limit:1}:{};
var handle = Object1.find(query, projection).observeChanges({
added: function (idx) {
count++;
if (!initializing)
self.changed("totalcounts", 1, {count: count});
},
removed: function (idx) {
count--;
self.changed("totalcounts", 1, {count: count});
}
});
initializing = false;
self.added("totalcounts", 1, {count: count});
self.ready();
self.onStop(function () {
handle.stop();
});
});
But as you see inside each of these methods there will be this line
self.added("totalcounts", 1, {count: count});
In fact on the client side when I need to access the count of an Object I do like this:
template.subscribe('object1Count', template.reactiveEventId.get());
...
TotalCounts = (typeof TotalCounts==='undefined')?new Mongo.Collection("totalcounts"):TotalCounts;
It apparently works, but now that I read it twice I wonder why, the "totalcounts" collection looks like the same for all the objects, so if I switch between pages needing different totalcounts (for different objects), I guess that the client destroys the local collection totalcounts and creates a new one. Does this happen also server side?
So finally my question is: what is the best practice? The projects need the total counts for various reasons: pagination, charts, etc.. I want to create the total counts server side and just pass the minimum data for that. Should I create different "totalcounts" for every object? What's the efficient way of doing this?
Thanks
self.added("totalcounts", 1, {count: count});
it means add to collection name totalcounts a document with _id is 1 and the rest of data is {count: count}.
Because they have the same _id then you can't make more than 1 subscription.
Btw, when the template is "unmounted" it will auto stop subscriptions.
I am building some little log reporting with meteor.
have a dead simple table, that should contain all the data that received from the external mongodb server.
the html:
<tbody>
{{#each impressions}}
<tr>
{{> impression}}
</tr>
{{/each}}
</tbody>
js :
Meteor.subscribe('impressions');
...
...
Template.logResults.helpers({
'impressions': function() {
var sTs = Router.current().params.fromTs;
var eTs = Router.current().params.toTs;
return Impressions.find({});
}
});
So far, so good. BUT, when I am changing the query to this one :
Impressions.find({
$and: [{
ts: {
$gte: sTs
}
}, {
ts: {
$lte: eTs
}
}]
});
The results aren't displayed on the HTML DOM,
I tried to debug that, and created a console.log of this exact query,
and surprisingly all the correct results return successfully to the console.
screenshot attached.
I am probably doing something wrong, maybe with publish/subscribe thing.
help someone?
Thanks.
P.S. I removed the insecure and auto-publish,
have this code on the server folder
Meteor.publish('impressions', function() {
return Impressions.find();
});
and this code on the main lib folder
Impressions = new Mongo.Collection("banners");
enter image description here
The router stores the parameters for the current route as strings (which makes sense because URLs are strings), so you need to explicitly convert the values to integers before querying the database. Give something like this a try:
var sTs = Number(Router.current().params.fromTs);
var eTs = Number(Router.current().params.toTs);
Notes:
parseInt or parseFloat may be a better choice depending on the nature of your input. See this question for more details.
You could do the type conversion in the router itself and pass the values in the data context to the helpers. See this section of the iron router guide.
I suspect it worked when you typed it into the console because you used numbers instead of strings. E.g. ts: {$gte: 123} instead of ts: {$gte: '123'}.
Using Meteor.js I've got the following code for my pub/sub working flawlessly. I'm able to pass my arguments through and return the cursors with no problem.
My objective is to display the distance between the current user location and the database result.
As mongodb has already calculated the distance to get the result set I don't want to calculate it again someplace else. I'd like to return the geoNear.results[n].dis results of the $geoNear documented here but can't work out a practical way to go about it. I appreciate the publish only returns a cursor to the docs but wondered if there was some way to attach the results somehow...
Meteor.publishComposite("public", function(location, distance) {
return {
find: function() {
return Tutors.find({},
{
$geoNear: {
$geometry: {
type: "Point" ,
coordinates: [ location.lng , location.lat ]
},
$maxDistance: distance,
},
}
);
}
}
});
My subscribe arguments are simply a lat/lng object and distance in metres.
What if I told you that you could use Mongo aggregation? The general idea here is to get the distance between the current user location and the database result to update automatically with a change in the 'Tutors' collection, thus use publication with an observe to achieve this.
Here's the set-up. The first step is to get the aggregation framework package which wraps up some Mongo methods for you. Just meteor add meteorhacks:aggregate and you should be home and dry. This will add an aggregate() method to your collections.
An alternative to adding the aggregation framework support is to call directly your mongoDB and access the underlying collection methods, which in this case you need the aggregate() method. So, use this to connect in the mongoDB :
var db = MongoInternals.defaultRemoteCollectionDriver().mongo.db,
Tutors = db.collection("tutors");
Now you can dive into the aggregation framework and build up your pipeline queries. The following example demonstrates how to get the aggregation in the publish reactive use a observe in the publish with ES6 in Meteor. This follows the 'counts-by-room' example in the meteor docs. With the observe, you know if a new location has been added, changes or removed. For simplicity re-run the aggregation each time (except remove) and if the location was previously published then update the publish, if the location was removed then remove the location from the publish and then for a new location use added:
Meteor.publish('findNearestTutors', function(opts) {
let initializing = 1, run = (action) => {
// Define the aggregation pipeline
let pipeline = [
{
$geoNear: {
near: {type: 'Point', coordinates: [Number(opts.lng), Number(opts.lat)]},
distanceField: 'distance',
maxDistance: opts.distance,
spherical: true,
sort: -1
}
}
]
Tutors.aggregate(pipeline).forEach((location) => {
// Add each of the results to the subscription.
this[action]('nearest-locations', location._id, location)
this.ready()
})
}
// Run the aggregation initially to add some data to your aggregation collection
run('added')
// Track any changes on the collection you are going to use for aggregation
let handle = Tutors.find({}).observeChanges({
added(id) {
// observeChanges only returns after the initial `added` callbacks
// have run. Until then, you don't want to send a lot of
// `self.changed()` messages - hence tracking the
// `initializing` state.
if (initializing && initializing--)
run('changed')
},
removed(id) {
run('changed')
},
changed(id) {
run('changed')
},
error(err) {
throw new Meteor.Error("Houston, we've got a problem here!", err.message)
}
})
// Stop observing the cursor when client unsubs.
// Stopping a subscription automatically takes
// care of sending the client any removed messages.
this.onStop(function () {
handle.stop();
})
})
Setup:
I got a large collection with the following entries
Name - String
Begin - time stamp
End - time stamp
Problem:
I want to get the gaps between documents, Using the map-reduce paradigm.
Approach:
I'm trying to set a new collection of pairs mid, after that I can compute differences from it using $unwind and Pair[1].Begin - Pair[0].End
function map(){
emit(0, this)
}
function reduce(){
var i = 0;
var pairs = [];
while ( i < values.length -1){
pairs.push([values[i], values[i+1]]);
i = i + 1;
}
return {"pairs":pairs};
}
db.collection.mapReduce(map, reduce, sort:{begin:1}, out:{replace:"mid"})
This works with limited number of document because of the 16MB document cap. I'm not sure if I need to get the collection into memory and doing it there, How else can I approach this problem?
The mapReduce function of MongoDB has a different way of handling what you propose than the method you are using to solve it. The key factor here is "keeping" the "previous" document in order to make the comparison to the next.
The actual mechanism that supports this is the "scope" functionality, which allows a sort of "global" variable approach to use in the overall code. As you will see, what you are asking when that is considered takes no "reduction" at all as there is no "grouping", just emission of document "pair" data:
db.collection.mapReduce(
function() {
if ( last == null ) {
last = this;
} else {
emit(
{
"start_id": last._id,
"end_id": this._id
},
this.Begin - last.End
);
last = this;
}
},
function() {}, // no reduction required
{
"out": { "inline": 1 },
"scope": { "last": null }
}
)
Out with a collection as the output as required to your size.
But this way by using a "global" to keep the last document then the code is both simple and efficient.
Ok, still in my toy app, I want to find out the average mileage on a group of car owners' odometers. This is pretty easy on the client but doesn't scale. Right? But on the server, I don't exactly see how to accomplish it.
Questions:
How do you implement something on the server then use it on the client?
How do you use the $avg aggregation function of mongo to leverage its optimized aggregation function?
Or alternatively to (2) how do you do a map/reduce on the server and make it available to the client?
The suggestion by #HubertOG was to use Meteor.call, which makes sense and I did this:
# Client side
Template.mileage.average_miles = ->
answer = null
Meteor.call "average_mileage", (error, result) ->
console.log "got average mileage result #{result}"
answer = result
console.log "but wait, answer = #{answer}"
answer
# Server side
Meteor.methods average_mileage: ->
console.log "server mileage called"
total = count = 0
r = Mileage.find({}).forEach (mileage) ->
total += mileage.mileage
count += 1
console.log "server about to return #{total / count}"
total / count
That would seem to work fine, but it doesn't because as near as I can tell Meteor.call is an asynchronous call and answer will always be a null return. Handling stuff on the server seems like a common enough use case that I must have just overlooked something. What would that be?
Thanks!
As of Meteor 0.6.5, the collection API doesn't support aggregation queries yet because there's no (straightforward) way to do live updates on them. However, you can still write them yourself, and make them available in a Meteor.publish, although the result will be static. In my opinion, doing it this way is still preferable because you can merge multiple aggregations and use the client-side collection API.
Meteor.publish("someAggregation", function (args) {
var sub = this;
// This works for Meteor 0.6.5
var db = MongoInternals.defaultRemoteCollectionDriver().mongo.db;
// Your arguments to Mongo's aggregation. Make these however you want.
var pipeline = [
{ $match: doSomethingWith(args) },
{ $group: {
_id: whatWeAreGroupingWith(args),
count: { $sum: 1 }
}}
];
db.collection("server_collection_name").aggregate(
pipeline,
// Need to wrap the callback so it gets called in a Fiber.
Meteor.bindEnvironment(
function(err, result) {
// Add each of the results to the subscription.
_.each(result, function(e) {
// Generate a random disposable id for aggregated documents
sub.added("client_collection_name", Random.id(), {
key: e._id.somethingOfInterest,
count: e.count
});
});
sub.ready();
},
function(error) {
Meteor._debug( "Error doing aggregation: " + error);
}
)
);
});
The above is an example grouping/count aggregation. Some things of note:
When you do this, you'll naturally be doing an aggregation on server_collection_name and pushing the results to a different collection called client_collection_name.
This subscription isn't going to be live, and will probably be updated whenever the arguments change, so we use a really simple loop that just pushes all the results out.
The results of the aggregation don't have Mongo ObjectIDs, so we generate some arbitrary ones of our own.
The callback to the aggregation needs to be wrapped in a Fiber. I use Meteor.bindEnvironment here but one can also use a Future for more low-level control.
If you start combining the results of publications like these, you'll need to carefully consider how the randomly generated ids impact the merge box. However, a straightforward implementation of this is just a standard database query, except it is more convenient to use with Meteor APIs client-side.
TL;DR version: Almost anytime you are pushing data out from the server, a publish is preferable to a method.
For more information about different ways to do aggregation, check out this post.
I did this with the 'aggregate' method. (ver 0.7.x)
if(Meteor.isServer){
Future = Npm.require('fibers/future');
Meteor.methods({
'aggregate' : function(param){
var fut = new Future();
MongoInternals.defaultRemoteCollectionDriver().mongo._getCollection(param.collection).aggregate(param.pipe,function(err, result){
fut.return(result);
});
return fut.wait();
}
,'test':function(param){
var _param = {
pipe : [
{ $unwind:'$data' },
{ $match:{
'data.y':"2031",
'data.m':'01',
'data.d':'01'
}},
{ $project : {
'_id':0
,'project_id' : "$project_id"
,'idx' : "$data.idx"
,'y' : '$data.y'
,'m' : '$data.m'
,'d' : '$data.d'
}}
],
collection:"yourCollection"
}
Meteor.call('aggregate',_param);
}
});
}
If you want reactivity, use Meteor.publish instead of Meteor.call. There's an example in the docs where they publish the number of messages in a given room (just above the documentation for this.userId), you should be able to do something similar.
You can use Meteor.methods for that.
// server
Meteor.methods({
average: function() {
...
return something;
},
});
// client
var _avg = { /* Create an object to store value and dependency */
dep: new Deps.Dependency();
};
Template.mileage.rendered = function() {
_avg.init = true;
};
Template.mileage.averageMiles = function() {
_avg.dep.depend(); /* Make the function rerun when _avg.dep is touched */
if(_avg.init) { /* Fetch the value from the server if not yet done */
_avg.init = false;
Meteor.call('average', function(error, result) {
_avg.val = result;
_avg.dep.changed(); /* Rerun the helper */
});
}
return _avg.val;
});