I'm currently implementing a realtime search function in my app and I've come across some behaviour which I'm confused about.
The background is: I have two subscriptions from the same MongoDB database on my server, named posts.
The first subscription subscribes to the latest 50 posts, and sends the data to the MiniMongo collection Posts.
The second subscriptions subscribes to the post matching whatever search is entered by the user, and sends this data to MiniMongo collection PostsSearch as per below.
// client
Posts = new Mongo.Collection('posts');
PostsSearch = new Mongo.Collection('postsSearch');
// server
Meteor.publish('postsPub', function(options, search) {
return Posts.find(search, options);
});
Meteor.publish('postsSearchPub', function(options, search) {
var self = this;
var subHandle = Posts.find(search, options).observeChanges({
added: function (id, fields) {
self.added("postsSearch", id, fields);
}
});
self.ready();
});
My question is, we know from the docs:
If you pass a name when you create the collection, then you are
declaring a persistent collection — one that is stored on the server
and seen by all users. Client code and server code can both access the
same collection using the same API.
However this isn't the case with PostsSearch. When a user starts searching on the client, the functionality works perfectly as expected - the correct cursors are sent to the client.
However I do not see a postsSearch in my MongoDB database and likewise, PostsSearch isn't populated on any other client other than my own.
How is this happening? What is self.added("postsSearch", id, fields); appearing to do that's it's able to send cursors down the wire to the client but not to the MongoDB database.
According to this doc, self.added("postsSearch", id, fields); informs the client-side that a document has been added to the postsSeach collection.
And according to Meteor.publish:
Alternatively, a publish function can directly control its published record set by calling the functions added (to add a new document to the published record set), ...
So I'm guessing that self.added does both of these operations: Adds a document to the published record set, and informs the client (that has subscribed to the current publication) of this addition.
Now if you see Meteor.subscribe:
When you subscribe to a record set, it tells the server to send records to the client. The client stores these records in local Minimongo collections, with the same name as the collection argument used in the publish handler's added, changed, and removed callbacks. Meteor will queue incoming records until you declare the Mongo.Collection on the client with the matching collection name.
This suggests 2 things:
You have to subscribe in order to receive the data from the server-side database.
Some kind of client-side code must exist in order to create a client-only postsSearch collection. (this is because you said, this collection doesn't exist on server-side database).
The 2nd point can be achieved quite easily, for example:
if(Meteor.isClient) {
postsSearch = new Mongo.Collection(null);
}
In the above example, the postsSearch collection will exist only on the client and not on the server.
And regarding the 1st, being subscribed to postsSearchPub will automatically send data for the postsSearch collection to the client (even if said collection doesn't exist in the server-side database. This is because of the explicit call to self.added).
Something to check out: According to this doc, self.ready(); calls the onReady callback of the subscription. It would be useful to see what is there in this callback, perhaps the client-only postsSearch collection is defined there?
From the doc:
this.added(collection, id, fields)
Call inside the publish function.
Informs the subscriber that a document has been added to the record set.
This means that the line self.added("postsSearch", id, fields); emulates the fact that an insert has been done to the PostsSearch collection although it's obviously not the case.
Concerning the absence of MongoDB collection, it could be related to Meteor laziness which creates the MongoDB collection at first insert, not sure though.
Related
I have a firestore DB where I'm storing polls in one collection and responses to polls in another collection. I want to get a document from the poll collection that isn't referenced in the responses collection for a particular user.
The naive approach would be to get all of the poll documents and all of the responses filtered by user ID then filter the polls on the client side. The problem is that there may be quite a few polls and responses so those queries would have to pull down a lot of data.
So my question is, is there a way to structure my data so that I can query for polls that haven't been completed by a user without having to pull down the collections in their entirety? Or more generally, is there some pattern to use when you need to query for documents in one collection that aren't referenced by another?
The documents in each of the collections look something like this:
Polls:
{
question: string;
answers: Answer[];
}
Responses:
{
userId: string;
pollId: string;
answerId: string;
}
Anyhelp would be much appreciated!
Queries in Firestore can only return documents from one collection (or from all collections with the same name) and can only contain conditions on the data that they actually return.
Since there's no way to filter based on a condition in some other documents, you'll need to include the information that you want to filter on in the polls documents.
For example, you could include a completionCount field in each poll document, that you initially set to 0, and then update only every poll completion. With that in place, the query becomes a simple query on the completionCount field of the polls collection.
For a specific user I'd actually add all polls to their profile document, and remove them from there. Duplicating data is usually the easiest (and sometimes only) way to implement use-cases such as this.
If you're worried about having to add each new poll to each new user profile when it is created, you can also query all polls on their creation timestamp when you next load a user profile and perform that sync at that moment.
load user profile,
check when they were last active,
query for new polls,
add them to user profile.
in a Meteor app, having real-time reactive updates between all connected clients is achieved with writing in collections, publishing and subscribing the right data. In normal case this means also database writes.
But what if I would like to sync particular data which does not need to be persistent and I would like to save the overhead of writing in the database ? Is it possible to use mini-mongo or other in-memory caching on the server by still preserving DDP synchronisation to all clients ?
Example
In my app I have a multiple collapsed threads and I want to show, which users currently expanded particular thread
Viewed by: Mike, Johny, Steven ...
I can store the information in the threads collection or make make a separate viewers collection and publish the information to the clients. But there is actually no meaning in making this information persistent an having the overhead of database writes.
I am confused by the collections documentation. which states:
OPTIONS
connection Object
The server connection that will manage this collection. Uses the default connection if not specified. Pass the return value of calling DDP.connect to specify a different server. Pass null to specify no connection.
and
... when you pass a name, here’s what happens:
...
On the client (and on the server if you specify a connection), a Minimongo instance is created.
But If I create a new collection and pass the option object with conneciton: null
// Creates a new Mongo collections and exports it
export const Presentations = new Mongo.Collection('presentations', {connection: null});
/**
* Publications
*/
if (Meteor.isServer) {
// This code only runs on the server
Meteor.publish(PRESENTATION_BY_MAP_ID, (mapId) => {
check(mapId, nonEmptyString);
return Presentations.find({ matchingMapId: mapId });
});
}
no data is being published to the clients.
TLDR: it's not possible.
There is no magic in Meteor that allow data being synced between clients while the data doesn't transit by the MongoDB database. The whole sync process through publications and subscriptions is triggered by MongoDB writes. Hence, if you don't write to database, you cannot sync data between clients (using the native pub/sub system available in Meteor).
After countless hours of trying everything possible I found a way to what I wanted:
export const Presentations = new Mongo.Collection('presentations', Meteor.isServer ? {connection: null} : {});
I checked the MongoDb and no presentations collection is being created. Also, n every server-restart the collection is empty. There is a small downside on the client, even the collectionHanlde.ready() is truthy the findOne() first returns undefined and is being synced afterwards.
I don't know if this is the right/preferable way, but it was the only one working for me so far. I tried to leave {connection: null} in the client code, but wasn't able to achieve any sync even though I implemented the added/changed/removed methods.
Sadly, I wasn't able to get any further help even in the meteor forum here and here
i try to build a homeautomation system with meteor. therefore i would like to do the following thing.
i have a collection with all my different liveValues i'm reading from any kind of source. each document is a value of a for example sensor with the actual value.
now i want to create a second collection called thing. in this collection i'd like to add all my "Things" for example "Roomtemperature living" with the data for this thing. one attribute should be the connection to one of liveValues.
Now i want to publish and subscribe with Meteor the Thing collection, because on the webinterface it doesn't matter what liveValue is behind the Thing.
Here, the in my optionen, complicated part starts.
How can i publish the data to the client and i will have a reactive update when the LiveValue has changend for the thing? because it's an differnt collection than "Thing" collection.
In my idea i would like to do this via one subscrition to one "thing" document and i will get back with this subscription the update of the liveValue of the liveValue collection.
Is this workable?
has somebody an idea how i can handle this?
i've heard something about meteor-reactive-publish but i'not sure if this is the solution. also i've heard that this needs a lots of power for the server.
thanks for your help.
So basically you want to merge the documents on server side to one reactive collection on client-side.
You should use observeChanges provided by Meteor Collections as described in the docs.
By this you can observe the changes on your server side collections and publish to your client-side aggregated collection, like this:
// Get the data from a kind of sensor
var cursor = SomeSensor.find({/* your query */});
var self = this;
// Observe the changes in the cursor and publish
// it to the 'things' collection in client
var observer = cursor.observeChanges({
added: function (document) {
self.added('things', document._id, document);
},
removed: function (document) {
self.removed('things', document._id, document);
},
changed: function (document) {
self.changed('things', document._id, document);
}
});
// Make the publication ready
self.ready();
// Stop the observer on subscription stop
self.onStop(function () {
observer.stop();
});
With this the things collection will have the data from all the sensors reactively.
Hope it helps you.
on meteor server side, this looks fine, which maps server side collection to publication
if (Meteor.isServer) { // This code only runs on the server
Meteor.publish('tasks', function tasksPublication() { return Tasks.find(); }); }
and the following on the client side is also understandable, which maps subscription to publication by name.
Meteor.subscribe('tasks');
But I couldn't find in any tutorial or docs explaining how subscription and client side collections are mapped to each other. There is no code mapping tasks (subscription) to Tasks (client side collection). Meteor might assume the client collection uses the same name as the server side by both including the same collection declaration (Tasks = new Mongo.collection('Tasks');). But what if I want to use a different collection name on the client side? or what if the info sent by the server is a mix of fields from multiple collections, how do clients know which collections to store this info when they get it from subscription?
This is part of the way Meteor works. It automatically synchronises data in collections between client and server. You don't need to worry about it, and you can't change it.
Your helper methods can pull data from different server collections, and put the data in arrays - you can read from different collections to do this, and you can do it reactively, so when the source collection changes it will run your helper again.
You can also define client only collections, which don't get saved to the server.
So you can probably do what you want, and if you then want to save something from your smooshed data, you would probably write a piece of code to extract the data to be updated into an object, and then save that.
Lets breakdown the code:
Meteor.publish('tasks', function() {
return Tasks.find();
});
Here we have defined a publication name 'tasks' which supplied data it has received from the function return Tasks.find().
Similarly, when subscribing, we refer to that particular publication- tasks in this case and we get those data.
Now coming to the part of linking it to the collection. In Meteor, when you define a collection, it should be defined on both client and server. So when you define a collection like Tasks = new Mongo.collection('tasklists');, On server, the Tasks object refers to the collection tasklists that the server will use to communicate with the mongoDB server. On the client, an object with the name Tasks is created to interact with the database minimongo created in client for tasklists. The Minimongo is a client side API in JS for MongoDB. (You can consider it as a client side replica of the mongoDB database).
So, on client side, you can define Tasks as anything as long as it is an object for the mongoDB collection tasklists- e.g AnyName=new Mongo.collection('tasklists');
Regarding how publication and subscription will know, which collection are we talking about: Publications send across something known as cursor which is related to a particular document(s) and collection in 'mongoDB'. As long as you get the collection name( tasklists) correct, you can have different object names(Tasks) on client and server.
In the Discover Meteor examples, what's the diff between "posts" and "Posts"? Why is it that when we do an insert from the server we use "posts" but when querying from the browser we use "Posts"? Wouldn't the system be confused by the case differences?
I see the variable assignment for client Posts to the server posts in posts.js. Is it a conventional notation to capitalize client and use small caps for server?
Posts = new Meteor.Collection('posts')
Why does server/fixtures.js use "Posts"? I was under the assumption that we query "Posts" in the browser (client), and use "posts" in the server, like we did in meteor mongo. So why are we now using Posts in the server?
Let's distinguish between the different names you might have to deal with when programming Meteor:
Variable names, such as Posts = new Meteor.Collection(...). These are used only so your code knows how to access this variable. Meteor doesn't know or care what it is, although the convention is to capitalize.
Collection names, such as new Meteor.Collection("posts"). This maps to the name of a MongoDB collection (on the server) or a minimongo collection (on the client).
Publication and subscription names, used in Meteor.publish("foo", ...) or Meteor.subscribe("foo"). These have to match up for the client to subscribe to some data on the server.
There are two things you need to match up in the Meteor data model:
Names of publications and their corresponding subscriptions
(usually) Names of collections on the client and server, if using the default collection model
A subscription name needs to always match up with the name of a publication. However, the collections that are sent for a given subscription needn't have anything to do with the subscription name. In fact, one can send over multiple cursors in one publication or one collection over different publications or even multiple subscriptions per publication, which appear merged as one in the client. You can also have different collection names in the server and client; read on...
Let's review the different cases:
Simple subscription model. This is the one you usually see in straightforward Meteor demos.
On client and server,
Posts = new Meteor.Collection("posts");
On server only:
Meteor.publish("postsPub", function() {
return Posts.find()
});
On client only:
Meteor.subscribe("postsPub")
This synchronizes the Posts collection (which is named posts in the database) using the publication called postsPub.
Multiple collections in one publication. You can send multiple cursors over for a single publication, using an array.
On client and server:
Posts = new Meteor.Collection("posts");
Comments = new Meteor.Collection("comments");
On server only:
Meteor.publish("postsAndComments", function() {
return [
Posts.find(),
Comments.find()
];
});
On client only:
Meteor.subscribe("postsAndComments");
This synchronizes the Posts collection as well as the Comments collection using a single publication called postsAndComments. This type of publication is well-suited for relational data; for example, where you might want to publish only certain posts and the comments associated only with those posts. See a package that can build these cursors automatically.
Multiple publications for a single collection. You can use multiple publications to send different slices of data for a single collection which are merged by Meteor automatically.
On server and client:
Posts = new Meteor.Collection("posts");
On server only:
Meteor.publish("top10Posts", function() {
return Posts.find({}, {
sort: {comments: -1},
limit: 10
});
});
Meteor.publish("newest10Posts", function() {
return Posts.find({}, {
sort: {timestamp: -1},
limit: 10
});
});
On client only:
Meteor.subscribe("top10Posts");
Meteor.subscribe("newest10Posts");
This pushes both the 10 posts with the most comments as well as the 10 newest posts on the site to the user, which sees both sets of data merged into a single Posts collection. If one of the newest posts is also a post with the most comments or vice versa, the Posts collection will contain less than 20 items. This is an example of how the data model in Meteor allows you to do powerful data merging operations without implementing the details yourself.
Multiple subscriptions per publication. You can get multiple sets of data from the same publication using different arguments.
On server and client:
Posts = new Meteor.Collection("posts");
On server only:
Meteor.publish("postsByUser", function(user) {
return Posts.find({
userId: user
});
});
On client only:
Meteor.subscribe("postsByUser", "fooUser");
Meteor.subscribe("postsByUser", "barUser");
This causes the posts by fooUser and barUser to both show up in the posts collection. This model is convenient when you have several different computations that are looking at different slices of your data and may be updated dynamically. Note that when you subscribe inside a Deps.autorun(...), Meteor calls stop() on any previous subscription handle with the same name automatically, but if you are using these subscriptions outside of an autorun you will need to stop them yourself. As of right now, you can't do two subscriptions with the same name inside an autorun computation, because Meteor can't tell them apart.
Pushing arbitrary data over a publication. You can completely customize publications to not require the same collection names on the server and client. In fact, the server can publish data that isn't backed by a collection at all. To do this, you can use the API for the publish functions.
On server only:
Posts = new Meteor.Collection("posts");
Meteor.publish("newPostsPub", function() {
var sub = this;
var subHandle = null;
subHandle = Posts.find({}, {
sort: {timestamp: -1},
limit: 10
})
.observeChanges({
added: function(id, fields) {
sub.added("newposts", id, fields);
},
changed: function(id, fields) {
sub.changed("newposts", id, fields);
},
removed: function(id) {
sub.removed("newposts", id);
}
});
sub.ready();
sub.onStop(function() {
subHandle.stop();
})
});
On client only:
NewPosts = new Meteor.Collection("newposts");
Meteor.subscribe("newPostsPub");
This synchronizes the newest 10 posts from the Posts collection on the server (called posts in the database) to the NewPosts collection on the client (called newposts in minimongo) using the publication/subscription called newPostsPub. Note that observeChanges differs from observe, which can do a bunch of other things.
The code seems complicated, but when you return a cursor inside a publish function, this is basically the code that Meteor is generating behind the scenes. Writing publications this way gives you a lot more control over what is and isn't sent to the client. Be careful though, as you must manually turn off observe handles and mark when the subscription is ready. For more information, see Matt Debergalis' description of this process (however, that post is out of date). Of course, you can combine this with the other pieces above to potentially get very nuanced and complicated publications.
Sorry for the essay :-) but many people get confused about this and I though it would be useful to describe all the cases.
You decide the naming conventions, and meteor doesn't care.
Posts becomes a collection of documents from the mongo server. You find posts by calling Posts.find({author: 'jim}). In the example you wrote, meteor is being told to internally call that collection 'posts'. Hopefully this is easy to remember if the names are similar...
There needs to be a way to express and track what info is available to clients. Sometimes there may be multiple sets of information, of varying detail. Example: a summary for a title listing, but detail for a particular document. These are often also named 'posts' so it can be initially confusing:
Meteor.publish "posts", -> # on server
Posts.find()
and then
dbs.subscriptions.posts = Meteor.subscribe 'posts' # on client
publication and subscription names must match, but it could all be named like this:
PostsDB = new Meteor.Collection('postdocumentsonserver')
so in mongo you'd need to type
db.postdocumentsonserver.find()
but otherwise you never need to care about 'postdocumentsonserver'. Then
Meteor.publish "post_titles", ->
PostsDB.find({},{fields:{name:1}})
matching
dbs.subscriptions.post_titles = Meteor.subscribe 'post_titles'