on meteor server side, this looks fine, which maps server side collection to publication
if (Meteor.isServer) { // This code only runs on the server
Meteor.publish('tasks', function tasksPublication() { return Tasks.find(); }); }
and the following on the client side is also understandable, which maps subscription to publication by name.
Meteor.subscribe('tasks');
But I couldn't find in any tutorial or docs explaining how subscription and client side collections are mapped to each other. There is no code mapping tasks (subscription) to Tasks (client side collection). Meteor might assume the client collection uses the same name as the server side by both including the same collection declaration (Tasks = new Mongo.collection('Tasks');). But what if I want to use a different collection name on the client side? or what if the info sent by the server is a mix of fields from multiple collections, how do clients know which collections to store this info when they get it from subscription?
This is part of the way Meteor works. It automatically synchronises data in collections between client and server. You don't need to worry about it, and you can't change it.
Your helper methods can pull data from different server collections, and put the data in arrays - you can read from different collections to do this, and you can do it reactively, so when the source collection changes it will run your helper again.
You can also define client only collections, which don't get saved to the server.
So you can probably do what you want, and if you then want to save something from your smooshed data, you would probably write a piece of code to extract the data to be updated into an object, and then save that.
Lets breakdown the code:
Meteor.publish('tasks', function() {
return Tasks.find();
});
Here we have defined a publication name 'tasks' which supplied data it has received from the function return Tasks.find().
Similarly, when subscribing, we refer to that particular publication- tasks in this case and we get those data.
Now coming to the part of linking it to the collection. In Meteor, when you define a collection, it should be defined on both client and server. So when you define a collection like Tasks = new Mongo.collection('tasklists');, On server, the Tasks object refers to the collection tasklists that the server will use to communicate with the mongoDB server. On the client, an object with the name Tasks is created to interact with the database minimongo created in client for tasklists. The Minimongo is a client side API in JS for MongoDB. (You can consider it as a client side replica of the mongoDB database).
So, on client side, you can define Tasks as anything as long as it is an object for the mongoDB collection tasklists- e.g AnyName=new Mongo.collection('tasklists');
Regarding how publication and subscription will know, which collection are we talking about: Publications send across something known as cursor which is related to a particular document(s) and collection in 'mongoDB'. As long as you get the collection name( tasklists) correct, you can have different object names(Tasks) on client and server.
Related
in a Meteor app, having real-time reactive updates between all connected clients is achieved with writing in collections, publishing and subscribing the right data. In normal case this means also database writes.
But what if I would like to sync particular data which does not need to be persistent and I would like to save the overhead of writing in the database ? Is it possible to use mini-mongo or other in-memory caching on the server by still preserving DDP synchronisation to all clients ?
Example
In my app I have a multiple collapsed threads and I want to show, which users currently expanded particular thread
Viewed by: Mike, Johny, Steven ...
I can store the information in the threads collection or make make a separate viewers collection and publish the information to the clients. But there is actually no meaning in making this information persistent an having the overhead of database writes.
I am confused by the collections documentation. which states:
OPTIONS
connection Object
The server connection that will manage this collection. Uses the default connection if not specified. Pass the return value of calling DDP.connect to specify a different server. Pass null to specify no connection.
and
... when you pass a name, here’s what happens:
...
On the client (and on the server if you specify a connection), a Minimongo instance is created.
But If I create a new collection and pass the option object with conneciton: null
// Creates a new Mongo collections and exports it
export const Presentations = new Mongo.Collection('presentations', {connection: null});
/**
* Publications
*/
if (Meteor.isServer) {
// This code only runs on the server
Meteor.publish(PRESENTATION_BY_MAP_ID, (mapId) => {
check(mapId, nonEmptyString);
return Presentations.find({ matchingMapId: mapId });
});
}
no data is being published to the clients.
TLDR: it's not possible.
There is no magic in Meteor that allow data being synced between clients while the data doesn't transit by the MongoDB database. The whole sync process through publications and subscriptions is triggered by MongoDB writes. Hence, if you don't write to database, you cannot sync data between clients (using the native pub/sub system available in Meteor).
After countless hours of trying everything possible I found a way to what I wanted:
export const Presentations = new Mongo.Collection('presentations', Meteor.isServer ? {connection: null} : {});
I checked the MongoDb and no presentations collection is being created. Also, n every server-restart the collection is empty. There is a small downside on the client, even the collectionHanlde.ready() is truthy the findOne() first returns undefined and is being synced afterwards.
I don't know if this is the right/preferable way, but it was the only one working for me so far. I tried to leave {connection: null} in the client code, but wasn't able to achieve any sync even though I implemented the added/changed/removed methods.
Sadly, I wasn't able to get any further help even in the meteor forum here and here
Due to the Meteor Docs there are 'server-side', 'client-side' and 'local' Collections. Is there a way to change the 'status' (e.g. if it's server-side, client-side or local) on a running app?
Use Case: A Web-Application where users can register and login. They can store sensible data. Depending on the Users personal preferences he should be able to choose if that data is stored local or on the server (General decision - not from case to case).
Current Approach: It's working fine if I either instantiate the Collection local CollectionName = new Mongo.Collection(null); or server side CollectionName = new Mongo.Collection('collectionName');.
But I can't think of an approach to make it possible to the user that he can change the Collection status.
Is there a way to do this?
Or is a workaround needed (e.g. Create both, a local and server-side Collaction, and just decide which to use for insert/update/find - what would mean a lot of duplicate code?!).
Edit: To make thinks clear: I want the user to be able to choose if his data is stored in a collection which is synced with the server or a collection without any syncing.
No, you can't change the type of a collection on a running app.
I think you are confused about what these terms mean. "Client-side" collections aren't permanently stored in localstorage. It just means it's a collection that's in the browser's memory. Just as "server-side" collections are those that reside in the server's memory. The difference is not how it's defined, but where the code runs. Most collections have a client-side and a server-side counterpart, and they are kept synchronized via pub/sub. Server-side collections are also synchronized with MongoDB (using the oplog).
Local collections can live in both places, but "local" means they aren't synchronized with anything.
I probably don't fully understand what you are trying to do, but local collections do not persist data.
If you pass null as the name, then you’re creating a local collection. It’s not synchronized anywhere; it’s just a local scratchpad that supports Mongo-style find, insert, update, and remove operations. (On both the client and the server, this scratchpad is implemented using Minimongo.)
This means any data added to them on the client will be blown away when the user closes their browser (unless you are also using one of the local collection persist meteor packages) and any data added to them on the server will be blown away when the meteor app is restarted. So I don't think you really want to use local collections.
Instead, I would use a regular collection (where a name is passed to the constructor) and either the standard allow or deny options (not really recommended anymore...but still a valid approach) or Meteor methods (the preferred approach) to control who can change data and what data is allowed to change.
Or, another option could be to pass your publication function a list of fields that the user wishes to see on the client for that given session. To do this you defined a new publication that receives a displayFields argument that you then use as the field specifier options in your collection .find().
Meteor.publish("userData", function (userId, displayFields) {
// validate the structure and contents of displayFields
// retrieve the data but only use the fields that the user requested
return UserData.find({user_id: userId}, {fields: displayFields});
});
Then on the client side you would subscribe to this and pass in the fields the user wishes to make visible on the client.
var displayFields = {
firstname: 1,
lastname: 0,
//...
};
this.subscribe("userData", [displayFields]);
Would a database like Cassandra and scheme like GraphQL work well together?
Cassandra ideology is based on the idea of optimizing your queries and denormalizing data. This doesn't seem to really mesh well with a GraphQL ideology where data seems to be accessible in every level of a query.
Example:
Suppose I architect my Cassandra table like so:
User:
name
address
etc... (many properties)
Group:
id
name
user_name (denormalized user, where we generally just need the name of a user)
But with GraphQL, it's one wouldn't exactly expect a denormalized User.
query getGroup {
group(id: 1) {
name
users {
name
}
}
}
So a couple of things:
1.) This GraphQL query could end up hitting our Cassandra database multiple times (assuming no caching). Getting the group name and for each of the users we might even hit it for each user. But lets say our resolve creates multiple User objects with one cassandra call.
2.) We can't really build a cassandra idiomatic database with denormalization and graphql in mind, can we? Otherwise we should expect certain properties of a User aren't returned to us with the query.
To sum up the question, what's the graphql strategy for working with denormalized data? Is it acceptable to omit certain properties that the client thinks are accessible? E.g the client tries to access address of user but we don't have that at the moment because our data is denormalized. Or should one not even worry about denormalization and just let graphQL make calls with a caching mechanism in between the db and graphql. E.g graphql first gets the group, then gets the user data for the group id.
This is a side effect of GraphQL where a query can get quite complex in retrieving the data. But as long as the user is actually requesting the data they need if you are smart about your resolvers the end result will actually be faster.
Consider tools like dataloader to cache when resolving a query.
As far as omitting certain properties graphql validates the response and will throw an error, although it will also return the data you gave. It would probably be better to implement some sort of timeout and throw a more descriptive error if there is an issue retrieving the data.
I'm currently implementing a realtime search function in my app and I've come across some behaviour which I'm confused about.
The background is: I have two subscriptions from the same MongoDB database on my server, named posts.
The first subscription subscribes to the latest 50 posts, and sends the data to the MiniMongo collection Posts.
The second subscriptions subscribes to the post matching whatever search is entered by the user, and sends this data to MiniMongo collection PostsSearch as per below.
// client
Posts = new Mongo.Collection('posts');
PostsSearch = new Mongo.Collection('postsSearch');
// server
Meteor.publish('postsPub', function(options, search) {
return Posts.find(search, options);
});
Meteor.publish('postsSearchPub', function(options, search) {
var self = this;
var subHandle = Posts.find(search, options).observeChanges({
added: function (id, fields) {
self.added("postsSearch", id, fields);
}
});
self.ready();
});
My question is, we know from the docs:
If you pass a name when you create the collection, then you are
declaring a persistent collection — one that is stored on the server
and seen by all users. Client code and server code can both access the
same collection using the same API.
However this isn't the case with PostsSearch. When a user starts searching on the client, the functionality works perfectly as expected - the correct cursors are sent to the client.
However I do not see a postsSearch in my MongoDB database and likewise, PostsSearch isn't populated on any other client other than my own.
How is this happening? What is self.added("postsSearch", id, fields); appearing to do that's it's able to send cursors down the wire to the client but not to the MongoDB database.
According to this doc, self.added("postsSearch", id, fields); informs the client-side that a document has been added to the postsSeach collection.
And according to Meteor.publish:
Alternatively, a publish function can directly control its published record set by calling the functions added (to add a new document to the published record set), ...
So I'm guessing that self.added does both of these operations: Adds a document to the published record set, and informs the client (that has subscribed to the current publication) of this addition.
Now if you see Meteor.subscribe:
When you subscribe to a record set, it tells the server to send records to the client. The client stores these records in local Minimongo collections, with the same name as the collection argument used in the publish handler's added, changed, and removed callbacks. Meteor will queue incoming records until you declare the Mongo.Collection on the client with the matching collection name.
This suggests 2 things:
You have to subscribe in order to receive the data from the server-side database.
Some kind of client-side code must exist in order to create a client-only postsSearch collection. (this is because you said, this collection doesn't exist on server-side database).
The 2nd point can be achieved quite easily, for example:
if(Meteor.isClient) {
postsSearch = new Mongo.Collection(null);
}
In the above example, the postsSearch collection will exist only on the client and not on the server.
And regarding the 1st, being subscribed to postsSearchPub will automatically send data for the postsSearch collection to the client (even if said collection doesn't exist in the server-side database. This is because of the explicit call to self.added).
Something to check out: According to this doc, self.ready(); calls the onReady callback of the subscription. It would be useful to see what is there in this callback, perhaps the client-only postsSearch collection is defined there?
From the doc:
this.added(collection, id, fields)
Call inside the publish function.
Informs the subscriber that a document has been added to the record set.
This means that the line self.added("postsSearch", id, fields); emulates the fact that an insert has been done to the PostsSearch collection although it's obviously not the case.
Concerning the absence of MongoDB collection, it could be related to Meteor laziness which creates the MongoDB collection at first insert, not sure though.
In the Discover Meteor examples, what's the diff between "posts" and "Posts"? Why is it that when we do an insert from the server we use "posts" but when querying from the browser we use "Posts"? Wouldn't the system be confused by the case differences?
I see the variable assignment for client Posts to the server posts in posts.js. Is it a conventional notation to capitalize client and use small caps for server?
Posts = new Meteor.Collection('posts')
Why does server/fixtures.js use "Posts"? I was under the assumption that we query "Posts" in the browser (client), and use "posts" in the server, like we did in meteor mongo. So why are we now using Posts in the server?
Let's distinguish between the different names you might have to deal with when programming Meteor:
Variable names, such as Posts = new Meteor.Collection(...). These are used only so your code knows how to access this variable. Meteor doesn't know or care what it is, although the convention is to capitalize.
Collection names, such as new Meteor.Collection("posts"). This maps to the name of a MongoDB collection (on the server) or a minimongo collection (on the client).
Publication and subscription names, used in Meteor.publish("foo", ...) or Meteor.subscribe("foo"). These have to match up for the client to subscribe to some data on the server.
There are two things you need to match up in the Meteor data model:
Names of publications and their corresponding subscriptions
(usually) Names of collections on the client and server, if using the default collection model
A subscription name needs to always match up with the name of a publication. However, the collections that are sent for a given subscription needn't have anything to do with the subscription name. In fact, one can send over multiple cursors in one publication or one collection over different publications or even multiple subscriptions per publication, which appear merged as one in the client. You can also have different collection names in the server and client; read on...
Let's review the different cases:
Simple subscription model. This is the one you usually see in straightforward Meteor demos.
On client and server,
Posts = new Meteor.Collection("posts");
On server only:
Meteor.publish("postsPub", function() {
return Posts.find()
});
On client only:
Meteor.subscribe("postsPub")
This synchronizes the Posts collection (which is named posts in the database) using the publication called postsPub.
Multiple collections in one publication. You can send multiple cursors over for a single publication, using an array.
On client and server:
Posts = new Meteor.Collection("posts");
Comments = new Meteor.Collection("comments");
On server only:
Meteor.publish("postsAndComments", function() {
return [
Posts.find(),
Comments.find()
];
});
On client only:
Meteor.subscribe("postsAndComments");
This synchronizes the Posts collection as well as the Comments collection using a single publication called postsAndComments. This type of publication is well-suited for relational data; for example, where you might want to publish only certain posts and the comments associated only with those posts. See a package that can build these cursors automatically.
Multiple publications for a single collection. You can use multiple publications to send different slices of data for a single collection which are merged by Meteor automatically.
On server and client:
Posts = new Meteor.Collection("posts");
On server only:
Meteor.publish("top10Posts", function() {
return Posts.find({}, {
sort: {comments: -1},
limit: 10
});
});
Meteor.publish("newest10Posts", function() {
return Posts.find({}, {
sort: {timestamp: -1},
limit: 10
});
});
On client only:
Meteor.subscribe("top10Posts");
Meteor.subscribe("newest10Posts");
This pushes both the 10 posts with the most comments as well as the 10 newest posts on the site to the user, which sees both sets of data merged into a single Posts collection. If one of the newest posts is also a post with the most comments or vice versa, the Posts collection will contain less than 20 items. This is an example of how the data model in Meteor allows you to do powerful data merging operations without implementing the details yourself.
Multiple subscriptions per publication. You can get multiple sets of data from the same publication using different arguments.
On server and client:
Posts = new Meteor.Collection("posts");
On server only:
Meteor.publish("postsByUser", function(user) {
return Posts.find({
userId: user
});
});
On client only:
Meteor.subscribe("postsByUser", "fooUser");
Meteor.subscribe("postsByUser", "barUser");
This causes the posts by fooUser and barUser to both show up in the posts collection. This model is convenient when you have several different computations that are looking at different slices of your data and may be updated dynamically. Note that when you subscribe inside a Deps.autorun(...), Meteor calls stop() on any previous subscription handle with the same name automatically, but if you are using these subscriptions outside of an autorun you will need to stop them yourself. As of right now, you can't do two subscriptions with the same name inside an autorun computation, because Meteor can't tell them apart.
Pushing arbitrary data over a publication. You can completely customize publications to not require the same collection names on the server and client. In fact, the server can publish data that isn't backed by a collection at all. To do this, you can use the API for the publish functions.
On server only:
Posts = new Meteor.Collection("posts");
Meteor.publish("newPostsPub", function() {
var sub = this;
var subHandle = null;
subHandle = Posts.find({}, {
sort: {timestamp: -1},
limit: 10
})
.observeChanges({
added: function(id, fields) {
sub.added("newposts", id, fields);
},
changed: function(id, fields) {
sub.changed("newposts", id, fields);
},
removed: function(id) {
sub.removed("newposts", id);
}
});
sub.ready();
sub.onStop(function() {
subHandle.stop();
})
});
On client only:
NewPosts = new Meteor.Collection("newposts");
Meteor.subscribe("newPostsPub");
This synchronizes the newest 10 posts from the Posts collection on the server (called posts in the database) to the NewPosts collection on the client (called newposts in minimongo) using the publication/subscription called newPostsPub. Note that observeChanges differs from observe, which can do a bunch of other things.
The code seems complicated, but when you return a cursor inside a publish function, this is basically the code that Meteor is generating behind the scenes. Writing publications this way gives you a lot more control over what is and isn't sent to the client. Be careful though, as you must manually turn off observe handles and mark when the subscription is ready. For more information, see Matt Debergalis' description of this process (however, that post is out of date). Of course, you can combine this with the other pieces above to potentially get very nuanced and complicated publications.
Sorry for the essay :-) but many people get confused about this and I though it would be useful to describe all the cases.
You decide the naming conventions, and meteor doesn't care.
Posts becomes a collection of documents from the mongo server. You find posts by calling Posts.find({author: 'jim}). In the example you wrote, meteor is being told to internally call that collection 'posts'. Hopefully this is easy to remember if the names are similar...
There needs to be a way to express and track what info is available to clients. Sometimes there may be multiple sets of information, of varying detail. Example: a summary for a title listing, but detail for a particular document. These are often also named 'posts' so it can be initially confusing:
Meteor.publish "posts", -> # on server
Posts.find()
and then
dbs.subscriptions.posts = Meteor.subscribe 'posts' # on client
publication and subscription names must match, but it could all be named like this:
PostsDB = new Meteor.Collection('postdocumentsonserver')
so in mongo you'd need to type
db.postdocumentsonserver.find()
but otherwise you never need to care about 'postdocumentsonserver'. Then
Meteor.publish "post_titles", ->
PostsDB.find({},{fields:{name:1}})
matching
dbs.subscriptions.post_titles = Meteor.subscribe 'post_titles'