Redux normalized state tree for posts and comments - mongodb

Redux recommends using normalized app state tree but I am not sure if it's the best practice in this case. Assume the following case:
Each Circle has_many Posts.
Each Post has_many Comments.
In the database on the backend, each model looks like this:
Circle:
{
_id: '1'
title: 'BoyBand'
}
Post:
{
_id: '1',
circle_id: '1',
body: "Some Post"
}
Comment:
{
_id: '1',
post_id: '1',
body: "Some Comment"
}
In the app state (the final result of all reducers) on the frontend looks like this:
{
circles: {
byId: {
1: {
title: 'BoyBand'
}
},
allIds: [1]
},
posts: {
byId: {
1: {
circle_id: '1',
body: 'Some Post'
}
},
allIds: [1]
},
comments: {
byId: {
1: {
post_id: '1',
body: 'Some Comment'
},
allIds: [1]
}
}
Now, when I go to CircleView, I fetch Circle from the backend which returns all posts and comments associated with it.
export const fetchCircle = (title) => (dispatch, getState) => {
dispatch({
type: constants.REQUEST_CIRCLE,
data: { title: title }
})
request
.get(`${API_URL}/circles/${title}`)
.end((err, res) => {
if (err) {
return
}
// When you fetch circle from the API, the API returns:
// {
// circle: circleObj,
// posts: postsArr,
// comments: commentsArr
// }
// so it's easier for the reducers to consume the data
dispatch({
type: constants.RECEIVE_CIRCLE,
data: (normalize(res.body.circle, schema.circle))
})
dispatch({
type: 'RECEIVE_POSTS',
data: (normalize(res.body.posts, schema.arrayOfPosts))
})
dispatch({
type: 'RECEIVE_COMMENTS',
data: (normalize(res.body.comments, schema.arrayOfComments))
})
})
}
Up to this point, I think I did everything in a fairly standard way. However, when I wanted to render each Post component, I realized that populating the posts with their comments became inefficient (O(N^2)) compared to when I kept my state tree in the following format.
{
circles: {
byId: {
1: {
title: 'BoyBand'
}
},
allIds: [1]
},
posts: {
byId: {
1: {
circle_id: '1',
body: 'Some Post'
comments: [arrOfComments]
}
},
allIds: [1]
}
}
This goes against my understanding where in a redux state tree, it's better to keep everything normalized.
Q. Should I in fact keep things denormalized in a case like this? How do I determine what to do?

I'd go for: yes normalize it, but do it on the backend!
Why?
Deleting is easier
because otherwise, you'd have to track down the posts and comments every time you'd want to delete a circle, or post.
Working with the data is easier
because otherwise, you'd have to do the same mutations on your data over and over again just so that you can select the dataset which is related to a particular circle or post.
You don't have any many-to-many relationship
you don't have multiple posts which link to the same comment so it just makes sense to have the data normalized.
You shouldn't be limited by an API
If this is a third-party API then make your backend fetch the API and normalize the data there. You shouldn't be restricted by the API and I don't know what kind of data you access but you can definitively save a DNS lookup for the user and serve cached data if the API is unavailable. If you rely on the API to being up you introduce a single point of failure.
About your performance issues, they should be insignificant if you normalize on the backend and you should measure it and take the critical code for a code review.

In my opinion, the list of comments is specific for any post. User cannot post one comment into multiple posts. And there's nothing wrong that comments are tightly coupled with the post. It's easy to update/remove specific comment(both postId and commentId are present). Removing a post is trivial. Same with circle. It's insignificantly harder to remove all comments of a specific user. And I think that there are no strict rules, the RIGHT way, etc... more often it depends. KiSS ;)
While thinking how to organize comments on client side I was reading this article, it's about possible db structures for similar situation. https://docs.mongodb.com/ecosystem/use-cases/storing-comments/

Related

Where to put version __v in redux state?

I have a model that is scattered all around the application. I have a redux state tree:
{
page: {
modelPart1: ...,
... : {
modelPart2: ...
}
}
I need to keep a reference to mongoDb __v in my state too. Where is the best place to place it?
I was thinking about a separate branch model_metadata that would keep the metadata about docs (_id, __v, ...).
{
model_metadata: { <------------------------ HERE
model: {
_id: id,
__v: 2
}
}
page: {
modelPart1: ...,
... : {
modelPart2: ...
}
}
Is it a valid approach or would you recommend a different one?
Every reducer only can access its own part of state, so when you do
combineReducers({
one,
another
});
and access state in one, it is equivalent to doing store.getState().one, and the same for another. So, you need to split the data in page property of state into two parts: actual data and metadata. Just like the object you retrieve from Mongo.
The point in having metadata and actual data being processed by the same reducer is that every time a reducer function is performed, you have everything you need about your object in state argument of that function. Splitting the data into two different reducers would make things way more complicated.
So, the new data representation in page would look like
{
model_metadata: { <------------------------ HERE
model: {
_id: id,
__v: 2
}
}
page: {
modelPart1: ...,
... : {
modelPart2: ...
}
}
while connecting to page would look like
connect(state => ({
page: state.page
})(...)

About normalization in Redux

Assume you're making Reddit where each Subreddit has many Post and each Post has many Comment. Then the API response probably looks like this:
subreddits: [{
title: "food"
posts: [{
id: "",
body: "..",
comments: [{
id: ".."
body: "..",
}]
..morePosts
},
title: "culture"
posts: [{
id: "",
body: "..",
comments: [{
id: ".."
body: "..",
}]
..morePosts
},
]
But since Redux discourages such nested state, we normalize the data structure before we feed them into reducers. Then, the data can be represented like this:
subredditByTitle: {
food: {
id: subreddit_1,
title: "food"
posts: [post_1, post_2]
}
culture: {
id: subreddit_2,
title: "culture"
posts: [post_3, post_4]
}
}
postsById: {
post_1: {
body: ".."
comments: [comment_1, comment_2]
},
post_2: {
body: "..",
comments: [comment_3, comment_4]
}
}
commentsById: {
comment_1: {
body: ".."
},
comment_2: {
body: ".."
}
}
But it feels a bit awkward to normalize the backend data like this when I use MongoDB, especially when I am using subdocuments. In a relational DB, it makes sense to have a lookup table (e.g. PostsById) for every DB table, does it make sense to do the same for every DB collection? My gut feeling is that instead of trying to normalize everything, it might be better to have one reducer for each document, but I am not sure what the best practice might be.
You should really normalize everything and have entities object in your store where you put all your entities. I tried many approaches but IMHO this is the only true way.
I am using it for things which would be unthinkable to do without this approach but they are out of scope of this answer other more common ones are pagination and asking for data only when you need them application feels super snappy when there is no unnecessary data loading.
I highly recommend to take a really good look at this tiny piece of code and the redux real world example as whole there is really much to learn from that. Your entities reducer would look different obviously but you should be able to write your own to suit your needs

What exactly is "data" that is passed to responses?

I'm writing a custom response that takes data as an input, and I am finding strange properties being added, namely:
add: [Function: add],
remove: [Function: remove]
When I log out some example data, I get:
[ { books:
[ { id: 1,
title: 'A Game of Thrones',
createdAt: '2015-08-04T04:53:38.043Z',
updatedAt: '2015-08-04T04:53:38.080Z',
author: 1 } ],
id: 1,
name: 'George R. R. Martin',
createdAt: '2015-08-04T04:53:38.040Z',
updatedAt: '2015-08-04T04:53:38.073Z' },
{ books:
[ { id: 2,
title: 'Ender\'s Game',
createdAt: '2015-08-04T04:53:38.043Z',
updatedAt: '2015-08-04T04:53:38.080Z',
author: 2 },
{ id: 3,
title: 'Speaker for the Dead',
createdAt: '2015-08-04T04:53:38.043Z',
updatedAt: '2015-08-04T04:53:38.081Z',
author: 2 } ],
id: 2,
name: 'Orson Scott Card',
createdAt: '2015-08-04T04:53:38.042Z',
updatedAt: '2015-08-04T04:53:38.074Z' } ]
Which looks innocent enough, but results in the strange add and remove functions when I use a custom serializer on it. If I take this data and hard-code it straight into the serializer, those are not present. Apparently something is lurking inside of data that's not being printed to the console.
So, what is data?
Edit: So, I'm still not quite sure what other magical properties live in here, but:
Object.keys(data[0].books))
reveals
[ '0', 'add', 'remove' ]
Which is where those are coming from. Why is this included in the data passed to custom responses? And what else might be hiding in there...
More importantly, how do I strip this gunk out and make data a normal object?
JSON.parse(JSON.stringify(data));
That cleans it up nicely, though it feels like a hack. (Actually, it's definitely a hack.)
I assume your data attribute is returned by a database query. e.g.:
Model.find(...).exec(function (err, data) { ... });
But what are these .add() and .remove() methods?
Here is what you can find in the docs:
For the most part, records are just plain old JavaScript objects (aka POJOs). However they do have a few protected (non-enumerable) methods for formatting their wrapped data, as well as a special method (.save()) for persisting programmatic changes to the database.
We can go deeper:
"collection" associations, on the other hand, do have a couple of special (non-enumerable) methods for associating and disassociating linked records. However, .save() must still be called on the original record in order for changes to be persisted to the database.
orders[1].buyers.add({ name: 'Jon Snow' });
orders[1].save(function (err) { ... });
So these methods (.add(), .remove(), .save()) are useful if you play with "collection" associations.
How to remove them?
You'll need to use .toObject() which returns a cloned model instance stripped of all instance methods.
You might want to use .toJSON() that also returns a cloned model instance. This one however includes all instance methods.

Meteor template subscriptions and performance

I'm looking for advice on my approach here- I want to be sure I'm doing things in the "meteor way" and keeping the code fast.
Current situation:
We have a collection for Questions. Each question has a nested collection of Answers. Through a REST API, a device relays the answers that were selected by users.
Based on the answers that were selected, we show a chart for each question- simple number breakdowns and percentage bars. To improve performance, we've been tracking the number of responses each Answer has received on the Answer itself.
The publication looks (basically) like this:
Meteor.publish('questionsBySiteID', function(site_id){
return Questions.find({site_id: site_id});
});
And the route like this:
Router.route('/sites/:_id/questions', {
name: 'questionsList',
waitOn: function(){
return [
Meteor.subscribe('questionsBySiteID', this.params._id),
];
},
data: function(){
return {
publishedQuestions: Questions.find(
{ site_id : this.params._id, active: true, deleted: {$ne: true} },
{ sort : { order: 1} }
),
archivedQuestions : Questions.find(
{ site_id : this.params._id, active: false, deleted: {$ne: true} },
{ sort : { updated_at: -1 } }
),
deletedQuestions : Questions.find(
{ site_id : this.params._id, deleted: true },
{ sort : { updated_at: -1 } }
)
};
}
});
Change required:
Now we want responses to be date-filterable. This means the denormalized response counts we've tracked on Answers aren't very useful. We've been tracking another collection (Responses) with more a "raw" version of the data. A Response object tracks the module (questions in this case), question_id, answer_id, timestamp, id for the customer the question belongs to, etc.
Question:
Is this something that template subscriptions help with? Perhaps we need a publication that accepts a question_id and optional start/end dates for the filter. The stats template for each question would be subscribed to applicable Responses data in Template.question.create(). Based on the question_id, the publication would need to find Responses for related answers within the date filter. And maybe we use the publish-counts package to count the number of times each answer was selected and publish those counts.
The Responses collection will be quite large, so I'm trying to be careful about what I publish here. I don't want to waitOn all Responses to be published.

Meteor Reactive Data Query for Comments with Usernames and Pictures

I am trying to implement a commenting system in a huge app and always run in the problem about cross reactiveness and publications.
The specific problem:
When a user writes a comment, I want to show the user's name and a profile picture. The comments are in one collection, the names and pictures in another.
When I make a subscription for every comment on this page and for every user whose id is in a comment of this page serversided, the app does not update the users available on the client when a new comment is added because "joins" are nonteactive on the server.
When I do that on the client, i have to unsubscribe and resubscribe all the time, a new comment is added and the load gets higher.
what is the best practise of implementing such a system in meteor? how can i get around that problem without a huge overpublishing?
As there is not official support for joins yet,among all the solutions out there in community
I found https://github.com/englue/meteor-publish-composite this package very helpful and I'm using it in my app.
This example perfectly suits your requirement https://github.com/englue/meteor-publish-composite#example-1-a-publication-that-takes-no-arguments
Meteor.publishComposite('topTenPosts', {
find: function() {
// Find top ten highest scoring posts
return Posts.find({}, { sort: { score: -1 }, limit: 10 });
},
children: [
{
find: function(post) {
// Find post author. Even though we only want to return
// one record here, we use "find" instead of "findOne"
// since this function should return a cursor.
return Meteor.users.find(
{ _id: post.authorId },
{ limit: 1, fields: { profile: 1 } });
}
},
{
find: function(post) {
// Find top two comments on post
return Comments.find(
{ postId: post._id },
{ sort: { score: -1 }, limit: 2 });
},
children: [
{
find: function(comment, post) {
// Find user that authored comment.
return Meteor.users.find(
{ _id: comment.authorId },
{ limit: 1, fields: { profile: 1 } });
}
}
]
}
]
});
//client
Meteor.subscribe('topTenPosts');
and the main thing is it is reactive