I am creating my first major app, and I thought of a way to optimize query performance. I am not sure though whether I should go through with it.
Here is a description of my approach.
Every time a user account is created, that user is assigned a random number between 1 and 10, which is stored in their user document. The number is called a number-Id. Here is how the user schema will look like:
let User = new Schema({
/// I left out all the other fields for clairty sake
numberId: {
type: Number,
default: Math.floor(Math.random() * 11),
index: true
}
}
Every time a user creates a blogpost and post, their number-Id is referenced inside the document of that blogpost and post. This is to make querying much faster by indexing the users number-id. Here is how the document of a blogpost would look like in MongoD:
{
"title": "my Blog Post",
"_id": "ObjectId("594824b2828d7b15ecd7b6a5")",
/// Here is the numberId of the user who posted the blogpost, it is added
/// to the document of the blogpost when it is created.
"postersNumberId": 2
/// the Id of the user who posted the blogpost
"postersId": "59481f901f0c7d249cf6b050"
}
Let's say I want to get all the blogposts made by a specific user. I can optimize my query much faster by using the number-Id of the user in question as an index, given that their number-Id is referenced in all the blogposts and comment posts they make.
BlogPost.find({postersId: user_id, postersNumberId: user.numberId});
It seems like this approach warrants that I store the users number-id in req.user in order for it to be readily available whenever I need it to optimize queries. So that means I would have to store the users data in a cookie via passport:
passport.serializeUser(function(user, done){
done(null, user._id);
});
passport.deserializeUser(function(id, done) {
User.findById(id, function (err, user){
if (err || !user) return done(err, null);
done(null, user);
});
});
Given this approach, I could now use all the information stored in the cookie, particularly the numberId, to optimize queries that retrieve the comments and blogposts a user makes:
BlogPost.find({postersId: req.user_id, postersNumberId: req.user.numberId});
However, I am using json-web-tokens to authenticate the user rather than cookies. So I will have to use a cookie to store the number-Id for indexing purposes in addition to using JWT for authentication. I've heard, however, that having cookies is bad for scalability, so I am worried that storing the users number-Id in req.user will eventually impact performance.
Should I continue with this approach, or no? What are the performance implications?
In addition to authentication JWT has a payload, which can be used to store additional information within the generated token itself:
var jwt = require('jsonwebtoken');
var token = jwt.sign({
data: {
numberId: 7
}
}, 'jwtSecret', {
expiresIn: '1h'
});
For retrieval:
jwt.verify(token, 'jwtSecret', function(err, decoded) {
if (err) {
console.log(err)
} else {
console.log(decoded);
//{ data: { numberId: 7 }, iat: 1498350787, exp: 1498354387 }
}
});
Related
I know that passport exposes the current user (I think using the 'passport' policy) in req.user. I want to do a similar thing for settings of a particular user, which are stored in a separate collection (so that they are available in req.settings). How can I do this?
The serializeUser method is responsible for that https://github.com/jaredhanson/passport#sessions
passport.serializeUser(function(user, done) {
done(null, user.id);
});
So instead of just serializing the user.id you could serialize the the settings, too:
passport.serializeUser(function(user, done) {
var sessionUser = { _id: user._id, name: user.name, settings: [...] }
done(null, sessionUser);
});
You might want to read Safe to store complete user info in session with Sails.js?
I use passportjs with passport-local strategy to authenticate users in my project. Official serializeUser deserializeUser approach is the following:
// serialize and deserialize
passport.serializeUser(function(user, done) {
done(null, user._id);
});
passport.deserializeUser(function(id, done) {
User.findById(id, function(err, user){
done(err, user.toJSON());
})
});
But due to performance reason I need to prevent query User.findById to my MongoDB database so I use the following approach:
passport.serializeUser(function(user, done){
done(null, user.toJSON());
});
passport.deserializeUser(function(user, done){
done(null, user);
});
But now I came up with the following problem: what if user change their data like name, age etc. How how could I update these without logout?
Actually I need to execute passport.serializeUser manually some how?
If you're using a database session store then you're still just retrieving the entire user.toJSON() from database upon every deserializeUser call, something you thought you were avoiding but actually are not.
If you're not using a database session store, then you may be storing it all in a cookie or something which seems highly unsafe, as user object may contain sensitive information like password-hash.
If you just want to store users in memory, you should do just that. Create a cache that holds user objects.
var Users = {};
passport.serializeUser(function(user, done) {
Users[user._id] = user; // store in cache
done(null, user._id);
});
passport.deserializeUser(function(id, done) {
if(Users[id]) return done(null, Users[id]); // retrieve from cache
User.findById(id, function(err, user){
if(err) return done(err);
Users[id] = user; // store in cache now if wasn't already found
done(null, user);
});
});
When I query Meteor.users I do not receive the services field or any other custom fields I have created outside of profile. Why is it that I only receive _id and profile on the client and how can I receive the entire Meteor.users object?
Thanks.
From the DOcs
By default, the current user's username, emails and profile are published to the client. You can publish additional fields for the current user with:
As said above If you want other fields you need to publish them
// server
Meteor.publish("userData", function () {
if (this.userId) {
return Meteor.users.find({_id: this.userId},
{fields: {'services': 1, 'others': 1}});
} else {
this.ready();
}
});
// client
Meteor.subscribe("userData");
The above answer does work, but it means you have to subscribe to said data, which you should do if you are getting data from users other than the currently logged in one.
But if all you care about is the logged in user's data, then you can instead use a null publication to get the data without subscribing.
On the server do,
Meteor.publish(null, function () {
if (! this.userId) {
return null;
}
return Meteor.users.find(this.userId, {
fields: {
services: 1,
profile: 1,
roles: 1,
username: 1,
},
});
});
And this is actually what the accounts package does under the hood
Getting into sails.js - enjoying the cleanliness of models, routes, and the recent addition of associations. My dilemma:
I have Users, and Groups. There is a many-many relationship between the two.
var User = {
attributes: {
username: 'string',
groups: {
collection: 'group',
via: 'users'
}
}
};
module.exports = User;
...
var Group = {
attributes: {
name: 'string',
users: {
collection: 'user',
via: 'groups',
dominant: true
}
}
};
module.exports = Group;
I'm having difficulty understanding how I would save a user and it's associated groups.
Can I access the 'join table' directly?
From an ajax call, how should I be sending in the list of group ids to my controller?
If via REST URL, is this already accounted for in blueprint functions via update?
If so - what does the URL look like? /user/update/1?groups=1,2,3 ?
Is all of this just not supported yet? Any insight is helpful, thanks.
Documentation for these blueprints is forthcoming, but to link two records that have a many-to-many association, you can use the following REST url:
POST /user/[userId]/groups
where the body of the post is:
{id: [groupId]}
assuming that id is the primary key of the Group model. Starting with v0.10-rc5, you can also simultaneously create and a add a new group to a user by sending data about the new group in the POST body, without an id:
{name: 'myGroup'}
You can currently only add one linked entity at a time.
To add an entity programmatically, use the add method:
User.findOne(123).exec(function(err, user) {
if (err) {return res.serverError(err);}
// Add group with ID 1 to user with ID 123
user.groups.add(1);
// Add brand new group to user with ID 123
user.groups.add({name: 'myGroup'});
// Save the user, committing the additions
user.save(function(err, user) {
if (err) {return res.serverError(err);}
return res.json(user);
});
});
Just to answer your question about accessing the join tables directly,
Yes you can do that if you are using Model.query function. You need to check the namees of the join tables from DB itself. Not sure if it is recommended or not but I have found myself in such situations sometimes when it was unavoidable.
There have been times when the logic I was trying to implement involved a lot many queries and it was required to be executed as an atomic transaction.
In those case, I encapsulated all the DB logic in a stored function and executed that using Model.query
var myQuery = "select some_db_function(" + <param> + ")";
Model.query(myQuery, function(err, result){
if(err) return res.json(err);
else{
result = result.rows[0].some_db_function;
return res.json(result);
}
});
postgres has been a great help here due to json datatype which allowed me to pass params as JSON and also return values as JSON
I am not able to update my mongoose schema because of a CastERror, which makes sence, but I dont know how to solve it.
Trip Schema:
var TripSchema = new Schema({
name: String,
_users: [{type: Schema.Types.ObjectId, ref: 'User'}]
});
User Schema:
var UserSchema = new Schema({
name: String,
email: String,
});
in my html page i render a trip with the possibility to add new users to this trip, I retrieve the data by calling the findById method on the Schema:
exports.readById = function (request, result) {
Trip.findById(request.params.tripId).populate('_users').exec(function (error, trip) {
if (error) {
console.log('error getting trips');
} else {
console.log('found single trip: ' + trip);
result.json(trip);
}
})
};
this works find. In my ui i can add new users to the trip, here is the code:
var user = new UserService();
user.email = $scope.newMail;
user.$save(function(response){
trip._users.push(user._id);
trip.$update(function (response) {
console.log('OK - user ' + user.email + ' was linked to trip ' + trip.name);
// call for the updated document in database
this.readOne();
})
};
The Problem is that when I update my Schema the existing users in trip are populated, means stored as objects not id on the trip, the new user is stored as ObjectId in trip.
How can I make sure the populated users go back to ObjectId before I update? otherwise the update will fail with a CastError.
see here for error
I've been searching around for a graceful way to handle this without finding a satisfactory solution, or at least one I feel confident is what the mongoosejs folks had in mind when using populate. Nonetheless, here's the route I took:
First, I tried to separate adding to the list from saving. So in your example, move trip._users.push(user._id); out of the $save function. I put actions like this on the client side of things, since I want the UI to show the changes before I persist them.
Second, when adding the user, I kept working with the populated model -- that is, I don't push(user._id) but instead add the full user: push(user). This keeps the _users list consistent, since the ids of other users have already been replaced with their corresponding objects during population.
So now you should be working with a consistent list of populated users. In the server code, just before calling $update, I replace trip._users with a list of ObjectIds. In other words, "un-populate" _users:
user_ids = []
for (var i in trip._users){
/* it might be a good idea to do more validation here if you like, to make
* sure you don't have any naked userIds in this array already, as you would
*/in your original code.
user_ids.push(trip._users[i]._id);
}
trip._users = user_ids;
trip.$update(....
As I read through your example code again, it looks like the user you are adding to the trip might be a new user? I'm not sure if that's just a relic of your simplification for question purposes, but if not, you'll need to save the user first so mongo can assign an ObjectId before you can save the trip.
I have written an function which accepts an array, and in callback returns with an array of ObjectId. To do it asynchronously in NodeJS, I am using async.js. The function is like:
let converter = function(array, callback) {
let idArray;
async.each(array, function(item, itemCallback) {
idArray.push(item._id);
itemCallback();
}, function(err) {
callback(idArray);
})
};
This works totally fine with me, and I hope should work with you as well