mongoosejs upsert with on save hook - mongodb

I'm trying to add data to my mongo database with mongoose, but there is a high probability that most of the data is already in the database, only a small number of fields need to be updated. Creation time for the record and last time updated need to be saved.
My first attempt at solving this problem included using the Model.save function, given that my model is called server, and data is an object coming from an external http service, which specifies the unique _id in data.
var instance = new Server(data);
instance.save(function(err){
if(err)
console.log(err);
});
also my pre-save hook:
ServerSchema.pre('save', function(next) {
this.lastseen = Date.now();
if (!this.isNew)
return next() //if the entry isn't new, lets not change the date registred
this.registered = Date.now();
next() //Don't forget this!
})
The problem here is that on duplicate _id the save chokes, with error E11000 duplicate key error index...
This now makes sense as save only does an update when the document instance is not created using the new operator.
So in my next attempt, I added code to attempt to lookup the document, then used underscore.js's _.extend to merge the new document with the one found in the database, then saved that to the database. The problem with this approach is that it require an extra call to the database for each chunk of data being processed.
My third attempt uses the Model.findByIdAndUpdate with {upsert:true} this works, in terms of stroring the data in the database, but schema defaults and my pre-save hook isn't triggered.
The fourth attempt uses code suggested by #aheckmann in this gist: https://gist.github.com/2764948
var server = new Server();
server.init(ping);
server.save(function(err){
if(err) {
console.log("DB Error: ",err);
return res.send('DB Error')
}
//if server approved, tell the inworld server to sync textures
if(server.approved)
res.send('success')
else
res.send('skip')
user.servers.addToSet(ping._id); //add the server to the user's list
user.save(function(err, usr){
if(err)
console.log("DB Error: ", err);
})
})
Here again, the pre-save hook isn't triggered. Am I to understand that the only to upsert with hooks is to attempt to find the document first with a findById ?
Q:
Is there a way to "upsert" Insert or Update based on the primary unique key without making more than one database call per chunk of data? Is there a method, or obvious fact that I am overlooking?

I don't think that you can do it with less then two calls to DB, unless you'll drop mongoose part and use mongo driver directly. But you can create a static method to do all the job for you:
ServerSchema.statics.findOrCreate(function(doc, next) {
this.findById(doc._id, function(err, res) {
res || (res = new this);
_.extend(res, doc); // add new data to the document
next(err, res); // if (err != null) then something went wrong
});
});
findByIdAndUpdate not triggers presave hook because it calls mongo driver directly.

Related

MongoDB pre save set field based on lookup logic

People register for an event. There are two collections in the database. One for new registrations coming in and one for the registrations of previous years. Both contain an email field as unique identifier.
I would like to know if its possible to check if a newly registered person has registered before in previous years. If so add a field, for example: returningCustomer: true. Otherwise add returningCustomer: false
I am using Mongoose and have a User model for new registrations. I don't have a model (yet) for previously registered users. Would that be neccesary? If it is possible to check if a person has registered before and a field can be added before saving, it might be handy to save the user to the returning customers collection immediatly as well.
I know it is possible to access the current document and collection using a pre save hook, but how about doing a lookup in another collection, write a bit of logic and add a field to the current document pre save?
userSchema.pre('save', function (doc, next) {
const exists = otherCollection.find({ email: doc.email });
exists ? doc.returningCustomer = true : doc.returningCustomer = false;
next();
});
You should have a model for the collection you want to lookup.
Then you can query the other collection before saving the current collection.
CurrentModel.pre('save', async function (next) {
const doc = await OtherModel.find({ field: this.field });
doc.length ? this.returningCustomer = false : this.returningCustomer = true;
next();
});

Meteor subscription to get only new documents

Just wondering if there is a way to set my meteor subscription to load only new documents from a mongo collection, avoiding to sync deletes and updates (Since they are not relevant in the data that is shown to user).
Why I need that? It seems anytime I do a Meteor.subscribe after an offline period, the WHOLE collection is sent again from server to client, while I only need the new records.
I think this happen to keep local/remote database integrity, but since my app is planned to work online/offline (I'm using also groundDB), it seems to me It will be very inefficient in terms of data usage.
Thanks in advance.
You can create a publish which sends only new documents. Like:
Meteor.publish('newDocumentsOnly', () => {
let initializing = true;
const handle = Collection.find().observeChanges({
added: (id, fields) => {
if (initializing) return;
this.added('Collection', id, fields);
}
});
initializing = false;
this.ready();
this.onStop(() => {
handle.stop();
});
});

create personal collections for logged in users

import {favRestaurants} from '/lib/collections';
import {Meteor} from 'meteor/meteor';
import {check} from 'meteor/check';
export default function () {
Meteor.methods({
'favRestaurants.create' (id, name, rating, priceLevel, type) {
check(id, String);
check(name, String);
check(rating, Number);
check(priceLevel, Number);
check(type, String);
const createdAt = new Date();
const restaurant = {id, name, rating, priceLevel, type, createdAt};
if(check(Meteor.user()) == null){
console.log('onlye logged in users can data');
}else{
FavRestaurants.insert(restaurant);
}
}
});
}
This is my insert method for adding data to the restaurants collections. When i console log the 'check(Meteor.user())' in the console i get null as output. By that logic you shouldn't be able to add data to the collection, although this is still possible.
I would also like to make the FavResaurants collection personal for each user. Iv'e tried to check if there is a user and then adding a collection in the main.js file on the client side.
Meteor.loggingIn(() => {
console.log('check for user method');
var restId = 0;
if(Meteor.user() != null){
console.log('created new collection for the user');
const FavRestaurants = new Mongo.Collection('favRestaurants' + restId);
}
restId++;
});
I dont get any output to console using this method, which i found in the meteor docs. Is the code in the wrong place? Any help is much appriciated.
According to the docs the Accounts.ui.config is the method i should use. But I'm not sure in code i should put it. So far the placement of this method has resulted in my application crashing.
Answering your first question, to allow only logged-in clients to access a method, you should use something like:
if (!Meteor.userId()) {
throw new Meteor.Error('403', 'Forbidden');
}
Now, I see you want a collection to store favorite restaurants for each user in client side. But as I see it, there'd be only one logged in user per client, so you don't need a separate collection for each user (as the collection is in each client), you can just refer the user with it's id, and then fetch a user's favorite restaurants by a query like:
FavRestaurants.find({user: Meteor.userId()});
Moreover, as the docs suggest, Meteor.loggingIn is a method which tells you if some user is in the process of logging in. What you are doing is over-riding it, which doesn't make sense.
You should do something like:
if (Meteor.loggingIn()) {
// Do your stuff
}
Hope it gives you more clarity.
Creating a collection per user is a bad approach.
Define your favRestaurants collection once and add a owner field in the restaurant document before insert.
Create a publish method to publish to client side the userid favrestaurant only.
One more thing, check your userid first in your Meteor method, it will avoid unnecessary proces.
Regs

Crossreference two collections

I have two collections A + B. Both are created at the same event. B is created some lines before A. Now I need to store in A the _id of B. How do I get the id of the just created B?
I am new to meteor and mongoDB, is the _id internally passed back on creation so that it is already available (I did not find an indication for this) or do I need to reread B? If so how do I do this best?
EDIT
I understand that the _id is passed back on the server after the insert.
Client:
Meteor.call('addB',b );
Server:
'addB':function(b){
return B.insert(b);
},
How can I pass B._id to the client so that I can do, on the client, something like:
a.id_of_B = B._id
Meteor.call('addA',a );
collection.insert returns the value of the created _id field. The docs says "Returns its unique _id.".
To return the values to the client a simple callback can be used:
Here again the link to the docs collection.insert
Client:
Meteor.call('addB',b function(error, result) {
BId = result;
});
Server:
'addB':function(b){
return B.insert(b);
},
Remember that in general you don't need Meteor.call() to insert into a collection. You can do both inserts on the client (if the collections are available there) and these inserts will automatically be synchronized back to the server.
var a = {...}; // some object
var b = {...}; // some other object
B.insert(b,function(err,id){ // asynchronous style
a.idOfB = id;
A.insert(a);
})
a.idOfB = B.insert(b); // synchronous style
A.insert(a);

Node + Mongoose: Get last inserted ID?

I want to retrieve the last inserted _id, using mongoose as MongoDB wrapper for node.js. I've found the following tutorial, but I can't change any node modules because the app runs on a public server:
Getting "Last Inserted ID" (hint - you have to hack Mongoose)
Any other ideas? This what I want to do:
Insert new user
Get user's _id value
Set a new session based on user's id
Redirect to /
Thanks!
I'm using mongoose version 1.2.0 and as soon as I created a new instance of a mongoose model, the _id is already set.
coffee> u = new User()
[object Object]
coffee> u._id
4dd68fc449aaedd177000001
I also verified that after I call u.save() the _id remains the same. I verified via MongoHub that this is indeed the real ID saved into MongoDB.
If you explicitly declare
_id: Schema.ObjectId
for your model, then the ObjectId will not be available after new or save.
This is probably a bug.
If you're looking to get the last inserted _id of a sub object, then create the object, and add it to the item. Here's an example in NowJS using MongoDB and Mongoose (to add some schema sugar) which then converts the result to JSON to send back to the client:
var nowRoomID = this.now.room;
var Conversation = mongoose.model('Conversation');
Conversation.findById(convID, function(error, conversation) {
var Blip = mongoose.model('Blip');
var createdBlip = new Blip();
createdBlip.author= nowUserName;
createdBlip.authorid = parsed.authorid;
createdBlip.body = revisedText;
createdBlip.created_at = new Date();
createdBlip.modified_at = new Date();
conversation.blips.push(createdBlip);
parsed._id = createdBlip._id; //NOTE: ID ACCESSED HERE
message = JSON.stringify(parsed);
conversation.save(function (err) {
if (!err) {
console.log('Success - saved a blip onto a conversation!');
nowjs.getGroup(nowRoomID).now.receiveMessage(nowUserName, message);
}
});
With MongoDB, if you don't explicitly set a document's _id value then the client driver will automatically set it to an ObjectId value. This is different from databases that might generate IDs on the server and need another query to retrieve it, like with SQL Server's scope_identity() or MySQL's last_insert_id().
This allows you to insert data asynchronously because don't need to wait for the server to return an _id value before you continue.
So, as shown is Peter's answer, the _id is available before the document is saved to the database.
I just get the id from the document passed to the callback, since save returns the saved document.
Check below url
http://mongodb.github.io/node-mongodb-native/markdown-docs/insert.html
you will find following code in given url
var document = {name:"David", title:"About MongoDB"};
collection.insert(document, {w: 1}, function(err, records){
console.log("Record added as "+records[0]._id);
});