I have two tables as seen below
The first table is for users and is populated via a registration form on the client side. When a new user is created, I need the second 'quotas' table to be populated with date, amount, and linked with the user id. The 'user_id' is used to pull the quotas information in a GET and display client side. I am having issues using the 'id' to populate the second table at the time of creation. I am using knex to make all queries. Would I be using join to link them in knex?
server
hydrateRouter // get all users
.route('/api/user')
.get((req, res) => {
knexInstance
.select('*')
.from('hydrate_users')
.then(results => {
res.json(results)
})
})
.post(jsonParser, (req, res, next) => { // register new users
const { username, glasses } = req.body;
const password = bcrypt.hashSync(req.body.password, 8);
const newUser = { username, password, glasses };
knexInstance
.insert(newUser)
.into('hydrate_users')
.then(user => {
res.status(201).json(user);
})
.catch(next);
})
client
export default class Register extends React.Component {
constructor(props) {
super(props);
this.state = {
username: '',
password: '',
glasses: 0
}
}
handleSubmit(event) {
event.preventDefault();
fetch('http://localhost:8000/api/user', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(this.state)
})
.then(response => response.json())
.then(responseJSON => {
this.props.history.push('/login');
})
}
server side route for displaying the water amount
hydrateRouter
.route('/api/user/waterconsumed/:user_id') // display water consumed/day
.all(requireAuth)
.get((req, res, next) => {
const {user_id} = req.params;
knexInstance
.from('hydrate_quotas')
.select('amount')
.where('user_id', user_id)
.first()
.then(water => {
res.json(water)
})
.catch(next)
})
Thank you!
Getting the id of an inserted row
So this is a common pattern in relational databases, where you can't create the egg until you have the unique id of the chicken that lays it! Clearly, the database needs to tell you how it wants to refer to the chicken.
In Postgres, you can simply use Knex's .returning function to make it explicit that you want the new row's id column returned to you after a successful insert. That'll make the first part of your query look like this:
knexInstance
.insert(newUser)
.into('users')
.returning('id')
Note: not all databases support this in the same way. In particular, if you happen to be developing locally using SQLite, it will return the number of rows affected by the query, not the id, since SQLite doesn't support SQL's RETURNING. Best is just to develop locally using Postgres to avoid nasty surprises.
Ok, so we know which chicken we're after. Now we need to make sure we've waited for the right id, then go ahead and use it:
.then(([ userId ]) => knexInstance
.insert({ user_id: userId,
date: knex.fn.now(),
amount: userConstants.INITIAL_QUOTA_AMOUNT })
.into('quotas')
)
Or however you choose to populate that second table.
Note: DATE is a SQL keyword. For that reason, it doesn't make a great column name. How about created or updated instead?
Responding with sensible data
So that's basic "I have the ID, let's insert to another table" strategy. However, you actually want to be able to respond with the user that was created... this seems like sensible API behaviour for a 201 response.
What you don't want to do is respond with the entire user record from the database, which will expose the password hash (as you're doing in your first code block from your question). Ideally, you'd probably like to respond with some UI-friendly combination of both tables.
Luckily, .returning also accepts an array argument. This allows us to pass a list of columns we'd like to respond with, reducing the risk of accidentally exposing something to the API surface that we'd rather not transmit.
const userColumns = [ 'id', 'username', 'glasses' ]
const quotaColumns = [ 'amount' ]
knexInstance
.insert(newUser)
.into('users')
.returning(userColumns)
.then(([ user]) => knexInstance
.insert({
user_id: user.id,
date: knex.fn.now(),
amount: userConstants.INITIAL_QUOTA_AMOUNT
})
.into('quotas')
.returning(quotaColumns)
.then(([ quota ]) => res.status(201)
.json({
...user,
...quota
})
)
)
Async/await for readability
These days, I'd probably avoid a promise chain like that in favour of the syntactic sugar that await provides us.
try {
const [ user ] = await knexInstance
.insert(newUser)
.into('users')
.returning(userColumns)
const [ quota ] = await knexInstance
.insert({
user_id: userId,
date: knex.fn.now(),
amount: userConstants.INITIAL_QUOTA_AMOUNT
})
.into('quotas')
.returning(quotaColumns)
res
.status(201)
.json({
...user,
...quota
})
} catch (e) {
next(Error("Something went wrong while inserting a user!"))
}
A note on transactions
There are a few assumptions here, but one big one: we assume that both inserts will be successful. Sure, we provide some error handling, but there's still the possibility that the first insert will succeed, and the second fail or time out for some reason.
Typically, we'd do multiple insertions in a transaction block. Here's how Knex handles this:
try {
const userResponse = await knexInstance.transaction(async tx => {
const [ user ] = await tx.insert(...)
const [ quota ] = await tx.insert(...)
return {
...user,
...quota
}
})
res
.status(201)
.json(userResponse)
} catch (e) {
next(Error('...'))
}
This is good general practice for multiple inserts that depend on each other, since it sets up an "all or nothing" approach: if something fails, the database will go back to its previous state.
Related
Good day everyone, I am working with watermelondb and I have the code below, but I don't know how to actually use it. I am new in watermelondb and I don't know how to pass data as props to the pullChanges and pushChanges objects. How do I pass necessary data like changes and lastPulledAt from the database into the sync function when I call it. And I need more explanation on the migrationsEnabledAtVersion: 1 too. Thanks in advance for your gracious answers.
import { synchronize } from '#nozbe/watermelondb/sync'
async function mySync() {
await synchronize({
database,
pullChanges: async ({ lastPulledAt, schemaVersion, migration }) => {
const urlParams = `last_pulled_at=${lastPulledAt}&schema_version=${schemaVersion}&migration=${encodeURIComponent(JSON.stringify(migration))}`
const response = await fetch(`https://my.backend/sync?${urlParams}`)
if (!response.ok) {
throw new Error(await response.text())
}
const { changes, timestamp } = await response.json()
return { changes, timestamp }
},
pushChanges: async ({ changes, lastPulledAt }) => {
const response = await fetch(`https://my.backend/sync?last_pulled_at=${lastPulledAt}`, {
method: 'POST',
body: JSON.stringify(changes)
})
if (!response.ok) {
throw new Error(await response.text())
}
},
migrationsEnabledAtVersion: 1,
})
}
Watermelondb's documentation is terrible and its link to typescript even worse.
I spent almost a week to get 100% synchronization with a simple table, now I'm having the same problems to solve the synchronization with associations.
Well, the object you need to return in pullChanges is of the following form:
return {
changes: {
//person is the name of the table in the models
person: {
created: [
{
// in created you need to send null in the id, if you don't send the id it doesn't work
id: null,
// other fields of your schema, not model
}
],
updated: [
{
// the fields of your schema, not model
}
],
deleted: [
// is a string[] consisting of the watermelondb id of the records that were deleted in the remote database
],
}
},
timestamp: new Date().getTime() / 1000
}
In my case, the remote database is not a watermelondb, it's a mySQL, and I don't have an endpoint in my API that returns everything in the watermelon format. For each table I do a search with deletedAt, updatedAt or createdAt > lastPulledAt and do the necessary filtering and preparations so that the data from the remote database is in the schema format of the local database.
In pushChanges I do the reverse data preparation process by calling the appropriate creation, update or deletion endpoints for each of the tables.
It's costly and annoying to do, but in the end it works fine, the biggest problem is watermelon's documentation which is terrible.
I am currently able to sign in just fine with previously created user credentials and use the app as normal, but am unable to create a new user. I am using React.js on the client side and an Express api on the backend. I am getting a mongoose validation error. All of the authentication came with the template the course has us use and I haven't touched any of those files. I even went back and compared commit history trees to ensure that nothing was changed.
Here is my user schema and sign-up route. I tried eliminating uniqueness from the model and that didn't impact it. I know there is a lot of potential places something could be going wrong, but if anyone has any suggestions on potential issues I would be forever grateful! I console logged the req.body.credentials within sign up and the data being sent over looks good.
Error code: 422 Unprocessable Entity
Server side error: 'The received params failed a Mongoose validation'
const mongoose = require('mongoose')
const { petSchema } = require('./pet.js')
const { pictureSchema } = require('./picture.js')
const userSchema = new mongoose.Schema({
email: {
type: String,
required: true,
unique: true
},
hashedPassword: {
type: String,
required: true
},
token: String,
pets: [petSchema],
pictures: [pictureSchema]
}, {
timestamps: true,
toObject: {
// remove `hashedPassword` field when we call `.toObject`
transform: (_doc, user) => {
delete user.hashedPassword
return user
}
}
})
module.exports = mongoose.model('User', userSchema)
// SIGN UP
// POST /sign-up
router.post('/sign-up', (req, res) => {
// start a promise chain, so that any errors will pass to `handle`
console.log(req.body.credentials)
Promise.resolve(req.body.credentials)
// reject any requests where `credentials.password` is not present, or where
// the password is an empty string
.then(credentials => {
if (!credentials ||
!credentials.password ||
credentials.password !== credentials.password_confirmation) {
throw new BadParamsError()
}
})
// generate a hash from the provided password, returning a promise
.then(() => bcrypt.hash(req.body.credentials.password, bcryptSaltRounds))
.then(hash => {
// return necessary params to create a user
return {
email: req.body.credentials.email,
hashedPassword: hash
}
})
// create user with provided email and hashed password
.then(user => User.create(user))
// send the new user object back with status 201, but `hashedPassword`
// won't be sent because of the `transform` in the User model
.then(user => res.status(201).json({ user: user.toObject() }))
// pass any errors along to the error handler
.catch(err => handle(err, res))
})
Solved. One of the subdocuments I had within user had a key with a value set to unique. I needed to eliminate that because my database was indexing users with a null value and throwing a duplicate error. I then needed to reset my database (I just renamed it to test it out) so that it didn't have any saved indexes with that configuration. I just deleted my collections within Heroku as well (luckily I didn't have significant amounts of data in there and this solution was perfectly fine for my situation). I am now able to sign up users again without any duplicate key errors.
I'm trying to return create a paginated list. I used graphql to query the data. With my query, I pass the number of records I need (In a variable named first) and the ID of the last fetched record (In a varible called after). Now I managed to write a query (Note that I used mongoose) to fetch the records. Now what I need to do is get the relavant information to perform the pagination like hasNextPage, hasPreviousPage, currentPage and totalPages.
To get most of these information I need to get the total number of records in the database. To do that I need to send another db request.
I also need to know the position of the record in the table. No idea how.
Here's the query:
new Promise((resolve, reject) =>
Company.where('_id')
.gt(after)
.limit(first)
.lean()
.exec((error, doc) => {
if (error) {
reject(error);
}
resolve({
edges: doc,
pageInfo: {
hasNextPage: '...',
hasPreviousPage: '...',
currentPage: '...',
totalPages: '...'
}
});
}))
Any idea how to do this efficiently?
you can try this module mongoose-paginate
here what i uses, for pagination,
var current = req.query.filter.current;
var limit = req.query.filter.limit;
console.log('params.query.filter.current',current);
var skip = Number(limit)*Number(current)-Number(limit);
console.log('skip::',skip);
Cours.find({'attributes.version.status': true}).skip(skip).limit(limit).sort({_id:'asc'}).exec(function (err, resulta) {
if (err) {
console.log('erreur trouverCours');
res.json({
protecteds: err
});
}
console.log('cours ::', resulta);
res.json({
"data": resulta
});
});
Getting into sails.js - enjoying the cleanliness of models, routes, and the recent addition of associations. My dilemma:
I have Users, and Groups. There is a many-many relationship between the two.
var User = {
attributes: {
username: 'string',
groups: {
collection: 'group',
via: 'users'
}
}
};
module.exports = User;
...
var Group = {
attributes: {
name: 'string',
users: {
collection: 'user',
via: 'groups',
dominant: true
}
}
};
module.exports = Group;
I'm having difficulty understanding how I would save a user and it's associated groups.
Can I access the 'join table' directly?
From an ajax call, how should I be sending in the list of group ids to my controller?
If via REST URL, is this already accounted for in blueprint functions via update?
If so - what does the URL look like? /user/update/1?groups=1,2,3 ?
Is all of this just not supported yet? Any insight is helpful, thanks.
Documentation for these blueprints is forthcoming, but to link two records that have a many-to-many association, you can use the following REST url:
POST /user/[userId]/groups
where the body of the post is:
{id: [groupId]}
assuming that id is the primary key of the Group model. Starting with v0.10-rc5, you can also simultaneously create and a add a new group to a user by sending data about the new group in the POST body, without an id:
{name: 'myGroup'}
You can currently only add one linked entity at a time.
To add an entity programmatically, use the add method:
User.findOne(123).exec(function(err, user) {
if (err) {return res.serverError(err);}
// Add group with ID 1 to user with ID 123
user.groups.add(1);
// Add brand new group to user with ID 123
user.groups.add({name: 'myGroup'});
// Save the user, committing the additions
user.save(function(err, user) {
if (err) {return res.serverError(err);}
return res.json(user);
});
});
Just to answer your question about accessing the join tables directly,
Yes you can do that if you are using Model.query function. You need to check the namees of the join tables from DB itself. Not sure if it is recommended or not but I have found myself in such situations sometimes when it was unavoidable.
There have been times when the logic I was trying to implement involved a lot many queries and it was required to be executed as an atomic transaction.
In those case, I encapsulated all the DB logic in a stored function and executed that using Model.query
var myQuery = "select some_db_function(" + <param> + ")";
Model.query(myQuery, function(err, result){
if(err) return res.json(err);
else{
result = result.rows[0].some_db_function;
return res.json(result);
}
});
postgres has been a great help here due to json datatype which allowed me to pass params as JSON and also return values as JSON
Is there a way to delete all children of an parent in Mongoose, similar to using MySQLs foreign keys?
For example, in MySQL I'd assign a foreign key and set it to cascade on delete. Thus, if I were to delete a client, all applications and associated users would be removed as well.
From a top level:
Delete Client
Delete Sweepstakes
Delete Submissions
Sweepstakes and submissions both have a field for client_id. Submissions has a field for both sweepstakes_id, and client_id.
Right now, I'm using the following code and I feel that there has to be a better way.
Client.findById(req.params.client_id, function(err, client) {
if (err)
return next(new restify.InternalError(err));
else if (!client)
return next(new restify.ResourceNotFoundError('The resource you requested could not be found.'));
// find and remove all associated sweepstakes
Sweepstakes.find({client_id: client._id}).remove();
// find and remove all submissions
Submission.find({client_id: client._id}).remove();
client.remove();
res.send({id: req.params.client_id});
});
This is one of the primary use cases of Mongoose's 'remove' middleware.
clientSchema.pre('remove', function(next) {
// 'this' is the client being removed. Provide callbacks here if you want
// to be notified of the calls' result.
Sweepstakes.remove({client_id: this._id}).exec();
Submission.remove({client_id: this._id}).exec();
next();
});
This way, when you call client.remove() this middleware is automatically invoked to clean up dependencies.
In case your references are stored other way around, say, client has an array of submission_ids, then in a similar way as accepted answer you can define the following on submissionSchema:
submissionSchema.pre('remove', function(next) {
Client.update(
{ submission_ids : this._id},
{ $pull: { submission_ids: this._id } },
{ multi: true }) //if reference exists in multiple documents
.exec();
next();
});
which will remove the submission's id from the clients' reference arrays on submission.remove().
Here's an other way I found
submissionSchema.pre('remove', function(next) {
this.model('Client').remove({ submission_ids: this._id }, next);
next();
});
I noticed that all of answers here have a pre assigned to the schema and not post.
my solution would be this: (using mongoose 6+)
ClientSchema.post("remove", async function(res, next) {
await Sweepstakes.deleteMany({ client_id: this._id });
await Submission.deleteMany({ client_id: this._id });
next();
});
By definition post gets executed after the process ends pre => process => post.
Now, you're probably wondering how is this different than the other solutions provided here.
What if a server error or the id of that client was not found?
On pre, it would delete all sweeptakes and submissions before the deleting process start for client. Thus, in case of an error, it would be better to cascade delete the other documents once client or the main document gets deleted.
async and await are optional here. However, it matters on large data. so that the user wouldn't get those "going to be deleted" cascade documents data if the delete progress is still on.
At the end, I could be wrong, hopefully this helps someone in their code.
Model
const orderSchema = new mongoose.Schema({
// Множество экземпляров --> []
orderItems: [{
type: mongoose.Schema.Types.ObjectId,
ref: 'OrderItem',
required: true
}],
...
...
});
asyncHandler (optional)
const asyncHandler = fn => (req, res, next) =>
Promise
.resolve(fn(req, res, next))
.catch(next)
module.exports = asyncHandler;
controller
const asyncHandler = require("../middleware/asyncErrHandler.middleware");
// **Models**
const Order = require('../models/order.mongo');
const OrderItem = require('../models/order-item.mongo');
// #desc Delete order
// #route DELETE /api/v1/orders/:id
// #access Private
exports.deleteOrder = asyncHandler(async (req, res, next) => {
let order = await Order.findById(req.params.id)
if (!order) return next(
res.status(404).json({ success: false, data: null })
)
await order.remove().then( items => {
// Cascade delete -OrderItem-
items.orderItems.forEach( el => OrderItem.findById(el).remove().exec())
}).catch(e => { res.status(400).json({ success: false, data: e }) });
res.status(201).json({ success: true, data: null });
});
https://mongoosejs.com/docs/api/model.html#model_Model-remove