How to validate for ObjectID - mongodb

Using Joi schema validation, is it possible to validate against MongoDB ObjectID's?
Something like this could be great:
_id: Joi.ObjectId().required().error(errorParser),

const Joi = require('#hapi/joi')
Joi.objectId = require('joi-objectid')(Joi)
const schema = Joi.object({
id: Joi.objectId(),
name: Joi.string().max(100),
date: Joi.date()
})
checkout https://www.npmjs.com/package/joi-objectid

I find that if I do
Joi.object({
id: Joi.string().hex().length(24)
})
it works without installing any external library or using RegEx
The hex makes sure the string contains only hexadecimal characters and the length makes sure that it is a string of exactly 24 characters

This package works if you are using the new version of Joi.
const Joi = require('joi-oid')
const schema = Joi.object({
id: Joi.objectId(),
name: Joi.string(),
age: Joi.number().min(18),
})
package: https://www.npmjs.com/package/joi-oid

If you want a TypeScript version of the above library integrated with Express without installing anything:
import Joi from '#hapi/joi';
import { createValidator } from 'express-joi-validation';
export const JoiObjectId = (message = 'valid id') => Joi.string().regex(/^[0-9a-fA-F]{24}$/, message)
const validator = createValidator({
passError: true,
});
const params = Joi.object({
id: JoiObjectId().required(),
});
router.get<{ id: string }>(
'/:id',
validator.params(params),
(req, res, next) => {
const { id } = req.params; // id has string type
....
}
);

I share mine
let id =
Joi.string()
.regex(/^[0-9a-fA-F]{24}$/)
.message('must be an oid')

With joi naked package you can use this:
const ObjectId = joi.object({
id: joi.binary().length(12),
})

This is a core function without any external package. I have used Joi and mongoose packages to achieve it.
const Joi = require("joi");
const ObjectID = require("mongodb").ObjectID;
const schema = Joi.object({
id: Joi.string().custom((value, helpers) => {
const filtered = ObjectID.isValid(value)
return !filtered ? helpers.error("any.invalid") : value;
},
"invalid objectId", ).required(),
name: Joi.string(),
age: Joi.number().min(18),
})

I see many correct answers here, but I also want to express my opinion.
I am not against installing npm packages, but installing one package just to validate an object id is overkill. I know programmers are lazy but C'mooon :)
Here is my full code function that validates two object id properties, very simple:
function validateRental(rental) {
const schema = Joi.object({
movieId: Joi.string()
.required()
.regex(/^[0-9a-fA-F]{24}$/, 'object Id'),
customerId: Joi.string()
.required()
.regex(/^[0-9a-fA-F]{24}$/, 'object Id'),
})
const { error } = schema.validate(rental)
return {
valid: error == null,
message: error ? error.details[0].message : null,
}
}
This way, if any of the properties contain invalid id like this:
{
"movieId": "123456",
"customerId": "63edf556f383d108d54a68a0"
}
This will be the error message:
`"movieId" with value "123456" fails to match the object Id pattern`

Related

Variables exporting error in MongoDB error

Question
I have provided my code below for reference. I'm using MongoDB and discord.js v12. So basically, I have made a !info command which shows some general info of the user.
What this code does is, it checks through the member's roles, and regarding which role they have, it calculates their total claim time (for giveaways etc.). The problem here, is with the donator role. I can't figure out why I can't use the donates variable outside the db.findOne block. Here, data.content.length shows the total donates of the users, which means donates * 5 is +5 claim time for each donate.
My Code
const moment = require('moment');
module.exports = {
name: 'info',
async execute(client, message, args, Discord){
const member = message.mentions.members.first() || message.guild.members.cache.get(args[0]) || message.member;
const db = require('../models/d-schema');
db.findOne({ guildid: message.guild.id, user: member.user.id }, async(err, data)=>{
if(err) throw err;
if(data){
const donates = parseInt(data.content.length);
}
})
var DefaultTime = 10;
var support = 0;
var donate = 0;
var boost = 0;
const userRoles = member.roles.cache.map((r) => r.name);
if (userRoles.includes("୨・supporter")) {
support = 3;
}
if (userRoles.includes("୨・donator")) {
donate = donates * 5;
}
if (userRoles.includes("୨・booster")) {
boost = 10;
}
const TotalTime = DefaultTime + support + donate + boost;
const embed = new Discord.MessageEmbed()
.setThumbnail(member.user.displayAvatarURL( {dynamic: true} ))
.addFields(
{name: member.user.tag, value: member.user, inline: true},
{name: 'Nickname', value: `${member.nickname !== null ? member.nickname : 'None'}`, inline: true},
{name: 'Is Bot', value: member.user.bot, inline: true},
{name: 'Joined', value: `${moment.utc(member.joinedAt).format("MMMM Do YYYY")}`, inline: true},
{name: 'Created', value: `${moment.utc(member.user.createdAt).format("MMMM Do YYYY")}`, inline: true},
{name: 'Claim Time', value: `${TotalTime} seconds`, inline: true},
)
.setFooter(`ID : ${member.user.id}`)
.setTimestamp()
.setColor('00ffcc')
message.channel.send(embed)
}
}
You cannot use the donates variable because you are declaring it inside the db.findOne() block. This is called variables scope. For better understanding you can read this answer.
If you want to use it outside of that block, you have to declare it beforehand, like this:
let donates;
db.findOne({ guildid: message.guild.id, user: member.user.id }, async(err, data)=>{
if(err) throw err;
if(data){
donates = parseInt(data.content.length);
}
})
Now you're able to use that variable outside of the db.findOne() block :)
Edit:
Alternative way:
It would be easier to use the function asynchronously. That way, everything can be scoped in the same block!
Example:
These two methods will give the same results:
const data = await Model.findOne({ ... });
console.log(data);
Model.findOne({ ... }, (err, data) => {
console.log(data);
});
Suggestion from Lioness100

How to make Mongoose model.insertMany insert documents with numerical and ordered ids?

I have this route in the backend express server:
router.route('/fillInformationAssetsSeverityEvaluation').post((req, res) => {
informationAssetsSeverityEvaluationRow.remove({}, (err) => {
if (err)
console.log(err);
else
// res.json("informationAssets Collection has been dropped!");
res.json('information Assets Severity Evaluation data has been received on the server side')
informationAssetsSeverityEvaluationRow.insertMany([req.body[0]], {
multi: true
}).then(documentsInserted => {
console.log('[req.body[0]]: ', [req.body[0]]);
console.log('documentsInserted: ', documentsInserted);
console.log('You have succesfully inserted ', documentsInserted.length, ' documents in informationAssetsSeverityEvaluation collection');
});
});
})
For the sake of simplicity, I am inserting only one document.
[req.body[0]]
{ REF: 'REFSHIT',
confFin: 'A',
confRep: 'A'}
But, in the real applications, I am inserting multiple documents similar to that.
This consoleLog :
console.log('documentsInserted: ', documentsInserted);
logs:
documentsInserted: [ { _id: 5d3453afc302d718e4870b53,
REF: 'REFSHIT',
confFin: 'A',
confRep: 'A'}]
As you see the id is automatically generated:
> _id: 5d3453afc302d718e4870b53
What I would like is: The ids of the different documents to be "numerically ordered". I.e:
Document 0 would have id 0
Document 1 would have id 1
Document 2 would have id 2
And so on and so forth.
After having made some research, I found out that I can do this manually by inserting the id manually inside the updateMany objects.
However, since I receive the documents objects from the request body, this is not a viable solution.
Any help?
Finally after trying four modules and a couple of days of trying for something that should be native to mongodb, I have found a simple solution. I hope it helps someone.
1/ Install mongoose-plugin-autoinc
2/
import mongoose from 'mongoose';
import { autoIncrement } from 'mongoose-plugin-autoinc';
const connection = mongoose.createConnection("mongodb://localhost/myDatabase");
const BookSchema = new mongoose.Schema({
author: { type: Schema.Types.ObjectId, ref: 'Author' },
title: String,
genre: String,
publishDate: Date
});
BookSchema.plugin(autoIncrement, 'Book');
const Book = connection.model('Book', BookSchema);
2/ In my case I have the models defined in models.js and the connection defined in server.js so I had to write this :
BookSchema.plugin(autoIncrement, 'Book');
in models.js
and instead of
const Book = connection.model('Book', BookSchema);
I have:
module.exports = {
informationAssetsRow: mongoose.model('informationAssetsRow', informationAssetsRow),
};
And in server.js:
const {
informationAssetsRow,
} = require('./models/models')

how to connect postgresql with graphql [duplicate]

GraphQL has mutations, Postgres has INSERT; GraphQL has queries, Postgres has SELECT's; etc., etc.. I haven't found an example showing how you could use both in a project, for example passing all the queries from front end (React, Relay) in GraphQL, but to a actually store the data in Postgres.
Does anyone know what Facebook is using as DB and how it's connected with GraphQL?
Is the only option of storing data in Postgres right now to build custom "adapters" that take the GraphQL query and convert it into SQL?
GraphQL is database agnostic, so you can use whatever you normally use to interact with the database, and use the query or mutation's resolve method to call a function you've defined that will get/add something to the database.
Without Relay
Here is an example of a mutation using the promise-based Knex SQL query builder, first without Relay to get a feel for the concept. I'm going to assume that you have created a userType in your GraphQL schema that has three fields: id, username, and created: all required, and that you have a getUser function already defined which queries the database and returns a user object. In the database I also have a password column, but since I don't want that queried I leave it out of my userType.
// db.js
// take a user object and use knex to add it to the database, then return the newly
// created user from the db.
const addUser = (user) => (
knex('users')
.returning('id') // returns [id]
.insert({
username: user.username,
password: yourPasswordHashFunction(user.password),
created: Math.floor(Date.now() / 1000), // Unix time in seconds
})
.then((id) => (getUser(id[0])))
.catch((error) => (
console.log(error)
))
);
// schema.js
// the resolve function receives the query inputs as args, then you can call
// your addUser function using them
const mutationType = new GraphQLObjectType({
name: 'Mutation',
description: 'Functions to add things to the database.',
fields: () => ({
addUser: {
type: userType,
args: {
username: {
type: new GraphQLNonNull(GraphQLString),
},
password: {
type: new GraphQLNonNull(GraphQLString),
},
},
resolve: (_, args) => (
addUser({
username: args.username,
password: args.password,
})
),
},
}),
});
Since Postgres creates the id for me and I calculate the created timestamp, I don't need them in my mutation query.
The Relay Way
Using the helpers in graphql-relay and sticking pretty close to the Relay Starter Kit helped me, because it was a lot to take in all at once. Relay requires you to set up your schema in a specific way so that it can work properly, but the idea is the same: use your functions to fetch from or add to the database in the resolve methods.
One important caveat is that the Relay way expects that the object returned from getUser is an instance of a class User, so you'll have to modify getUser to accommodate that.
The final example using Relay (fromGlobalId, globalIdField, mutationWithClientMutationId, and nodeDefinitions are all from graphql-relay):
/**
* We get the node interface and field from the Relay library.
*
* The first method defines the way we resolve an ID to its object.
* The second defines the way we resolve an object to its GraphQL type.
*
* All your types will implement this nodeInterface
*/
const { nodeInterface, nodeField } = nodeDefinitions(
(globalId) => {
const { type, id } = fromGlobalId(globalId);
if (type === 'User') {
return getUser(id);
}
return null;
},
(obj) => {
if (obj instanceof User) {
return userType;
}
return null;
}
);
// a globalId is just a base64 encoding of the database id and the type
const userType = new GraphQLObjectType({
name: 'User',
description: 'A user.',
fields: () => ({
id: globalIdField('User'),
username: {
type: new GraphQLNonNull(GraphQLString),
description: 'The username the user has selected.',
},
created: {
type: GraphQLInt,
description: 'The Unix timestamp in seconds of when the user was created.',
},
}),
interfaces: [nodeInterface],
});
// The "payload" is the data that will be returned from the mutation
const userMutation = mutationWithClientMutationId({
name: 'AddUser',
inputFields: {
username: {
type: GraphQLString,
},
password: {
type: new GraphQLNonNull(GraphQLString),
},
},
outputFields: {
user: {
type: userType,
resolve: (payload) => getUser(payload.userId),
},
},
mutateAndGetPayload: ({ username, password }) =>
addUser(
{ username, password }
).then((user) => ({ userId: user.id })), // passed to resolve in outputFields
});
const mutationType = new GraphQLObjectType({
name: 'Mutation',
description: 'Functions to add things to the database.',
fields: () => ({
addUser: userMutation,
}),
});
const queryType = new GraphQLObjectType({
name: 'Query',
fields: () => ({
node: nodeField,
user: {
type: userType,
args: {
id: {
description: 'ID number of the user.',
type: new GraphQLNonNull(GraphQLID),
},
},
resolve: (root, args) => getUser(args.id),
},
}),
});
We address this problem in Join Monster, a library we recently open-sourced to automatically translate GraphQL queries to SQL based on your schema definitions.
This GraphQL Starter Kit can be used for experimenting with GraphQL.js and PostgreSQL:
https://github.com/kriasoft/graphql-starter-kit - Node.js, GraphQL.js, PostgreSQL, Babel, Flow
(disclaimer: I'm the author)
Have a look at graphql-sequelize for how to work with Postgres.
For mutations (create/update/delete) you can look at the examples in the relay repo for instance.
Postgraphile https://www.graphile.org/postgraphile/ is Open Source
Rapidly build highly customisable, lightning-fast GraphQL APIs
PostGraphile is an open-source tool to help you rapidly design and
serve a high-performance, secure, client-facing GraphQL API backed
primarily by your PostgreSQL database. Delight your customers with
incredible performance whilst maintaining full control over your data
and your database. Use our powerful plugin system to customise every
facet of your GraphQL API to your liking.
You can use an ORM like sequelize if you're using Javascript or Typeorm if you're using Typescript
Probably FB using mongodb or nosql in backend. I've recently read a blog entry which explain how to connect to mongodb. Basically, you need to build a graph model to match the data you already have in your DB. Then write resolve, reject function to tell GQL how to behave when posting a query request.
See https://www.compose.io/articles/using-graphql-with-mongodb/
Have a look at SequelizeJS which is a promise based ORM that can work with a number of dialects; PostgreSQL, MySQL, SQLite and MSSQL
The below code is pulled right from its example
const Sequelize = require('sequelize');
const sequelize = new Sequelize('database', 'username', 'password', {
host: 'localhost',
dialect: 'mysql'|'sqlite'|'postgres'|'mssql',
pool: {
max: 5,
min: 0,
acquire: 30000,
idle: 10000
},
// SQLite only
storage: 'path/to/database.sqlite',
// http://docs.sequelizejs.com/manual/tutorial/querying.html#operators
operatorsAliases: false
});
const User = sequelize.define('user', {
username: Sequelize.STRING,
birthday: Sequelize.DATE
});
sequelize.sync()
.then(() => User.create({
username: 'janedoe',
birthday: new Date(1980, 6, 20)
}))
.then(jane => {
console.log(jane.toJSON());
});

Asynchronous Issues with JEST and MongoDB

I am getting inconsistent results with JEST when I try to remove items from a MongoDB Collection using the beforeEach() Hook.
My Mongoose schema and model defined as:
// Define Mongoose wafer sort schema
const waferSchema = new mongoose.Schema({
productType: {
type: String,
required: true,
enum: ['A', 'B'],
},
updated: {
type: Date,
default: Date.now,
index: true,
},
waferId: {
type: String,
required: true,
trim: true,
minlength: 7,
},
sublotId: {
type: String,
required: true,
trim: true,
minlength: 7,
},
}
// Define unique key for the schema
const Wafer = mongoose.model('Wafer', waferSchema);
module.exports.Wafer = Wafer;
My JEST tests:
describe('API: /WT', () => {
// Happy Path for Posting Object
let wtEntry = {};
beforeEach(async () => {
wtEntry = {
productType: 'A',
waferId: 'A01A001.3',
sublotId: 'A01A001.1',
};
await Wafer.deleteMany({});
// I also tried to pass in done and then call done() after the delete
});
describe('GET /:id', () => {
it('Return Wafer Sort Entry with specified ID', async () => {
// Create a new wafer Entry and Save it to the DB
const wafer = new Wafer(wtEntry);
await wafer.save();
const res = await request(apiServer).get(`/WT/${wafer.id}`);
expect(res.status).toBe(200);
expect(res.body).toHaveProperty('productType', 'A');
expect(res.body).toHaveProperty('waferId', 'A01A001.3');
expect(res.body).toHaveProperty('sublotId', 'A01A001.1');
});
}
So the error I always get is related to duplicate keys when I run my tests more than once:
MongoError: E11000 duplicate key error collection: promis_tests.promiswts index: waferId_1_sublotId_1 dup key: { : "A01A001.3", : "A01A001.1" }
But I do not understand how I can get this duplicate key error if the beforeEach() were firing properly. Am I trying to clear the collection improperly? I've tried passing in a done element to the before each callback and invoking it after delete command. I've also tried implementing the delete in beforeAll(), afterEach(), and afterAll() but still get inconsistent results. I'm pretty stumped on this one. I might just removed the schema key all together but I would like to understand what is going on here with the beforeEach(). Thanks in advance for any advice.
It might be because you are not actually using the promise API that mongoose has to offer. By default, mongooses functions like deleteMany() do not return a promise. You will have to call .exec() at the end of the function chain to return a promise e.g. await collection.deleteMany({}).exec(). So you are running into a race condition. deleteMany() also accepts a callback, so you could always wrap it in a promise. I would do something like this:
describe('API: /WT', () => {
// Happy Path for Posting Object
const wtEntry = {
productType: 'A',
waferId: 'A01A001.3',
sublotId: 'A01A001.1',
};
beforeEach(async () => {
await Wafer.deleteMany({}).exec();
});
describe('GET /:id', () => {
it('Return Wafer Sort Entry with specified ID', async () => {
expect.assertions(4);
// Create a new wafer Entry and Save it to the DB
const wafer = await Wafer.create(wtEntry);
const res = await request(apiServer).get(`/WT/${wafer.id}`);
expect(res.status).toBe(200);
expect(res.body).toHaveProperty('productType', 'A');
expect(res.body).toHaveProperty('waferId', 'A01A001.3');
expect(res.body).toHaveProperty('sublotId', 'A01A001.1');
});
}
Also, always expect the assertions with asynchronous code
https://jestjs.io/docs/en/asynchronous.html
You can read more about mongoose promises and query objects here
https://mongoosejs.com/docs/promises.html
Without deleting the schema index this seems to be the most reliable solution. Not 100% sure why it works over async await Wafer.deleteMany({});
beforeEach((done) => {
wtEntry = {
productType: 'A',
waferId: 'A01A001.3',
sublotId: 'A01A001.1',
};
mongoose.connection.collections.promiswts.drop(() => {
// Run the next test!
done();
});
});

Joi validate presence of a key or another one

I'm using Joi for http body validation. I want to allow keyA to be present OR a keyB to be present and required but not both. I don't find in documentation what I'm looking for and it seems that a cycling loop appear in my schema.
const messageSchema = Joi.object().keys({
keyA: Joi.when('keyB', { is: Joi.exist(), then: Joi.forbidden(), otherwise: Joi.string().required() }),
keyB: Joi.when('keyA', { is: Joi.exist(), then: Joi.forbidden(), otherwise: Joi.string().uri().required() }),
});
Any idea ?
Finally found what I was looking for :
const messageSchema = Joi.object().keys({
keyA: Joi.string(),
keyB: Joi.string().uri(),
}).or('keyA', 'keyB');