Why is mongoose not allowing me to find by $in on _id unless I have a schema wrapping another schema? - mongodb

Please, if anyone can help explain the below situation. I've lost hours only to discover a working solution but it doesn't make sense why it needs to be implemented this way.
For some reason if I connect mongo over nodejs, using 1 mongoose schema, I cannot search via $in.
When I wrap that single schema in another schema, with no additional information $in returns a match.
Note: the command line works always. So it is something to do with the way the schema needs to be impelemnted.
I am new to mongoose so maybe there is something obvious i am missing but it does not make any sense. Any help appreciated.
thanks
creation of data in docker
Note there is only one record:
#!/bin/bash
set -e
mongo <<EOF
use ${MONGO_DB_NAME}
db.createCollection("persons")
db.persons.insert({"name": "CPU"})
EOF
schema.ts
import { Model, Schema, Document, Connection } from 'mongoose';
interface IPerson {
_id: string;
name: string;
};
type TSchemaPerson = Schema<IPerson>;
const SchemaPerson: TSchemaPerson = new Schema({
_id: String,
name: String
});
// ##########################################################################
// ODD BEHAVIOUR ?
// I have to wrap SchemaPerson inside SchemaPersons,
// even though its basically a duplication in order to get $in to return a result
// --------------------------------------------------------
// \/
const SchemaPersons: Schema<TSchemaPerson> = new Schema({SchemaPerson});
// ##########################################################################
const collectionName: string = 'Person';
export type TDocumentPerson = Model<Document & IPerson>;
export type TConnPersons = (con: Connection) => TDocumentPerson;
// ##########################################################################
// ODD BEHAVIOUR ?
// If I return SchemaPerson instead of SchemaPersons here then $in does not return a result --------
// \/
const ModelPersons: TConnPersons = con => con.model(collectionName, SchemaPersons);
// ##########################################################################
export default ModelPersons;
graphql - resolver.ts
const persons: QueryResolvers['persons'] = async (parent, args, context) => {
console.log('persons()');
const { dbConn }: IContext = context;
if (dbConn) {
const Person: TDocumentPerson = ModelPersons(dbConn);
let result;
let resultIn;
try {
result = await Person.find().exec();
console.log('result = ', result); // Result in both cases = [ { _id: 5f36e199ebd39f53b437834b, name: 'CPU' } ]
resultIn = await Person.find({
_id: {
$in: [new ObjectId('5f36e199ebd39f53b437834b')]
}
}).exec();
// ##########################################################################
// ODD BEHAVIOUR ?
// Result with SchemaPersons = [ { _id: 5f36e199ebd39f53b437834b, name: 'CPU' } ]
// Result with SchemaPerson = [ ]
console.log('resultIn = ', resultIn);
// ##########################################################################
} catch (err) {
console.log('err = ', err);
}
}
};
Note if I change the model to a collection then it does work. See below:
schema.ts - connecting as collection
import { Schema, Collection, Connection } from 'mongoose';
interface IPerson {
_id: string;
name: string;
};
type TSchemaPerson = Schema<IPerson>;
const SchemaPerson: TSchemaPerson = new Schema({
_id: String,
name: String
});
export type TCollectionPersons = Collection & IPerson;
export type TConnPersons = (con: Connection) => TCollectionPersons;
const CollectionPersons: TConnPersons = con => con.collection('persons', SchemaPerson)
export default CollectionPersons;
resolver.ts
const persons: TCollectionPersons = CollectionPersons(dbConn);
let result;
let resultIn;
let resultById
try {
result = await persons.find().toArray();
console.log('result = ', result);
// Result [ { _id: 5f36e199ebd39f53b437834b, name: 'CPU' } ]
resultIn = await persons.find({
_id: {
$in: [new ObjectId('5f36e199ebd39f53b437834b')]
}
}).toArray();
console.log('resultIn = ', resultIn);
// Result [ { _id: 5f36e199ebd39f53b437834b, name: 'CPU' } ]
// cannot use findById on collection.
// resultById = await persons.findById('5f36e199ebd39f53b437834b');
} catch (err) {
console.log('err = ', err);
}
}

Related

Mongoose Schema properties validation with Typescript NextJS

i am trying to save new document to mongo db, the Schema validation is not working for me, i am trying ti make required true, but i still can add new document without the required field.
this is my schema:
// lib/models/test.model.ts
import { Model, Schema } from 'mongoose';
import createModel from '../createModel';
interface ITest {
first_name: string;
last_name: string;
}
type TestModel = Model<ITest, {}>;
const testSchema = new Schema<ITest, TestModel>({
first_name: {
type: String,
required: [true, 'Required first name'],
},
last_name: {
type: String,
required: true,
},
});
const Test = createModel<ITest, TestModel>('tests', testSchema);
module.exports = Test;
this is createModel:
// lib/createModel.ts
import { Model, model, Schema } from 'mongoose';
// Simple Generic Function for reusability
// Feel free to modify however you like
export default function createModel<T, TModel = Model<T>>(
modelName: string,
schema: Schema<T>
): TModel {
let createdModel: TModel;
if (process.env.NODE_ENV === 'development') {
// In development mode, use a global variable so that the value
// is preserved across module reloads caused by HMR (Hot Module Replacement).
// #ts-ignore
if (!global[modelName]) {
createdModel = model<T, TModel>(modelName, schema);
// #ts-ignore
global[modelName] = createdModel;
}
// #ts-ignore
createdModel = global[modelName];
} else {
// In production mode, it's best to not use a global variable.
createdModel = model<T, TModel>(modelName, schema);
}
return createdModel;
}
and this is my tests file:
import { connection } from 'mongoose';
import type { NextApiRequest, NextApiResponse } from 'next';
const Test = require('../../../lib/models/test.model');
import { connect } from '../../../lib/dbConnect';
const ObjectId = require('mongodb').ObjectId;
export default async function handler(
req: NextApiRequest,
res: NextApiResponse
) {
switch (req.method) {
case 'POST': {
return addPost(req, res);
}
}
}
async function addPost(req: NextApiRequest, res: NextApiResponse) {
try {
connect();
// const { first_name, last_name } = req.body;
const test = new Test({
first_name: req.body.first_name,
last_name: req.body.last_name,
});
let post = await test.save();
// return the posts
return res.json({
message: JSON.parse(JSON.stringify(post)),
success: true,
});
// Erase test data after use
//connection.db.dropCollection(testModel.collection.collectionName);
} catch (err) {
//res.status(400).json(err);
res.status(400).json({
message: err,
success: false,
});
}
}
in the Postman, i send a request body without the required field (first_name) and i still can add it.
any help?

How can I increment a counter variable in LoopBack 4 with a MongoDB datasource?

I'm trying to convert my Nodejs Express app to Loopback 4 and I can't figure out how to increment a counter. In my Angular 9 app when a user clicks an icon a counter is incremented. This works perfectly in Express
In Express
const updateIconCount = async function (dataset, collection = 'icons') {
let query = { _id: new ObjectId(dataset.id), userId: dataset.userId };
return await mongoController.update(
collection,
query,
{ $inc: { counter: 1 } },
function (err, res) {
logAccess(res, 'debug', true, 'function update updateIconLink');
if (err) {
return false;
} else {
return true;
}
}
);
};
I tried to first get the value of counter and then increment but every time I save VS Code reformats the code in an an unusual way. In this snippet I commented out the line of code that causes this reformatting. I can set the counter value, e.g. 100.
In Loopback 4
#patch('/icons/count/{id}', {
responses: {
'204': {
description: 'Icons PATCH success',
},
},
})
async incrementCountById(
#param.path.string('id') id: string,
#requestBody({
content: {
'application/json': {
schema: getModelSchemaRef(Icons, {partial: true}),
},
},
})
icons: Icons,
): Promise<void> {
// let targetIcon = this.findById(id).then(icon => {return icon});
icons.counter = 100;
console.log(icons.counter);
await this.iconsRepository.updateById(id, icons);
}
How do I implement { $inc: { counter: 1 } } in Loopback 4?
Added to aid solution
My mongo.datasource.ts
import {inject, lifeCycleObserver, LifeCycleObserver} from '#loopback/core';
import {juggler} from '#loopback/repository';
const config = {
name: 'mongo',
connector: 'mongodb',
url: '',
host: '192.168.253.53',
port: 32813,
user: '',
password: '',
database: 'firstgame',
useNewUrlParser: true,
allowExtendedOperators: true,
};
// Observe application's life cycle to disconnect the datasource when
// application is stopped. This allows the application to be shut down
// gracefully. The `stop()` method is inherited from `juggler.DataSource`.
// Learn more at https://loopback.io/doc/en/lb4/Life-cycle.html
#lifeCycleObserver('datasource')
export class MongoDataSource extends juggler.DataSource
implements LifeCycleObserver {
static dataSourceName = 'mongo';
static readonly defaultConfig = config;
constructor(
#inject('datasources.config.mongo', {optional: true})
dsConfig: object = config,
) {
super(dsConfig);
}
}
Amended endpoint
#patch('/icons/count/{id}', {
responses: {
'204': {
description: 'Icons PATCH success',
},
},
})
async incrementCountById(
#param.path.string('id') id: string,
#requestBody({
content: {
'application/json': {
schema: getModelSchemaRef(Icons, {partial: true}),
},
},
})
icons: Icons,
): Promise<void> {
console.log(id);
// #ts-ignore
await this.iconsRepository.updateById(id, {$inc: {counter: 1}});//this line fails
// icons.counter = 101; //these lines will set the icon counter to 101 so I know it is connecting to the mongodb
// await this.iconsRepository.updateById(id, icons);
}
You can use the mongo update-operators.
Basically, you just have to set allowExtendedOperators=true at your MongoDB datasource definition (guide). After that, you can directly use these operators.
Usage example:
// increment icon.counter by 3
await this.iconsRepository.updateById(id, {$inc: {counter: 3}} as Partial<Counter>);
Currently, these operators are missing from the lb4 types so you must cheat typescript to accept them. It's ugly but that's the only solution I could find right now.
You can follow this issue to see what's going on with these operators.

MongoDB query - pass the function to the Model.find()

I have issue with querying MongoDB (Mongoose) by passing the function as parameter in Model.find() -> like this Model.find(searchCondition). I hope that you can help me.
// Fetching patients from the database
exports.getPatients = (req, res, next) => {
const criterionSearchCategory = req.query.kriterijumPretrage;
const ageSearchCategory = req.query.kriterijumGodina;
const searchInputValue = req.query.pojamPretrage;
console.log({ [criterionSearchCategory]: { [ageSearchCategory]: Number([searchInputValue]) }});
// Patient search condition, based on selected option from select dropdown
function searchCondition() {
if (criterionSearchCategory == 'undefined') {
return {};
} else if (criterionSearchCategory == 'age') {
return { [criterionSearchCategory]: { [ageSearchCategory] : Number([searchInputValue]) }}
} else {
return { [criterionSearchCategory]: { $in: [ "/^" + searchInputValue + "/i" ]}}
}
}
...
const patientQuery = Patient.find(searchCondition);
getPatients(patientsPerPage: number, currentPage: number, criterionSearchCategory: string, searchInputValue: string, ageSearchCategory: any) {
const queryParams = `?pacijenataPoStranici=${patientsPerPage}&trenutnaStranica=${currentPage}&kriterijumPretrage=${criterionSearchCategory}&pojamPretrage=${searchInputValue}&kriterijumGodina=${ageSearchCategory}`;
this.http
.get<{ message: string, patients: any, maxPatients: number }>( BACKEND_URL + queryParams)
// Execute map on every data that makes it through Observable stream
.pipe(map((patientData) => {
I want to menton when I pass the query params manually, for example const patientQuery = Patient.find({ age: { '$gt': 30 } }); appropriate patients will be fetched correctly , but when I pass the function , like this const patientQuery = Patient.find(searchCondition); then does not work.
The first question, is it possible to pass the function as parameter like this?
Any suggestion will be appreciate. Thank you

feathers-mongodb Service.find({query: {_id}}) returns null

I have the schemas below:
students.graphql.schema.js
export default [
`
type StudentsWithPagination {
total: Int
items: [Students]
}
type Students {
_id: String!
name: String
address: Addresses
}
`,
];
addresses.graphql.schema.js
export default [
`
type AddressesWithPagination {
total: Int
items: [Addresses]
}
type Addresses {
_id: String!
title: String
}
`,
];
I have created two services by running feathers generate service students.service.js and addresses.services.js.
When I search addresses by title, I get result. However, when I search by _id, I get null. Something like:
const studentsResolvers = {
Students: {
address: student => {
const query = {
_id: student.address
}
return Addresses.find({ query }).then(result => {
console.log(result)
})
}
}
}
The code above produces null though student.address returns the right address._id. I still get null even I hardcode student.address with the right address._id
The code above will return null unless I search by address title. Something like:
const query = {
title: 'my-location'
}
_id is of type String, not ObjectID.
What am I doing wrong?
As documented in the feathers-mongodb adapter, since MongoDB itself (unlike Mongoose) does not have a schema, all query parameters have to be converted to the type in the database in a hook manually. The example can be adapted accordingly for $in queries:
const ObjectID = require('mongodb').ObjectID;
app.service('users').hooks({
before: {
find(context) {
const { query = {} } = context.params;
if(query._id) {
query._id = new ObjectID(query._id);
}
if(query.age !== undefined) {
query.age = parseInt(query.age, 10);
}
context.params.query = query;
return Promise.resolve(context);
}
}
});

apollostack/graphql-server - how to get the fields requested in a query from resolver

I am trying to figure out a clean way to work with queries and mongdb projections so I don't have to retrieve excessive information from the database.
So assuming I have:
// the query
type Query {
getUserByEmail(email: String!): User
}
And I have a User with an email and a username, to keep things simple. If I send a query and I only want to retrieve the email, I can do the following:
query { getUserByEmail(email: "test#test.com") { email } }
But in the resolver, my DB query still retrieves both username and email, but only one of those is passed back by apollo server as the query result.
I only want the DB to retrieve what the query asks for:
// the resolver
getUserByEmail(root, args, context, info) {
// check what fields the query requested
// create a projection to only request those fields
return db.collection('users').findOne({ email: args.email }, { /* projection */ });
}
Of course the problem is, getting information on what the client is requesting isn't so straightforward.
Assuming I pass in request as context - I considered using context.payload (hapi.js), which has the query string, and searching it through various .split()s, but that feels kind of dirty. As far as I can tell, info.fieldASTs[0].selectionSet.selections has the list of fields, and I could check for it's existence in there. I'm not sure how reliable this is. Especially when I start using more complex queries.
Is there a simpler way?
In case you don't use mongDB, a projection is an additional argument you pass in telling it explicitly what to retrieve:
// telling mongoDB to not retrieve _id
db.collection('users').findOne({ email: 'test#test.com' }, { _id: 0 })
As always, thanks to the amazing community.
2020-Jan answer
The current answer to getting the fields requested in a GraphQL query, is to use the graphql-parse-resolve-info library for parsing the info parameter.
The library is "a pretty complete solution and is actually used under the hood by postgraphile", and is recommended going forward by the author of the other top library for parsing the info field, graphql-fields.
Use graphql-fields
Apollo server example
const rootSchema = [`
type Person {
id: String!
name: String!
email: String!
picture: String!
type: Int!
status: Int!
createdAt: Float
updatedAt: Float
}
schema {
query: Query
mutation: Mutation
}
`];
const rootResolvers = {
Query: {
users(root, args, context, info) {
const topLevelFields = Object.keys(graphqlFields(info));
return fetch(`/api/user?fields=${topLevelFields.join(',')}`);
}
}
};
const schema = [...rootSchema];
const resolvers = Object.assign({}, rootResolvers);
// Create schema
const executableSchema = makeExecutableSchema({
typeDefs: schema,
resolvers,
});
Sure you can. This is actually the same functionality that is implemented on join-monster package for SQL based db's. There's a talk by their creator: https://www.youtube.com/watch?v=Y7AdMIuXOgs
Take a look on their info analysing code to get you started - https://github.com/stems/join-monster/blob/master/src/queryASTToSqlAST.js#L6-L30
Would love to see a projection-monster package for us mongo users :)
UPDATE:
There is a package that creates a projection object from info on npm: https://www.npmjs.com/package/graphql-mongodb-projection
You can generate MongoDB projection from info argument. Here is the sample code that you can follow
/**
* #description - Gets MongoDB projection from graphql query
*
* #return { object }
* #param { object } info
* #param { model } model - MongoDB model for referencing
*/
function getDBProjection(info, model) {
const {
schema: { obj }
} = model;
const keys = Object.keys(obj);
const projection = {};
const { selections } = info.fieldNodes[0].selectionSet;
for (let i = 0; i < keys.length; i++) {
const key = keys[i];
const isSelected = selections.some(
selection => selection.name.value === key
);
projection[key] = isSelected;
}
console.log(projection);
}
module.exports = getDBProjection;
With a few helper functions you can use it like this (typescript version):
import { parceGqlInfo, query } from "#backend";
import { GraphQLResolveInfo } from "graphql";
export const user = async (parent: unknown, args: unknown, ctx: unknown, info: GraphQLResolveInfo): Promise<User | null> => {
const { dbQueryStr } = parceGqlInfo(info, userFields, "id");
const [user] = await query(`SELECT ${dbQueryStr} FROM users WHERE id=$1;`, [1]);
return user;
};
Helper functions.
Few points:
gql_uid used as ID! string type from primary key to not change db types
required option is used for dataloaders (if field was not requested by user)
allowedFields used to filter additional fields from info like '__typename'
queryPrefix is used if you need to prefix selected fields like select u.id from users u
const userFields = [
"gql_uid",
"id",
"email"
]
// merge arrays and delete duplicates
export const mergeDedupe = <T>(arr: any[][]): T => {
// #ts-ignore
return ([...new Set([].concat(...arr))] as unknown) as T;
};
import { parse, simplify, ResolveTree } from "graphql-parse-resolve-info";
import { GraphQLResolveInfo } from "graphql";
export const getQueryFieldsFromInfo = <Required = string>(info: GraphQLResolveInfo, options: { required?: Required[] } = {}): string[] => {
const { fields } = simplify(parse(info) as ResolveTree, info.returnType) as { fields: { [key: string]: { name: string } } };
let astFields = Object.entries(fields).map(([, v]) => v.name);
if (options.required) {
astFields = mergeDedupe([astFields, options.required]);
}
return astFields;
};
export const onlyAllowedFields = <T extends string | number>(raw: T[] | readonly T[], allowed: T[] | readonly T[]): T[] => {
return allowed.filter((f) => raw.includes(f));
};
export const parceGqlInfo = (
info: GraphQLResolveInfo,
allowedFields: string[] | readonly string[],
gqlUidDbAlliasField: string,
options: { required?: string[]; queryPrefix?: string } = {}
): { pureDbFields: string[]; gqlUidRequested: boolean; dbQueryStr: string } => {
const fieldsWithGqlUid = onlyAllowedFields(getQueryFieldsFromInfo(info, options), allowedFields);
return {
pureDbFields: fieldsWithGqlUid.filter((i) => i !== "gql_uid"),
gqlUidRequested: fieldsWithGqlUid.includes("gql_uid"),
dbQueryStr: fieldsWithGqlUid
.map((f) => {
const dbQueryStrField = f === "gql_uid" ? `${gqlUidDbAlliasField}::Text AS gql_uid` : f;
return options.queryPrefix ? `${options.queryPrefix}.${dbQueryStrField}` : dbQueryStrField;
})
.join(),
};
};