I'm working with the latest mongoose version of today (6.2.7) and I'm having a really weird bug.
This is my Schema:
const testSchema = new Schema<ITestSchema>({
age: Number
}, { timestamps: true });
const testModel = model<ITestSchema>("test", testSchema);
When I'm creating new collections out of it everything is working perfect! and I'm getting timestamps (updatedAt and createdAt) added to the collection.
But When I'm working with sessions the timestamps are not added and i see only "age", "_d" and "__v".
This is the example code for the creation with the sessions:
const test = async () => {
const session: ClientSession = await mongoose.startSession();
try {
session.startTransaction();
const newTest = new testModel({
age: 30,
}, { session });
await newTest.save({ session });
await session.commitTransaction();
} catch (error) {
await session.abortTransaction();
throw error;
} finally {
await session.endSession();
}
};
I tried read the doc several times and searched for similar issues online but could not find any.
Thanks 3>
i know the question is old but i just came across this and found a solution, you mistake is that you are using session in 2 different places both when creating the model and when saving, the thing is if you use session only on testModel it also won't work, so the session must only be used when calling save function
sample code:
const modelObject = {
'_id' : new Types.ObjectId(),
...dto
}
const newItem = new this.model(modelObject)
return newItem.save({
session
});
Related
Hi I am working in a NestJS project using mongodb and mongoose.
I have to create transactions with a lot of promises inside, so i think it was a good idea to use Promise.all() inside my transaction for performace issues.
Unfortunately when i started working with my transactions i have a first issue, i was using
session.startTransaction(); and my code was throwing the following error:
Given transaction number 2 does not match any in-progress transactions. The active transaction number is 1, the error was thrown sometimes, not always but it was a problem
So i read the following question Mongoose `Promise.all()` Transaction Error, and i started to use withTransaction(), this solved the problem, but now mi code does not work propertly.
the code basically takes an array of bookings and then creates them, also needs to create combos of the bookings, what I need is that if a creation of a booking or a combo fail nothing should be inserted, for perfomance I use Promise.all().
But when i execute the function sometimes it creates more bookings than expected, if bookingsArray is from size 2, some times it creates 3 bookings and i just don't know why, this occurs very rarely but it is a big issue.
If i remove the Promise.all() from the transaction it works perfectly, but without Promise.all() the query is slow, so I wanted to know if there is any error in my code, or if you just cannot use Promise.all() inside a mongodb transaction in Nestjs
Main function with the transaction and Promise.all(), this one sometimes create the wrong number of bookings
async createMultipleBookings(
userId: string,
bookingsArray: CreateBookingDto[],
): Promise<void> {
const session = await this.connection.startSession();
await session.withTransaction(async () => {
const promiseArray = [];
for (let i = 0; i < bookingsArray.length; i++) {
promiseArray.push(
this.bookingRepository.createSingleBooking(
userId,
bookingsArray[i],
session,
),
);
}
promiseArray.push(
this.bookingRepository.createCombosBookings(bookingsArray, session),
);
await Promise.all(promiseArray);
});
session.endSession();
}
Main function with the transaction and withot Promise.all(), works fine but slow
async createMultipleBookings(
userId: string,
bookingsArray: CreateBookingDto[],
): Promise<void> {
const session = await this.connection.startSession();
await session.withTransaction(async () => {
for (let i = 0; i < bookingsArray.length; i++) {
await this.bookingRepository.createSingleBooking(
userId,
bookingsArray[i],
session,
);
}
await this.bookingRepository.createCombosBookings(bookingsArray, session);
});
session.endSession();
}
Functions called inside the main function
async createSingleBooking(
userId: string,
createBookingDto: CreateBookingDto,
session: mongoose.ClientSession | null = null,
) {
const product = await this.productsService.getProductById(
createBookingDto.productId,
session,
);
const user = await this.authService.getUserByIdcustomAttributes(
userId,
['profile', 'name'],
session,
);
const laboratory = await this.laboratoryService.getLaboratoryById(
product.laboratoryId,
session,
);
if (product.state !== State.published)
throw new BadRequestException(
`product ${createBookingDto.productId} is not published`,
);
const bookingTracking = this.createBookingTraking();
const value = product.prices.find(
(price) => price.user === user.profile.role,
);
const bookingPrice: Price = !value
? {
user: user.profile.role,
measure: Measure.valorACotizar,
price: null,
}
: value;
await new this.model({
...createBookingDto,
userId,
canceled: false,
productType: product.productType,
bookingTracking,
bookingPrice,
laboratoryId: product.laboratoryId,
userName: user.name,
productName: product.name,
laboratoryName: laboratory.name,
facultyName: laboratory.faculty,
createdAt: new Date(),
}).save({ session });
await this.productsService.updateProductOutstanding(
createBookingDto.productId,
session,
);
}
async createCombosBookings(
bookingsArray: CreateBookingDto[],
session: mongoose.ClientSession,
): Promise<void> {
const promiseArray = [];
for (let i = 1; i < bookingsArray.length; i++) {
promiseArray.push(
this.combosService.createCombo(
{
productId1: bookingsArray[0].productId,
productId2: bookingsArray[i].productId,
},
session,
),
);
}
await Promise.all(promiseArray);
}
also this is how i create the connection element:
export class BookingService {
constructor(
#InjectModel(Booking.name) private readonly model: Model<BookingDocument>,
private readonly authService: AuthService,
private readonly bookingRepository: BookingRepository,
#InjectConnection()
private readonly connection: mongoose.Connection,
) {}
I am trying to create multiple objects with mongodb, mongoose and express (by using Insomnia). I have managed to create the first object, but when I try to create the following one it gives me the following error:
{
"success": false,
"message": {
"driver": true,
"name": "MongoError",
"index": 0,
"code": 11000
}
}
I tried to solve it by including different data in the object (although I didn't specify uniqueness in the properties), but it will throw the 11000 (duplicate key) error anyways.
The log of req.body returns { inmovilizadoInmaterial: 12, inmovilizadoMaterial: 13 } (as the data included for the registration are those values)
Here's the model
const mongoose = require("mongoose");
const { Schema } = mongoose;
const balanceSchema = new Schema(
{
inmovilizadoInmaterial: { type: Number, default: 0 },
inmovilizadoMaterial: { type: Number, default: 0 },
}
);
module.exports = mongoose.model("Balance", balanceSchema);
Here's the router that connects the model and the controller the object:
const express = require('express');
const router = express.Router();
const balanceController = require('../controllers/balance.controller');
router.post('/create', balanceController.create);
module.exports = router;
And finally here's the controller that has the function that creates the object.
const balanceController = {};
const Balance = require('../models/Balance')
balanceController.create = async(req,res)=>{
const balance = new Balance(req.body);
balance.save();
}
module.exports = balanceController;
I know it must be a very simple mistake but I'm very new to the technology.
Thank you very much in advance!
Can you try to change your controller like this:
balanceController.create = async(req,res)=>{
try {
console.log('Req body:', req.body);
const balance = await Balance.create(req.body);
res.status(200).json({success: true});
} catch(error) {
console.log('ERROR: ', error);
res.status(400).json({success: false, error});
}
}
Note that req.body will be logged so you can check whether your controller receiving correct data (maybe you are sending empty req.body from the frontend). Also error will be logged in. If this does not work, add both output of console.log(req.body) and console.log(error) to your question.
I have connected MongoDB to my discord.js code and have made a setwelcome command as per-server data so that each server can customize their own welcome message. Everything works great, I just want to know if there is any way that I can make the message appear as an embed? Here's the code:
//importing all the needed files and languages
const mongo = require('./mongo')
const command = require('./command')
const welcomeSchema = require('./schemas/welcome-schema')
const mongoose = require('mongoose')
const Discord = require('discord.js')
mongoose.set('useFindAndModify', false);
//my code is inside this export
module.exports = (client) => {
//this next line is for later
const cache = {}
command(client, 'setwelcome', async (message) => {
const { member, channel, content, guild } = message
//checking to see that only admins can do this
if (!member.hasPermissions === 'ADMINISTRATOR') {
channel.send('You do not have the permission to run this command')
return
}
//simplifying commands
let text = content
//this is to store just the command and not the prefix in mongo compass
const split = text.split(' ')
if (split.length < 2) {
channel.send('Please provide a welcome message!')
return
}
split.shift()
text = split.join(' ')
//this is to not fetch from the database after code ran once
cache[guild.id] = [channel.id, text]
//this is to store the code inside mongo compass
await mongo().then(async (mongoose) => {
try {
await welcomeSchema.findOneAndUpdate({
_id: guild.id
}, {
_id: guild.id,
channelId: channel.id,
text,
}, {
upsert: true
})
} finally {
mongoose.connection.close()
}
})
})
//this is to fetch from the database
const onJoin = async (member) => {
const { guild } = member
let data = cache[guild.id]
if (!data) {
console.log('FETCHING FROM DATABASE')
await mongo().then( async (mongoose) => {
try {
const result = await welcomeSchema.findOne({ _id: guild.id })
cache[guild.id] = data = [result.channelId, result.text]
} finally {
mongoose.connection.close()
}
})
}
//this is to simplify into variables
const channelId = data[0]
const text = data[1]
/*this is where the message sends on discord. the second of these 2 lines is what I want embedded
which is basically the welcome message itself*/
const channel = guild.channels.cache.get(channelId)
channel.send(text.replace(/<#>/g, `<#${member.id}>`))
}
//this is to test the command
command(client, 'simjoin', message => {
onJoin(message.member)
})
//this is so the command works when someone joins
client.on('guildMemberAdd', member => {
onJoin(member)
})
}
I know how to usually make an embed, but I'm just confused at the moment on what to put as .setDescription() for the embed.
Please advise.
If you just want to have the message be sent as an embed, create a MessageEmbed and use setDescription() with the description as the only argument. Then send it with channel.send(embed).
const embed = new Discord.MessageEmbed();
embed.setDescription(text.replace(/<#>/g, `<#${member.id}>`));
channel.send(embed);
By the way, if you are confused about how to use a specific method you can always search for the method name on the official discord.js documentation so you don’t have to wait for an answer here. Good luck creating your bot!
the following function is supposed to do:
save the project into collection 'project_list'
push the uuid of the new project to a client in the collection 'client_list'
with transaction enabled, if anything goes wrong (such as the client id is given in the new project JSON object does not exist in the collection 'client_list'), then both step1 and step2 will be canceled.
noticed that the code is giving the client_id as 'invalid-uuid' to updateOne which will not add the project uuid to the client because the client_id does not exist in the collection but the new project is saved into collection 'project_list' with or without me doing commitTransaction.
I was reading this https://mongoosejs.com/docs/transactions.html which suggest me to abort the transaction by calling await session.abortTransaction(); but i dont know how to implement it.
thanks in advance.
transactionCreateProject: async function (newProjectJson) {
const mongoConnectConfig = {useNewUrlParser: true, useUnifiedTopology: true};
await mongoose.connect(CONSTANTS.ADMIN_MONGO_URL, mongoConnectConfig)
const db = mongoose.connection;
db.on('error', console.error.bind(console, 'MongoDB connection error：'))
db.once("open", function (callback) {
console.log("Connection Succeeded")
});
let project = mongoose.model('project', mongooseProjectSchema)
let client = mongoose.model('client', mongooseClientSchema)
const session = await mongoose.startSession();
session.startTransaction();
try {
console.log(newProjectJson.clientUUID)
let newProjectMongooseObject = new project()
for (let i = 0; i < Object.keys(newProjectJson).length; i++){
// assigning the attributes of new project to mongoose object
newProjectMongooseObject[Object.keys(newProjectJson)[i]] = newProjectJson[Object.keys(newProjectJson)[i]]
}
newProjectMongooseObject['createdAt'] = dateObject.toISOString();
let saveRes = await newProjectMongooseObject.save()
let addProjectToClientRes = await clientMongooseModel.updateOne({ 'uuid': "invalid-uuid"}, {$push: {'projects': { projectID: newProjectJson.uuid}}})
}
catch (e) {
console.log(e)
return null
}
finally {
console.log("in finally, session ending")
session.endSession();
}
return null
},
I am trying to build a music app and while working on the back end for the app (using express), I am facing this weird issue of documents not saving in mongo collections.
I made a post route to which user submits form data, which contains the song's mp3 file and the name of the song (it will have more data later on).
I am using multer to parse multipart form data.
I am able to save the mp3 file to mongoDB using multer-gridfs-storage. I want to save the song info such as name, artists etc in a different collection and here is the schema for the collection:
import mongoose from 'mongoose';
const Schema = mongoose.Schema;
const SongsInfo = new Schema({
name: {
type: String,
required: true,
},
});
const Song = mongoose.model('Song', SongsInfo);
export default Song;
index.js file:
import Grid from 'gridfs-stream';
import GridFsStorage from 'multer-gridfs-storage';
const app = express();
const conn = mongoose.createConnection(mongoURI);
let gfs;
conn.once('open', () => {
console.log('Connected to mongodb');
gfs = Grid(conn.db, mongoose.mongo);
gfs.collection('songFiles');
});
// storage engine
const storage = new GridFsStorage({
url: mongoURI,
file: (req, file) => new Promise((resolve, reject) => {
crypto.randomBytes(16, (err, buf) => {
if (err) {
return reject(err);
}
const filename = buf.toString('hex') +
path.extname(file.originalname);
let fileInfo;
fileInfo = {
filename,
bucketName: 'songFiles',
};
resolve(fileInfo);
});
}),
});
let upload;
middleWare(app);
app.post('/api/uploadSong', async (req, res) => {
upload = multer({ storage }).any();
upload(req, res, async (err) => {
console.log('in');
if (err) {
// console.log(err);
return res.end('Error uploading file.');
}
const { name } = req.body;
// push a Song into songs collection
const songInfo = new Song({
name,
});
const si = await songInfo.save(); // (*)
console.log(songInfo);
res.json({
songInfo: si,
file: req.file,
});
});
});
On line (*) the server just freezes until the request gets timed out.
No errors shown on console. Don't know what to do :(
I solved the issue finally!
So what i did was bring the models in index.js file and changed up some stuff here and there..
index.js
const app = express();
mongoose.connect(mongoURI); //(*)
const conn = mongoose.connection; // (*)
let gfs;
conn.once('open', () => {
console.log('Connected to mongodb');
gfs = Grid(conn.db, mongoose.mongo);
gfs.collection('songFiles');
});
// models
const Schema = mongoose.Schema; //(***)
const SongsInfo = new Schema({
name: {
type: String,
required: true,
},
});
const Song = mongoose.model('Song', SongsInfo);
// storage engine
const storage = new GridFsStorage({
url: mongoURI,
file: (req, file) => new Promise((resolve, reject) => {
crypto.randomBytes(16, (err, buf) => {
if (err) {
return reject(err);
}
const filename = buf.toString('hex') + path.extname(file.originalname);
let fileInfo;
if (file.mimetype === 'image/jpeg' || file.mimetype === 'image/png') {
fileInfo = {
filename,
bucketName: 'imageFiles',
};
} else {
fileInfo = {
filename,
bucketName: 'songFiles',
};
}
resolve(fileInfo);
});
}),
});
let upload;
middleWare(app);
app.post('/api/uploadSong', async (req, res) => {
upload = multer({ storage }).any();
upload(req, res, async (err) => {
console.log('in');
if (err) {
return res.end('Error uploading file.');
}
const { name } = req.body;
// push a Song into songs collection
const songInfo = new Song({
name,
});
songInfo.save((er, song) => {
if (!er) {
res.send('err');
} else {
console.log(err, song);
res.send('err');
}
});
});
});
At line (***) I used the same instance of mongoose that was initialized.. in the previous file I imported it again from the package...
Kia ora!
For those stumbling here three years on. I came across this issue as well. Abhishek Mehandiratta has the answer hidden in their code snippet.
The Fix
I went from instantiating mongoose with:
connect(connStr)
to doing:
const conn = createConnection(connStr)
This is the breaking change. So for an easy fix, change your code back to using connect(...).
I too, followed the documentation and Youtube tutorials to alter my code in such a way. It's an unfortunate misleading for developers who have not encountered a need to understand the difference.
I changed it back, and now it's working again (even with 'multer-gridfs-storage'). You can reference the connection with:
import {connection} from "mongoose";
connection.once('open', ...)
Why is this happening?
BenSower writes up the differences between connect and createConnection here. So from my basic understanding of BenSower's write up, you're referencing the wrong connection pool when you create your 'Song' schema, and thus, referencing the wrong pool when saving. I guess this results in a time out, have not looked further, but I'm sure a more in-depth answer exists somewhere.
import mongoose from 'mongoose';
const Song = mongoose.model('Song', SongsInfo);
As BenSower states in their answer "For me, the big disadvantage of creating multiple connections in the same module, is the fact that you can not directly access your models through mongoose.model, if they were defined "in" different connections". They're spot on with this one. Having finally overcome this obstacle, I shall have a look into createConnection on another project. For now, good ol' connect() will fix this issue, and do just fine :D