I'm looking at upgrading my projects mongoose version from 5.10.19 to latest (6.5.1). I'm noticing that I have a lot more collections in my database than I did before. I made a simple example to test this out and when I run it on mongoose 5, I only see the collection "mains" but mongoose 6 creates "mains" and "subs". I'd expect that the subdocument models would not have their own collection like mongoose 5 behaves.
import { connect, model, Schema } from 'mongoose';
const mongoUrl = 'mongodb://localhost:27017/test';
(async () => {
const subSchema: Schema = new Schema({ color: String, yes: Boolean });
const mainSchema: Schema = new Schema({ name: String, sub: subSchema });
const MainModel = model('Main', mainSchema);
model('Sub', subSchema);
await connect(mongoUrl, { ssl: true, sslValidate: false });
console.log(`Successfully connected to mongodb: "${mongoUrl}"`);
await MainModel.create({ name: 'One', sub: { color: 'Yellow', yes: true } });
})()
.then(() => {
console.log('\nSuccess');
process.exit();
})
.catch(() => {
console.log('\nFailure');
process.exit();
});
Is there a mongoose setting I'm missing that's causing this to happen?
Also, on Node 12.20.12.
It's probably because of this change:
autoCreate is true by default unless readPreference is secondary or secondaryPreferred, which means Mongoose will attempt to create every model's underlying collection before creating indexes.
So turning off autoCreate (and possibly autoIndex) on subSchema should fix it. Alternatively, just don't create a model of it (only the schema).
In my current express application I want to use the new ability of mongodb multi-doc transactions.
First of all it is important to point out how I connect and handle the models
My app.js (server) firstly connects to the db by using db.connect().
I require all models in my db.index file. Since the models will be initiated with the same mongoose reference, I assume that future requires of the models in different routes point to the connected and same connection. Please correct me if I'm wrong with any of these assumptions.
I save the connection reference inside the state object and returning it when needed for my transaction later
./db/index.ts
const fs = require('fs');
const path = require('path');
const mongoose = require('mongoose');
const state = {
connection = null,
}
// require all models
const modelFiles = fs.readdirSync(path.join(__dirname, 'models'));
modelFiles
.filter(fn => fn.endsWith('.js') && fn !== 'index.js')
.forEach(fn => require(path.join(__dirname, 'models', fn)));
const connect = async () => {
state.connection = await mongoose.connect(.....);
return;
}
const get = () => state.connection;
module.exports = {
connect,
get,
}
my model files are containing my required schemas
./db/models/example.model.ts
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const ExampleSchema = new Schema({...);
const ExampleModel = mongoose.model('Example', ExampleSchema);
module.exports = ExampleModel;
Now the route where I try to do a basic transaction. F
./routes/item.route.ts
const ExampleModel = require('../db/models/example.model');
router.post('/changeQty', async (req,res,next) => {
const connection = db.get().connection;
const session = await connection.startSession(); // works fine
// start a transaction
session.startTransaction(); // also fine
const {someData} = req.body.data;
try{
// jsut looping that data and preparing the promises
let promiseArr = [];
someData.forEach(data => {
// !!! THIS TRHOWS ERROR !!!
let p = ExampleModel.findOneAndUpdate(
{_id : data.id},
{$incr : {qty : data.qty}},
{new : true, runValidators : true}
).session(session).exec();
promiseArr.push(p);
})
// running the promises parallel
await Promise.all(promiseArr);
await session.commitTransaction();
return res.status(..)....;
}catch(err){
await session.abortTransaction();
// MongoError : Given transaction number 1 does not match any in-progress transactions.
return res.status(500).json({err : err});
}finally{
session.endSession();
}
})
But I always get the following error, which probably has to do sth with the connection reference of my models. I assume, that they don't have access to the connection which started the session, so they are not aware of the session.
MongoError: Given transaction number 1 does not match any in-progress
transactions.
Maybe I somehow need to initiate the models inside db.connect with the direct connection reference ?
There is a big mistake somewhere and I hope you can lead me to the correct path. I appreciate Any help, Thanks in advance
This is because you're doing operations in parallel:
So you've got a bunch of race conditions. Just use async/await
and make your life easier.
let p = await ExampleModel.findOneAndUpdate(
{_id : data.id},
{$incr : {qty : data.qty}},
{new : true, runValidators : true}
).session(session).exec();
Reference : https://github.com/Automattic/mongoose/issues/7311
If that does not work try to execute promises one by one rather than promise.all().
I am trying to make db.collection.renameCollection with mongoose, but i can't find that function anywhere. Did they miss to add it or i am looking at wrong place?
As quick example of what i am doing is:
var conn = mongoose.createConnection('localhost',"dbname");
var Collection = conn.model(collectionName, Schema, collectionName);
console.log(typeof Collection.renameCollection);
Which show undefined.
var con = mongoose.createConnection('localhost',"dbname");
con.once('open', function() {
console.log(typeof con.db[obj.name]);
});
This give also undefined.
Here's an example that will perform a rename operation using Mongoose.
const mongoose = require('mongoose');
mongoose.Promise = Promise;
mongoose.connect('mongodb://localhost/test').then(() => {
console.log('connected');
// Access the underlying database object provided by the MongoDB driver.
let db = mongoose.connection.db;
// Rename the `test` collection to `foobar`
return db.collection('test').rename('foobar');
}).then(() => {
console.log('rename successful');
}).catch(e => {
console.log('rename failed:', e.message);
}).then(() => {
console.log('disconnecting');
mongoose.disconnect();
});
As you can see, the MongoDB driver exposes the renameCollection() method as rename(), which is documented here: http://mongodb.github.io/node-mongodb-native/2.2/api/Collection.html#rename
I have mongoosastic setup within a MEAN stack program. Everything works correctly except when I delete a document from mongodb it is not deleted in the elasticsearch index. So every time I do a search that includes delete items, the deleted item is returned but is null when it is hydrated. Does mongoosastic handle deleting from the ES index? Do I have to program an index refresh?
var mongoose = require('mongoose');
var mongoosastic = require("mongoosastic");
var Schema = mongoose.Schema;
var quantumSchema = new mongoose.Schema({
note: {
type: String,
require: true,
es_indexed: true
}
});
quantumSchema.plugin(mongoosastic);
var Quantum = mongoose.model('Quantum', quantumSchema);
Quantum.createMapping(function(err, mapping){
if(err){
console.log('error creating mapping (you can safely ignore this)');
console.log(err);
}else{
console.log('mapping created!');
console.log(mapping);
}
});
I had the same error. If you look in the Documentation it states that you have to explicit remove the document after deleting it.
This is the way i am doing a deletion now.
const deleteOne = Model => async (id)=> {
const document = await Model.findByIdAndDelete(id);
if (!document) {
return new Result()
.setSuccess(false)
.setError('Unable to delete Entity with ID: ' + id + '.')
}
//this ensures the deletion from the elasticsearch index
document.remove();
return new Result()
.setSuccess(true)
.setData(document)
}
I dont know what version of mongoosastic you're using but i use mongoosastic#3.6.0 and my indexed doc get deleted whenever i remove it either using Model.findByIdAndRemove or Model.remove. Therefore try to cross check the way you delete you're docs.
I solved the problem by changing the way I delete the data.
I was using:
Quantum.findByIdAndRemove(quantumid)
I switched it to:
Quantum.findById(quantumid, function(err, quantum) {
quantum.remove(function(err, quantum) {
if (err) {
console.log(err);
return;
}
});
});
I did not research the reason for this working, but it solved the problem and I moved on.
I have a large collection of 300 question objects in a database test. I can interact with this collection easily through MongoDB's interactive shell; however, when I try to get the collection through Mongoose in an express.js application I get an empty array.
My question is, how can I access this already existing dataset instead of recreating it in express? Here's some code:
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
mongoose.connect('mongodb://localhost/test');
mongoose.model('question', new Schema({ url: String, text: String, id: Number }));
var questions = mongoose.model('question');
questions.find({}, function(err, data) { console.log(err, data, data.length); });
This outputs:
null [] 0
Mongoose added the ability to specify the collection name under the schema, or as the third argument when declaring the model. Otherwise it will use the pluralized version given by the name you map to the model.
Try something like the following, either schema-mapped:
new Schema({ url: String, text: String, id: Number},
{ collection : 'question' }); // collection name
or model mapped:
mongoose.model('Question',
new Schema({ url: String, text: String, id: Number}),
'question'); // collection name
Here's an abstraction of Will Nathan's answer if anyone just wants an easy copy-paste add-in function:
function find (name, query, cb) {
mongoose.connection.db.collection(name, function (err, collection) {
collection.find(query).toArray(cb);
});
}
simply do find(collection_name, query, callback); to be given the result.
for example, if I have a document { a : 1 } in a collection 'foo' and I want to list its properties, I do this:
find('foo', {a : 1}, function (err, docs) {
console.dir(docs);
});
//output: [ { _id: 4e22118fb83406f66a159da5, a: 1 } ]
You can do something like this, than you you'll access the native mongodb functions inside mongoose:
var mongoose = require("mongoose");
mongoose.connect('mongodb://localhost/local');
var connection = mongoose.connection;
connection.on('error', console.error.bind(console, 'connection error:'));
connection.once('open', function () {
connection.db.collection("YourCollectionName", function(err, collection){
collection.find({}).toArray(function(err, data){
console.log(data); // it will print your collection data
})
});
});
Update 2022
If you get an MongoInvalidArgumentError: The callback form of this helper has been removed. error message, here's the new syntax using async/await:
const mongoose = require("mongoose");
mongoose.connect('mongodb://localhost/productsDB');
const connection = mongoose.connection;
connection.on('error', console.error.bind(console, 'connection error:'));
connection.once('open', async function () {
const collection = connection.db.collection("Products");
collection.find({}).toArray(function(err, data){
console.log(data); // it will print your collection data
});
});
I had the same problem and was able to run a schema-less query using an existing Mongoose connection with the code below. I've added a simple constraint 'a=b' to show where you would add such a constraint:
var action = function (err, collection) {
// Locate all the entries using find
collection.find({'a':'b'}).toArray(function(err, results) {
/* whatever you want to do with the results in node such as the following
res.render('home', {
'title': 'MyTitle',
'data': results
});
*/
});
};
mongoose.connection.db.collection('question', action);
Are you sure you've connected to the db? (I ask because I don't see a port specified)
try:
mongoose.connection.on("open", function(){
console.log("mongodb is connected!!");
});
Also, you can do a "show collections" in mongo shell to see the collections within your db - maybe try adding a record via mongoose and see where it ends up?
From the look of your connection string, you should see the record in the "test" db.
Hope it helps!
Something else that was not obvious, to me at least, was that the when using Mongoose's third parameter to avoid replacing the actual collection with a new one with the same name, the new Schema(...) is actually only a placeholder, and doesn't interfere with the exisitng schema so
var User = mongoose.model('User', new Schema({ url: String, text: String, id: Number}, { collection : 'users' })); // collection name;
User.find({}, function(err, data) { console.log(err, data, data.length);});
works fine and returns all fields - even if the actual (remote) Schema contains none of these fields. Mongoose will still want it as new Schema(...), and a variable almost certainly won't hack it.
Go to MongoDB website, Login > Connect > Connect Application > Copy > Paste in 'database_url' > Collections > Copy/Paste in 'collection' .
var mongoose = require("mongoose");
mongoose.connect(' database_url ');
var conn = mongoose.connection;
conn.on('error', console.error.bind(console, 'connection error:'));
conn.once('open', function () {
conn.db.collection(" collection ", function(err, collection){
collection.find({}).toArray(function(err, data){
console.log(data); // data printed in console
})
});
});
I tried all the answers but nothing worked out, finally got the answer hoe to do it.
var mongoose = require('mongoose');
mongoose.connect('mongodb://0.0.0.0:27017/local');
// let model = require('./test1');
setTimeout(async () => {
let coll = mongoose.connection.db.collection(<Your collection name in plural form>);
// let data = await coll.find({}, {limit:2}).toArray();
// let data = await coll.find({name:"Vishal"}, {limit:2}).toArray();
// let data = await coll.find({name:"Vishal"}, {projection:{player:1, _id:0}}).toArray();
let data = await coll.find({}, {limit:3, sort:{name:-1}}).toArray();
console.log(data);
}, 2000);
I have also mentioned some of the criteria to filter out. Delete and update can also be done by this.
Thanks.
Make sure you're connecting to the right database as well as the right collection within the database.
You can include the name of the database in the connection string.
notice databasename in the following connection string:
var mongoose = require('mongoose');
const connectionString = 'mongodb+srv://username:password#hosturl.net/databasename';
mongoose.connect(connectionString);