I want to save the large html content in mongodb but it doesn't allow to save more than 1 kb of content in single field of collection.
Basically i am building a article submission web app using NUXT & database i choose is Mongodb. Now i am stuck in this crucial task.
any help would be great!
var mongoose = require('mongoose');
var ArticleSchema = new mongoose.Schema({
title : { type : String , unique : true, required : true},
description: { type : String , unique : true, required : true},
content: { type : String , unique : true, required : true},
banner: { type : String },
active: { type: Boolean, default: false },
created_date: { type: Date, default: Date.now },
updated_date: { type: Date, default: Date.now },
postedBy: {
type : Object , required : true
},
comments: [{
text: String,
postedBy: {
type : Object , required : true
}
}]
});
module.exports = mongoose.model('Article', ArticleSchema);
Getting this error while saving large content:
Btree::insert: key too large to index, failing tech-expert.articles.$content_1 3500 { : \"<p>Here is Sample text!Here is Sample text!Here is Sample text!Here is Sample text!Here is Sample text!Here is Sample text!Here is Sample text!Here is...\" }"}
Seems like you have an index on content field. Consider removing this index, or replacing it with text index
UPD: Mongoose would automatically create Unique index if you schema says unique: true - just get rid of this part for the content field. Mongoose Schema Type
Related
I am looking for the best way to model this scenario:
There is a ProductA model. Users can "like" or "dislike" ProductA documents. The documents are then added to an array in the User model called "likes" & "dislikes."
var UserSchema = new mongoose.Schema({
...,
likes: [{ type: mongoose.Schema.Types.ObjectId, ref: 'ProductA' }],
dislikes: [{ type: mongoose.Schema.Types.ObjectId, ref: 'ProductA' }],
...,
});
Now, I have realized that I want to add in a new Product: "ProductB." How do I restructure this database to keep this scalable and allow me to add new products? I am not sure what the best way to do this would be in MongoDB.
I believe my ideal scenario is the following psuedo-model:
var InteractionSchema= new mongoose.Schema({
product: // anonymous reference to an object
productType: //enum of which product it is
user: // user who made the interaction
interactionType: // like or dislike enum
});
I could not find any reference to how to handle anonymous references in MongoDB however. I would appreciate some advice
If I understand your requirement correctly, you can have three collections at a time:
products (contains all the products)
users (contains user information)
user_product_likes (contains user's like/dislike)
Respective schema can be,
UserInformationSchema :
{
name : {
type: String,
required: false
..
},
..
}
ProductSchema :
{
product_type : {
type: Integer,
},
...
}
InteractionSchema :
{
product_id : {
type: Integer
required: true
},
user_id : {
type: Integer
required: true
},
like : {
type: Boolean
required: false,
default:false
},
dislike : {
type: Booelan,
required: false,
default: false
}
}
I created a simple dynamic fields in React-Redux with a plus button to add as many field as I want (hobbies) of an already existing form. I'm using mongodb as a database and so I have this error that tells me that my fields/data don't have iDs.
so how can I generate iDs for my data?
this below is my model with featherJs. as you can see this is how I added my hobbies array in the existing model called myService. I can see that my hobbies are created in mongo (using Robo 3T) which is great but i'm having difficulty reusing them (hobbies) in an other component in Redux. I'm not sure if I should give IDs to this fields or create a new service just for them. I never coded something in backend so I'm confused. what's the rule for this kind of situations.
Any other suggestions would be helpful.
warning in Redux: Each child in a list should have a unique "key" prop.
error in api : Cast to ObjectId failed for value at path "_id" for model "
const { Schema } = mongooseClient;
const myService = new Schema({
type: { type: String, enum: VALID_TYPES, required: true },
user: {
type: mongooseClient.Schema.Types.ObjectId,
ref: 'user',
required: true
},
comment: String,
hobbies: [{
type: mongooseClient.Schema.Types.ObjectId,
ref: 'hobbies',
default: [],
required: false }],
date: {
begin: { type: Date, default: Date.now },
current: { type: Date, default: Date.now },
end: { type: Date, required: true },
},
}, {
timestamps: true
});
return mongooseClient.model('myService', myService);
};
My collection has approximately 500 documents, which will be double that in a few weeks:
How can I make getting all the documents faster? I'm currently using db.registrations.find(), so that I can have all the documents available for searching, sorting, and filtering data. Using skip/limit makes the query display quickly, but you can't search all the registrations for players, and that's necessary.
My schema:
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var playerSchema = new mongoose.Schema({
first: {
type: String,
required: true
},
last: {
type: String,
required: true
},
email: {
type: String,
required: true
},
phone: {
type: String,
required: true
},
address: {
address: String,
city: String,
state: String,
zip: String,
country: {
type: String,
"default" : "USA"
},
adult: Boolean
}
});
var registrationsSchema = new mongoose.Schema({
event : {
type: String,
required: true
},
day : {
type: String,
required: true
},
group : {
type: String,
required: true
},
field : {
type: String,
},
price : {
type: Number,
required: true
},
players : [playerSchema],
net : Number,
team : {
type: Number,
min: 0,
max: 7,
"default" : null
},
notes: String,
paymentID : {
type: String,
required: true,
"default": "VOLUNTEER"
},
paymentStatus : {
type: String,
"default" : "PAID"
},
paymentNote : String,
// users : [userSchema],
active : {
type: Boolean,
default: true
},
users: [{
type: Schema.Types.ObjectId,
ref: 'User'
}],
createdOn : {
type : Date,
"default" : Date.now
},
updatedLast : {
type: Date
}
});
mongoose.model('Registration', registrationsSchema);
There is no big deal to load 1000 records from mongodb using mongoose. I did it in the past (2-3k records) and it worked well as long as I respected this 2 rules:
Don't load all the mongoose stuff
Use lean query.
This way it won't load all the mongoose methods / attributes and it will get just the data of your objects. You can't use .save() or other methods but it's way faster to load.
Use stream to load your data.
Streams are a good way to load large dataset with nodejs/mongoose. It will read the data block by block from mongodb and send them to your application for usage. You will avoid the tipical case :
I wait 2 seconds my data and my server is idle
My server is 100% CPU during 2 seconds to process the data I got and the db is idle.
With streams, in this example your total time will be ~2s instead of 2+2=4s
To load data from stream with mongoose use the .cursor() function to change your request to a nodejs stream.
Here is an example to load all your players fast
const allPlayers = [];
const cursor = Player.find({}).lean().cursor();
cursor.on('data', function(player) { allPlayers.push(player); });
cursor.on('end', function() { console.log('All players are loaded here'); });
You can achieve your objective using the following ways.
By default if you query for mongoose document, it will load with all of it's attributes and other necessary meta-data required(ie.,with lean = false). so if you use lean() function, it will load only a plain java script objects and it won't return even setter and corresponding getter methods. So that you can get all the documents very very fast. And you will get High performance. that's what the magic of lean() function on the back ground.
Another suggestion is as a thumb rule, please maintain proper indexing as per your requirement for each collection to get good performance while querying.
Hope this will help you!
I have two schemas, Mails and Users, as defined here:
var MailsSchema = new Schema({
from : [{ type: ObjectId, ref: 'UsersSchema' }],
to : [{ type: ObjectId, ref: 'UsersSchema' }],
date : { type: Date, default: Date.now },
subject : { type: String, default: '' },
message : { type: String, default: '' },
isRead : { type: Boolean, default: false }
});
var UsersSchema = new Schema({
username : String,
pass : String,
firstName : String,
lastName : String,
age : Number
});
I want to get list of mail that send to specific user, populate the username in 'from' field,
I tried to do it that way:
mails.find({ to: id }).populate('from', 'username').exec(function (err, docs)...
The problem is:
I'm getting an array in 'from' field, I want to get only the username.
I.e. instead of getting this result:
[{"_id":"53fa2902da480e4c0d315fcd","__v":0,"isRead":false,"message":"SABABA","subject":"MA KORE","date":"2014-08-24T18:03:46.428Z","to":["53f9dbe164b1375c1c5b997a"],"from":[{"_id":"53f9dbe164b1375c1c5b997a","username":"adaroh"}]}]
I want to get that result:
[{"_id":"53fa2902da480e4c0d315fcd","__v":0,"isRead":false,"message":"SABABA","subject":"MA KORE","date":"2014-08-24T18:03:46.428Z","to":["53f9dbe164b1375c1c5b997a"],"from":"adaroh"}]
That's just how populate works. You can modify the docs yourself to filter out the extra id. In your exec, for each doc that gets populated you can look at the from array and do
doc.from[i] = doc.from[i].username
The example here
http://mongoosejs.com/docs/populate.html
Provides the following code
var story1 = new Story({
title: "A man who cooked Nintendo"
, _creator: aaron._id
});
_creator is defined above as follows
_creator : { type: Schema.ObjectId, ref: 'Person' }
If I modify the code to the following
var story1 = new Story({
title: "A man who cooked Nintendo"
, _creator: {name: 'test'}
});
It seems to happily insert the data into MongoDB.
{ "title" : "A man who cooked Nintendo", "_creator" : { "name" : "test" }, "_id" : ObjectId("4fb7a55315c5f2de07000002"), "fans" : [ ] }
How would I catch the error before insertion? I would like to check that it is not only an ObjectId but also that it corresponds to a valid Person.
To continue what #JohnnyHK proposed, here is a complete solution(assuming _creator is a reference to a numeric id).
If you want to check if the value is a valid ObjectId
function isObjectId(n) {
return mongoose.Types.ObjectId.isValid(n);
}
validate: [validator, 'my error type']
_creator : { type: Schema.ObjectId, ref: 'Person', validate: isObjectId }
The method isValid does not exist (anymore?), your best bet is a simple regular expression, as provided here
You could add validation to the _creator field of the schema as described here.
_creator : { type: Schema.ObjectId, ref: 'Person', validate: ... }