I am running papa parse on the server to read and parse a csv file usign the following code:
function getData() {
console.log("Started getData...");
const file = fs.createReadStream(filePath);
papa.parse(file, {
header: true,
complete: createRecord
});
}
My complete callback function looks like this:
function createRecord(row, file) {
records = row.data;
console.log("Started createRecord...");
records.forEach((record) => {
console.log(record)
Activity.create(record);
});
}
I then write this to a MondoDB database. Everything works perfectly well, except for the first field name that is parsed by papa. The record output from the console.log(record) call above looks like this:
{
'Activity_Date': '2018-08-28',
Time_Start: '2018-08-28 20:20',
Time_End: '2018-08-28 21:01',
Duration: 0.027962963,
Subject: 'CS410',
Semester: 'Fall 2018',
Week: 1,
Task: 'Lectures',
Day: 2
}
Notice the quotes around Activity_Date. This causes Mongo to ignore the first field and only commit the remainder to the database.
Attempting to edit the csv file differently has not resulted in anything useful. What is weird is that if I look at the meta data from papa parse, it gives me the field names without the qoutes.
The Mongo Schema for Activity looks like this:
const mongoose = require("mongoose");
const activitySchema = new mongoose.Schema({
Activity_Date: String,
Time_Start: String,
Time_End: String,
Duration: Number,
Subject: String,
Semester: String,
Week: Number,
Task: String,
Day: Number
});
module.exports = mongoose.model("Activity", activitySchema)
Any assistance in getting rid of the quotes will be very welcome!
Thanks
It has been a while since asked but i will leave what worked for me. Try using this in the config object:
transformHeader: h => h.trim()
Related
Need help with arrays in Mongoose. Here is how my schema looks :-
const alertsSchema = new mongoose.Schema({
Alert_Date: String,
Alert_StartTime: String,
Alert_EndTime: String,
Alert_RuleName: String,
Alert_RuleId: String,
Alert_EntryNumber: String,
Alert_AlertId: String,
Alert_Description: String,
Alert_TriggerTime: String,
Alert_Activities: [{
Alert_Activities_ActivityType: String,
Alert_Activities_ActivityTime: String,
Alert_Activities_AreaName: String,
Alert_Activities_AreaType: String,
Alert_Activities_Position: Array,
Alert_Activities_Duration: Number,
Alert_Activities_SecondVesselId: String,
Alert_Activities_SecondVesselName: String,
Alert_Activities_SecondVesselClassCalc: String,
Alert_Activities_SecondVesselSize: String,
Alert_Activities_SecondVesselMMSI: String,
Alert_Activities_SecondVesselIMO: String,
}],
})
The Alert_Activities is an array coming from my upstream node js application. I implemented a fswatch functionality and as soon as a particular file changes, I am looking to save the record in my collection. The upstream file will always contain an array. Generally on an average of around 4 to 5 records. In short Alert_Activities will be there for every element of the array.
I am running a for loop and then trying to save all four elements in one go into my collection.myObject is the full array read from the upstream file using fs.read
for(i=0; i<myObject.length; i++){
var newAlertData = new alertRegister({
Alert_Date: date1,
Alert_StartTime: startNotificationDate,
Alert_EndTime: endNotificationDate,
Alert_RuleName: myObject[i].ruleName,
Alert_RuleId: myObject[i].ruleId,
Alert_EntryNumber: myObject[i].entryNumber,
Alert_AlertId: myObject[i].alertId,
Alert_Description: myObject[i].description,
Alert_TriggerTime: myObject[i].triggerTime,
// Alert_Activities: myObject[i].activities,
});
newAlertData.save(function(err,data1){
if(err){
console.log(err)
} else {
console.log("data saved")
}
})
The Alert_Activities will obviously not save. What is the right way to do this in Mongoose?
If you are dealing with Mongoose Documents, not with .lean() javascript objects, probably, you'll need to mark array field(s) as modified via markModified and only then use .save().
Also, directModifiedPaths could help you to check, what fields has been modified in the document.
I have noticed that your Alert_Activities, is actually an array of objects, so make sure that the result mongoose documents, which you are trying to .save() really satisfy all the validation rules.
If changes on some fields do successfully saved but the others - don't, then if's definitely something wrong with field names, or validation. And the DB/mongoose doesn't trigger you an error, because the document has already been saved, even in particular.
I am new to MongoDB/Mongoose, and I am having an issue with PUT request to update the phone number of name with duplicate entry.
Objective: My front end can take in person's name and its phone number. Whenever it takes in a name that already exist in our record, it will ask user if they want to replace the number. Furthermore, name input should be case insensitive, meaning as long as spelling is right, it should update the phone number.
e.g. If {name: Test, number: 123} exist in our record, inputting {name: TEST, number: 456} will provide a pop-up menu confirming if user want to change their name. If selected Ok, the record would change to {name: Test, number:456} while reflecting its change on front end side and on backend DB.
Currently, my schema is defined as following (on a different file).
const personSchema = new mongoose.Schema({
name: String,
number: String,
})
Current code for updating functionality is following, which does work for most part:
app.put('/api/persons/:id', (req, res, next) => {
const body = req.body
const person = {
name: body.name,
number: body.number,
}
Person.findByIdAndUpdate(req.params.id, person, { new: true })
.then(updatedPerson => res.json(updatedPerson))
.catch(err => next(err))
// catch is for error handling
})
Main issue: However, above code does not work when I input a name with same name but with different casing. For instance, Name "Test" and "TEST" is considered differently. Primary source of error, based on what I printed out on console, is that the ID is different and thus my code can't find the same entry.
i.e. ID for storing record with name "Test" is different from the ID of new entry {name: TEST, number: 123} and hence my input entry ID doesn't exit in my database.
Above is bit weird sense whenever I input the name with same case, it does work.
Based on some searching, I found a different stackoverflow suggestion that uses regex and findoneandupdate, so I tried the following:
app.put('/api/persons/:id', (req, res, next) => {
const body = req.body
const person = {
name: body.name,
number: body.number,
}
// for testing purpose,
// this prints out the correct name
Person.find({ name: new RegExp(`^${body.name}$`, `i`) })
.then(result => {
console.log(result[0])
})
Person.findOneAndUpdate({ name: new RegExp(`^${body.name}$`, `i`) }, person, { new: true })
.then(updatePerson => {
res.json(updatePerson)
})
.catch(err => next(err))
})
There was few issue with this:
Casing of person's name changes (so if the new input has name of "TEST", it will change it all caps when it is supposed to preserve the casing of initial entry)
Above worked via REST Client extension on VS code, which is similar to Postman. However actually testing on frontend had a same issue of not finding ID.
I was wondering what is the correct way to update the entry with either findByIdAndUpdate (preferably) or findOneAndUpdate while taking case insensitive entry and preserving the name.
For reference, following is what my front end looks like:
You can add a field query_name : String that is the lowercase version of the name.
I would use mongoose hooks.
const personSchema = new mongoose.Schema({
name: String,
number: String,
original_name : String
})
personSchema.pre('save', function(next) {
this.query_name = this.name.toLowerCase();
next();
});
You can read more about it and maybe find another solution here
https://mongoosejs.com/docs/middleware.html
Note:
*Don't use arrow function because it does't have 'this' property.
Image that I have the following MongoDB model in Mongoose:
const stats = new mongoose.Schema({
startDate: Date,
endDate: Date
totals: {
revenue: Number,
tax: Number,
profit: Number
}
})
Now I want to reuse this schema in the schema itself. Each document contains the totals of a whole month, and for each separated day. So the complete model would be something like this:
const model = new mongoose.Schema({
stats,
days: [{stats}]
})
This is the code which I currently have, but when I try to create a new document in this model, there is nothing saved except for a empty array at days.
So it looks like that the stats property is not recognized as an schema (probably due to the missing name/key). How can I archive something like this?
Edit 1:
Fixed 1 fault in the model. The model is now looking like this:
const model = new mongoose.Schema({
stats,
days: [stats]
})
Now the model is created with data in days[] but the model itself is empty.
Try defining it as:
const model = new mongoose.Schema({
...stats.obj,
days: [{stats:{ type: stats}}]
})
I'm having a problem that is really bugging me. I don't even want to use this solution I don't think but I want to know if there is one.
I was creating a comment section with mongodb and mongoose and keeping the comments attached to the resource like this:
const MovieSchema = new mongoose.Schema({
movieTitle: {type: String, text: true},
year: Number,
imdb: String,
comments: [{
date: Date,
body: String
}]
})
When editing the comments body I understood I could access a nested document like this:
const query = {
imdb: req.body.movie.imdb,
"comments._id": new ObjectId(req.body.editedComment._id)
}
const update = {
$set: {
"comments.$.body": req.body.newComment
}
}
Movie.findOneAndUpdate(query, update, function(err, movie) {
//do stuff
})
I then wanted to roll out a first level reply to comments, where every reply to a comment or to another reply just appeared as an array of replies for the top level comment (sort of like Facebook, not like reddit). At first I wanted to keep the replies attached to the comments just as I had kept the comments attachted to the resource. So the schema would look something like this:
const MovieSchema = new mongoose.Schema({
movieTitle: {type: String, text: true},
year: Number,
imdb: String,
comments: [{
date: Date,
body: String,
replies: [{
date: Date,
body: String
}]
}]
})
My question is how would you go about accessing a nested nested document. For instance if I wanted to edit a reply it doesn't seem I can use two $ symbols. So how would I do this in mongodb, and is this even possible?
I'm pretty sure I'm going to make Comments have its own model to simplify things but I still want to know if this is possible because it seems like a pretty big drawback of mongodb if not. On the other hand I'd feel pretty stupid using mongodb if I didn't figure out how to edit a nested nested document...
according to this issue: https://jira.mongodb.org/browse/SERVER-27089
updating nested-nested elements can be done this way:
parent.update({},
{$set: {“children.$[i].children.$[j].d”: nuValue}},
{ arrayFilters: [{ “i._id”: childId}, { “j._id”: grandchildId }] });
this is included in MongoDB 3.5.12 development version, in the MongoDB 3.6 production version.
according to https://github.com/Automattic/mongoose/issues/5986#issuecomment-358065800 it's supposed to be supported in mongoose 5+
if you're using an older mongodb or mongoose versions, there are 2 options:
find parent, edit result's grandchild, save parent.
const result = await parent.findById(parentId);
const grandchild = result.children.find(child => child._id.equals(childId))
.children.find(grandchild => grandchild._id.equals(grandchildId));
grandchild.field = value;
parent.save();
know granchild's index "somehow", findByIdAndUpdate parent with:
parent.findByIdAndUpdate(id,
{ $set: { [`children.$.children.${index}.field`]: value }});
I'm using Mongoose on a Node.js server to save data into MongoDB. What I want to do is to check and see if a model object exists in the collection already.
For example heres my model:
var ApiRequest = new Schema({
route: String,
priority: String,
maxResponseAge: String,
status: String,
request: Schema.Types.Mixed,
timestamp: { type: Date, default: Date.now }
});
And here's what I would like to do:
var Request = mongoose.model('api-request', ApiRequest);
function newRequest(req, type) {
return new Request({
'route' : req.route.path,
'priority' : req.body.priority,
'maxResponseAge' : req.body.maxResponseAge,
'request' : getRequestByType(req, type)
});
}
function main(req, type, callback) {
var tempReq = newRequest(req, type);
Request.findOne(tempReq, '', function (err, foundRequest) {
// Bla bla whatever
callback(err, foundRequest);
});
}
The big issues I'm finding are that the tempReq which is a model has an _id variable and a timestamp that is going to be different from what is saved in the database. So I would like to ignore those fields and compare by everything else.
As a note my actual models have more variables than this hence the reason I don't want to use .find({ param : val, ....})..... and instead would like to use the existing model for comparison.
Any ideas? Thanks!
You need to use plain JS objects instead of Mongoose model instances as query objects (the first parameter to find).
So either:
Change newRequest to return a plain object and later pass that into a new Request() call if you need to add it to the database.
OR
In your main function turn tempReq into a query object like this:
var query = tempReq.toObject();
delete query._id;
Request.findOne(query, ...