Why does express/mongodb updateOne post allow $set of some values, but will not update others? - mongodb

I am baffled by this post method, it will update fields 'x and y', but any attempt to set an array of widgets fails.
It is finding the correct item to update, passing all the required information through, but it will not allow insertion of, or update to 'widgets' fields.
Even if I remove the data intended for widgets and arbitrarily send through 'foo' it will not update with a field 'widgets'.
What am I doing wrong here???
API Call to Update Widgets. The Arbitrary X and Y values will update on the database, but any attempt to update widget makes no change
const saveUpdatedWidgets = async (update, _id) => {
console.log("called to update widgets ",update.widgets," in pagecard saveUpdatedWidgets")
let widgetObject = []
for(let u=0;u<update.widgets.length;u++){
widgetObject.push({
id: update.widgets[u].id,
text: update.widgets[u].text
})
}
Api.withToken().post('/pagewidget/'+_id,
{widgets: widgetObject, x:250, y:250}
).then(function (response) {
console.log("?worked ",response.data)
}).catch(function (error) {
console.log("page save failed for some reason on pagecard: ",error.response);
});
};
This will return the following in the console:
Code for post method is:
//THIS ROUTER WILL NOT UPDATE ANY WIDGETS FOR SOME REASON
router.post('/pagewidget/:_id',auth, async(req,res)=>{
console.log("request to update ",req.body," for id ",req.params," in pagewidgetsave post")
const query = { "_id": req.params };
const addedWidgets = req.body;
const newValues = { $set: addedWidgets }
try {
const thePage = await Pages.updateOne( query, newValues);
res.status(201).send(thePage)
console.log("updated Page: ",thePage);
}
catch(e){
console.log(e);
res.status(400).send(e)
}
})
Results from the console running node shows that values are going through, but only x and y actually update in database..
Here is the axios api.js file if there are any issues here:
import axios from 'axios';
const baseURL = process.env.REACT_APP_BASE_URL || "http://localhost:3001"
export default {
noToken() {
return axios.create({
baseURL: baseURL
});
},
withToken() {
const tokenStr = window.sessionStorage.getItem("token")
return axios.create({
baseURL: baseURL,
headers: {"Authorization" : `Bearer ${tokenStr}`}
});
}
}
What is going on!!?? It finds the page OK, and updates x and y values, but can't update widgets, even if the values for widget are just a string or number...

I found the issue. the MongoDB documentation doesn't mention this too well, and in its examples for updateOne() it passes an object for the update argument.
BUT, if you are setting a new field, this argument must be wrapped inside an array to use $set, this is because it can accept both methods to $set and to $unset. (see mongoDB docs)
(i.e. updateOne({query} , [{$set: {field:"value"}, {$unset: {otherfield:"othervalue"}])
In the end the post method just had to change to const thePage = await Pages.updateOne( query, [newValues]); (with newValues stored as an object inside an array, to allow addition of $unset if it was needed.
This is why it would update existing values OK, but it would not set new values into the database.
What a journey....
Full code for post method here
router.post('/pagewidget/:_id',auth, async(req,res)=>{
const query = {"_id": req.params._id};
const addedWidgets = req.body;
const newValues = { $set: addedWidgets }
try {
const thePage = await Pages.updateOne( query, [newValues]);
res.status(201).send(thePage)
console.log("updated Page: ",thePage);
}
catch(e){
console.log(e);
res.status(400).send(e)
}
})

Related

Get nth document in a collection in Firestore [duplicate]

I am trying to paginate documents with firestore beta. I am following the official docs
I see there is a pageSize param to say how many documents you want to show, but i dont see any offset option. Does anybody know how can i do a pagination system?
thank you
From your documentation link, I assume that you're using the REST API. In the REST API, there is a pageToken parameter, which you can specify. This can be derived from the nextPageToken returned from the previous request.
Previous response
{
"documents": [
{
object(Document)
}
],
"nextPageToken": ABCDEF1234567890,
}
Next request
projects/my-project/databases/my-database/documents or projects/my-project/databases/my-database/documents/chatrooms?pageSize=20&pageToken=ABCDEF1234567890
Firestore range queries are based on an having anchor document. So you must know the (order-by properties of the) document that the range start with, and then use ref.startAfter(anchorDdoc).limit(10) (or ref.startAfter(valueOfAnchorDoc).limit(10)) to get the next page.
The Firestore server-side Admin SDKs have an offset() call that allows the server to determine the document to starts at. But the client-side SDKs do not have this method.
This may help paginating without page numbers (previous and next):
app.get('/page/next/:startAt',(req, res)=>{
(async ()=>{
try{
const startAt = req.params.startAt;
let query = db.collection('movies');
let response = [];
console.log(query);
await query.orderBy('time', 'desc').where("time", "<", new Date(parseInt(startAt)*1000)).limit(16).get().then(snapshots => {
let docs = snapshots.docs;
for (let doc of docs){
console.log(doc.data());
const selectedItems = {
id: doc.id,
name: doc.data().name,
desp: doc.data().desp,
site: doc.data().site,
time: doc.data().time
}
response.push(selectedItems);
}
return response; //each then should return a vlue
})
return res.status(200).send(response);
}catch (error){
console.log(error);
return res.status(500).send(error);
}
})();
});
app.get('/page/back/:backAt',(req, res)=>{
(async ()=>{
try{
const backAt = req.params.backAt;
let query = db.collection('movies');
let response = [];
console.log(query);
await query.orderBy('time', 'desc').where("time", ">", new Date(parseInt(backAt)*1000)).limit(16).get().then(snapshots => {
let docs = snapshots.docs;
for (let doc of docs){
console.log(doc.data());
const selectedItems = {
id: doc.id,
name: doc.data().name,
desp: doc.data().desp,
site: doc.data().site,
time: doc.data().time
}
response.push(selectedItems);
}
return response; //each then should return a vlue
})
return res.status(200).send(response);
}catch (error){
console.log(error);
return res.status(500).send(error);
}
})();
});

how to read back result when using mongodb transaction?

All mongodb transaction examples I have seen so far don't allow reading results back. For example (pseudo code):
begin_transaction
collection_one.update(...)
collection_two.update(...)
commit_transaction
The problem is, what if I want to update collection_two based on the result of updating collection_one?
For example?
begin_transaction
result = collection_one.update(...)
if (result.update_count() > 0)
{
collection_two.update(...)
}
commit_transaction
I have never seen an example like the above? It seems that when use transaction, I can't get the result back.
Another example,
begin_transaction
result = collection_children.find({name: 'xxx'})
collection_team.delete({name in result})
commit_transaction
Basically, I want to perform a find on a collection, and based the find result to perform a second action on a different collection.
And I want the 2 actions together be atomic.
Here is an example of how this works as expected with Mongoose. The same example obviously is possible without Mongoose.
var author = new Author({ email: "test22#test.com", other: [{a: 1}});
var book = new Book({ title: 'ABC' })
let doFoo = async () => {
const session = await mongoose.startSession();
session.startTransaction();
try {
const opts = { session, new: true };
let _author = await author.save() // Notice we get back the record
let _book = await book.save()
// Notice we are using now the new id of the just saved record
await Author.findOneAndUpdate({ _id: _author.id }, { $set: { other: { foo: 2 } }}, opts);
await Book.findOneAndUpdate({ _id: _book.id }, { $set: { title: "ABC" }}, opts);
await session.commitTransaction();
session.endSession();
} catch (error) {
await session.abortTransaction();
session.endSession();
throw error; // Rethrow so calling function sees error
}
}
doFoo()
So in the example above we create/save two different records in their respective collections and after that based on the new records we go back and update them.

Circular Reference Issue in Mongoose pre-hook

In my MongoDB/Node backend environment I am using Mongoose pre and post hook middleware to check what's changed on the document, in order to create some system notes as a result.
One problem I'm running into is that when I try and lookup the record for the document in question I get a "Customer.findOne()" is not a function error. This is ONLY a problem when I'm looking up a record from the same collection from which the model just launched this pre and post hook triggers file. In other words, if my "Customer" model kicks off functions in a pre hook function in an external file, then I get an error if I then try and lookup a Customer with a standard findOne():
My customer model looks something like this:
module.exports = mongoose.model(
"Customer",
mongoose
.Schema(
{
__v: {
type: Number,
select: false
},
deleted: {
type: Boolean,
default: false
},
// Other props
searchResults: [
{
matchKey: String,
matchValue: String
}
]
},
{
timestamps: true
}
)
.pre("save", function(next) {
const doc = this;
trigger.preSave(doc);
next();
})
.post("save", function(doc) {
trigger.postSave(doc);
})
.post("update", function(doc) {
trigger.postSave(doc);
})
.post("findOneAndUpdate", function(doc) {
trigger.postSave(doc);
})
);
... the problematic findOne() function in the triggers file being called from the model looks like this:
const Customer = require("../../models/customer");
exports.preSave = async function(doc) {
this.preSaveDoc = await Customer.findOne({
_id: doc._id
}).exec();
};
To clarify, this is NOT a problem if I'm using a findOne() to lookup a record from a different collection in this same triggers file. Then it works fine. See below when finding a Contact -- no problem here:
const Contact = require("../../models/contact");
exports.preSave = async function(doc) {
this.preSaveDoc = await Contact.findOne({
_id: doc._id
}).exec();
};
The workaround I've found is to use Mongo instead of Mongoose, like so:
exports.preSave = async function(doc) {
let MongoClient = await require("../../config/database")();
let db = MongoClient.connection.db;
db.collection("customers")
.findOne({ _id: doc._id })
.then(doc => {
this.preSaveDoc = doc;
});
}
... but I'd prefer to use Mongoose syntax here. How can I use a findOne() in a pre-hook function being called from the same model/collection as the lookup type?
I have ran similar issue few days ago.
Effectively it is a circular dependency problem. When you call .findOne() on your customer model it doesn't exist as it is not exported yet.
You should probably try something like that :
const customerSchema = mongoose.Schema(...);
customerSchema.pre("save", async function(next) {
const customer = await Customer.findOne({
_id: this._id
}).exec();
trigger.setPreSaveDoc(customer);
next();
})
const Customer = mongoose.model("Customer", customerSchema)
module.export Customer;
Here customer will be defined because it is not called (the pre hook) before its creation.
As an easier way (I am not sure about it) but you could try to move the Contact import in your Trigger file under the save function export. That way I think the decencies may works.
Did it helps ?

Mongo `pre` hook not firing as expected on `save()` operation

I am using pre and post hooks in my MongoDB/Node backend in order to compare a pre-save and post-save version of a document so I can generate notes via model triggers based on what's changed. In one of my models/collections this is working, but in another, it's not working as expected, and I'm not sure why.
In the problem case, some research has determined that even though I am calling a pre hook trigger on an operation that uses a save(), when I console out the doc state passed in that pre hook, it's already had the change applied. In other words, the hook is not firing before the save() operation, but after, from what I can tell.
Here is my relevant model code:
let Schema = mongoose
.Schema(CustomerSchema, {
timestamps: true
})
.pre("save", function(next) {
const doc = this;
console.log("doc in .pre: ", doc); // this should be the pre-save version of the doc, but it is the post-save version
console.log("doc.history.length in model doc: ", doc.history.length);
trigger.preSave(doc);
next();
})
.post("save", function(doc) {
trigger.postSave(doc);
})
.post("update", function(doc) {
trigger.postSave(doc);
});
module.exports = mongoose.model("Customer", Schema);
The relevant part of the save() operation that I'm doing looks like this (all I'm doing is pushing a new element to an array on the doc called "history"):
exports.updateHistory = async function(req, res) {
let request = new CentralReqController(
req,
res,
{
// Allowed Parameters
id: {
type: String
},
stageId: {
type: String
},
startedBy: {
type: String
}
},
[
// Required Parameters
"id",
"stageId",
"startedBy"
]
);
let newHistoryObj = {
stageId: request.parameters.stageId,
startDate: new Date(),
startedBy: request.parameters.startedBy,
completed: false
};
let customerToUpdate = await Customer.findOne({
_id: request.parameters.id
}).exec();
let historyArray = await customerToUpdate.history;
console.log("historyArray.length before push in update func: ", historyArray.length);
historyArray.push(newHistoryObj);
await customerToUpdate.save((err, doc) => {
if (doc) console.log("history update saved...");
if (err) return request.sendError("Customer history update failed.", err);
});
};
So, my question is, if a pre hook on a save() operation is supposed to fire BEFORE the save() happens, why does the document I look at via my console.log show a document that's already had the save() operation done on it?
You are a bit mistaken on what the pre/post 'save' hooks are doing. In pre/post hook terms, save is the actual save operation to the database. That being said, the this you have in the pre('save') hook, is the object you called .save() on, not the updated object from the database. For example:
let myCustomer = req.body.customer; // some customer object
// Update the customer object
myCustomer.name = 'Updated Name';
// Save the customer
myCustomer.save();
We just updated the customers name. When the .save() is called, it triggers the hooks, like you stated above. Only the difference is, the this in the pre('save') hook is the same object as myCustomer, not the updated object from the database. On the contrary, the doc object in the `post('save') hook IS the updated object from the database.
Schema.pre('save', function(next) {
console.log(this); // Modified object (myCustomer), not from DB
)};
Schema.post('save', function(doc) {
console.log(doc); // Modified object DIRECTLY from DB
});

Is there a way to perform a "dry run" of an update operation?

I am in the process of changing the schema for one of my MongoDB collections. (I had been storing dates as strings, and now my application stores them as ISODates; I need to go back and change all of the old records to use ISODates as well.) I think I know how to do this using an update, but since this operation will affect tens of thousands of records I'm hesitant to issue an operation that I'm not 100% sure will work. Is there any way to do a "dry run" of an update that will show me, for a small number of records, the original record and how it would be changed?
Edit: I ended up using the approach of adding a new field to each record, and then (after verifying that the data was right) renaming that field to match the original. It looked like this:
db.events.find({timestamp: {$type: 2}})
.forEach( function (e) {
e.newTimestamp = new ISODate(e.timestamp);
db.events.save(e);
} )
db.events.update({},
{$rename: {'newTimestamp': 'timestamp'}},
{multi: true})
By the way, that method for converting the string times to ISODates was what ended up working. (I got the idea from this SO answer.)
My advice would be to add the ISODate as a new field. Once confirmed that all looks good you could then unset the the string date.
Create a test environment with your database structure. Copy a handful of records to it. Problem solved. Not the solution you were looking for, I'm sure. But, I believe, this is the exact circumstances that a 'test environment' should be used for.
Select ID of particular records that you would like to monitor. place in the update {_id:{$in:[<your monitored id>]}}
Another option which depends of the amount of overhead it will cause you -
You can consider writing a script, that performs the find operation, add printouts or run in debug while the save operation is commented out. Once you've gained confidence you can apply the save operation.
var changesLog = [];
var errorsLog = [];
events.find({timestamp: {$type: 2}}, function (err, events) {
if (err) {
debugger;
throw err;
} else {
for (var i = 0; i < events.length; i++) {
console.log('events' + i +"/"+(candidates.length-1));
var currentEvent = events[i];
var shouldUpdateCandidateData = false;
currentEvent.timestamp = new ISODate(currentEvent.timestamp);
var change = currentEvent._id;
changesLog.push(change);
// // ** Dry Run **
// currentEvent.save(function (err) {
// if (err) {
// debugger;
// errorsLog.push(currentEvent._id + ", " + currentEvent.timeStamp + ', ' + err);
// throw err;
// }
// });
}
console.log('Done');
console.log('Changes:');
console.log(changesLog);
console.log('Errors:');
console.log(errorsLog);
return;
}
});
db.collection.find({"_manager": { $exists: true, $ne: null }}).forEach(
function(doc){
doc['_managers']=[doc._manager]; // String --> List
delete doc['_manager']; // Remove "_managers" key-value pair
printjson(doc); // Debug by output the doc result
//db.teams.save(doc); // Save all the changes into doc data
}
)
In my case the collection contain _manager and I would like to change it to _managers list. I have tested it in my local working as expected.
In the several latest versions of MongoDB (at least starting with 4.2), you could do that using a transaction.
const { MongoClient } = require('mongodb')
async function main({ dryRun }) {
const client = new MongoClient('mongodb://127.0.0.1:27017', {
maxPoolSize: 1
})
const pool = await client.connect()
const db = pool.db('someDB')
const session = pool.startSession()
session.startTransaction()
try {
const filter = { id: 'some-id' }
const update = { $rename: { 'newTimestamp': 'timestamp' } }
// This is the important bit
const options = { session: session }
await db.collection('someCollection').updateMany(
filter,
update,
options // using session
)
const afterUpdate = db.collection('someCollection')
.find(
filter,
options // using session
)
.toArray()
console.debug('updated documents', afterUpdate)
if (dryRun) {
// This will roll back any changes made within the session
await session.abortTransaction()
} else {
await session.commitTransaction()
}
} finally {
await session.endSession()
await pool.close()
}
}
const _ = main({ dryRun: true })