I'm trying to update a field in a collection with data from the same collection, but from a sub-sub field in it, and either can't get the syntax right, or I'm just doing it wrong.
I've spent quite some time now digging around here, but can't seem to get anywhere.
Here's the example structure of the users collection:
{
"_id": "12345qwerty",
"services": {
"oauth": {
"CharacterID": 12345678,
"CharacterName": "Official Username",
},
},
"name": "what I want to change",
"username": "OfficialUsername"
}
What I'm trying to do would be pretty trivial with SQL, ie: update all the display names to match a trusted source...
update users
set name = services.oauth.CharacterName;
...but I'm having trouble getting in MongoDB, and I have a feeling im doing it wrong.
Here's what i have so far, but it doesn't work as expected.
db.users.find().snapshot().forEach(
function (elem) {
db.users.update(
{ _id: elem._id },
{ $set: { name: elem.services.oauth.CharacterName } }
);
}
);
I can set the name to be anything at the base level, but can't set it to be something from the sublevel, as it doesn't recognise the sub-fields.
Any help would be greatly appreciated!
db.users.update({"services.oauth.CharacterName": {$exists: true}},{$set: {"name": "services.oauth.CharacterName"}},{multi:true})
I am setting name at the root of your document to be equal to the value in services.oauth.CharacterName in the sub sub document. multi = true will update multiple document, I am only updating documents that have the services.oauth.CharacterName value.
Related
I have a collection called "EN" and I created an index as follow:
db.EN.createIndex( { "Prod_id": 1 } );
When I run db.EN.getIndexes() I get this:
[{ "v": 2, "key": {
"_id": 1 }, "name": "_id_" }, { "v": 2, "key": {
"Prod_id": 1 }, "name": "Prod_id_1" }]
However, when I run the following query:
db.EN.find({'Icecat-interface.Product.#Prod_id':'ABCD'})
.explain()
I get this:
{ "explainVersion": "1", "queryPlanner": {
"namespace": "Icecat.EN",
"indexFilterSet": false,
"parsedQuery": {
"ICECAT-interface.Product.Prod_id": {
"$eq": "ABCD"
}
},
"queryHash": "D12BE22E",
"planCacheKey": "9F077ED2",
"maxIndexedOrSolutionsReached": false,
"maxIndexedAndSolutionsReached": false,
"maxScansToExplodeReached": false,
"winningPlan": {
"stage": "COLLSCAN",
"filter": {
"ICECAT-interface.Product.Prod_id": {
"$eq": "ABCD"
}
},
"direction": "forward"
},
"rejectedPlans": [] }, "command": {
"find": "EN",
"filter": {
"ICECAT-interface.Product.Prod_id": "ABCD"
},
"batchSize": 1000,
"projection": {},
"$readPreference": {
"mode": "primary"
},
"$db": "Icecat" }, "serverInfo": {
It's using COLLSCAN instead of the index, why is this happening?
MongoDB version is 5.0.9-8
Thanks
EDIT (and solution)
It turns that the field name has "#" in front and the index was created without this character so was not picking it up at all.
Once I created a new index using the field name as it was supposed to be it worked OK.
It was interesting though to see how indexing works and best practices
Your find operation is defined as
.find({'Icecat-interface.Product.#Prod_id':'ABCD'})
What is Icecat-interface.Product.#?
The parsedQuery in the explain output confirms that MongoDB is attempting to look for a document that has has a value of "ABCD" for a different field name than the one you have aindexed. From the explain you've provided, that field name is "ICECAT-interface.Product.Prod_id". As the field name being queried and the one that is indexed are different, MongoDB cannot use the index to perform the operation.
Marginally related, the # character that is used in the find is absent in the explain output. This appears to because the actual operation that was used to generate the explain was slightly different. This is also noticeable by the fact that the explain include a batchSize of 1000 which is absent in the operation that was shown as the one being explained.
Depending on what the Icecat-interface.Product.# prefix is supposed to be, the solution is probably to simply remove that from the query predicate in the find itself.
Edit to respond to the comment and the edit to the question. Regarding the comment first:
When I run this: .find({'Prod_id':'ABCD'}) it uses COLLSCAN which to me is wrong, as I have an index on that field, unless I'm missing something here
MongoDB will look to use an index if its first key is used by the query. So an index on { y: 1 } would not be eligible for use by a query of .find({ x: 1}). Similarly to a generic x and y example, Icecat-interface.Product.Prod_id and Prod_id are different field names. So if you query on one but only an index on the other exists, then a collection scan is the only way for the database to execute the query.
This then overlaps some with the edit to the question. In the edited question the new explain plan shows the database successfully using an index. However, that index is { "ICECAT-interface.Product.Prod_id": 1 } which is not the index that you originally show being created or present on the collection ({ "Prod_id": 1 }).
Moreover, you also mention that you "don't get any result back, even with products I know are in the DB". Which field in the database contains the value that you are searching on ('ABCD')? This is going to directly inform what results you get back and what index is used to find the results. Remember that you can search on any arbitrary field in MongoDB, even if it doesn't exist in the database.
I would recommend some extra attention be paid to the namespaces and field names that are being used. Unless this { "ICECAT-interface.Product.Prod_id": 1 } index was created after the db.EN.getIndexes() output was gathered, you may be inadvertently connecting to different systems or namespaces since that index is definitely present somewhere.
Based on your live comments while I'm writing this, seems like you've solved the field name mystery.
I am trying to updateMany() documents which are an array of documents, if they have a certain value. I am still learning mongodb and I know how to update all documents if they have this value with my new value. The problem I think I am having is that they are an array of documents so I need to go into each document find the field called courses go inside of it and find the document called name and then update it the value of name and I have been trying, but no luck. Also In the request I update the first collection no problem its when I get into the next collection where it does nothing like I already mentioned.
request.body.code = request.sanitize(request.body.code);
request.body.name = request.sanitize(request.body.name);
request.body.courses.forEach(course => {
course.code = request.sanitize(course.code);
course.name = request.sanitize(course.name);
});
let newValues = { $set: {"code": request.body.code, "name":request.body.name } };
result = await mongoClient.db(DB_NAME).collection("all_courses").updateOne(selector, newValues);
// Okay now try to update other collection of array documents which isnt working yet
selector = { "code": "PROG2700"};
newValues = { $set: {"code": request.body.code, "name": request.body.name } };
console.log("this is new values");
console.log(request.body.courses);
result = await mongoClient.db(DB_NAME).collection("technologies").updateMany(selector,newValues);
That is in my server doing the mongodb work and trying to update the documents.
let sendString:Object = {
"id": my,
"code": edit_Course?.code,
"name": state_category_name,
"courses":
[
{
"code": edit_Course?.code,
"name": state_category_name
}
]
}
That is what I am sending to the server as the request.body
{
"_id": "5fdaa52060689aa159e4122e",
"name": "Webpack",
"description": "Webpack is great",
"difficulty": 2,
"courses": [
{
"code": "PROG2700",
"name": "Client Side Programming"
},
{
"code": "PROG3017",
"name": "Full Stack Programming"
}
]
}
That is what the mongodb structure of each document looks like in the technologies collection that I am having trouble updating. Its the name field I need to update in every document under courses. Could anyone give me a helping hand, many thanks!
Your selector filter doesn't work for the technologies class because the matching code field is part of an array named courses. To account for this, the filter should look something like this:
{ "courses.code": "PROG2700" }
This is an example of querying a field in a document embedded in an array.
The next step is to update the right document in the array. This can be done in MongoDB using the positional operator $. The update document would look like this:
{ $set: { "courses.$.code": request.body.code, "courses.$.name": request.body.name }}
In this update document, the positional operator tells MongoDB to update the code and name fields of the first document in the courses array with a code of PROG2700.
Put together, the update might look like this:
filter = { "courses.code": "PROG2700" };
update = { $set: { "courses.$.code": request.body.code, "courses.$.name": request.body.name }};
result = await mongoClient.db(DB_NAME).collection("technologies").updateMany(filter,update);
I'm not quite understanding how fuzzy works in the $searchBeta stage of aggregation. I'm not getting the desired result that I want when I'm trying to implement full-text search on my backend. Full text search for MongoDB was released last year (2019), so there really aren't many tutorials and/or references to go by besides the documentation. I've read the documentation, but I'm still confused, so I would like some clarification.
Let's say I have these 5 documents in my db:
{
"name": "Lightning Bolt",
"set_name": "Masters 25"
},
{
"name": "Snapcaster Mage",
"set_name": "Modern Masters 2017"
},
{
"name": "Verdant Catacombs",
"set_name": "Modern Masters 2017"
},
{
"name": "Chain Lightning",
"set_name": "Battlebond"
},
{
"name": "Battle of Wits",
"set_name": "Magic 2013"
}
And this is my aggregation in MongoDB Compass:
db.cards.aggregate([
{
$searchBeta: {
search: { //search has been deprecated, but it works in MongoDB Compass; replace with 'text'
query: 'lightn',
path: ["name", "set_name"],
fuzzy: {
maxEdits: 1,
prefixLength: 2,
maxExpansion: 100
}
}
}
}
]);
What I'm expecting my result to be:
[
{
"name": "Lightning Bolt", //lightn is in 'Lightning'
"set_name": "Masters 25"
},
{
"name": "Chain Lightning", //lightn is in 'Lightning'
"set_name": "Battlebond"
}
]
What I actually get:
[] //empty array
I don't really understand why my result is empty, so it would be much appreciated if someone explained what I'm doing wrong.
What I think is happening:
db.cards.aggregate... is looking for documents in the "name" and "set_name" fields for words that have a max edit of one character variation from the "lightn" query. The documents that are in the cards collection contain edits that are greater than 2, and therefor your expected result is an empty array. "Fuzzy is used to find strings which are similar to the search term or terms"; used with maxEdits and prefixLength.
Have you tried the term operator with the wildcard option? I think the below aggregation would get you the results you were actually expecting.
e.g.
db.cards.aggregate([
{$searchBeta:
{"term":
{"path":
["name","set_name"],
"query": "l*h*",
"wildcard":true}
}}]).pretty()
You need to provide an index to use with your search query.
The index is basically the analyzer that your query will use to process your results regarding if you want to a full match of the text, or you want a partial match etc.
You can read more about Analyzers from here
In your case, an index based on STANDARD analyzer will help.
After you create your index your code, modified below, will work:
db.cards.aggregate([
{
$search:{
text: { //search has been deprecated, but it works in MongoDB Compass; replace with 'text'
index: 'index_name_for_analyzer (STANDARD in your case)'
query: 'lightn',
path: ["name"] //since you only want to search in one field
fuzzy: {
maxEdits: 1,
prefixLength: 2,
maxExpansion: 100
}
}
}
}
]);
I have a document which looks like this:
"id":X,
"name": "X",
"email": "X",
"shows": ["id":X, ,"name":X,"season":X,"ep_id":X, "active" : true/false]
Inside shows can be many records with unique ID.
Given a show id,how can i check if active field is true or not for this record?
I want to do some action if the value of the field is false.
I've tried this but it gives me the full document.
userM.find({id: userID, 'shows.id': showID,'shows.active':true})
Thanks
Edit:
I managed to find a way using forEach manipulation on the query results.
Try This:
db.userM.find(
{ shows: { $elemMatch: { "id": "X", "active":true } } }
)
I've been wrestling with the asynchronous nature of MongoDB, Mongoose and JavaScript and how to best make multiple updates to a collection.
I have an Excel sheet of client and contact data. There are some clients with multiple contacts, one per line, and the client data is the same (so the client name can be used as a unique key - in fact in the schema it's defined with unique: true).
The logic I want to achieve is:
Search the Client collection for the client with clientName as the key
If a matching clientName isn't found then create a new document for that client (not an upsert, I don't want to change anything if the client document is already in the database)
Check to see if the contact is already present in the array of contacts within the client document using firstName and lastName as the keys
If the contact isn't found then $push that contact onto the array
Of course, we could easily have a situation where the client doesn't exists (and so is created) and then immediately, the very next row of the sheet, is another contact for the same client so then I'd want to find that existing (just created) client and $push that 2nd new contact into the array.
I've tried this but it's not working:
Client.findOneAndUpdate(
{clientName: obj.client.clientname},
{$set: obj.client, $push: {contacts: obj.contact}},
{upsert: true, new: true},
function(err, client){
console.log(client)
}
)
and I've had a good look at other questions, e.g.:
create mongodb document with subdocuments atomically?
https://stackoverflow.com/questions/28026197/upserting-complex-collections-through-mongoose-via-express
but can't get a solution... I'm coming to the conclusion that maybe I have to use some app logic to do the find, then decisions in my code, then writes, rather than use a single Mongoose/Mongo statement, but then the issues of asynchronicity rear their ugly head.
Any suggestions?
The approach to handling this is not a simple one, as mixing "upserts" with adding items to "arrays" can easily lead to undesired results. It also depends on if you want logic to set other fields such as a "counter" indicating how many contacts there are within an array, which you only want to increment/decrement as items are added or removed respectively.
In the most simple case however, if the "contacts" only contained a singular value such as an ObjectId linking to another collection, then the $addToSet modifier works well, as long as there no "counters" involved:
Client.findOneAndUpdate(
{ "clientName": clientName },
{ "$addToSet": { "contacts": contact } },
{ "upsert": true, "new": true },
function(err,client) {
// handle here
}
);
And that is all fine as you are only testing to see if a doucment matches on the "clientName", if not upsert it. Whether there is a match or not, the $addToSet operator will take care of unique "singular" values, being any "object" that is truly unique.
The difficulties come in where you have something like:
{ "firstName": "John", "lastName": "Smith", "age": 37 }
Already in the contacts array, and then you want to do something like this:
{ "firstName": "John", "lastName": "Smith", "age": 38 }
Where your actual intention is that this is the "same" John Smith, and it's just that the "age" is not different. Ideally you want to just "update" that array entry end neiter create a new array or a new document.
Working this with .findOneAndUpdate() where you want the updated document to return can be difficult. So if you don't really want the modified document in response, then the Bulk Operations API of MongoDB and the core driver are of most help here.
Considering the statements:
var bulk = Client.collection.initializeOrderedBulkOP();
// First try the upsert and set the array
bulk.find({ "clientName": clientName }).upsert().updateOne({
"$setOnInsert": {
// other valid client info in here
"contacts": [contact]
}
});
// Try to set the array where it exists
bulk.find({
"clientName": clientName,
"contacts": {
"$elemMatch": {
"firstName": contact.firstName,
"lastName": contact.lastName
}
}
}).updateOne({
"$set": { "contacts.$": contact }
});
// Try to "push" the array where it does not exist
bulk.find({
"clientName": clientName,
"contacts": {
"$not": { "$elemMatch": {
"firstName": contact.firstName,
"lastName": contact.lastName
}}
}
}).updateOne({
"$push": { "contacts": contact }
});
bulk.execute(function(err,response) {
// handle in here
});
This is nice since the Bulk Operations here mean that all statements here are sent to the server at once and there is only one response. Also note here that the logic means here that at most only two operations will actually modify anything.
In the first instance, the $setOnInsert modifier makes sure that nothing is changed when the document is just a match. As the only modifications here are within that block, this only affects a document where an "upsert" occurs.
Also note on the next two statements you do not try to "upsert" again. This considers that the first statement was possibly successful where it had to be, or otherwise did not matter.
The other reason for no "upsert" there is because the condtions needed to test the presence of the element in the array would lead to the "upsert" of a new document when they were not met. That is not desired, therefore no "upsert".
What they do in fact is respectively check whether the array element is present or not, and either update the existing element or create a new one. Therefore in total, all operations mean you either modify "once" or at most "twice" in the case where an upsert occurred. The possible "twice" creates very little overhead and no real problem.
Also in the third statement the $not operator reverses the logic of the $elemMatch to determine that no array element with the query condition exists.
Translating this with .findOneAndUpdate() becomes a bit more of an issue. Not only is it the "success" that matters now, it also determines how the eventual content is returned.
So the best idea here is to run the events in "series", and then work a little magic with the result in order to return the end "updated" form.
The help we will use here is both with async.waterfall and the lodash library:
var _ = require('lodash'); // letting you know where _ is coming from
async.waterfall(
[
function(callback) {
Client.findOneAndUpdate(
{ "clientName": clientName },
{
"$setOnInsert": {
// other valid client info in here
"contacts": [contact]
}
},
{ "upsert": true, "new": true },
callback
);
},
function(client,callback) {
Client.findOneAndUpdate(
{
"clientName": clientName,
"contacts": {
"$elemMatch": {
"firstName": contact.firstName,
"lastName": contact.lastName
}
}
},
{ "$set": { "contacts.$": contact } },
{ "new": true },
function(err,newClient) {
client = client || {};
newClient = newClient || {};
client = _.merge(client,newClient);
callback(err,client);
}
);
},
function(client,callback) {
Client.findOneAndUpdate(
{
"clientName": clientName,
"contacts": {
"$not": { "$elemMatch": {
"firstName": contact.firstName,
"lastName": contact.lastName
}}
}
},
{ "$push": { "contacts": contact } },
{ "new": true },
function(err,newClient) {
newClient = newClient || {};
client = _.merge(client,newClient);
callback(err,client);
}
);
}
],
function(err,client) {
if (err) throw err;
console.log(client);
}
);
That follows the same logic as before in that only two or one of those statements is actually going to do anything with the possibility that the "new" document returned is going to be null. The "waterfall" here passes a result from each stage onto the next, including the end where also any error will immediately branch to.
In this case the null would be swapped for an empty object {} and the _.merge() method will combine the two objects into one, at each later stage. This gives you the final result which is the modified object, no matter which preceeding operations actually did anything.
Of course, there would be a differnt manipulation required for $pull, and also your question has input data as an object form in itself. But those are actually answers in themselves.
This should at least get you started on how to approach your update pattern.
I don't use mongoose so I'll post a mongo shell update; sorry for that. I think the following would do:
db.clients.update({$and:[{'clientName':'apple'},{'contacts.firstName': {$ne: 'nick'}},{'contacts.lastName': {$ne: 'white'}}]},
{$set:{'clientName':'apple'}, $push: {contacts: {'firstName': 'nick', 'lastName':'white'}}},
{upsert: true });
So:
if the client "apple" does not exists, it is created, with a contact with given first and last name. If it exists, and does not have the given contact, it has it pushed. If it exists, and already has the given contact, nothing happens.