Lets say I have a document structured like that :
datas: {
foo: {
...
keytoupdate: [...]
},
whatever: {
...
keytoupdate: [...]
},
anystring: {
...
keytoupdate: [...]
},
...: {
...
keytoupdate: [...]
}
}
I know that :
Each direct child property of the "datas" document has a "keytoupdate" field.
The direct child properties of the "datas" document varies from case to case: not necessarily the same name, neither the same number.
I want to update each "keytoupdate" fields, no matter how many of them there are.
The question is: How can I do that ? Is there any magic operator like $ that does the same job for Array ?
Thank you !
I'll answer my own question : there is no way to do that, we can't play with dynamic keys, just forget about it ! But there are 2 workarounds :
The best solution, as suggested by #chridam, is to redesign the schema to make an array of objects, where the keys are parts of the arrays, you can see this question for more details.
If you can't, the other (but not good) solution is to make a request for each field that might be in your document, instead of trying to do this in one request. This is a very bad solution, especially if your document may have lots of fields, and you have to known which fields that could be in your documents. This is a bad solution, absolutely not optimized, but it has the merit of being simple to implement
Related
I have an object structure like this:
{
name: "...",
pockets: [
{
cdate: "....",
items: [...]
}
...
]
}
In an update operation, I want to add some records into the items field of the last pocket item. Using dot notation is the only way that I know to access a sub document, but I can't get what I want. So, I'm looking for something like these:
pockets.-1.items
pockets.$last.items
Is it possible to modify the last element? If yes, how?
I don't know of a way to do this using a single-line query. But you could select the record, update and then save it.
var query = <insert query here>;
var mydocs = db.mycollection.find(query);
for (var i=0 ; i<mydocs.length ; i++) {
mydocs[i].pockets[pockets.length-1].items.push('new item');
db.mycollection.save(mydoc);
}
I don't believe it is possible to do it atomically. There is a request for this functionality to be added to MongoDB.
If you can assure thread-safety in your application code, you could probably use a sequence of $pop from pockets array (that removes the last element from pockets) to variable p and then $addToSet to p.items, now you can $push p back into pockets. But if your application doesn't have a way to assure only one process may be doing this at one time, then another process could modify the array in the middle of those steps and you may end up losing that update.
You might also look into "Update if current" semantics here to see another way you can work around possible race by multiple threads issue.
TLDR: What is request.resource.data.size() counting in the firestore rules when writing, say, some booleans and a nested Object to a document? Not sure what the docs mean by "entries in the map" (https://firebase.google.com/docs/reference/rules/rules.firestore.Resource#data, https://firebase.google.com/docs/reference/rules/rules.Map) and my assumptions appear to be wrong when testing in the rules simulator (similar problem with request.resource.data.keys().size()).
Longer version: Running into a problem in Firestore rules where not being able to update data as expected (despite similar tests working in the rules simulator). Have narrowed down the problem to point where can see that it is a rule checking for request.resource.data.size() equaling a certain number.
An example of the data being passed to the firestore update function looks like
Object {
"parentObj": Object {
"nestedObj": Object {
"key1": Timestamp {
"nanoseconds": 998000000,
"seconds": 1536498767,
},
},
},
"otherKey": true,
}
where the timestamp is generated via firebase.firestore.Timestamp.now().
This appears to work fine in the rules simulator, but not for the actual data when doing
let obj = {}
obj.otherKey = true
// since want to set object key name dynamically as nestedObj value,
// see https://stackoverflow.com/a/47296152/8236733
obj.parentObj = {} // needed for adding nested dynamic keys
obj.parentObj[nestedObj] = {
key1: fb.firestore.Timestamp.now()
}
firebase.firestore.collection('mycollection')
.doc('mydoc')
.update(obj)
Among some other rules, I use the rule request.resource.data.size() == 2 and this appears to be the rules that causes a permission denied error (since commenting out this rules get things working again). Would think that since the object is being passed with 2 (top-level) keys, then request.resource.data.size()=2, but this is apparently not the case (nor is it the number of keys total in the passed object) (similar problem with request.resource.data.keys().size()). So there's a long example to a short question. Would be very helpful if someone could clarify for me what is going wrong here.
From my last communications with firebase support around a month ago - there were issues with request.resource.data.size() and timestamp based security rules for queries.
I was also told that request.resource.data.size() is the size of the document AFTER a successful write. So if you're writing 2 additional keys to a document with 4 keys, that value you should be checking against is 6, not 2.
Having said all that - I am still having problems with request.resource.data.size() and any alternatives such as request.resource.size() which seems to be used in this documentation
https://firebase.google.com/docs/firestore/solutions/role-based-access
I also have some places in my security rules where it seems to work. I personally don't know why that is though.
Been struggling with that for a few hours and I see now that the doc on Firebase is clear: "the request.resource variable contains the future state of the document". So with ALL the fields, not only the ones being sent.
https://firebase.google.com/docs/firestore/security/rules-conditions#data_validation.
But there is actually another way to ONLY count the number of fields being sent with request.writeFields.size(). The property writeFields is a table with all the incoming fields.
Beware: writeFields is deprecated and may stop working anytime, but I have not found any replacement.
EDIT: writeFields apparently does not work in the simulator anymore...
We recently added a new property to one of our Mongoose schemas that defines takes a String with an enum validator and a default value. We now need to query for documents using that property but it isn't set for pre-existing documents until after the query happens. Is there any way to get around this without having to re-save every document in that collection that existed before this change, or if not is there a best practice for how to do that cleanly?
The new property:
sales_category: {
type: String,
required: true,
enum: ["Prospect", "Subscriber", "Activated Trial", "Expired Subscriber", "Expired Free Trial"],
default: "Prospect"
}
The query:
Account.find({sales_category: "Prospect"}).populate("account_holder").exec(function(err, accounts) {
Edit: I just found https://stackoverflow.com/a/14288276/8324 which seems to imply that there is no clean way to do this, the suggestion to leave it as it is and just invert any query for {sales_category: "Prospect"} to a query for not any of the other categories seems like the best solution provided the enum never changes. I'm not sure if we can guarantee that it in this use case though so I think I might end up falling back to the "re-save everything" solution, even if it doesn't feel great.
I'll leave this open for now in case someone has a better solution.
What about using an OR conditional with an $exists operator?
Account.find().or([{sales_category: "Prospect"}, { sales_category: { $exists: false }}]).populate("account_holder").exec(function(err, accounts) {...});
One note: this doesn't use an index which would only be a concern if this is used on a large collection.
I know this has been asked before, but I have yet to find a solution that works efficiently. I am working with the MongoDB C# driver, though this is more of a general question about MongoDB operations.
I have a document structure that looks something like this:
field1: value1
field2: value2
...
users: [ {...user 1 subdocument...}, {...user 2 subdocument...}, ... ]
Some facts:
Each user subdocument includes further sub-arrays & subdocuments (so they're fairly complex).
The average users array only contains about 5 elements, but in the worst case can surpass 100.
Several thousand update operations on multiple users may be conducted per day in this system, each on one document at a time. Larger arrays will receive more frequent updates due to their data size.
I am trying to figure out how to do this efficiently. From what I've heard, you cannot directly set several array elements to new values all at once, so I had to try something else.
I tried using the $pullAll / $AddToSet + $each operations to remove the old array and replace it with a modified one. I am aware that $pullall can remove only the elements that I need as well, but I would like to preserve the order of elements.
The C# code:
try
{
WriteConcernResult wcr = collection.Update(query,
Update.Combine(Update.PullAll("users"),
Update.AddToSetEach("users", newUsers.ToArray())));
}
catch (WriteConcernException wce)
{
return wce.Message;
}
In this case newUsers is aList<BsonValue>converted to an array. However I am getting the following exception message:
Cannot update 'users' and 'users' at the same time
By the looks of it, I can't have two update statements in use on the same field in the same write operation.
I also tried Update.Set("users", newUsers.ToArray()), but apparently the Set statement doesn't work with arrays, just basic values:
Argument 2: cannot convert from 'MongoDB.Bson.BsonValue[]' to 'MongoDB.Bson.BsonValue'
So then I tried converting that array to a BsonDocument:
Update.Set("users", newUsers.ToArray().ToBsonDocument());
And got this:
An Array value cannot be written to the root level of a BSON document.
I could try replacing the whole document, but that seems like overkill and definitely not very efficient.
So the only thing I can think of now is to run two separate write operations: one to remove the unwanted old users and another to replace them with their newer versions:
WriteConcernResult wcr = collection.Update(query, Update.PullAll("users"));
WriteConcernResult wcr = collection.Update(query, Update.AddToSetEach("users", newUsers.ToArray()));
Is this my best option? Or is there another, better way of doing this?
Your code should work with a minor change:
Update.Set("users", new BsonArray(newUsers));
BsonArray is a BsonValue, where as an array of documents is not and we don't implicitly convert arrays like we do other primitive values.
this extension method solve my problem:
public static class MongoExtension
{
public static BsonArray ToBsonArray(this IEnumerable list)
{
var array = new BsonArray();
foreach (var item in list)
array.Add((BsonValue) item);
return array;
}
}
So i need some advice as to what i'm doing incorrectly.
My database is setup up exactly like a file system consisting of folders and files.
It begins with a folder, but can have a relatively infinite number of subfolders and or files.
{
"name":"folder1",
"uniqueID":"zzz0",
"subcontents": [ {"name":"subfolder1", "uniqueID":"zzz1"},
{"name":"subfile1", "uniqueID":"zzz2"},
{"name":"subfile2", "uniqueID":"zzz3"},
{"name":"subfolder2", "subcontents": [...etc...], "uniqueID":"zzz4"},
]
}
Each folder/file document have a uniqueID so that I can reference to it (seen above zzz#). My question is, can I make a mongoDB query to pull out only a single document?
Like say for example db.fileSystemCollection.find({"uniqueID":"zzz4"}) and it would give me the following result? Do i have to use indexes to do this? I've been trying but the query returns empty every time.
intended result ---> {"name":"subfolder2", "subcontents": [...etc...], "uniqueID":"zzz4"}
[EDIT]
Based on the responses below, I will consider an XML database instead on mongoDB. The json structure cant be rearranged to work with MongoDB (too much data).
Short answer is no, as it's stated by Chris.
Your embedded representation of a tree is really good for intuitive understanding (and implementation as well). But if you want to allow effective searches on your tree using indices in MongoDB, you might consider another ways for tree storage. A bunch of ways is listed at http://docs.mongodb.org/manual/tutorial/model-tree-structures/
Please keep in mind that every representation has its own pros and cons depending on your access patterns.
Since for filesystem-like structure it's likely to have the ability to find all the sub contents of a given folder, you may use child references pattern for this:
{
"name":"folder1",
"uniqueID":"zzz0",
"subcontents": [ "zzz1",
"zzz2",
"zzz3",
"zzz4"
]
}
{
"name":"subfolder1",
"uniqueID":"zzz1"
}
...
No; searching for {uniqueID: "zzz4"} will only get you documents whose top-level uniqueID matches.
What you probably want is to maintain an array on the document which lists all the unique IDs in that tree. So your document would be:
{
"name":"folder1",
"uniqueID":"zzz0",
"idList": ["zzz0", "zzz1", "zzz2", "zzz3", "zzz4"],
"subcontents": [ {"name":"subfolder1", "uniqueID":"zzz1"},
{"name":"subfile1", "uniqueID":"zzz2"},
{"name":"subfile2", "uniqueID":"zzz3"},
{"name":"subfolder2", "subcontents": [...etc...], "uniqueID":"zzz4"},
]
}
Then you can index that:
db.fileSystemCollection.ensureIndex({"idList": 1})
Then you can find on it:
db.fileSystemCollection.find({"idList": "zzz4})
That'll return you those documents.
As an aside, if you're trying to store files in Mongo, have you looked at GridFS?