Querying nested mongo document [duplicate] - mongodb

This question already has answers here:
MongoDB - how to query for a nested item inside a collection?
(3 answers)
Closed 6 years ago.
I have few JSON documents stored in MongoDB that are basically error reports, i.e. each document contains a nested data structure mismatches between datasets. They look like the following:
{
"nodename": "BLAH"
"errors": {
"PC": {
"value": {
"PC93196": [
"post",
"casper"
],
"PC03196": [
"netdb"
]
}
}
}
}
So in the above case, a process/tool determines the PC value of PC93196 and PC03196. netdb in this case reported a different PC value to that of post and casper.
How do I query for all documents that have errors where the process/tool casper is involved?

The document schema doesn't really lend itself to being queried very easily in the way you're trying to. This is because the value object is composed of named objects, rather than being an Array of objects. I.e. the value object has two properties: PC93196 and PC03196.
If your document was structured slightly differently it would be much easier to query in the way you want. For example a document structure like this:
{
"nodename": "BLAH",
"errors": {
"PC": [
{
"name": "PC93196",
"type": [
"post",
"casper"
]
},
{
"name": "PC03196",
"type": [
"netdb"
]
}
]
}
}
Would allow you to write a query like this:
db.errors.aggregate([{$unwind:"$errors.PC"},{$match:{"errors.PC.type":"casper"}}])
which would provide a result:
{ "_id" : ObjectId("5763c480dad14fd061657f91"), "nodename" : "BLAH", "errors" : { "PC" : { "name" : "PC93196", "type" : [ "post", "casper" ] } } }

Related

Search and update in array of objects MongoDB

I have a collection in MongoDB containing search history of a user where each document is stored like:
"_id": "user1"
searchHistory: {
"product1": [
{
"timestamp": 1623482432,
"query": {
"query": "chocolate",
"qty": 2
}
},
{
"timestamp": 1623481234,
"query": {
"query": "lindor",
"qty": 4
}
},
],
"product2": [
{
"timestamp": 1623473622,
"query": {
"query": "table",
"qty": 1
}
},
{
"timestamp": 1623438232,
"query": {
"query": "ike",
"qty": 1
}
},
]
}
Here _id of document acts like a foreign key to the user document in another collection.
I have backend running on nodejs and this function is used to store a new search history in the record.
exports.updateUserSearchCount = function (userId, productId, searchDetails) {
let addToSetData = {}
let key = `searchHistory.${productId}`
addToSetData[key] = { "timestamp": new Date().getTime(), "query": searchDetails }
return client.db("mydb").collection("userSearchHistory").updateOne({ "_id": userId }, { "$addToSet": addToSetData }, { upsert: true }, async (err, res) => {
})
}
Now, I want to get search history of a user based on query only using the db.find().
I want something like this:
db.find({"_id": "user1", "searchHistory.somewildcard.query": "some query"})
I need a wildcard which will replace ".somewildcard." to search in all products searched.
I saw a suggestion that we should store document like:
"_id": "user1"
searchHistory: [
{
"key": "product1",
"value": [
{
"timestamp": 1623482432,
"query": {
"query": "chocolate",
"qty": 2
}
}
]
}
]
However if I store document like this, then adding search history to existing document becomes a tideous and confusing task.
What should I do?
It's always a bad idea to save values are keys, for this exact reason you're facing. It heavily limits querying that field, obviously the trade off is that it makes updates much easier.
I personally recommend you do not save these searches in nested form at all, this will cause you scaling issues quite quickly, assuming these fields are indexed you will start seeing performance issues when the arrays get's too large ( few hundred searches ).
So my personal recommendation is for you to save it in a new collection like so:
{
"user_id": "1",
"key": "product1",
"timestamp": 1623482432,
"query": {
"query": "chocolate",
"qty": 2
}
}
Now querying a specific user or a specific product or even a query substring is all very easily supported by creating some basic indexes. an "update" in this case would just be to insert a new document which is also much faster.
If you still prefer to keep the nested structure, then I recommend you do switch to the recommended structure you posted, as you mentioned updates will become slightly more tedious, but you can still do it quite easily using arrayFilters for updating a specific element or just using $push for adding a new search

The dollar ($) prefixed field '$oid' in 'features.0._id.$oid' is not valid for storage

I have the following Mongo collection attached below.
it is existing one not new collection created, I am trying to push new item in the features array, but I am not able to achieve that due to the following error: The dollar ($) prefixed field '$oid' in 'features.0._id.$oid' is not valid for storage.
using this script:
db.getCollection("test").update({ _id: "RYB0001" },
{
$push: {
"features": {
"_id": {
"$oid": ObjectId(),
},
"featureItems": [],
"typeId": "type3"
}
}
});
If it is a driver version issue, any workaround can I do to push the new item to the current collection?
MongoDb version: 4.0.4
Mongo Collection screenshot
The $oid notation is part of MongoDB's Extended JSON. I assume the data in your database doesn't actually have that key-value pair and it was only represented that way after using something like JSON.stringify(obj)
> db.test.find({});
{ "_id" : "RYB0001", "features" : [ { "_id" : ObjectId("5e40d46a97abdef3faa0d5d9"), "featureItems" : [ ], "typeId" : "type3" } ] }
> JSON.stringify(db.test.find({})[0]);
{"_id":"RYB0001","features":[{"_id":{"$oid":"5e40d46a97abdef3faa0d5d9"},"featureItems":[],"typeId":"type3"}]}
You'll need to generate a new ObjectId using its constructor "_id": new ObjectId(), note the new keyword there, I think #Thilo may have missed that in their comment.
db.getCollection("test").update({ _id: "RYB0001" },
{
$push: {
"features": {
"_id": new ObjectId(),
"featureItems": [],
"typeId": "type3"
}
}
});

Loopback and MongoDB Embedded Documents Defining JSON Model

Can / how do you define an embedded document in a model's json definition with LoopbackJS without creating a model to represent the sub-document?
For example, consider this following MongoDB document:
{
_id: ObjectId("some_mongodb_id"),
subDocs: [
{
"propertyA": "a1",
"propertyB": "b1"
},
{
"propertyA": "a2",
"propertyB": "b2"
}
]
}
I could create two models in loopback:
some-model.json:
...
"properties": {
"subDocs": [
"SubDocsModel"
]
}
sub-docs-model.json:
...
"properties": {
"propertyA": "string",
"propertyB": "string"
}
Rather than doing that, however, I'd like to just declare the sub-doc model inline in the some-model.json since it's just there to document the shape of some-model's document.
Is that possible? Something like:
some-model.json:
...
"properties":{
"subDocs": [
{
"propertyA": {
"type": "string"
},
"propertyB": {
"type": "string"
}
}
]
}
I tried the above, but what I end up with is a field in my mongodb document that's of type string with the value [object Object]...
The purpose would be (1) to document the shape of the sub-document, and (2) to allow for validation by loopback without adding custom logic.
You can define it as an object
some-model.json:
"properties": {
"subDocs": ["object"]
}
But if you want validation or have a structure for sub-docs, you need to create a loopback model for that.
Loopback does not do any validation, ... for properties with type object.

Morphia query to filter and fetch an embedded list element

I am new to NoSQL and morphia. I am using Morphia to query MongoDB.
I have a sample collection as below:
[
{
"serviceId": "id1",
"serviceName": "ding",
"serviceVersion": "1.0",
"files": [
{
"fileName": "b.html",
"fileContents": "contentsA"
},
{
"fileName": "b.html",
"fileContents": "contentsB"
}
]
},
{
"serviceId": "id2",
"serviceName": "ding",
"serviceVersion": "2.0",
"files": [
{
"fileName": "b.html",
"fileContents": "contentsA"
},
{
"fileName": "b.html",
"fileContents": "contentsB"
}
]
}
]
I would like to fetch an element in "files" List , given service name, service version and filename., using Morphia.
I was able to get what I want using the query below:
db.ApiDoc.find({ serviceName: "ding", serviceVersion: "2.0"}, { files: { $elemMatch: { fileName: "b.html" } } }).sort({ "_id": 1}).skip(0).limit(30);
What I tried so far :
I tried using "elemmatch" api that morphia has, but no luck.
query = ...createQuery(
Result.class);
query.and(query.criteria("serviceName").equal("ding"),
query.criteria("serviceVersion").equal(
"2.0"));
query.filter("files elem",BasicDBObjectBuilder.start("fileName", "a.html").get());
I seem to get the entire Result collection with all the files. I would like to get only the matched files(by filename).
can some one help me how I can get this to work?
Thanks
rajesh
I don't believe it's possible to get just the matching sub element. You can request just to have the 'files' array returned but all elements will be included in the result set and you will have to refilter in your code.
The other option is to make Files a collection of its own with a serviceId field and then you'll have more power to load only certain files.
It's possible to do that.
the filter doesn't really work like projection.
try this :
datastore.createQuery(Result.class)
.field("serviceName").equal("dong")
.field("serviceVersion").equal("2.0")
.field("files.filename").equal("a.html")
.project("files.$.filename", true);

How to update names field nested in an array in mongodb

I have this object and I'd like to update the name field "field" of all the document in the collections. I read the mongodb documentation and it says $rename doesn't work in this case. I should execute a forEach but I don't know how which command use
{
"name": "foo"
"array": [
"object": {
"field": "name"
}
]
}
Do it manually:
db.collection.find().forEach(function(doc) {
if (doc.array) {
doc.array.forEach(function(edoc) {
if (edoc.object) {
doc.object.new_field = edoc.object.field
delete edoc.object.field
}
})
db.test.update({ "_id" : doc._id }, doc)
}
})
This should get you started. It handles missing or empty array arrays, but not an array value of the wrong type, or an object value of the wrong type.
$rename modifier for update Ops should work (http://docs.mongodb.org/manual/reference/operator/update/rename/)
Imagine a collection like yours:
{
"name": "foo",
"array":[
{"field": "name" }
]
}
You will be able to do something like this:
db.rename.update({},{$rename:{"name":"newName"}});
And the document will be as follows:
{
"newName": "foo",
"array":[
{"field": "name" }
]
}
In order to update all the collection you should use the multi option as follows:
db.rename.update({},{$rename:{"name":"newName"}}, {multi:true})
Regards