Morphia query to filter and fetch an embedded list element - mongodb

I am new to NoSQL and morphia. I am using Morphia to query MongoDB.
I have a sample collection as below:
[
{
"serviceId": "id1",
"serviceName": "ding",
"serviceVersion": "1.0",
"files": [
{
"fileName": "b.html",
"fileContents": "contentsA"
},
{
"fileName": "b.html",
"fileContents": "contentsB"
}
]
},
{
"serviceId": "id2",
"serviceName": "ding",
"serviceVersion": "2.0",
"files": [
{
"fileName": "b.html",
"fileContents": "contentsA"
},
{
"fileName": "b.html",
"fileContents": "contentsB"
}
]
}
]
I would like to fetch an element in "files" List , given service name, service version and filename., using Morphia.
I was able to get what I want using the query below:
db.ApiDoc.find({ serviceName: "ding", serviceVersion: "2.0"}, { files: { $elemMatch: { fileName: "b.html" } } }).sort({ "_id": 1}).skip(0).limit(30);
What I tried so far :
I tried using "elemmatch" api that morphia has, but no luck.
query = ...createQuery(
Result.class);
query.and(query.criteria("serviceName").equal("ding"),
query.criteria("serviceVersion").equal(
"2.0"));
query.filter("files elem",BasicDBObjectBuilder.start("fileName", "a.html").get());
I seem to get the entire Result collection with all the files. I would like to get only the matched files(by filename).
can some one help me how I can get this to work?
Thanks
rajesh

I don't believe it's possible to get just the matching sub element. You can request just to have the 'files' array returned but all elements will be included in the result set and you will have to refilter in your code.
The other option is to make Files a collection of its own with a serviceId field and then you'll have more power to load only certain files.

It's possible to do that.
the filter doesn't really work like projection.
try this :
datastore.createQuery(Result.class)
.field("serviceName").equal("dong")
.field("serviceVersion").equal("2.0")
.field("files.filename").equal("a.html")
.project("files.$.filename", true);

Related

How to use nested query using &or with &any in mongodb?

I'm learning mongoDB queries and have a problem given my collection looks like:
"filename": "myfile.png",
"updatedCoordinates": [
{
"xmin": 537.640869140625,
"xmax": 1049.36376953125,
"ymin": 204.90736389160156,
"ymax": 714.813720703125,
"label": "LABEL_0",
"status": "UNCHANGED"
},
{
"xmin": 76.68355560302734,
"xmax": 544.8860473632812,
"ymin": 151.90313720703125,
"ymax": 807.1371459960938,
"label": "LABEL_0",
"status": "UNCHANGED"
}],
"predictedCoordinates": [
{
"xmin": 537.640869140625,
"xmax": 1049.36376953125,
"ymin": 204.90736389160156,
"ymax": 714.813720703125,
"status": "UNCHANGED",
"label": "LABEL_0"
}
]
and the eligible values of status are: UNCHANGED, CHANGED, UNDETECTED
How would I query: Get all the in instances from the db where status == CHANGED / UNDECTED for ANY of the values inside either updatedCoordinates or predictedCoordinates ?
It means that if status of minimum of 1 entry inside either updated or predicted is set to changed or undetected, it's eligible for my query.
I tried:
{"$or":[{"updatedCoordinates.status": "CHANGED"}, {"predictedCoordinates.status": "CHANGED"}]}
With Python dict, I can query as:
def find_eligible(single_instance:dict):
for key in ["predictedCoordinates", "updatedCoordinates"]:
for i in single_instance[key]:
if i["status"] in ["CHANGED", "UNDETECTED]: return True
return False
But retrieving 400K instances first just to filter a few ones is not a good idea.
Try running this query:
db.collection.find({
"$or": [
{
"updatedCoordinates.status": {
"$in": [
"CHANGED",
"UNDETECTED"
]
}
},
{
"predictedCoordinates.status": {
"$in": [
"CHANGED",
"UNDETECTED"
]
}
}
]
})
Mongodb playground link: https://mongoplayground.net/p/Qda-G5L1mbR
Simple use of Mongo's dot notation allows access to nested values in arrays / objects, like so:
db.collection.find({
"updatedCoordinates.status": "CHANGED"
})
Mongo Playground

Search and update in array of objects MongoDB

I have a collection in MongoDB containing search history of a user where each document is stored like:
"_id": "user1"
searchHistory: {
"product1": [
{
"timestamp": 1623482432,
"query": {
"query": "chocolate",
"qty": 2
}
},
{
"timestamp": 1623481234,
"query": {
"query": "lindor",
"qty": 4
}
},
],
"product2": [
{
"timestamp": 1623473622,
"query": {
"query": "table",
"qty": 1
}
},
{
"timestamp": 1623438232,
"query": {
"query": "ike",
"qty": 1
}
},
]
}
Here _id of document acts like a foreign key to the user document in another collection.
I have backend running on nodejs and this function is used to store a new search history in the record.
exports.updateUserSearchCount = function (userId, productId, searchDetails) {
let addToSetData = {}
let key = `searchHistory.${productId}`
addToSetData[key] = { "timestamp": new Date().getTime(), "query": searchDetails }
return client.db("mydb").collection("userSearchHistory").updateOne({ "_id": userId }, { "$addToSet": addToSetData }, { upsert: true }, async (err, res) => {
})
}
Now, I want to get search history of a user based on query only using the db.find().
I want something like this:
db.find({"_id": "user1", "searchHistory.somewildcard.query": "some query"})
I need a wildcard which will replace ".somewildcard." to search in all products searched.
I saw a suggestion that we should store document like:
"_id": "user1"
searchHistory: [
{
"key": "product1",
"value": [
{
"timestamp": 1623482432,
"query": {
"query": "chocolate",
"qty": 2
}
}
]
}
]
However if I store document like this, then adding search history to existing document becomes a tideous and confusing task.
What should I do?
It's always a bad idea to save values are keys, for this exact reason you're facing. It heavily limits querying that field, obviously the trade off is that it makes updates much easier.
I personally recommend you do not save these searches in nested form at all, this will cause you scaling issues quite quickly, assuming these fields are indexed you will start seeing performance issues when the arrays get's too large ( few hundred searches ).
So my personal recommendation is for you to save it in a new collection like so:
{
"user_id": "1",
"key": "product1",
"timestamp": 1623482432,
"query": {
"query": "chocolate",
"qty": 2
}
}
Now querying a specific user or a specific product or even a query substring is all very easily supported by creating some basic indexes. an "update" in this case would just be to insert a new document which is also much faster.
If you still prefer to keep the nested structure, then I recommend you do switch to the recommended structure you posted, as you mentioned updates will become slightly more tedious, but you can still do it quite easily using arrayFilters for updating a specific element or just using $push for adding a new search

Get data from list of Sub documents in Mongo collection using Spring JPA

Below is the sample of my Mongo-collection data-structure
{
"id": "5d91fe25da1917111182ce5a",
"customName": "Chess Application",
"status":"not_ready",
"environments": [
{
"environmentId": "6bbbbda6-b01a-4b9e-99d5-a1d0f696449a",
"environmentName": "Dev",
"environmentType": "dev",
},
{
"environmentId": "3b958d27-8fb7-4edd-bbb0-1dd86437d313",
"environmentName": "qa",
"environmentType": "qa",
}
]
}
Am using spring-JPA to get the data.. I will get only the environmentId as input and i will scan all the collections and get the collection that has this environmentId
Note: the Environment-id here is not mongo-created ID. It is the UUID generated by my Java app during insertion
i used findByEnvironmentsIsIn() method and it is not helpful . Any idea on how to get only one object from the list-of-sub-documents ?
#Query("{'environments' : { $elemMatch: { 'environmentId': { $in: ?0 }}}}")
List<Object> findByEnvironmentsIsIn( Set<String> environmentIds);
I guess this should work for you

How to search through a list of objects nested inside an array with Spring data MongoDB?

I have got a collection of documents and each documents contains a nested array of objects.
{
"id": "309324739",
"debters": [
{
"user": {
"name": "John Doe",
"internal": true
},
"debt": 1463,
},
{
"user": {
"name": "Alex Tree",
"internal": false
},
"debt": 53443,
},
}
What I'm trying to do is to return find the document by id and then find inside the debters list that has a false flag?
I tried the following query...
Debters findByIdAndDebters_User_InternalIsFalse(#Param("id") String id,);
But I'm getting an error saying that it can find "internal" property. What am I doing wrong and how can I loop through array using this magic mongo repository query?
you need to write a native query for that which is similar to
#Query("{'debters.user.internal':false,'_id':''}")
Debters findByIdAndDebtersUserInternalIsFalse(#Param("id") String id,);

sailsjs array query Exact match

i want to query mongodb in sailsjs.
this is structure of my db
{
"users": [
"52ed09e1d015533c124015d5",
"52ed4bc75ece1fb013fed7f5"
],
"user_msgs": [
{
"sender": "52ed09e1d015533c124015d5",
"sendTo": "52ed4bc75ece1fb013fed7f5",
"msg": "ss"
}
],
"createdAt": ISODate("2014-02-06T16:12:17.751Z"),
"updatedAt": ISODate("2014-02-06T16:12:17.751Z"),
"_id": ObjectID("52f3b461f46da23c111582f6")
}
I want to search those "users" who who match array [
"52ed09e1d015533c124015d5",
"52ed4bc75ece1fb013fed7f5"
]
Message.find({user: ["52ed09e1d015533c124015d5","52ed4bc75ece1fb013fed7f5"]})
this query returns all objects which contains 1 OR 2 ..but i need only those which exacly match 1 AND 2,
i have also tried $all, and etc.. but did not worked
please tell me how to write query with sailsjs supported syntex to get those user
You'll need to use the native Mongo adapter for this:
Message.native(function(err, collection) {
collection.find({users:{'$all':["52ed09e1d015533c124015d5","52ed4bc75ece1fb013fed7f5"]}}).toArray(function(err, results) {
// Do something with results
});
});
Message.find()
.where({users: "52ed09e1d015533c124015d5", "52ed4bc75ece1fb013fed7f5"})
.exec(function(){
// do something
});
While the above code may work to pull in just those users. I think a better solution would be to define your user ID's in your message model.
I would add the following attributes to your messages model:
senderID: {
type: 'string'
},
receiverID: {
type: 'string'
}
Now you can make that query more efficient by using the following query:
Message.find()
.where({senderID: "52ed09e1d015533c124015d5"})
.where({receiverID: "52ed4bc75ece1fb013fed7f5"})
.exec(function(){
// do something
});
This is the route I would take.