I'm learning mongoDB queries and have a problem given my collection looks like:
"filename": "myfile.png",
"updatedCoordinates": [
{
"xmin": 537.640869140625,
"xmax": 1049.36376953125,
"ymin": 204.90736389160156,
"ymax": 714.813720703125,
"label": "LABEL_0",
"status": "UNCHANGED"
},
{
"xmin": 76.68355560302734,
"xmax": 544.8860473632812,
"ymin": 151.90313720703125,
"ymax": 807.1371459960938,
"label": "LABEL_0",
"status": "UNCHANGED"
}],
"predictedCoordinates": [
{
"xmin": 537.640869140625,
"xmax": 1049.36376953125,
"ymin": 204.90736389160156,
"ymax": 714.813720703125,
"status": "UNCHANGED",
"label": "LABEL_0"
}
]
and the eligible values of status are: UNCHANGED, CHANGED, UNDETECTED
How would I query: Get all the in instances from the db where status == CHANGED / UNDECTED for ANY of the values inside either updatedCoordinates or predictedCoordinates ?
It means that if status of minimum of 1 entry inside either updated or predicted is set to changed or undetected, it's eligible for my query.
I tried:
{"$or":[{"updatedCoordinates.status": "CHANGED"}, {"predictedCoordinates.status": "CHANGED"}]}
With Python dict, I can query as:
def find_eligible(single_instance:dict):
for key in ["predictedCoordinates", "updatedCoordinates"]:
for i in single_instance[key]:
if i["status"] in ["CHANGED", "UNDETECTED]: return True
return False
But retrieving 400K instances first just to filter a few ones is not a good idea.
Try running this query:
db.collection.find({
"$or": [
{
"updatedCoordinates.status": {
"$in": [
"CHANGED",
"UNDETECTED"
]
}
},
{
"predictedCoordinates.status": {
"$in": [
"CHANGED",
"UNDETECTED"
]
}
}
]
})
Mongodb playground link: https://mongoplayground.net/p/Qda-G5L1mbR
Simple use of Mongo's dot notation allows access to nested values in arrays / objects, like so:
db.collection.find({
"updatedCoordinates.status": "CHANGED"
})
Mongo Playground
Related
I have a collection in MongoDB containing search history of a user where each document is stored like:
"_id": "user1"
searchHistory: {
"product1": [
{
"timestamp": 1623482432,
"query": {
"query": "chocolate",
"qty": 2
}
},
{
"timestamp": 1623481234,
"query": {
"query": "lindor",
"qty": 4
}
},
],
"product2": [
{
"timestamp": 1623473622,
"query": {
"query": "table",
"qty": 1
}
},
{
"timestamp": 1623438232,
"query": {
"query": "ike",
"qty": 1
}
},
]
}
Here _id of document acts like a foreign key to the user document in another collection.
I have backend running on nodejs and this function is used to store a new search history in the record.
exports.updateUserSearchCount = function (userId, productId, searchDetails) {
let addToSetData = {}
let key = `searchHistory.${productId}`
addToSetData[key] = { "timestamp": new Date().getTime(), "query": searchDetails }
return client.db("mydb").collection("userSearchHistory").updateOne({ "_id": userId }, { "$addToSet": addToSetData }, { upsert: true }, async (err, res) => {
})
}
Now, I want to get search history of a user based on query only using the db.find().
I want something like this:
db.find({"_id": "user1", "searchHistory.somewildcard.query": "some query"})
I need a wildcard which will replace ".somewildcard." to search in all products searched.
I saw a suggestion that we should store document like:
"_id": "user1"
searchHistory: [
{
"key": "product1",
"value": [
{
"timestamp": 1623482432,
"query": {
"query": "chocolate",
"qty": 2
}
}
]
}
]
However if I store document like this, then adding search history to existing document becomes a tideous and confusing task.
What should I do?
It's always a bad idea to save values are keys, for this exact reason you're facing. It heavily limits querying that field, obviously the trade off is that it makes updates much easier.
I personally recommend you do not save these searches in nested form at all, this will cause you scaling issues quite quickly, assuming these fields are indexed you will start seeing performance issues when the arrays get's too large ( few hundred searches ).
So my personal recommendation is for you to save it in a new collection like so:
{
"user_id": "1",
"key": "product1",
"timestamp": 1623482432,
"query": {
"query": "chocolate",
"qty": 2
}
}
Now querying a specific user or a specific product or even a query substring is all very easily supported by creating some basic indexes. an "update" in this case would just be to insert a new document which is also much faster.
If you still prefer to keep the nested structure, then I recommend you do switch to the recommended structure you posted, as you mentioned updates will become slightly more tedious, but you can still do it quite easily using arrayFilters for updating a specific element or just using $push for adding a new search
In the sample_training database, companies collection, there is data like the following one:
Exercise: How many companies in the sample_training.companies collection have offices in the city of "Seattle"?
The query I thought of was with the dot notation as follows:
db.companies.find({ "offices.0.city": "Seattle" }).count()
This returns 110.
However, the site gives the following query as the correct one
db.companies.find({ "offices": { "$elemMatch": { "city": "Seattle" } } }).count()
This returns 117.
I have checked that my query seems to work fine as well, but I don't know why they differ in their result.
The difference is you are only looking at the first element (index 0) in the array. You are specifying the index to look at.
Meaning, if we have the following database:
[
{
"offices": [
{
"city": "Dallas"
},
{
"city": "Jacksonville"
}
]
}
]
With the following query:
db.collection.find({
"offices.0.city": "Jacksonville"
})
It would return nothing.
Whereas, if we used this query, the element does not have to be the first index.
db.collection.find({
"offices": {
"$elemMatch": {
"city": "Jacksonville"
}
}
})
Live Demos:
Working - https://mongoplayground.net/p/wnX-arcooa7
Not Working - https://mongoplayground.net/p/zFWV00TzZjj
I went to the wine db - https://www.pdbmbook.com/playground/mongo/wine/view/pgdb____1635519319_617c0b57588c7
And I did:
db.products.find( { "type": "rose"}).count();
Result = 3
db.products.find({ "products.0.type": "rose" }).count();
Result: 0
db.products.find({ "products": { "$elemMatch": { "type": "rose" } } }).count()
Result: 0
I suspect I get back 0 since the online playground I used is limited in functionality. Nevertheless I would assume any field that references the index of the object e.g. "offices.0.city" would mean you are starting the count higher up the tree or at 0.
Hi I am trying to increment the count of the matching requirement in an array. My sample collection looks like the following:
{
"_id": ObjectId("60760ba2e870fa518f2ae48b"),
"userId": "6075f7289822d94dca8066b4",
"requirements": [
{
"searchText": "zee5",
"planType": "basic",
"mode": "PRIVATE",
"count": 32.0
},
{
"searchText": "sony",
"planType": "standard",
"mode": "PUBLIC",
"count": 12.0
},
{
"searchText": "prime",
"planType": "premium",
"mode": "PRIVATE",
"count": 2
}
]
}
If a user searches for prime, with filter premium and PRIVATE, then the count of the last requirement should be updated. If he searches for prime, with filter standard and PRIVATE, then the new requirement will be inserted with count 1.
I am doing in two steps. First I fire an update with the following query and then if no update, I fire a push query with count 1:
db.getCollection('userProfile').update({ "$and" : [{ "requirements.searchText" : {$eq:"prime"}}, {"requirements.mode" : {$eq: "PUBLIC"}}, {"requirements.planType": {$eq: "standard"}}, { "userId" : "6075f7289822d94dca8066b4"}]}, {$inc: {"requirements.$.count" : 1}})
I was expecting that the above query will not update any requirement, since there is no exact match. Interestingly, it increments the count of the second requirement with (sony, standard, public). What is wrong with the query? How can I get it right?
Demo - with Update - https://mongoplayground.net/p/-ISXaAayxxv
Demo No update - https://mongoplayground.net/p/88bTj3lz7U_
Use $elemMatch to make sure all properties are present in the same object inside the array
The $elemMatch operator matches documents that contain an array field with at least one element that matches all the specified query criteria.
db.collection.update(
{
"requirements": {
$elemMatch: { "searchText": "prime","mode": "PUBLIC", "planType": "standard" }
},
"userId": "6075f7289822d94dca8066b4"
},
{ $inc: { "requirements.$.count": 1 } }
)
Problem -
Your current query will match any document with all these fields in
requirements array in any object, if they match 1 property in 1 index of the array and another match in the next index query will find the document valid.
"searchText": "prime",
"mode": "PUBLIC",
"planType": "standard"
Suppose, I have a MongoDB query
db.tm_watch.count({trademark: {$in: [ObjectId('1'), ObjectId('2')]}});
that returns the count of documents that have trademark equal to 1 or 2.
I have tried this query to convert it into elasticsearch one.
es_query = {
"query": {
"bool": {
"must": [
{"terms": {"trademark": ids}},
{"term": {"team": req.user.team.id}},
],
}
}
}
esClient.count({
index: 'tm_watch',
type: 'docs',
body: es_query
}
but I don't know is this correct since I'm new to Elasticsearch.
Thanks!
The ES equivalent to mongodb's .count method is the Count API.
Assuming your index name is tm_watch and the field trademark has a .keyword multi-field mapping, you could use a terms query:
POST tm_watch/_count
{
"query": {
"terms": {
"trademark.keyword": [ "1", "2" ]
}
}
}
I am new to NoSQL and morphia. I am using Morphia to query MongoDB.
I have a sample collection as below:
[
{
"serviceId": "id1",
"serviceName": "ding",
"serviceVersion": "1.0",
"files": [
{
"fileName": "b.html",
"fileContents": "contentsA"
},
{
"fileName": "b.html",
"fileContents": "contentsB"
}
]
},
{
"serviceId": "id2",
"serviceName": "ding",
"serviceVersion": "2.0",
"files": [
{
"fileName": "b.html",
"fileContents": "contentsA"
},
{
"fileName": "b.html",
"fileContents": "contentsB"
}
]
}
]
I would like to fetch an element in "files" List , given service name, service version and filename., using Morphia.
I was able to get what I want using the query below:
db.ApiDoc.find({ serviceName: "ding", serviceVersion: "2.0"}, { files: { $elemMatch: { fileName: "b.html" } } }).sort({ "_id": 1}).skip(0).limit(30);
What I tried so far :
I tried using "elemmatch" api that morphia has, but no luck.
query = ...createQuery(
Result.class);
query.and(query.criteria("serviceName").equal("ding"),
query.criteria("serviceVersion").equal(
"2.0"));
query.filter("files elem",BasicDBObjectBuilder.start("fileName", "a.html").get());
I seem to get the entire Result collection with all the files. I would like to get only the matched files(by filename).
can some one help me how I can get this to work?
Thanks
rajesh
I don't believe it's possible to get just the matching sub element. You can request just to have the 'files' array returned but all elements will be included in the result set and you will have to refilter in your code.
The other option is to make Files a collection of its own with a serviceId field and then you'll have more power to load only certain files.
It's possible to do that.
the filter doesn't really work like projection.
try this :
datastore.createQuery(Result.class)
.field("serviceName").equal("dong")
.field("serviceVersion").equal("2.0")
.field("files.filename").equal("a.html")
.project("files.$.filename", true);