MongoDB querying by sub-document value not working as expected - mongodb

I have two schemas, a Profile and a LevelOfNeed.
Profile
{
"_id" : ObjectId("56d35960a695dfa140137fca"),
. . .
"levelOfNeedServiced" : ObjectId("56d35828a695dfa140137fc7")
}
Level of Need
{
"_id" : ObjectId("56d35828a695dfa140137fc7"),
"sortOrder" : 2,
"description" : "Moderate Needs",
"additionalCost" : 3,
"__v" : 0
}
I currently have 4 documents for LevelOfNeed. What I need to do is select all of the Profile documents where the levelOfNeedServiced.sortOrder is >= a value.
Example:
db.getCollection('profiles').find({
'levelOfNeedServiced.sortOrder': { $gte: 2 }
})
Given my data, I would expect to see the example Profile, but this returns no results. What am I doing wrong?
Update 1
Previously, I was running MongoDB 3.0.9. I've since upgraded to 3.2.3, however I'm still getting the same results. According to the docs, I should be able to query on an embedded document field value.
Update 2
The aggregate function solution works as expected, but since I already had an array of LevelOfNeed objects, I was able to use that to get to the related documents I needed using the $in operator.

Unfortunately mongodb does not support joins until version 3.2. In version 3.2 it provides the $lookup aggregation operator to lookup referenced documents across collections.
You could use it as below:
db.Profile.aggregate([
{
$lookup:{
"from":"LevelOfNeed",
"localField":"levelOfNeedServiced",
"foreignField":"_id",
"as":"joined"
}
},
{
$match:{
"joined.sortOrder":{$gte:2}
}
},
{
$project:{"levelOfNeedServiced":1,...} //include things you want to project.
}
])
Your code:
db.getCollection('profiles').find({
'levelOfNeedServiced.sortOrder': { $gte: 2 }
})
does not work as intended because, the field levelOfNeedServiced is identified as a field containing an ObjectId and not the resolved LevelOfNeed document.

Related

How to get the last unique transactions from a collection in MongoDB

I have a collection that contains the following fields: agentId, postBalance, preBalance, etc. I want to fetch the last unique record for an agent that contains the field stated earlier based on a date filter.
db.transaction.find(
{
"createdAt" : {
"$gte": ISODate("2022-09-01T00:00:00Z"),
"$lt": ISODate("2022-09-02T00:00:00Z")
}
},
{
“agentId”: 1,
“walletBalance”: 1
}
)
The query above returns duplicate values and not the latest one. How best do I optimise this query. I am using Mongo Compass so I don't mind any query that comes in that format. I have read up on $last, $natural but they don't seem to solve my issue.
Have you tried to add sort by "createdAt" and limit of 1, or just using findOne method with same sort?

Sort before querying

Is it possible to run a sort on a Mongo collection before running the filtering query? I have older code in which I was using a method of getting a random result from the database by having a field which was a random float between 0 and 1, then querying with findOne to get the first document with a value greater than a random float generated at that time. The sample set was small, so didn't notice a problem at the time, but recently noticed that with one query, I was almost always getting the same value. The "first" document had a random > .9, so nearly every query matched it first.
I realized, for this solution to work, I need to sort by random, then find the first value greater than my random. As I understand it, this isn't as necessary a solution as in the past, as $sample exists as of 3.2, but I figure learning how I could do this would be good? Plus, my understanding is that $sample can return the same document multiple times (where N > 1 obviously, so not directly applicable to my question).
So for example, the following data:
> db.links.find()
{ "_id" : ObjectId("553c072bc87652a80e00002a"), "random" : 0.9162904409691691 }
{ "_id" : ObjectId("553c3332c87652c80700002a"), "random" : 0.00427396921440959 }
{ "_id" : ObjectId("553c3c5cc87652a80e00002b"), "random" : 0.2409569111187011 }
{ "_id" : ObjectId("553c3c66c876521c10000029"), "random" : 0.35101076657883823 }
{ "_id" : ObjectId("553c3c6ec87652200700002e"), "random" : 0.3234482416883111 }
{ "_id" : ObjectId("553c68d5c87652a80e00002c"), "random" : 0.5221220930106938 }
Any attempt to run db.mycollection.findOne({'random': {'$gte': x}}) where x is any value up to .91 always return the first object (_id 553c072). Anything greater returns nothing. If I could sort by the random value in ascending order then filter, it would keep searching until it found the correct value.
I would strongly recommend you to drop your custom solution and simply switch to using the MongoDB built-in $sample stage which will return a random result from your collection.
EDIT based on your comment:
Here's how you can do what you originally asked for:
db.links.find({ "random": { $gte: /* put your value here */ } })
.sort({ "random": 1 /* sort by "random" field in ascending order */ })
.limit(1)
You can, but don't need to use the aggregation framework, too:
db.links.aggregate({
$match: {
"random": {
$gte: /* put your value here */ // filter the collection
}
}
}, {
$sort: {
"random": 1 // sort by "random" field in ascending order
}
}, {
$limit: 1 // return only the first element
})

multi updating a key along the documents of a collection using pymongo

I have lots of documents inside a collection.
The structure of each of the documents inside the collection is as it follows:
{
"_id" : ObjectId(....),
"valor" : {
"AB" : {
"X" : 0.0,
"Y" : 142.6,
},
"FJ" : {
"X" : 0.2,
"Y" : 3.33
....
The collection has currently about 200 documents and I have noticed that one of the keys inside valor has the wrong name. In this case we will say "FJ" shall be "JOF" in all the docs of the collection.
Im pretty sure it is possible to change the key in all the docs using the update function of pymongo. The problem I am facing is that when I visit the online doc available https://docs.mongodb.com/v3.0/reference/method/db.collection.update/ only explains how to change the values(which I would like to remain how they currently are and change only the keys).
This is what I have tried:
def multi_update(spec_key,key_updte):
rdo=col.update((valor.spec_key),{"$set":(valor.key_updte)},multi=True)
return rdo
print(multi_update('FJ','JOF'))
But outputs name 'valor' is not defined . I thought I shall use valor.specific_key to access to the corresponding json
how can I update a key only along the docs of the collection?
You have two problems. First, valor is not an identifier in your Python code, it's a field name of a MongoDB document. You need to quote it in single or double quotes in Python in order to make it a string and use it in a PyMongo update expression.
Your second problem is, MongoDB's update command doesn't allow you set one field to the value of another, nor to rename a field. However, you can reshape all the documents in your collection using the aggregate command with a $project stage and store the results in a second collection using a $out stage.
Here's a complete example to play with:
db = MongoClient().test
collection = db.collection
collection.delete_many({})
collection.insert_one({
"valor" : {
"AB" : {
"X" : 0.0,
"Y" : 142.6,
},
"FJ" : {
"X" : 0.2,
"Y" : 3.33}}})
collection.aggregate([{
"$project": {
"valor": {
"AB": "$valor.AB",
"FOJ": "$valor.FJ"
}
}
}, {
"$out": "collection2"
}])
This is the dangerous part. First, check that "collection2" has all the documents you want, in the desired shape. Then:
collection.drop()
db.collection2.rename("collection")
import pprint
pprint.pprint(collection.find_one())

how to remove all documents from a collection except one in MongoDB

Is there any way to remove all the documents except one from a collection based on condition.
I am using MongoDB version 2.4.9
You can do this in below way,
db.inventory.remove( { type : "food" } )
Above query will remove documents with type equals to "food"
To remove document that not matches condition you can do,
db.inventory.remove( { type : { $ne: "food" } } )
or
db.inventory.remove( { type : { $nin: ["Apple", "Mango"] } } )
Check here for more info.
To remove all documents except one, we can use the query operator $nin (not in) over a specified array containing the values related to the documents that we want to keep.
db.collections.remove({"field_name":{$nin:["valueX"]}})
The advantage of $nin array is that we can use it to delete all documents except one or two or even many other documents.
To delete all documents except two:
db.collections.remove({"field_name":{$nin:["valueX", "valueY"]}})
To delete all documents except three:
db.collections.remove({"field_name":{$nin:["valueX", "valueY", "valueZ"]}})
Query
db.collection.remove({ "fieldName" : { $ne : "value"}})
As stated above by Taha EL BOUFFI, the following worked for me.
db.collection.remove({"fieldName" : { $nin: ["value"]}});

Multiple update of embedded documents' properties

I have the following collection:
{
"Milestones" : [
{ "ActualDate" : null,
"Index": 0,
"Name" : "milestone1",
"TargetDate" : ISODate("2011-12-13T22:00:00Z"),
"_id" : ObjectId("4ee89ae7e60fc615c42e28d1")},
{ "ActualDate" : null,
"Index" : 0,
"Name" : "milestone2",
"TargetDate" : ISODate("2011-12-13T22:00:00Z"),
"_id" : ObjectId("4ee89ae7e60fc615c42e28d2") } ]
,
"Name" : "a", "_id" : ObjectId("4ee89ae7e60fc615c42e28ce")
}
I want to update definite documents: that have specified _id, List of Milestones._id and ActualDate is null.
I dotnet my code looks like:
var query = Query.And(new[] { Query.EQ("_id", ObjectId.Parse(projectId)),
Query.In("Milestones._id", new BsonArray(values.Select(ObjectId.Parse))),
Query.EQ("Milestones.ActualDate", BsonNull.Value) });
var update = Update.Set("Milestones.$.ActualDate", DateTime.Now.Date);
Coll.Update(query, update, UpdateFlags.Multi, SafeMode.True);
Or in native code:
db.Projects.update({ "_id" : ObjectId("4ee89ae7e60fc615c42e28ce"), "Milestones._id" : { "$in" : [ObjectId("4ee89ae7e60fc615c42e28d1"), ObjectId("4ee89ae7e60fc615c42e28d2"), ObjectId("4ee8a648e60fc615c41d481e")] }, "Milestones.ActualDate" : null },{ "$set" : { "Milestones.$.ActualDate" : ISODate("2011-12-13T22:00:00Z") } }, false, true)
But only the first item is being updated.
This is not possible in current moment. Flag multi in update means update of multiple root documents. Positional operator can match only one nested array item. There is such feature in mongodb jira. You can vote up and wait.
Current solution can be only load document, update as you wish and save back or multiple atomic update for each nested array id.
From documentation at mongodb.org:
Currently the $ operator only applies to the first matched item in the
query
As answered by Andrew Orsich, this is not possible for the moment, at least not as you wish. But loading the document, modifying the array then saving it back will work. The risk is that some other process could modify the array in the meantime, so you would overwrite its changes. To avoid this, you can use optimistic locking, especially if the array is not modified every second.
load the document, including a new attribute: milestones_version
modify the array as needed
save back to mongodb, but now add a query constraint on the milestones_version, and increment it:
db.Projects.findAndModify({
query: {
_id: your_project_id,
milestones_version: expected_milestones_version
},
update: {
$set: {
Milestones: modified_milestones
},
$inc: {
milestones_version: 1
}
},
new: 1
})
If another process modified the milestones array (and hence the milestones_version) before we did, then this command will do nothing and simply return null. We just need to reload the document and try again. If the array is not modified every second, then this will be very rare and will not have any impact on performance.
The main problem with this solution is that you have to edit every Project, one by one (no multi: true). You could still write a javascript function and have it run on the server though.
According to their JIRA page "This new feature is available starting with the MongoDB 3.5.12 development version, and included in the MongoDB 3.6 production version"
https://jira.mongodb.org/browse/SERVER-1243