Mongo Scala Driver: PullByFilter Based on Nested Field Value - mongodb

I have a model band that contains a list of tours
Band:
{
name: String,
email: String,
createdAt: String,
tours: Tour[],
...
}
where a Tour is:
{
name: String,
region: String,
published: Boolean,
...
}
The goal is simply to create an end point that receives a Band Name and Tour Name deletes a tour based on this input.
The following works:
bandService.getBandByName(req.getParam("bandName")).flatMap{ b =>
val tour = b.tours.filter(t => t.name == req.getParam("tourName")).head
mongoDataBaseConnector.bands.findOneAndUpdate(
equal("bandName", req.getParam("bandName")),
pull("tours", tour)
).toFuture().flatMap(u => bandService.getBandByName(req.getParam("bandName")))
However, this requires me to first resolve the band by the name received first, filter, find the tour and pass in the exact object in to the pull I am trying to avoid this by using pullByFilter but can't seem to get this to work. Unfortunately couldn't find any examples of this function in the scala driver.
This is what I am trying:
mongoDataBaseConnector.bands.findOneAndUpdate(
and(
equal("bandName", req.getParam("bandName")),
equal("tours.name", req.getParam("tourName"))),
pullByFilter(and(
equal("tours.$.name", req.getParam("tourName")),
equal("tours.$.region", req.getParam("region"))
))
).toFuture().flatMap(u => bandService.getBandByName(req.getParam("bandName")))
this gives the following error:
com.mongodb.MongoCommandException: Command failed with error 2 (BadValue): 'Cannot apply $pull to a non-array value' on server cluster0-shard-00-01-sqs4t.mongodb.net:27017. The full response is {"operationTime": {"$timestamp": {"t": 1568589476, "i": 8}}, "ok": 0.0, "errmsg": "Cannot apply $pull to a non-array value", "code": 2, "codeName": "BadValue", "$clusterTime": {"clusterTime": {"$timestamp": {"t": 1568589476, "i": 8}}, "signature": {"hash": {"$binary": "Qz/DqAdG11H8KRkW8gtvRAAE61Q=", "$type": "00"}, "keyId": {"$numberLong": "6710077417139994625"}}}}
Any ideas are appreciated. Is this even possible with this function?

Okay, I've been working on a nearly identical problem all day. What I have isn't great, but it's enough that I'm willing to accept it and hopefully someone will be able to make it into a more fully-formed solution. Or at least I can save someone else some time.
pullByFilter only accepts one argument. It seems that argument can be another Bson filter which is applied directly to each of the children, or a BsonValue that will be matched against. Since none of the Bson filters step into the child documents, you have to create a BsonValue, specifically, a document. I believe this should solve your problem:
mongoDataBaseConnector.bands.findOneAndUpdate(
and(
equal("bandName", req.getParam("bandName")),
equal("tours.name", req.getParam("tourName"))),
pullByFilter(BsonDocument().append("tours", BsonDocument()
.append("name", req.getParam("tourName"))
.append("region", req.getParam("region"))
))
).toFuture().flatMap(u => bandService.getBandByName(req.getParam("bandName")))
I would have liked to be able to switch back using Bson filters after getting to the inner document, but it seems that once you're in BsonValues, you can't go back.
Edit: To help whoever comes after me: I think there should be a way to solve this more cleanly with arrayFilters, but I haven't yet found it.

Related

Mongodb create index for boolean and integer fields

user collection
[{
deleted: false,
otp: 3435,
number: '+919737624720',
email: 'Test#gmail.com',
name: 'Test child name',
coin: 2
},
{
deleted: false,
otp: 5659,
number: '+917406732496',
email: 'anand.satyan#gmail.com',
name: 'Nivaan',
coin: 0
}
]
I am using below command to create index Looks like for string it is working
But i am not sure this is correct for number and boolean field.
db.users.createIndex({name:"text", email: "text", coin: 1, deleted: 1})
I am using this command to filter data:
db.users.find({$text:{$search:"anand.satya"}}).pretty()
db.users.find({$text:{$search:"test"}}).pretty()
db.users.find({$text:{$search:2}}).pretty()
db.users.find({$text:{$search:false}}).pretty()
string related fields working. But numeric and boolean fields are not working.
Please check how i will create index for them
The title and comments in this question are misleading. Part of the question is more focused on how to query with fields that contain boolean and integer fields while another part of the question is focused on overall indexing strategies.
Regarding indexing, the index that was shown in the question is perfectly capable of satisfying some queries that include predicates on coin and deleted. We can see that when looking at the explain output for a query of .find({$text:{$search:"test"}, coin:123, deleted: false}):
> db.users.find({$text:{$search:"test"}, coin:123, deleted: false}).explain().queryPlanner.winningPlan.inputStage
{
stage: 'FETCH',
inputStage: {
stage: 'IXSCAN',
filter: {
'$and': [ { coin: { '$eq': 123 } }, { deleted: { '$eq': false } } ]
},
keyPattern: { _fts: 'text', _ftsx: 1, coin: 1, deleted: 1 },
indexName: 'name_text_email_text_coin_1_deleted_1',
isMultiKey: false,
isUnique: false,
isSparse: false,
isPartial: false,
indexVersion: 2,
direction: 'backward',
indexBounds: {}
}
}
Observe here that the index scan stage (IXSCAN) is responsible for providing the filter for the coin and deleted predicates (as opposed to the database having to do that after FETCHing the full document.
Separately, you mentioned in the question that these two particular queries aren't working:
db.users.find({$text:{$search:2}}).pretty()
db.users.find({$text:{$search:false}}).pretty()
And by 'not working' you are referring to the fact that no results are being returned. This is also related to the following discussion in the comments which seemed to have a misleading takeaway:
You'll have to convert your coin and deleted fields to string, if you want it to be picked up by $search – Charchit Kapoor
So. There is no way for searching boolean or integger field. ? – Kiran S youtube channel
Nope, not that I know of. – Charchit Kapoor
You can absolutely use boolean and integer values in your query predicate to filter data. This playground demonstrates that.
What #Charchit Kapoor is mentioning that can't be done is using the $text operator to match and return results whose field values are not strings. Said another way, the $text operator is specifically used to perform a text search.
If what you are trying to achieve are direct equality matches for the field values, both strings and otherwise, then you can delete the text index as there is no need for using the $text operator in your query. A simplified query might be:
db.users.find({ name: "test"})
Demonstrated in this playground.
A few additional things come to mind:
Regarding indexing overall, databases will generally consider using an index if the first key is used in the query. You can read more about this for MongoDB specifically on this page. The takeaway is that you will want to create the appropriate set of indexes to align with your most commonly executed queries. If you have a query that just filters on coin, for example, then you may wish to create an index that has coin as its first key.
If you want to check if the exact string value is present in multiple fields, then you may want to do so using the $or operator (and have appropriate indexes for the database to use).
If you do indeed need more advanced text searching capabilities, then it would be appropriate to either continue using the $text operator or consider Atlas Search if the cluster is running in Atlas. Doing so does not prevent you from also having indexes that would support your other queries, such as on { coin: 2 }. It's simply that the syntax for performing such a query needs to be updated.
There is a lot going on here, but the big takeaway is that you can absolutely filter data based on any data type. Doing so simply requires using the appropriate syntax, and doing so efficiently requires an appropriate indexing strategy to be used along side of the queries.

Mongoose Model.deleteMany() only deletes first element of matches

I'm trying to use the Model.deleteMany() function from mongoose. I'm trying to do something like this:
MyModel.deleteMany({"_id": {$in: ['objectid 1', 'objectid 2'
But this only deletes the first element of the matches from DB (in this case, if 'objectid 1' exists in the DB, it deletes that, but if it isn't nothing happens (both n and deletedCount is 0 in the returned data from the function). I tried using something other than the _id as a query, and this worked. If I had three elements with the same 'name' field, I could delete these.
I tried _id both with and without quotation marks. I also tried converting the object id strings to actual object ids before passing them to deleteMany, but this had no difference either. I have also of course tried to google this, but everything I've found are examples of usage, where it looks like I'm doing the exact same thing as the various blog posts.
I haven't added much code here because I don't really see what else I could be adding. I'm already printing out the input to the $in object, and this is correct. The strangest thing, I think, is that the first element of the list is deleted. Is it treated as a deleteOne request for some reason? are there any config options I need?
As per request, I've added the query and the documents I'd hope to delete:
//Request
MemberModel.deleteMany({"_id": {$in: [
5ee4f6308631dc413c7f04b4,
5ee4f6308631dc413c7f04b5,
5ee4f6308631dc413c7f04b6
]}};
//Expected to be deleted
[
{
"_id": "5ee4f62f8631dc413c7f04b5",
"firstName": "Name",
"lastName": "Nameson",
"__v": 0
},
{
"_id": "5ee4f62f8631dc413c7f04b6",
"firstName": "Other",
"lastName": "Person",
"__v": 0
}
]
If you have any ideas for what I could try, that would be much appreciated.

How to filter by count on gremlin map (OrientDB)

I have a somewhat similar business problem to - Gremlin filter by count
, but I'm running on OrientDB 3.0.16
This query:
g.V().hasLabel('skill').
groupCount()
Returns from OrientDB, as expected:
{
"result": [
{
"com": 1,
"netcompactframework": 1,
"netremoting": 2,
"netframework": 3,
"net": 1,
"netclr": 1
}
],
"elapsedMs": 18
}
I tried to apply an unfold and where filter after it:
g.V().hasLabel('skill').
groupCount().
unfold().
where(select(values).is(gt(1)))
But I get an error:
{
"errors": [
{
"reason": 501,
"code": 501,
"content": "java.lang.UnsupportedOperationException: Cannot convert netremoting=2 - class java.util.HashMap$Node to JSON"
}
]
}
It seems that problem is with unfold() as OrientDB is trying to convert the map entry string into JSON and fails
Any ideas?
Is this an OrientDB specific issue? Maybe there is another way to perform the same logic in gremlin?
That looks like a serialization error of some sort, but I'm not sure of the circumstance under which you are operating to get that problem. When you unfold() a Map, it gets converted to Java Map.Entry and returning that seems to be a problem for the serializer which along the way encounters the internal class HashMap$Node. I think you can work around this problem by folding back to a Map:
g.V().hasLabel('skill').
groupCount().
unfold().
where(select(values).is(gt(1))).
group().
by(keys).
by(select(values))
I would be curious to know what circumstances cause you to get that error. Standard GraphSON serializers in Gremlin Server should be able to handle HashMap$Node so it's curious that you'd be getting this problem at all.

API $expand and &count

Is it possible to use $expand but instead of returning a collection of objects, just return the count of objects?
For example, get an account and a count of its annotations in a single WebApi call
I've tried a few things.
Obvious attempt: accounts(6CDEEB72-2AC8-E711-A825-000D3AE0A7F8)?$select=name&$expand=Account_Annotation($count=true) returns all fields of all Annotations but doesn't count anything.
Next I tried accounts(6CDEEB72-2AC8-E711-A825-000D3AE0A7F8)?$select=name&$expand=Account_Annotation($select=annotationid&$count=true) returns an error: "Found an unbalanced bracket expression". I think this is related to the & symbol in the $expand
I found a non-crm blog that said this could be resolved with a ; but when I tried accounts(6CDEEB72-2AC8-E711-A825-000D3AE0A7F8)?$select=name&$expand=Account_Annotation($select=annotationid;$count=true) it doesn't give an error, but the $count instruction seems to be ignored
A crazy attempt of accounts(6CDEEB72-2AC8-E711-A825-000D3AE0A7F8)?$select=name&$count=Account_Annotation($select=annotationid) returns a "not a valid count" error
I'm guessing that this is not a valid combination, but I thought I would post here in case someone else has achieved this successfully.
I got this working, but I'm not sure it's what you are looking for exactly, it only works if I put the count before expand:
api/data/v9.0/cmnt_salesexample?
$select=endcustomeraccountid&$count=true&$expand=endcustomerid($select=accountid)
output:
{
"#odata.context": "https://xyz",
"#odata.count": 5000,
"value": [
{
"#odata.etag": "W/\"3560581422\"",
"endcustomerid": "54321"
},
{
"#odata.etag": "W/\"3510396844\"",
"endcustomerid": "12345"
},
...

What does the "+" mean in Mongodb for comparisons?

so I'm trying to learn mongodb via nodeschool with local servers and I noticed in my exercises, that if I don't prefix the variable I am comparing with a '+' character, I don't get anything returned to me.
Here is the actual function in question, arg is first argument from the command line entered by the user:
parrots.find({
age: {$gt: arg}
},
{name: 1, age: 1, _id: 0}).toArray(function(err, documents) {
if (err) {
throw err
}
console.log(documents);
db.close();
});
however if I simply edit the line where I check if age is greater than the age provided by the user to
age: {$gt: +arg}
suddenly I get an JSON object returned to me that was within the database.
I'm just not sure why age: {$gt: arg} doesn't work since even in the documentation for the find method, there was no example that used +
Thank you
The arguments as they come from the command line are Strings, the +arg syntax is a shorthand to cast that string to a JavaScript Number. It is not specific to Mongo, just a JS idiosyncrasy.