I've probably missed something obvious, but within the ReactiveMongo API (v0.8) how do you set a limit on the number of documents returned by a query?
I'd like to return the single most recent document added to a collection. This is my code so far:
def getLatest()(implicit reader: reactivemongo.bson.handlers.RawBSONReader[T]): Future[Option[T]] = {
collection.find (QueryBuilder(
queryDoc = Some(BSONDocument()),
sortDoc = Some(BSONDocument("_id" -> BSONInteger(-1)))
)).headOption().mapTo[Option[T]]
}
headOption() works to retrieve a single result, but I'm not explicitly using any kind of Mongo limit clause so I'm worried about this query's impact on the DB. Please help me improve this code. Thanks in advance.
In 0.8 you have to set the batchSize option to 1 in order to tell MongoDB to close the database cursor automatically:
val maybedoc = collection.find(BSONDocument(), QueryOpts().batchSize(1)).headOption
// or using QueryBuilder like you do
val maybedoc2 = collection.find (QueryBuilder(
queryDoc = Some(BSONDocument()),
sortDoc = Some(BSONDocument("_id" -> BSONInteger(-1)))
), QueryOpts().batchSize(1)).headOption()
In 0.9 collections have been refactored and greatly simplified. Now you can do this:
val maybedoc = collection.
find(BSONDocument()).
sort(BSONDocument("_id" -> -1)).
one[BSONDocument]
The one[T] method in 0.9 sets the batchSize flag for you and returns an Option[T].
Yes, the headOption() function limits the query to just one result:
def headOption()(implicit ec: ExecutionContext) :Future[Option[T]] = {
collect[Iterable](1).map(_.headOption)
}
https://github.com/zenexity/ReactiveMongo/blob/0.8/src/main/scala/api/cursor.scala#L180
Related
Using ReactiveMongo 0.11 for Scala 2.11. I have an issue where my queries are failing to descend. The following is my Index and ReactiveMongo query:
collection.indexesManager.ensure(Index(Seq("userId" -> IndexType.Ascending, "lastActivity" -> IndexType.Descending), background = true))
def listEfforts(userId: String, page: Int, pageSize: Int): Future[\/[ErrMsg, List[EffortDesc]]] = {
val query = BSONDocument("userId" -> userId)
val sort = BSONDocument("lastActivity" -> -1)
val skipN = (page - 1) * pageSize
val queryOptions = new QueryOpts(skipN = skipN, batchSizeN = pageSize, flagsN = 0)
collection.find(query).options(queryOptions).sort(sort).cursor[EffortDesc](ReadPreference.primaryPreferred).collect[List](pageSize).flatMap {
case list => Future(\/.right(list))
}
}
What's happening is my results are all ascending, even though my sort variable has been set to -1. lastActivity is a Unix timestamp field in milliseconds. I've tried other debugging issues (like recompiling, etc.)
Any idea what could be causing this? Thanks for your help!
Found the issue. If I put a IndexType.Descending on lastActivity field and then additionally sort as "descending" (via "lastActivity" -> -1) MongoDB will first return a descended sort according to its index and then sort it again.
I'm not sure if this is normal/expected behavior in Mongo but changing -1 to 1 fixed the issue.
Using either
("fieldName", BSONInteger.apply(1))
or
("fieldName", BSONInteger.apply(-1))
works for me.
I'm currently trying to create a percolator query with Elastic4s. I've got about this far but I can't seem to find any examples so I'm not sure how this quite works. So I've got:
val percQuery = percolate in esIndex / esType query myQuery
esClient.execute(percQuery)
Every time it runs it doesn't match anything. I figured out I need to be able to percolate on an Id but I can't seem to find any examples on how to do it, not even in the docs. I know with Elastic4s creating queries other than a percolator query lets you specify an id field like:
val query = index into esIndex / esType source myDoc id 12345
I've tried this way for percolate but it doesn't like the id field, does anyone know how this can be done?
I was using Dispatch Http to do this previously but I'm trying to move away from it. Before, I was doing this to submit the percolator query:
url(s"$esUrl/.percolator/$queryId)
.setContentType("application/json", "utf-8")
.setBody(someJson)
.POST
notice the queryId just need something similar to that but in elastic4s.
So you want to add a document and return the queries that are waiting for that id to be added? That seems an odd use for percolate as it will be a one time use only, as only one document can be added per id. You can't do a percolate currently on id in elastic4s, and I'm not sure if you can even do it in elasticsearch itself.
This is the best attempt I can come up with, where you have your own "id" field, which could mirror the 'proper' _id field.
object Test extends App {
import ElasticDsl._
val client = ElasticClient.local
client.execute {
create index "perc" mappings {
"idtest" as(
"id" typed StringType
)
}
}.await
client.execute {
register id "a" into "perc" query {
termQuery("id", "a")
}
}.await
client.execute {
register id "b" into "perc" query {
termQuery("id", "b")
}
}.await
val resp1 = client.execute {
percolate in "perc/idtest" doc("id" -> "a")
}.await
// prints a
println(resp1.getMatches.head.getId)
val resp2 = client.execute {
percolate in "perc/idtest" doc("id" -> "b")
}.await
// prints b
println(resp2.getMatches.head.getId)
}
Written using elastic4s 1.7.4
So after much more researching I figured out how this works with elastic4s. To do this in Elastic4s you actually have to use register instead of percolate like so:
val percQuery = register id queryId into esIndex query myQuery
This will register a percolator query at the id.
I have some conditions and a level to check while fetching records from collection using MongoDB.
For this I am using below
def cursorOutput = dataSetCollection.find(whereObject,criteriaObject)
Which is working fine. but I want to use distinct with combination to above query.
def distinctMIdList = dataSetCollection.distinct(hierarchyField,whereObject)
Above is the query for distinct. How to combine two query.
I tried below which is not working
def cursorOutput = dataSetCollection.find(whereObject,criteriaObject).distinct("manager id")
whereObject is a condition to fetch results and criteriaObject is fields to fetch.
Distinct query is giving me result with only Manager id field but I am looking for other field also(use of criteriaObject).
Finally how to combine above two query. I searched for pipeline where $distinct is not available.
Thank you.
Map function:
var mapFunction = function() {
if(/*your criteria*/) {
emit("manager_id", this.manager_id);
}
};
Reduce Function:
var reduceFunction = function(managerFiled,values){
return Array.unique(values);
}
I am migrating from the mongodb csharp driver 1.10.0 to 2.0.0.
One of the collection I am using is very big and has to fulfill many queries with different filter attributes. That is why I was relying on some index hint statements. With the v1.10 driver it looks like
myCollection.Find(query).SetHint("myIndexName");
I searched the v2 driver api but this hint method seems to be completly removed in the v2 driver. Is there an alternative? How should I do index hints with the v2 driver?
Note: The Solutions provided works for latest mongodb csharp drivers as well
You can use the FindOptions.Modifiers property.
var modifiers = new BsonDocument("$hint", "myIndexName");
await myCollection.Find(query, new FindOptions { Modifiers = modifiers }).ToListAsync();
May I ask why you are using the hint? Was the server consistently choosing the wrong index? You shouldn't need to do this except in exceptional cases.
Craig
Ideally, try to make the query in a way that mongodb optimizer can use the index automatically.
If you are using FindAsync then you will have a property named Hint. Use it like this:
If you have index named "myIndexName" which you want your query should use forcefully, then use like this:.
BsonString bsonString = new BsonString("myIndexName");
cursor = await collection.FindAsync(y => y.Population > 400000000,
new FindOptions<Person, Person>()
{
BatchSize = 200,
NoCursorTimeout = true,
AllowPartialResults = true,
Projection = "{'_id':1,'Name':1,'Population':1}"
Hint = bsonString.AsBsonValue,
}).ConfigureAwait(false);
You can fine BsonString class in MongoDB.Bson
With agregate you can force indice like this:
BsonString bsonString = new BsonString("ix_indice");
var query = this.collection.Aggregate(new AggregateOptions() { Hint = bsonString }).Match(new BsonDocument {..});
If you are using the Linq IQueryable, you can specify the hint (and other options) like this:
BsonDocument hint = new BsonDocument("myFieldName", 1);
// or
BsonDocument hint = new BsonString("myIndexName");
await collection.AsQueryable(new AggregateOptions { Hint = hint })
myFieldName can also reference a complex field, e.g. Metadata.FileName
myIndexName is the name of an index. I prefer to reference the field (first option) directly, instead of an index, for simple cases.
I am using Slick to connect to Postgres Database in our application. I have a generic filtering logic, where a Filter object will be passed from the UI, and it should return results with pagination. The Filter object should be generic so that it can be re-used.
Pseudo code of filter object is given below:
Filter = {
type: table
prop: List_of_conditions
page : 1
size : 10
}
Currently, I am building a native SQL from the Filter object and executing it. However, I am not able to use take and drop before the query is actually getting executed. It is currently getting all the results, and then dropping the unnecessary records.
I know how to do it with the slick queries, but not sure how to use pagination with the native queries?
val res = StaticQuery.queryNA[Entity](queryStr).list.drop((filter.pageNo- 1) * filter.pageSize).take(filter.pageSize)
I am using Slick 2.1
When you are using plain sql, you can't use the collection operators to build a query. You have to do it all in SQL:
val limit = filter.pageSize
val offset = (filter.pageNo- 1) * filter.pageSize
val res = StaticQuery.queryNA[Entity](queryStr ++ s" LIMIT $limit OFFSET $offset").list
I haven't tested it but i would suggest you to try to move .list call to the end
val res = StaticQuery.queryNA[Entity](queryStr).drop((filter.pageNo- 1) * filter.pageSize).take(filter.pageSize).list