Having some issues getting ReactiveMongo 0.11 to paginate my query. The behavior is that it is returning all results, instead of the page-by-page results.
Here's my query:
def listConvos(userId: String, page: Int, pageSize: Int) = {
val query = BSONDocument("userId" -> userId)
val sort = BSONDocument("lastActivity" -> -1)
val skipN = (page-1) * pageSize
val queryOptions = new QueryOpts(skipN = skipN, batchSizeN = pageSize, flagsN = 0)
collection.find(query).options(queryOptions).sort(sort).cursor[ConvoDesc](ReadPreference.primaryPreferred).collect[List]()
Note that pagination starts at 1. Using the IntelliJ debugger, the values for the variables above are:
userId = "29aosidfj43903p"
query = BSONDocument(<non-empty>)
sort = BSONDocument(<non-empty>)
queryOptions = QueryOpts(10,10,0)
page = 2
pageSize = 10
I can also confirm I've set a compound index on the userId and lastActivity fields. Thanks for your help!
The solution was to also include the page size in the collect method like so:
collection.find(query).options(queryOptions).sort(sort).cursor[ConvoDesc](ReadPreference.primaryPreferred).collect[List](pageSize)
Related
I am trying to implement Pagination with Reactive Mongo.
def get(page: Int, pageSize: Int, filter: JsObject = Json.obj()): Future[Seq[Thing]] = {
val actualPage = if(page > 1) page else 1
collection
.find(filter)
.options(QueryOpts(skipN = (actualPage - 1) * pageSize, batchSizeN = pageSize))
.cursor[Thing]()
.collect[Seq](pageSize, Cursor.FailOnError[Seq[Thing]]())
}
This works and returns me a sequence of documents filtering by the page and pageSize. The problem is I have no idea how many results match the specified query, so I cant tell how many page's exist.
Is there a way I can also get a $count as part of the same query? Or do I need to resort to querying the database a second time with the below?
collection.count(filter)
Ideally I would change my return type to Future[Page[Seq[Thing]]] where Page would contain the total result count. e.g.
case class Page[T](results: Seq[T], count: Int)
I am using mongolite package to retrieve data from MongoDB using R script. I need one framework data to be input to other. As below
connecting mongodb
library(mongolite)
mongoSample<-mongolite::mongo(collection = "Sample1", db = "Test", url =
"mongodb://User:123#Wyyuyu:13333/ty2_U",verbose = TRUE)
mongoSample2<-mongolite::mongo(collection = "Sample2", db = "Test", url =
"mongodb://User:123#Wyyuyu:13333/ty2_U",verbose = TRUE)
getting specific data from collection below query is working fine.
D$find(query = '{"_id": "xyz"}', fields = "{\"_id\":1, \"description\":1,\"name\":1}", sort = "{}", skip = 0, limit = 0, handler = NULL, pagesize = 1000)
But I want to filter passing values from another framework.
**As in SQL
SELECT * FROM Sample1 WHERE Id IN (SELECT Id FROM Sample2)
Any help on this will be much appreciated.
Using ReactiveMongo 0.11 for Scala 2.11. I have an issue where my queries are failing to descend. The following is my Index and ReactiveMongo query:
collection.indexesManager.ensure(Index(Seq("userId" -> IndexType.Ascending, "lastActivity" -> IndexType.Descending), background = true))
def listEfforts(userId: String, page: Int, pageSize: Int): Future[\/[ErrMsg, List[EffortDesc]]] = {
val query = BSONDocument("userId" -> userId)
val sort = BSONDocument("lastActivity" -> -1)
val skipN = (page - 1) * pageSize
val queryOptions = new QueryOpts(skipN = skipN, batchSizeN = pageSize, flagsN = 0)
collection.find(query).options(queryOptions).sort(sort).cursor[EffortDesc](ReadPreference.primaryPreferred).collect[List](pageSize).flatMap {
case list => Future(\/.right(list))
}
}
What's happening is my results are all ascending, even though my sort variable has been set to -1. lastActivity is a Unix timestamp field in milliseconds. I've tried other debugging issues (like recompiling, etc.)
Any idea what could be causing this? Thanks for your help!
Found the issue. If I put a IndexType.Descending on lastActivity field and then additionally sort as "descending" (via "lastActivity" -> -1) MongoDB will first return a descended sort according to its index and then sort it again.
I'm not sure if this is normal/expected behavior in Mongo but changing -1 to 1 fixed the issue.
Using either
("fieldName", BSONInteger.apply(1))
or
("fieldName", BSONInteger.apply(-1))
works for me.
I've probably missed something obvious, but within the ReactiveMongo API (v0.8) how do you set a limit on the number of documents returned by a query?
I'd like to return the single most recent document added to a collection. This is my code so far:
def getLatest()(implicit reader: reactivemongo.bson.handlers.RawBSONReader[T]): Future[Option[T]] = {
collection.find (QueryBuilder(
queryDoc = Some(BSONDocument()),
sortDoc = Some(BSONDocument("_id" -> BSONInteger(-1)))
)).headOption().mapTo[Option[T]]
}
headOption() works to retrieve a single result, but I'm not explicitly using any kind of Mongo limit clause so I'm worried about this query's impact on the DB. Please help me improve this code. Thanks in advance.
In 0.8 you have to set the batchSize option to 1 in order to tell MongoDB to close the database cursor automatically:
val maybedoc = collection.find(BSONDocument(), QueryOpts().batchSize(1)).headOption
// or using QueryBuilder like you do
val maybedoc2 = collection.find (QueryBuilder(
queryDoc = Some(BSONDocument()),
sortDoc = Some(BSONDocument("_id" -> BSONInteger(-1)))
), QueryOpts().batchSize(1)).headOption()
In 0.9 collections have been refactored and greatly simplified. Now you can do this:
val maybedoc = collection.
find(BSONDocument()).
sort(BSONDocument("_id" -> -1)).
one[BSONDocument]
The one[T] method in 0.9 sets the batchSize flag for you and returns an Option[T].
Yes, the headOption() function limits the query to just one result:
def headOption()(implicit ec: ExecutionContext) :Future[Option[T]] = {
collect[Iterable](1).map(_.headOption)
}
https://github.com/zenexity/ReactiveMongo/blob/0.8/src/main/scala/api/cursor.scala#L180
I'm trying to build a list page like the one in the "Computers" sample. My environment is Play 2.0 and PostrgreSQL 9.0
I have the following method in my User object:
def list(page: Int = 0, pageSize: Int = 10, orderBy: Int = 1, filter: String = "%"): Page[User] = {
val offset = pageSize * page
val mode = if (orderBy > 0) "ASC NULLS FIRST" else "DESC NULLS LAST"
Logger.debug("Users.list with params: page[%d] pageSize[%d] orderBy[%d] filter[%s] order[%s]".format(page, pageSize, orderBy, filter, mode))
DB.withConnection {
implicit connection =>
val users = SQL(
"""
select * from publisher
where name ilike {filter}
order by {orderBy} %s
limit {pageSize} offset {offset}
""".format(mode)
).on(
'pageSize -> pageSize,
'offset -> offset,
'filter -> filter,
'orderBy -> scala.math.abs(orderBy)
).as(User.simple *)
val totalRows = SQL(
"""
select count(*) from publisher
where name like {filter}
"""
).on(
'filter -> filter
).as(scalar[Long].single)
Page(users, page, offset, totalRows)
}
}
Doesn't matter which value of 'orderBy' I provide, the order is always based on id of the entities.
The query generated by Anorm is valid PostgreSQL and it works fine when running it against the database directly. But it seems like if Anorm parser was ignoring the order in which the results are returned, and instead returns a list ordered by 'id'.
I've even tried to simplify the query to a "select * from publisher order by 2 ASC/DESC", but nothing is fixed, the ordering is ignored by Anorm on return.
Any suggestion on how to solve this issue?
Thanks to Guillaume on the mailing list of Play I found a workaround.
All placeholders work except the one in order by. The worse part is that when you follow the logs, the driver generates the correct query and PostgreSQL is receiving it. I'm not sure what's the deal, very confusing, but if I remove that placeholder, it just works.
Depressing :(
I solved it like this:
val users = SQL(
"""
select * from publisher
where name ilike {filter}
order by %d %s
limit {pageSize} offset {offset}
""".format(scala.math.abs(orderBy), mode)
).on(
'pageSize -> pageSize,
'offset -> offset,
'filter -> filter
).as(User.simple *)
Now you'll be screaming "SQL INJECTION". Relax. Although it may be possible somehow, orderBy is an integer (which we turn into abs value for more safety). If you try to call the controller that provides orderBy with a string, Play returns a 404 error. So only integers are allowed. And if there is no column corresponding to the given integer, the order by is ignored. So, not ideal, but not so bad.