I'm trying to build a list page like the one in the "Computers" sample. My environment is Play 2.0 and PostrgreSQL 9.0
I have the following method in my User object:
def list(page: Int = 0, pageSize: Int = 10, orderBy: Int = 1, filter: String = "%"): Page[User] = {
val offset = pageSize * page
val mode = if (orderBy > 0) "ASC NULLS FIRST" else "DESC NULLS LAST"
Logger.debug("Users.list with params: page[%d] pageSize[%d] orderBy[%d] filter[%s] order[%s]".format(page, pageSize, orderBy, filter, mode))
DB.withConnection {
implicit connection =>
val users = SQL(
"""
select * from publisher
where name ilike {filter}
order by {orderBy} %s
limit {pageSize} offset {offset}
""".format(mode)
).on(
'pageSize -> pageSize,
'offset -> offset,
'filter -> filter,
'orderBy -> scala.math.abs(orderBy)
).as(User.simple *)
val totalRows = SQL(
"""
select count(*) from publisher
where name like {filter}
"""
).on(
'filter -> filter
).as(scalar[Long].single)
Page(users, page, offset, totalRows)
}
}
Doesn't matter which value of 'orderBy' I provide, the order is always based on id of the entities.
The query generated by Anorm is valid PostgreSQL and it works fine when running it against the database directly. But it seems like if Anorm parser was ignoring the order in which the results are returned, and instead returns a list ordered by 'id'.
I've even tried to simplify the query to a "select * from publisher order by 2 ASC/DESC", but nothing is fixed, the ordering is ignored by Anorm on return.
Any suggestion on how to solve this issue?
Thanks to Guillaume on the mailing list of Play I found a workaround.
All placeholders work except the one in order by. The worse part is that when you follow the logs, the driver generates the correct query and PostgreSQL is receiving it. I'm not sure what's the deal, very confusing, but if I remove that placeholder, it just works.
Depressing :(
I solved it like this:
val users = SQL(
"""
select * from publisher
where name ilike {filter}
order by %d %s
limit {pageSize} offset {offset}
""".format(scala.math.abs(orderBy), mode)
).on(
'pageSize -> pageSize,
'offset -> offset,
'filter -> filter
).as(User.simple *)
Now you'll be screaming "SQL INJECTION". Relax. Although it may be possible somehow, orderBy is an integer (which we turn into abs value for more safety). If you try to call the controller that provides orderBy with a string, Play returns a 404 error. So only integers are allowed. And if there is no column corresponding to the given integer, the order by is ignored. So, not ideal, but not so bad.
Related
Let's say I want to re-order some elements in a single-linked list of records...
The approach that came to mind is to build an array of (previous, current) id tuples and then unnest that in the FROM clause of an UPDATE statement.
I.e. something like this:
private fun updateCompositionElementOrdering(elementIdsInExpectedOrder: List<Int>) {
val PREVIOUS_ELEMENT = DSL.field("previous_element_id", Int::class.javaObjectType)
val CURRENT_ELEMENT = DSL.field("current_element_id", Int::class.javaObjectType)
val previousAndCurrent = elementIdsInExpectedOrder.mapIndexed { i, currentElement ->
val previousElementId = if (i == 0) null else elementIdsInExpectedOrder[i - 1]
DSL.row(previousElementId, currentElement)
}
ctx
.update(COMPOSITION_ELEMENT)
.set(COMPOSITION_ELEMENT.PREVIOUS_COMPOSITION_ELEMENT_ID, PREVIOUS_ELEMENT)
.from(
DSL.unnest(previousAndCurrent).`as`(DSL.name("p"), PREVIOUS_ELEMENT.unqualifiedName, CURRENT_ELEMENT.unqualifiedName)
)
.where(COMPOSITION_ELEMENT.ID.eq(CURRENT_ELEMENT))
.execute()
}
where private val ctx: DSLContext, of course.
But this results in:
jOOQ; bad SQL grammar [
update "public"."composition_element"
set
"previous_composition_element_id" = previous_element_id,
"modified_by_id" = ?
from
unnest(cast(? as any[])) as "values" (previous_element_id, current_element_id)
where "public"."composition_element"."id" = current_element_id
];
nested exception is org.postgresql.util.PSQLException: ERROR: syntax error at or near "any"
(this is with jOOQ 3.17.4 and Postgres 13.5)
jOOQ currently doesn't support unnesting arrays of rows. It probably should: https://github.com/jOOQ/jOOQ/issues/14505
... but this is a hairy thing also in PostgreSQL. For example:
select *
from unnest(array[row(1, 2), row(3, 4)]) as t (a, b)
Produces:
SQL Error [42601]: ERROR: a column definition list is required for functions returning "record"
You'd need to specify the column type:
select *
from unnest(array[row(1, 2), row(3, 4)]) as t (a integer, b integer)
There's a feature request for this, but it's not too popular, given how esoteric the feature is: https://github.com/jOOQ/jOOQ/issues/5926
Maybe, better just use DSL.values()?
DSL.values(previousAndCurrent)
.`as`(DSL.name("p"),
PREVIOUS_ELEMENT.unqualifiedName,
CURRENT_ELEMENT.unqualifiedName)
I am trying to make a complex query in swift to get data from DynamoDB.
I am able to get all information by using the userID. However there are times that I may not know the entirety of the userID and need to make a more complex query.
For instance, if I know the first name and the last name, and the user id format is "firstname:lastname:email", I need to be able to query all userID's that include the first and last name, then add a where for another column.
I am very new to dynamo and want to accomplish something like the sql query below.
SQL example:
SELECT * FROM mytable
WHERE column2 LIKE '%OtherInformation%'
AND (column1 LIKE '%lastname%' OR column1 LIKE '%firstname%')
Here is the code I have in swift4 for getting the userID if I know it exaclty, not entirely sure how to modify this for complex queries.
func queryDBForUser(Fname: String, Lname: String) {
let userId = Fname + "." + Lname + ":" + (UIDevice.current.identifierForVendor?.uuidString)!
self.UserId = userId
let objectMapper = AWSDynamoDBObjectMapper.default()
let queryExpression = AWSDynamoDBQueryExpression()
queryExpression.keyConditionExpression = "#userId = :userId"
queryExpression.expressionAttributeNames = ["#userId": "userId",]
queryExpression.expressionAttributeValues = [":userId": userId,]
objectMapper.query(CheckaraUsers.self, expression: queryExpression, completionHandler: {(response: AWSDynamoDBPaginatedOutput? ,error: Error?) -> Void in
if let error = error {
print("Amazon DynamoDB Error: \(error)")
return
}
I have also tried many variations along the lines of the following code, with no luck:
queryExpression.keyConditionExpression = "#FirstName = :firstName and #LastName = :lastName,"
queryExpression.expressionAttributeNames = ["#FirstName": "FirstName" , "#LastName": "LastName"]
queryExpression.expressionAttributeValues = [":FirstName": Fname,":LastName": Lname]
Any help would be greatly appreciated, thanks in advance!
You won't be able to do this with a DynamoDB query. When you query a table (or index) in DynamoDB you must always specify the complete primary key. In your case that would mean the full value of "firstname:lastname:email".
You could sort of do this with a DynamoDB scan and a filter expression, but that will look at every item in your table, so it could be slow and expensive. Amazon will charge you for the read capacity necessary to look at every item in the table.
So if you really wanted to, the filter expression for the scan operation would be something like:
"contains (#FirstName, :firstName) and contains (#LastName, : lastName)"
Note that contains looks for an exact substring match, so if you want case insensitive matches (like ILIKE in SQL) it won't work.
If you need to do these types of queries then you need to evaluate whether or not DynamoDB is the right choice for you. DynamoDB is a NoSQL key/value store basically. It trades limited querying functionality for scalability and performance. If you are coming at DynamoDB from a SQL background and are expecting to be able to do freeform queries of anything in your table, you will be disappointed.
Got the query working by adding a secondary index to my DynamoDB table, although this is not what I initially wanted, it still works as now I can query for a value that exists in both columns I needed, without doing a table scan and filtering after.
query code:
queryExpression.indexName = "Index-Name" queryExpression.keyConditionExpression = "#LastName = :LastName and #otherValue = :otherValue"
queryExpression.expressionAttributeNames = ["#LastName": "LastName" , "#otherValue": "otherValue"]
queryExpression.expressionAttributeValues = [":LastName": Lname,":otherValue": self.otherValue!]
I'm powering a search bar via AJAX that passes a selected filter (radio button) that relates to a database column and a search string for whatever is entered in the search bar. The scala/play/anorm code I am using is this:
def searchDB(searchString: String, filter: String): List[DatabaseResult] = {
DB.withConnection { implicit c =>
SQL(
"""
SELECT name, email, emailsecondary, picture, linkedin, title, company, companylink, companydesc, location, github, stackoverflow, twitter, blog
FROM mailinglistperson
WHERE {filter} LIKE '%{searchString}%'
""").on(
'filter -> filter,
'searchString -> searchString
).as(databaseResultParser.*)
}
}
When I run a query on the database (PostgreSQL) using psql that is isomorphic to the above anorm code, it returns 2 results, i.e.:
select id, name, email from mailinglistperson where company like '%kixer%';
But the anorm code returns 0 results when passed the exact same values (I've verified the values via println's)
EDIT: When I switch the anorm code to use String Interpolation I get:
[error] - play.core.server.netty.PlayDefaultUpstreamHandler - Cannot invoke the action
java.lang.RuntimeException: No parameter value for placeholder: 3
EDIT2: I also tried passing the '%...%' along with searchString into LIKE and still got 0 results.
There are two issues - the name of the column, and the filter value
As for the filter value: You have to omit the single ticks in the SQL command, and you should pass the placeholder "%" in the argument. The ticks are handled automatically in case of a string.
As for the column name: It's like a string parameter, so again ticks are handled automatically as well:
[debug] c.j.b.PreparedStatementHandle - select ... from ... where 'filter' like '%aaa%'
One solution: Use normal string interpolation s"""... $filter ...""".
All together:
SQL(
s"""
SELECT name, email, ...
FROM mailinglistperson
WHERE $filter LIKE {searchString}
""").on(
'searchString -> "%" + searchString + "%"
).as(databaseResultParser.*)
but that should be accompanied by a check before, something like
val validColumns = List("name", "email")
if (validColumns.contains(filter)) == false) {
throw new IllegalArgumentException("...")
}
to guard against SQL injection.
Update
As pointed out by cchantep: If Anorm >= 2.4 is used, one can use mixed interpolation (both for column names and values):
SQL"... WHERE #$filter LIKE $searchString"
In this case it's partially safe against SQL injection: that only covers the values, and not the column name.
Update 2
As for logging the SQL statements, see Where to see the logged sql statements in play2?
But as you are using PostgreSQL I suggest the definitive source: The PostgreSQL log: In postgresql.conf:
log_statement = 'all' # none, ddl, mod, all
then you will see in the PostgreSQL log file something like this:
LOG: select * from test50 where name like $1
DETAIL: Parameter: $1 = '%aaa'
I'm developing a web application based on Activator and Postgres.
I am trying to perform the following SQL query:
SELECT *
FROM table t_0
WHERE t_0.category IN (?)
then to populate the query I am using the following Scala code
val readQuery = """ SELECT * FROM table t_0 WHERE t_0.category IN (?) """
val categories = Array("free time", "living")
val insertValues = Array(categories)
val queryResult = await { connection.sendPreparedStatement(readQuery, insertValues) }
Even though there are some records in the database I always get an empty set, I have already tried with some forms of Array[Byte], but I have never managed to get results.
Does anybody have some tips or trick that I can use?
Thanks!
I'm using Scalaquery and have run into a problem when I attempt to limit my query based on a date field. I'm using Scala 2.9.2, ScalaQuery 2.9.1:0.10.0-M1 Consider the code below:
case class MyCase (opr_date: Date)
object MyClass extends BasicTable[MyCase]("MYTABLE") {
def opr_date = column[Date]("OPR_DATE")
def * = opr_date <> (MyCase, MyCase.unapply _)
def test(date: Date) = db.withSession {
logDebug("test date: " + date)
val qry = for {
d <- MyClass if (d.opr_date === date)
} yield d.opr_date
logDebug(qry.selectStatement)
qry.list
}
}
This query never returns any rows. Here is the calling code:
"The data" should {
"be available " in {
val testDate = CommonFormat.parseDate("2012-10-27", CommonFormat.EURO_SHORT).getTime
val records = MyClass.test2(new java.sql.Date(testDate))
records.size must be_>(0)
}
}
The query returns 0 rows and produces the following SQL when I print the select:
SELECT "t1"."OPR_DATE" FROM "MYTABLE" "t1" WHERE ("t1"."OPR_DATE"={d '2012-10-27'})
I have data available for the test date. If I paste the SQL into a SQL editor and edit the date so that its not the JDBC template format ('27-Oct-2012') the query returns the expected rows. Can anyone tell me what I'm doing wrong? Shouldn't this work?
I found out this morning that this was a data problem. The query works fine. It turns out I was connecting to the wrong server. We have a confusing setup of multiple environments and back-up systems that share the same database name. After connecting to the correct server the query works as expected. I saw different results between my code and the editor-tool because they were pointing at different servers (same database name ugh).Thank you to all who took time to look into this for me. I appreciate your efforts.