Using Open JPA 2.0, Database is DB2 9.7. For query like SELECT COUNT(1) FROM USER WHERE FNAME := fname, JPA is converting the query to SELECT COUNT(CAST(? AS BIGINT)) FROM TABLENAME.
How to avoid the CAST to BIGINT?
Code sample below:
query = entityManager.createNamedQuery("qry.checkuser");
query.setParameter("fname", fname);
Long count = (Long)query.getSingleResult();
Which one is the problem in the CAST?
I think you can't avoid it since is SQL generated by the JPA provider.
BTW, I allways use Number super class instead of specific subclass:
query = entityManager.createNamedQuery("qry.checkuser", Number.class);
query.setParameter("fname", fname);
Number count = query.getSingleResult();
// Do whatever is needed
if (count.longValue()...
This way there's no problem if the JPA provider returns a integer, long or BigXXXX.
What worked for me is:
SELECT COUNT(USER_ID) FROM USER WHERE FNAME := fname
Basically we need to use a non nullable column like Primary key column and with this change the CAST can be avoided which consumes additional CPU, a minor gain.
Related
I'm trying to parameterize my postgresql query in order to prevent SQL injection in my ruby on rails application. The SQL query will sum a different value in my table depending on the input.
Here is a simplified version of my function:
def self.calculate_value(value)
calculated_value = ""
if value == "quantity"
calculated_value = "COALESCE(sum(amount), 0)"
elsif value == "retail"
calculated_value = "COALESCE(sum(amount * price), 0)"
elsif value == "wholesale"
calculated_value = "COALESCE(sum(amount * cost), 0)"
end
query = <<-SQL
select CAST(? AS DOUBLE PRECISION) as ? from table1
SQL
return Table1.find_by_sql([query, calculated_value, value])
end
If I call calculate_value("retail"), it will execute the query like this:
select location, CAST('COALESCE(sum(amount * price), 0)' AS DOUBLE PRECISION) as 'retail' from table1 group by location
This results in an error. I want it to execute without the quotes like this:
select location, CAST(COALESCE(sum(amount * price), 0) AS DOUBLE PRECISION) as retail from table1 group by location
I understand that the addition of quotations is what prevents the sql injection but how would I prevent it in this case? What is the best way to handle this scenario?
NOTE: This is a simplified version of the queries I'll be writing and I'll want to use find_by_sql.
Prepared statement can not change query structure: table or column names, order by clause, function names and so on. Only literals can be changed this way.
Where is SQL injection? You are not going to put a user-defined value in the query text. Instead, you check the given value against the allowed list and use only your own written parts of SQL. In this case, there is no danger of SQL injection.
I also want to link to this article. It is safe to create a query text dynamically if you control all parts of that query. And it's much better for RDBMS than some smart logic in query.
I'm transforming some old PostgreSQL code to jOOQ, and I'm currently struggling with SQL that has multiple WITH clauses, where each one depends on previous. It would be best to keep the SQL logic the way it was written and not to change it (e.g. multiple queries to DB).
As it seems, there is no way to do SELECT on something that is UPDATE ... RETURNING, for example
dsl.select(DSL.asterisk())
.from(dsl.update(...)
.returning(DSL.asterisk())
)
I've created some test tables, trying to create some sort of MVCE:
create table dashboard.test (id int primary key not null, data text); --test table
with updated_test AS (
UPDATE dashboard.test SET data = 'new data'
WHERE id = 1
returning data
),
test_user AS (
select du.* from dashboard.dashboard_user du, updated_test -- from previous WITH
where du.is_active AND du.data = updated_test.data
)
SELECT jsonb_build_object('test_user', to_jsonb(tu.*), 'updated_test', to_jsonb(ut.*))
FROM test_user tu, updated_test ut; -- from both WITH clauses
So far this is my jOOQ code (written in Kotlin):
dsl.with("updated_test").`as`(
dsl.update(Tables.TEST)
.set(Tables.TEST.DATA, DSL.value("new data"))
.returning(Tables.TEST.DATA) //ERROR is here: Required Select<*>, found UpdateResultStep<TestRecord>
).with("test_user").`as`(
dsl
.select(DSL.asterisk())
.from(
Tables.DASHBOARD_USER,
DSL.table(DSL.name("updated_test")) //or what to use here?
)
.where(Tables.DASHBOARD_USER.IS_ACTIVE.isTrue
.and(Tables.DASHBOARD_USER.DATA.eq(DSL.field(DSL.name("updated_test.data"), String::class.java)))
)
)
.select() //here goes my own logic for jsonBBuildObject (which is tested and works for other queries)
.from(
DSL.table(DSL.name("updated_test")), //what to use here
DSL.table(DSL.name("test_user")) //or here
)
Are there any workarounds for this? I'd like to avoid changing SQL if possible.
Also, in this project this trick is used very often to get JSON(B) from UPDATE clause (table has JSON(B) columns too):
with _updated AS (update dashboard.test SET data = 'something' WHERE id = 1 returning *)
select to_jsonb(_updated.*) from _updated;
and it will be a real step back for us if there is no workaround for this.
I'm using JOOQ version 3.13.3, and Postgres 12.0.
This is currently not supported in jOOQ, see:
https://github.com/jOOQ/jOOQ/issues/3185
https://github.com/jOOQ/jOOQ/issues/4474
The workaround is, as always, when some vendor specific syntax is unsupported, to resort to plain SQL templating
E.g.
// If you don't need to map data types
dsl.fetch("with t as ({0}) {1}", update, select);
// If you need to map data types
dsl.resultQuery("with t as ({0}) {1}", update, select).coerce(select.getSelect()).fetch();
In a SQL Server database I have a column of decimal datatype defined something like this:
CREATE TABLE MyTable
(
Id INT,
Number DECIMAL(9, 4)
)
I use Entity Framework and I would like to return column Number converted to a string with only the digits right of the decimal separator that are actually needed. A strict constraint is that a result must be an IQueryable.
So my query is:
IQueryable queryable = (
from myTable in MyDatabase.NyTable
select new
{
Id = myTable.Id,
Number = SqlFunctions.StringConvert(myTable.Number,9,4)
}
);
The problem with is that it always convert number to string with 4 decimals, even if they are 0.
Examples:
3 is converted to "3.0000"
1.2 is converted to "1.2000"
If I use other parameters for StringConvert i.e.
SqlFunctions.StringConvert(myTable.Number, 9, 2)
the results are also not OK:
0.375 gets rounded to 0.38.
StringConvert() function is translated into SQL Server function STR.
https://learn.microsoft.com/en-us/sql/t-sql/functions/str-transact-sql?view=sql-server-2017
This explains the weird results.
In the realm of Entity Framework and LINQ I was not able to find a working solution.
What I look for is something like C# function
String.Format("0.####", number)
but this cannot be used in a LINQ query.
In plain simple SQL I could write my query like this
SELECT
Id,
Number = CAST(CAST(Number AS REAL) AS VARCHAR(15))
FROM
MyTable
I have not managed to massage LINQ to produce query like that.
A workaround would be to forget doing this in LINQ, which is quite inflexible and messy thing, borderline on useless and just return type DECIMAL from database and do my formatting on a client side before displaying. But this is additional, unnecessary code and I would hate to di it that way if there perhaps is a simpler way via LINQ.
Is it possible to format numbers in LINQ queries?
I would absolutely return a decimal from he database and format it when needed. Possible directly after the query. But usually this is done at display time to take into account culture specific formatting from the the client.
var q =
(from myTable in MyDatabase.NyTable
select new
{
Id = myTable.Id,
Number = myTable.Number
})
.AsEnumerable()
.Select(x => new { Id = x.Id, Number = x.Number.ToString("G29") });
In a Spring Boot application, I have a SQL query that is executed on a postgresql server as follows :
#Query(value = "select count(*) from servers where brand= coalesce(?1, brand) " +
"and flavour= coalesce(?2, flavour) ; ",
nativeQuery = true)
Integer icecreamStockCount(String country, String category);
However,
I get the following error when I execute the method :
ERROR: COALESCE types bytea and character varying in PostgreSQL
How do I pass String value = null to the query?
**NOTE : ** I found that my question varied from JPA Query to handle NULL parameter value
You need not coalesce, try this
#Query("select count(*) from servers where (brand = ?1 or ?1 is null)" +
" and (flavour = ?2 or ?2 is null)")
Integer icecreamStockCount(String country, String category);
When I encounted this error, I ended up using a combination of OR and CAST to solve the issue.
SELECT COUNT(*)
FROM servers
WHERE (?1 IS NULL OR brand = CAST(?1 AS CHARACTER VARYING))
AND (?2 IS NULL OR flavour = CAST(?2 AS CHARACTER VARYING))
This works even if ?1, ?2, brand and flavor are all nullable fields.
Note that passing null for ?1 means "all servers regardless of brand" rather than "all servers without a brand". For the latter, you could use IS DISTINCT FROM as follows.
SELECT COUNT(*)
FROM servers
WHERE (CAST(?1 AS CHARACTER VARYING) IS NOT DISTINCT FROM brand)
AND (CAST(?2 AS CHARACTER VARYING) IS NOT DISTINCT FROM flavour)
Finally, certain parameter types such as Boolean cannot be cast in SQL from BYTEA to BOOLEAN, for those cases you need a double cast:
SELECT COUNT(*)
FROM servers
WHERE (?1 IS NULL OR is_black = CAST(CAST(?1 AS CHARACTER VARYING) AS BOOLEAN))
In my eyes this is a problem in Hibernate which could be solved by passing Java null parameters as plain SQL NULLs rather than interpreting null as a value of type BYTEA.
If you really need to use native query, there is a problem because it's an improvement not implemented yet, see hibernate. If you don't need to use native you can do (where ?1 is null or field like ?1). Assuming you do need native,
you may treat the String before by setting this empty and then calling the repository and this one would be like:
#Query(value = "select count(*) from servers where (?1 like '' or brand like ?1) " +
"and (?2 like '' or flavour like ?2)",
nativeQuery = true)
Integer icecreamStockCount(String country, String category);
There is always javax.persistence.EntityManager bean as option for native query situations and I recommend it instead of previous approach. Here you can append to your query the way you want, as follows:
String queryString = "select count(*) from servers ";
if (!isNull(country)) queryString += "where brand like :country";
Query query = entityManager.createNativeQuery(queryString);
if (!isNull(country)) query.setParameter("country", country);
return query.getResultList();
Observations:
Newer versions have improved this '+' concatenation Strings. But you can build your queryString the way you want with StringBuilder or String Format, it doesn't matter.
Be careful with SQL injection, the setParameter method avoid this kind of problem, for more information see this Sql Injection Baeldung
So this is not the exact answer to the question above, but I was facing a similar issue, I figured I would add it here, for those that come across this question.
I was using a native query, in my case, it was not a singular value like above, but I was passing in a list to match this part of the query:
WHERE (cm.first_name in (:firstNames) OR :firstNames is NULL)
I was getting the bytea error, in the end I was able to send an empty list.
(null == entity.getFirstName()? Collections.emptyList() : entity.getFirstName())
In this case, sending the empty list to the resolver worked, where as null did not.
hope this saves you some time.
null parameters are not allowed before Hibernate 5.0.2.
See https://hibernate.atlassian.net/browse/HHH-9165
and the replies to https://www.postgresql.org/message-id/6ekbd7dm4d6su5b9i4hsf92ibv4j76n51f#4ax.com
There is geometry type column in database like Postgis or h2gis(I am using it).
In the console provided by database, I can create a geometry value with select ST_GeomFromText('POINT(12.3 12)', 4326).
Or select a column with geometry type simply by select * from geom.
However I don't know how to insert a geometry value (a string actually) into a table or the opposite direction conversion.
There are also several miscellaneous question below.
Here is the table definition in slick:
class TableSimple(tag:Tag) extends Table[ (Double,String,String) ](tag,"tb_simple"){
def col_double = column[Double]("col_double",O.NotNull)
def col_str = column[String]("col_str",O.NotNull)
def geom = column[String]("geom",O.DBType("Geometry"))
def * = (col_double,col_str,geom)
}
1. About select
The most simple one:
sql" select col_double,col_str, geom from tb_simple ".as[(Double,String,String)]
won't work unless casting geom to string explicitly like:
sql" select col_double,col_str, cast( geom as varchar) from tb_simple ".as[(Double,String,String)]
The first sql throws the error java.lang.ClassNotFoundException: com.vividsolutions.jts.io.ParseException
Q1: How does slick know com.vividsolutions.jts.io.ParseException (it is lib used by h2gis)? Is it an error on the server side or client side(slick side)?
Q2: How to convert/treat column geom as string without writing too much code(e.g. create a new column type in slick)?
2. About insert
First of all the following sql works
StaticQuery.updateNA(""" insert into tb_simple values(11,'abcd',ST_GeomFromText('POINT(5.300000 1.100000)', 4326)) """).execute
I hope code like TableQuery[TableSimple] += (10.3,"hello","ST_GeomFromText('POINT(0.300000 1.100000)'") would work but it doesn't.
It shouldn't because slick translate it to
insert into tb_simple values(11,'abcd','ST_GeomFromText(''POINT(5.300000 1.100000)'', 4326)')
Notice the function ST_GeomFromText become a part of string, that's why it doesn't work.
Q3: Can I implant a string directly for a column instead of wrapped with '' in slick?
I hope I can insert a row as easy as TableQuery[TableSimple] += (10.3,"hello","ST_GeomFromText('POINT(0.300000 1.100000)'") or similar code.
Q4 What's the most convenient way in Slick to implement bidirectional conversion to and from String for a geometry or other self-defined column in the database?
Answering you main question: Slick-pg offers mapping of geometry types in the db to actual geometry types in your model.
It works for Postgis, but maybe it can also work with H2Gis.
You can find slick-pg at https://github.com/tminglei/slick-pg