I need to compare a nullable entity property via IN expression within the following JPQL query:
#NamedQuery(name = "query",
query = "SELECT e FROM MyEntity e WHERE e.status IN :statuses")
Now, I like the shown collection-valued input parameter statuses to optionally contain null as an element:
final List<MyEntity> actual = entityManager.createNamedQuery("query", MyEntity.class)
.setParameter("statuses", Arrays.asList(null, 1L))
.getResultList();
However with Hibernate/Derby an actual result list only contains entities with status 1L but not null.
I have not found anything in the JPA 2.2 specification about this case. Did I miss something or is this vendor-specific?
The answers to this question only solve part of my problem. In their proposed solutions, the null comparison is hard-baked into the query and cannot be controlled via the collection-valued parameter.
As a Java programmer, where null = null yields true it might come as a surprise that in SQL (and JPQL) null = null is itself null which is "falsy". As a result, null in (null) yields null as well.
Instead you need to treat null seperately with a IS NULL check: e.status IS NULL OR e.status IN :statuses.
This is described in 4.11 Null Values of the JPA Specification:
Comparison or arithmetic operations with a NULL value always yield an unknown value.
Two NULL values are not considered to be equal, the comparison yields an unknown value.
Related
SELECT * FROM Entity e WHERE e.Status <> ANY(ARRAY[1,2,3]);
Here Status is a nullable integer column. Using the above query i am unable to fetch the records whose status value is NULL.
SELECT * FROM Entity e WHERE (e.Status is NULL OR e.Status = 4);
This query does the trick. Could someone explain me why the first query was not working as expected.
NULL kinda means "unknown", so the expressions
NULL = NULL
and
NULL != NULL
are neither true nor false, they're NULL. Because it is not known whether an "unknown" value is equal or unequal to another "unknown" value.
Since <> ANY uses an equality test, if the value searched in the array is NULL, then the result will be NULL.
So your second query is correct.
It is spelled out in the docs Array ANY:
If the array expression yields a null array, the result of ANY will be null. If the left-hand expression yields null, the result of ANY is ordinarily null (though a non-strict comparison operator could possibly yield a different result). Also, if the right-hand array contains any null elements and no true comparison result is obtained, the result of ANY will be null, not false (again, assuming a strict comparison operator). This is in accordance with SQL's normal rules for Boolean combinations of null values.
FYI:
e.Status is NULL OR e.Status = 4
can be shortened to:
e_status IS NOT DISTINCT FROM 4
per Comparison operators.
In python, None !=1 will return True.
But why in Pyspark "Null_column" != 1 will return false?
example:
data = [(1,5),(2,5)]
columns=["id","test"]
df_null=spark.createDataFrame(data,columns)
df_null = df_null.withColumn("nul_val",lit(None))
df_null.printSchema()
df_null.show()
but df_null.filter(df_null.nul_val != 1).count() will return 0
Please check NULL Semantics - Spark 3.0.0 for how to handle comparison with null in spark.
But to summerize, in Spark, null is undefined , so any comparison with null will result in undefined and should be avoided to avoid unwanted results. And in your case, since undefined is not True, the count will be 0.
Apache spark supports the standard comparison operators such as ‘>’, ‘>=’, ‘=’, ‘<’ and ‘<=’. The result of these operators is unknown or NULL when one of the operarands or both the operands are unknown or NULL.
If you want to compare with a column that might contain null, use the null-safe operation <=> which results in False if one of the operands is null:
In order to compare the NULL values for equality, Spark provides a null-safe equal operator (‘<=>’), which returns False when one of the operand is NULL
So, back to your problem. To solve it I would do a null-check and the comparison with 1:
df_null.filter((df_null.nul_val.isNull()) | (df_null.nul_val != 1)).count()
Another solution would be to replace null with 0, if that does not destroy any other logic:
df_null.fill(value=0,subset=["nul_val"]).filter(df_null.nul_val != 1).count()
I would like to write a query which returns all rows where action1 is not 't' (including instances where action1 is null)
This query does not return the null records:
select *
from actions
where action1 <> 't'
This query does return the null records, but I'm surprised both predicates are necessary
select *
from event_actions
where action_c2 is null or action_c2 <> 't'
Is there a way to write this query without both predicates?
You can use the IS DISTINCT FROM construction in the comparison, which treats null as if it is a known value that is different from any non-null value i.e.
select *
from actions
where action1 is distinct from 't'
a is distinct from b is equivalent to
case
when a is null then b is null
else a is not null then b is not null and a = b
end
For reference: https://wiki.postgresql.org/wiki/Is_distinct_from
I am using python3.6 and py-postgresql==1.2.1.
I have the following statement:
db.prepapre("SELECT * FROM seasons WHERE user_id=$1 AND season_id=$2 LIMIT 1), where season_id can be NULL.
I want to be able to be able to get the latest record with a NULL season_id by passing None as the $2 param, but it does not work. Instead, I need to create this second statement:
db.prepapre("SELECT * FROM seasons WHERE user_id=$1 AND season_id IS NULL LIMIT 1)
It must have something to do with season_id = NULL not working and season_id IS NULL is, but is there a way to make this work?
From Comparison Functions and Operators:
Do not write expression = NULL because NULL is not “equal to” NULL. (The null value represents an unknown value, and it is not known whether two unknown values are equal.)
Some applications might expect that expression = NULL returns true if expression evaluates to the null value. It is highly recommended that these applications be modified to comply with the SQL standard. However, if that cannot be done the transform_null_equals configuration variable is available. If it is enabled, PostgreSQL will convert x = NULL clauses to x IS NULL.
and:
19.13.2. Platform and Client Compatibility
transform_null_equals (boolean)
When on, expressions of the form expr = NULL (or NULL = expr) are treated as expr IS NULL, that is, they return true if expr evaluates to the null value, and false otherwise. The correct SQL-spec-compliant behavior of expr = NULL is to always return null (unknown). Therefore this parameter defaults to off.
You could rewrite your query:
SELECT *
FROM seasons
WHERE user_id = $1
AND (season_id = $2 OR ($2 IS NULL AND season_id IS NULL))
-- ORDER BY ... --LIMIT without sorting could be dangerous
-- you should explicitly specify sorting
LIMIT 1;
In a Spring Boot application, I have a SQL query that is executed on a postgresql server as follows :
#Query(value = "select count(*) from servers where brand= coalesce(?1, brand) " +
"and flavour= coalesce(?2, flavour) ; ",
nativeQuery = true)
Integer icecreamStockCount(String country, String category);
However,
I get the following error when I execute the method :
ERROR: COALESCE types bytea and character varying in PostgreSQL
How do I pass String value = null to the query?
**NOTE : ** I found that my question varied from JPA Query to handle NULL parameter value
You need not coalesce, try this
#Query("select count(*) from servers where (brand = ?1 or ?1 is null)" +
" and (flavour = ?2 or ?2 is null)")
Integer icecreamStockCount(String country, String category);
When I encounted this error, I ended up using a combination of OR and CAST to solve the issue.
SELECT COUNT(*)
FROM servers
WHERE (?1 IS NULL OR brand = CAST(?1 AS CHARACTER VARYING))
AND (?2 IS NULL OR flavour = CAST(?2 AS CHARACTER VARYING))
This works even if ?1, ?2, brand and flavor are all nullable fields.
Note that passing null for ?1 means "all servers regardless of brand" rather than "all servers without a brand". For the latter, you could use IS DISTINCT FROM as follows.
SELECT COUNT(*)
FROM servers
WHERE (CAST(?1 AS CHARACTER VARYING) IS NOT DISTINCT FROM brand)
AND (CAST(?2 AS CHARACTER VARYING) IS NOT DISTINCT FROM flavour)
Finally, certain parameter types such as Boolean cannot be cast in SQL from BYTEA to BOOLEAN, for those cases you need a double cast:
SELECT COUNT(*)
FROM servers
WHERE (?1 IS NULL OR is_black = CAST(CAST(?1 AS CHARACTER VARYING) AS BOOLEAN))
In my eyes this is a problem in Hibernate which could be solved by passing Java null parameters as plain SQL NULLs rather than interpreting null as a value of type BYTEA.
If you really need to use native query, there is a problem because it's an improvement not implemented yet, see hibernate. If you don't need to use native you can do (where ?1 is null or field like ?1). Assuming you do need native,
you may treat the String before by setting this empty and then calling the repository and this one would be like:
#Query(value = "select count(*) from servers where (?1 like '' or brand like ?1) " +
"and (?2 like '' or flavour like ?2)",
nativeQuery = true)
Integer icecreamStockCount(String country, String category);
There is always javax.persistence.EntityManager bean as option for native query situations and I recommend it instead of previous approach. Here you can append to your query the way you want, as follows:
String queryString = "select count(*) from servers ";
if (!isNull(country)) queryString += "where brand like :country";
Query query = entityManager.createNativeQuery(queryString);
if (!isNull(country)) query.setParameter("country", country);
return query.getResultList();
Observations:
Newer versions have improved this '+' concatenation Strings. But you can build your queryString the way you want with StringBuilder or String Format, it doesn't matter.
Be careful with SQL injection, the setParameter method avoid this kind of problem, for more information see this Sql Injection Baeldung
So this is not the exact answer to the question above, but I was facing a similar issue, I figured I would add it here, for those that come across this question.
I was using a native query, in my case, it was not a singular value like above, but I was passing in a list to match this part of the query:
WHERE (cm.first_name in (:firstNames) OR :firstNames is NULL)
I was getting the bytea error, in the end I was able to send an empty list.
(null == entity.getFirstName()? Collections.emptyList() : entity.getFirstName())
In this case, sending the empty list to the resolver worked, where as null did not.
hope this saves you some time.
null parameters are not allowed before Hibernate 5.0.2.
See https://hibernate.atlassian.net/browse/HHH-9165
and the replies to https://www.postgresql.org/message-id/6ekbd7dm4d6su5b9i4hsf92ibv4j76n51f#4ax.com