I have a table t with boolean column b.
db.select(T.B).from(T).where(T.B.isTrue()).fetch();
db.select(T.B).from(T).where(T.B.eq(Boolean.TRUE)).fetch();
both these statements are translated to same sql query:
select "public"."T"."B" from "public"."T" where "public"."T"."B" = true;
and this is not working because for boolean comparison keywoard "is" must be used:
select "public"."T"."B" from "public"."T" where "public"."T"."B" is true;
Is there a way to write this properly?
jOOQ currently doesn't support the SQL <boolean test> expression (i.e. x IS [ NOT ] <truth value>)
However, in the "true" case, this isn't really necessary anyway. You can pass the column to the where clause directly:
where(T.B)
... for the same effect. To get the inverse predicate, you can write
where(not(T.B))
If you want a null-safe comparison, you can also use Field.isNotDistinctFrom() or Field.isDistinctFrom()
where(T.B.isNotDistinctFrom(true))
where(T.B.isDistinctFrom(true))
Related
So I work with Postgres SQL, and I have a jsonb column with the following structure:
{
"Store1":[
{
"price":5.99,
"seller":"seller"
},
{
"price":56.43,
"seller":"seller"
}
],
"Store2":[
{
"price":45.65,
"seller":"seller"
},
{
"price":44.66,
"seller":"seller"
}
]
}
I have a jsonb like this for every product in the database. I want to run an SQL query that will answer the following question:
For each product, is one of the prices in this JSON is bigger/equal/smaller than X?
Basically filter the product to include only the ones who have at least one price that satisfies a mathematical condition.
How can I do it efficiently? What's the best way in Postgres to iterate a JSON like this, with a relatively complex inner structure?
Also, if I could control the way the data is structured (to an extent, I can), what changes can I do to make this query more efficient?
Thanks!
Use a json path expression:
WHERE col ## '$.*[*].price < 20'
or
WHERE col #? '$.*[*] ? (#.price < 20)'
If you need to compare to another column or make the query parameterised, you can either build the jsonpath dynamically
WHERE col ## format('$.*[*].price < %s', $1)::jsonpath
WHERE col #? format('$.*[*] ? (#.price < %s)', $1)::jsonpath
or you can use the respective function and pass variables as an object:
WHERE jsonb_path_match(col, '$.*[*].price < $limit', jsonb_build_object('limit', $1))
WHERE jsonb_path_exists(col, format('$.*[*] ? (#.price < $limit)', jsonb_build_object('limit', $1))
I admit I had to check my cheat sheet to figure out the right combination of operator and expression. Takeaways:
if a comparison operator needs to work with multiple values, it generally functions as an ANY
## does not work with ? (# …) filter expressions since they don't return a boolean,
#? does not work with predicates since they always return a value (even if it's false)
What changes can I do to make this query more efficient?
As #jjanes commented on my other answer, the jsonpath match col ## '$.*[*].price < $limit' isn't going to be fast and needs to do full table scan, at least for < and >. To make a useful index, a different approach is required. An index can only have a single value to compare with, not any number. For that, we need to change the condition from EXISTS(SELECT prices_of(col) WHERE price < $limit) to (SELECT MIN(prices_of(col))) < $limit.
With this idea it is possible to build an expression index on the result of a custom immutable function:
CREATE FUNCTION min_price(data jsonb) RETURNS float
LANGUAGE SQL
IMMUTABLE
RETURNS NULL ON NULL INPUT
RETURN (
SELECT min((offer ->> 'price')::float)
FROM jsonb_each(data) AS entries(name, store),
LATERAL jsonb_array_elements(store) AS elements(offer)
);
CREATE INDEX example_min_data_price_idx ON example (min_price(data));
which you can use as
SELECT * FROM example WHERE min_price(data) < 20;
Looking for rows with a price larger than a certain number requires a separate index on max_price(data). If you want to use the index in a JOIN with more conditions, consider making it a multi-column index.
Looking for row with a price equalling a certain number can be optimised by indexing the jsonb column and using a jsonpath:
CREATE INDEX example_data_idx ON example USING GIN (data jsonb_ops);
SELECT * FROM example WHERE data ## '$.*[*].price == 20';
SELECT * FROM example WHERE data #? '$.*[*] ? (#.price == 20)';
Unfortunately you can't use jsonb_path_ops here since that doesn't support the wildcard.
I'm trying to parameterize my postgresql query in order to prevent SQL injection in my ruby on rails application. The SQL query will sum a different value in my table depending on the input.
Here is a simplified version of my function:
def self.calculate_value(value)
calculated_value = ""
if value == "quantity"
calculated_value = "COALESCE(sum(amount), 0)"
elsif value == "retail"
calculated_value = "COALESCE(sum(amount * price), 0)"
elsif value == "wholesale"
calculated_value = "COALESCE(sum(amount * cost), 0)"
end
query = <<-SQL
select CAST(? AS DOUBLE PRECISION) as ? from table1
SQL
return Table1.find_by_sql([query, calculated_value, value])
end
If I call calculate_value("retail"), it will execute the query like this:
select location, CAST('COALESCE(sum(amount * price), 0)' AS DOUBLE PRECISION) as 'retail' from table1 group by location
This results in an error. I want it to execute without the quotes like this:
select location, CAST(COALESCE(sum(amount * price), 0) AS DOUBLE PRECISION) as retail from table1 group by location
I understand that the addition of quotations is what prevents the sql injection but how would I prevent it in this case? What is the best way to handle this scenario?
NOTE: This is a simplified version of the queries I'll be writing and I'll want to use find_by_sql.
Prepared statement can not change query structure: table or column names, order by clause, function names and so on. Only literals can be changed this way.
Where is SQL injection? You are not going to put a user-defined value in the query text. Instead, you check the given value against the allowed list and use only your own written parts of SQL. In this case, there is no danger of SQL injection.
I also want to link to this article. It is safe to create a query text dynamically if you control all parts of that query. And it's much better for RDBMS than some smart logic in query.
If I have a jsonb column called value with fields such as:
{"id": "5e367554-bf4e-4057-8089-a3a43c9470c0",
"tags": ["principal", "reversal", "interest"],,, etc}
how would I find all the records containing given tags, e.g:
if given: ["reversal", "interest"]
it should find all records with either "reversal" or "interest" or both.
My experimentation got me to this abomination so far:
select value from account_balance_updated
where value #> '{}' :: jsonb and value->>'tags' LIKE '%"principal"%';
of course this is completely wrong and inefficient
Assuming you are using PG 9.4+, you can use the jsonb_array_elements() function:
SELECT DISTINCT abu.*
FROM account_balance_updated abu,
jsonb_array_elements(abu.value->'tags') t
WHERE t.value <# '["reversal", "interest"]'::jsonb;
As it turned out you can use cool jsonb operators described here:
https://www.postgresql.org/docs/9.5/static/functions-json.html
so original query doesn't have to change much:
select value from account_balance_updated
where value #> '{}' :: jsonb and value->'tags' ?| array['reversal', 'interest'];
in my case I also needed to escape the ? (??|) because I am using so called "prepared statement" where you pass query string and parameters to jdbc and question marks are like placeholders for params:
https://docs.oracle.com/javase/tutorial/jdbc/basics/prepared.html
I'm trying to select keys from JSONB type with true values. So far I managed to do that using this query but I feel like there is a better way:
SELECT json.key
FROM jsonb_each_text('{"aaa": true, "bbb": false}'::JSONB) json
WHERE json.value = 'true';
What I don't like is the WHERE clause where I'm comparing strings. Is there a way to cast it to boolean?
If yes, would it work for truthy and falsy values too? (explanation of truthy and falsy values in javascript: http://www.codeproject.com/Articles/713894/Truthy-Vs-Falsy-Values-in-JavaScript).
jsonb has an equality operator (=; unlike json), so you could write
SELECT key
FROM jsonb_each('{"aaa": true, "bbb": false}')
WHERE value = jsonb 'true'
(with jsonb_each_text() you rely on some JSON values' text representation).
You can even include some additional values, if you want:
WHERE value IN (to_jsonb(TRUE), jsonb '"true"', to_jsonb('truthy'))
IN uses the equality operator under the hood.
I want to check if something is present in my database before saving it in order to avoid key duplicate errors. I'm using Play! 2.2.6 with anorm and Postgresql 9.3.
So I wrote a little function (I omit the errors check):
def testIfExist(fieldName: String, value: String): Boolean = {
DB.withConnection { implicit connection =>
SQL( """SELECT exists(SELECT 1 FROM events where {fieldName}={value} LIMIT 1)""")
.on(
'fieldName -> fieldName,
'value -> value
).execute()
}
}
But it always return true although my database is totally empty.
So I tested to replace
SELECT exists(SELECT 1 FROM events where {fieldName}={value} LIMIT 1
by
SELECT exists(SELECT 1 FROM events where name='aname' LIMIT 1
and it still always return true...
I also tested the same query directly in psql and the response is what I except : false...
execute returns true if anything was returned in the result set. In this case it will be 0 or 1. It will only return false if the query was an update (returns no result set). You need to use as with a ResultSetParser to parse the results.
There's another problem with this code as well. You can't supply column names in prepared statements. {fieldName}={value}. This will get turned into a string comparison, which will probably always be false. Instead, you can use string interpolation to insert the field name into the query. Though be wary, fieldName should be be from user defined input as it is vulnerable to SQL injection. (Your users shouldn't need know about your columns anyway)
SQL(s"SELECT exists(SELECT 1 FROM events where ${fieldName} = {value} LIMIT 1)")
.on("value" -> value)
.as(scalar[Boolean].single)