How to query deep jsonb in Postgres? - postgresql

I have a jsonb column (called info) in Postgres which structure looks like this:
{ name: 'john', house_id: null, extra_attrs: [{ attr_id: 4, attr_value: 'a value' }, { attr_id: 5, attr_value: 'another value' }] }
It can have N extra_attrs but we know that each of them will have just two keys: the attr_id and the attr_value.
Now, what is the best way to query for info that has extra_attrs with a specific attr_id and attr_value. I have done it like this, and it works:
Given the following data structure to query for:
[{ attr_id: 4, values: ['a value', 'something else'] }, { attr_id: 5, values: ['another value'] }]
The following query works:
select * from people
where (info #> '{"extra_attrs": [{ "attr_id": 4, "attr_value": "a value" }]} OR info #> '{"extra_attrs": [{ "attr_id": 4, "attr_value": "something else" }]) AND info #> '{"extra_attrs": [{ "attr_id": 5, "attr_value": "another value" }]}
I am wondering if there is a better way to do so or this is fine.

One alternate method would involve json functions and transforming data to apply the filter on:
SELECT people.info
FROM people,
LATERAL (SELECT DISTINCT True is_valid
FROM jsonb_array_elements(info->'extra_attrs') y
WHERE (y->>'attr_id', y->>'attr_value') IN (
('4', 'a value'),
('4', 'something else'),
('5','another value')
)
) y
WHERE is_valid
I believe this method more convenient for dynamic filters since the id/value pairs are added in only 1 place.
A similar (and perhaps slightly faster) method would use WHERE EXISTS and compare json documents like below.
SELECT people.info
FROM people
WHERE EXISTS (SELECT TRUE
FROM jsonb_array_elements(info->'extra_attrs') attrs
WHERE attrs #> ANY(ARRAY[
JSONB '{ "attr_id": 4, "attr_value": "a value" }',
JSONB '{ "attr_id": 4, "attr_value": "something else" }',
JSONB '{ "attr_id": 5, "attr_value": "another value" }'
]
)
)

Related

Postgresql: Can the minus operator not be used with a parameter? Only hardcoded values?

The following query deletes an entry using index:
const deleteGameQuery = `
update users
set games = games - 1
where username = $1
`
If I pass the index as a parameter, nothing is deleted:
const gameIndex = rowsCopy[0].games.findIndex(obj => obj.game == gameID).toString();
const deleteGameQuery = `
update users
set games = games - $1
where username = $2
`
const { rows } = await query(deleteGameQuery, [gameIndex, username]);
ctx.body = rows;
The gameIndex parameter is just a string, the same as if I typed it. So why doesn't it seem to read the value? Is this not allowed?
The column games is a jsonb data type with the following data:
[
{
"game": "cyberpunk-2077",
"status": "Backlog",
"platform": "Any"
},
{
"game": "new-pokemon-snap",
"status": "Backlog",
"platform": "Any"
}
]
The problem is you're passing text instead of an integer. You need to pass an integer. I'm not sure exactly how your database interface works to pass integers, try removing toString() and ensure gameIndex is a Number.
const gameIndex = rowsCopy[0].games.findIndex(obj => obj.game == gameID).
array - integer and array - text mean two different things.
array - 1 removes the second element from the array.
select '[1,2,3]'::jsonb - 1;
[1, 3]
array - '1' searches for the entry '1' and removes it.
select '["1","2","3"]'::jsonb - '1';
["2", "3"]
-- Here, nothing is removed because 1 != '1'.
select '[1,2,3]'::jsonb - '1';
[1, 2, 3]
When you pass in a parameter, it is translated by query according to its type. If you pass a Number it will be translated as 1. If you pass a String it will be translated as '1'. (Or at least that's how it should work, I'm not totally familiar with Javascript database libraries.)
As a side note, this sort of data is better handled as a join table.
create table games (
id bigserial primary key,
name text not null,
status text not null,
platform text not null
);
create table users (
id bigserial primary key,
username text not null
);
create table game_users (
game_id bigint not null references games,
user_id bigint not null references users,
-- If a user can only have each game once.
unique(game_id, user_id)
);
-- User 1 has games 1 and 2. User 2 has game 2.
insert into game_users (game_id, user_id) values (1, 1), (2, 1), (2,2);
-- User 1 no longer has game 1.
delete from game_users where game_id = 1 and user_id = 1;
You would also have a platforms table and a game_platforms join table.
Join tables are a little mind bending, but they're how SQL stores relationships. JSONB is very useful, but it is not a substitute for relationships.
You can try to avoid decomposing objects outside of postgress and manipulate jsonb structure inside the query like this:
create table gameplayers as (select 1 as id, '[
{
"game": "cyberpunk-2077",
"status": "Backlog",
"platform": "Any"
},
{
"game": "new-pokemon-snap",
"status": "Backlog",
"platform": "Any"
},
{
"game": "gameone",
"status": "Backlog",
"platform": "Any"
}
]'::jsonb games);
with
ungroupped as (select * from gameplayers g, jsonb_to_recordset(g.games)
as (game text, status text, platform text)),
filtered as (select id,
jsonb_agg(
json_build_object('game', game,
'status', status,
'platfrom', platform
)
) games
from ungroupped where game not like 'cyberpunk-2077' group by id)
UPDATE gameplayers as g set games=f.games
from filtered f where f.id=g.id;

Different path formats for PostgreSQL JSONB functions

I'm confused by how path uses different formats depending on the function in the PostgreSQL JSONB documentation.
If I had a PostgreSQL table foo that looks like
pk
json_obj
0
{"values": [{"id": "a_b", "value": 5}, {"id": "c_d", "value": 6]}
1
{"values": [{"id": "c_d", "value": 7}, {"id": "e_f", "value": 8]}
Why does this query give me these results?
SELECT json_obj, -- {"values": [{"id": "a_b", "value": 5}, {"id": "c_d", "value": 6]}
json_obj #? '$.values[*].id', -- true
json_obj #> '$.values[*].id', -- ERROR: malformed array literal
json_obj #> '{values, 0, id}', -- "a_b"
JSONB_SET(json_obj, '$.annotations[*].id', '"hi"') -- ERROR: malformed array literal
FROM foo;
Specifically, why does #? support $.values[*].id (described on that page in another section) but JSONB_SET uses some other path format {bar,3,baz}?
Ultimately, what I would like to do and don't know how, is to remove non-alphanumeric characters (e.g. underscores in this example) in all id values represented by the path $.values[*].id.
The reason is that the operators have different data types on the right hand side.
SELECT oprname, oprright::regtype
FROM pg_operator
WHERE oprleft = 'jsonb'::regtype
AND oprname IN ('#?', '#>');
oprname | oprright
---------+----------
#> | text[]
#? | jsonpath
(2 rows)
Similarly, the second argument of jsonb_set is a text[].
Now '$.values[*].id' is a valid jsonpath, but not a valid text[] literal.
Thanks for the answers and comments about why the data types were different.
I wanted to post how I solved my problem:
Ultimately, what I would like to do and don't know how, is to remove
non-alphanumeric characters (e.g. underscores in this example) in all
id values represented by the path $.values[*].id.
WITH unnested AS (
SELECT f.pk, JSONB_ARRAY_ELEMENTS(f.json_obj -> 'values') AS value
FROM foo f
),
updated_values AS (
SELECT un.pk, JSONB_SET(un.value, '{id}', TO_JSONB(LOWER(REGEXP_REPLACE(un.value ->> 'id', '[^a-zA-Z0-9]', '', 'g'))), FALSE) AS new_value
FROM unnested un
WHERE value -> 'id' IS NOT NULL -- Had some values that didn't have 'id' keys
)
UPDATE foo f2
SET json_obj = JSONB_SET(f2.json_obj, '{values}', (SELECT JSONB_AGG(uv.new_value) FROM updated_values uv WHERE uv.pk = f2.pk), FALSE)
WHERE JSONB_PATH_EXISTS(f2.json_obj, '$.values[*].id') -- Had some values that didn't have 'id' keys

org.postgresql.util.PSQLException: ERROR: set-returning functions are not allowed in WHERE

I need a help for the below mentioned detail,
I am using Postgres + Spring-Data-JPA. Moreover, I have used the jsonb data type for storing data.
I am trying to execute a query, but it gives me the following error:
ERROR: set-returning functions are not allowed in WHERE
The cause here is that I have added a jsonb condition in the WHERE clause (kindly refer to the below query for more detail).
Query (I have renamed column name just because of hiding the actual column name):
select distinct
jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'firstName' as firstName,
jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'lastName' as lastName,
jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'country' as country
from
tale1 table10_
left outer join
table2 table21_
on table10_.end_user_id=table21_.end_user_id
left outer join
table3 table32_
on table10_.manufacturer_id=table32_.manufacturer_id
where
table21_.end_user_uuid=(
?
)
and table21_.is_active=true
and table32_.manufacturer_uuid=(
?
)
and table32_.is_active=true
and table10_.is_active=true
and table32_.is_active=true
and jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'action' = ('PENDING')
order by
jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'firstName',
jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'lastName'
limit ?
The following line in the above query is causing the error:
and jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'action' = ('PENDING')
Can anyone please guide me about how do fetch data from the inner JSON? Especially in my case I have an inner List and a few elements inside.
I recommend a lateral join with jsonb_array_elements for cases like that. Here is an example:
CREATE TABLE tale1 (
id integer PRIMARY KEY,
initiated_referral_detail jsonb NOT NULL
);
INSERT INTO tale1 VALUES
(1, '{
"name": "one",
"listOfAttribue": [
{ "id": 1, "action": "DONE"},
{ "id": 2, "action": "PENDING" },
{ "id": 3, "action": "ACTIVE" }
]
}');
INSERT INTO tale1 VALUES
(2, '{
"name": "two",
"listOfAttribue": [
{ "id": 1, "action": "DONE"},
{ "id": 2, "action": "ACTIVE" }
]
}');
To find all ids where the associated JSON contains an array element with action = PENDING, you can query like this:
SELECT DISTINCT id
FROM tale1 CROSS JOIN LATERAL
jsonb_array_elements(initiated_referral_detail -> 'listOfAttribue') AS attr
WHERE attr ->> 'action' = 'PENDING';

Postgres: How to string pattern match query a json column?

I have a column with json type but I'm wondering how to select filter it i.e.
select * from fooTable where myjson like "orld";
How would I query for a substring match like the above. Say searching for "orld" under "bar" keys?
{ "foo": "hello", "bar": "world"}
I took a look at this documentation but it is quite confusing.
https://www.postgresql.org/docs/current/static/datatype-json.html
Use the ->> operator to get json attributes as text, example
with my_table(id, my_json) as (
values
(1, '{ "foo": "hello", "bar": "world"}'::json),
(2, '{ "foo": "hello", "bar": "moon"}'::json)
)
select t.*
from my_table t
where my_json->>'bar' like '%orld'
id | my_json
----+-----------------------------------
1 | { "foo": "hello", "bar": "world"}
(1 row)
Note that you need a placeholder % in the pattern.

Postgres jsonb_set multiple nested fields

I have DB table with a jsonb column that has an entity, with nested child entities. Let's say we have:
SELECT jsonb_set('{"top": {"nested": {"leaf" : 1}}}', '{top,nested,leaf}', '2');
Which work's fine by updating top.nested.leaf to 2.
But what if we wanted to do multiple fields, such as:
SELECT jsonb_set('{"top": {"nested": {"leaf" : 1}, "other_nested": {"paper": 0}}}', '[{top,nested,leaf}, {top,other_nested,paper}]', '[2, 2]');
The above does not work and says:
ERROR: malformed array literal: "[{top,nested,leaf}, {top,other_nested,paper}]"
LINE 1: ...": {"leaf" : 1}, "other_nested": {"paper": 0}}}', '[{top,nes...
^
DETAIL: "[" must introduce explicitly-specified array dimensions.
Any ideas?
https://www.postgresql.org/docs/current/static/functions-json.html
jsonb_set(target jsonb, path text[], new_value jsonb[, create_missing boolean])
neither path, nor new value can't have several values. you have to run it twice for wanted result, eg:
SELECT jsonb_set(
'{"top": {"nested": {"leaf" : 1}, "other_nested": {"paper": 0}}}'
, '{top,nested,leaf}'
, '2'
);
SELECT jsonb_set(
'{"top": {"nested": {"leaf" : 1}, "other_nested": {"paper": 0}}}'
, '{top,other_nested,paper}'
, '2'
);