I'm confused by how path uses different formats depending on the function in the PostgreSQL JSONB documentation.
If I had a PostgreSQL table foo that looks like
pk
json_obj
0
{"values": [{"id": "a_b", "value": 5}, {"id": "c_d", "value": 6]}
1
{"values": [{"id": "c_d", "value": 7}, {"id": "e_f", "value": 8]}
Why does this query give me these results?
SELECT json_obj, -- {"values": [{"id": "a_b", "value": 5}, {"id": "c_d", "value": 6]}
json_obj #? '$.values[*].id', -- true
json_obj #> '$.values[*].id', -- ERROR: malformed array literal
json_obj #> '{values, 0, id}', -- "a_b"
JSONB_SET(json_obj, '$.annotations[*].id', '"hi"') -- ERROR: malformed array literal
FROM foo;
Specifically, why does #? support $.values[*].id (described on that page in another section) but JSONB_SET uses some other path format {bar,3,baz}?
Ultimately, what I would like to do and don't know how, is to remove non-alphanumeric characters (e.g. underscores in this example) in all id values represented by the path $.values[*].id.
The reason is that the operators have different data types on the right hand side.
SELECT oprname, oprright::regtype
FROM pg_operator
WHERE oprleft = 'jsonb'::regtype
AND oprname IN ('#?', '#>');
oprname | oprright
---------+----------
#> | text[]
#? | jsonpath
(2 rows)
Similarly, the second argument of jsonb_set is a text[].
Now '$.values[*].id' is a valid jsonpath, but not a valid text[] literal.
Thanks for the answers and comments about why the data types were different.
I wanted to post how I solved my problem:
Ultimately, what I would like to do and don't know how, is to remove
non-alphanumeric characters (e.g. underscores in this example) in all
id values represented by the path $.values[*].id.
WITH unnested AS (
SELECT f.pk, JSONB_ARRAY_ELEMENTS(f.json_obj -> 'values') AS value
FROM foo f
),
updated_values AS (
SELECT un.pk, JSONB_SET(un.value, '{id}', TO_JSONB(LOWER(REGEXP_REPLACE(un.value ->> 'id', '[^a-zA-Z0-9]', '', 'g'))), FALSE) AS new_value
FROM unnested un
WHERE value -> 'id' IS NOT NULL -- Had some values that didn't have 'id' keys
)
UPDATE foo f2
SET json_obj = JSONB_SET(f2.json_obj, '{values}', (SELECT JSONB_AGG(uv.new_value) FROM updated_values uv WHERE uv.pk = f2.pk), FALSE)
WHERE JSONB_PATH_EXISTS(f2.json_obj, '$.values[*].id') -- Had some values that didn't have 'id' keys
Related
The following query deletes an entry using index:
const deleteGameQuery = `
update users
set games = games - 1
where username = $1
`
If I pass the index as a parameter, nothing is deleted:
const gameIndex = rowsCopy[0].games.findIndex(obj => obj.game == gameID).toString();
const deleteGameQuery = `
update users
set games = games - $1
where username = $2
`
const { rows } = await query(deleteGameQuery, [gameIndex, username]);
ctx.body = rows;
The gameIndex parameter is just a string, the same as if I typed it. So why doesn't it seem to read the value? Is this not allowed?
The column games is a jsonb data type with the following data:
[
{
"game": "cyberpunk-2077",
"status": "Backlog",
"platform": "Any"
},
{
"game": "new-pokemon-snap",
"status": "Backlog",
"platform": "Any"
}
]
The problem is you're passing text instead of an integer. You need to pass an integer. I'm not sure exactly how your database interface works to pass integers, try removing toString() and ensure gameIndex is a Number.
const gameIndex = rowsCopy[0].games.findIndex(obj => obj.game == gameID).
array - integer and array - text mean two different things.
array - 1 removes the second element from the array.
select '[1,2,3]'::jsonb - 1;
[1, 3]
array - '1' searches for the entry '1' and removes it.
select '["1","2","3"]'::jsonb - '1';
["2", "3"]
-- Here, nothing is removed because 1 != '1'.
select '[1,2,3]'::jsonb - '1';
[1, 2, 3]
When you pass in a parameter, it is translated by query according to its type. If you pass a Number it will be translated as 1. If you pass a String it will be translated as '1'. (Or at least that's how it should work, I'm not totally familiar with Javascript database libraries.)
As a side note, this sort of data is better handled as a join table.
create table games (
id bigserial primary key,
name text not null,
status text not null,
platform text not null
);
create table users (
id bigserial primary key,
username text not null
);
create table game_users (
game_id bigint not null references games,
user_id bigint not null references users,
-- If a user can only have each game once.
unique(game_id, user_id)
);
-- User 1 has games 1 and 2. User 2 has game 2.
insert into game_users (game_id, user_id) values (1, 1), (2, 1), (2,2);
-- User 1 no longer has game 1.
delete from game_users where game_id = 1 and user_id = 1;
You would also have a platforms table and a game_platforms join table.
Join tables are a little mind bending, but they're how SQL stores relationships. JSONB is very useful, but it is not a substitute for relationships.
You can try to avoid decomposing objects outside of postgress and manipulate jsonb structure inside the query like this:
create table gameplayers as (select 1 as id, '[
{
"game": "cyberpunk-2077",
"status": "Backlog",
"platform": "Any"
},
{
"game": "new-pokemon-snap",
"status": "Backlog",
"platform": "Any"
},
{
"game": "gameone",
"status": "Backlog",
"platform": "Any"
}
]'::jsonb games);
with
ungroupped as (select * from gameplayers g, jsonb_to_recordset(g.games)
as (game text, status text, platform text)),
filtered as (select id,
jsonb_agg(
json_build_object('game', game,
'status', status,
'platfrom', platform
)
) games
from ungroupped where game not like 'cyberpunk-2077' group by id)
UPDATE gameplayers as g set games=f.games
from filtered f where f.id=g.id;
Using postgres version 10.13
This is my datatable jsongraphs
id
jsongraph
1
{ "data": {"scopes_by_id": { "121": { "id": 121, "pk": 121, "name": "Prework" } }, "commonsites_by_id": {"123": {"id": 123, "pk": 123, "name": "Somewhere over the rainbow"}}}}
2
{ "data": {"scopes_by_id": { "156": { "id": 156, "pk": 156, "name": "ABC" } }, "commonsites_by_id": {"123": {"id": 123, "pk": 123, "name": "Somewhere over the rainbow"}}}}
I want the distinct values of scope id and site id which should be (121, 123), (156,123)
So I tried
SELECT DISTINCT
jsongraph->'data'->'scopes_by_id'->>'pk' ,
jsongraph->'data'->'commonsites_by_id'->>'pk' from jsongraphs;
This won't work because the path should be like data->scopes_by_id->121->>pk but I cannot know beforehand the value of 121 in between.
Is there a way to get the values of what I need by filling in some kind of wildcard in the path?
E.g.data->scopes_by_id->{*}->>pk like that?
ANd because this is legacy data, it's also hard to change the data itself.
As the nesting level seems to be fixed, you could do something like this:
select j.id, scopes.*, commonsites.*
from jsongraphs j
cross join lateral (
select jsonb_agg(j.jsongraph #> array['data','scopes_by_id', t1.scope_id, 'pk']) as scope_ids
from jsonb_each_text(j.jsongraph #> '{data,scopes_by_id}') as t1(scope_id)
) scopes
cross join lateral (
select jsonb_agg(j.jsongraph #> array['data','commonsites_by_id', t2.site_id, 'pk']) as common_ids
from jsonb_each_text(j.jsongraph #> '{data,commonsites_by_id}') as t2(site_id)
) commonsites
order by id;
The sub-queries extract all key below the respective part (e.g. scopes_by_id) and then uses the #>' operator to access the path for each id inside the original JSON value. And finally all PK values are aggregated back into a single array.
This returns the PK values from each part separately as an array in order to handle the situation where you have a different number of "scope ids" and "commonsite ids"
If you just want "the first" id from each section, you can remove the aggregation and use a LIMIT clause:
select j.id, scopes.*, commonsites.*
from jsongraphs j
cross join lateral (
select j.jsongraph #> array['data','scopes_by_id', t1.scope_id, 'pk'] as scope_id
from jsonb_each_text(j.jsongraph #> '{data,scopes_by_id}') as t1(scope_id)
limit 1
) scopes
cross join lateral (
select j.jsongraph #> array['data','commonsites_by_id', t2.site_id, 'pk'] as common_id
from jsonb_each_text(j.jsongraph #> '{data,commonsites_by_id}') as t2(site_id)
limit 1
) commonsites
order by id;
Not sure on which level you want to apply the "distinct" part for this.
In Postgres 12 or later, you could achieve the same with:
select id,
jsonb_path_query_array(j.jsongraph, 'strict $.data.scopes_by_id.**.pk') as scopes,
jsonb_path_query_array(j.jsongraph, 'strict $.data.commonsites_by_id.**.pk') as common
from jsongraphs ;
order by id;
Online example
In a Postgres (9+) table there is a column of type jsonb with the following json:
{
"dynamicFields":[
{
"name":"040",
"subfields":[
{
"name":"a",
"value":"abc"
},
{
"name":"a",
"value":"xyz"
}
]
}
]
}
I would like to write a query that return only the rows where the field name equals 040 and subfield a equals xyz.
This is as far as I got, so far:
select e.obj from my_table
cross join lateral jsonb_array_elements(my_column-> 'dynamicFields') as e(obj)
where e.obj ->> 'name' = '040' and e.obj ->> 'subfields' #> '{"name": "a", "value": "xyz"}'::jsonb
How should this query be to achieve this?
e.obj ->> 'subfields' has a text result. You'll want to use e.obj -> 'subfields' that returns the jsonb value where the #> operator works. Also the containment checks needs to have another array as the right hand side, so that it will test whether all values in the right array are contained in the left array - it doesn't work to pass the element object directly.
select e.obj from my_table
cross join lateral jsonb_array_elements(my_column-> 'dynamicFields') as e(obj)
where e.obj ->> 'name' = '040' and e.obj -> 'subfields' #> '[{"name": "a", "value": "xyz"}]'::jsonb
-- ^ ^ ^
(online demo)
As you have an equality condition you can use the "contains" operator directly by providing a JSON value of what you want. There is no need to unnest the arrays.
select *
from my_table
where my_column -> 'dynamicFields' #> '[{"name": "040", "subfields": [{"name":"a", "value": "xyz"}]}]'
Starting with Postgres 12 an alternative is to a SQL/JSON path operator:
select *
from my_table
where my_column #? '$.dynamicFields[*] ? (#.name == "040").subfields[*] ? (#.name == "a" && #.value == "xyz")'
I need a help for the below mentioned detail,
I am using Postgres + Spring-Data-JPA. Moreover, I have used the jsonb data type for storing data.
I am trying to execute a query, but it gives me the following error:
ERROR: set-returning functions are not allowed in WHERE
The cause here is that I have added a jsonb condition in the WHERE clause (kindly refer to the below query for more detail).
Query (I have renamed column name just because of hiding the actual column name):
select distinct
jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'firstName' as firstName,
jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'lastName' as lastName,
jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'country' as country
from
tale1 table10_
left outer join
table2 table21_
on table10_.end_user_id=table21_.end_user_id
left outer join
table3 table32_
on table10_.manufacturer_id=table32_.manufacturer_id
where
table21_.end_user_uuid=(
?
)
and table21_.is_active=true
and table32_.manufacturer_uuid=(
?
)
and table32_.is_active=true
and table10_.is_active=true
and table32_.is_active=true
and jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'action' = ('PENDING')
order by
jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'firstName',
jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'lastName'
limit ?
The following line in the above query is causing the error:
and jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'action' = ('PENDING')
Can anyone please guide me about how do fetch data from the inner JSON? Especially in my case I have an inner List and a few elements inside.
I recommend a lateral join with jsonb_array_elements for cases like that. Here is an example:
CREATE TABLE tale1 (
id integer PRIMARY KEY,
initiated_referral_detail jsonb NOT NULL
);
INSERT INTO tale1 VALUES
(1, '{
"name": "one",
"listOfAttribue": [
{ "id": 1, "action": "DONE"},
{ "id": 2, "action": "PENDING" },
{ "id": 3, "action": "ACTIVE" }
]
}');
INSERT INTO tale1 VALUES
(2, '{
"name": "two",
"listOfAttribue": [
{ "id": 1, "action": "DONE"},
{ "id": 2, "action": "ACTIVE" }
]
}');
To find all ids where the associated JSON contains an array element with action = PENDING, you can query like this:
SELECT DISTINCT id
FROM tale1 CROSS JOIN LATERAL
jsonb_array_elements(initiated_referral_detail -> 'listOfAttribue') AS attr
WHERE attr ->> 'action' = 'PENDING';
I have a column with json type but I'm wondering how to select filter it i.e.
select * from fooTable where myjson like "orld";
How would I query for a substring match like the above. Say searching for "orld" under "bar" keys?
{ "foo": "hello", "bar": "world"}
I took a look at this documentation but it is quite confusing.
https://www.postgresql.org/docs/current/static/datatype-json.html
Use the ->> operator to get json attributes as text, example
with my_table(id, my_json) as (
values
(1, '{ "foo": "hello", "bar": "world"}'::json),
(2, '{ "foo": "hello", "bar": "moon"}'::json)
)
select t.*
from my_table t
where my_json->>'bar' like '%orld'
id | my_json
----+-----------------------------------
1 | { "foo": "hello", "bar": "world"}
(1 row)
Note that you need a placeholder % in the pattern.