I have trouble figuring out how to query PostgreSQL JsonB datatype. I have a simple table with the structure:
CREATE TABLE myTable (id BIGINT PRIMARY KEY, answers JSONB)
All Json documents in the column "answers" are of the form:
[
{"R" : "aaa", "V" : 25},
{"R" : "aaa", "V" : 31},
{"R" : "bbb", "V" : 38}
...
]
There can be many elements in the list, but all elements will have a "R" and a "V" item.
I would like to retrieve a table using SQL, listing Ids, and a Json list of all the "V"s where "R" == "aaa" .
For the example above, I would get:
Id for the record
[25, 31] -- the two "V" values where "R" == "aaa"
Any thoughts? Any help appreciated I have spent some time on the JSon paths examples available on the web, but haven't found something similar.
Thanks in advance.
Note: Postgresql 12+ only
Using jsonpath:
WITH data(id, json_arr) AS (
VALUES (1, $$[
{ "R": "aaa", "V": 25 },
{ "R": "aaa", "V": 31 },
{ "R": "bbb", "V": 38 }
]$$::JSONB)
)
SELECT id,
-- $[*] : "inside the top level array"
-- ? (#.R == "aaa") : "keep only when the "R" key's value is "aaa""
-- .V : "select the "V" key of those elements"
jsonb_path_query_array(json_arr, '$[*] ? (#.R == "aaa").V')
FROM data
returns:
+--+----------------------+
|id|jsonb_path_query_array|
+--+----------------------+
|1 |[25, 31] |
+--+----------------------+
Note: you can also use
jsonb_path_query_array(
json_arr,
'$[*] ? (#.R == $r_value).V',
'{"r_value": "aaa"}' -- pass the 'r_value' to the query above
)
I would expand the elements out, filter by R, and then reaggregate. This results in an int[] array.
select m.id, array_agg((a.obj->>'V')::int) as vvals
from mytable m
cross join lateral jsonb_array_elements(answers) as a(obj)
where a.obj->>'R' = 'aaa';
all in pure JSONB:
SELECT id, jsonb_agg(ans->'V') FROM (
SELECT id, jsonb_array_elements(answers) AS ans
FROM myTable
) zz
WHERE ans->'R' = '"aaa"'
GROUP BY id;
tl;dr -- is there some way to get values as a jsonb_array from a jsonb object in postgres?
Trying to use recursive cte in postgres to flatten an arbitrarily deep tree structure like this:
{
"type": "foo",
"properties": {...},
"children": [
"type": "bar",
"properties": {...},
"children": [
"type": "multivariate",
"variants": {
"arbitrary-name": {
properties: {...},
children: [...],
},
"some-other-name": {
properties: {...},
children: [...],
},
"another": {
properties: {...},
children: [...],
}
}
]
]
}
Generally following this post, however I'm stuck at processing the type: "multivariate" node where what I really want is essentially jsonb_agg(jsonb_object_values(json_object -> 'variants'))
Update:
Sorry, I obviously should have included the query I tried:
WITH RECURSIVE tree_nodes (id, json_element) AS (
-- non recursive term
SELECT
id, node
FROM trees
UNION
-- recursive term
SELECT
id,
CASE
WHEN jsonb_typeof(json_element) = 'array'
THEN jsonb_array_elements(json_element)
WHEN jsonb_exists(json_element, 'children')
THEN jsonb_array_elements(json_element -> 'children')
WHEN jsonb_exists(json_element, 'variants')
THEN (select jsonb_agg(element.value) from jsonb_each(json_element -> 'variants') as element)
END AS json_element
FROM
tree_nodes
WHERE
jsonb_typeof(json_element) = 'array' OR jsonb_typeof(json_element) = 'object'
)
select * from tree_nodes;
The schema is just an id & a jsonb node column
This query gives an error:
ERROR: set-returning functions are not allowed in CASE
LINE 16: THEN jsonb_array_elements(json_element -> 'children')
^
HINT: You might be able to move the set-returning function into a LATERAL FROM item.
I just want Object.values(json_element -> 'variants') 😫
Update 2:
After reading this all again, I realized this is a problem due to me using a recent version of PostgreSQL (10.3), which apparently no longer allows returning a set from a CASE statement, which was kind of the crux of getting this tree-flattening approach to work afaict. There's probably some way to achieve the same thing in recent versions of PostgreSQL but I have no idea how I'd go about doing it.
Use jsonb_each() in the FROM clause together with jsonb_agg(<jsonb_each_alias>.value) in the SELECT, for example:
select
id,
jsonb_agg(child.value)
from
(values
(101, '{"child":{"a":1,"b":2}}'::jsonb),
(102, '{"child":{"c":3,"d":4,"e":5}}'::jsonb
)) as t(id, json_object), -- example table, replace values block with actual tablespec
jsonb_each(t.json_object->'child') as child
group by t.id
You can always chain other jsonb functions which return setof jsonb (e.g. jsonb_array_elements) in the FROM if you need to iterate the higher level arrays before the jsonb_each; for example:
select
id,
jsonb_agg(sets.value)
from
(values
(101, '{"children":[{"a_key":{"a":1}},{"a_key":{"b":2}}]}'::jsonb),
(102, '{"children":[{"a_key":{"c":3,"d":4,"e":5}},{"a_key":{"f":6}}]}'::jsonb
)) as t(id, json_object), -- example table, replace values block with actual tablespec
jsonb_array_elements(t.json_object->'children') elem,
jsonb_each(elem->'a_key') as sets
group by t.id;
Update Answer
In answer to your comment and question edit about needing to walk the 'children' of each tree node and extract the 'variants'; I would achieve this by splitting the CTE into multiple stages:
with recursive
-- Constant table for demonstration purposes only; remove this and replace below references to "objects" with table name
objects(id, object) as (values
(101, '{"children":[{"children":[{"variants":{"aa":11}},{"variants":{"ab":12}}],"variants":{"a":1}},{"variants":{"b":2}}]}'::jsonb),
(102, '{"children":[{"children":[{"variants":{"cc":33,"cd":34,"ce":35}},{"variants":{"f":36}}],"variants":{"c":3,"d":4,"e":5}},{"variants":{"f":6}}]}'::jsonb)
),
tree_nodes as ( -- Flatten the tree by walking all 'children' and creating a separate record for each root
-- non-recursive term: get root element
select
o.id, o.object as value
from
objects o
union all
-- recursive term - get JSON object node for each child
select
n.id,
e.value
from
tree_nodes n,
jsonb_array_elements(n.value->'children') e
where
jsonb_typeof(n.value->'children') = 'array'
),
variants as (
select
n.id,
v.value
from
tree_nodes n,
jsonb_each(n.value->'variants') v -- expand variants
where
jsonb_typeof(n.value->'variants') = 'object'
)
select
id,
jsonb_agg(value)
from
variants
group by
id
;
This ability of breaking a query up into a "pipeline" of operations is one of my favourite things about CTEs - it makes the query much easier to understand, maintain and debug.
db<>fiddle
Expanded the test data with more children elements and deeper structure (more nested elements):
{
"type": "foo",
"children": [
{
"type" : "bar1",
"children" : [{
"type" : "blubb",
"children" : [{
"type" : "multivariate",
"variants" : {
"blubb_variant1": {
"properties" : {
"blubb_v1_a" : 100
},
"children" : ["z", "y"]
},
"blubb_variant2": {
"properties" : {
"blubb_v2_a" : 300,
"blubb_v2_b" : 4200
},
"children" : []
}
}
}]
}]
},
{
"type" : "bar2",
"children" : [{
"type" : "multivariate",
"variants" : {
"multivariate_variant1": {
"properties" : {
"multivariate_v1_a" : 1,
"multivariate_v1_b" : 2
},
"children" : [1,2,3]
},
"multivariate_variant2": {
"properties" : {
"multivariate_v2_a" : 3,
"multivariate_v2_b" : 42,
"multivariate_v2_d" : "fgh"
},
"children" : [4,5,6]
},
"multivariate_variant3": {
"properties" : {
"multivariate_v3_a" : "abc",
"multivariate_v3_b" : "def"
},
"children" : [7,8,9]
}
}
},
{
"type" : "blah",
"variants" : {
"blah_variant1": {
"properties" : {
"blah_v1_a" : 1,
"blah_v1_b" : 2
},
"children" : [{
"type" : "blah_sub1",
"variants" : {
"blah_sub1_variant1" : {
"properties" : {
"blah_s1_v1_a" : 12345,
"children" : ["a",1, "bn"]
}
}
}
}]
},
"blah_variant2": {
"properties" : {
"blah_v2_a" : 3,
"blah_v2_b" : 42,
"blah_v2_c" : "fgh"
},
"children" : [4,5,6]
}
}
}]
}
]
}
Result:
variants json
----------------------- ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
"multivariate_variant1" {"children": [1, 2, 3], "properties": {"multivariate_v1_a": 1, "multivariate_v1_b": 2}}
"multivariate_variant2" {"children": [4, 5, 6], "properties": {"multivariate_v2_a": 3, "multivariate_v2_b": 42, "multivariate_v2_d": "fgh"}}
"multivariate_variant3" {"children": [7, 8, 9], "properties": {"multivariate_v3_a": "abc", "multivariate_v3_b": "def"}}
"blah_variant1" {"children": [{"type": "blah_sub1", "variants": {"blah_sub1_variant1": {"properties": {"children": ["a", 1, "bn"], "blah_s1_v1_a": 12345}}}}], "properties": {"blah_v1_a": 1, "blah_v1_b": 2}}
"blah_variant2" {"children": [4, 5, 6], "properties": {"blah_v2_a": 3, "blah_v2_b": 42, "blah_v2_c": "fgh"}}
"blubb_variant1" {"children": ["z", "y"], "properties": {"blubb_v1_a": 100}}
"blubb_variant2" {"children": [], "properties": {"blubb_v2_a": 300, "blubb_v2_b": 4200}}
"blah_sub1_variant1" {"properties": {"children": ["a", 1, "bn"], "blah_s1_v1_a": 12345}}
The Query:
WITH RECURSIVE json_cte(variants, json) AS (
SELECT NULL::jsonb, json FROM (
SELECT '{/*FOR TEST DATA SEE ABOVE*/}'::jsonb as json
)s
UNION
SELECT
row_to_json(v)::jsonb -> 'key', -- D
CASE WHEN v IS NOT NULL THEN row_to_json(v)::jsonb -> 'value' ELSE c END -- C
FROM json_cte
LEFT JOIN LATERAL jsonb_array_elements(json -> 'children') as c ON TRUE -- A
LEFT JOIN LATERAL jsonb_each(json -> 'variants') as v ON TRUE -- B
)
SELECT * FROM json_cte WHERE variants IS NOT NULL
The WITH RECURSIVE structure checks elements in a recursive ways. The first UNION part is the starting point. The second part is the recursive part where the last calculation is taken for the next step.
A: if in the current JSON a children element exists all elements will be expanded into one row per child
B: if the current JSON has an element variants all elements will be expanded into a record. Note that in the example one JSON element can either contain a variants or a children element.
C: if there is a variants element then the expanded record will be converted back into a json. The resulting structure is {"key" : "name_of_variant", "value" : "json_of_variant"}. The values will be the JSON for the next recursion (the JSON of the variants can have own children elements. That's why it works). Otherwise the expanded children elements will be the next data
D: if there is a variants element then the key is printed
I have a JSONB column with JSON that looks like this:
{"id" : "3", "username" : "abcdef"}
Is there way to update the JSON to :
{"id" : 3, "username" : "abcdef"}
Update the jsonb column using the concatenate operator ||, e.g.:
create table example(id int, js jsonb);
insert into example values
(1, '{"id": "3", "username": "abcdef"}');
update example
set js = js || jsonb_build_object('id', (js->>'id')::int)
returning *;
id | js
----+---------------------------------
1 | {"id": 3, "username": "abcdef"}
(1 row)
select json_build_object('id',CAST(j->>'id' AS NUMERIC),'username',j->>'username')::jsonb from (
select '{"id" : "3", "username" : "abcdef"}' :: jsonb AS j
)t
I would like some help with flask, pymongo and some ideas.
What I want to do:
I have a list:
lg=[{'name_dep': name_dep, 'cod_prg': cod_prg, 'name_prg': name_prg,
'cod_form': cod_form, 'an_s': frm['an_s'], 'serie': serie,
'nr_s':frm['nr_s'], 'nr_gr': frm['nr_gr'], 'nr_sgr': frm['nr_sgr']}]
where frm=mongo.db.colecttion1
I want to create a temporary collection of which documents have the structure:
{
"_id" : ObjectId("576d3494b7e8f8d6d0d7d0e6"),
"name_dep" : "name_dep1",
"cod_prg" : "cod1",
"an_s" : "1",
"name_prg" : "name1",
"forms" : [
{
"nr_gr" : "5",
"nr_sgr" : "10",
"nr_s" : "150"
}
{
"nr_gr" : "6",
"nr_sgr" : "12",
"nr_s" : "90"
}
{
"nr_gr" : "2",
"nr_sgr" : "4",
"nr_s" : "50"
}
],
"cod_form" : "1",
"serie" : "XX"
}
The code for achieving that is:
if mongo.db.form_temp:
mongo.db.form_temp.drop()
coll_temp=mongo.db.form_temp
formatie1=lg[0]
coll_temp.insert({'name_dep':formatie1['name_dep'], 'cod_prg':formatie1['cod_prg'], 'name_prg':formatie1['name_prg'], 'cod_form':formatie1['cod_form'], 'an_s':formatie1['an_s'], 'serie':formatie1['serie'], 'forms':[{'nr_s':formatie1['nr_s'], 'nr_gr':formatie1['nr_gr'], 'nr_sgr':formatie1['nr_sgr']}]})
for y in range (1, len(lg)-1):
x=lg[y]
dep_in_coll=mongo.db.coll_temp.find_one({ '$and': [{'name_dep':x['name_dep']}, {'cod_prg':x['cod_prg']}]})
if dep_in_coll is None:
coll_temp.insert({'name_dep':x['name_dep'], 'cod_prg':x['cod_prg'], 'name_prg':x['name_prg'], 'cod_form':x['cod_form'], 'an_s':x['an_s'], 'serie':x['serie'], 'forms':[{'nr_s':x['nr_s'], 'nr_gr':x['nr_gr'], 'nr_sgr':x['nr_sgr']}]})
else:
iddoc=dep_in_coll['_id']
listforms=dep_in_coll['forms']
listforms.append([{'nr_s':x['nr_s'], 'nr_gr':x['nr_gr'], 'nr_sgr':x['nr_sgr']}])
dep_in_coll.update({'_id':iddoc}, {'$set' : {'forms': listforms}})
What I'doing is to drop the temporary collection if exists and then to create the collection again and insert the first element of the list lg.
Then iterate through the lg list and for each the same values of the pair name_dep , cod_prg modify the forms list by adding the fields nr_s, nr_gr, nr_sgr with their values.
My code doesn't run as expected, but inserting the documents in the temp collection, the collection having the same no. of documents as lg list, without modifying the forms list.
Thank you.
Schema
"_id" : ObjectId(""),
"top_level_array" : [
{
"array_one" : [
"1",
"2",
"3",
"4",
"5"
],
"name" : "user name",
"array_two" : [
"1",
"2"
],
"array_three" : [
"1",
"2"
]
},
{
// another document
},
{
// another document
}
]
Question
I can successfully set the value of a single array with:
db.users.update({email_addr:"email#email.com", "top_level_array.name":"user name"},{$set:{"top_level_array.$.array_one":["just one value now"]}})
How can I set the value of multiple arrays at the same time?
This worked for me:
Separate the different arrays to update with a comma eg:
db.users.update({email_addr:"email#email.com", "top_level_array.name":"user name"},{$set:{"top_level_array.$.array_one":["just one value now"],"top_level_array.$.array_two":["another value"]}})
And to clarify the context of this solution, the query is conditional in that it searches for a match on name within a document, and then updates arrays that are at the same level as name ie array_one and array_two.