I have multiple jsonb columns and i want to extract data from each column.
I am using postgressql.
Input:-
ID Jsoncol1 Jsoncol2 Jsoncol3 Date
0 {"#class": "team": {"id":"Captain","dob": [1990, 9, 11]}} {"#class": "group": {"id":"Colour","dob": [1990, 9, 11]}} {"#class": "person": {"id":"Red","dob": [1990, 9, 11]}]} 13/05/2019
Output:-
ID Team Group Person Date
0 Captain Colour Red 13/05/2019
Without #class: (which makes the JSON value invalid) the data could look like that
id | jsoncol1 | jsoncol2 | jsoncol3 | mydate
-: | :---------------------------------------------- | :---------------------------------------------- | :-------------------------------------------- | :---------
0 | {"team": {"id":"Captain","dob": [1990, 9, 11]}} | {"group": {"id":"Colour","dob": [1990, 9, 11]}} | {"person": {"id":"Red","dob": [1990, 9, 11]}} | 2019-05-13
The problem seems to be, extracting the id value from the JSON object:
demo:db<>fiddle
SELECT
id,
jsoncol1 -> 'team' ->> 'id' AS team,
jsoncol2 -> 'group' ->> 'id' AS "group",
jsoncol3 -> 'person' ->> 'id' AS person,
mydate
FROM mytable
-> Postgres JSON documentation
Related
Using the hypothetical schema:
CREATE TABLE obj (id INT, name VARCHAR);
CREATE TABLE objprop (obj_id INT, key VARCHAR, value VARCHAR);
INSERT INTO obj VALUES
(1, 'Object 1'),
(2, 'Object 2'),
(3, 'Object 3');
INSERT INTO objprop VALUES
(1, 'created', '2020-02-16'),
(1, 'updated', '2020-02-28'),
(2, 'created', '2020-02-01');
Could I obtain a list of objects (one per row), and a JSON field that represents object's properties?
I know I can use the ARRAY() function with a subquery to retrieve an array of values, for example:
SELECT id, name, ARRAY(SELECT value FROM objprop where obj_id=id) values FROM obj;
+----+----------+------------------------------+
| id | name | values |
+----+----------+------------------------------+
| 1 | Object 1 | {'2020-02-16', '2020-02-28'} |
| 2 | Object 2 | {'2020-02-01'} |
| 3 | Object 3 | {} |
+----+----------+------------------------------+
But could I make a query that instead of an ARRAY, it would return me a JSON column with the subquery in it? My goal is to obtain for example:
+---+----------+----------------------------------------------------------------------------------------+
| 1 | Object 1 | [{"key": "created", "value": "2020-02-16"}, {"key": "updated", "value": "2020-02-28"}] |
| 2 | Object 2 | [{"key": "created", "value": "2020-02-01"}] |
| 3 | Object 3 | [] |
+---+----------+----------------------------------------------------------------------------------------+
SELECT
id,
name,
COALESCE((
SELECT json_agg(json_build_object('key', key, 'value', value))
FROM objprop where obj_id=id
), '[]'::json) vals
FROM
obj;
I have a table of transactions where I need to group by customerId and create aggregated JSON records in an array.
customerId|transactionVal|day
1234| 2|2019-01-01
1234| 3|2019-01-04
14| 1|2019-01-01
What I'm looking to do is return something like this:
customerId|transactions
1234|[{'transactionVal': 2, 'day': '2019-01-01'}, {'transactionVal': 3, 'day': '2019-01-04'}]
14|[{'transactionVal': 1, 'day': '2019-01-01'}]
I will need to later iterate through each transaction in the array to calculate % changes in the transactionVal.
I searched for a while but could not seem to find something to handle this as the table is quite large > 70mil rows.
Thanks!
Should be possible to use array_agg and json_build_object like so:
ok=# select customer_id,
array_agg(json_build_object('thing', thing, 'time', time))
from test group by customer_id;
customer_id | array_agg
-------------+---------------------------------------------------------------------------------------------------
2 | {"{\"thing\" : 1, \"time\" : 1}"}
1 | {"{\"thing\" : 1, \"time\" : 1}",
"{\"thing\" : 2, \"time\" : 2}",
"{\"thing\" : 3, \"time\" : 3}"}
(2 rows)
ok=# select * from test;
customer_id | thing | time
-------------+-------+------
1 | 1 | 1
2 | 1 | 1
1 | 2 | 2
1 | 3 | 3
(4 rows)
ok=#
In our database we have a data set sorta like the following:
+----+-------------------------------+
| id | stuff |
+----+-------------------------------+
| 1 | {} |
+----+-------------------------------+
| 2 | {"a": "something", "b": null} |
+----+-------------------------------+
| 3 | {"c": null, "d": null} |
+----+-------------------------------+
I would like to match only, in this case, the one with id = 2, reason being that at least one of the values in the object is not null.
How can this be done with PostgreSQL?
I know one can do something like WHERE stuff != '{}' but that of course only checks for an empty object
Or something like WHERE (stuff->>'a') IS NOT NULL, but the thing is the list of keys in the objects are not hardcoded, could be anything
Use the function jsonb_each_text() or json_each_text(), example:
with my_table(id, jdata) as (
values
(1, '{}'::jsonb),
(2, '{"a": "something", "b": null}'),
(3, '{"c": null, "d": null}')
)
select distinct t.*
from my_table t
cross join jsonb_each_text(jdata)
where value is not null;
id | jdata
----+-------------------------------
2 | {"a": "something", "b": null}
(1 row)
This query (proposed by Abelisto, see the comments) should be more performant on a larger dataset:
select t.*
from my_table t
where exists (
select 1
from jsonb_each_text(jdata)
where value is not null);
I currently have a table which contains a column with a JSON object representing Twitter cashtags.
For example, this is my original query:
SELECT
DATA->'id' as tweet_id,
DATA->'text' as tweet_text,
DATA->'entities'->'symbols' as cashtags
FROM documents
LIMIT 10
The cashtags column will return something like
[{"text":"HEMP","indices":[0,5]},{"text":"MSEZ","indices":[63,68]}]
How can I traverse this datatype, which is listed as jsonb, in order to say, only return results where the text is equal to HEMP or MSEZ?
The value data->'entities'->'symbols' is a json array. You can unnest the array using the function jsonb_array_elements(), e.g.:
SELECT
data->'id' as tweet_id,
data->'text' as tweet_text,
value as cashtag
FROM documents,
jsonb_array_elements(data->'entities'->'symbols')
where value->>'text' in ('HEMP', 'MSEZ');
tweet_id | tweet_text | cashtag
----------+------------+---------------------------------------
1 | "my_tweet" | {"text": "HEMP", "indices": [0, 5]}
1 | "my_tweet" | {"text": "MSEZ", "indices": [63, 68]}
(2 rows)
or:
SELECT DISTINCT
data->'id' as tweet_id,
data->'text' as tweet_text,
data->'entities'->'symbols' as cashtags
FROM documents,
jsonb_array_elements(data->'entities'->'symbols')
WHERE value->>'text' in ('HEMP', 'MSEZ');
tweet_id | tweet_text | cashtags
----------+------------+------------------------------------------------------------------------------
1 | "my_tweet" | [{"text": "HEMP", "indices": [0, 5]}, {"text": "MSEZ", "indices": [63, 68]}]
(1 row)
Imagining I have a table cars with a field data inside:
CARS
name | data
car 1 | { "doors" => "5", "engine" => "1.1" }
car 2 | { "doors" => "3", "engine" => "1.1", "air_conditioning" => "true" }
car 3 | { "doors" => "5", "engine" => "1.4" }
Assuming data keys are dynamic (more can be added), how can I create a pivot table from that data like this:
CROSSTAB
name | doors | engine | air_conditioning
car 1 | 5 | 1.1 |
car 2 | 3 | 1.1 | "true"
car 3 | 5 | 1.4 |
Here's how to get the result you asked for:
CREATE TABLE hstore_test (id bigserial primary key, title text, doors integer, engine text, air_conditioning boolean)
INSERT INTO hstore_test (title, doors, engine, air_conditioning)
VALUES ('Car1', 2, '1.1', false), ('Car2', 4, '1.2', true), ('Car3', 3, '1.3', false), ('Car4', 5, '1.4', null);
DROP TABLE IF EXISTS hstore_persist;
CREATE TABLE hstore_persist AS
SELECT hstore(t) car_data FROM hstore_test AS t;
SELECT car_data->'title' "name", car_data->'doors' doors, car_data->'engine' engine, car_data->'air_conditioning' air_conditioning
FROM hstore_persist
This will result in the table
name | doors | engine | air_conditioning
Car1 | 2 | 1.1 | f
Car2 | 4 | 1.2 | t
Car3 | 3 | 1.3 | f
Car4 | 5 | 1.4 |
There is nothing "crosstab" about it, though. This is just using the accessor methods of an hstore to display the data in the way you show in the example.