I have a table such as:
ID | Details
1 | {"name": "my_name", "phone": "1234", "address": "my address"}
2 | {"name": "his_name", "phone": "4321", "address": "his address"}
In this, Details is a jsonb object. I want to add another field named 'tags' to jsonb which should have some particular keys. In this case, "name", "phone". The final state after execution of the query should be:
ID | Details
1 | {"tags": {"name": "my_name", "phone": "1234"},"name": "my_name", "phone": "1234", "address":"my address"}
2 | {"tags": {"name": "his_name", "phone": "4321"},"name": "his_name", "phone": "4321", "address":"his address"}
I can think of the following steps to get this done:
Loop over each row and extract the details["name"] and details["phone"] in variables.
Add these variables to the jsonb.
I cant think of how the respective postgres query for this should be. Please guide.
use jsonb_build_object
update t set details
= jsonb_build_object ( 'tags',
jsonb_build_object( 'name', details->>'name', 'phone',details->>'phone')
)
|| details
DEMO
Use the concatenate operator, of course!
https://www.postgresql.org/docs/current/functions-json.html
update t1 set details = details || '{"tags": {"name": "my_name"}}' where id = 1
You can extract the keys you are interested in, build a new json value and append that to the column:
update the_table
set details = details || jsonb_build_object('tags',
jsonb_build_object('name', details -> 'name',
'phone', details -> 'phone'));
Related
I've created a stream and a table in this way:
CREATE STREAM user_stream
(id VARCHAR, name VARCHAR, age INT)
WITH (kafka_topic='user_topic', value_format='json', partitions=1);
CREATE TABLE user_table AS
SELECT
id,
LATEST_BY_OFFSET(name) as name,
LATEST_BY_OFFSET(age) as age
FROM user_stream
GROUP BY id
EMIT CHANGES;
And submit some event to the user_topic as:
{ "id": "user_1", "name": "Sherie Shine", "age": 31 }
{ "id": "user_2", "name": "Liv Denman", "age": 52 }
{ "id": "user_3", "name": "Frona Ness", "age": 44 }
Then query the table as:
SELECT * FROM user_table WHERE age > 40 EMIT CHANGES;
We'll get two rows:
+------------+----------------+-------+
|SID |NAME |AGE |
+------------+----------------+-------+
|user_2 |Frona Ness |44 |
|user_3 |Liv Denman |52 |
Post another message to the user_topic:
{ "id": "user_3", "age": 35 }
I'm expecting user_3 will be removed from the current query, but I've received nothing.
If I interrupt the current query with Ctrl+C and issue the same query again, I'll see only user_2 as it's the only one with age > 40 now.
How can we handle the update to remove a row from the filter?
The issue is gone after upgrading confluent-6.1.1 to confluent-6.2.0.
I have a table called "Bookmarks" that contains several standard rows and also a JSONB column called "columnsettings"
The content of this JSONB column looks like this.
[
{
"data": "id",
"width": 25
},
{
"data": "field_1",
"width": 125
},
{
"data": "field_12",
"width": 125
},
{
"data": "field_11",
"width": 125
},
{
"data": "field_2",
"width": 125
},
{
"data": "field_7",
"width": 125
},
{
"data": "field_8",
"width": 125
},
{
"data": "field_9",
"width": 125
},
{
"data": "field_10",
"width": 125
}
]
I am trying to write an update statement which would update this columnsettings by removing a specific node I specify. For example, I might want to update the columnsettings and remove just the node where data='field_2' as an example.
I have tried a number of things...I believe it will look something like this, but this is wrong.
update public."Bookmarks"
set columnsettings =
jsonb_set(columnsettings, (columnsettings->'data') - 'field_2');
What is the correct syntax to remove a node within a JSONB Array like this?
I did get a version working when there is a single row. This correctly updates the JSONB column and removes the node
UPDATE public."Bookmarks" SET columnsettings = columnsettings - (select position-1 from public."Bookmarks", jsonb_array_elements(columnsettings) with ordinality arr(elem, position) WHERE elem->>'data' = 'field_2')::int
However, I want it to apply to every row in the table. When there is more than 1 row, I get the error " more than one row returned by a subquery used as an expression"
How do I get this query to update all rows in the table?
UPDATED, the answer provided solved my issue.
I now have another JSONB column where I need to do the same filtering. The structure is a bit different, it looks likke this
{
"filters": [
{
"field": "field_8",
"value": [
1
],
"header": "Colors",
"uitype": 7,
"operator": "searchvalues",
"textvalues": [
"Red"
],
"displayfield": "field_8_options"
}
],
"rowHeight": 1,
"detailViewWidth": 1059
}
I tried using the syntax the same way as follows:
UPDATE public."Bookmarks"
SET tabsettings = filtered_elements.tabsettings
FROM (
SELECT bookmarkid, JSONB_AGG(el) as tabsettings
FROM public."Bookmarks",
JSONB_ARRAY_ELEMENTS(tabsettings) AS el
WHERE el->'filters'->>'field' != 'field_8'
GROUP BY bookmarkid
) AS filtered_elements
WHERE filtered_elements.bookmarkid = public."Bookmarks".bookmarkid;
This gives an error: "cannot extract elements from an object"
I thought I had the syntax correct, but how should this line be formatted?
WHERE el->'filters'->>'field' != 'field_8'
I tried this format as well to get to the array. This doesn't given an error, but it doesn't find any matches...even though there are records.
UPDATE public."Bookmarks"
SET tabsettings = filtered_elements.tabsettings
FROM (
SELECT bookmarkid, JSONB_AGG(el) as tabsettings
FROM public."Bookmarks",
JSONB_ARRAY_ELEMENTS(tabsettings->'filters') AS el
WHERE el->>'field' != 'field_8'
GROUP BY bookmarkid
) AS filtered_elements
WHERE filtered_elements.bookmarkid = public."Bookmarks".bookmarkid;
UPDATED .
This query now seems to work if there is more than one "filter" in the array.
However, if there is only 1 element in array which should be excluded, it doesn't remove the item.
UPDATE public."Bookmarks"
SET tabsettings = filtered_elements.tabsettings
FROM (
SELECT bookmarkid,
tabsettings || JSONB_BUILD_OBJECT('filters', JSONB_AGG(el)) as tabsettings
FROM public."Bookmarks",
-- this must be an array
JSONB_ARRAY_ELEMENTS(tabsettings->'filters') AS el
WHERE el->>'field' != 'field_8'
GROUP BY bookmarkid
) AS filtered_elements
WHERE filtered_elements.bookmarkid = public."Bookmarks".bookmarkid;
You can deconstruct, filter, and re-construct the JSONB array. Something like this should work:
UPDATE bookmarks
SET columnsettings = filtered_elements.columnsettings
FROM (
SELECT id, JSONB_AGG(el) as columnsettings
FROM bookmarks,
JSONB_ARRAY_ELEMENTS(columnsettings) AS el
WHERE el->>'data' != 'field_2'
GROUP BY id
) AS filtered_elements
WHERE filtered_elements.id = bookmarks.id;
Using JSONB_ARRAY_ELEMENTS, you transform the JSONB array into rows, one per object, which you call el. Then you can access the data attribute to filter out the "field_2" entry. Finally, you group by id to put the remainign values back together, and update the corresponding row.
EDIT If your data is a nested array in an object, override the object on the specific key:
UPDATE bookmarks
SET tabsettings = filtered_elements.tabsettings
FROM (
SELECT id,
tabsettings || JSONB_BUILD_OBJECT('filters', JSONB_AGG(el)) as tabsettings
FROM bookmarks,
-- this must be an array
JSONB_ARRAY_ELEMENTS(tabsettings->'filters') AS el
WHERE el->>'field' != 'field_2'
GROUP BY id
) AS filtered_elements
WHERE filtered_elements.id = bookmarks.id;
How can I refactor this update statement so that it updates a jsonb column in my postgresql table?
update contacts as c set latitude = v.latitude,longitude = v.longitude,
home_house_num = v.home_house_num,home_predirection = v.home_predirection,
home_street_name = v.home_street_name, home_street_type = v.home_street_type
from (values (16247746,40.814140,-74.259250,'25',null,'Moran','Rd'),
(16247747,20.900840,-156.373700,'581','South','Pili Loko','St'))
as v(contact_id,latitude,longitude,home_house_num,home_predirection,home_street_name,home_street_type) where c.contact_id = v.contact_id
My table looks like this ...
|contact_id (bigInt) |data (jsonb)|
-----------------------------------
|111231 |{"email": "albsmith#gmail.com", "home_zip": "07052", "lastname": "Smith",
"firstname": "Al", "home_phone": "1111111111", "middlename": "B",
"home_address1": "25 Moran Rd", "home_street_name": "Moran Rd"}
Note that I do not want to overwrite any other fields that may already exist in the jsonb object that are not specified in the update statement. For example, in this example, I would not want to overwrite the email or name fields
You could restructure your update into something like:
update
contacts c
set
data = c.data || v.data
FROM
(values
(
16247746,
'{
"latitude": 40.814140,
"longitude": -74.259250,
"home_house_num": "25",
"home_predirection": null,
"home_street_name": "Moran",
"home_street_type": "Rd"
}'::JSONB
),
(
16247747,
'{
"latitude": 20.900840,
"longitude": -156.373700,
"home_house_num": "581",
"home_predirection": "Sourth",
"home_street_name": "Pili Loko",
"home_street_type": "St"
}'::JSONB
)
) as v(
contact_id,
data
)
where
c.contact_id = v.contact_id
;
I am using the COPY command to load the JSON dataset from S3 to Redshift table. The data is getting loaded partially but it ignores records which has missing data(key-value/array) i.e. from the below example only the first record will get loaded.
Query:
COPY address from 's3://mybucket/address.json'
credentials 'aws_access_key_id=XXXXXXX;aws_secret_access_key=XXXXXXX'
maxerror as 250
json 's3:/mybucket/address_jsonpath.json';
My question is how can I load all the records from address.json even when some records will have missing key/data, similar to the below sample data set.
Sample of JSON
{
"name": "Sam P",
"addresses": [
{
"zip": "12345",
"city": "Silver Spring",
"street_address": "2960 Silver Ave",
"state": "MD"
},
{
"zip": "99999",
"city": "Curry",
"street_address": "2960 Silver Ave",
"state": "PA"
}
]
}
{
"name": "Sam Q",
"addresses": [ ]
}
{
"name": "Sam R"
}
Is there an alternative to FILLRECORD for JSON dataset?
I am looking for an implementation or a workaround which can load all the above 3 records in the Redshift table.
There is no FILLRECORD equivalent for COPY from JSON. It is explicitly not supported in the documentation.
But you have a more fundamental issue - the first record contains an array of multiple addresses. Redshift's COPY from JSON does not allow you to create multiple rows from nested arrays.
The simplest way to resolve this is to define the files to be loaded as an external table and use our nested data syntax to expand the embedded array into full rows. Then use an INSERT INTO to load the data to a final table.
DROP TABLE IF EXISTS spectrum.partial_json;
CREATE EXTERNAL TABLE spectrum.partial_json (
name VARCHAR(100),
addresses ARRAY<STRUCT<zip:INTEGER
,city:VARCHAR(100)
,street_address:VARCHAR(255)
,state:VARCHAR(2)>>
)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
LOCATION 's3://my-test-files/partial_json/'
;
INSERT INTO final_table
SELECT ext.name
, address.zip
, address.city
, address.street_address
, address.state
FROM spectrum.partial_json ext
LEFT JOIN ext.addresses address ON true
;
-- name | zip | city | street_address | state
-- -------+-------+---------------+-----------------+-------
-- Sam P | 12345 | Silver Spring | 2960 Silver Ave | MD
-- Sam P | 99999 | Curry | 2960 Silver Ave | PA
-- Sam Q | | | |
-- Sam R | | | |
NB: I tweaked your example JSON a little to make this simpler. For instance you had un-keyed objects as the values for name that I made into plain string values.
how about...
{
"name": "Sam R"
"address": ""
}
I need a help for the below mentioned detail,
I am using Postgres + Spring-Data-JPA. Moreover, I have used the jsonb data type for storing data.
I am trying to execute a query, but it gives me the following error:
ERROR: set-returning functions are not allowed in WHERE
The cause here is that I have added a jsonb condition in the WHERE clause (kindly refer to the below query for more detail).
Query (I have renamed column name just because of hiding the actual column name):
select distinct
jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'firstName' as firstName,
jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'lastName' as lastName,
jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'country' as country
from
tale1 table10_
left outer join
table2 table21_
on table10_.end_user_id=table21_.end_user_id
left outer join
table3 table32_
on table10_.manufacturer_id=table32_.manufacturer_id
where
table21_.end_user_uuid=(
?
)
and table21_.is_active=true
and table32_.manufacturer_uuid=(
?
)
and table32_.is_active=true
and table10_.is_active=true
and table32_.is_active=true
and jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'action' = ('PENDING')
order by
jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'firstName',
jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'lastName'
limit ?
The following line in the above query is causing the error:
and jsonb_array_elements(initiated_referral_detail->'listOfAttribue')::jsonb
->> 'action' = ('PENDING')
Can anyone please guide me about how do fetch data from the inner JSON? Especially in my case I have an inner List and a few elements inside.
I recommend a lateral join with jsonb_array_elements for cases like that. Here is an example:
CREATE TABLE tale1 (
id integer PRIMARY KEY,
initiated_referral_detail jsonb NOT NULL
);
INSERT INTO tale1 VALUES
(1, '{
"name": "one",
"listOfAttribue": [
{ "id": 1, "action": "DONE"},
{ "id": 2, "action": "PENDING" },
{ "id": 3, "action": "ACTIVE" }
]
}');
INSERT INTO tale1 VALUES
(2, '{
"name": "two",
"listOfAttribue": [
{ "id": 1, "action": "DONE"},
{ "id": 2, "action": "ACTIVE" }
]
}');
To find all ids where the associated JSON contains an array element with action = PENDING, you can query like this:
SELECT DISTINCT id
FROM tale1 CROSS JOIN LATERAL
jsonb_array_elements(initiated_referral_detail -> 'listOfAttribue') AS attr
WHERE attr ->> 'action' = 'PENDING';