I'm not sure if this is possible or not, but I'm trying to get into the nitty gritty of what I can do in postgres 9.6.1 and this seems like maybe its possible. So given this table:
DROP TABLE IF EXISTS live_data;
CREATE TABLE live_data (
rec_id TEXT,
control_data JSONB
);
CREATE UNIQUE INDEX rec_id_idx ON live_data (rec_id);
I want to be able to upsert individual props on the control_data json without having to upsert a whole new json string.
With no rows in that table yet I tried this:
INSERT INTO live_data(rec_id, control_data) VALUES ('1', '{"set":{"some_prop": 99}}')
ON CONFLICT (rec_id) DO UPDATE SET control_data->'set'->'some_prop' = 99;
FWIW I get this error on that query :
syntax error at or near "->"
Am I writing that query wrong and/or is what I want to do just not currently possible?
Use jsonb_set():
INSERT INTO live_data(rec_id, control_data)
VALUES ('1', '{"set":{"some_prop": 99}}');
INSERT INTO live_data(rec_id, control_data)
VALUES ('1', '{"set":{"some_prop": 88}}')
ON CONFLICT (rec_id) DO
UPDATE SET control_data =
jsonb_set(live_data.control_data, array['set','some_prop'], '88', true)
RETURNING *;
rec_id | control_data
--------+----------------------------
1 | {"set": {"some_prop": 88}}
(1 row)
If your json column has a value like below,
[
{
"Code":"xyz",
"Amount":[
{
"Type":"Pay1",
"Amount":"999",
"username":"henry"
},
{
"Type":"Pay2",
"Amount":"499",
"username":"rohilpatel",
"Bonus":"100"
}
],
"Currency":"$"
}
]
Below sql query will add the key-value pair OR update if exist at specified path. [Upsert will work like this way]
update tableName
SET columnName = jsonb_set(columnName, '{0,Amount,1,Bonus}', '200')
Related
We are using PostgREST to automatically generate a REST API for a Postgres database. Our primary keys have an external representation that's different from how we store them internally. For simplicity's sake lets pretend the ids are stored as integers but we represent them as hexadecimal strings outwardly.
It's simple enough to get PostgREST to convert to the external representation for read operations:
CREATE DOMAIN hexid AS bigint;
CREATE TABLE fruits (
fruit_id hexid PRIMARY KEY,
name text
);
CREATE OR REPLACE VIEW api_fruits AS
SELECT to_hex(fruit_id) as fruit_id, name FROM fruits;
INSERT INTO fruits(fruit_id, name) VALUES('51955', 'avocado');
PostgREST generates the expected representation when we GET api_fruits:
[
{
"fruit_id": "caf3",
"name": "avocado"
}
]
But that's about as far as we get with this solution. It's a one way transformation so we won't be able to POST/PATCH records this way. The way PostgREST works is to transform such requests into equivalent INSERT and UPDATE statements. But this view with its custom formatting is not updatable. This is what would happen if we tried:
ERROR: cannot insert into column "fruit_id" of view "api_fruits"
DETAIL: View columns that are not columns of their base relation are not updatable.
STATEMENT: WITH pgrst_source AS (WITH pgrst_payload AS (SELECT $1::json AS json_data), pgrst_body AS ( SELECT CASE WHEN json_typeof(json_data) = 'array' THEN json_data ELSE json_build_array(json_data) END AS val FROM pgrst_payload) INSERT INTO "api_x"."api_fruits"("fruit_id", "name") SELECT "fruit_id", "name" FROM json_populate_recordset (null::"api_x"."api_fruits", (SELECT val FROM pgrst_body)) _ RETURNING "api_x"."api_fruits".*) SELECT '' AS total_result_set, pg_catalog.count(_postgrest_t) AS page_total, CASE WHEN pg_catalog.count(_postgrest_t) = 1 THEN coalesce((
WITH data AS (SELECT row_to_json(_) AS row FROM pgrst_source AS _ LIMIT 1)
SELECT array_agg(json_data.key || '=eq.' || json_data.value)
FROM data CROSS JOIN json_each_text(data.row) AS json_data
WHERE json_data.key IN ('')
), array[]::text[]) ELSE array[]::text[] END AS header, '' AS body, nullif(current_setting('response.headers', true), '') AS response_headers, nullif(current_setting('response.status', true), '') AS response_status FROM (SELECT * FROM pgrst_source) _postgrest_t
We can't INSERT into "View columns that are not columns of their base relation".
The obvious workaround is to serve fruit_id as a straight column, just an integer. With some post and preprocessing at the nginx level we can hex encode it there (and hex decode incoming ids). I'm wondering if we can do better than that though. For large API operations, re-encoding the JSON will use a lot of memory and CPU time and it seems so unnecessary.
It would have been great to be able to use a custom CREATE CAST to take the incoming hexadecimal strings and turn them back into integers, something like this:
CREATE CAST (json AS hexid) WITH FUNCTION json_to_hexid AS ASSIGNMENT;
But alas custom casts are ignored on CREATE DOMAIN types. And we can't make a true custom column type because our cloud Postgres host (Google Cloud SQL) doesn't allow custom extensions.
It feels like some combination of INSTEAD OF triggers or rules could work. But when using query parameters to filter results using query parameters (e.g. select a fruit by id), I don't think there's an appropriate trigger to use. INSTEAD OF doesn't work for straight SELECT does it?
For example I've tested doing something like this to take care of INSERT and allow POST with PostgREST. It works:
CREATE OR REPLACE FUNCTION api_fruits_insert()
RETURNS trigger AS
$$
BEGIN
INSERT INTO fruits(fruit_id, name) VALUES (('x' || lpad(NEW.fruit_id, 16, '0'))::bit(64)::bigint::hexid, NEW.name);
RETURN NEW;
END
$$ LANGUAGE 'plpgsql';
CREATE TRIGGER api_fruits_insert
INSTEAD OF INSERT
ON api_fruits
FOR EACH ROW
EXECUTE PROCEDURE api_fruits_insert();
The trouble is in the WHERE clause. Let's PATCH api_fruits?fruit_id=in.(7b,caf3) with {"name": "pear"}. This works out of the box since the name column is updatable but look at the query:
WITH pgrst_source AS (WITH pgrst_payload AS (SELECT $1::json AS json_data), pgrst_body AS ( SELECT CASE WHEN json_typeof(json_data) = 'array' THEN json_data ELSE json_build_array(json_data) END AS val FROM pgrst_payload) UPDATE "api_x"."api_fruits" SET "name" = _."name" FROM (SELECT * FROM json_populate_recordset (null::"api_x"."api_fruits" , (SELECT val FROM pgrst_body) )) _ WHERE "api_x"."api_fruits"."fruit_id" = ANY ($2) RETURNING 1) SELECT '' AS total_result_set, pg_catalog.count(_postgrest_t) AS page_total, array[]::text[] AS header, '' AS body, nullif(current_setting('response.headers', true), '') AS response_headers, nullif(current_setting('response.status', true), '') AS response_status FROM (SELECT * FROM pgrst_source) _postgrest_t
DETAIL: parameters: $1 = '{
"name": "pear"
}', $2 = '{7b,caf3}'
So we have essentially UPDATE api_fruits SET name='berry' WHERE fruit_id IN ('7b', 'caf3');. Surprisingly this works but it's a full table scan so Postgres can evaluate to_hex(fruit_id) for each row looking for matches. The same happens if we try to GET a record by fruit_id. How would we rewrite the WHERE clauses?
It really feels like some combination of just the right Postgres and PostgREST features should be able to get us to a point where it's all happening in Postgres without nginx's help and without excessive complexity. Any ideas?
I wanted to perform a conditional insert in PostgreSQL. Something like:
INSERT INTO {TABLE_NAME} (user_id, data) values ('{user_id}', '{data}')
WHERE not exists(select 1 from files where user_id='{user_id}' and data->'userType'='Type1')
Unfortunately, insert and where does not cooperate in PostGreSQL. What could be a suitable syntax for my query? I was considering ON CONFLICT, but couldn't find the syntax for using it with JSON object. (Data in the example)
Is it possible?
Rewrite the VALUES part to a SELECT, then you can use a WHERE condition:
INSERT INTO { TABLE_NAME } ( user_id, data )
SELECT
user_id,
data
FROM
( VALUES ( '{user_id}', '{data}' ) ) sub ( user_id, data )
WHERE
NOT EXISTS (
SELECT 1
FROM files
WHERE user_id = '{user_id}'
AND data -> 'userType' = 'Type1'
);
But, there is NO guarantee that the WHERE condition works! Another transaction that has not been committed yet, is invisible to this query. This could lead to data quality issues.
You can use INSERT ... SELECT ... WHERE ....
INSERT INTO elbat
(user_id,
data)
SELECT 'abc',
'xyz'
WHERE NOT EXISTS (SELECT *
FROM files
WHERE user_id = 'abc'
AND data->>'userType' = 'Type1')
And it looks like you're creating the query in a host language. Don't use string concatenation or interpolation for getting the values in it. That's error prone and makes your application vulnerable to SQL injection attacks. Look up how to use parameterized queries in your host language. Very likely for the table name parameters cannot be used. You need some other method of either whitelisting the names or properly quoting them.
I have a json column (json_col) in a postgres database with the following structure:
{
"event1":{
"START_DATE":"6/18/2011",
"END_DATE":"7/23/2011",
"event_type":"active with prior experience"
},
"event2":{
"START_DATE":"8/20/11",
"END_DATE":"2/11/2012",
"event_type":"active"
}
}
[example of table structure][1]
How can I make a select statement in postgres to return the start_date and end_date with a where statement where "event_type" like "active"?
Attempted Query:
select person_id, json_col#>>'START_DATE' as event_start, json_col#>>'END_DATE' as event_end
from data
where json_col->>'event_type' like '%active%';
Returns empty columns.
Expected Response:
event_start
6/18/2011
8/20/2011
It sounds like you want to unnest your json structure, ignoring the top level keys and just getting the top level values. You can do this with jsonb_each, looking at resulting column named 'value'. You would put the function call in the FROM list as a lateral join (but since it is a function call, you don't need to specify the LATERAL keyword, it is implicit)
select value->>'START_DATE' from data, jsonb_each(json_col)
where value->>'event_type' like '%active%';
My database contains a table which has a column with jsonb type, and I want to update a part of these data using functions/operators from postgreSQL. Given we have this:
{
"A":[
{"index":"1"},
{"index":"2"}
],
"B":[
{"index":"3"},
{"index":"4"}
]
}
Let's say we went to add a key with an empty array to objects from "A" array, in order to have:
{
"A":[
{"index":"1", "myArray":[]},
{"index":"2", "myArray":[]}
],
"B":[
{"index":"3"},
{"index":"4"}
]
}
How can I proceed?
I've already tried this kind of things without success:
UPDATE myTable SET myColumn = (myColumn::jsonb)->>'A' || '{"myArray":[]}'
UPDATE myTable SET myColumn = (
SELECT jsonb_agg(jsonb_set(
element,
array['A'],
to_jsonb(((element ->> 'A')::jsonb || '{"myArray":[]}')::jsonb)
))
FROM jsonb_array_elements(myColumn::jsonb) element
)::json
UPDATE myTable SET myColumn = (
SELECT jsonb_each((element ->> 'A')::jsonb) || '{"myArray":[]}'::jsonb
FROM jsonb_array_elements(myColumn::jsonb) element
)::json
Obviously, all of these tests have been big failure. I have difficulties to understand how works postgreSQL functions.
Somebody can help?
The approach with jsonb_array_elements and jsonb_set was the right idea, but somehow you nested them the wrong way round:
UPDATE myTable SET myColumn = jsonb_set(myColumn, '{A}', (
SELECT jsonb_agg( element || '{"myArray":[]}' )
FROM jsonb_array_elements(myColumn -> 'A') element
));
(online demo)
Btw if your column already has jsonb data type, you shouldn't need any casts.
I have a table review(movie_id, user_id, reviewtext, date, time, likes, status)/
I get the error
column "exist" does not exist LINE 1: INSERT INTO review values ($1, $2, $3,$4,$5 ,0,"exist") ^ )
when I want to insert values into a postgresql database. I can not modify the code anymore so is there any way to make this work by altering the database like adding a column?
The code to insert is as follows:
$query = $this->db->prepare('INSERT INTO review values (:movieid, :userid, :review,:date,:time ,0,"exist")');
$result = $query->execute(Array(':movieid' => $movieid, ':userid' => $userid, ':review' => $review, ':date' => $date, ':time' => $time));
I understand that a way to fix this is to use single quotes for the column 'status' but the only thing I can do is alter the database.
No you can't.
If you had used proper insert - with named columns:
insert into review (column1, column2, column3) values (....)
then it could be theoretically possible to do by adding column "exist" and a trigger. But this would be very far away from being sane solution.