I have simple table with a jsonb column
CREATE TABLE things (
id SERIAL PRIMARY KEY,
data jsonb
);
with data that looks like:
{
"id": 1,
"title": "thing",
"things": [
{
"title": "thing 1",
"moreThings": [
{ "title": "more thing 1" }
]
}
]
}
So how do I append inside of a deeply nested array like moreThings?
For single level nested array I could do this and it works:
UPDATE posts SET data = jsonb_set(data, '{things}', data->'things' || '{ "text": "thing" }', true);
But the same doesn't work for deeply nested arrays:
UPDATE posts SET data = jsonb_set(data, '{things}', data->'things'->'moreThings' || '{ "text": "thing" }', true)
How can I append to moreThings?
It works just fine:
UPDATE things
SET data =
jsonb_set(data,
'{things,0,moreThings}',
data->'things'->0->'moreThings' || '{ "text": "thing" }',
TRUE
)
WHERE id = 1;
If you have a table that consists only of a primary key and a jsonb attribute and you regularly want to manipulate this jsonb in the database, you are certainly doing something wrong. Your life will be much easier if you normalize the data some more.
Related
I have a complex object with JSONB (PostgreSQL 12), that has nested arrays in nested arrays and so on. I search for all invoices, that contains specific criteria.
create table invoice:
invoice_number primary key text not null,
parts: jsonb,
...
Object:
"parts": [
{
"groups": [
{
"categories": [
{
"items": [
{
...
"articleName": "article1",
"articleSize": "M",
},
{
...
"articleName": "article2"
"articleSize": "XXL",
}
]
}
]
}
]
},
{
"groups": [
...
]
},
]
I've a build a native query to search for items with a specific articleName:
select * from invoice i,
jsonb_array_elements(i.invoice_parts) parts,
jsonb_array_elements(parts -> 'groups') groups,
jsonb_array_elements(groups -> 'categories') categories,
jsonb_array_elements(categories -> 'items') items
where items ->> 'articleName' like '%name%' and items ->> 'articleSize' = 'XXL';
I assume i could improve search speed with indexing. I've read about Trigram indexes. Would it be the best type of indexing for my case? If yes -> how to build it for such complex object.
Thanks in regards for any advices.
The only option that might speed up this, is to create a GIN index on the parts column and use a JSON path operator:
select *
from invoice
where parts #? '$.parts[*].groups[*].categories[*].items[*] ? (#.articleName like_regex "name" && #.articleSize == "XXL")'
But I doubt this is going to be fast enough, even if that uses the index.
I am trying to add items to a JSON array using jsonb_insert as:
jsob_insert(document, '{path,to,array,0}','{"key":"value"}')
This inserts {"key":"value"} to the head of array. However, I want to do this insert iff the JSON object {"key":"value"} does not already exist in the array. Currently, it just adds it to the array without caring about duplicacy (which is how it is intended to work). I am just wondering if there is a way to take care of duplicate records this way.
Wrap that call in a case that checks whether the object already exists in the array:
with invars as (
select '{
"path": {
"to": {
"array": [
{
"key": "value"
}
]
}
}
}'::jsonb as document
)
select case
when jsonb_path_exists(document, '$.path.to.array[*].key ? (# == "value")')
then document
else jsonb_insert(document, '{path,to,array,0}', '{"key": "value"}')
end as desired_result,
jsonb_insert(document, '{path,to,array,0}', '{"key": "value"}') as old_result
from invars;
db<>fiddle here
I have a REST API that outputs JSON data similar to this example:
{
"GroupIds": [
"1234",
"2345",
"3456",
"4567"
],
"Id": "w5a19-a493-bfd4-0a0c8djc05",
"Name": "Test Item",
"Description": "test item description",
"Notes": null,
"ExternalId": null,
"ExpiryDate": null,
"ActiveStatus": 0,
"TagIds": [
"784083-4c77-b8fb-0135046c",
"86de96-44c1-a497-0a308607",
"7565aa-437f-af36-8f9306c9",
"d5d841-1762-8c14-d8420da2",
"bac054-2b6e-a19b-ef5b0b0c"
],
"ResourceIds": []
}
Using ADF, I want to parse through this JSON object and insert a row for each value in the GroupIds array along with the objects Id and Name... So ultimately the above JSON should translate to a table like this:
GroupID
Id
Name
1234
w5a19-a493-bfd4-0a0c8djc05
Test Item
2345
w5a19-a493-bfd4-0a0c8djc05
Test Item
3456
w5a19-a493-bfd4-0a0c8djc05
Test Item
4567
w5a19-a493-bfd4-0a0c8djc05
Test Item
Is there some configuration I can use in the Copy Activity settings to accomplish this?
You can use Data flow activity to get desired result.
First add the REST API source then use select transformer and add required columns.
After this select Derived Column transformer and use unfold function to flatten JSON array.
Another way is to use Flatten formatter.
I tend to use a more ELT pattern for this, ie passing the JSON to a Stored Proc activity and letting the SQL database handle the JSON. This assumes you already have access to a SQL DB which is very capable with JSON.
A simplified example:
DECLARE #json NVARCHAR(MAX) = '{
"GroupIds": [
"1234",
"2345",
"3456",
"4567"
],
"Id": "w5a19-a493-bfd4-0a0c8djc05",
"Name": "Test Item",
"Description": "test item description",
"Notes": null,
"ExternalId": null,
"ExpiryDate": null,
"ActiveStatus": 0,
"TagIds": [
"784083-4c77-b8fb-0135046c",
"86de96-44c1-a497-0a308607",
"7565aa-437f-af36-8f9306c9",
"d5d841-1762-8c14-d8420da2",
"bac054-2b6e-a19b-ef5b0b0c"
],
"ResourceIds": []
}'
SELECT
g.[value] AS groupId,
m.Id,
m.[Name]
FROM OPENJSON( #json, '$' )
WITH
(
Id VARCHAR(50) '$.Id',
[Name] VARCHAR(50) '$.Name',
GroupIds NVARCHAR(MAX) AS JSON
) m
CROSS APPLY OPENJSON( #json, '$.GroupIds' ) g;
You could convert this to a stored procedure where #json is the parameter and convert the SELECT to an INSERT.
My results:
I worked through a very similar example with more screenprints here which is worth a look. It's a different pattern to using Mapping Data Flows but if you already have SQL available then it makes sense to use it rather than fire up separate compute with duplicate cost. If you are not logging to a SQL DB or have access to one, then Mapping Data Flows approach might make sense to you.
I'm creating a stored procedure in postgres which takes a json and inserts into a table by performing a json_to_recordset command.
Prior to inserting the record set into a SQL table with that command I want to add an additional field as a index to the json in the json array.
Is this possible?
For each index in the array I want to add "current_status": "pending"
[
{
"batch_id": "40",
"state_id": "10"
},
{
"batch_id": "40",
"state_id": "10"
}
]
after
[
{
"batch_id": "40",
"state_id": "10",
"current_status": "pending"
},
{
"batch_id": "40",
"state_id": "10"
"current_status": "pending"
}
]
Another option is updating the only NEW records in the table after the fact.
I'm new to postgres and have been reading through the documentation.
Based on your added comment, the current_status = 'pending' should be added as part of the insert into your target table instead of appending the key to the json objects.
insert into target_table (batch_id, state_id, current_status)
select batch_id, state_id, 'pending' as current_status
from json_to_recordset(<json>) as x(batch_id text, state_id text);
Adding to Mike's correct answer, in the case we want to modify existing tables with json arrays to include the status we may:
-- First create the table
CREATE TABLE myJson AS
SELECT '[
{
"batch_id": "40",
"state_id": "10"
},
{
"batch_id": "50",
"state_id": "60"
}
]'::json js;
WITH unnest_and_concat AS (
SELECT json_array_elements(js)::jsonb || json_build_object('current_status', 'pending')::jsonb jee
FROM myJson)
SELECT json_agg(jee)::json
FROM unnest_and_concat;
Of course this is only intended to work for a table with these rows (for illustration). If the objective is to update the entire table then we can do that (ideally with a LATERAL JOIN) mixed with an update statement. Looks like:
UPDATE myJson
SET old_col=new_col
FROM <insert subquery or table>
WHERE myJson.id = new_table.id;
Nevertheless, I would recommend modifying upon insert rather than updating.
I want to retrive record which are matching to booking's client id & want to show it to client. I am doing the following:
$mongoDb = $mongoDb->selectCollection('booking');
$bookingInfo = $mongoDb->find(array("client.id" => $_SESSION['client_id']));
My mongo database record looks like this:
"paymentDue": "",
"client": {
"contacts": [
{
"name": "loy furison",
"email": "loy#hotmail.com"
}
],
"id": "5492abba64363df013000029",
"name": "Birdfire"
},
want to fire the query with key value as client.id in find function. But this query doesnt work..whats the issue
I got a little logic that is different by key name only. If i find it with client.name then i shows me records & there i need to insert these in json object & then through foreach loop each record if i retrive & compare then it works...got it but the expected doesnt work why?????...didnt get:-!