What is the proper way to convert any text (or varchar) to jsonB type in Postgres (version 9.6) ?
For example, here I am using two methods and I am getting different results:
Method 1:
dev=# select '[{"field":15,"operator":0,"value":"1"},{"field":15,"operator":0,"value":"2"},55]'::jsonb;
jsonb
----------------------------------------------------------------------------------------------
[{"field": 15, "value": "1", "operator": 0}, {"field": 15, "value": "2", "operator": 0}, 55]
(1 row)
Method 2 , which doesn't produce the desired results, btw:
dev=# select to_jsonb('[{"field":15,"operator":0,"value":"1"},{"field":15,"operator":0,"value":"2"},55]'::text);
to_jsonb
----------------------------------------------------------------------------------------------------
"[{\"field\":15,\"operator\":0,\"value\":\"1\"},{\"field\":15,\"operator\":0,\"value\":\"2\"},55]"
(1 row)
dev=#
Here, it was converted to a string, not an array.
Why doesn't the second method creates an array ?
According to Postgres documentation:
to_jsonb(anyelemnt)
Returns the value as json or jsonb. Arrays and composites are
converted (recursively) to arrays and objects; otherwise, if there is
a cast from the type to json, the cast function will be used to
perform the conversion; otherwise, a scalar value is produced. For any
scalar type other than a number, a Boolean, or a null value, the text
representation will be used, in such a fashion that it is a valid json
or jsonb value.
IMHO you are providing a JSON formatted string, then you should use the first method.
to_json('Fred said "Hi."'::text) --> "Fred said \"Hi.\""
If you try to get an array of element using to_json(text) you'll get the next error:
select *
from jsonb_array_elements_text(to_jsonb('[{"field":15,"operator":0,"value":"1"},{"field":15,"operator":0,"value":"2"},55]'::text));
cannot extract elements from a scalar
But if you previously cast it to json:
select *
from jsonb_array_elements_text(to_jsonb('[{"field":15,"operator":0,"value":"1"},{"field":15,"operator":0,"value":"2"},55]'::json));
+--------------------------------------------+
| value |
+--------------------------------------------+
| {"field": 15, "value": "1", "operator": 0} |
+--------------------------------------------+
| {"field": 15, "value": "2", "operator": 0} |
+--------------------------------------------+
| 55 |
+--------------------------------------------+
If your text is just a json format text, you could just explicitly cast it to json/jsonb like this:
select '{"a":"b"}'::jsonb
Atomic type conversion and CSV-to-JSONb
A typical parse problem in open data applications is to parse line by line a CSV (or CSV-like) text into JSONB correct (atomic) datatypes. Datatypes can be defined in SQL jargon ('int', 'text', 'float', etc.) or JSON jargon ('string', 'number'):
CREATE FUNCTION csv_to_jsonb(
p_info text, -- the CSV line
coltypes_sql text[], -- the datatype list
rgx_sep text DEFAULT '\|' -- CSV separator, by regular expression
) RETURNS JSONb AS $f$
SELECT to_jsonb(a) FROM (
SELECT array_agg(CASE
WHEN tp IN ('int','integer','smallint','bigint') THEN to_jsonb(p::bigint)
WHEN tp IN ('number','numeric','float','double') THEN to_jsonb(p::numeric)
WHEN tp='boolean' THEN to_jsonb(p::boolean)
WHEN tp IN ('json','jsonb','object','array') THEN p::jsonb
ELSE to_jsonb(p)
END) a
FROM regexp_split_to_table(p_info,rgx_sep) WITH ORDINALITY t1(p,i)
INNER JOIN unnest(coltypes_sql) WITH ORDINALITY t2(tp,j)
ON i=j
) t
$f$ language SQL immutable;
-- Example:
SELECT csv_to_jsonb(
'123|foo bar|1.2|true|99999|{"x":123,"y":"foo"}',
array['int','text','float','boolean','bigint','object']
);
-- results [123, "foo bar", 1.2, true, 99999, {"x": 123, "y": "foo"}]
-- that is: number, string, number, true, number, object
Related
I have a column products in a table test which is of type of text in the below format:
[{"is_bulk_product": false, "rate": 0, "subtotal": 7.17, "qty": 2, "tax": 0.90}]
It is an array with nested dictionary values. When I tried to alter the column using this:
alter table test alter COLUMN products type jsonb using products::jsonb;
I get below error:
ERROR: 22P02: invalid input syntax for type json
DETAIL: Character with value 0x09 must be escaped.
CONTEXT: JSON data, line 1: ...some_id": 2613, "qty": 2, "upc": "1234...
LOCATION: json_lex_string, json.c:789
Time: 57000.237 ms (00:57.000)
how can we make sure the json is valid before altering the column ?
Thank You
Your written JSON string is correct, so this SQL code execute without exception:
select '[{"is_bulk_product": false, "rate": 0, "subtotal": 7.17, "qty": 2, "tax": 0.90}]'::jsonb
Maybe the table has an incorrect JSON format in other records. You can firstly check this by selecting data, example:
select products::jsonb from test;
And you have incorrect syntax on your SQL code, you can cast products field to JSONB but not a test, test is your table name:
alter table test
alter COLUMN products type jsonb using products::jsonb;
I have table with JSON-b field like this:
id | data
----------
1 | '{"points": [{"id": 10, "address": "Test 1"}, {"id": 20, "address": "Test 2"}, {"id": 30, "address": "Test 3"}]}'
2 | '{"points": [{"id": 40, "address": "Test 444"}, {"id": 20, "address": "Test 222"}, {"id": 50, "address": "Test 555"}]}'
The JSON-b field "data" contains "points" array.
How to get all "points" whose point id is contained in an array [40, 20]? Like classic IN:
... IN (40,20)
Query must use GIN index!!! Array IDs will be sub-query.
You could almost do it with a functional index using a jsonb_path_query_array to extract the data. But as far as I can tell, not quite.
create index on t using gin (jsonb_path_query_array(x,'$.points[*].id'));
And then query with:
select * from t where jsonb_path_query_array(x,'$.points[*].id') ?| '{20,40}';
The problem is that ?| only works with text elements, while in your data the values of 'id' are integers, not text. I thought jsonpath would provide a way to convert them to text, but if it does, I cannot find it.
So instead I think you will have to define your own function which accepts jsonb, and returns int[] or text[] (or jsonb which is an array of text conversions). Then you can build an index on the results of this function. Don't forget to declare it immutable.
You will need to unnest the array (essentially normalizing your data model "on-the-fly") then you can use a subquery to check the value:
select t.*
from the_table t
where exists (select *
from jsonb_array_elements(t.data -> 'points') as x(element)
where (x.element ->> 'id')::int in (select id
from other_table))
I have a table in postgresql that has two columns:
Table "schemaname.tablename"
Column | Type | Collation | Nullable | Default
--------+-------------------+-----------+----------+---------
_key | character varying | | not null |
value | jsonb | | |
Indexes:
"tablename_pkey" PRIMARY KEY, btree (_key)
and I'd like to convert a nested property value of the jsonb that looks like this:
{
"somekey": "[k1=v1, k2=v2, k3=v2]",
}
into this:
{
"somekey": [
"java.util.LinkedHashMap",
{
"k1": "v1",
"k2": "v2",
"k3": "v3"
}
]
}
I've managed to parse the comma separted string into an array of strings but aside from having to still apply another split on '=' I don't really know how to do the actual UPDATE on all rows of the table and generate the proper jsonb value for "somekey" key.
select regexp_split_to_array(RTRIM(LTRIM(value->>'somekey','['),']'),',') from schemaname.tablename;
Any ideas?
Try this one (self-contained test data):
WITH tablename (_key, value) AS (
VALUES
('test', '{"somekey":"[k1=v1, k2=v2, k3=v2]"}'::jsonb),
('second', '{"somekey":"[no one=wants to, see=me, with garbage]"}'::jsonb),
('third', '{"somekey":"[some,key=with a = in it''s value, some=more here]"}'::jsonb)
)
SELECT
tab._key,
jsonb_insert(
'{"somekey":["java.util.LinkedHashMap"]}', -- basic JSON structure
'{somekey,0}', -- path to insert after
jsonb_object( -- create a JSONB object on-the-fly from the key-value array
array_agg(key_values) -- aggregate all key-value rows into one array
),
true -- we want to insert after the matching element, not before it
) AS json_transformed
FROM
tablename AS tab,
-- the following is an implicit LATERAL join (function based on eahc row for previous table)
regexp_matches( -- produces multiple rows
btrim(tab.value->>'somekey', '[]'), -- as you started with
'(\w[^=]*)=([^,]*)', -- define regular expression groups for keys and values
'g' -- we want all key-value sets
) AS key_values
GROUP BY 1
;
...resulting in:
_key | json_transformed
--------+-------------------------------------------------------------------------------------------------------
second | {"somekey": ["java.util.LinkedHashMap", {"see": "me", "no one": "wants to"}]}
third | {"somekey": ["java.util.LinkedHashMap", {"some": "more here", "some,key": "with a = in it's value"}]}
test | {"somekey": ["java.util.LinkedHashMap", {"k1": "v1", "k2": "v2", "k3": "v2"}]}
(3 rows)
I hope the inline comments explain how it works in enough detail.
Without requiring aggregate/group by:
The following requires no grouping as we don't need aggregate function array_agg, but are a little bit less strict on the key-value format and will break a query easily because of some data (the previous variant will just drop some key-value):
WITH tablename (_key, value) AS (
VALUES
('test', '{"somekey":"[k1=v1, k2=v2, k3=v2]"}'::jsonb),
('second', '{"somekey":"[no one=wants to, see=me, with garbage]"}'::jsonb)
)
SELECT
_key,
jsonb_insert(
'{"somekey":["java.util.LinkedHashMap"]}', -- basic JSON structure
'{somekey,0}', -- path to insert after
jsonb_object( -- create a JSONB object on-the-fly from the key-value array
key_values -- take the keys + values as split using the function
),
true -- we want to insert after the matching element, not before it
) AS json_transformed
FROM
tablename AS tab,
-- the following is an implicit LATERAL join (function based on eahc row for previous table)
regexp_split_to_array( -- produces an array or keys and values: [k, v, k, v, ...]
btrim(tab.value->>'somekey', '[]'), -- as you started with
'(=|,\s*)' -- regex to match both separators
) AS key_values
;
...results into:
_key | json_transformed
--------+--------------------------------------------------------------------------------
test | {"somekey": ["java.util.LinkedHashMap", {"k1": "v1", "k2": "v2", "k3": "v2"}]}
second | {"somekey": ["java.util.LinkedHashMap", {"see": "me", "no one": "wants to"}]}
(2 rows)
Feeding it with garbage (as in the "second" row before) or with an = character in the value (as in the "third" row before) would result in the following error here:
ERROR: array must have even number of elements
I have been searching for this answer for long on stackoverflow, but did not find anything useful.
I have data in my_tbl as:
folder_id | links
--------------+----------------------------------------------------------------
761 | [{"ids": "[82293,82292]", "index": "index_1"}, {"ids": "[82293,82292]", "index": "index_2"}]
769 | [{"ids": "[82323,82324]", "index": "index_3"}]
572 | [{"ids": "[80031,79674,78971]", "index": "index_4"}]
785 | [{"ids": "[82367,82369]", "index": "index_5"}, {"ids": "[82368,82371]", "index": "index_6"}]
768 | [{"ids": "[82292,82306]", "index": "index_7"}]
I want to get all the rows where links->>'ids' contain 82292. So in this case, it should return me folder_id 761 and 768.
I achieved so far is separating ids arrays from links by -
jsonb_array_elements(links) ->> 'ids'
not sure how to proceed further.
You can use jsonb_array_elements to convert a json array to a rowset. You can use this to at the value of "ids". Using the #> operator you can check if the array contains a value:
select distinct folder_id
from YourTable
cross join lateral
jsonb_array_elements(links) ids(ele)
where (ele->>'ids')::jsonb #> '[82292]'
A tricky part is that "[82293,82292]" is stored as a string, not an array. The ele->>'ids' expression retrieves "[82293,82292]" as string using the ->> operator (the double > makes it return text instead of jsonb.) The content of the string is converted to an array by ::jsonb.
Example at rextester.com.
Is it possible to use LIKE operator for single key/value inside array of objects for jsonb field in PostgreSQL 9.4? For example I have:
id | body
------------------------------------------------------------
1 | {"products": [{"name": "qwe", "description": "asd"}, {"name": "zxc", "description": "vbn"}]}
I know, I can get a product with something like this:
select * from table where 'body'->'products' #> '[{"name": "qwe"}]'::jsonb
The question is: can I get this product if I don't know full name of it?
Try to get the key and value by using jsonb_each() function:
WITH json_test(data) AS ( VALUES
('{"products": [{"name": "qwe", "description": "asd"}, {"name": "zxc", "description": "vbn"}]}'::JSONB)
)
SELECT doc.key,doc.value
FROM json_test jt,
jsonb_array_elements(jt.data->'products') array_elements,
jsonb_each(array_elements) doc
WHERE
doc.key = 'name'
AND
doc.value::TEXT LIKE '%w%';
Output will be the following:
key | value
------+-------
name | "qwe"
(1 row)