i have the following table schema:
i want to convert the items column into a json string but the output of TO_JSON_STRING isn't what i need.
when i run this query i get:
SELECT id, store, TO_JSON_STRING(items) AS items_json
FROM nested_array_example
but what i need is items_json to be:
{"table": "3", "lamp": "7", "swedish_thing": "729"}
is there a way to do that in bigquery?
here is the query to generate the table's data:
INSERT INTO `project_name.data_seT_name.nested_array_example` (store, items, id)
VALUES ("ikea", [("table","3"),("lamp","7"),("swedish_thing",'729'),("swedish_thing_made_in_china",'5723')], '1')
Consider below approach
select id, store,
( select '{' || string_agg(format('"%s": "%s"', name, value)) || '}'
from t.items
) items_json
from `project_name.data_seT_name.nested_array_example` t
if applied to sample data from your question - output is
Related
Source data
I am working on an ELT project to load data from CSV files into PostgreSQL where I will transform it. The CSV files have many columns that are consistent across files, but also contain activity columns that are inconsistent with names like Date (05/19/2020), Type (05/19/2020), etc.
In the loading script I am merging all of the columns with dates in the column name into one jsonb column so I don't have to constantly add new columns to the raw data table.
The resulting jsonb column in the raw data table looks like this:
id
activity
12345678
{"Date (05/19/2020)": null, "Type (05/19/2020)": null, "Date (06/03/2020)": "06/01/2020", "Type (06/03/2020)": "E"}
98765432
{"Date (05/19/2020)": "05/18/2020", "Type (05/19/2020)": "B", "Date (10/23/2020)": "10/26/2020", "Type (10/23/2020)": "T"}
JSON to columns
Using the amazing create_jsonb_flat_view function from this post I can convert the jsonb to columns like this:
id
Date (05/19/2020)
Type (05/19/2020)
Date (06/03/2020)
Type (06/03/2020)
Type (10/23/2020
Date (10/23/2020)
Type (10/23/2020)
10629465
null
null
06/01/2020
E
98765432
05/18/2020
B
10/26/2020
T
Need to move part of column name to row
Now, this is where I'm stuck. I need to remove the portion of the column name that is the Activity Date (e.g. (05/19/2020)) and create a row for each id and ActivityDate with additional columns for Date and Type like this:
id
ActivityDate
Date
Type
12345678
05/19/2020
null
null
12345678
06/03/2020
06/01/2020
E
98765432
05/19/2020
05/18/2020
B
98765432
10/23/2020
10/26/2020
T
I followed your link to the create_jsonb_flat_view article yesterday and then forgot this question. While I thank you for pointing me there, I think that mentioning it worked against you.
A more conventional approach using regexp_replace() works here. I left the date values as strings, but you can convert them with to_date() if needed:
with parse as (
select id, e.k, e.v,
regexp_replace(e.k, '\s+\([0-9/]{10}\)', '') as k_no_date,
regexp_replace(e.k, '^.+([0-9/]{10}).+', '\1') as k_date_only
from rawinput
cross join lateral jsonb_each_text(activity) as e(k, v)
)
select id,
k_date_only as activity_date,
min(v) filter (where k_no_date = 'Date') as date,
min(v) filter (where k_no_date = 'Type') as type
from parse
group by id, k_date_only;
db<>fiddle here
#Mike-Organek's Answer works beautifully!
However, I was curious if the regexp_replace() calls might be slowing the query down a bit and it seemed I could get the same results using a simpler function.
Since Mike gave me a great example to start with I modified it to split on the space between Date and (05/19/2020).
For 20,000 rows, it went from taking an avg of 7 sec on my local machine to an avg of .9 sec.
Here is the resulting query:
with parse as (
select id, e.k, e.v,
split_part(e.k, ' ', 1) as k_no_date,
trim(split_part(e.k, ' ', 2),'()') as k_date_only
from rawinput
cross join lateral jsonb_each_text(activity) as e(k, v)
)
select id,
k_date_only as activity_date,
min(v) filter (where k_no_date = 'Date') as date,
min(v) filter (where k_no_date = 'Type') as type
from parse
group by id, k_date_only;
I'm using postgresql 11, I have a jsonb which represent a row of that table, it's look like
{"userid":"test","rolename":"Root","loginerror":0,"email":"superadmin#ae.com",...,"thirdpartyauthenticationkey":{}}
is there any method that I could gather all the "values" of the jsonb into a string which is separated by ',' and without the keys?
The string I want to obtain with the jsonb above is like
(test, Root, 0, superadmin#ae.com, ..., {})
I need to keep the ORDER of those values as what their keys were in the jsonb. Could I do that with postgresql?
You can use the jsonb_populate_record function (assuming your json data does match the users table). This will force the text value to match the order of your users table:
Schema (PostgreSQL v13)
CREATE TABLE users (
userid text,
rolename text,
loginerror int,
email text,
thirdpartyauthenticationkey json
)
Query #1
WITH d(js) AS (
VALUES
('{"userid":"test", "rolename":"Root", "loginerror":0, "email":"superadmin#ae.com", "thirdpartyauthenticationkey":{}}'::jsonb),
('{"userid":"other", "rolename":"User", "loginerror":324, "email":"nope#ae.com", "thirdpartyauthenticationkey":{}}'::jsonb)
)
SELECT jsonb_populate_record(null::users, js),
jsonb_populate_record(null::users, js)::text AS record_as_text,
pg_typeof(jsonb_populate_record(null::users, js)::text)
FROM d
;
jsonb_populate_record
record_as_text
pg_typeof
(test,Root,0,superadmin#ae.com,{})
(test,Root,0,superadmin#ae.com,{})
text
(other,User,324,nope#ae.com,{})
(other,User,324,nope#ae.com,{})
text
Note that if you're building this string to insert it back into postgresql then you don't need to do that, since the result of jsonb_populate_record will match your table:
Query #2
WITH d(js) AS (
VALUES
('{"userid":"test", "rolename":"Root", "loginerror":0, "email":"superadmin#ae.com", "thirdpartyauthenticationkey":{}}'::jsonb),
('{"userid":"other", "rolename":"User", "loginerror":324, "email":"nope#ae.com", "thirdpartyauthenticationkey":{}}'::jsonb)
)
INSERT INTO users
SELECT (jsonb_populate_record(null::users, js)).*
FROM d;
There are no results to be displayed.
Query #3
SELECT * FROM users;
userid
rolename
loginerror
email
thirdpartyauthenticationkey
test
Root
0
superadmin#ae.com
[object Object]
other
User
324
nope#ae.com
[object Object]
View on DB Fiddle
You can use jsonb_each_text() to get a set of a text representation of the elements, string_agg() to aggregate them in a comma separated string and concat() to put that in parenthesis.
SELECT concat('(', string_agg(value, ', '), ')')
FROM jsonb_each_text('{"userid":"test","rolename":"Root","loginerror":0,"email":"superadmin#ae.com","thirdpartyauthenticationkey":{}}'::jsonb) jet (key,
value);
db<>fiddle
You didn't provide DDL and DML of a (the) table the JSON may reside in (if it does, that isn't clear from your question). The demonstration above therefore only uses the JSON you showed as a scalar. If you have indeed a table you need to CROSS JOIN LATERAL and GROUP BY some key.
Edit:
If you need to be sure the order is retained and you don't have that defined in a table's structure as #Marth's answer assumes, then you can of course extract every value manually in the order you need them.
SELECT concat('(',
concat_ws(', ',
j->>'userid',
j->>'rolename',
j->>'loginerror',
j->>'email',
j->>'thirdpartyauthenticationkey'),
')')
FROM (VALUES ('{"userid":"test","rolename":"Root","loginerror":0,"email":"superadmin#ae.com","thirdpartyauthenticationkey":{}}'::jsonb)) v (j);
db<>fiddle
I have a table named Test and in that one column (Subject) contains JSON values.
This is the query which i am using
select Name,Subject
from Test
where id =1;
And the following are the JSON values present inside table.
{
"subject":{
"Maths":"20",
"Physics":"21",
"English":"22"
},
"Staff":{
"English":"Anna",
"maths":"Rahul",
"Physics":"John"
}
}
Now my question is how to write a query to get English mark from JSON value.
Expected o/p is 22.
I am new to postgres, can any one help me in this thanks in advance
You can combine the -> and ->> operators
select Name,Subject, subject -> 'subject' ->> 'English' as english_mark
from Test
where id =1;
Alternatively use the #>> operator where you provide the path to the element you want as an array of keys:
select Name,Subject, subject #>> '{subject, English}' as english_mark
from Test
where id =1;
I want to dynamically concat strings in a jsonb array in my psql query which is as follows :
with cte as (select generate_series(1, 3) as num) select '["name", "number"||cte.num]'::jsonb as res from cte;
But I am getting this error :
ERROR: invalid input syntax for type json
LINE 1: ...e as (select generate_series(1, 5) as num) select '["name", ...
^
DETAIL: Token "|" is invalid.
CONTEXT: JSON data, line 1: ["name", "number"|...
So, here I am generating numbers from 1 to 3 and I want to append those numbers with "number" string in my jsonb array. This is what I want in my result:
res
----------------------
["name", "number1"]
["name", "number2"]
["name", "number3"]
Please tell me how do I get this. Thanks.
Something like this should work.
SELECT ( '["name", "number'|| num ||'"]' )::jsonb as res
FROM generate_series(1, 3) num
You were adding the pipes as characters after an element in a list. You should first build the array string (in parenthesis), and then convert it to jsonb.
I'm trying to insert data into a JSONB field based on a dependent table.
Essentially I want to do this (ignore why this is just an example query):
insert into myschema.teams (team_name, params)
select users.team_name, '{"team_name": teams.team_id, "user_name": users.username }'
from myschema.users
where users.team_name is not null;
As written I'm getting these errors:
ERROR: invalid input syntax for type json
LINE 2: ... '{"team_name...
^
DETAIL: Token "teams" is invalid.
CONTEXT: JSON data, line 1: {"team_name": teams...
You are using a string literal that doesn't contain valid JSON. There is no interpolation going on - you need use the jsonb_build_object function to create the JSONB value from dynamic values. (You could also do string concatenation and the cast from text to json, but please don't).
insert into myschema.teams (team_name, params)
select users.team_name, jsonb_build_object('team_name', teams.team_name, 'user_name', users.username)
from myschema.users
where users.team_name is not null;