Related
Basically I'm trying to compare two JSONB rows and return a numeric value. But I wanna be able to query for it. I'm not sure whether I should use a custom SQL function, a calculated field, or a Postgres generated column, so I need a bit of advice.
I have a jsonb column for each user that keeps a few hundreds of keys/values as such:
USERS TABLE:
| username | user_jsonb_column |
|-----------------------------------------------------------|
| 'user1' | {"key1":"value1", "key2":"value2" ... } |
|--------------|--------------------------------------------|
| 'user2' | {"key2":"value2", "key3":"value3" ... } |
I am trying to calculate the similarity of the jsonb rows of 2 users with a very simple SQL query as such:
SELECT ROUND ((
SELECT COUNT(*) from (
SELECT jsonb_each(user_jsonb_column)
FROM users WHERE username = 'johndoe'
INTERSECT
SELECT jsonb_each(user_jsonb_column)
FROM users WHERE username = 'janedoe'
)::decimal AS SAME_PAIRS
/ --divide it by
SELECT COUNT(*) from (
SELECT jsonb_object_keys(user_jsonb_column)
FROM users WHERE username = 'johndoe'
INTERSECT
SELECT jsonb_object_keys(user_jsonb_column)
FROM users WHERE username = 'janedoe'
) as SAME_KEYS
) * 100) as similarity_percentage
This is working as intended and gives me the similarity result between 2 json objects as a percentage.
I am trying to turn this into a function so that I can query for the similarity percentage of 2 users as such:
query {
calculate_similarity_percentage(
args: {user1: "johndoe", user2: "janedoe"}
){
similarity_percentage_value
}
}
But I'm stuck at this point because I'm not sure whether I should think in terms of a trackable custom SQL function (which should return SETOF <TABLE> but I need a numeric value), a computed field (which can also return BASE type), or maybe a Postgres generated column in my situation.
I've been reading https://hasura.io/docs/1.0/graphql/core/schema/custom-functions.html and https://hasura.io/docs/1.0/graphql/core/schema/computed-fields.html but I couldn't quite figure out how to approach this, so any kind of help or comment would be appreciated.
Update: Yes, as Laurenz Albe pointed out, I am able to create a function like this:
CREATE OR REPLACE FUNCTION public.calculate_similarity_percentage(text, text)
RETURNS numeric
LANGUAGE sql
STABLE
AS $function$
SELECT ROUND(
(select count(*) from (
SELECT jsonb_each(user_jsonb_column) FROM users WHERE username = $1
INTERSECT
SELECT jsonb_each(user_jsonb_column) FROM users WHERE username = $2
) as SAME_PAIRS
)::decimal / (
select count(*) from (
SELECT jsonb_object_keys(user_jsonb_column) FROM users WHERE username = $1
INTERSECT
SELECT jsonb_object_keys(user_jsonb_column) FROM users WHERE username = $2
) as SAME_KEYS
)
* 100) as similarity_percentage
$function$
Then I can execute this function:
SELECT calculate_similarity_percentage('johndoe','janedoe')
And it returns this without any problem:
similarity_percentage
62
However, I would like Hasura to track this function so that I can query it on graphQL as:
query MyQuery {
calculate_similarity_percentage(args: {user1: "johndoe", user2: "janedoe"}) {
similarity_percentage
}
}
But if I try to track the function above, Hasura says:
**SQL Execution Failed**
in function "calculate_similarity_percentage":
the function "calculate_similarity_percentage" cannot be tracked for the following reasons:
• the function does not return a "COMPOSITE" type
• the function does not return a SETOF
• the function does not return a SETOF table
I have no idea if I can find a workaround and return a numeric value as a "COMPOSITE" or SETOF table.
Here is how I kind of solved my case. But this was not the optimal solution so I'm not accepting this as an answer.
I ended up creating another table like this:
USER_RELATION_TABLE:
| user1_col | user2_col |
|--------------------------|
| 'johndoe' | 'janedoe' |
|--------------------------|
| 'brad' | 'angelina' |
|--------------------------|
| ... | ... |
Then I added a computed field on the relation table with the following function:
CREATE OR REPLACE FUNCTION public.calculate_similarity_percentage(user_relation_row user_relation_table)
RETURNS numeric
LANGUAGE sql
STABLE
AS $function$
SELECT ROUND(
(select count(*) from (
SELECT jsonb_each(user_jsonb_column) FROM users
WHERE username = user_relation_row.user1_col
INTERSECT
SELECT jsonb_each(user_jsonb_column) FROM users
WHERE username = user_relation_row.user2_col
) as SAME_PAIRS
)::decimal / (
select count(*) from (
SELECT jsonb_object_keys(user_jsonb_column) FROM users
WHERE username = user_relation_row.user1_col
INTERSECT
SELECT jsonb_object_keys(user_jsonb_column) FROM users
WHERE username = user_relation_row.user2_col
) as SAME_KEYS
)
* 100) as similarity_percentage
$function$
Now I can query it on the graphQL like this:
query MyQuery {
user_relation_table {
similarity
}
}
i have a starting table where there are some meteo data stored every 15 minutes, one field stores leaf wet at 1 minute sampling in a numeric array form, thus i have a 15 values array each row.
Now i want to create a 1 hour aggregation of this table, crating an array of 60 values for this field.
I tried array_cat at first place, but says
array_cat(numeric[]) not existing
the function obviuously exists, so i tought the format was not the one expected, i tried first unnesting and then aggregating, not working again.
Finally i was able to aggregate trough string conversion, but it's not what i wanted (i might in the future apply some numeric elaboration oh that 60-values array)
I paste the query for further investigations
SELECT dati1_v.id_stazione,
to_char(dati1_v.data_ora, 'YYYY-MM-DD HH24:00:00'::text) AS date_hour,
round(avg(dati1_v.temp1_media), 2) AS t_avg,
round(avg(dati1_v.ur1_media), 2) AS hum_avg,
sum(dati1_v.pioggia) AS rain_tot,
max(dati1_v.pioggia) AS rain_max,
round((avg((SELECT avg(lw.lw) AS avg FROM unnest(dati1_v.lw_top_array) lw(lw))) - lws.top_min) /
(lws.top_max - lws.top_min) * 100::numeric, 2) AS lw_top_avg,
array_agg((SELECT round((avg(lw.lw) - lws.top_min) / (lws.top_max - lws.top_min) * 100::numeric, 2) AS round
FROM unnest(dati1_v.lw_top_array) lw(lw))) AS lw_top_array,
array_cat(dati1_v.lw_top_array) AS lw_top_array_tot,
-- array_agg((select lw_top_array from unnest(dati1_v.lw_top_array))) AS lw_top_array_tot,
-- array_agg(array_to_string(dati1_v.lw_top_array, ',')) AS lw_top_array_tot,
round((avg((SELECT avg(lw.lw) AS avg FROM unnest(dati1_v.lw_bottom_array) lw(lw))) - lws.bottom_min) /
(lws.bottom_max - lws.bottom_min) * 100::numeric, 2) AS lw_bottom_avg,
array_agg((SELECT round((avg(lw.lw) - lws.bottom_min) / (lws.bottom_max - lws.bottom_min) * 100::numeric,
2) AS round
FROM unnest(dati1_v.lw_bottom_array) lw(lw))) AS lw_bottom_array
FROM dati1_v,
lw_settings lws
WHERE lws.id = 1
GROUP BY dati1_v.id_stazione, to_char(dati1_v.data_ora, 'YYYY-MM-DD HH24:00:00'::text), lws.top_min, lws.top_max,
lws.bottom_min, lws.bottom_max
ORDER BY dati1_v.id_stazione, to_char(dati1_v.data_ora, 'YYYY-MM-DD HH24:00:00'::text)
in particular, my tries were related to this specific block:
array_cat(dati1_v.lw_top_array) AS lw_top_array_tot,
-- array_agg((select lw_top_array from unnest(dati1_v.lw_top_array))) AS lw_top_array_tot,
-- array_agg(array_to_string(dati1_v.lw_top_array, ',')) AS lw_top_array_tot
Thanks
For me in similar case helped UNNEST in subquery and ARRAY_AGG of unnnested
SELECT
ARRAY_AGG(
DISTINCT lw_top
) as lw_top_array
FROM (
SELECT
UNNEST(lw_top_array) AS lw_top
FROM
dati1_v
) as tmp;
for me helped next query
SELECT
my_table.key,
array_agg(_unnested.item) as array_coll
from my_table
left join LATERAL (SELECT unnest(my_table.array_coll) as item) _unnested ON TRUE
GROUP by my_table.key
In PostgreSQL, the Group_concat function is not available but you can get similar result as string_agg and array_to_string.
string_agg(array_to_string(file_ids, ','), ',') filter ( where file_ids notnull ) AS file_ids_str
array_to_string and array_to_string works in next way
array_to_string([1, 2, 456], ',') => '1,2,456'
string_agg(['a', 'ab'], ',') => 'a,ab'
the only problem is that result is string with ',' as separator
I'm a really bad in sql , my query is
select * from car_wash where
(select ST_Within((select car_wash.lon_lat from car_wash),(select ST_Buffer(ST_GeomFromText('POINT(65.3 323.2)'),20)))) = true
AND car_wash.was_deleted=false;
But i know that it isn't correct because nested query can return more than 1 column, how to modify this query to use where clause
select *
from car_wash cw
where
ST_Within (
cw.lon_lat,
ST_Buffer(ST_GeomFromText('POINT(65.3 323.2)'),20)
)
AND
not car_wash.was_deleted
I don't use postgresql, but maybe something like this works:
select * from car_wash
where EXISTS (select ST_Within((select car_wash.lon_lat from car_wash),
(select ST_Buffer(ST_GeomFromText('POINT(65.3 323.2)'),20))) within
WHERE within = true)
AND car_wash.was_deleted=false;
If it doesn't work, I have a variant, so tell me when.
I have table called users in PostgreSQL.I want to get all users in table from my Laravel application.
I can get from table directly as:
$testData = DB::table('users')->get();
output is:
[{"id":1,"name":"Chris Sevilleja","username":"sevilayha","email":"chris#scotch.io","password":"$2y$08$i\/ATa68ierqRL47ZxHX4EesJGEcdtKPckZs8GDGpYS.IR4aaQn.\/q","created_at":"2016-09-07 09:32:41","updated_at":"2016-09-07 09:32:41"},{"id":2,"name":"Bemagoni chandrashekar","username":"chandrashekar","email":"chandrashekar#zessta.com","password":"$2y$08$QG9JsAerYp3UYpXNNyImhuz\/6hiWv8XpURpJX1uJ.hAm8l1RG2JrC","created_at":"2016-09-07 09:32:41","updated_at":"2016-09-07 09:32:41"},{"id":3,"name":"dwefr","username":"ewre","email":"ewrt#egyfrhgt.com","password":"$2y$08$s2XBZvoAvpEqjjhbx8QPw.eSe5TIuJC25XbaFkTskzAKAGi99QWga","created_at":"2016-09-07 09:34:27","updated_at":"2016-09-07 09:34:27"},{"id":4,"name":"r3t54y6u677i","username":"retrytyu","email":"etyru#dddj.com","password":"$2y$08$Xb0Xm4KwdSwcBSHX0F6ETOWU60X.NO5D7\/uwVv\/xUAXTUS8LSPCLu","created_at":"2016-09-07 09:46:26","updated_at":"2016-09-07 09:46:26"},{"id":5,"name":"r3t45y6u","username":"ertrh","email":"435t4y65#sss.com","password":"$2y$08$36DLu49nZ2YOWtU.c625meiyi3\/fmsHxiTuxIU9z9UcyrbIpFSKKW","created_at":"2016-09-07 10:02:05","updated_at":"2016-09-07 10:02:05"},{"id":6,"name":"ewrtryuyj","username":"ryt","email":"wrewtey#sssks.com","password":"$2y$08$GULHtX3GGXPGgm8gA9yDbeawZlQ5QwD2TX7nrCvEU4j7jrSgPWAQO","created_at":"2016-09-07 10:04:17","updated_at":"2016-09-07 10:04:17"},{"id":7,"name":"chandu","username":"chandu","email":"ch1235hdhd#dhdjd.com","password":"$2y$08$gpAhcl\/Sg.lGvb.zk.I\/m.PfcttGI6OPFMsMxQVm15dYOtQDvIWSG","created_at":"2016-09-07 10:06:18","updated_at":"2016-09-07 10:06:18"},{"id":8,"name":"dewfergt","username":"erwrgf","email":"fgf#dgrf.com","password":"$2y$08$ikYAXV1prZsEj2MPxXM4S.Tqn160Jv25cFOQLghK8ptFiSBaIFGZO","created_at":"2016-09-07 10:07:20","updated_at":"2016-09-07 10:07:20"},{"id":9,"name":"rteryt","username":"wrwter","email":"wretr#gjjd.com","password":"$2y$08$pkOUBl1NlNdBShNWiklya.0zlPzPrEH2edCfvdCiHLnj80GY1sdtm","created_at":"2016-09-08 07:11:46","updated_at":"2016-09-08 07:11:46"},{"id":10,"name":"Raghu","username":"raghu","email":"raghu#gdgdjd.com","password":"$2y$08$m9wke.vvTTZytYw91I4\/q.qxKoCobLOW7dbCvs66xFyJyy2R9phni","created_at":"2016-09-10 10:06:40","updated_at":"2016-09-10 10:06:40"},{"id":11,"name":"wewert","username":"ewretr","email":"ewrtey#vsss.com","password":"$2y$08$IXD0eXYTPPGE1MfALonEFey0lr\/KBMZ0.3AIO3sWVgu7IZdWhXwTG","created_at":"2016-09-12 07:48:58","updated_at":"2016-09-12 07:48:58"}]
Through PostgreSQL function calling:
my PostgreSQL function:
-- Function: public."RegiterUsers2"()
-- DROP FUNCTION public."RegiterUsers2"();
CREATE OR REPLACE FUNCTION public."RegiterUsers2"()
RETURNS SETOF users AS
'select * from users'
LANGUAGE sql VOLATILE
COST 100
ROWS 1000;
ALTER FUNCTION public."RegiterUsers2"()
OWNER TO postgres;
Laravel code:
$allUserData=DB::select('SELECT public."RegiterUsers2"()')
output:
[{"RegiterUsers2":"(1,\"Chris Sevilleja\",sevilayha,chris#scotch.io,$2y$08$i\/ATa68ierqRL47ZxHX4EesJGEcdtKPckZs8GDGpYS.IR4aaQn.\/q,\"2016-09-07 09:32:41\",\"2016-09-07 09:32:41\")"},{"RegiterUsers2":"(2,\"Bemagoni chandrashekar\",chandrashekar,chandrashekar#zessta.com,$2y$08$QG9JsAerYp3UYpXNNyImhuz\/6hiWv8XpURpJX1uJ.hAm8l1RG2JrC,\"2016-09-07 09:32:41\",\"2016-09-07 09:32:41\")"},{"RegiterUsers2":"(3,dwefr,ewre,ewrt#egyfrhgt.com,$2y$08$s2XBZvoAvpEqjjhbx8QPw.eSe5TIuJC25XbaFkTskzAKAGi99QWga,\"2016-09-07 09:34:27\",\"2016-09-07 09:34:27\")"},{"RegiterUsers2":"(4,r3t54y6u677i,retrytyu,etyru#dddj.com,$2y$08$Xb0Xm4KwdSwcBSHX0F6ETOWU60X.NO5D7\/uwVv\/xUAXTUS8LSPCLu,\"2016-09-07 09:46:26\",\"2016-09-07 09:46:26\")"},{"RegiterUsers2":"(5,r3t45y6u,ertrh,435t4y65#sss.com,$2y$08$36DLu49nZ2YOWtU.c625meiyi3\/fmsHxiTuxIU9z9UcyrbIpFSKKW,\"2016-09-07 10:02:05\",\"2016-09-07 10:02:05\")"},{"RegiterUsers2":"(6,ewrtryuyj,ryt,wrewtey#sssks.com,$2y$08$GULHtX3GGXPGgm8gA9yDbeawZlQ5QwD2TX7nrCvEU4j7jrSgPWAQO,\"2016-09-07 10:04:17\",\"2016-09-07 10:04:17\")"},{"RegiterUsers2":"(7,chandu,chandu,ch1235hdhd#dhdjd.com,$2y$08$gpAhcl\/Sg.lGvb.zk.I\/m.PfcttGI6OPFMsMxQVm15dYOtQDvIWSG,\"2016-09-07 10:06:18\",\"2016-09-07 10:06:18\")"},{"RegiterUsers2":"(8,dewfergt,erwrgf,fgf#dgrf.com,$2y$08$ikYAXV1prZsEj2MPxXM4S.Tqn160Jv25cFOQLghK8ptFiSBaIFGZO,\"2016-09-07 10:07:20\",\"2016-09-07 10:07:20\")"},{"RegiterUsers2":"(9,rteryt,wrwter,wretr#gjjd.com,$2y$08$pkOUBl1NlNdBShNWiklya.0zlPzPrEH2edCfvdCiHLnj80GY1sdtm,\"2016-09-08 07:11:46\",\"2016-09-08 07:11:46\")"},{"RegiterUsers2":"(10,Raghu,raghu,raghu#gdgdjd.com,$2y$08$m9wke.vvTTZytYw91I4\/q.qxKoCobLOW7dbCvs66xFyJyy2R9phni,\"2016-09-10 10:06:40\",\"2016-09-10 10:06:40\")"},{"RegiterUsers2":"(11,wewert,ewretr,ewrtey#vsss.com,$2y$08$IXD0eXYTPPGE1MfALonEFey0lr\/KBMZ0.3AIO3sWVgu7IZdWhXwTG,\"2016-09-12 07:48:58\",\"2016-09-12 07:48:58\")"}]
I need like this
{"id":1,
"name":"Chris Sevilleja",
"username":"sevilayha",
"email":"chris#scotch.io",
"password":"$2y$08$i\/ATa68ierqRL47ZxHX4EesJGEcdtKPckZs8GDGpYS.IR4aaQn",
"created_at":"2016-09-07 09:32:41",
"updated_at":"2016-09-07 09:32:41"},
so what is wrong with postgres function and how can I get like above one.
When you call a function returning many columns and you want each one, you must call in the form of:
SELECT col1, col2, ... FROM function(...)
Or:
SELECT * FROM function(...)
So in your case you simple want:
$allUserData=DB::select('SELECT * FROM public."RegiterUsers2"()')
I have the following heap of text:
"BundleSize,155648,DynamicSize,204800,Identifier,com.URLConnectionSample,Name,
URLConnectionSample,ShortVersion,1.0,Version,1.0,BundleSize,155648,DynamicSize,
16384,Identifier,com.IdentifierForVendor3,Name,IdentifierForVendor3,ShortVersion,
1.0,Version,1.0,".
What I'd like to do is extract data from this in the following manner:
BundleSize:155648
DynamicSize:204800
Identifier:com.URLConnectionSample
Name:URLConnectionSample
ShortVersion:1.0
Version:1.0
BundleSize:155648
DynamicSize:16384
Identifier:com.IdentifierForVendor3
Name:IdentifierForVendor3
ShortVersion:1.0
Version:1.0
All tips and suggestions are welcome.
It isn't quite clear what do you need to do with this data. If you really need to process it entirely in the database (looks like the task for your favorite scripting language instead), one option is to use hstore.
Converting records one by one is easy:
Assuming
%s =
BundleSize,155648,DynamicSize,204800,Identifier,com.URLConnectionSample,Name,URLConnectionSample,ShortVersion,1.0,Version,1.0
SELECT * FROM each(hstore(string_to_array(%s, ',')));
Output:
key | value
--------------+-------------------------
Name | URLConnectionSample
Version | 1.0
BundleSize | 155648
Identifier | com.URLConnectionSample
DynamicSize | 204800
ShortVersion | 1.0
If you have table with columns exactly matching field names (note the quotes, populate_record is case-sensitive to key names):
CREATE TABLE data (
"BundleSize" integer, "DynamicSize" integer, "Identifier" text,
"Name" text, "ShortVersion" text, "Version" text);
You can insert hstore records into it like this:
INSERT INTO data SELECT * FROM
populate_record(NULL::data, hstore(string_to_array(%s, ',')));
Things get more complicated if you have comma-separated values for more than one record.
%s = BundleSize,155648,DynamicSize,204800,Identifier,com.URLConnectionSample,Name,URLConnectionSample,ShortVersion,1.0,Version,1.0,BundleSize,155648,DynamicSize,16384,Identifier,com.IdentifierForVendor3,Name,IdentifierForVendor3,ShortVersion,1.0,Version,1.0,
You need to break up an array into chunks of number_of_fields * 2 = 12 elements first.
SELECT hstore(row) FROM (
SELECT array_agg(str) AS row FROM (
SELECT str, row_number() OVER () AS i FROM
unnest(string_to_array(%s, ',')) AS str
) AS str_sub
GROUP BY (i - 1) / 12) AS row_sub
WHERE array_length(row, 1) = 12;
Output:
"Name"=>"URLConnectionSample", "Version"=>"1.0", "BundleSize"=>"155648", "Identifier"=>"com.URLConnectionSample", "DynamicSize"=>"204800", "ShortVersion"=>"1.0"
"Name"=>"IdentifierForVendor3", "Version"=>"1.0", "BundleSize"=>"155648", "Identifier"=>"com.IdentifierForVendor3", "DynamicSize"=>"16384", "ShortVersion"=>"1.0"
And inserting this into the aforementioned table:
INSERT INTO data SELECT (populate_record(NULL::data, hstore(row))).* FROM ...
the rest of the query is the same.