I am using a LEFT JOIN there will be cases where there is no right-table match therefore empty (null) values are substituted for the right-table columns. As a result I am getting [null] as one of the JSON aggregates.
SELECT C.id, C.name, json_agg(E) AS emails FROM contacts C
LEFT JOIN emails E ON C.id = E.user_id
GROUP BY C.id;
Postgres 9.3 creates output for example
id | name | emails
-----------------------------------------------------------
1 | Ryan | [{"id":3,"user_id":1,"email":"hello#world.com"},{"id":4,"user_id":1,"email":"again#awesome.com"}]
2 | Nick | [null]
How can I ignore/remove null so I have an empty JSON array [] when the right-table column is null?
In 9.4 you can use coalesce and an aggregate filter expression.
SELECT C.id, C.name,
COALESCE(json_agg(E) FILTER (WHERE E.user_id IS NOT NULL), '[]') AS emails
FROM contacts C
LEFT JOIN emails E ON C.id = E.user_id
GROUP BY C.id, C.name
ORDER BY C.id;
The filter expression prevents the aggregate from processing the rows that are null because the left join condition is not met, so you end up with a database null instead of the json [null]. Once you have a database null, then you can use coalesce as usual.
http://www.postgresql.org/docs/9.4/static/sql-expressions.html#SYNTAX-AGGREGATES
something like this, may be?
select
c.id, c.name,
case when count(e) = 0 then '[]' else json_agg(e) end as emails
from contacts as c
left outer join emails as e on c.id = e.user_id
group by c.id
sql fiddle demo
you also can group before join (I'd prefer this version, it's a bit more clear):
select
c.id, c.name,
coalesce(e.emails, '[]') as emails
from contacts as c
left outer join (
select e.user_id, json_agg(e) as emails from emails as e group by e.user_id
) as e on e.user_id = c.id
sql fiddle demo
If this is actually a PostgreSQL bug, I hope it's been fixed in 9.4. Very annoying.
SELECT C.id, C.name,
COALESCE(NULLIF(json_agg(E)::TEXT, '[null]'), '[]')::JSON AS emails
FROM contacts C
LEFT JOIN emails E ON C.id = E.user_id
GROUP BY C.id;
I personally don't do the COALESCE bit, just return the NULL. Your call.
I used this answer (sorry, I can't seem to link to your username) but I believe I improved it a bit.
For the array version we can
get rid of the redundant double select
use json_agg instead of the array_to_json(array_agg()) calls
and get this:
CREATE OR REPLACE FUNCTION public.json_clean_array(p_data JSON)
RETURNS JSON
LANGUAGE SQL IMMUTABLE
AS $$
-- removes elements that are json null (not sql-null) or empty
SELECT json_agg(value)
FROM json_array_elements(p_data)
WHERE value::text <> 'null' AND value::text <> '""';
$$;
For 9.3, for the object version, we can:
get rid of the non-used WITH clause
get rid of the redundant double select
and get this:
CREATE OR REPLACE FUNCTION public.json_clean(p_data JSON)
RETURNS JSON
LANGUAGE SQL IMMUTABLE
AS $$
-- removes elements that are json null (not sql-null) or empty
SELECT ('{' || string_agg(to_json(key) || ':' || value, ',') || '}') :: JSON
FROM json_each(p_data)
WHERE value::TEXT <> 'null' AND value::TEXT <> '""';
$$;
For 9.4, we don't have to use the string assembly stuff to build the object, as we can use the newly added json_object_agg
CREATE OR REPLACE FUNCTION public.json_clean(p_data JSON)
RETURNS JSON
LANGUAGE SQL IMMUTABLE
AS $$
-- removes elements that are json null (not sql-null) or empty
SELECT json_object_agg(key, value)
FROM json_each(p_data)
WHERE value::TEXT <> 'null' AND value::TEXT <> '""';
$$;
Probably less performant than Roman Pekar's solution, but a bit neater:
select
c.id, c.name,
array_to_json(array(select email from emails e where e.user_id=c.id))
from contacts c
A bit different but might be helpful for others:
If all objects in the array are of same structure (e.g. because you use jsonb_build_object to create them) you can define a "NULL object with the same structure" to use in array_remove:
...
array_remove(
array_agg(jsonb_build_object('att1', column1, 'att2', column2)),
to_jsonb('{"att1":null, "att2":null}'::json)
)
...
I made my own function for filtering json arrays:
CREATE OR REPLACE FUNCTION public.json_clean_array(data JSON)
RETURNS JSON
LANGUAGE SQL
AS $$
SELECT
array_to_json(array_agg(value)) :: JSON
FROM (
SELECT
value
FROM json_array_elements(data)
WHERE cast(value AS TEXT) != 'null' AND cast(value AS TEXT) != ''
) t;
$$;
I use it as
select
friend_id as friend,
json_clean_array(array_to_json(array_agg(comment))) as comments
from some_entity_that_might_have_comments
group by friend_id;
of course only works in postgresql 9.3. I also have a similar one for object fields:
CREATE OR REPLACE FUNCTION public.json_clean(data JSON)
RETURNS JSON
LANGUAGE SQL
AS $$
SELECT
('{' || string_agg(to_json(key) || ':' || value, ',') || '}') :: JSON
FROM (
WITH to_clean AS (
SELECT
*
FROM json_each(data)
)
SELECT
*
FROM json_each(data)
WHERE cast(value AS TEXT) != 'null' AND cast(value AS TEXT) != ''
) t;
$$;
EDIT: You can see a few utils (a few are not originally mine but they were take from other stackoverflow solutions) here at my gist:
https://gist.github.com/le-doude/8b0e89d71a32efd21283
This way works, but there's gotta be a better way :(
SELECT C.id, C.name,
case when exists (select true from emails where user_id=C.id) then json_agg(E) else '[]' end
FROM contacts C
LEFT JOIN emails E ON C.id = E.user_id
GROUP BY C.id, C.name;
demo: http://sqlfiddle.com/#!15/ddefb/16
At the time this question was asked, the following example might not have been as efficient of a choice, due to the nature of how the email_list would basically not limit itself based on the outer query, but newer versions of postgres handle this much better (also, I'd recommend jsonb over json)
WITH email_list (user_id, emails) as (
SELECT user_id, json_agg(emails) FROM emails GROUP BY user_id
)
SELECT C.id, C.name, COALESCE(E.emails, '[]'::json) as emails
FROM contacts C LEFT JOIN email_list E ON C.id = E.user_id;
The COALESCE is only needed if you actually do want to have an empty array, otherwise the entire value would be null, which can be preferable output in some languages.
Related
Recently started working on Postgres and need to pivot data.
I wrote the following query:
select *
from crosstab (
$$
with tmp_kv as (
select distinct pat_id
,col.name as key, replace(replace(replace(value, '[',''), ']', ''),'"','') as value
from (
select p.Id as pat_id, nullif(kv.key,'undefined')::int as key, trim(kv.value::text,'"') as value
from pat_table p
left join e_table e on e.pat_id = p.id and e.id is null
,jsonb_each_text(p.data) as kv
) t
left join lateral (
select name::text as name from public.config_fields fld
where id = t.key
) col on true
)
select pat_id, key, value
from tmp_kv
where nullif(trim(key),'') is not null
order by pat_id, key
$$,$$
select distinct key from tmp_kv -- (Get error "relation "tmp_kv" does not exist" )
where nullif(trim(key),'') is not null
order by 1
$$
) as (
pat_id bigint
...
...
);
Query works if I take the WITH clause out into temporary table. But will be deploying it to production with read replicas, so need it to be working with a CTE. Is there a way?
The two queries passed as strings to the crosstab() function are separate queries.
A CTE can only be attached to a single query.
What you ask for is strictly impossible.
Since you have to spell out the (static) return type for crosstab() anyway, and the result of the query in the 2nd parameter has to match that, it's pointless to use a query with a dynamic result as 2nd parameter to begin with.
I have a select query return and it shows the result like below:
select * from table gives the result like below
I have parameter called Apple If I pass the parameter somewhere in query I should get the result like below
How to get this in postgresql. If anyone knows please share the answer below.
I would do this with a helper function for clarity. And it might be reusable.
create or replace function filter_jsonb_array(arr jsonb, fruit text)
returns jsonb language sql immutable as
$$
select coalesce
(
(select jsonb_agg(j) from jsonb_array_elements(arr) j where j ->> 'fruit' = fruit),
'[]'::jsonb
);
$$;
and then
select "Column_A", "Column_B", filter_jsonb_array("Column_JSONARRAY", 'Apple') from table_;
If you do not want a function then the function body can be placed directly into the select query.
select
"Column_A",
"Column_B",
coalesce
(
(select jsonb_agg(j) from jsonb_array_elements("Column_JSONARRAY") j where j ->> 'fruit' = 'Apple'),
'[]'::jsonb
) "Column_JSONARRAY"
from table_;
Considering your datatype of column Column_JSONARRAY is JSONB, try This:
with cte as (
SELECT column_a, column_b, (column_jsonarray ->> ( index_-1 )::int)::jsonb AS "column_jsonarray"
FROM table_
CROSS JOIN jsonb_array_elements(column_jsonarray)
WITH ORDINALITY arr(array_,index_)
WHERE array_->>'fruit' in ('Apple')
)
select t1.column_a, t1.column_b, jsonb_agg(t2.column_jsonarray)
from table_ t1
left join cte t2 on t1.column_a =t2.column_a and t1.column_b =t2.column_b
group by t1.column_a, t1.column_b
I have a view from a query select * from table which returns the below data
I want to group by the name column which have same name and merge the JSONArray column like mentioned below
One way to do this, is to unnest the arrays and then aggregate them back:
select t.id, t.name, jsonb_agg(a.e)
from the_table t
cross join lateral jsonb_array_elements(t.json_array) as a(e)
group by t.id, t.name;
If you do that a lot, a custom aggregate makes this a bit easier to user (but probably not faster)
create function jsonb_array_combine(p_one jsonb, p_two jsonb)
returns jsonb
as
$$
select jsonb_agg(e)
from (
select e
from jsonb_array_elements(p_one) as o(e)
union all
select e
from jsonb_array_elements(p_two) as t(e)
) t
$$
language sql
immutable;
create aggregate jsonb_array_agg(jsonb)
(
SFUNC = jsonb_array_combine(jsonb, jsonb),
STYPE = jsonb
);
Then you can use it like this:
select t.id, t.name, jsonb_array_agg(t.json_array)
from the_table t
group by t.id, t.name;
I have three tables:
create table id_table (
id integer
);
insert into id_table values (1),(2),(3);
create table alphabet_table (
id integer,
letter text
);
insert into alphabet_table values (1,'a'),(2,'b'),(3,'c');
create table greek_table (
id integer,
letter text
);
insert into greek_table values (1,'alpha'),(2,'beta');
I like to create a function that join id_table with either alphabet_table or greek_table on id. The choice of the table depends on an input value specified in the function. I wrote:
CREATE OR REPLACE FUNCTION choose_letters(letter_type text)
RETURNS table (id integer,letter text) AS $$
BEGIN
RETURN QUERY
select t1.id,
case when letter_type = 'alphabet' then t2.letter else t3.letter end as letter
from id_table t1,
alphabet_table t2 ,
greek_table t3
where t1.id = t2.id and t1.id = t3.id;
END;
$$LANGUAGE plpgsql;
I ran select choose_letter('alphabet'). The problem with this code is that when id_table joins with alphabet_table, it does not pick up id, No 3. It seems that inner joins are done for both alphabet_table and greek_table (so it only picks up the common ids, 1 and 2). To avoid this problem, I wrote:
CREATE OR REPLACE FUNCTION choose_letters(letter_type text)
RETURNS table (id integer, letter text) AS $$
BEGIN
RETURN QUERY
select t1.id,
case when letter_type = 'alphabet' then t2.letter else t3.letter end as letter
from id_table t1
left join alphabet_table t2 on t1.id=t2.id
left join greek_table t3 on t1.id=t3.id
where t2.letter is not null or t3.letter is not null;
END;
$$LANGUAGE plpgsql;
Now it pick up all the 3 ids when id_table and alphabet_table join. However, When I ran select choose_letter('greek'). The id no. 3 appears with null value in letter column despite the fact that I specified t3.letter is not null in where clause.
What I'm looking for is that when I ran select choose_letters('alphabet'), the output needs to be (1,'a'), (2,'b'),(3,'c'). select choose_letters('greek') should produce (1,'alpha'),(2,'beta). No missing values nor null. How can I accomplish this?
Learn to use proper, explicit JOIN syntax. Simple rule: Never use commas in the FROM clause.
You can do what you want with LEFT JOIN and some other logic:
select i.id,
coalesce(a.letter, g.letter) as letter
from id_table i left join
alphabet_table a
on i.id = a.id and letter_type = 'alphabet' left join
greek_table g
on i.id = g.id and letter_type <> 'alphabet'
where a.id is not null or g.id is not null;
The condition using letter_type needs to be in the on clauses. Otherwise, alphabet_table will always have a match.
Gordon Linoff's answer above is certainly correct, but here is an alternative way to write the code.
It may or may not be better from a performance perspective, but it is logically equivalent. If performance is a concern you'd need to run EXPLAIN ANALYZE on the query an inspect the plan, and do other profiling.
Some good parts about this are the inner join makes the join clause and where clause simpler and easier to reason about. It's also more straight forward for the execution engine to parallelize the query.
On the downside it looks like code duplication, however, the DRY principle is often misapplied to SQL. Repeating code is less important than repeating data reads. What you're aiming to do is not scan the same data multiple times.
If there is no index on the fields you are joining or the letter_type then this could end up doing a full table scan twice, and be worse. If you do have the indexes then it can do it with index range scans nicely.
SELECT
i.id,
a.letter
FROM id_table i
INNER JOIN alphabet_table a
ON i.id = a.id
WHERE letter_type = 'alphabet'
UNION ALL
SELECT
i.id,
g.letter
FROM id_table i
INNER JOIN greek_table g
ON i.id = g.id
WHERE letter_type <> 'alphabet'
The first problem is your tables or not structured properly, You would have created single table like char_table (id int, letter text, type text) type will specify whether it is alphabet or Greek.
Another solution is you can write two SQL queries one in if condition other one is in else part
My requirement is i have to compare data between two different tables with same schema in same database ,
For the moment I am making comparison in different fields in same table and if some validation fails error will be stored in one table like :
IF (NEW.vision IS NULL and new.vispres IS NOT NULL)
THEN INSERT INTO exception_detail( noces,exception) VALUES
(new.no,'please check the values if vision is null then vispres should also be null');
END IF;
The same kind of comparison i want to do with two tables for same element (no) eg
IF (TABLE1.NEW.vispres IS NULL and TABLE2.new.vispres IS NOT NULL)
THEN INSERT INTO exception_detail( noces,exception) VALUES
(new.no,'please check the values if vispres is null for number 5 in table 1 then vispres should also be null for number 5 in Table 2 ');
END IF;
Please help
Thank you in advance
Can I do something like :
SELECT q1.* FROM TABLE1 q1
INNER JOIN TABLE2 q2 ON (q1.noces = q2.noces);
I think it will give all the records from both tables where noces is same
In continuation now i want to compare each row of the output, and if data is not same it must throw exception, IS there a possibility like :
foreach row of above output{
if (q1.name != q2.name)
Do something ;
if (q2.address < q1.address)
Do something ;
}
but it all must be in one query or trigger
I would look at essentially three cases.
ID's the same, but data differs
ID in table_a not found in table_b
ID in table_b not found in table_a
SQL might look like this:
SELECT (r).* FROM ( -- Wrapper query
SELECT a as r
FROM table_a a
JOIN table_b b ON a.id = b.id AND a <> b
UNION ALL
SELECT a as r
FROM table_a a
LEFT JOIN table_b b ON a.id = b.id
WHERE b.id IS NULL
UNION ALL
SELECT b AS r
FROM table_b b
LEFT JOIN table_a a ON a.id = b.id
WHERE a.id IS NULL
) t;
It might be possible to fold this into a case and a full outer join but this should give you the basic idea.