group by error with postgres and pomm orm - postgresql

I want to execute the following SQL query :
SELECT date, COUNT(id_customers)
FROM event
WHERE event_id = 3
GROUP BY date
When I try this query in my database, it works perfectly. But in my code I get an error which I can't resolve.
I use symfony2 with the orm pomm. It's Postgresql.
Here is my code :
$sql = "SELECT e.date, COUNT(id_customers) FROM event e WHERE event_id = $* GROUP BY e.date";
return $this->query($sql, [$eventId])->extract();
Here is the error :
request.CRITICAL: Uncaught PHP Exception InvalidArgumentException:
"No such field 'id'. Existing fields are {date, count}"
at /home/vagrant/sourcefiles/vendor/pomm-project/model-manager/sources/lib/Model/FlexibleEntity/FlexibleContainer.php line 64
{"exception":" [object] (InvalidArgumentException(code: 0): No such field 'id'.
Existing fields are {date, count}
at /home/vagrant/sourcefiles/vendor/pomm-project/model-manager/sources/lib/Model/FlexibleEntity/FlexibleContainer.php:64)"} []
So I tried to had the id in my select, by I get this error :
request.CRITICAL: Uncaught PHP Exception
PommProject\Foundation\Exception\SqlException: "
SQL error state '42803' [ERROR] ==== ERROR: column "e.id" must appear in the GROUP BY clause or be used in an aggregate function LINE 1: SELECT e.id, e.date, COUNT(id_customers) FROM event e WHERE ... ^
==== «PREPARE === SELECT e.id, e.date, COUNT(id_customers) FROM event e WHERE event_id = $1 GROUP BY e.date ===»." at /home/vagrant/sourcefiles/vendor/pomm-project/foundation/sources/lib/Session/Connection.php line 327 {"exception":"[object] (PommProject\Foundation\Exception\SqlException(code: 0): \nSQL error state '42803' [ERROR]\n====\nERROR: column \"e.id\" must appear in the GROUP BY clause or be used in an aggregate function\nLINE 1: SELECT e.id, e.date, COUNT(id_customers) FROM event e WHERE ...\n ^\n\n====\n«PREPARE ===\nSELECT e.id, e.date, COUNT(id_customers) FROM event e WHERE event_id = $1 GROUP BY e.date\n ===». at /home/vagrant/sourcefiles/vendor/pomm-project/foundation/sources/lib/Session/Connection.php:327)"} []
The only thing that works is when I had the id in the group by, but this is not the result I want.
Someone can explain me why this is working in the database and not in the php ?

this is because you are fetching flexible entities without their primary key. There is an identity mapper behind the scene that ensure fetching twice the same entity will return the same instance.
In this case, you do not need to fetch entities (hence the extract after the query). So you can just use the QueryManager pooler to return converted arrays like the following:
$sql = "SELECT e.date, COUNT(id_customers) FROM event e WHERE event_id = $* GROUP BY e.date";
// Return an iterator that fetches converted arrays on demand:
return $this
->getSession()
->getQueryManager()
->query($sql, [$eventId])
;

i think its because the alias,
try this
$sql = "SELECT e.date, COUNT(e.id_customers) FROM event e WHERE event_id = $* GROUP BY e.date";
return $this->query($sql, [$eventId])->extract();

This is exactly how GROUP BY works in PostgreSQL:
When GROUP BY is present, it is not valid for the SELECT list
expressions to refer to ungrouped columns except within aggregate
functions, since there would be more than one possible value to return
for an ungrouped column.
It means that each field in your query either must be present in GROUP BY statement or handled by any of the aggregation functions. This is one of the differences between GROUP BY in MySQL and PostreSQL.
In other words you can add id at GROUP BY statement and do not worry about it ;)

Related

Why am I getting error 17410 end of data on my rest api when I add a parameter to the query

I have ORDS 21 installed in a local 19c oracle database. I have created a stored procedure to list all the departments from the dept table with a cursor that lists the employees from the emp table. If I just list all departments and their employees the api works fine, but if I add a parameter to the query to specify which department, I get an error 17410 no data left .
Both queries are backed by plsql stored procedures. I have created many stored procedures using this same format with parameters and nested cursors before without a problem.
MWE for query that works:
create or replace procedure get_dept
as
l_cur sys_refcursor;
begin
open l_cur for
select d.deptno,d.dname,
cursor (select e.empno,e.ename
from emp e
where e.deptno = d.deptno
order by e.deptno,e.empno
) as employees
from dept d
order by d.deptno;
-- return the resultset in json format
apex_json.open_object;
apex_json.write('emps',l_cur);
apex_json.close_object;
end get_dept;
/
MWE for query that does not work:
create or replace procedure get_dept1
(
p_dept_no in varchar2
) as
l_cur sys_refcursor;
begin
open l_cur for
select d.deptno,d.dname,
cursor (select e.empno,e.ename
from emp e
where e.deptno = d.deptno
order by e.deptno,e.empno
) as employees
from dept d
where d.deptno = to_number(p_dept_no)
order by d.deptno;
-- return the resultset in json format
apex_json.open_object;
apex_json.write('emps',l_cur);
apex_json.close_object;
end get_dept1;
/

SQL Error: ERROR: not all tokens processed

Am getting below error in Postgres while executing insert and delete queries. I have around 50 inserts and 50 delete statements. When executed an getting the error as,
SQL Error: ERROR: not all tokens processed
The error is not consistent all the time,
For example,
My 20th delete statement is getting failed
Next time when the same queries are executed, 25th delete statement is getting failed
And when those statements are executed alone, there is no failure.
Not sure if it is a database load issue or infrastructure related issue.
Any suggestion would be helpful
Below is the query,
WITH del_table_1 AS
(
delete from table_1 where to_date('01-'||col1,'DD-mm-YYYY') < current_date-1
RETURNING *
)
update control_table set deleted_count = cnt, status = 'Completed',
update_user_id = 'User', update_datetime = current_date from
(select 'Table1' as table_name, count(*) as cnt from del_table_1) aa
where
control_table.table_name = aa.table_name
and control_table.table_name = 'Table1'
and control_table.status = 'Pending';

PostgreSql Group By and aggreate function error

My problem is, when I run the following query in MySQL, it looks like this
Query;
SELECT
CONCAT(b.tarih, '#', CONCAT(b.enlem, ',', b.boylam), '#', b.aldigi_yol) AS IlkMesaiEnlemBoylamImei,
CONCAT(tson.max_tarih, '#', CONCAT(tson.max_enlem, ',', tson.max_boylam), '#', tson.max_aldigi_yol) AS SonMesaiEnlemBoylamImei,
Max(CAST(b.hiz AS UNSIGNED)) As EnYuksekHiz,
TIME_FORMAT(Sec_TO_TIME(TIMESTAMPDIFF(SECOND, (b.tarih), (tson.max_tarih))), '%H:%i') AS DurmaSuresi
FROM
(Select id as max_id, tarih as max_tarih, enlem as max_enlem, boylam as max_boylam, aldigi_yol as max_aldigi_yol from _213gl2015016424 where id in(
SELECT MAX(id)
FROM _213gl2015016424 where (tarih between DATE('2016-11-30 05:45:00') AND Date('2017-01-13 14:19:06')) AND CAST(hiz AS UNSIGNED) > 0
GROUP BY DATE(tarih))
) tson
LEFT JOIN _213gl2015016424 a ON a.id = tson.max_id
LEFT JOIN _213gl2015016424 b ON DATE(b.tarih) = DATE(a.tarih)
WHERE b.tarih is not null And (b.tarih between DATE('2016-11-30 05:45:00') AND Date('2017-01-13 14:19:06')) AND b.hiz > 0
GROUP BY tson.max_tarih
Output is order by date;
Result query
When I try to run a query in PostgreSQL, I get group by mistake.
Query;
SELECT
CONCAT(b.tarih, '#', CONCAT(b.enlem, ',', b.boylam), '#', b.toplamyol) AS IlkMesaiEnlemBoylamImei,
CONCAT(tson.max_tarih, '#', CONCAT(tson.max_enlem, ',', tson.max_boylam), '#', tson.max_toplamyol) AS SonMesaiEnlemBoylamImei,
Max(CAST(b.hiz AS OID)) As EnYuksekHiz,
to_char(to_timestamp((extract(epoch from (tson.max_tarih)) - extract(epoch from (b.tarih)))) - interval '2 hour','HH24:MI') AS DurmaSuresi
FROM
(Select id as max_id, tarih as max_tarih, enlem as max_enlem, boylam as max_boylam, toplamyol as max_toplamyol from _213GL2016008691 where id in(
SELECT MAX(id)
FROM _213GL2016008691 where (tarih between DATE('2018-02-01 03:31:54') AND DATE('2018-03-01 03:31:54')) AND CAST(hiz AS OID) > 0
GROUP BY DATE(tarih))
) tson
LEFT JOIN _213GL2016008691 a ON a.id = tson.max_id
LEFT JOIN _213GL2016008691 b ON DATE(b.tarih) = DATE(a.tarih)
WHERE b.tarih is not null And (b.tarih between DATE('2018-02-12 03:31:54') AND DATE('2018-02-13 03:31:54')) AND b.hiz > 0
GROUP BY tson.max_tarih
Group by error is : To use the aggregate function, you must add the column "b.tarih" to the GROUP BY list.
When I add it I get the same error for another column.I'm waiting for your help.
You are using a feature of MySQL that is not standard SQL and you can also deactivate.
You are grouping by tson.max_tarih in your query. That means that for all rows that share the same value in that field, you will get only one row as a result of that group.
If you have several different values in the rest of the fields (enlem, boylam, etc...) which one are you trying to get in as the result of the query? That's the question that PostgreSQL is asking you.
MySQL is just returning any value for those fields among the rows in the group. PostgreSQL requires you to actually specify it.
Two typical solutions would be grouping by the rest of the fields (b.tarih, b.enlem) or specifying the value those fields to something like MAX(b.tarih), etc.

Joining with set-returning function (SRF) and access columns in SQLAlchemy

Suppose I have an activity table and a subscription table. Each activity has an array of generic references to some other object, and each subscription has a single generic reference to some other object in the same set.
CREATE TABLE activity (
id serial primary key,
ob_refs UUID[] not null
);
CREATE TABLE subscription (
id UUID primary key,
ob_ref UUID,
subscribed boolean not null
);
I want to join with the set-returning function unnest so I can find the "deepest" matching subscription, something like this:
SELECT id
FROM (
SELECT DISTINCT ON (activity.id)
activity.id,
x.ob_ref, x.ob_depth,
subscription.subscribed IS NULL OR subscription.subscribed = TRUE
AS subscribed,
FROM activity
LEFT JOIN subscription
ON activity.ob_refs #> array[subscription.ob_ref]
LEFT JOIN unnest(activity.ob_refs)
WITH ORDINALITY AS x(ob_ref, ob_depth)
ON subscription.ob_ref = x.ob_ref
ORDER BY x.ob_depth DESC
) sub
WHERE subscribed = TRUE;
But I can't figure out how to do that second join and get access to the columns. I've tried creating a FromClause like this:
act_ref_t = (sa.select(
[sa.column('unnest', UUID).label('ob_ref'),
sa.column('ordinality', sa.Integer).label('ob_depth')],
from_obj=sa.func.unnest(Activity.ob_refs))
.suffix_with('WITH ORDINALITY')
.alias('act_ref_t'))
...
query = (query
.outerjoin(
act_ref_t,
Subscription.ob_ref == act_ref_t.c.ob_ref))
.order_by(activity.id, act_ref_t.ob_depth)
But that results in this SQL with another subquery:
LEFT OUTER JOIN (
SELECT unnest AS ob_ref, ordinality AS ref_i
FROM unnest(activity.ob_refs) WITH ORDINALITY
) AS act_ref_t
ON subscription.ob_refs #> ARRAY[act_ref_t.ob_ref]
... which fails because of the missing and unsupported LATERAL keyword:
There is an entry for table "activity", but it cannot be referenced from this part of the query.
So, how can I create a JOIN clause for this SRF without using a subquery? Or is there something else I'm missing?
Edit 1 Using sa.text with TextClause.columns instead of sa.select gets me a lot closer:
act_ref_t = (sa.sql.text(
"unnest(activity.ob_refs) WITH ORDINALITY")
.columns(sa.column('unnest', UUID),
sa.column('ordinality', sa.Integer))
.alias('act_ref'))
But the resulting SQL fails because it wraps the clause in parentheses:
LEFT OUTER JOIN (unnest(activity.ob_refs) WITH ORDINALITY)
AS act_ref ON subscription.ob_ref = act_ref.unnest
The error is syntax error at or near ")". Can I get TextAsFrom to not be wrapped in parentheses?
It turns out this is not directly supported by SA, but the correct behaviour can be achieved with a ColumnClause and a FunctionElement. First import this recipe as described by zzzeek in this SA issue. Then create a special unnest function that includes the WITH ORDINALITY modifier:
class unnest_func(ColumnFunction):
name = 'unnest'
column_names = ['unnest', 'ordinality']
#compiles(unnest_func)
def _compile_unnest_func(element, compiler, **kw):
return compiler.visit_function(element, **kw) + " WITH ORDINALITY"
You can then use it in joins, ordering, etc. like this:
act_ref = unnest_func(Activity.ob_refs)
query = (query
.add_columns(act_ref.c.unnest, act_ref.c.ordinality)
.outerjoin(act_ref, sa.true())
.outerjoin(Subscription, Subscription.ob_ref == act_ref.c.unnest)
.order_by(act_ref.c.ordinality.desc()))

ERROR: missing FROM-clause entry for table "movies"

I am new to SQL and need to query a database to extract certain information before I can import it into another software I am familiar with to analyse the data. This query was sent to me by a friend who I don't have access to at the moment, and I cannot figure out why it gives me the following error:
ERROR: missing FROM-clause entry for table "movies"
LINE 8: FROM (SELECT movies.movieid
Here is the query:
SELECT innerselect.movieid
,innerselect.title
,innerselect.year
,innerselect.imdbid
,innerselect.budget[1] AS budget_currency
,TO_NUMBER(innerselect.budget[2], '999999999999990.00') AS budget_total
,innerselect.businesstext
FROM (SELECT movies.movieid
,movies.title
,movies.year
,movies.imdbid
,business.businesstext
,regexp_matches(business.businesstext, '^BT:[ ](USD)[ ](-?(?!0)(?:\d+|\d{1,3}(?:,\d{3})+))', 'g') AS budget -- creates a PostgreSQL Array which contains the content matched with the RegEx Groups FROM movies LEFT JOIN business ON movies.movieid=business.movieid WHERE movies.movieid > 2753500
) AS innerselect
Any help would be greatly appreciated.
Problem is you put the FROM on the same line as the comment, so the FROM clause was ignored.
SELECT innerselect.movieid
,innerselect.title
,innerselect.year
,innerselect.imdbid
,innerselect.budget[1] AS budget_currency
,TO_NUMBER(innerselect.budget[2], '999999999999990.00') AS budget_total
,innerselect.businesstext
FROM (SELECT movies.movieid
,movies.title
,movies.year
,movies.imdbid
,business.businesstext
,regexp_matches(business.businesstext, '^BT:[ ](USD)[ ](-?(?!0)(?:\d+|\d{1,3}(?:,\d{3})+))', 'g') AS budget -- creates a PostgreSQL Array which contains the content matched with the RegEx Groups
FROM movies LEFT JOIN business ON movies.movieid=business.movieid WHERE movies.movieid > 2753500
) AS innerselect