Does the ORM read_group on Odoo 11 has the same features of Odoo 12? - group-by

I am working on an Odoo 11 project and unfortunately, we can't migrate to Odoo 12.
Reading the book Odoo 12 Development Cookbook - Third Edition I found a way to create my aggregate method WITHOUT coding SQL with GROUP BY, just using the read_group method available on Odoo API.
Here is what I do to get it using SQL:
SELECT SUM(ai.amount_total) as sum_total,
AVG(ai.amount_total) as avg_total
FROM account_invoice as ai
WHERE ai.partner_id = :partner_id AND ai.type = 'out_invoice'
And here is how I am trying to get it working on my model class:
class MyModel(models.Model):
# fields definition
#api.multi
def get_aggregated_invoices(self):
self.ensure_one()
invoices_partner_domain = [
('partner_id', '=', self.partner_id.id),
('type', '=', 'out_invoice')
]
grouped_result = self.env['account.invoice'].read_group(
invoices_partner_domain,
[
'partner_id',
'sum_total:sum(amount_total)',
'avg_total:avg(amount_total)'
],
['partner_id']
)
if len(grouped_result) > 0:
print(str(grouped_result)) # << DEBUG breakpoint here
return {
'sum_total': grouped_result[0]['sum_total'],
'sum_total': grouped_result[0]['avg_total'],
}
else:
return None
Debugging the code I got this result:
Am I missing anything or Odoo 11 does not have the same features as Odoo 12?
If not, is there any way to do it through Odoo API or should I need to stick with my GROUP BY SQL statement instead?

The v12 version introduces the use of aggregation functions in fields parameter.
:param list fields: list of fields present in the list view specified on the object.
Each element is either field (field name, using the default aggregation),
or field:agg (aggregate field with aggregation function agg),
or name:agg(field) (aggregate field with agg and return it as name).
The possible aggregation functions are the ones provided by PostgreSQL
(https://www.postgresql.org/docs/current/static/functions-aggregate.html)
and count_distinct, with the expected meaning.
The difference is in the process of the aggregated field in the _read_group_raw function, Odoo will try to use the aggregation function if defined and valid, if not it will use the field group_operator as an aggregation function.
Odoo 11 will not check if we defined an aggregation function and use the group_operator. In your example, the aggregated_fields will be an empty list.
The default group_operator for amount_total field is sum. If you use the SQL debug level, you will see that Odoo executes the following query:
SELECT min("account_invoice".id) AS id, count("account_invoice".id) AS "partner_id_count" , sum("account_invoice"."amount_total") AS "amount_total" ,"account_invoice"."partner_id" as "partner_id"
FROM "account_invoice" LEFT JOIN "res_partner" as "account_invoice__partner_id" ON ("account_invoice"."partner_id" = "account_invoice__partner_id"."id")
WHERE (("account_invoice"."partner_id" = %s) AND ("account_invoice"."type" = %s))
GROUP BY "account_invoice"."partner_id","account_invoice__partner_id"."display_name"
ORDER BY "account_invoice__partner_id"."display_name"
When calling the read_group function as following (use sum as an aggregation function):
grouped_result = self.env['account.invoice'].read_group(
invoices_partner_domain,
[
'partner_id',
'amount_total',
],
['partner_id']
)

Related

SELECT from result of UPDATE ... RETURNING in jOOQ

I'm transforming some old PostgreSQL code to jOOQ, and I'm currently struggling with SQL that has multiple WITH clauses, where each one depends on previous. It would be best to keep the SQL logic the way it was written and not to change it (e.g. multiple queries to DB).
As it seems, there is no way to do SELECT on something that is UPDATE ... RETURNING, for example
dsl.select(DSL.asterisk())
.from(dsl.update(...)
.returning(DSL.asterisk())
)
I've created some test tables, trying to create some sort of MVCE:
create table dashboard.test (id int primary key not null, data text); --test table
with updated_test AS (
UPDATE dashboard.test SET data = 'new data'
WHERE id = 1
returning data
),
test_user AS (
select du.* from dashboard.dashboard_user du, updated_test -- from previous WITH
where du.is_active AND du.data = updated_test.data
)
SELECT jsonb_build_object('test_user', to_jsonb(tu.*), 'updated_test', to_jsonb(ut.*))
FROM test_user tu, updated_test ut; -- from both WITH clauses
So far this is my jOOQ code (written in Kotlin):
dsl.with("updated_test").`as`(
dsl.update(Tables.TEST)
.set(Tables.TEST.DATA, DSL.value("new data"))
.returning(Tables.TEST.DATA) //ERROR is here: Required Select<*>, found UpdateResultStep<TestRecord>
).with("test_user").`as`(
dsl
.select(DSL.asterisk())
.from(
Tables.DASHBOARD_USER,
DSL.table(DSL.name("updated_test")) //or what to use here?
)
.where(Tables.DASHBOARD_USER.IS_ACTIVE.isTrue
.and(Tables.DASHBOARD_USER.DATA.eq(DSL.field(DSL.name("updated_test.data"), String::class.java)))
)
)
.select() //here goes my own logic for jsonBBuildObject (which is tested and works for other queries)
.from(
DSL.table(DSL.name("updated_test")), //what to use here
DSL.table(DSL.name("test_user")) //or here
)
Are there any workarounds for this? I'd like to avoid changing SQL if possible.
Also, in this project this trick is used very often to get JSON(B) from UPDATE clause (table has JSON(B) columns too):
with _updated AS (update dashboard.test SET data = 'something' WHERE id = 1 returning *)
select to_jsonb(_updated.*) from _updated;
and it will be a real step back for us if there is no workaround for this.
I'm using JOOQ version 3.13.3, and Postgres 12.0.
This is currently not supported in jOOQ, see:
https://github.com/jOOQ/jOOQ/issues/3185
https://github.com/jOOQ/jOOQ/issues/4474
The workaround is, as always, when some vendor specific syntax is unsupported, to resort to plain SQL templating
E.g.
// If you don't need to map data types
dsl.fetch("with t as ({0}) {1}", update, select);
// If you need to map data types
dsl.resultQuery("with t as ({0}) {1}", update, select).coerce(select.getSelect()).fetch();

Group by date intervals using JPA's Criteria API

I'm trying to group entities by date intervals using JPA's Criteria API. I use this way of querying for entities as this is a part of the service that serves API requests which may ask for any field of any entity, including sorting, filtering, grouping and aggregations. Everything works fine except for grouping by date fields. My underlying DBMS i PostgreSQL.
To give a minimal example, here's my entity class:
#Entity
#Table(name = "receipts")
public class DbReceipt {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
private Date sellDate;
// Many other fields
}
This example discusses grouping my "month" interval (therefore grouping by year+month), but in the end I'm looking for a solution that would let me group by any interval, such as "year", "day" or "minutes".
What I'm trying to achieve is the following query, but using Criteria API:
SELECT TO_CHAR(sell_date, 'YYYY-MM') AS alias1 FROM receipts GROUP BY alias1;
My attempt to do so is this:
#Service
public class ReceiptServiceImpl extends ReceiptService {
#Autowired
private EntityManager em;
#Override
public void test() {
CriteriaBuilder cb = em.getCriteriaBuilder();
CriteriaQuery<Object[]> query = cb.createQuery(Object[].class);
Root<?> root = query.from(DbReceipt.class);
Expression<?> expr = cb.function("to_char", String.class, root.get("sellDate"), cb.literal("YYYY-MM"));
query.groupBy(expr);
query.multiselect(expr);
TypedQuery<Object[]> typedQuery = em.createQuery(query);
List<Object[]> resultList = typedQuery.getResultList();
}
}
The reason I use to_char function and not MONTH and similar is that I need entities like 2019-05 and 2020-05 to not be grouped together. I also narrow this example down to only year and month to keep things short, but the goal is to group by any date interval.
The code above creates the following query (SQL logging enabled) which results in an error:
Hibernate: select to_char(dbreceipt0_.sell_date, ?) as col_0_0_ from receipts dbreceipt0_ group by to_char(dbreceipt0_.sell_date, ?)
24-05-2020 12:16:30.071 [http-nio-1234-exec-5] WARN o.h.e.jdbc.spi.SqlExceptionHelper.logExceptions - SQL Error: 0, SQLState: 42803
24-05-2020 12:16:30.071 [http-nio-1234-exec-5] ERROR o.h.e.jdbc.spi.SqlExceptionHelper.logExceptions - ERROR: column "dbreceipt0_.sell_date" must appear in the GROUP BY clause or be used in an aggregate function
Position: 16
which to me is caused by the fact that the whole expression is put into the 'group by' part of the query, rather than just an alias. Now, I've tried to assign an alias to the expression (which returns Selection<T> and groupBy accepts expressions, therefore I can only really use that in the multiselect), but that didn't affect how the query is performed - nothing changed.
How do I achieve grouping by year and month as described above using Criteria API? Maybe there's a different way other than using to_char? Maybe there's a way to give an alias to the groupBy method that would cause it to group by an alias instead of the whole expression?
I think it's a bug in PostgreSQL (the error comes from there, not from Hibernate). I have tried a slightly modified version of your code with EclipseLink + Derby and works perfectly.
Note that I had to use numbers instead of strings because Derby DB doesn't have an equivalent of TO_CHAR function.
Expression<Integer> year = cb.function("YEAR", Integer.class, root.get("sellDate"));
Expression<Integer> month = cb.function("MONTH", Integer.class, root.get("sellDate"));
Expression<Integer> expr = cb.sum(month, cb.prod(12, year));
query.groupBy(expr);
query.multiselect(expr);
This returns the following SQL:
SELECT (MONTH(MY_DATE) + (12 * YEAR(MY_DATE)))
FROM MY_DATE_TABLE
GROUP BY (MONTH(MY_DATE) + (12 * YEAR(MY_DATE)))
Note that there are no portable solutions for manipulating dates in JPA criteria queries. If the number of groups to be queried simultaneously is not too high I'd go with a more practical approach where you find the dates in Java and pass them as literals to the query builder.
Another workaround is to query with a groupBy(root.get("sellDate")) and then aggregate the results in Java according to the desired time period.
Post Scriptum: I don't think it's relevant, however I modified the query's return type from Object[] to Object.

Conditional WHERE clause in KDB?

Full Query:
{[tier;company;ccy; startdate; enddate] select Deal_Time, Deal_Date from DEALONLINE_REMOVED where ?[company = `All; 1b; COMPANY = company], ?[tier = `All;; TIER = tier], Deal_Date within(startdate;enddate), Status = `Completed, ?[ccy = `All;1b;CCY_Pair = ccy]}
Particular Query:
where ?[company = `All; 1b; COMPANY = company], ?[tier = `All; 1b; TIER = tier],
What this query is trying to do is to get the viewstate of a dropdown.
If there dropdown selection is "All", that where clause i.e. company or tier is invalidated, and all companies or tiers are shown.
I am unsure if the query above is correct as I am getting weird charts when displaying them on KDB dashboard.
What I would recommend is to restructure your function to make use of the where clause using functional qSQL.
In your case, you need to be able to filter based on certain input, if its "All" then don't filter else filter on that input. Something like this could work.
/Define sample table
DEALONLINE_REMOVED:([]Deal_time:10#.z.p;Deal_Date:10?.z.d;Company:10?`MSFT`AAPL`GOOGL;TIER:10?`1`2`3)
/New function which joins to where clause
{[company;tier]
wc:();
if[not company=`All;wc:wc,enlist (=;`Company;enlist company)];
if[not tier=`All;wc:wc,enlist (=;`TIER;enlist tier)];
?[DEALONLINE_REMOVED;wc;0b;()]
}[`MSFT;`2]
If you replace the input with `All you will see that everything is returned.
The full functional select for your query would be as follows:
whcl:{[tier;company;ccy;startdate;enddate]
wc:(enlist (within;`Deal_Date;(enlist;startdate;enddate))),(enlist (=;`Status;enlist `Completed)),
$[tier=`All;();enlist (=;`TIER;enlist tier)],
$[company=`All;()enlist (=;`COMPANY;enlist company)],
$[ccy=`All;();enlist (=;`CCY_Pair;enlist ccy)];
?[`DEALONLINE_REMOVED;wc;0b;`Deal_Time`Deal_Date!`Deal_Time`Deal_Date]
}
The first part specifies your date range and status = `Completed in the where clause
wc:(enlist (within;`Deal_Date;(enlist;startdate;enddate))),(enlist (=;`Status;enlist `Completed)),
Next each of these conditionals checks for `All for the TIER, COMPANY and CCY_Pair column filtering. It then joins these on to the where clause when a specific TIER, COMPANY or CCY_Pair are specified. (otherwise an empty list is joined on):
$[tier=`All;();enlist (=;`TIER;enlist tier)],
$[company=`All;();enlist (=;`COMPANY;enlist company)],
$[ccy=`All;();enlist (=;`CCY_Pair;enlist ccy)];
Finally, the select statement is called in its functional form as follows, with wc as the where clause:
?[`DEALONLINE_REMOVED;wc;0b;`Deal_Time`Deal_Date!`Deal_Time`Deal_Date]

AREL: writing complex update statements with from clause

I tried looking for an example of using Arel::UpdateManager to form an update statement with a from clause (as in UPDATE t SET t.itty = "b" FROM .... WHERE ...), couldn.t find any. The way I've seen it, Arel::UpdateManager sets the main engine on initialization and allows to set the various fields and values to update. Is there actually a way to do this?
Another aside would be to find out how to express Postgres posix regex matching into ARel, but this might be impossible by now.
As far as I see the current version of arel gem is not support FROM keyword for the sql query. You can generate a query using the SET, and WHERE keywords only, like:
UPDATE t SET t.itty = "b" WHERE ...
and the code, which copies a value from field2 to field1 for the units table, will be like:
relation = Unit.all
um = Arel::UpdateManager.new(relation.engine)
um.table(relation.table)
um.ast.wheres = relation.wheres.to_a
um.set(Arel::Nodes::SqlLiteral.new('field1 = "field2"'))
ActiveRecord::Base.connection.execute(um.to_sql)
Exactly you can use the additional method to update a relation. So we create the Arel's UpdateManager, assigning to it the table, where clause, and values to set. Values shell be passed to the method as an argument. Then we need to add FROM keyword to the generated SQL request, we add it only if we have access to external table of the specified one by the UPDATE clause itself. And at the last we executes the query. So we get:
def update_relation!(relation, values)
um = Arel::UpdateManager.new(relation.engine)
um.table(relation.table)
um.ast.wheres = relation.wheres.to_a
um.set(values)
sql = um.to_sql
# appends FROM field to the query if needed
m = sql.match(/WHERE/)
tables = relation.arel.source.to_a.select {|v| v.class == Arel::Table }.map(&:name).uniq
tables.shift
sql.insert(m.begin(0), "FROM #{tables.join(",")} ") if m && !tables.empty?
# executes the query
ActiveRecord::Base.connection.execute(sql)
end
The you can issue the the relation update as:
values = Arel::Nodes::SqlLiteral.new('field1 = "field2", field2 = NULL')
relation = Unit.not_rejected.where(Unit.arel_table[:field2].not_eq(nil))
update_relation!(relation, values)

Laravel 3 Eloquent How to select column as

I'm trying to figure out how to give a column an alias using Eloquent.
So, in other words, how do I execute the following mysql query using Eloquent?
SELECT occupation AS test FROM users WHERE occupation = 'PIMP';
Thx in adv!
Eloquent returns a regular Fluent query. So you can try something like this (assuming your model name is 'User'):
$user = User::where_occupation('pimp')->get(array('occupation as test'));
This is how I have been able to do this in Laravel 5 using select() and passing the col name and the alias to that method (here I'm also using groupby() to restrict it to "DISTINCT" return values then using toarray() to return any array instead of a Collection Object:
$results = asset::select('model_code__c AS option')->whereRAW("model_code__c <> '' AND status = 'A'")->groupby('model_code__c')->get()->toarray();