I need to build a complex query, in part from a "filter framework". A filter framework is a reflective component that adds up as many and filters as possible automagically.
I have successfully created the filter framework that generates and filters based on user input via MVC. The following is a typical query
SELECT * from Entity
WHERE
attribute1 = ?
AND attribute2 > ?
AND attribute3 IN (????);
In Mybatis Dynamic SQL, you chain up calls to AbstractWhereDSL.and and you are done.
Given that I have a reference to an automatically-generated WhereApplier, which is a Consumer<AbstractWhereDSL<?>>, I need to chain up another condition that is a complex condition. The resulting query MUST be
SELECT * from Entity
WHERE
attribute1 = ?
AND attribute2 > ?
AND attribute3 IN (????)
AND (
(attribute4 = ? AND attribute5 IN (???))
OR (attribute4 = ? AND attribute5 IN (???))
OR (attribute4 = ? AND attribute5 IN (???))
)
;
AbstractWhereDSL accepts only BindableColumn and ExistsPredicate.
If I do something like
ret = ret.applyWhere(where -> where
.and(attribute4,isEqualTo(x)).and(attribute5,isIn(y))
.or(attribute4,isEqualTo(z)).and(attribute5,isIn(a))
);
I will end up into a flat query
SELECT * from Entity
WHERE
attribute1 = ?
AND attribute2 > ?
AND attribute3 IN (????)
AND attribute4 = ?
AND attribute5 IN (???)
OR attribute4 = ?
AND attribute5 IN (???)
OR attribute4 = ?
AND attribute5 IN (???)
;
Which is bad in my scenario.
Question: how can I create a sub-predicate in a where condition in Mybtis Dynamic SQL?
Context: while I have discussed this multiple times with the team, I need to segregate queries according to attribute-based visibility. The attributes are two and must be in combination. I have already optimized the query to group all values of attribute5 (legal entity ID) that come along with the same attribute4 (workflow type) in the permission set, otherwise the final query becomes an endless list of (attribute4 = ? and attribute5 = ?)
In this context, it would be extremely simple to do attribute4 in (?) and attribute5 in (?) but some results would be acceptable (you can see legal entity ACME in workflow PERFORMANCE, but not in COMPLIANCE, to provide an example)
If you are using mybatis-dynamic-sql library version 1.4.0, It supports grouping of where conditions. Please refer below git link for reference
https://github.com/mybatis/mybatis-dynamic-sql/blob/master/src/test/java/examples/complexquery/GroupingTest.java
Related
I am currently using EFCore 1.1 (preview release) with SQL Server.
I am doing what I thought was a simple OUTER JOIN between an Order and OrderItem table.
var orders = from order in ctx.Order
join orderItem in ctx.OrderItem
on order.OrderId equals orderItem.OrderId into tmp
from oi in tmp.DefaultIfEmpty()
select new
{
order.OrderDt,
Sku = (oi == null) ? null : oi.Sku,
Qty = (oi == null) ? (int?) null : oi.Qty
};
The actual data returned is correct (I know earlier versions had issues with OUTER JOINS not working at all). However the SQL is horrible and includes every column in Order and OrderItem which is problematic considering one of them is a large XML Blob.
SELECT [order].[OrderId], [order].[OrderStatusTypeId],
[order].[OrderSummary], [order].[OrderTotal], [order].[OrderTypeId],
[order].[ParentFSPId], [order].[ParentOrderId],
[order].[PayPalECToken], [order].[PaymentFailureTypeId] ....
...[orderItem].[OrderId], [orderItem].[OrderItemType], [orderItem].[Qty],
[orderItem].[SKU] FROM [Order] AS [order] LEFT JOIN [OrderItem] AS
[orderItem] ON [order].[OrderId] = [orderItem].[OrderId] ORDER BY
[order].[OrderId]
(There are many more columns not shown here.)
On the other hand - if I make it an INNER JOIN then the SQL is as expected with only the columns in my select clause:
SELECT [order].[OrderDt], [orderItem].[SKU], [orderItem].[Qty] FROM
[Order] AS [order] INNER JOIN [OrderItem] AS [orderItem] ON
[order].[OrderId] = [orderItem].[OrderId]
I tried reverting to EFCore 1.01, but got some horrible nuget package errors and gave up with that.
Not clear whether this is an actual regression issue or an incomplete feature in EFCore. But couldn't find any further information about this elsewhere.
Edit: EFCore 2.1 has addressed a lot of issues with grouping and also N+1 type issues where a separate query is made for every child entity. Very impressed with the performance in fact.
3/14/18 - 2.1 Preview 1 of EFCore isn't recommended because the GROUP BY SQL has some issues when using OrderBy() but it's fixed in nightly builds and Preview 2.
The following applies to EF Core 1.1.0 (release).
Although shouldn't be doing such things, tried several alternative syntax queries (using navigation property instead of manual join, joining subqueries containing anonymous type projection, using let / intermediate Select, using Concat / Union to emulate left join, alternative left join syntax etc.) The result - either the same as in the post, and/or executing more than one query, and/or invalid SQL queries, and/or strange runtime exceptions like IndexOutOfRange, InvalidArgument etc.
What I can say based on tests is that most likely the problem is related to bug(s) (regression, incomplete implementation - does it really matter) in GroupJoin translation. For instance, #7003: Wrong SQL generated for query with group join on a subquery that is not present in the final projection or #6647 - Left Join (GroupJoin) always materializes elements resulting in unnecessary data pulling etc.
Until it get fixed (when?), as a (far from perfect) workaround I could suggest using the alternative left outer join syntax (from a in A from b in B.Where(b = b.Key == a.Key).DefaultIfEmpty()):
var orders = from o in ctx.Order
from oi in ctx.OrderItem.Where(oi => oi.OrderId == o.OrderId).DefaultIfEmpty()
select new
{
OrderDt = o.OrderDt,
Sku = oi.Sku,
Qty = (int?)oi.Qty
};
which produces the following SQL:
SELECT [o].[OrderDt], [t1].[Sku], [t1].[Qty]
FROM [Order] AS [o]
CROSS APPLY (
SELECT [t0].*
FROM (
SELECT NULL AS [empty]
) AS [empty0]
LEFT JOIN (
SELECT [oi0].*
FROM [OrderItem] AS [oi0]
WHERE [oi0].[OrderId] = [o].[OrderId]
) AS [t0] ON 1 = 1
) AS [t1]
As you can see, the projection is ok, but instead of LEFT JOIN it uses strange CROSS APPLY which might introduce another performance issue.
Also note that you have to use casts for value types and nothing for strings when accessing the right joined table as shown above. If you use null checks as in the original query, you'll get ArgumentNullException at runtime (yet another bug).
Using "into" will create a temporary identifier to store the results.
Reference : MDSN: into (C# Reference)
So removing the "into tmp from oi in tmp.DefaultIfEmpty()" will result in the clean sql with the three columns.
var orders = from order in ctx.Order
join orderItem in ctx.OrderItem
on order.OrderId equals orderItem.OrderId
select new
{
order.OrderDt,
Sku = (oi == null) ? null : oi.Sku,
Qty = (oi == null) ? (int?) null : oi.Qty
};
I want to realize the following (for this example: MySQL) statement in JPA 2.1 with Eclipselink as the implementation:
select *
from account
where (mailing_zip_code, id) > (56237, 275)
order by mailing_zip_code asc, id asc
limit 10;
What I want to achieve is an implementation of the seek method for endless scrolling without using an offset in the query. With focus on the where clause, I am not able to find a correct name for this construct. In some places it is called 'composite value', but I am not able to find results by this name, but it seems to be compliant with the SQL-92 standard.
This statement will be extended by filter parameters, so naturally I would like to build it with the JPA criteria API.
I searched the API for some time now, does anybody now if and how this is possible?
After realizing that
select *
from account
where (mailing_zip_code, id) > (56237, 275)
order by mailing_zip_code asc, id asc
limit 10;
is just a shorter way to write
select *
from account
where mailing_zip_code > '56237'
or (mailing_zip_code = '56237' AND id > 276)
order by mailing_zip_code asc, id asc
limit 10;
the criteria query was not that hard after all (appended just the predicate):
if (null != startId) {
Predicate dateGreater = cb.greaterThan(entity.get(sortBy), startValue);
Predicate dateEqual = cb.equal(entity.get(sortBy), startValue);
Predicate idGreater = cb.greaterThan(entity.get("id"), startId);
Predicate dateEqualAndIdGreater = cb.and(dateEqual, idGreater);
cb.or(dateGreater, dateEqualAndIdGreater);
}
If there is a nicer way, I would be very happy to learn about it.
Entity Framework and Linq-To-Entities are really giving me some headaches. I have a fairly simple query:
var result = feed.FeedItems.Any(ei => ei.ServerId == "12345");
feed is a single EF entity I selected earlier in a separate query from the same context.
But the generated SQL just throws away the .Any condition and requests all FeedItems of the feed object which can be several thousands of records which is a waste of Network bandwith. Seems the actual .Any comparison is done in C#:
exec sp_executesql N'SELECT [t0].[Id], [t0].[FeedId], [t0].[ServerId], [t0].[Published], [t0].[Inserted], [t0].[Title], [t0].[Content], [t0].[Author], [t0].[WebUri], [t0].[CommentsUri]
FROM [dbo].[FeedItem] AS [t0]
WHERE [t0].[FeedId] = #p0',N'#p0 int',#p0=3
I also tried:
!feed.FeedItems.Where(ei => ei.ServerId == "12345").Any();
But it doesn't change anything. Even removing Any() and querying for the complete list of items does not change the query.
I don't get it ... why isn't this working as I would expect? There should be a
WHERE ServerId == 1234
clause in the SQL statement.
Thanks very much for any help/clarification :)
As Nicholas already noticed, looks like query executed in FeedItems property (possibly you are returning List or IEnumerable) and whole list of items are returned from database. After that you are applying Any to in-memory collection. That's why you don't see WHERE ServerId == 1234 in SQL query.
When you apply Any to IQueryable generated query will look like:
SELECT
(CASE
WHEN EXISTS(
SELECT NULL AS [EMPTY]
[dbo].[FeedItem] AS [t0]
WHERE [t0].[ServerId] = #p0
) THEN 1
ELSE 0
END) AS [value]
I'm new to Entity Framework. I have a database query which I need to convert to Entity Framework. how to write the query in LinQ to Enity
Can someone help me on that?
SELECT
FLD1,
FLD2,
SUM(FLD3),
(TO_CHAR(FLD4,'MM/DD/YYYY'))
FROM
TABLE1
WHERE
(FLD2=XXX ) AND
(FLD3 BETWEEN TO_DATE(VARDATE,'MMDDYYYY') AND TO_DATE(VARDATE1,'MMDDYYYY'))
GROUP BY
FLD1,
FLD2,
FLD4
Well...info is sparse and you filled it with a lot of different cases something like this would do.
_context.SomeObject
.Where(x=>x.SomeField == "SomeValue" && x.SomeField > 5 && x.SomeField < 10)
.Select(x=>new { x.SomeField1, x.SomeField2, x.SomeField2, SomeField4 = x.SomeChildCollection.Sum(y=>y.SomeChildvalue)
.GroupBy(x=>new {x.SomeField1, x.SomeField2, x.SomeField3})
.ToList()
This would result in a group where the key was an object with the values SomeField1, SomeField2, SomeField3, and the object would be the an anonymous projection with the 4 properties in the Select.
In some kinds of comparisions regarding dates you might need to use EntityFunctions.
I have the following SQL query, which I'm struggling to convert to LINQ.
Purpose: Get the top 10 coupons from the table, ordered by the date they expire (i.e. list the ones that are about to expire first) and then randomly choosing one of those for publication.
Notes: Because of the way the database is structured, there maybe duplicate Codes in the Coupon table. Therefore, I am using a GROUP BY to enforce distinction, because I can't use DISTINCT in the sub select query (which I think is correct). The SQL query works.
SELECT TOP 1
c1.*
FROM
Coupon c1
WHERE
Code IN (
SELECT TOP 10
c2.Code
FROM
Coupon c2
WHERE
c2.Published = 0
GROUP BY
c2.Code,
c2.Expires
ORDER BY
c2.Expires
)
ORDER BY NEWID()
Update:
This is as close as I have got, but in two queries:
var result1 = (from c in Coupons
where c.Published == false
orderby c.Expires
group c by new { c.Code, c.Expires } into coupon
select coupon.FirstOrDefault()).Take(10);
var result2 = (from c in result1
orderby Guid.NewGuid()
select c).Take(1);
Here's one possible way:
from c in Coupons
from cs in
((from c in coupons
where c.published == false
select c).Distinct()
).Take(10)
where cs.ID == c.ID
select c
Keep in mind that LINQ creates a strongly-typed data set, so an IN statement has no general equivalent. I understand trying to keep the SQL tight, but LINQ may not be the best answer for this. If you are using MS SQL Server (not SQL Server Compact) you might want to consider doing this as a Stored Procedure.
Using MercurioJ's slightly buggy response, in combination with another SO suggested random row solution my solution was:
var result3 = (from c in _dataContext.Coupons
from cs in
((from c1 in _dataContext.Coupons
where
c1.IsPublished == false
select c1).Distinct()
).Take(10)
where cs.CouponId == c.CouponId
orderby _dataContext.NewId()
select c).Take(1);