I have two ecto queries, say query1 and query2, now I am performing full join between those two ecto queries, something like this
query1 |> join(:full, [a], b in subquery(query2), a.id == b.id)
Everything is working fine, but some of the records are returned as nil, something like this
[%user{}, %user{}, %user{}, nil, %user{}, %user{}, nil, %user{}, %user{}, nil]
I think the same issue has also been discussed in this ecto thread.
Is there some workaround for this ecto join issue.
That Ecto thread is a separate issue. :)
You are using a full join, which means the left and right side of a join will be returned even when there is no match. If you use an :inner join, it should work as expected:
query1 |> join(:inner, [a], b in subquery(query2), a.id == b.id)
If you need to keep a full join, maybe you want to return both a and b in your example?
query1
|> join(:inner, [a], b in subquery(query2), a.id == b.id)
|> select([a, b], {a, b})
This way it is guaranteed one of them is not nil.
Related
I use spring data jpa with native query
I have already some query like this
How to use native query none parameter.
String q="SELECT t1.blockNumber-1 FROM someTAble t1 LEFT JOIN someTAble t2 ON t2.blockNumber = t1.blockNumber-1 WHERE t2.blockNumber IS NULL AND t1.blockNumber> 0 ORDER BY t1.blockNumber";
#Query(value = q,nativeQuery = true)
List<Entity> findByBlockNumberIs();
they are occur errors Column 'sequence' not found.
That query means are when i insert some Contiguous data int value then i find missing data.
But
this query working
SELECT *,t1.blockNumber-1 FROM someTAble t1 LEFT JOIN someTAble t2 ON t2.blockNumber = t1.blockNumber-1 WHERE t2.blockNumber IS NULL AND t1.blockNumber> 0 ORDER BY t1.blockNumber
The difference between the two queries is whether there is a '*' or not
how to change simple to my query.
How to i changed error
OR How to use spring data jpa predicate
QEntity qBe1= QEntity .blockEntity;
QEntity qBe2= QEntity .blockEntity;
build.and(qBe2.blockNumber.eq(be.getBlockNumber()-1))
.and(qBe2.blockNumber.isNull().and(qBe1.blockNumber.gt(0)));
is predicate can use left join?
well...
use this.
List<Integer> findByBlockNumber()
Wonder if someone can help me with an array aggregator issue
I’ve got a query that does a join using a joining table then it filters down all values that are inside a given array and filters out values that are in another array.
The code looks like this:
Product
|> join(:inner, [j], jt in "job_tech", on: j.id == jt.product_id)
|> join(:inner, [j, jt], t in Tech, on: jt.ingredient_id == t.id)
|> group_by([j], j.id)
|> having_good_ingredients(good_ingredients)
|> not_having_bad_ingredients(bad_ingredients)
With having_good_ingredients look like this:
def having_good_ingredients(query, good_ingredients) do
if Enum.count(good_ingredients) > 0 do
query
|> having(fragment("array_agg(t2.name) && (?)::varchar[]", ^good_ingredients))
else
query
end
end
This works but it’ll grab all values that satisfy any of the values in the good_stacks array where I want them to only satisfy if all of the stacks work
i.e. if I’ve got [A, C] in my array, I want to return values that have A AND C, not just A and not just C.
Anyone have any ideas?
I believe you want to use the #> operator, instead of the overlap && operator:
having(fragment("array_agg(t2.name) #> (?)::varchar[]", ^good_ingredients))
Reference: https://www.postgresql.org/docs/current/functions-array.html#ARRAY-OPERATORS-TABLE
I'm trying to query an Ecto table with append-only semantics, so I'd like the most recent version of a complete row for a given ID. The technique is described here, but in short: I want to JOIN a table on itself with a subquery that fetches the most recent time for an ID. In SQL this would look like:
SELECT r.*
FROM rules AS r
JOIN (
SELECT id, MAX(inserted_at) AS inserted_at FROM rules GROUP BY id
) AS recent_rules
ON (
recent_rules.id = r.id
AND recent_rules.inserted_at = r.inserted_at)
I'm having trouble expessing this in Ecto. I tried something like this:
maxes =
from(m in Rule,
select: {m.id, max(m.inserted_at)},
group_by: m.id)
from(r in Rule,
join: m in ^maxes, on: r.id == m.id and r.inserted_at == m.inserted_at)
But trying to run this, I hit a restriction:
queries in joins can only have where conditions in query
suggesting maxes must just be a SELECT _ FROM _ WHERE form.
If I try switching maxes and Rule in the JOIN:
maxes =
from(m in Rule,
select: {m.id, max(m.inserted_at)},
group_by: m.id)
from(m in maxes,
join: r in Rule, on: r.id == m.id and r.inserted_at == m.inserted_at)
then I'm not able to SELECT the whole row, just id and MAX(inserted_at).
Does anyone know how to do this JOIN? Or a better way to query append-only in Ecto? Thanks 🙂
Doing m in ^maxes is not running a subquery but either query composition (if in a from) or converting the query to a join (in a join). In both cases, you are changing the same query. Given your initial query, I believe you want subqueries.
Also note that a subquery requires the select to return a map, so we can refer to the fields later on. Something along these lines should work:
maxes =
from(m in Rule,
select: %{id: m.id, inserted_at: max(m.inserted_at)},
group_by: m.id)
from(r in Rule,
join: m in ^subquery(maxes), on: r.id == m.id and r.inserted_at == m.inserted_at)
PS: I have pushed a commit to Ecto that clarifies the error message in cases like yours.
invalid query was interpolated in a join.
If you want to pass a query to a join, you must either:
1. Make sure the query only has `where` conditions (which will be converted to ON clauses)
2. Or wrap the query in a subquery by calling subquery(query)
I have this table structure:
Classic many to many relationship. I want to get all the orders for products belonging to the category for a small number of products I provide. It may be easier to show the SQL that does exactly what I want:
select o.*
from [Order] o join Product p2 on o.FKCatalogNumber=p2.CatalogNumber
where p2.FKCategoryId IN
(select c.Id
from Category c join Product p1 on p1.FKCategoryId=c.Id
where p1.CatalogNumber in ('0001', '0002')
This example gives me all the orders belonging to the categories that catalog #'s 0001 and 0002 are in.
But I am unable to wrap my head around the equivalent EF syntax for this query. I'm embarrassed to say I spent half the day on this. I bet it's easy for someone out there.
I came up with this but it's not working (and probably not even close):
string[] catNumbers = {"0001", "0002"};
var orders = ctx.Categories
.SelectMany(c => c.Products, (c, p) => new {c, p})
.Where(#t => catNumbers.Contains(#t.p.CatalogNumber))
.Select(#t => #t.p.Orders)
.ToList();
You can use query syntax (which looks very similar to SQL) in LINQ, so if you're more comfortable with SQL then you may prefer to write your query like this:
string[] catNumbers = {"0001", "0002"};
var orders = from o in ctx.Orders
join p2 in ctx.Products on o.FKCatalogNumber equals p2.CatalogNumber
where
(
from c in ctx.Categories
join p1 in ctx.Products on c.ID equals p1.FKCategoryId
where catNumbers.Contains(p1.CatalogNumber)
select c.ID
).Contains(p2.FKCategoryId)
select o;
As you can see, it's actually just your SQL query rearranged slightly, but it compiles as C#.
Note that:
the [Order] o syntax for referencing tables is replaced by o in ctx.Orders
LINQ enforces which way round you do the join condition so I had to flip your on o.FKCatalogNumber=p2.CatalogNumber to be on o.FKCatalogNumber equals p2.CatalogNumber
instead of your where p2.FKCategoryId IN (...), the equivalent c# is (...).Contains(p2.FKCategoryId)
the select comes last, not first
but those are the only major changes. Otherwise, it's written just like SQL.
I'd also draw your attention to a distinction regarding this comment:
the equivalent EF syntax for this query
The syntax here isn't specific to EF, but is just LINQ - Language Integrated Querying. It has two flavours: query syntax (sometimes called declarative) and method syntax (sometimes called fluent). LINQ works on just about any collection that implements IEnumerable or IQueryable, including EF's DbSet.
For more info on the different ways of querying, this MSDN page is a decent place to start. There's also this handy reference table showing the equivalent query syntax for each method-syntax operator, where applicable.
You can still nest queries in EF. The following looks like it works for me:
string[] catNumbers = {"0001", "0002"};
var orders = ctx.Orders
.Where(o => ctx.Products
.Where(p => catNumbers.Contains(p.CatalogNumber))
.Select(p => p.CategoryId)
.Contains(o.Product.CategoryId)
);
This produces the following SQL:
SELECT
[Extent1].[Id] AS [Id],
[Extent1].[CatalogNumber] AS [CatalogNumber]
FROM [dbo].[Orders] AS [Extent1]
WHERE EXISTS (SELECT
1 AS [C1]
FROM [dbo].[Products] AS [Extent2]
INNER JOIN [dbo].[Products] AS [Extent3] ON [Extent2].[CategoryId] = [Extent3].[CategoryId]
WHERE ([Extent1].[CatalogNumber] = [Extent3].[CatalogNumber]) AND ([Extent2].[CatalogNumber] IN (N'0001', N'0002'))
)
PostgreSQL Unicode 9.01 doesn't like:
SELECT table1.fielda,
SUM (CAST (table2.fielda AS INT)) AS header.specific
FROM *etc*
What is wrong with SUM-CAST?
Error Message:
Incorrect column expression: 'SUM (CAST
(specifics_nfl_3pl_work_order_item.delivery_quantity AS INT))
Query:
SELECT specifics_nfl_3pl_work_order.work_order_number,
specifics_nfl_3pl_work_order.goods_issue_date,
specifics_nfl_3pl_work_order.order_status_id,
SUM (CAST (specifics_nfl_3pl_work_order_item.delivery_quantity AS INT)) AS units
FROM public.specifics_nfl_3pl_work_order specifics_nfl_3pl_work_order,
public.specifics_nfl_3pl_work_order_item specifics_nfl_3pl_work_order_item,
public.specifics_nfl_order_status specifics_nfl_order_status
WHERE specifics_nfl_3pl_work_order.order_status_id In (3,17,14)
AND specifics_nfl_3pl_work_order_item.specifics_nfl_work_order_id=
specifics_nfl_3pl_work_order.id
AND ((specifics_nfl_3pl_work_order.sold_to_id<>'0000000000')
AND (specifics_nfl_3pl_work_order.goods_issue_date>={d '2013-08-01'}))
It would be really great if you can help.
If I were you, then I would do these steps:
give your table short aliases
format the query
use proper ANSI joins:
remove spaces between function name and (
select
o.work_order_number,
o.goods_issue_date,
o.order_status_id,
sum(cast(oi.delivery_quantity as int)) as units
from public.specifics_nfl_3pl_work_order as o
inner join public.specifics_nfl_3pl_work_order_item as oi on
oi.specifics_nfl_work_order_id = o.id
-- inner join public.specifics_nfl_order_status os -- seems redundant
where
o.order_status_id In (3,17,14) and
o.sold_to_id <> '0000000000' and
o.goods_issue_date >= {d '2013-08-01'}
Actually I really think you need group by clause here:
select
o.work_order_number,
o.goods_issue_date,
o.order_status_id,
sum(cast(oi.delivery_quantity as int)) as units
from public.specifics_nfl_3pl_work_order as o
inner join public.specifics_nfl_3pl_work_order_item as oi on
oi.specifics_nfl_work_order_id = o.id
where
o.order_status_id In (3,17,14) and
o.sold_to_id <> '0000000000' and
o.goods_issue_date >= {d '2013-08-01'}
group by
o.work_order_number,
o.goods_issue_date,
o.order_status_id
if it still doesn't work - try to comment sum and see is it working?
But you have a table2 or only table1?
Try:
SELECT table1.fielda,
SUM (CAST (table1.fielda AS INT)) AS "header.specific"
FROM etc
In addition to what #Roman already cleared up, there are more problems here:
SELECT o.work_order_number
,o.goods_issue_date
,o.order_status_id
,SUM(CAST(oi.delivery_quantity AS INT)) AS units -- suspicious
FROM public.specifics_nfl_3pl_work_order o,
JOIN public.specifics_nfl_3pl_work_order_item oi
ON oi.specifics_nfl_work_order_id = o.id
CROSS JOIN public.specifics_nfl_order_status os -- probably wrong
WHERE o.order_status_id IN (3,17,14)
AND o.sold_to_id <> '0000000000' -- suspicious
AND o.goods_issue_date> = {d '2013-08-01'} -- nonsense
GROUP BY 1, 2, 3
o.goods_issue_date> = {d '2013-08-01'} is syntactical nonsense. Maybe you mean:
o.goods_issue_date> = '2013-08-01'
You have the table specifics_nfl_order_status in your FROM list, but without any expression connecting it to the rest. This effectively results in a CROSS JOIN, which results in a Cartesian product and is almost certainly wrong in a very expensive way: every row is combined with every row of the rest:
CROSS JOIN public.specifics_nfl_order_status os
Either remove the table (since you don't use it) or add a WHERE or ON clause to connect it to the rest. Note, that it is not just redundant, it has a dramatic effect on the result as it is.
This WHERE clause is suspicious:
AND o.sold_to_id <> '0000000000'
Seems like you are storing numbers as strings or otherwise confusing the two.
Also, CAST (oi.delivery_quantity AS INT) should not be needed to begin with. The column should be of data type integer or some other appropriate numeric type to begin with. Be sure to use proper data types.
The default setting of search_path includes public, and you may not need to schema-qualify tables. Instead of public.specifics_nfl_3pl_work_order, it may suffice to use:
specifics_nfl_3pl_work_order
GROUP BY 1, 2, 3 is using positional parameters, just a notational shortcut for:
GROUP BY o.work_order_number, o.goods_issue_date, o.order_status_id
Details in the manual.
According to comments you are using MS Query to create the query. This is not the best of ideas. Produces the kind of inferior code you presented us with. You may want to get rid of that while you are working with PostgreSQL.