I'm trying to translate the following raw sql into SQLAlchemy ORM
Raw SQL:
objects = request.db.execute("select user_login.userlogin_id, user_friend.status,"\
" user_login.email, user_info.fname, user_info.lname"\
" from user_login "\
" LEFT OUTER JOIN user_info on "\
" user_info.userlogin_id = user_login.userlogin_id "\
" LEFT OUTER JOIN user_friend on "\
" (user_info.userlogin_id = user_friend.send_from_userlogin_id or"\
" user_info.userlogin_id = user_friend.send_to_userlogin_id )"\
" where email='%s' " %request.POST['q'])
SQLAlchemy version:
objects = request.db.query(UserLogin.email, UserLogin.userlogin_id, UserInfo.fname,
UserInfo.lname, UserFriend.status)\
.outerjoin(UserInfo, UserLogin.userlogin_id == UserInfo.userlogin_id)\
.outerjoin(UserFriend,
or_(UserLogin.userlogin_id == UserFriend.send_from_userlogin_id \
,UserLogin.userlogin_id == UserFriend.send_to_userlogin_id) )\
.filter(UserLogin.email==request.POST['q'])
I took a look at the log and it is the following which looks a bit strange to me, as it doesn't have any parentheses:
INFO [sqlalchemy.engine.base.Engine][Dummy-4] SELECT user_login.email AS
user_login_email, user_login.userlogin_id AS user_login_userlogin_id,
user_info.fname AS user_info_fname, user_info.lname AS user_info_lname,
user_friend.status AS user_friend_status FROM user_login LEFT OUTER JOIN
user_info ON user_login.userlogin_id = user_info.userlogin_id LEFT OUTER JOIN
user_friend ON
user_login.userlogin_id = user_friend.send_from_userlogin_id OR
user_login.userlogin_id = user_friend.send_to_userlogin_id
WHERE user_login.email = %(email_1)s
How should I structure the sqlalchemy such that there will be () for user_login.userlogin_id = user_friend.send_from_userlogin_id OR
user_login.userlogin_id = user_friend.send_to_userlogin_id for the last LEFT OUTER JOIN?
Related
After upgrading to spring-data-jpa 3.0.0 a JPQL query that uses Pageable is returning less elements than expected.
I executed the generated SQL from the console and that returns the correct number of elements.
I don't see a count SQL query being generated when using spring-data-jpa 3.0.0
JPQL Query:
#query(value = "select fsd from FeeScheduleDrugEntity fsd "
+ "left join DrugNdcEntity ndc on fsd.drug.id = ndc.drug.id and fsd.drug.noc = true "
+ "left join FeeScheduleSourceEntity fsse on fsse.id = fsd.drugFeeScheduleSource.id "
+ "where fsd.feeSchedule.id = :feeScheduleId ")
Generated SQL:
select f1_0.fee_schedule_drug_id,
f1_0.allowable_per_billing_unit,
f1_0.fee_schedule_drug_source,
f1_0.created_at,
f1_0.created_by,
f1_0.drug_id,
f1_0.fee_schedule_item_source_id,
f1_0.fee_schedule_id,
f1_0.modified_at,
f1_0.modified_by
from core.fee_schedule_drug f1_0
join core.drug d2_0 on d2_0.drug_id = f1_0.drug_id
left join core.drug_ndc d1_0 on f1_0.drug_id = d1_0.drug_id and d2_0.is_noc = true
left join core.fee_schedule_item_source f2_0
on f2_0.fee_schedule_item_source_id = f1_0.fee_schedule_item_source_id
where f1_0.fee_schedule_id=?
order by d1_0.ndc asc
offset ? rows fetch first ? rows only
I am expecting the same number of results from my query
I have the query below, and its sql code. It's running really slow, so it was re written in sql, now I'm just not sure how to convert the sql back to a lambda expression.
This is the part of the expression giving me the problems, somewhere in
r.RecordProducts.Any()
records = records
.Include(r => r.Employer)
.Include(r => r.Contractor)
.Include(r => r.RecordProducts)
.ThenInclude(rp => rp.ProductDefendant.Defendant)
.Where(r => EF.Functions.Like(r.Employer.DefendantCode, "%" + input.DefendantCode + "%")
|| EF.Functions.Like(r.Contractor.DefendantCode, "%" + input.DefendantCode + "%")
|| r.RecordProducts.Any(rp => EF.Functions.Like(rp.ProductDefendant.Defendant.DefendantCode, "%" + input.DefendantCode + "%") && rp.IsActive == true));
the any clause does an exist and some funky stuff in the sql where clause below
SELECT [t].[Id], [t].[StartDate], [t].[EndDate], [t].[WitnessName], [t].[SourceCode], [t].[JobsiteName], [t].[ShipName], [t].[EmployerCode]
FROM (
SELECT DISTINCT [r].[RecordID] AS [Id], [r].[StartDate], [r].[EndDate], [r.Witness].[FullName] AS [WitnessName], CASE
WHEN [r].[SourceID] IS NOT NULL
THEN [r.Source].[SourceCode] ELSE N'zzzzz'
END AS [SourceCode], CASE
WHEN [r].[JobsiteID] IS NOT NULL
THEN [r.Jobsite].[JobsiteName] ELSE N'zzzzz'
END AS [JobsiteName], CASE
WHEN [r].[ShipID] IS NOT NULL
THEN [r.Ship].[ShipName] ELSE N'zzzzz'
END AS [ShipName], CASE
WHEN [r].[EmployerID] IS NOT NULL
THEN [r.Employer].[DefendantCode] ELSE N'zzzzz'
END AS [EmployerCode]
FROM [Records] AS [r]
LEFT JOIN [Ships] AS [r.Ship] ON [r].[ShipID] = [r.Ship].[ShipID]
LEFT JOIN [Jobsites] AS [r.Jobsite] ON [r].[JobsiteID] = [r.Jobsite].[JobsiteID]
LEFT JOIN [Sources] AS [r.Source] ON [r].[SourceID] = [r.Source].[SourceID]
LEFT JOIN [Witnesses] AS [r.Witness] ON [r].[WitnessID] = [r.Witness].[WitnessID]
LEFT JOIN [Defendants] AS [r.Contractor] ON [r].[ContractorID] = [r.Contractor].[DefendantID]
LEFT JOIN [Defendants] AS [r.Employer] ON [r].[EmployerID] = [r.Employer].[DefendantID]
WHERE ([r].[IsActive] = 1) AND (([r.Employer].[DefendantCode] LIKE (N'%' + 'cert') + N'%' OR [r.Contractor].[DefendantCode] LIKE (N'%' + 'cert') + N'%') OR EXISTS (
SELECT 1
FROM [Records_Products] AS [rp]
INNER JOIN [Product_Defendant] AS [rp.ProductDefendant] ON [rp].[DefendantProductID] = [rp.ProductDefendant].[DefendantProductID]
INNER JOIN [Defendants] AS [rp.ProductDefendant.Defendant] ON [rp.ProductDefendant].[DefendantID] = [rp.ProductDefendant.Defendant].[DefendantID]
WHERE ([rp.ProductDefendant.Defendant].[DefendantCode] LIKE (N'%' + 'cert') + N'%' AND ([rp].[IsActive] = 1)) AND ([r].[RecordID] = [rp].[RecordID])))
) AS [t]
ORDER BY [t].[SourceCode]
OFFSET 0 ROWS FETCH NEXT 500 ROWS ONLY
Here is the new sql that works better, just not sure how to convert it back to a lambda expression
SELECT [t].[Id]
,[t].[StartDate]
,[t].[EndDate]
,[t].[WitnessName]
,[t].[SourceCode]
,[t].[JobsiteName]
,[t].[ShipName]
,[t].[EmployerCode]
FROM (
SELECT DISTINCT [r].[RecordID] AS [Id]
,[r].[StartDate]
,[r].[EndDate]
,[r.Witness].[FullName] AS [WitnessName]
,CASE
WHEN [r].[SourceID] IS NOT NULL
THEN [r.Source].[SourceCode]
ELSE N'zzzzz'
END AS [SourceCode]
,CASE
WHEN [r].[JobsiteID] IS NOT NULL
THEN [r.Jobsite].[JobsiteName]
ELSE N'zzzzz'
END AS [JobsiteName]
,CASE
WHEN [r].[ShipID] IS NOT NULL
THEN [r.Ship].[ShipName]
ELSE N'zzzzz'
END AS [ShipName]
,CASE
WHEN [r].[EmployerID] IS NOT NULL
THEN [r.Employer].[DefendantCode]
ELSE N'zzzzz'
END AS [EmployerCode]
FROM [Records] AS [r]
LEFT JOIN [Ships] AS [r.Ship] ON [r].[ShipID] = [r.Ship].[ShipID]
LEFT JOIN [Jobsites] AS [r.Jobsite] ON [r].[JobsiteID] = [r.Jobsite].[JobsiteID]
LEFT JOIN [Sources] AS [r.Source] ON [r].[SourceID] = [r.Source].[SourceID]
LEFT JOIN [Witnesses] AS [r.Witness] ON [r].[WitnessID] = [r.Witness].[WitnessID]
LEFT JOIN [Defendants] AS [r.Contractor] ON [r].[ContractorID] = [r.Contractor].[DefendantID]
LEFT JOIN [Defendants] AS [r.Employer] ON [r].[EmployerID] = [r.Employer].[DefendantID]
LEFT JOIN (
SELECT [rp].[RecordID]
FROM [Records_Products] AS [rp]
INNER JOIN [Product_Defendant] AS [rp.ProductDefendant] ON [rp].[DefendantProductID] = [rp.ProductDefendant].[DefendantProductID]
INNER JOIN [Defendants] AS [rp.ProductDefendant.Defendant] ON [rp.ProductDefendant].[DefendantID] = [rp.ProductDefendant.Defendant].[DefendantID]
WHERE (
[rp.ProductDefendant.Defendant].[DefendantCode] LIKE (N'%' + 'cert') + N'%'
AND ([rp].[IsActive] = 1)
)
) AS RecordProduct ON [r].[RecordID] = RecordProduct.[RecordID]
WHERE ([r].[IsActive] = 1)
AND (
(
[r.Employer].[DefendantCode] LIKE (N'%' + 'cert') + N'%'
OR [r.Contractor].[DefendantCode] LIKE (N'%' + 'cert') + N'%'
)
OR RecordProduct.RecordID IS NOT NULL --OR EXISTS ( -- SELECT 1 -- FROM [Records_Products] AS [rp] -- INNER JOIN [Product_Defendant] AS [rp.ProductDefendant] ON [rp].[DefendantProductID] = [rp.ProductDefendant].[DefendantProductID] -- INNER JOIN [Defendants] AS [rp.ProductDefendant.Defendant] ON [rp.ProductDefendant].[DefendantID] = [rp.ProductDefendant.Defendant].[DefendantID] -- WHERE ([rp.ProductDefendant.Defendant].[DefendantCode] LIKE (N'%' + 'cert') + N'%' -- AND ([rp].[IsActive] = 1)) AND ([r].[RecordID] = [rp].[RecordID]) -- ) )) AS [t]ORDER BY [t].[SourceCode]OFFSET 0 ROWS FETCH NEXT 500 ROWS ONLY
)
)
The linq expression you supplied and the SQL generated do not match. For one, the linq expression is performing an Include on the various related tables which would have included all of those entity columns in the top-level SELECT which are not present in your example SQL. I also don't see conditions in the Linq expression for the Take 500 & OrderBy, or IsActive assertion on Record.
To be able to help determine the source of any performance concern we need to see the complete Linq expression and the resulting SQL.
Looking at the basis of the Linq expression you provided:
records = records
.Include(r => r.Employer)
.Include(r => r.Contractor)
.Include(r => r.RecordProducts)
.ThenInclude(rp => rp.ProductDefendant.Defendant)
.Where(r => EF.Functions.Like(r.Employer.DefendantCode, "%" + input.DefendantCode + "%")
|| EF.Functions.Like(r.Contractor.DefendantCode, "%" + input.DefendantCode + "%")
|| r.RecordProducts.Any(rp => EF.Functions.Like(rp.ProductDefendant.Defendant.DefendantCode, "%" + input.DefendantCode + "%") && rp.IsActive == true));
There are a few suggestions I can make:
There is no need for the Functions.Like. You should be able to achieve the same with Contains.
Avoid using Include and instead utilize Select to retrieve the columns from the resulting structure that you actually need. Populate these into ViewModels or consume them in the code. The less data you pull back, the better optimized the SQL can be for indexing, and the less data pulled across the wire. Consuming entities also leads to unexpected lazy-load scenarios as systems mature and someone forgets to Include a new relation.
.
records = records
.Where(r => r.IsActive
&& (r.Employer.DefendantCode.Contains(input.DefendantCode)
|| r.Contractor.DefendantCode.Contains(input.DefendantCode)
|| r.RecordProducts.Any(rp => rp.IsActive
&& rp.ProductDefendant.Defendant.DefendantCode.Contains(input.DefendantCode))
.OrderBy(r => r.SourceCode)
.Select(r => new RecordViewModel
{
// Populate the data you want here.
}).Take(500).ToList();
This also adds the IsActive check, OrderBy, and Take(500) based on your sample SQL.
how do you query raw SQL on rails?
So I have this raw sql that I need to run on rails but giving me a syntax error. I also escape the extra parenthesis but still got a syntax error near the first inner join.
here's my code:
Spree::CorporateAccount.joins(" (((((
( inner join spree_memberships on spree_corporate_accounts.id = spree_memberships.corporate_account_id)
inner join spree_users on spree_memberships.user_id = spree_users.id)
left join spree_variant_price_sets on spree_corporate_accounts.variant_price_set_id = spree_variant_price_sets.id)
left join spree_addresses on spree_corporate_accounts.bill_address_id = spree_addresses.id)
left join spree_states on spree_addresses.state_id = spree_states.id)
left join spree_countries on spree_addresses.country_id = spree_countries.id)
left join spree_partner_accounts on spree_corporate_accounts.id = spree_partner_accounts.partnerable_id
").where(" spree_memberships.deleted_at IS null
AND (spree_partner_accounts.partnerable_type = 'Spree::CorporateAccount' OR spree_partner_accounts.partnerable_type IS NULL)
AND admin = true
")
but on sql this is perfectly fine.
SELECT
spree_corporate_accounts.id,
spree_corporate_accounts.company_name,
spree_memberships.ADMIN,
spree_users.email,
spree_users.doctor AS name,
spree_partner_accounts.account_key,
CASE spree_corporate_accounts.billing_type when 1 THEN 'Postbill' WHEN 2 THEN 'Creditcard' ELSE 'Individual'END,
spree_variant_price_sets.name AS priceset,
spree_addresses.address1,
spree_addresses.address2,
spree_addresses.city,
spree_addresses.zipcode,
spree_states.name AS state,
spree_countries.name AS country,
spree_addresses.phone,
spree_users.created_at
from (((((( spree_corporate_accounts inner join spree_memberships on spree_corporate_accounts.id = spree_memberships.corporate_account_id)
inner join spree_users on spree_memberships.user_id = spree_users.id)
left join spree_variant_price_sets on spree_corporate_accounts.variant_price_set_id = spree_variant_price_sets.id)
left join spree_addresses on spree_corporate_accounts.bill_address_id = spree_addresses.id)
left join spree_states on spree_addresses.state_id = spree_states.id)
left join spree_countries on spree_addresses.country_id = spree_countries.id)
left join spree_partner_accounts on spree_corporate_accounts.id = spree_partner_accounts.partnerable_id
where spree_memberships.deleted_at IS null
and (spree_partner_accounts.partnerable_type = 'Spree::CorporateAccount' OR spree_partner_accounts.partnerable_type IS NULL )
AND admin = true
if I do this. It will yield a different result. so what im thinking is the parenthesis evaluate the result first and then go to the next.
Spree::CorporateAccount.joins("inner join spree_memberships on spree_corporate_accounts.id = spree_memberships.corporate_account_id")
.joins("inner join spree_users on spree_memberships.user_id = spree_users.id")
.joins("left join spree_variant_price_sets on spree_corporate_accounts.variant_price_set_id = spree_variant_price_sets.id")
.joins("left join spree_addresses on spree_corporate_accounts.bill_address_id = spree_addresses.id")
.joins("left join spree_states on spree_addresses.state_id = spree_states.id")
.joins("left join spree_countries on spree_addresses.country_id = spree_countries.id")
.joins("left join spree_partner_accounts on spree_corporate_accounts.id = spree_partner_accounts.partnerable_id ")
.where("spree_memberships.deleted_at IS null
AND spree_partner_accounts.partnerable_type = 'Spree::CorporateAccount' OR spree_partner_accounts.partnerable_type IS NULL
AND admin = true
")
I have multiple dataframes I need to concat the addresses and zip based condition.Actually I had sql query which i need to convert to dataframe join
I had written UDF which is working fine for concating multiple columns to obtain a single column,
val getConcatenated = udf( (first: String, second: String,third: String,fourth: String,five: String,six: String) => { first + "," + second + "," +third + "," +fourth + "," +five + "," +six } )
MySQl Query
select
CONCAT(al.Address1,',',al.Address2,',',al.Zip) AS AtAddress,
CONCAT(rl.Address1,',',rl.Address2,',',rl.Zip) AS RtAddress,
CONCAT(d.Address1,',',d.Address2,','d.Zip) AS DAddress,
CONCAT(s.Address1,',',s.Address2,',',s.Zip) AS SAGddress,
CONCAT(vl.Address1,',',vl.Address2,',vl.Zip) AS VAddress,
CONCAT(sg.Address1,',',sg.Address2,',sg.Zip) AS SAGGddress
FROM
si s inner join
at a on s.cid = a.cid and s.cid =a.cid
inner join De d on s.cid = d.cid AND d.aid = a.aid
inner join SGrpM sgm on s.cid = sgm.cid and s.sid =sgm.sid and sgm.status=1
inner join SeG sg on sgm.cid =sg.cid and sgm.gid =sg.gid
inner join bd bu on s.cid = bu.cid and s.sid =bu.sid
inner join locas al on a.ALId = al.lid
inner join locas rl on a.RLId = rl.lid
inner join locas vl on a.VLId = vl.lid
I am facing issue when joining the dataframes which gives me null value.
val DS = DS_SI.join(at,Seq("cid","sid"),"inner").join(DS_DE,Seq("cid","aid"),"inner") .join(DS_SGrpM,Seq("cid","sid"),"inner").join(DS_SG,Seq("cid","gid"),"inner") .join(at,Seq("cid","sid"),"inner")
.join(DS_BD,Seq("cid","sid"),"inner").join(DS_LOCAS("ALId") <=> DS_LOCATION("lid") && at("RLId") <=> DS_LOCAS("lid")&& at("VLId") <=> DS_LOCAS("lid"),"inner")
Iam trying to join my dataFrames like above which is not giving be proper results and then I want to concat by adding the column
.withColumn("AtAddress",getConcatenated())
.withColumn("RtAddress",getConcatenated())....
Any one tell me how effectively we can achieve this and am I joining the dataframes correctly or any better approach for this .....
You can use concat_ws(separator, columns_to_concat).
Example:
import org.apache.spark.sql.functions._
df.withColumn("title", concat_ws(", ", DS_DE("Address2"), DS_DE("Address2"), DS_DE("Zip")))
how do when I'm using fetch? "with" does not work.
ISession session = NHibernateHelper.GetSession();
string sql = "FROM Filial fi " +
"left join fetch fi.LstUsuario usr " +
"with usr.NumSequencial = :idUsr " +
"order by fi.Nome";
IQuery query = session.CreateQuery(sql);
query.SetParameter("idUsr", usuario.NumSequencial, NHibernateUtil.Int64);
lstFilial = (List<Filial>)query.List<Filial>();
Error message: unexpected token: with [FROM Entidade.Filial fi left join fetch fi.LstUsuario usr with usr.NumSequencial = :idUsr order by fi.Nome]
Code in SQL
select *
from TFILIAL fi
left join TPERMIS_USR_FILIAL usrFi
on fi.anum_sequ_filial = usrfi.anum_sequ_filial
and usrfi.anum_sequ_usuario = 2;
Thank you in advance.