how can merge result of this 3 line
var newscatid=Dbcontext.tbl_NewsPosition.where(x => x.Fk_NewsID==4 and IsMainPosition=true).select(x => x.Fk_NewsCatId);
from p in Dbcontext.tbl_cat.where(x => x.Id== newscatid) select new { parentCat = b.CatName};
from ch in Dbcontext.tbl_cat.where(x => x.Fk_ParentId== newscatid) select new { childCat = ch.CatName};
This is what I’m trying to obtain:
+-----------+----------+
| parentCat | childCat |
+-----------+----------+
| Sport | Footbal |
| | |
+-----------+----------+
and these are my tables:
Try this one:
var result = from p in Dbcontext.tbl_cat
join ch in Dbcontext.tbl_cat on p.Id equals ch.Fk_ParentId
join np in Dbcontext.tbl_NewsPosition on p.Id equals np.Fk_NewsCatId
where np.Fk_NewsID==4 && np.IsMainPosition
select new { parentCat = p.CatName, childCat = ch.CatName };
Please refer below code:
var newscatid=Dbcontext.tbl_NewsPosition.where(x => x.Fk_NewsID==4 and
IsMainPosition=true).select(x => x.Fk_NewsCatId);'
var data =
from p in Dbcontext.tbl_cat
join ch in Dbcontext.tbl_cat on p.Id equals ch.Fk_ParentId
where p.Id==newscatid
select new
{
parentCat = p.CatName ,
childCat = ch.CatName
};
Related
I have an UuidConditionSet, when the if condition is wrong, I want apply an empty string to my select statement(or just ignore this UuidConditionSet), but I got this error. How to solve this problem?
mismatched input 'FROM' expecting <EOF>(line 10, pos 3)
This is the select
(SELECT
item,
amount,
date
from my_table
where record_type = 'myType'
and ( date_format(date, "yyyy-MM-dd") >= '2020-02-27'
and date_format(date, "yyyy-MM-dd") <= '2020-02-28' )
and ()
var UuidConditionSet = ""
var UuidCondition = Seq.empty[String]
if(!UuidList.mkString.isEmpty) {
UuidCondition = for {
Uuid <- UuidList
UuidConditionSet = s"${SQLColumnHelper.EVENT_INFO_STRUCT_NAME}.${SQLColumnHelper.UUID} = '".concat(eventUuid).concat("'")
} yield UuidConditionSet
UuidConditionSet = UuidCondition.reduce(_.concat(" or ").concat(_))
}
s"""SELECT
| ${SQLColumnHelper.STRUCT_NAME_ITEM},
| ${SQLColumnHelper.STRUCT_NAME_AMOUNT},
| ${SQLColumnHelper.DATE}
| from ${sqlTableHelper.TABLE}
| where ${SQLColumnHelper.EVENT_INFO_STRUCT_NAME} = '${RECORD_TYPE}'
| and ( date_format(${SQLColumnHelper.DATE}, "${Constant.STAY_DATE_FORMAT}") >= '${stayDateRangeTuple._1}'
| and date_format(${SQLColumnHelper.DATE}, "${Constant.STAY_DATE_FORMAT}") <= '${stayDateRangeTuple._2}' )
| and ($UuidConditionSet)
You can use pattern matching on the list UuidList to check the size and return an empty string if the list is empty. Also, you can use IN instead of multiple ORs here.
Try this:
val UuidCondition = UuidList match {
case l if (l.size > 0) => {
l.map(u => s"'$u'").mkString(
s"and ${SQLColumnHelper.EVENT_INFO_STRUCT_NAME}.${SQLColumnHelper.UUID} in (",
",",
")"
)
}
case _ => ""
}
s"""SELECT
| ${SQLColumnHelper.STRUCT_NAME_ITEM},
| ${SQLColumnHelper.STRUCT_NAME_AMOUNT},
| ${SQLColumnHelper.DATE}
| from ${sqlTableHelper.TABLE}
| where ${SQLColumnHelper.EVENT_INFO_STRUCT_NAME} = '${RECORD_TYPE}'
| and date_format(${SQLColumnHelper.DATE}, "${Constant.STAY_DATE_FORMAT}") >= '${stayDateRangeTuple._1}'
| and date_format(${SQLColumnHelper.DATE}, "${Constant.STAY_DATE_FORMAT}") <= '${stayDateRangeTuple._2}'
| $UuidCondition
"""
Schema of my json is next
and field Reports_Rows_Rows_Cells if it not null looks like next:
[Row(Attributes=[Row(Id='account', Value='bd9e85e0-0478-433d-ae9f-0b3c4f04bfe4')], Value='Business Bank Account'),
Row(Attributes=[Row(Id='account', Value='bd9e85e0-0478-433d-ae9f-0b3c4f04bfe4')], Value='10105.54'),
Row(Attributes=[Row(Id='account', Value='bd9e85e0-0478-433d-ae9f-0b3c4f04bfe4')], Value='4938.48')]
What I want is to create table which has all above columns and column Reports_Rows_Rows_Cells should look like
-------- |-------|Reports_Rows_Rows_Cells_Value | Reports_Rows_Rows_Cells_Value | Reports_Rows_Rows_Cells_Value|
| Business Bank Account |10105.54 | 4938.48
Not after parsing json my table look like next:
-------- |-------|Reports_Rows_Rows_Cells_Value|
| Business Bank Account |
| 10105.54 |
| 4938.48 |
My code which I use to parse json
def flatten_df(nested_df):
# for ncol in nested_df.columns:
array_cols = [c[0] for c in nested_df.dtypes if c[1][:5] == 'array']
for col in array_cols:
nested_df = nested_df.withColumn(col, explode_outer(nested_df[col]))
nested_cols = [c[0] for c in nested_df.dtypes if c[1][:6] == 'struct']
if len(nested_cols) == 0:
return nested_df
flat_cols = [c[0] for c in nested_df.dtypes if c[1][:6] != 'struct']
flat_df = nested_df.select(flat_cols +
[F.col(nc+'.'+c).alias(nc+'_'+c)
for nc in nested_cols
for c in nested_df.select(nc+'.*').columns])
return flatten_df(flat_df)
I have following Data:
+-----+-----+----+
|Col1 |t0 |t1 |
+-----+-----+----+
| A |null |20 |
| A |20 |40 |
| B |null |10 |
| B |10 |20 |
| B |20 |120 |
| B |120 |140 |
| B |140 |320 |
| B |320 |340 |
| B |340 |360 |
+-----+-----+----+
And what I want is something like this:
+-----+-----+----+----+
|Col1 |t0 |t1 |grp |
+-----+-----+----+----+
| A |null |20 |1A |
| A |20 |40 |1A |
| B |null |10 |1B |
| B |10 |20 |1B |
| B |20 |120 |2B |
| B |120 |140 |2B |
| B |140 |320 |3B |
| B |320 |340 |3B |
| B |340 |360 |3B |
+-----+-----+----+----+
Explanation:
The extra column is based on the Col1 and the difference between t1 and t0.
When the difference between that two is too high => a new number is generated. (in the dataset above when the difference is greater than 50)
I build t0 with:
val windowSpec = Window.partitionBy($"Col1").orderBy("t1")
df = df.withColumn("t0", lag("t1", 1) over windowSpec)
Can someone help me how to do it?
I searched but didn't get a good idea.
I'm a little bit lost because I need the value of the previous calculated row of grp...
Thanks
I solved it myself
val grp = (coalesce(
($"t" - lag($"t", 1).over(windowSpec)),
lit(0)
) > 50).cast("bigint")
df = df.withColumn("grp", sum(grp).over(windowSpec))
With this I don't need both colums (t0 and t1) anymore but can use only t1 (or t) without compute t0.
(I only need to add the value of Col1 but the most important part the number is done and works fine.)
I got the solution from:
Spark SQL window function with complex condition
thanks for your help
You can use udf function to generate the grp column
def testUdf = udf((col1: String, t0: Int, t1: Int)=> (t1-t0) match {
case x : Int if(x > 50) => 2+col1
case _ => 1+col1
})
Call the udf function as
df.withColumn("grp", testUdf($"Col1", $"t0", $"t1"))
The udf function above won't work properly due to null values in t0 which can be replaced by 0
df.na.fill(0)
I hope this is the answer you are searching for.
Edited
Here's the complete solution using udaf . The process is complex . You've already got easy answer but it might help somebody who might use it
First defining udaf
class Boendal extends UserDefinedAggregateFunction {
def inputSchema = new StructType().add("Col1", StringType).add("t0", IntegerType).add("t1", IntegerType).add("rank", IntegerType)
def bufferSchema = new StructType().add("buff", StringType).add("buffer1", IntegerType)
def dataType = StringType
def deterministic = true
def initialize(buffer: MutableAggregationBuffer) = {
buffer.update(0, "")
buffer.update(1, 0)
}
def update(buffer: MutableAggregationBuffer, input: Row) = {
if (!input.isNullAt(0)) {
val buff = buffer.getString(0)
val col1 = input.getString(0)
val t0 = input.getInt(1)
val t1 = input.getInt(2)
val rank = input.getInt(3)
var value = 1
if((t1-t0) < 50)
value = 1
else
value = (t1-t0)/50
val lastValue = buffer(1).asInstanceOf[Integer]
// if(!buff.isEmpty) {
if (value < lastValue)
value = lastValue
// }
buffer.update(1, value)
var finalString = ""
if(buff.isEmpty){
finalString = rank+";"+value+col1
}
else
finalString = buff+"::"+rank+";"+value+col1
buffer.update(0, finalString)
}
}
def merge(buffer1: MutableAggregationBuffer, buffer2: Row) = {
val buff1 = buffer1.getString(0)
val buff2 = buffer2.getString(0)
buffer1.update(0, buff1+buff2)
}
def evaluate(buffer: Row) : String = {
buffer.getString(0)
}
}
Then some udfs
def rankUdf = udf((grp: String)=> grp.split(";")(0))
def removeRankUdf = udf((grp: String) => grp.split(";")(1))
And finally call the udaf and udfs
val windowSpec = Window.partitionBy($"Col1").orderBy($"t1")
df = df.withColumn("t0", lag("t1", 1) over windowSpec)
.withColumn("rank", rank() over windowSpec)
df = df.na.fill(0)
val boendal = new Boendal
val df2 = df.groupBy("Col1").agg(boendal($"Col1", $"t0", $"t1", $"rank").as("grp2")).withColumnRenamed("Col1", "Col2")
.withColumn("grp2", explode(split($"grp2", "::")))
.withColumn("rank2", rankUdf($"grp2"))
.withColumn("grp2", removeRankUdf($"grp2"))
df = df.join(df2, df("Col1") === df2("Col2") && df("rank") === df2("rank2"))
.drop("Col2", "rank", "rank2")
df.show(false)
Hope it helps
Here is the entity and table structure-
class Person { int PersonId, string PersonName}
class Report { int ReportId, datetime ReportTime, int PersonId }
Table: Persons
----------------------------
| PersonId | PersonName |
----------------------------
| 1 | Abc |
----------------------------
| 2 | Xyz |
----------------------------
Table: Reports
----------------------------------------------
| ReportId | ReportTime | PersonId |
----------------------------------------------
| 10 | 2017-02-27 11:12 | 1 |
---------------------------- -----------------
| 14 | 2017-02-27 15:23 | 1 |
---------------------------- -----------------
I want to select data as follows (PersonName from Person and last record of his Id in reports table)-
-------------------------------------
| PersonName | ReportTime |
-------------------------------------
| Abc | 2017-02-27 15:23 |
-------------------------------------
How can i do it in Lambda or LINQ?
Use Queryable.GroupJoin:
from p in db.Persons
join r in db.Reports on p.PersonId equals r.PersonId into g
where g.Any() // if some persons do not have reports
select new {
p.PersonName,
ReportTime = g.Max(r => r.ReportTime)
}
Lambda (note that it will return Nullable ReportTime with nulls for persons which don't have any reports)
db.Persons.GroupJoin(
db.Reports,
p => p.PersonId,
r => r.PersonId,
(p,g) => new { p.PersonName, ReportTime = g.Max(r => (DateTime?)r.ReportTime) })
Try this:
List<Person> people = new List<Person>
{
new Person {PersonId = 1, PersonName = "AB" },
new Person {PersonId = 2, PersonName = "CD" },
new Person {PersonId = 3, PersonName = "EF" },
};
List<Report> reports = new List<Report>()
{
new Report {ReportId = 1, ReportTime = DateTime.Now, PersonId = 1 },
new Report {ReportId = 2, ReportTime = DateTime.Now.AddHours(-1), PersonId = 1 },
new Report {ReportId = 3, ReportTime = DateTime.Now.AddHours(-2), PersonId = 1 },
new Report {ReportId = 4, ReportTime = DateTime.Now.AddMinutes(-3), PersonId = 2 },
new Report {ReportId = 5, ReportTime = DateTime.Now, PersonId = 2 }
};
var res = (from rep in reports
group rep by rep.PersonId into repGrp
join ppl in people on repGrp.FirstOrDefault().PersonId equals ppl.PersonId
select new
{
PersonName = ppl.PersonName,
ReportDate = repGrp.Max(r => r.ReportTime),
}).ToList();
}
I currently get problem to dynamic linq expression below
My Models
public class Orders
{
public int OrderId ;
public ICollection<OrderStatuses> Statuses;
}
public class Statuses
{
public int StatusId;
public int OrderId;
public string Note;
public DateTime Created;
}
My Sample data :
Orders
| ID | Name |
----------------------
| 1 | Order 01 |
| 2 | Order 02 |
| 3 | Order 03 |
Statuses
|ID | OrderId | Note | Created |
---------------------------------------
| 1 | 1 | Ordered | 2016-03-01|
| 2 | 1 | Pending | 2016-04-02|
| 3 | 1 | Completed | 2016-05-19|
| 4 | 1 | Ordered | 2015-05-19|
| 5 | 2 | Ordered | 2016-05-20|
| 6 | 2 | Completed | 2016-05-19|
| 7 | 3 | Completed | 2016-05-19|
I'd like to get number of orders which have note value equal to 'Ordered' and max created time.
Below is sample number of orders that I expect from query
| Name | Note | Last Created|
-------------------------------------|
| Order 01 | Ordered | 2016-03-01 |
| Order 02 | Ordered | 2016-05-20 |
Here my idea but it's seem to wrong way
var outer = PredicateBuilder.True<Order>();
var orders = _entities.Orders
.GroupBy(x => x.OrderId)
.Select(x => new { x.Key, Created = x.Max(g => g.Created) })
.ToArray();
var predicateStatuses = PredicateBuilder.False<Status>();
foreach (var item in orders)
{
predicateStatuses = predicateStatuses.Or(x => x.OrderId == item.Key && x.Created == item.Created);
}
var predicateOrders = PredicateBuilder.False<JobOrder>();
predicateOrders = predicateOrders.Or(predicateStatuses); (I don't how to passed expression which different object type (Order and Status) here or I have to write an extension method or something
outer = outer.And(predicateOrders);
Please suggest me how to solve this dynamic linq expression in this case.
Thanks in advance.
There's nothing dynamic about your query, at least, it doesn't need to be. You can express it as a regular query.
var query =
from o in db.Orders
join s in db.Statuses on o.Id equals s.OrderId
where s.Note == "Ordered"
orderby s.Created descending
group new { o.Name, s.Note, LastCreated = s.Created } by o.Id into g
select g.First();
p.s., your models doesn't seem to match the data at all so I'm ignoring that. Adjust as necessary.
Thanks so much for #Jeff Mercado answer. Finally, I customized your answer to solve my problem below:
var predicateStatuses = PredicateBuilder.False<Order>();
predicateStatuses = predicateStatuses.Or(p => (
from j in db.Statuses
where j.OrderId == p.ID
group j by j.OrderId into g
select g.OrderByDescending(t=>t.Created)
.FirstOrDefault()
).FirstOrDefault().Note == 'Ordered'
);