How EXACTLY is Slick's SimpleLiteral used? - scala

I want to use some extra features of PostgreSQL in my code but I don't want to fill the place with SQL string interpolations.
Currently I have:
/** Use 'now()' through Slick. */
val psqlNow = SimpleFunction.nullary[java.sql.Date]("now")
//Not really my code, but we only care for 2 lines.
def aQuery(limiter: Column[Int]) = {
myTable
.filter(_.validFrom >= psqlNow)
.filter(_.validUntil <= psqlNow)
.filter(_.fakeId === limiter).map(e => (e.fakeId, e.name)
}
But I want to use 'CURRENT_DATE', which I is a literal (and using it in place of "now" throws an exception). Can someone provide an actual example, because I can't get this to compile:
/** Use 'CURRENT_DATE' through Slick. */
val psqlNow = SimpleLiteral("CURRENT_DATE")(...WHAT GOES HERE?...)
//Not really my code, but we only care for 2 lines.
def aQuery(limiter: Column[Int]) = {
myTable
.filter(_.validFrom >= psqlNow)
.filter(_.validUntil <= psqlNow)
.filter(_.fakeId === limiter).map(e => (e.fakeId, e.name)
}
And I also want to change the following to lifted Slick, can I do it with SimpleLiteral (to somehow put 'count(*) OVER() recordsFiltered' into the generated query?
SELECT *, count(*) OVER() recordsFiltered FROM example
WHERE id = $1
The examples are trivial, the actual code is a series of folds over filtering criteria.

import scala.slick.ast.TypedType
val current_date = Column.forNode[java.sql.Date](new SimpleLiteral("CURRENT_DATE")(implicitly[TypedType[java.sql.Date]]))
does the trick. Better support is missing at the moment.
I added a PR, so in Slick 2.2 it will be supported like this:
val current_date = SimpleLiteral[java.sql.Date]("CURRENT_DATE")
See https://github.com/slick/slick/pull/981

Related

Exclude null columns in an update statement - JOOQ

I have a POJO that has the fields that can be updated. But sometimes only a few fields will need to be updated and the rest are null. How do I write an update statement that ignores the fields that are null? Would it be better to loop through the non missing ones and dynamically add to a set statement, or using coalesce?
I have the following query:
jooqService.using(txn)
.update(USER_DETAILS)
.set(USER_DETAILS.NAME, input.name)
.set(USER_DETAILS.LAST_NAME, input.lastName)
.set(USER_DETAILS.COURSES, input.courses)
.set(USER_DETAILS.SCHOOL, input.school)
.where(USER_DETAILS.ID.eq(input.id))
.execute()
If there is a better practice?
I don't know Jooq but it looks like you could simply do this:
val jooq = jooqService.using(txn).update(USER_DETAILS)
input.name.let {jooq.set(USER_DETAILS.NAME, it)}
input.lastName.let {jooq.set(USER_DETAILS.LAST_NAME, it)}
etc...
EDIT: Mapping these fields explicitly as above is clearest in my opinion, but you could do something like this:
val fields = new Object[] {USER_DETAILS.NAME, USER_DETAILS.LAST_NAME}
val values = new Object[] {input.name, input.lastName}
val jooq = jooqService.using(txn).update(USER_DETAILS)
values.forEachIndexed { i, value ->
value.let {jooq.set(fields[i], value)}
}
You'd still need to enumerate all the fields and values explicitly and consistently in the arrays for this to work. It seems less readable and more error prone to me.
In Java, it would be somthing like this
var jooqQuery = jooqService.using(txn)
.update(USER_DETAILS);
if (input.name != null) {
jooqQuery.set(USER_DETAILS.NAME, input.name);
}
if (input.lastName != null) {
jooqQuery.set(USER_DETAILS.LAST_NAME, input.lastName);
}
// ...
jooqQuery.where(USER_DETAILS.ID.eq(input.id))
.execute();
Another option rather than writing this UPDATE statement is to use UpdatableRecord:
// Load a POJO into a record using a RecordUnmapper
UserDetailsRecord r =
jooqService.using(txn)
.newRecord(USER_DETAILS, input)
(0 .. r.size() - 1).forEach { if (r[it] == null) r.changed(it, false) }
r.update();
You can probably write an extension function to make this available for all jOOQ records, globally, e.g. as r.updateNonNulls().

LINQ to SQL compare nullable in subquery

I fail to translate a sql query to a linq query that could calculate some stock.
This is my test query that I'm trying to convert to a linq query.
SELECT
i.*,
(SELECT COUNT(t.*) FROM tickets t
WHERE t.starttime::time = i.sessionstarttime::time
AND t.starttime::date = '2018-04-06'::date)
as stock
FROM items I
-- note that the hardcoded date ('2018-04-06') is a function parameter
( tl;dr; how would you convert this PostgreSQL query to LINQ? )
My attempts so far are the variations of the following query:
var items = await _context.Items.Select(x => new Item
{
Id = x.Id,
IsTicket = x.IsTicket,
Name = x.Name,
Price = x.Price,
SaleItems = x.SaleItems,
SessionStartTime = x.SessionStartTime,
DateCreated = x.DateCreated,
DateEdit = x.DateEdit,
UserIdCreated = x.UserIdCreated,
UserIdEdited = x.UserIdEdited,
// calculate stock in subquery
Stock = _context.Tickets.Count(
t => t.StartTime.Date == ticketDate
&& x.SessionStartTime.HasValue
&& t.StartTime.Hour == x.SessionStartTime.Value.Hours // this is the part that is failing
&& t.State != TicketState.Canceled)
}).ToListAsync();
t.StartTime is Datetime and x.SessionStartTime is Nullable Timespan
So when I comment the line && t.StartTime.Hour == x.SessionStartTime.Value.Hours everything is fine, but with it I get warnings that it could not be translated and will be evaluated locally. But I don't want to download the whole ticket table just to count them.
The t.StartTime.Hour part is fine, I tried to perform static comparisons with both parameters. t.StartTime.Hour == 5 was translated without any problems, but x.SessionStartTime.Value.Hours == 5 failed to translate.
Also the problematic part in the application output:
([t].StartTime.Hour == Convert([x].SessionStartTime, TimeSpan).Hours))
So I guess that convert part is failing.
So what I'm missing and how I could work around this problem. Any help will be appreciated.
Update:
After experimenting a bit I have found two workarounds, that I wouldn't call the answers.
First I noticed that EF is trying to convert Nullable<TimeSpan> to a regular TimeSpan from the mentioned output: ([t].StartTime.Hour == Convert([x].SessionStartTime, TimeSpan).Hours))
I thought I could prevent that conversion by converting to a string and comparing the strings (I have a feeling this will bite me in the future):
t.StartTime.ToString().Contains(x.SessionStartTime.ToString())
The second workaround is only viable for my scenario since I know the items query is final and I can materialise it without calculated Stock, and then loop through the results and calculate it on a separate query. But this seems to add additional calls to the database and sacrifice some performance.
foreach(var x in items.Where(x=>x.SessionStartTime.HasValue))
{
// accessing the t.StartTime.TimeOfDay property seems to fail the LINQ to SQL as well
var hours = x.SessionStartTime.Value.Hours;
var minutes = x.SessionStartTime.Value.Minutes;
x.Stock = _context.Tickets.Count(t => t.StartTime.Date == ticketDate
&& t.StartTime.Hour == hours
&& t.StartTime.Minute == minutes);
}

Anorm String Interpolation not replacing variables

We are using Scala Play, and I am trying to ensure that all SQL queries are using Anorm's String Interpolation. It works with some queries, but many are not actually replacing the variables before the query is executing.
import anorm.SQL
import anorm.SqlStringInterpolation
object SecureFile
{
val table = "secure_file"
val pk = "secure_file_idx"
...
// This method works exactly as I would hope
def insert(secureFile: SecureFile): Option[Long] = {
DBExec { implicit connection =>
SQL"""
INSERT INTO secure_file (
subscriber_idx,
mime_type,
file_size_bytes,
portal_msg_idx
) VALUES (
${secureFile.subscriberIdx},
${secureFile.mimeType},
${secureFile.fileSizeBytes},
${secureFile.portalMsgIdx}
)
""" executeInsert()
}
}
def delete(secureFileIdx: Long): Int = {
DBExec { implicit connection =>
// Prints correct values
println(s"table: ${table} pk: ${pk} secureFileIdx: ${secureFileIdx} ")
// Does not work
SQL"""
DELETE FROM $table WHERE ${pk} = ${secureFileIdx}
""".executeUpdate()
// Works, but unsafe
val query = s"DELETE FROM ${table} WHERE ${pk} = ${secureFileIdx}"
SQL(query).executeUpdate()
}
}
....
}
Over in the PostgreSQL logs, it's clear that the delete statement has not acquired the correct values:
2015-01-09 17:23:03 MST ERROR: syntax error at or near "$1" at character 23
2015-01-09 17:23:03 MST STATEMENT: DELETE FROM $1 WHERE $2 = $3
2015-01-09 17:23:03 MST LOG: execute S_1: ROLLBACK
I've tried many variations of execute, executeUpdate, and executeQuery with similar results. For the moment, we are using basic string replacement but of course this is bad because it's not using PreparedStatements.
For anyone else sitting on this page scratching their head and wondering what they might be missing...
SQL("select * from mytable where id = $id")
is NOT the same as
SQL"select * from mytable where id = $id"
The former does not do String interpolation whereas the latter does.
This is easily overlooked in the aforementioned docs as all the samples provided just happen to have a (non-related) closing parenthesis on them (like this sentence does)
Anorm String interpolation was introduced to pass parameter (e.g. SQL"Select * From Test Where id = $x), with interpolation arguments (e.g. $x) set on underlying PreparedStament according proper type conversion (see use cases on https://www.playframework.com/documentation/2.3.x/ScalaAnorm ).
Next Anorm release will also have the #$foo syntax to mix interpolation for parameter with standard string interpolation. This will allow to write DELETE FROM #$table WHERE #${pk} = ${secureFileIdx} and having it executed as DELETE FROM foo WHERE bar = ? (if literal table is "foo" and pk is "bar"), with literal secureFileIdx passed as parameter. See related pull request.
Until next revision is release, you can build Anorm from its master sources ti benefit from this change.

Dynamic SQL Query based on values selected in dropdown/radiobutton

I have a controller in a simple grails application which looks like the following:
The where condition here is a static one. I want the filters in where condition(Hourly,Afiiliates)to be dyanamic based on selection made using radio buttons(for timeperiod) and using dropdown (for mv)
class Tablev1classController {
def dataSource
def listJson = {
def sql = new Sql(dataSource)
def rows = sql.rows("select date_hour, total_revenue as sales, visits, marketing, organic,single_page_visits,total_units,orders,total_revenue_ly as sales_ly, visits_ly,marketing_ly, organic_ly,total_units_ly,orders_ly,single_page_visits_ly from xyz.mu_ewacs_marketing_vehicle_tylylw where time_period = 'HOURLY' && mv = 'AFFILIATES'")
sql.close()
render rows as JSON
}
Any help would be appreciated.
What you're probably looking for are named or ordinal parameters. See Sql javadoc, chapters "Avoiding SQL injection" and "Named and named ordinal parameters".
Using ordinal parameters it could be done like this:
def rows = sql.rows("select ... from xyz.whatever
where time_period = ? && mv = ?", [timePeriod, mv])

Linq to Entities does not recognize the method System.DateTime.. and cannot translate this into a store expression

I have a problem that has taken me weeks to resolve and I have not been able to.
I have a class where I have two methods. The following is supposed to take the latest date from database. That date represents the latest payment that a customer has done to "something":
public DateTime getLatestPaymentDate(int? idCustomer)
{
DateTime lastDate;
lastDate = (from fp in ge.Payments
from cst in ge.Customers
from brs in ge.Records.AsEnumerable()
where (cst.idCustomer == brs.idCustomer && brs.idHardBox == fp.idHardbox
&& cst.idCustomer == idCustomer)
select fp.datePayment).AsEnumerable().Max();
return lastDate;
}//getLatestPaymentDate
And here I have the other method, which is supposed to call the previous one to complete a Linq query and pass it to a Crystal Report:
//Linq query to retrieve all those customers'data who have not paid their safebox(es) annuity in the last year.
public List<ReportObject> GetPendingPayers()
{
List<ReportObject> defaulterCustomers;
defaulterCustomers = (from c in ge.Customer
from br in ge.Records
from p in ge.Payments
where (c.idCustomer == br.idCustomer
&& br.idHardBox == p.idHardBox)
select new ReportObject
{
CustomerId = c.idCustomer,
CustomerName = c.nameCustomer,
HardBoxDateRecord = br.idHardRecord,
PaymentDate = getLatestPaymentDate(c.idCustomer),
}).Distinct().ToList();
}//GetPendingPayers
No compile error is thrown here, but when I run the application and the second method tries to call the first one in the field PaymentDate the error mentioned in the header occurs:
Linq to Entities does not recognize the method System.DateTime.. and cannot translate this into a store expression
Please anybody with an useful input that put me off from this messy error? Any help will be appreciated !
Thanks a lot !
Have a look at these other questions :
LINQ to Entities does not recognize the method
LINQ to Entities does not recognize the method 'System.DateTime Parse(System.String)' method
Basically, you cannot use a value on the C# side and translate it into SQL. The first question offers a more thorough explanation ; the second offers a simple solution to your problem.
EDIT :
Simply put : the EF is asking the SQL server to perform the getLatestPaymentDate method, which it has no clue about. You need to execute it on the program side.
Simply perform your query first, put the results into a list and then do your Select on the in-memory list :
List<ReportObject> defaulterCustomers;
var queryResult = (from c in ge.Customer
from br in ge.Records
from p in ge.Payments
where (c.idCustomer == br.idCustomer
&& br.idHardBox == p.idHardBox)).Distinct().ToList();
defaulterCustomers = from r in queryResult
select new ReportObject
{
CustomerId = r.idCustomer,
CustomerName = r.nameCustomer,
HardBoxDateRecord = r.idHardRecord,
PaymentDate = getLatestPaymentDate(r.idCustomer),
}).Distinct().ToList();
I don't have access to your code, obviously, so try it out and tell me if it works for you!
You'll end up with an in-memory list