Exclude null columns in an update statement - JOOQ - postgresql

I have a POJO that has the fields that can be updated. But sometimes only a few fields will need to be updated and the rest are null. How do I write an update statement that ignores the fields that are null? Would it be better to loop through the non missing ones and dynamically add to a set statement, or using coalesce?
I have the following query:
jooqService.using(txn)
.update(USER_DETAILS)
.set(USER_DETAILS.NAME, input.name)
.set(USER_DETAILS.LAST_NAME, input.lastName)
.set(USER_DETAILS.COURSES, input.courses)
.set(USER_DETAILS.SCHOOL, input.school)
.where(USER_DETAILS.ID.eq(input.id))
.execute()
If there is a better practice?

I don't know Jooq but it looks like you could simply do this:
val jooq = jooqService.using(txn).update(USER_DETAILS)
input.name.let {jooq.set(USER_DETAILS.NAME, it)}
input.lastName.let {jooq.set(USER_DETAILS.LAST_NAME, it)}
etc...
EDIT: Mapping these fields explicitly as above is clearest in my opinion, but you could do something like this:
val fields = new Object[] {USER_DETAILS.NAME, USER_DETAILS.LAST_NAME}
val values = new Object[] {input.name, input.lastName}
val jooq = jooqService.using(txn).update(USER_DETAILS)
values.forEachIndexed { i, value ->
value.let {jooq.set(fields[i], value)}
}
You'd still need to enumerate all the fields and values explicitly and consistently in the arrays for this to work. It seems less readable and more error prone to me.

In Java, it would be somthing like this
var jooqQuery = jooqService.using(txn)
.update(USER_DETAILS);
if (input.name != null) {
jooqQuery.set(USER_DETAILS.NAME, input.name);
}
if (input.lastName != null) {
jooqQuery.set(USER_DETAILS.LAST_NAME, input.lastName);
}
// ...
jooqQuery.where(USER_DETAILS.ID.eq(input.id))
.execute();

Another option rather than writing this UPDATE statement is to use UpdatableRecord:
// Load a POJO into a record using a RecordUnmapper
UserDetailsRecord r =
jooqService.using(txn)
.newRecord(USER_DETAILS, input)
(0 .. r.size() - 1).forEach { if (r[it] == null) r.changed(it, false) }
r.update();
You can probably write an extension function to make this available for all jOOQ records, globally, e.g. as r.updateNonNulls().

Related

How to fix FirstOrDefault returning Null in Linq

My Linq Query keeps returning the null error on FirstOrDefault
The cast to value type 'System.Int32' failed because the materialized value is null
because it can't find any records to match on the ClinicalAssetID form the ClinicalReading Table, fair enough!
But I want the fields in my details page just to appear blank if the table does not have matching entry.
But how can I handle the null issue when using the order by function ?
Current Code:
var ClinicalASSPATINCVM = (from s in db.ClinicalAssets
join cp in db.ClinicalPATs on s.ClinicalAssetID equals cp.ClinicalAssetID into AP
from subASSPAT in AP.DefaultIfEmpty()
join ci in db.ClinicalINSs on s.ClinicalAssetID equals ci.ClinicalAssetID into AI
from subASSINC in AI.DefaultIfEmpty()
join co in db.ClinicalReadings on s.ClinicalAssetID equals co.ClinicalAssetID into AR
let subASSRED = AR.OrderByDescending(subASSRED => subASSRED.MeterReadingDone).FirstOrDefault()
select new ClinicalASSPATINCVM
{
ClinicalAssetID = s.ClinicalAssetID,
AssetTypeName = s.AssetTypeName,
ProductName = s.ProductName,
ModelName = s.ModelName,
SupplierName = s.SupplierName,
ManufacturerName = s.ManufacturerName,
SerialNo = s.SerialNo,
PurchaseDate = s.PurchaseDate,
PoNo = s.PoNo,
Costing = s.Costing,
TeamName = s.TeamName,
StaffName = s.StaffName,
WarrantyEndDate = subASSPAT.WarrantyEndDate,
InspectionDate = subASSPAT.InspectionDate,
InspectionOutcomeResult = subASSPAT.InspectionOutcomeResult,
InspectionDocumnets = subASSPAT.InspectionDocumnets,
LastTypeofInspection = subASSINC.LastTypeofInspection,
NextInspectionDate = subASSINC.NextInspectionDate,
NextInspectionType = subASSINC.NextInspectionType,
MeterReadingDone = subASSRED.MeterReadingDone,
MeterReadingDue = subASSRED.MeterReadingDue,
MeterReading = subASSRED.MeterReading,
MeterUnitsUsed = subASSRED.MeterUnitsUsed,
FilterReplaced = subASSRED.FilterReplaced
}).FirstOrDefault(x => x.ClinicalAssetID == id);
Tried this but doesn't work
.DefaultIfEmpty(new ClinicalASSPATINCVM())
.FirstOrDefault()
Error was:
CS1929 'IOrderedEnumerable<ClinicalReading>' does not contain a definition for 'DefaultIfEmpty' and the best extension method overload 'Queryable.DefaultIfEmpty<ClinicalASSPATINCVM>(IQueryable<ClinicalASSPATINCVM>, ClinicalASSPATINCVM)' requires a receiver of type 'IQueryable<ClinicalASSPATINCVM>'
Feel a little closer with this but still errors
let subASSRED = AR.OrderByDescending(subASSRED => (subASSRED.MeterReadingDone != null) ? subASSRED.MeterReadingDone : String.Empty).FirstOrDefault()
Error:
CS0173 Type of conditional expression cannot be determined because there is no implicit conversion between 'System.DateTime?' and 'string'
The original error means that some of the following properties of the ClinicalASSPATINCVM class - MeterReadingDone, MeterReadingDue, MeterReading, MeterUnitsUsed, or FilterReplaced is of type int.
Remember that subASSRED here
let subASSRED = AR.OrderByDescending(subASSRED => subASSRED.MeterReadingDone).FirstOrDefault()
might be null (no corresponding record).
Now look at this part of the projection:
MeterReadingDone = subASSRED.MeterReadingDone,
MeterReadingDue = subASSRED.MeterReadingDue,
MeterReading = subASSRED.MeterReading,
MeterUnitsUsed = subASSRED.MeterUnitsUsed,
FilterReplaced = subASSRED.FilterReplaced
If that was LINQ to Objects, all these would generate NRE (Null Reference Exception) at runtime. In LINQ to Entities this is converted and executed as SQL. SQL has no issues with expression like subASSRED.SomeProperty because SQL supports NULL naturally even if SomeProperty normally does not allow NULL. So the SQL query executes normally, but now EF must materialize the result into objects, and the C# object property is not nullable, hence the error in question.
To solve it, find the int property(es) and use the following pattern inside query:
SomeIntProperty = (int?)subASSRED.SomeIntProperty ?? 0 // or other meaningful default
or change receiving object property type to int? and leave the original query as is.
Do the same for any non nullable type property, e.g. DateTime, double, decimal, Guid etc.
You're problem is because your DefaultIfEmpty is executed AsQueryable. Perform it AsEnumerable and it will work:
// create the default element only once!
static readonly ClinicalAssPatInVcm defaultElement = new ClinicalAssPatInVcm ();
var result = <my big linq query>
.Where(x => x.ClinicalAssetID == id)
.AsEnumerable()
.DefaultIfEmpty(defaultElement)
.FirstOrDefault();
This won't lead to a performance penalty!
Database management systems are extremely optimized for selecting data. One of the slower parts of a database query is the transport of the selected data to your local process. Hence it is wise to let the DBMS do most of the selecting, and only after you know that you only have the data that you really plan to use, move the data to your local process.
In your case, you need at utmost one element from your DBMS, and if there is nothing, you want to use a default object instead.
AsQueryable will move the selected data to your local process in a smart way, probably per "page" of selected data.
The page size is a good compromise: not too small, so you don't have to ask for the next page too often; not too large, so that you don't transfer much more items than you actually use.
Besides, because of the Where statement you expect at utmost one element anyway. So that a full "page" is fetched is no problem, the page will contain only one element.
After the page is fetched, DefaultIfEmpty checks if the page is empty, and if so, returns a sequence containing the defaultElement. If not, it returns the complete page.
After the DefaultIfEmpty you only take the first element, which is what you want.

how to update a value in dataframe and drop a row on this basis of a given value in scala

I need to update the value and if the value is zero then drop that row. Here is the snapshot.
val net = sc.accumulator(0.0)
df1.foreach(x=> {net += calculate(df2, x)})
def calculate(df2:DataFrame, x : Row):Double = {
var pro:Double = 0.0
df2.foreach(y => {if(xxx){ do some stuff and update the y.getLong(2) value }
else if(yyy){ do some stuff and update the y.getLong(2) value}
if(y.getLong(2) == 0) {drop this row from df2} })
return pro;
}
Any suggestions? Thanks.
You cannot change the DataFrame or RDD. They are read only for a reason. But you can create a new one and use transformations by all the means available. So when you want to change for example contents of a column in dataframe just add new column with updated contents by using functions like this:
df.withComlumn(...)
DataFrames are immutable, you can not update a value but rather create new DF every time.
Can you reframe your use case, its not very clear what you are trying to achieve with the above snippet (Not able to understand the use of accumulator) ?
You can rather try df2.withColumn(...) and use your udf here.

Way to Extract list of elements from Scala list

I have standard list of objects which is used for the some analysis. The analysis generates a list of Strings and i need to look through the standard list of objects and retrieve objects with same name.
case class TestObj(name:String,positions:List[Int],present:Boolean)
val stdLis:List[TestObj]
//analysis generates a list of strings
var generatedLis:List[String]
//list to save objects found in standard list
val lisBuf = new ListBuffer[TestObj]()
//my current way
generatedLis.foreach{i=>
val temp = stdLis.filter(p=>p.name.equalsIgnoreCase(i))
if(temp.size==1){
lisBuf.append(temp(0))
}
}
Is there any other way to achieve this. Like having an custom indexof method that over rides and looks for the name instead of the whole object or something. I have not tried that approach as i am not sure about it.
stdLis.filter(testObj => generatedLis.exists(_.equalsIgnoreCase(testObj.name)))
use filter to filter elements from 'stdLis' per predicate
use exists to check if 'generatedLis' has a value of ....
Don't use mutable containers to filter sequences.
Naive solution:
val lisBuf =
for {
str <- generatedLis
temp = stdLis.filter(_.name.equalsIgnoreCase(str))
if temp.size == 1
} yield temp(0)
if we discard condition temp.size == 1 (i'm not sure it is legal or not):
val lisBuf = stdLis.filter(s => generatedLis.exists(_.equalsIgnoreCase(s.name)))

What would be the opposite to hasFields?

I'm using logical deletes by adding a field deletedAt. If I want to get only the deleted documents it would be something like r.table('clients').hasFields('deletedAt'). My method has a withDeletes parameter which determines if deleted documents are excluded or not.
Finally, people at the #rethinkdb IRC channel suggested me to use the filter method and that did the trick:
query = adapter.table(table).filter(filters)
if withDeleted
query = adapter.filter (doc) ->
return doc.hasFields 'deletedAt'
else
query = adapter.filter (doc) ->
return doc.hasFields('deletedAt').not()
query.run connection, (err, results) ->
...
My question is why do I have to use filter and not something like:
query = adapter.table(table).filter(filters)
query = if withDeleted then query.hasFields 'deletedAt' else query.hasFields('deletedAt').not()
...
or something like that.
Thanks in advance.
The hasFields function can be called on both objects and sequences, but not cannot.
This query:
query.hasFields('deletedAt')
Behaves the same as this one (on sequences of objects):
query.filter((doc) -> return doc.hasFields('deletedAt'))
However, this query:
query.hasFields('deletedAt').not()
Behaves like this:
query.filter((doc) -> return doc.hasFields('deletedAt')).not()
But that doesn't make sense. you want the not to be inside the filter, not after it. Like this:
query.filter((doc) -> return doc.hasFields('deletedAt').not())
One nice that about RethinkDB is that because of the way queries are built up in host language it's very easy to define new fluent syntax by just defining functions in your language. For example if you wanted to have a lacksFields function you could define it in Python (sorry I don't really know coffeescript) like so:
def lacks_fields(stream, *args):
res = stream
for arg in args:
res = res.filter(lambda x: ~x.has_fields(arg))
return res
Then you can use a nice fluent syntax like:
lacks_fields(stream, "foo", "bar", "buzz")

How does the following Java "continue" code translate to Scala?

for (String stock : allStocks) {
Quote quote = getQuote(...);
if (null == quoteLast) {
continue;
}
Price price = quote.getPrice();
if (null == price) {
continue;
}
}
I don't necessarily need a line by line translation, but I'm looking for the "Scala way" to handle this type of problem.
You don't need continue or breakable or anything like that in cases like this: Options and for comprehensions do the trick very nicely,
val stocksWithPrices =
for {
stock <- allStocks
quote <- Option(getQuote(...))
price <- Option(quote.getPrice())
} yield (stock, quote, price);
Generally you try to avoid those situations to begin with by filtering before you even start:
val goodStocks = allStocks.view.
map(stock => (stock, stock.getQuote)).filter(_._2 != null).
map { case (stock, quote) => (stock,quote, quote.getPrice) }.filter(_._3 != null)
(this example showing how you'd carry along partial results if you need them). I've used a view so that results will be computed as-needed, instead of creating a bunch of new collections at each step.
Actually, you'd probably have the quotes and such return options--look around on StackOverflow for examples of how to use those instead of null return values.
But, anyway, if that sort of thing doesn't work so well (e.g. because you are generating too many intermediate results that you need to keep, or you are relying on updating mutable variables and you want to keep the evaluation pattern simple so you know what's happening when) and you can't conceive of the problem in a different, possibly more robust way, then you can
import scala.util.control.Breaks._
for (stock <- allStocks) {
breakable {
val quote = getQuote(...)
if (quoteLast eq null) break;
...
}
}
The breakable construct specifies where breaks should take you to. If you put breakable outside a for loop, it works like a standard Java-style break. If you put it inside, it acts like continue.
Of course, if you have a very small number of conditions, you don't need the continue at all; just use the else of the if-statement.
Your control structure here can be mapped very idiomatically into the following for loop, and your code demonstrates the kind of filtering that Scala's for loop was designed for.
for {stock <- allStocks.view
quote = getQuote(...)
if quoteLast != null
price = quote.getPrice
if null != price
}{
// whatever comes after all of the null tests
}
By the way, Scala will automatically desugar this into the code from Rex Kerr's solution
val goodStocks = allStocks.view.
map(stock => (stock, stock.getQuote)).filter(_._2 != null).
map { case (stock, quote) => (stock,quote, quote.getPrice) }.filter(_._3 != null)
This solution probably doesn't work in general for all different kinds of more complex flows that might use continue, but it does address a lot of common ones.
If the focus is really on the continue and not on the null handling, just define an inner method (the null handling part is a different idiom in scala):
def handleStock(stock: String): Unit {
val quote = getQuote(...)
if (null == quoteLast) {
return
}
val price = quote.getPrice();
if (null == price) {
return
}
}
for (stock <- allStocks) {
handleStock(stock)
}
The simplest way is to embed the skipped-over code in an if with reversed-sense to what you have.
See http://www.scala-lang.org/node/257