I'm using Entity-Framework 6.1.3 with a sqlite database.
During page load I'm initializing some properties in a loop (see below).
foreach (var trade in model.Trades)
{
trade.ExchangeRates = Db.ExchangeRates.Local;
trade.BaseCurrency = Prj_TradAc.Properties.Settings.Default.BaseCurrency;
}
Db.ExchangeRates.Local never hits the Database which is expected.
So I was expecting to only assign a reference to Db.ExchangeRates.Local which should be fast.
However with only ~500 Trades the loop takes almost 10s!
When I do the following
var ers = Db.ExchangeRates.Local;
foreach (var trade in model.Trades)
{
trade.ExchangeRates = ers;
trade.BaseCurrency = Prj_TradAc.Properties.Settings.Default.BaseCurrency;
}
the same loop with the same amount of data takes ~40ms
So why is accessing DBSet.Local so slow?
EDIT:
Db.Configuration.AutoDetectChangesEnable = false
also makes the assignment fast. However I still do not understand why this is an issue here. My properties which I'm assigning to are just linked to fields - so no operation is going on here. There shouldn't be a change to the DBSet during assignment.
Anytime you access Local property (using the property getter), and DbContext.Configuration.AutoDetectChangesEnabled property is true (by default), EF calls ObjectContext.DetectChanges method which slows down the process.
Related
I get an object from within an array in my model (a JSONmodel type) which is
{
"task": [
{
"dbid": 465,
"bk_cnt": 11,
}, {
"dbid": 472,
"bk_cnt": 16,
}
]
}
I bind this model to a table and connect the bk_cnt up to an objectNumber in a cell. No problem so far.
In code I want to change the value of the first bk_cnt value from 11 to 20 on press of a button. Inside the event I have:
var model = this.getView().getModel() // get the model
var tasks = model.getProperty("/task"); // get as a JS object
tasks[0].bk_cnt = 20 // update the model...will it update the view?
// model.refresh() // it will if this is uncommented.
Problem: Though it is bound to the view, the displayed value of bk_cnt does not change. if I add model.refresh() it does. This code is extracted from a larger section and one of the larger features is sorting by column click. When I click a column to re-sort (no change to the model), the value 20 appears.
What gives?
Musings: I have read that the model.getProprty() function returns a javascript object with a live reference back to the model, and that a change to the value of the object will automatically be reflected in the view for any bound controls. Does this statement fall down on array attributes ?
EDIT: Still feeling around the issue I find that
model.setProperty("/task/0/bk_cnt", 20)
Does not require a model.refresh() to update the view. Not a total surprise as this command is directly acting through the model. This leaves me thinking that the 'live' object returned by getProperty() is only live when it is a primitive datatype like a string or integer, but not for a JS object. Or am I missing something ?
EDIT 2: #Ash points out in his answer that there is a further approach which is to access the JS object from the model property, set whatever attributes need to be updated in the JS object, then replace that into the model, e.g.
var tasks = model.getProperty("/task");
tasks[0].bk_cnt = 20
model.setProperty('/task', tasks)
Second edit done to complete the trio of approaches for future readers.
The Model object is an abstraction layer ON TOP of a javascript object. There is no way that a change within an object is notified anywhere. You need to explicitly trigger the notifications through model.refresh() or model.setProperty().
So both of your solutions are valid, another one (which I favor) would be
var tasks = model.getProperty("/task");
tasks[0].bk_cnt = 20
model.setProperty('/task', tasks)
But this actually depends on how you bind your model to your UI objects :)
When I create a query in squeryl, it returns a Query[T] object. The query was not yet executed and will be, when I iterate over the Query object (Query[T] extends Iterable[T]).
Around the execution of a query there has to be either a transaction{} or a inTransaction{} block.
I'm just speaking of SELECT queries and transactions wouldn't be necessary, but the squeryl framework needs them.
I'd like to create a query in the model of my application and pass it directly to the view where a view helper in the template iterates over it and presents the data.
This is only possible when putting the transaction{} block in the controller (the controller includes the call of the template, so the template which does the iteration is also inside). It's not possible to put the transaction{} block in the model, because the model doesn't really execute the query.
But in my understanding the transaction has nothing to do with the controller. It's a decision of the model which database framework to use, how to use it and where to use transactions. So I want the transaction{} block to be in the model.
I know that I can - instead of returning the Query[T] instance - call Iterable[T].toList on this Query[T] object and then return the created list. Then the whole query is executed in the model and everything is fine. But I don't like this approach, because all the data requested from the database has to be cached in this list. I'd prefer a way where this data is directly passed to the view. I like the MySql feature of streaming the result set when it's large.
Is there any possibility? Maybe something like a function Query[T].executeNow() which sends the request to the database, is able to close the transaction, but still uses the MySQL streaming feature and receives the rest of the (selected and therefore fixed) result set when it's accessed? Because the result set is fixed in the moment of the query, closing the transaction shouldn't be a problem.
The general problem that I see here is that you try to combine the following two ideas:
lazy computation of data; here: database results
hiding the need for a post-processing action that must be triggered when the computation is done; here: hiding from your controller or view that the database session must be closed
Since your computation is lazy and since you are not obliged to perform it to the very end (here: to iterate over the whole result set), there is no obvious hook that could trigger the post-processing step.
Your suggestion of invoking Query[T].toList does not exhibit this problem, since the computation is performed to the very end, and requesting the last element of the result set can be used as a trigger for closing the session.
That said, the best I could come up with is the following, which is an adaptation of the code inside org.squeryl.dsl.QueryDsl._using:
class IterableQuery[T](val q: Query[T]) extends Iterable[T] {
private var lifeCycleState: Int = 0
private var session: Session = null
private var prevSession: Option[Session] = None
def start() {
assert(lifeCycleState == 0, "Queries may not be restarted.")
lifeCycleState = 1
/* Create a new session for this query. */
session = SessionFactory.newSession
/* Store and unbind a possibly existing session. */
val prevSession = Session.currentSessionOption
if(prevSession != None) prevSession.get.unbindFromCurrentThread
/* Bind newly created session. */
session.bindToCurrentThread
}
def iterator = {
assert(lifeCycleState == 1, "Query is not active.")
q.toStream.iterator
}
def stop() {
assert(lifeCycleState == 1, "Query is not active.")
lifeCycleState = 2
/* Unbind session and close it. */
session.unbindFromCurrentThread
session.close
/* Re-bind previous session, if it existed. */
if(prevSession != None) prevSession.get.bindToCurrentThread
}
}
Clients can use the query wrapper as follows:
var manualIt = new IterableQuery(booksQuery)
manualIt.start()
manualIt.foreach(println)
manualIt.stop()
// manualIt.foreach(println) /* Fails, as expected */
manualIt = new IterableQuery(booksQuery) /* Queries can be reused */
manualIt.start()
manualIt.foreach(b => println("Book: " + b))
manualIt.stop()
The invocation of manualIt.start() could already be done when the object is created, i.e., inside the constructor of IterableQuery, or before the object is passed to the controller.
However, working with resources (files, database connections, etc.) in such a way is very fragile, because the post-processing is not triggered in case of exceptions. If you look at the implementation of org.squeryl.dsl.QueryDsl._using you will see a couple of try ... finally blocks that are missing from IterableQuery.
I am new to using intersytems cache and face an issue where I am querying data stored in cache, exposed by classes which do not seem to accurately represent the data in the underlying system. The data stored in the globals is almost always larger than what is defined in the object code.
As such I get errors like the one below very frequently.
Msg 7347, Level 16, State 1, Line 2
OLE DB provider 'MSDASQL' for linked server 'cache' returned data that does not match expected data length for column '[cache]..[namespace].[tablename].columname'. The (maximum) expected data length is 5, while the returned data length is 6.
Does anyone have any experience with implementing some type of quality process to ensure that the object definitions (sql mappings) are maintained in such away that they can accomodate the data which is being persisted in the globals?
Property columname As %String(MAXLEN = 5, TRUNCATE = 1) [ Required, SqlColumnNumber = 2, SqlFieldName = columname ];
In this particular example the system has the column defined with a max len of 5, however the data stored in the system is 6 characters long.
How can I proactively monitor and repair such situations.
/*
I did not create these object definitions in cache
*/
It's not completely clear what "monitor and repair" would mean for you, but:
How much control do you have over the database side? Cache runs code for a data-type on converting from a global to ODBC using the LogicalToODBC method of the data-type class. If you change the property types from %String to your own class, AppropriatelyNamedString, then you can override that method to automatically truncate. If that's what you want to do. It is possible to change all the %String property types programatically using the %Library.CompiledClass class.
It is also possible to run code within Cache to find records with properties that are above the (somewhat theoretical) maximum length. This obviously would require full table scans. It is even possible to expose that code as a stored procedure.
Again, I don't know what exactly you are trying to do, but those are some options. They probably do require getting deeper into the Cache side than you would prefer.
As far as preventing the bad data in the first place, there is no general answer. Cache allows programmers to directly write to the globals, bypassing any object or table definitions. If that is happening, the code doing so must be fixed directly.
Edit: Here is code that might work in detecting bad data. It might not work if you are doing cetain funny stuff, but it worked for me. It's kind of ugly because I didn't want to break it up into methods or tags. This is meant to run from a command prompt, so it would have to be modified for your purposes probably.
{
S ClassQuery=##CLASS(%ResultSet).%New("%Dictionary.ClassDefinition:SubclassOf")
I 'ClassQuery.Execute("%Library.Persistent") b q
While ClassQuery.Next(.sc) {
If $$$ISERR(sc) b Quit
S ClassName=ClassQuery.Data("Name")
I $E(ClassName)="%" continue
S OneClassQuery=##CLASS(%ResultSet).%New(ClassName_":Extent")
I '$IsObject(OneClassQuery) continue //may not exist
try {
I 'OneClassQuery.Execute() D OneClassQuery.Close() continue
}
catch
{
D OneClassQuery.Close()
continue
}
S PropertyQuery=##CLASS(%ResultSet).%New("%Dictionary.PropertyDefinition:Summary")
K Properties
s sc=PropertyQuery.Execute(ClassName) I 'sc D PropertyQuery.Close() continue
While PropertyQuery.Next()
{
s PropertyName=$G(PropertyQuery.Data("Name"))
S PropertyDefinition=""
S PropertyDefinition=##CLASS(%Dictionary.PropertyDefinition).%OpenId(ClassName_"||"_PropertyName)
I '$IsObject(PropertyDefinition) continue
I PropertyDefinition.Private continue
I PropertyDefinition.SqlFieldName=""
{
S Properties(PropertyName)=PropertyName
}
else
{
I PropertyName'="" S Properties(PropertyDefinition.SqlFieldName)=PropertyName
}
}
D PropertyQuery.Close()
I '$D(Properties) continue
While OneClassQuery.Next(.sc2) {
B:'sc2
S ID=OneClassQuery.Data("ID")
Set OneRowQuery=##class(%ResultSet).%New("%DynamicQuery:SQL")
S sc=OneRowQuery.Prepare("Select * FROM "_ClassName_" WHERE ID=?") continue:'sc
S sc=OneRowQuery.Execute(ID) continue:'sc
I 'OneRowQuery.Next() D OneRowQuery.Close() continue
S PropertyName=""
F S PropertyName=$O(Properties(PropertyName)) Q:PropertyName="" d
. S PropertyValue=$G(OneRowQuery.Data(PropertyName))
. I PropertyValue'="" D
.. S PropertyIsValid=$ZOBJClassMETHOD(ClassName,Properties(PropertyName)_"IsValid",PropertyValue)
.. I 'PropertyIsValid W !,ClassName,":",ID,":",PropertyName," has invalid value of "_PropertyValue
.. //I PropertyIsValid W !,ClassName,":",ID,":",PropertyName," has VALID value of "_PropertyValue
D OneRowQuery.Close()
}
D OneClassQuery.Close()
}
D ClassQuery.Close()
}
The simplest solution is to increase the MAXLEN parameter to 6 or larger. Caché only enforces MAXLEN and TRUNCATE when saving. Within other Caché code this is usually fine, but unfortunately ODBC clients tend to expect this to be enforced more strictly. The other option is to write your SQL like SELECT LEFT(columnname, 5)...
The simplest solution which I use for all Integration Services Packages, for example is to create a query that casts all nvarchar or char data to the correct length. In this way, my data never fails for truncation.
Optional:
First run a query like: SELECT Max(datalength(mycolumnName)) from cachenamespace.tablename.mycolumnName
Your new query : SELECT cast(mycolumnname as varchar(6) ) as mycolumnname,
convert(varchar(8000), memo_field) AS memo_field
from cachenamespace.tablename.mycolumnName
Your pain of getting the data will be lessened but not eliminated.
If you use any type of oledb provider, or if you use an OPENQUERY in SQL Server,
the casts must occur in the query sent to Intersystems CACHE db, not in the the outer query that retrieves data from the inner OPENQUERY.
Consider the following code.
var items = from i in context.Items
select i;
var item = items.FirstOrDefault();
item.this = "that";
item.that = "this";
var items2 = from i in context.Items
where i.this == "that"
select i;
var data = items2.FirstOrDefault();
context.SaveChanges();
I'm trying to confirm that items2 will not include my modifications to item. In other words, items2's copy of item will not include the unsaved changes.
Have you tried it? =)
By default, your objects are being tracked and cached by the context, so that the objects in your second query actually do reflect changes in the first.
You may want to call context.Items.AsNoTracking() on the one of your two "items" to get the behavior you want.
Edit: Actually, this is a strange question. I just noticed that your items2 hasn't even hit the database yet, since you haven't called ToList() or FirstorDefault(). It remains an IQueryable that will hit the database after your code snippet and will therefore contain the changed value.
HOWEVER, if you call ToList() on items2, you'll encounter the caching scenario I outlined above.
In case of "var item" your query is executed the moment you used FirstOrDefault(). But for var items2 the query is still not executed. Now in your case result of items2 will always be affected by the updates you have done in the first query.
It will contain the modifications, only way to do is create a new context and query the new context.
I have a very small entity framework setup containing only a few related classes/tables and a view. I need to be able to pull a specific record from this view, namely, I need to be able to grab the record that meets two criteria, it has a specific ProfileID and a specific QuoteID.
This line is what's causing the problem:
TWProfileUpchargeTotal upchargeTotals = _context.TWProfileUpchargeTotals.Where(p => p.Profileid == profile.id && p.quoteid == _quote.quoteid).First();
I'm looping through the profiles I know about and getting their information from the view, so profile.id changes each time.
The first time this code executes it gets the correct record from the view.
The second and third (and presumably beyond that) time it executes, it retrieves the exact same record.
Any idea why or what I'm doing wrong here?
Thanks, in advance.
You've been bitten by the LINQ "gotcha" called closure. The following post (and many others) on SO detail this:
closure
What you need to do is declare a variable WITHIN the foreach you've ommited from the above code and assign the profile.id to this and use this in the Where clause.
foreach(Profile profile in ListOfProfiles)
{
var localProfile = profile;
TWProfileUpchargeTotal upchargeTotals = _context.TWProfileUpchargeTotals.Where(p => p.Profileid == localProfile.id && p.quoteid == _quote.quoteid).First();
}