For function createReactiveStateEngine , is there a way to apply multiple filtering? I could not get it working by passing a list of conditions.
My code is listed as follows:
metrics=array(ANY, 2, 2)
metrics[0]=<ema(ffill(qty), 50)/ema(ffill(price), 50)>
metrics[1]=<msum(price, 10)/mavg(price, 10)*ratios(qty)>
sym=take("A"+string(1..3000), 6000000)
price=round(rand(100.0, 6000000), 2)
qty=rand(1..100, 6000000)
dummy = table(sym, price, qty)
outputTable = table(6000000:0, `sym`factor1`factor2, [STRING,DOUBLE,DOUBLE])
priceFilter = [<price> 0>,<prev(price) != price>]
priceFltEngine = createReactiveStateEngine(name="price_filter", metrics=metrics, dummyTable=dummy, outputTable=outputTable, keyColumn="sym", filter=priceFilter, keepOrder=true)
I received the following exception:
filter must be a piece of code to indicate the filtering logic.
Related
My Linq Query keeps returning the null error on FirstOrDefault
The cast to value type 'System.Int32' failed because the materialized value is null
because it can't find any records to match on the ClinicalAssetID form the ClinicalReading Table, fair enough!
But I want the fields in my details page just to appear blank if the table does not have matching entry.
But how can I handle the null issue when using the order by function ?
Current Code:
var ClinicalASSPATINCVM = (from s in db.ClinicalAssets
join cp in db.ClinicalPATs on s.ClinicalAssetID equals cp.ClinicalAssetID into AP
from subASSPAT in AP.DefaultIfEmpty()
join ci in db.ClinicalINSs on s.ClinicalAssetID equals ci.ClinicalAssetID into AI
from subASSINC in AI.DefaultIfEmpty()
join co in db.ClinicalReadings on s.ClinicalAssetID equals co.ClinicalAssetID into AR
let subASSRED = AR.OrderByDescending(subASSRED => subASSRED.MeterReadingDone).FirstOrDefault()
select new ClinicalASSPATINCVM
{
ClinicalAssetID = s.ClinicalAssetID,
AssetTypeName = s.AssetTypeName,
ProductName = s.ProductName,
ModelName = s.ModelName,
SupplierName = s.SupplierName,
ManufacturerName = s.ManufacturerName,
SerialNo = s.SerialNo,
PurchaseDate = s.PurchaseDate,
PoNo = s.PoNo,
Costing = s.Costing,
TeamName = s.TeamName,
StaffName = s.StaffName,
WarrantyEndDate = subASSPAT.WarrantyEndDate,
InspectionDate = subASSPAT.InspectionDate,
InspectionOutcomeResult = subASSPAT.InspectionOutcomeResult,
InspectionDocumnets = subASSPAT.InspectionDocumnets,
LastTypeofInspection = subASSINC.LastTypeofInspection,
NextInspectionDate = subASSINC.NextInspectionDate,
NextInspectionType = subASSINC.NextInspectionType,
MeterReadingDone = subASSRED.MeterReadingDone,
MeterReadingDue = subASSRED.MeterReadingDue,
MeterReading = subASSRED.MeterReading,
MeterUnitsUsed = subASSRED.MeterUnitsUsed,
FilterReplaced = subASSRED.FilterReplaced
}).FirstOrDefault(x => x.ClinicalAssetID == id);
Tried this but doesn't work
.DefaultIfEmpty(new ClinicalASSPATINCVM())
.FirstOrDefault()
Error was:
CS1929 'IOrderedEnumerable<ClinicalReading>' does not contain a definition for 'DefaultIfEmpty' and the best extension method overload 'Queryable.DefaultIfEmpty<ClinicalASSPATINCVM>(IQueryable<ClinicalASSPATINCVM>, ClinicalASSPATINCVM)' requires a receiver of type 'IQueryable<ClinicalASSPATINCVM>'
Feel a little closer with this but still errors
let subASSRED = AR.OrderByDescending(subASSRED => (subASSRED.MeterReadingDone != null) ? subASSRED.MeterReadingDone : String.Empty).FirstOrDefault()
Error:
CS0173 Type of conditional expression cannot be determined because there is no implicit conversion between 'System.DateTime?' and 'string'
The original error means that some of the following properties of the ClinicalASSPATINCVM class - MeterReadingDone, MeterReadingDue, MeterReading, MeterUnitsUsed, or FilterReplaced is of type int.
Remember that subASSRED here
let subASSRED = AR.OrderByDescending(subASSRED => subASSRED.MeterReadingDone).FirstOrDefault()
might be null (no corresponding record).
Now look at this part of the projection:
MeterReadingDone = subASSRED.MeterReadingDone,
MeterReadingDue = subASSRED.MeterReadingDue,
MeterReading = subASSRED.MeterReading,
MeterUnitsUsed = subASSRED.MeterUnitsUsed,
FilterReplaced = subASSRED.FilterReplaced
If that was LINQ to Objects, all these would generate NRE (Null Reference Exception) at runtime. In LINQ to Entities this is converted and executed as SQL. SQL has no issues with expression like subASSRED.SomeProperty because SQL supports NULL naturally even if SomeProperty normally does not allow NULL. So the SQL query executes normally, but now EF must materialize the result into objects, and the C# object property is not nullable, hence the error in question.
To solve it, find the int property(es) and use the following pattern inside query:
SomeIntProperty = (int?)subASSRED.SomeIntProperty ?? 0 // or other meaningful default
or change receiving object property type to int? and leave the original query as is.
Do the same for any non nullable type property, e.g. DateTime, double, decimal, Guid etc.
You're problem is because your DefaultIfEmpty is executed AsQueryable. Perform it AsEnumerable and it will work:
// create the default element only once!
static readonly ClinicalAssPatInVcm defaultElement = new ClinicalAssPatInVcm ();
var result = <my big linq query>
.Where(x => x.ClinicalAssetID == id)
.AsEnumerable()
.DefaultIfEmpty(defaultElement)
.FirstOrDefault();
This won't lead to a performance penalty!
Database management systems are extremely optimized for selecting data. One of the slower parts of a database query is the transport of the selected data to your local process. Hence it is wise to let the DBMS do most of the selecting, and only after you know that you only have the data that you really plan to use, move the data to your local process.
In your case, you need at utmost one element from your DBMS, and if there is nothing, you want to use a default object instead.
AsQueryable will move the selected data to your local process in a smart way, probably per "page" of selected data.
The page size is a good compromise: not too small, so you don't have to ask for the next page too often; not too large, so that you don't transfer much more items than you actually use.
Besides, because of the Where statement you expect at utmost one element anyway. So that a full "page" is fetched is no problem, the page will contain only one element.
After the page is fetched, DefaultIfEmpty checks if the page is empty, and if so, returns a sequence containing the defaultElement. If not, it returns the complete page.
After the DefaultIfEmpty you only take the first element, which is what you want.
I fail to translate a sql query to a linq query that could calculate some stock.
This is my test query that I'm trying to convert to a linq query.
SELECT
i.*,
(SELECT COUNT(t.*) FROM tickets t
WHERE t.starttime::time = i.sessionstarttime::time
AND t.starttime::date = '2018-04-06'::date)
as stock
FROM items I
-- note that the hardcoded date ('2018-04-06') is a function parameter
( tl;dr; how would you convert this PostgreSQL query to LINQ? )
My attempts so far are the variations of the following query:
var items = await _context.Items.Select(x => new Item
{
Id = x.Id,
IsTicket = x.IsTicket,
Name = x.Name,
Price = x.Price,
SaleItems = x.SaleItems,
SessionStartTime = x.SessionStartTime,
DateCreated = x.DateCreated,
DateEdit = x.DateEdit,
UserIdCreated = x.UserIdCreated,
UserIdEdited = x.UserIdEdited,
// calculate stock in subquery
Stock = _context.Tickets.Count(
t => t.StartTime.Date == ticketDate
&& x.SessionStartTime.HasValue
&& t.StartTime.Hour == x.SessionStartTime.Value.Hours // this is the part that is failing
&& t.State != TicketState.Canceled)
}).ToListAsync();
t.StartTime is Datetime and x.SessionStartTime is Nullable Timespan
So when I comment the line && t.StartTime.Hour == x.SessionStartTime.Value.Hours everything is fine, but with it I get warnings that it could not be translated and will be evaluated locally. But I don't want to download the whole ticket table just to count them.
The t.StartTime.Hour part is fine, I tried to perform static comparisons with both parameters. t.StartTime.Hour == 5 was translated without any problems, but x.SessionStartTime.Value.Hours == 5 failed to translate.
Also the problematic part in the application output:
([t].StartTime.Hour == Convert([x].SessionStartTime, TimeSpan).Hours))
So I guess that convert part is failing.
So what I'm missing and how I could work around this problem. Any help will be appreciated.
Update:
After experimenting a bit I have found two workarounds, that I wouldn't call the answers.
First I noticed that EF is trying to convert Nullable<TimeSpan> to a regular TimeSpan from the mentioned output: ([t].StartTime.Hour == Convert([x].SessionStartTime, TimeSpan).Hours))
I thought I could prevent that conversion by converting to a string and comparing the strings (I have a feeling this will bite me in the future):
t.StartTime.ToString().Contains(x.SessionStartTime.ToString())
The second workaround is only viable for my scenario since I know the items query is final and I can materialise it without calculated Stock, and then loop through the results and calculate it on a separate query. But this seems to add additional calls to the database and sacrifice some performance.
foreach(var x in items.Where(x=>x.SessionStartTime.HasValue))
{
// accessing the t.StartTime.TimeOfDay property seems to fail the LINQ to SQL as well
var hours = x.SessionStartTime.Value.Hours;
var minutes = x.SessionStartTime.Value.Minutes;
x.Stock = _context.Tickets.Count(t => t.StartTime.Date == ticketDate
&& t.StartTime.Hour == hours
&& t.StartTime.Minute == minutes);
}
I am new to jayData and am trying to filter on an entity set. The filter needs to perform an complex evaluation beyond what I saw in the samples.
Here is a working sample of what I am trying to accomplish (the listView line isn't and is just there to show what I plan to do with the data):
function () {
var weekday = moment().isoWeekday()-1;
console.log(weekday);
var de = leagueDB.DailyEvents.toArray(function (events) {
console.log(events);
var filtered = [];
for (var e = 0; e < events.length;e++) {
console.log(events[e]);
console.log(events[e].RecurrenceRule);
var rule = RRule.fromString(events[e].RecurrenceRule);
var ruleOptions = rule.options.byweekday;
var isDay = ruleOptions.indexOf(weekday);
console.log(ruleOptions, isDay);
if(isDay =! -1)
{
filtered.push(events[e]);
}
}
$("#listView").kendoListView({dataSource:filtered});
});
Basically it is just evaluating a recurring rule string to see if the current day meets that criteria, if so add that event to the list for viewing.
But it blows up when I try to do this:
eventListLocal:leagueDB.DailyEvents.filter(function(e){
console.log("The Weekday is:"+viewModel.weekday);
console.log(e);
console.log("The recurrence rule is:"+e.RecurrenceRule);
var rruleOptions = viewModel.rruleOptions(e.RecurrenceRule);
if (rruleOptions !== -1) {
return true;
}
}).asKendoDataSource()
The error that is generating is:
Exception: Unable to resolve type:undefined
The thing is it seems to be occurring on "e" and the console logs like the event is not being passed in. However, I am not seeing a list either. In short I am really confused as to what is going on.
Any help would be appreciated.
Thanks,
You can't write filter expressions such as this.
When you write .filter(...), jaydata will parse your expression and then it will generate filter for underlying provider, for example where for webSql and $filter for oDataProvider.
Both JayData expression parser and the data provider itself should understand your filter.
Your filter is not suitable for this approach, because most of your codes are not familiar for jaydata expression parser and the underlying data provider, for example your console.log etc.
You can simplify your filter, or you should load all your data into an array, and then you can use filter method of array itself, there, you can write any filter you like, and your filter will work. Of course this has performance issue in some scenarios when your data set is large.
Read more on http://jaydata.org/tutorials/entityexpressions-the-heart-of-jaydata
I'm using Rails 4 in a web app, Postgresql database and squeel gem for queries.
I have this function in my model statistic.rb
def properties_mean_ppm(mode, rooms, type, output_currency_id)
sql_result = properties(mode, rooms, type).select{
avg(price_dolar / property_area).as(prom)
}
avg = sql_result[0].prom
final_avg = change_currency(avg, DOLAR_ID, output_currency_id)
return final_avg.to_f
end
price_dolar and property_area are columns in the properties table.
It works fine in Rails console and displays the result, but when I use it on the controller it gives an error:
ActiveModel::MissingAttributeError (missing attribute: id)
And indicates the line
avg = sql_result.to_a[0].prom
I also tried using sql_result[0].prom or sql_result.take or sql_result.first, they all have the same error.
The sql_result is this:
#<ActiveRecord::Relation [#<Property >]>
This is the action called in the controller
def properties_mean_ppm
#statistic = Statistic.find(params[:id])
mode = params[:mode] ? params[:mode] : ANY_MODE
type = params[:type] ? params[:type] : ANY_TYPE
one_room = #statistic.properties_mean_ppm(mode, 1, type, UF)
end
I know how to get the result using only SQL without activerecord but that would be very inefficient for me because I have lots of filters called before in the properties() function
Seems like calling that from the properties object made the controller to expect a Property with an id as a result. So I made it work without squeel.
sql_result = properties(mode, rooms, type).select(
"avg(precio_dolar / dimension_propiedad) as prom, 1 as id"
)
And giving a fixed id to the result
In WordPress as you must already known, when using get_posts() or query_posts() or even WP_Query, it is not possible to order the returned posts by specifying a list of post ID in the order we want.
Instead we have to loop through the results and re-order them on the PHP side. This is a performance hit and a bad practice. Instead we should use built-in MySQL functions to retrieve the posts in the desired order upfront.
Thankfully there is the posts_orderby which can be used to specify a custom ORDERBY statement, like this:
// My list of post IDs in my custom order
$my_post_ids = array(1,3,2);
// Apply filter to the ORDERBY SQL statement
add_filter('posts_orderby', 'my_custom_orderby');
function my_custom_orderby($orderby_statement) {
global $my_post_ids;
$orderby_statement = 'FIELD(ID, '.implode(',',$my_post_ids).')';
return $orderby_statement;
}
// My custom query
$my_custom_query = new WP_Query(array('post_type' => 'post', 'post__in' => $my_post_ids);
However there is a problem with the above code, is that it will affect the order of all queries on the page! Including queries made by plugins, shortcodes, and so on.
Easy fix!
The simple way to fix this, is to apply the filter only one time, and remove it as soon as it is called, by putting a remove_filter() within the filter itself, so it is run only once:
// My list of post IDs in my custom order
$my_post_ids = array(1,3,2);
// Apply filter to the ORDERBY SQL statement
add_filter('posts_orderby', 'my_custom_orderby');
function my_custom_orderby($orderby_statement) {
// Disable this filter for future queries!
remove_filter(current_filter(), __FUNCTION__);
global $my_post_ids;
$orderby_statement = 'FIELD(ID, '.implode(',',$my_post_ids).')';
return $orderby_statement;
}
// My custom query
$my_custom_query = new WP_Query(array('post_type' => 'post', 'post__in' => $my_post_ids);
Because I set this filter just before my custom query, once I execute my custom query it should be filtered by the posts_orderby filter set above, which is then immediately disabled so it won't affect any future queries.
In theory, that's great, and it works great in most case scenarios!
An issue with WPML
However I have encountered a case when using the WPML plugin where this filter affects other queries than mine and causes errors. I believe the WPML plugin is creating a query of its own that is executed just before my own custom query, making my filter applies to the WPML query instead of mine!
Is there any possible way to add a check within the filter to make sure that it affects the correct query?
Thank you very much
Edit:
The fix for WPML
For information, while the accepted answer for this question is correct, it didn't solve the problem I was having with WPML. Here is how I fixed the WPML conflict:
// My list of post IDs in my custom order
$my_post_ids = array(1,3,2);
// Apply filter to the ORDERBY SQL statement
add_filter('posts_orderby', 'my_custom_orderby');
function my_custom_orderby($orderby_statement) {
// Disable this filter for future queries!
remove_filter(current_filter(), __FUNCTION__);
global $my_post_ids, $wpdb;
$orderby_statement = 'FIELD('.$wpdb->base_prefix.'posts.ID, '.implode(',',$my_post_ids).')';
return $orderby_statement;
}
// My custom query
$my_custom_query = new WP_Query(array('post_type' => 'post', 'post__in' => $my_post_ids);
This filter takes two parameters, $orderby and &$this. "this" being the WP_Query object. I'm not sure how to detect that WPML is making the call, but we can check that your call is the one being
made.
$my_post_ids = array(1,3,2);
add_filter( 'posts_orderby', 'my_custom_orderby', 10, 2 );
function my_custom_orderby( $orderby_statement, $object )
{
global $my_post_ids;
if( $my_post_ids != $object->query['post__in'] )
return $orderby_statement;
// Disable this filter for future queries!
remove_filter( current_filter(), __FUNCTION__ );
$orderby_statement = 'FIELD(ID, ' . implode( ',', $my_post_ids ) . ')';
return $orderby_statement;
}