I'm trying understand CQRS to see if it can help out in an reporting environment.
Problem: An CQRS designed system is already in production, happily generating commands, events and updating the necessary query views. A new report is required. This report takes a number of parameters; Start Date, End Date, Product Type, and Product Category.
How do I generate the aggregate views for:
A query store that will initially be empty
And, can pass parameters with very different values
Do I try and solve this using a CQRS approach, or is there a better alternative?
Thanks
If it is not reasonable to precompute all your report data into flat view, then just don't do that. You may want to join a bunch of tables for your report. It's your decision what can be precomputed, and what is not worth it (cpu, storage considerations).
In your particular case (StartDate, EndDate,..) - i can't see what is the problem to generate a single ViewModel table for it, and just query directly against the parameters.
Figure out which events are required to gather all report data.
Query all those events, republish them to the endpoint that handles updating the new report table(s).
Wait until all events have been processed.
Put some indexes on the columns that will function as report query criteria.
Done!
Related
I have a report on PowerBI that has many pages/tabs and each one also has alot of data being displayed. As I didn't design this, I'm going through the report to eliminate as much as I can and possibly splitting the report as alot of the data only requries refreshing once a week.
This is where my query comes in, I have information on one page that requires a refresh every two hours over a 12 hour duration, one field of data that requires a daily refresh and two more fields only require refreshing when required.
Is it possible to segment scheduled refreshes throughout a single part of the report, or does scheduled refresh only allow the entire report to be refreshed? (I.E. Sales status is hourly, Outbound status is daily, and sales summary is weekly)
I'd rather avoid having to split reports, as it is very handy to have them on one page; rather than having to open two and view them on multiple monitors.
I am just starting out on PowerBI reports, having been shown enough to get what I need done; but plan to delve further in, this being my first port of call if it is possible.
Thanks for any reponses in advance.
Brian.
No. It's Not Possible.
PowerBI Internal working like Tabular Model.
In Import mode we can not do incremental refresh also.
So other option is you can create Reporting layer and define denormilized with calucaluated columns Reporting tables.( Sales ,summary )
and use Direct query or Refresh and Do ETL for This table.
So you can schedule ETL for specific Tables i.e.Sales or summary.
Currently I have a pricerule that has only one action element to fetch a price from pricelist.
In order to achieve this pricerule I'm adding entries into required tables like
PRICERULE,
PRELEMENT,
PRELEMENTATTR
and other tables.
Now I need to add more conditions and branches to this pricerule in order to fit for the requirements(something like this).
But I found forming this pricerule by inserting entries directly into tables (as I did for simple pricerule) is quite complex. Because after forming the pricerule it has to be updated on weekly basis. Updates will be like changing the markup/markdown percentage or changing the start and end date of this markup etc.
So my question is:
Instead of directly updating the tables, is there any IBM WCS OOB functionality to achieve this?
I've a problem with the dashboard in Tableau. In the dashboard there are many worksheets, and all the columns that are in the report are calculable. The problem is that dashboard is being formed for a very long time. The report contains approximately 2 million rows. And it is generated about 5 minutes.
Tell me, what are the solutions in this case?
Maybe I can somehow adjust the page display and not all the records at once?
To reduce the calculation time, try to exclude data you don't need with a data source filter in tableau. You can also hide or delete unused calculated fields. Other things you can do is reduce sheets that are not used.
Here's a link: https://www.tableau.com/about/blog/2016/1/5-tips-make-your-dashboards-more-performant-48574
Steps to follow to reduce calculation time:
Extract the data and use Extract data and also keep option as extract instead of live.Also replace the data source using extract data.
Use "User Filter" to reduce calculation time so that tableau will display of particular user data only.
I hope this will work to solve your problems.
I have one more idea to resolve this issue.
1)when you loan first time your dashboard put into Dashboard Action Filter
First Time load dashboard data exclude in your sheet.
Dashboard Menu->Action->add action->select sheet and exclude option.
2) Live to Extract data source and select radio button extract.
3)use user filter.
I am following the other answers (use extract, dashboard action filter...) and I want to add one point:
Drag every field used by any tablesheet on the dashboard on "Detail" of every tablesheet you are using on the Dashboard. Now Tableau loads all needed data while loading the first tablesheet and can use this data for the other sheets.
i.e. A dashboard contains three tablesheets (A, B, C) now you drag every field used by A on "Deatil" of B and C, every field used by B on "Deatil" of A and C, every field used by C on "Deatil" of B and A.
We are also having a similar issue with 150 million rows but I want to check if you are doing following steps. This may help you. This goes back to fundamentals of Tableau reporting.
1/ Try to make sure your data set is in star schema format. This will help a lot in report.
2/ Try to have tables and views in DB in such a way that same columns are used in Tableau. Any extra columns in tables adds to the performance issue.
3/Make sure indexing is done properly for all the fields that are joined.
4/ In my experience Dashboard adds extra performance lag. So make sure you try to get as much performance tuning on sheets as possible before even going to dashboard.
5/ If required try to use materialized views.
hope this helps.
Try to capture performance metrics using performance recorder option in Tableau.
Check for the underlying DB tables and joins present on the data source layer.
Try using optimized sets and parameters as required and get rid of less relevant filters.
Try using data extracts with scheduled refresh with data source filter for limited business years data.
For PowerApps, what data source, other than SharePoint lists are accessible via Powershell?
There are actually two issues that I am dealing with. The first is dynamic updating and the second is the 500 item limit that SharePoint lists are subject to.
I need to dynamically update my data source, which I am currently doing with PowerShell. My data source is not static and updating records by hand is time-consuming and error prone. The driving force behind my question is that the SharePoint list view threshold is 5,000 records however you are limited to 500 visible and searchable records when using SharePoint lists in the Gallery View and my data source contains greater than 500 but less than 1000 records. If you have any items beyond the 500th record that should match the filter criteria, they will not be found. So SharePoint lists are not optional for me until that limitation is remediated
Reference: https://powerapps.microsoft.com/en-us/tutorials/function-filter-lookup/
To your first question, Powershell can be used for almost anything on the Microsoft stack. You could use SQL server, Dynamics 365, SP, Azure, and in the future there will be an SDK for the Common Data Service. There are a lot of connectors, and Powershell can work with a good majority of them.
Take note that working with these data structures through Powershell is independent from Powerapps. Powerapps just takes the data that the data connector gives it, and if you have something updating the data in the background (Powershell, cron job, etc.), In order to get a dynamic list of items, you can use a Timer control and a Refresh function on your data source to update the list every ~5-20 seconds.
To your second question about SharePoint, there is an article that came out around the time you asked this regarding working with large lists. I wouldn't say it completely solves your question, but this article seems to state using the "Filter" function on basic column types would possibly work for you:
...if you’d like to filter the set of items that you are showing in the gallery control, you will make use of a “Filter” expression, rather than the “Search” expression, which is the default that existing apps used. With our changes, SharePoint connector now supports “equals” type of queries on columns that support filtering (Single line of text, choice, numbers, dates and people), so make sure that the columns and the expressions you use are supported and watch for the same warning to avoid reverting back to the top 500 items.
It also notes that if you want to pull from a list larger than the 5k threshold, you would need to use indexes, I have not fully tested this yet but it seems that this could potentially solve your problem.
A little background:
I have 2 entities (Product and Case). The product entity will hold all product records. A section in the Case will have the ability to choose products and auto-populate all related fields that are located in the product record for that specific product. For example, Product record has fields like hazardous, range, lot ect. The same field appear on the Case record. These fields should only be populated based on the product that was selected.
I was able to accomplish the above by creating a 1:N relationship and adding it to my Case form. I then created a workflow to populate the related fields (hazardous, range, lot ect). However, these fields only populate when the record is saved. Is there a way to make it update the fields once the product is chosen?
I want to refrain form using any type of JavaScript. If possible, I would like to strictly use workflows to accomplish this (if at all possible).
Real time information in your case can be only accomplished by using JavaScript. Maps works too but they have a special behavior.
Workflows that fire when the record is created only execute after all core operations are done (Native logic, Plug-in logic...) and you can't fire workflows if the record is not created.
So using workflows is a good idea even if you can't see the information