We are building our new Next generation server for a medium sized back office application.
We already decided we would like to use a java framework for the client side (gwt \ vaadin \ zkoss)
What we would like now is to create a Proof Of Concept example of each technology.
our back office ui is pretty standard, we have tables \ grids with filters that should show entries straight from the DB.
Problem is we got huge amount of rows in each table (1M minimum)
which mean that we must use a load on demand tables for them.
My questions is: how do i implement a load on demand table for my big tables? I looked around and saw the following concept again and again:
you create a container, you populate it with data, the data is being displayed on the client side.
problem is i tried this naive way to populate the containers with 1M entries and it was awful. are there any built in on demand containers?>
any code examples \ references will be a huge help!
You would want to use GWT Cell Table, which has the AsyncDataProvider, that lets you handle the user's paging and sorting events by grabbing data from your server.
It also provides an alternative ListDataProvider, which lets you grab your data as a list of objects, then set that data to your table. If you use ListDataProvider, you have to define how to sort your objects with Comparators, and table will handle sorting and paging against that list.
Google "gwt celltable asyncdataprovider example" for more examples and tutorials.
Vaadin has a nice concept of lazy loading data in most of the components.
For example the table, list, dropdown's etc. have that concept.
The only thing you realy need to know at start, is the number of total rows.
Everything else can then be handled "ondemand".
For example the Table component initially only loads about 30 rows (can be customized)
and then fetches rows as needed. (Or better they are usually fetched just before the user scrols to the next rows)
A example is this demo
http://demo.vaadin.com/dashboard/#!/transactions
How you retrieve the data from your backend depends on the technology used.
But vaadin has working concepts where you don't need to load all 1mio. rows into memory,
it will handle the "fetch on demand" as the rows need to be displayed.
Related
I want to know if we can do combination filtering in ag grid. Some volumn filtering on client and some on server. is that possible?
I was checking adaptabletools website they have built similar feature with serverOptions.link below. I was trying to achieve similar thing via ag-grid api. Can you please advise
https://api.adaptabletools.com/interfaces/_src_adaptableoptions_searchoptions_.searchoptions.html
Update on this question as I developed AdapTable which the OP refers to in her question.
We DO enable and facilitate server-side searching, sorting and filtering while keeping ag-Grid in ClientSideRowModel mode and many of users take huge advantage of it.
You can learn more at:
https://docs.adaptabletools.com/docs/key-topics/server-functionality
However note that this is suited to the use case where you have a few hundred thousand rows and want to have the best of both worlds; if you have millions of rows of data that needs searching and filtering then you should use ag-Grid's Server or Infinite Row Models (both of which AdapTable fully supports but in different ways to that mentioned in the OP).
Well Client-Side RowModel is the default. The grid will load all of the data into the grid in one go. The grid can then perform filtering, sorting, grouping, pivoting and aggregation all in memory.
The Server-Side Row Model builds on the Infinite Row Model. In addition to lazy-loading the data as the user scrolls down, it also allows lazy-loading of grouped data with server-side grouping and aggregation. Advanced users will use Server-Side Row Model to do ad-hoc slice and dice of data with server-side aggregations.
Ideally developer should choose either of them. Also AG Grid doesn't allow any method to set RowmodelType type programmatically.
So the simple answer is no it can't be done easily.
But I think you can do some workaround by creating another hidden AG Grid which will be created with RowmodelType = 'client side'. update the data in this 2nd grid whenever data changes in the first grid. also switch between grid(using hide show logic) when user wants to filter on client side(may be you can provide a radio button for that) and you can set filterstate/columnstate etc.. settings from 2nd to 1st grid.
I've written a report that displays all of the possible combinations between two lists (it's a grid with columns from one list, and a grid of columns from another list). It forms a spreadsheet like structure and places an X in cells where the two lists have a many-to-many relation.
Of course it's paired down with categories and there is a limit to the number of items that can be in a category, as these sorts of queries are frowned upon for a good reason since the number of items in both lists multipled by one another result in the number of possible cells that fall between. Also note that this is a small database, and not one that's to be used by many people at one time (SQLite3).
I've already got the logic to display it, by querying the tables, and their weak entity, the output even comes back out as expected; but now I'm trying to clean up the code, and wondering where to place the logic for this given that I'd like to use a REST API between the database and the web frontend.
Should the front-end query the REST API for the necessary values, and then string them together into a report, or should the REST API do that and just return JSON for the table in the report to the front end so that it can be processed into an HTML table? It also seems to me that maybe I need a backend API layer that hooks into REST API to keep that decoupled as well, but it almost feels like too much.
I have a case where my DataGrid might contain thousands of rows of data. My page size is only 50. So I want to get only so much data onto the client from server and load rows of data as and when required. Is there a default support from GWT to do that?
I tried using PageSizePager but then realized that the data is already sent to the client and that defeats what am trying to achieve.
Thanks.
Indeed. The aim (among others) of DataGrid, as well as of any other cell widget, is to display large data sets as fast as possible by using pagination and cells.
I suggest you to start with the official docs about CellTable (same applies to DataGrid), and pagination/data retrieval.
You'll probably need an AsyncDataProvider to asynchronously fetch data and an AbstractPager (say, SimplePager) to force retrieval of more data.
You can also decide to extend the current range of rows (say, infinite scroll), instead having multiple pages (by using KeyboardPagingPolicy.INCREASE_RANGE).
In general, the question is too general and can be hardly answered in few lines (or without specify more). The bits are all in the links above, and don't forget to have a look at the cell widgets samples in the showcase, as they cover almost everything you will need.
Hope to get you started.
I must do a RPC call when my Presenter is revealled. That call result in a String[] with large amount of data. But this call is very very slow. It takes about 1min to finish.
After some tests, i discovered that ListBox.addItem() takes over 30% of this call. It's a huge time for just add String on that Widget.
What can I do to minimize this time?
Assuming that i need to load everything when my Presenter reveal.
Things that I've already done:
Put my Query inside a View(Doesn't affect too much)
Server read a Txt file instead of call DB(worst then View).
Use Collections classes ArrayList,Vector...(Vector reduced time by 5%)
I've noticed that GWT designed a LightweightCollections to improve use of Collections on Client side(It's my next step).
But what can I do about ListBox?
Too much choice is no choice.
You will not be able to tune up GWT Listbox/ValueListBox for purpose of displaying such huge amount data ( i am guessing entries in 1000's considering 20 seconds i.e 30% of 1 min). GWT Listbox is meant for selection. You cannot expect user to see 1000's of values , scroll and then select. Its a User Interaction nightmare.
The right approach is use to Async loaded SuggestBox for such huge data. With SuggestBox you can filters and display lesser data as choice based on users input keys.
If using SuggestBox is not feasible you must give a try for CellList in Cell Widgets ( they might show better performance ) - https://developers.google.com/web-toolkit/doc/latest/DevGuideUiCellWidgets
I am not sure but give GWTChosen a try - http://jdramaix.github.com/gwtchosen/
What are some possible designs to deal with frequently changing data forms?
I have a basic CRUD web application where the main data entry form changes yearly. So each record should be tied to a specific version of the form. This requirement is kind of new, so the existing application was not built with this in mind.
I'm looking for different ways of handling this, hoping to avoid future technical debt. Here are some options I've come up with:
Create a new object, UI and set of tables for each version. This is obviously the most naive approach.
Keep adding all the fields to the same object and DB tables, but show/hide them based on the form version. This will become a mess after a few changes.
Build form definitions, then dynamically build the UI and store the data as some dictionary like format (e.g. JSON/XML or maybe an document oriented database) I think this is going to be too complex for the scope of this app, especially for the UI.
What other possibilities are there? Does anyone have experience doing this? I'm looking for some design patterns to help deal with the complexity.
First, I will speak to your solutions above and then I will give my answer.
Creating a new table for each
version is going to require new
programming every year since you will
not be able to dynamically join to
the new table and include the new
columns easily. That seems pretty obvious and really makes this a bad choice.
The issues you mentioned with adding
the columns to the same form are
correct. Also, whatever database you
are using has a max on how many
columns it can handle and how many
bytes it can have in a row. That could become another concern.
The third option I think is the
closest to what you want. I would
not store the new column data in a
JSON/XML unless it is for duplication
to increase speed. I think this is
your best option
The only option you didn't mention
was storing all of the data in 1
database field and using XML to
parse. This option would make it
tough to query and write reports
against.
If I had to do this:
The first table would have the
columns ID (seeded), Name,
InputType, CreateDate,
ExpirationDate, and CssClass. I
would call it tbInputs.
The second table would have the have
5 columns, ID, Input_ID (with FK to
tbInputs.ID), Entry_ID (with FK to
the main/original table) value, and
CreateDate. The FK to the
main/original table would allow you
to find what items were attached to
what form entry. I would call this
table tbInputValues.
If you don't
plan on having that base table then
I would use a simply table that tracks the creation date, creator ID,
and the form_id.
Once you have those you will just need to create a dynamic form that pulls back all of the inputs that are currently active and display them. I would put all of the dynamic controls inside of some kind of container like a <div> since it will allow you to loop through them without knowing the name of every element. Then insert into tbInputValues the ID of the input and its value.
Create a form to add or remove an
input. This would mean you would
not have much if any maintenance
work to do each year.
I think this solution may not seem like the most eloquent but if executed correctly I do think it is your most flexible solution that requires the least amount of technical debt.
I think the third approach (XML) is the most flexible. A simple XML structure is generated very fast and can be easily versioned and validated against an XSD.
You'd have a table holding the XML in one column and the year/version this xml applies to.
Generating UI code based on the schema is basically a bad idea. If you do not require extensive validation, you can opt for a simple editable table.
If you need a custom form every year, I'd look at it as kind of a job guarantee :-) It's important to make the versioning mechanism and extension transparent and explicit though.
For this particular app, we decided to deal with the problem as if there was one form that continuously grows. Due to the nature of the form this seemed more natural than more explicit separation. We will have a mapping of year->field for parts of the application that do need to know which data is for which year.
For the UI, we will be creating a new page for each year's form. Dynamic form creation is far too complex in this situation.