Displaying web based visualisation or graphing of data based on a postgresql database? - postgresql

I am working on a web application for a client that uses a postgresql database. I want the client to be able to go to a certain area of the site where the data from the database is displayed in graph form (for example, sales figures over a 6 month period). Is there a plugin I could use for this (I don't have any experience of this, so an easy one, or one with tutorials available would be great). I had a look at BIRT, which says it has a web based option but I couldn't really figure it out. I don't want the client to have to download and go through another program, I just want them to go to a url within their site, and it's all just presented to them there and then.
Any sort of pointers in the right direction would be greatly appreciated.
Thank you.

HighCharts, at http://www.highcharts.com/, works well for this case -- I use it fairly often. It supports Ajax data feeds in JSON format, so you can write an endpoint which returns the JSON representing the data from Postgres and which gets called from a JavaScript function which creates the graphs using that data (you would place that call in a ready function).
Also, if you're using Postgres 9.3 or higher, it supports JSON natively, so you can do the JSON conversion in the SQL query itself, as opposed to post-processing the results in your Python or other backend code.
Highcharts is reasonably flexible and allows for a variety of nice-looking, functional charts and graphs. If you want to get much fanicer, d3 may be worth a look. These are some the types of graphs/charts it can do: https://github.com/mbostock/d3/wiki/Gallery
I have not used d3 myself, however.
For the scenario you described above, Highcharts seems like it would work just fine.

It's been a while, and a lot has happened since 2016. There is now ChartJS as well - http://chartjs.org/, for example, which is easier to use than HighCharts and very flexible (I've used both).
What they both don't do is dynamic data. If you want that your client decides which data he wants to watch - that part you need to write yourself.

Related

Is it possible to create visualizations(charts) on our own using tableau?

The time taken to load the charts(visualization) using tableau in my mobile app is close to 10 or 15 secs. So i am looking for alternative methods like creating my own chart but only to use the intelligence tableau provides on creating the visualizations.
In short, i want to know whether there is some support or API available from tableau using which i can draw the visualizations by myself in my mobile app without losing any chart data or functionality like digging deeper in charts.
For example, an API from tableau using which i can create the chart in my mobile app. After creating it, if user touches any data that should be sent again to tableau to get further event action like showing filters/tables etc.
Is this even possible? I did lot of research in tableau and got to know about tableau SDK, tableau API's, tableau extensions etc. but unable to find what i am looking for.
Tableau provides a thorough white paper Designing Efficient Workbooks. I would start by reading that and trying some of the recommendations.
If you are looking for an API driven visualization system, you can check out MuzeJS.
You load your data in an in-browser DataModel, run relational algebra enabled data operators to get the right subset of data, and then just pass to Muze engine, which automatically renders the best visualization for it.
It is similar to the kind of intelligence Tableau provides but since it is a developer first API, you can customize as you want, whether it be for the mobile or the desktop. In fact, most of the charts will be available for both seamlessly.
Any changes to data (including application of data operations) automatically updates the visualization, without you having to do anything else.
Add to that, if you’ve to connect multiple charts (for cross-interactivity, drill-down etc.), you’ve to manually write the ‘glue’ code. With Muze, all charts rendered from the same DataModel are automatically connected (enabling cross-filtering). It also provides composability and the ability to facet your visualization, providing a multi-grid layout.
You can go through some of the examples and the documentation to see if it suits your needs

Postgres to ElasticSearch data indexing for ELK Stack

So I know the JDBC Rivers plugin is deprecated so even though it is being used I'd ideally not want to look at using something that is no longer supported.
However I have a few tables in a Postgres database with values that I need to be able to search in a Kibana view. Im new to the ELK stack but i've been messing around with some of their samples to get familiar.
I've seen some mentions of using Stored Procedures/Triggers from Postgres to send to Logstash. Although im not sure if this is the best way. Im not a developer but a QA so my coding skills are "ok" as im used to writing automation tests/etc...
What would be the best way to do this? I would want to probably capture updates to these tables (probably new inserts or updates) OR be able to poll the data every X period of time (30s or something). Lets pretend it's for a weather station and the tables contain humidity data from different weather sensors.
I'd want to be able to search in a Kibana view the Values/Station ID/etc...
Is this doable? Is there maybe a better way than using Triggers/Stored procedures?
I ended up using the JDBC driver and following https://www.elastic.co/blog/logstash-jdbc-input-plugin to get it moving and working (Which it does move). But it was a lot of setup for anyone that may see this answer.

Display function graph on the web

I would like to display a couple of function graphs in a web browser. That is, I have a few functions, like distribution or density functions, regularly generated by data analysis tools and I'd like to make those accessible via a web browser. They should be rendered as line charts, actual values displayed e.g. in a tooltip when hovering over the chart. The ability to zoom in would be nice but not essential. Also, a nice dashboard like arrangement would be nice. Now I have the data (each function represented as a list of floats), how do I best render them?
Generating static images might be an option, but they're not interactive at all. I see there are javascript frameworks to display charts, but I have the feeling there must be some ready-made solution out there to grab numbers from a file, or DB, or wherever, and show functions; hence, I don't want to start writing my own javascript. I thought about Grafana, but that's time series oriented and my data is not time series. I thought about importing the data into Google spreadsheets and create charts there programmatically, but I'd like to have at least a few thousands data points per chart and am not sure if that would work out performancewise.
What do you suggest?
use the simplest php Graph library. PHPMYGRAPH.
http://phpmygraph.abisvmm.nl/

Umbraco: Storing spatial data

I'm researching Umbraco for use as a base in a large CMS project, however the project calls for the SQL Server 2008 database to store spatial data against content.
Being new to Umbraco I'm still reading through the documentation and slowly building up an idea of it's architecture. However so far it doesn't look like Umbraco supports the storage of spatial data.
There only appears to be four database datatype options: date, integer, ntext, nvarchar
Is it possible to store spatial data to the database?
Update: Futher research into how Umbraco works has showed me I was on the wrong track. It seems the way to do this is store the lat/long data in the data inside the usual XML format Umbraco uses.
Then to use the Spatial.net extensions that have been built on top of Lucene.net, rather than use the limited search capabilities Examine exposes.
However this is all still theoretical, I've just not been able to achieve this. If I do before someone answers this question, I'll post my findings here to help others.
You could take a look at how to make user controls (with Visual Studio) in Umbraco.
It is also possible the versatility in Umbraco 'Document Types' is enough for you.
It is possible to extend Umbraco in any sort of way to get the solution you want. I don't know how you want the spatial data to interact with your frontend - so it is difficult to provide a direct solution.
Although there are ways to store spatial data and perform queries against it using Spatial.net, it's not a very elegant solution.
Instead I've created an additional table in SQL Server 2008 with the geometry/geography datatype and a reference to the Umbraco content it's connected with.
I've then got a event hook which updates this whether content is added/updated/deleted.

JasperReports and custom data sources

I'm looking at embedding JasperReports into an existing web app for reporting. The webapp sits on top of an existing database which is ancient and complex, and really not suitable for report writers to use to write reports against directly.
What I want to look at is writing some kind of wrapper around our existing data access layer (written to make our life easier talking to aforementioned ancient and complex db). Does anyone have any experience of writing custom data sources for JasperResports, or of doing anything like this?
Updated
I guess I probably wasn't clear in my question - which is probably because my requirements aren't clear either. I want to provide some way that the end-users can use something like iReport to author reports against the database, and then to use JasperReportServer for scheduling/viewing of the reports. However, the database is really, really nasty and was never designed for use in this way. We've got a access layer around it that the webapp uses to talk to it. I want to keep my end users away from the DB altogether, and the idea of a custom data source that used the access layer seemed a good option. However, I've found very little documentation on how to do that. Maybe it's just a whole lot easier than I think it is, and I'm just trying to make a dead simple thing too complicated.
Updated
Thanks for the answers. I don't think my problem has been solved, but I think the answers have helped to inform the requirements phase.
Jasper reports allows you to use a "JavaBean" data source. You can load your data into any Java Bean structure and build the reports against that. Works well.
See the "Custom Data Source" section here.
Every JasperReports template can have two different data sources. One is hooking it directly to a database using some jdbc driver or, in your case, providing a collection of java beans (POJO's), usually list.
JasperReports template is similar to a method definition. It has a name, i.e. compiled JR object and parameters (data source and a list of input parameters of some of the most popular Java types).
My suggestion is to use iReport tool. Open some example that comes with the JasperReports bundle, analyze it and tweak it. It's not so complicated.
UPDATE
Letting customers authoring JasperReports templates, compiling and adding to the classpath means that you'll need to open your system too much. Usually clients provide description of a desired report and developer(s) create the data source and design the template. JasperReports can have parameters. If these parameters are exposed through UI users can change the behavior of reports in the runtime.
If you really need to allow more flexibility then use the API provided by JasperReports for authoring templates. I could imagine some simple DLS for advanced users to communicate with your system creating on-fly reports.