I need to create a dashboard to be used in a control room, where a bunch of operators will need to monitor the number of tasks assigned to other employees (among other aspects).
Source data will be coming from a RDBMs (PostgreSQL, in this case). We have people with assigned and numbered tasks that also have a status, and the DB data is like this (purely fictional: but it resembles the real one)
Having to create and mantain a dashboard i was thinking to use tools like Grafana, Kibana or similars, to plot something like this
The problem is that Grafana, for example, doesn't let me use alphabetical values for the x-axis. It only allow numeric values, while i've names to plot (Mark, Luke, Brian).
Is there a best practice than i can follow? Am i trying to use the wrong tools?
Actually solution is easier then you think although it also took me some time to figure it out. I will place here an example for some unspecified shop data grouped over countries - you just need to change it for your task. Example was tested on Grafana 5.0.3
PostgreSQL query for metrics
SELECT
$__time( partition_date ),
country as metric,
sum(value) as value
FROM
aggregations.my_data_for_dashboard
WHERE
shop = 'myshopname' AND
$__timeFilter(partition_date )
group by 1, 2
Grafana will show usual metrics:
In "Axes" tab look at "X-Axis" section, item "Mode" - switch "Time" to "Series" and Grafana will show bar chart for countries.
Related
I created a SQL query that counts the number of servers running test jobs on a specific Jenkins server at a specified time. I'm trying to chart it on Grafana but for some reason, it's displaying the value as NaN.
The data source is a MySQL server. I'm running Grafana version 8.1.5. I went on the server (phpMyAdmin) to check the results of the query and I can see numbers.
When we look at the grafana chart/panel, the bars on the chart look like it matches the values, but the chart shows NaN instead of the value.
What setting do I need to change so that it can print the numeric value on the chart instead of displaying NaN?
EDIT: Looks like I didn't include enough info to my question.
Here's the query that's used, although, I don't know how feasible it is to include the tables with data to demo the issue:
SELECT COUNT(DISTINCT(tb_name)) as val, jenkins FROM (
SELECT
hw.collected_date AS mytime,
hw.jenkins AS jenkins,
hw.name AS tb_name
FROM hw_report as hw LEFT OUTER JOIN jenkins_owner jo
ON hw.jenkins = jo.jenkins
WHERE jo.org IN ('Enterprise Readiness')) as y
WHERE mytime = '2021-12-03 00:00:00'
GROUP BY jenkins
HAVING count(distinct(tb_name)) > '0'
Here are screenshots of my panel. The "Show values" setting is set to "Always", which II assume should show the values.
One thing I noticed is when I hover over the bar, it shows
COUNT(DISTINCT(name_cnt))
instead of a numeric value. Not sure if this is indicative of anything. I checked other charts in the dashboard that someone created and their bars either have a numeric value or it's just a column name (like name_cnt)
Sorry I'm a noob and didn't know what to configure, look up, or mention in my question. I accidentally came across the answer.
So it seems like I need to select something under the "Transform" tab for the panel. Calculation needs to be "Total" instead of "Count" and I need to hide the fields specified in my SQL query and show the "Total"
I use grafana to plot timeseries data. In a timeseries plot i want to add a constant line which comes from a monitoring level. The value of that level is dynamic (from a postgres database) the timeseries come from a ifluxdb Datasource.
The monitoring level have no timestamp. The result should look like this:
I have searched quite a while how to do this, but not found a good explanation.
You can now add thresholds (bottom of the edit screen).
Each threshold can be represented as a solid line and/or region with individual color
Another way to do it in a dirty way is to create panel with a mixed data source.
Create your variable in grafana - it can be a query, constant or custom. Just remember to keep it a single floating point.
Add your original query and add the prometheus data source to query your variable.
${net_ordered_storage}
You will have to play a little bit with the number of data points displayed (query options>max data points) and minimum step of data point in prometheus query to make grafana connect dots.
Green horizontal line from variable
To draw a line like that you have to "fake" a timeseries. (thresholds don't work since they can not be dynamic as far as I know)
First thing to keep in mind is that grafana needs timestamp to plot it, for this reason the global variables ${__to} and ${__from} come in handy.
Then, to draw a line, grafana needs at least two points. ([t0, t1][y0, y1])
So this is the sql (postgre) query that lead to the desired result:
SELECT
${__from} AS time,
level_1,
FROM my_table where display_name = '${my_grafana_var:raw}'
union all
SELECT
${__to} AS time,
level_1,
FROM my_table where display_name = '${my_grafana_var:raw}';
It is possible to add dynamic thresholds using the Config from query option.
Add a new query below your standard metric query. This will likely be called B.
Here you query the static reference value.
Eg SELECT expected_number_of_widgets FROM baseline
Open Transformation tab
Find Configuration from Query results
Choose Config query = B
At the dropdown for expected_number_of_widgets, choose Use as: Threshold1.
In the right panel, under Thresholds, make sure Show Tresholds is enabled and remove the default threshold of 80.
For more details, see https://grafana.com/docs/grafana/latest/panels-visualizations/query-transform-data/transform-data/#config-from-query-results
I have some data like this below
data image see link
I would like to make a dashboard that will show you all the related empires based on what you choose (those that existed at the same time AND those in one of it's regions of influence). For example if I choose Rome then it will only show Egypt, Greek and Gaul and not show Byzantine because it is from a later time and not show China because it is in a different region. See below
See expected result picture in link
The simple way to achieve this task is to "Self-Join"
I would self-join the data again with Inner join on Region and Era
then, to handle the duplicate rows I would create a calculation
[Empire_Data1] = [Empires_Data2]
and put as false in the filter shelf.
then if you drag both Empires field you will the output you are looking for,
Since this is like 20 rows of data, you can perform a self join without any challenge.
But you have a lot of rows as in hundreds of thousands or more then, you might want to prep your data before connecting to the tableau.
What is the best way to store metrics data used in displaying graphs?
Currently I have a table analytics(domain::text, interval_in_days::int, grouping::text, metric::text, type::text, labels[], data[], summary::json)
domain is the overall category of the metrics. Like what part of the application they're under. Could be sales or support etc.
the interval_in_days and grouping are 'view options' the end user can specify at the interface level to have a different view of the data points.
grouping can be date, day_of_week or time_of_day
interval_in_days can be 7, 30 or 90
labels is an array of the labels on the x-axis and data are the corresponding datapoints.
type is either data_series or summary. If data series, the row represent's the data used for drawing the graph, while a summary has the summary:json field populated with an object like {total_number_of_X: 132, median_X: 320.. etc}
metric is simply the metric the corresponding graph represents, so there's a separate graph for each value of metric
From this it follows that for each metric/graph I display, I have 9 (3 intervals * 3 groupings). For each domain I have a single row with type summary.
Every few hours I aggregate a lot of data across multiple tables into the analytics table. So I don't have to perform expensive queries adhoc.
I feel this is not the optimal approach, so I'm really interested in seeing how other people accomplishes the same task or any suggestions.
There is nothing wrong with storing 9 rows of raw data and later aggregating them to something more comfortable. It's a common approach and has performance benefits in some situations.
What I would really re-think in your design are the datatypes. From your description it seems you can transform all ::text fields into something like ::varchar(20). Then you can use STORAGE PLAIN on these columns and your table will become more efficient.
Also, consider adding foreign keys to describe what is stored in individual columns. For example, you stated grouping can be date, day_of_week or time_of_day, so you could have a groupings table that will list these options. But again, the foreign key would have to be covered by an index, so you may want to skip on that due to performance reasons.
Using IBM Bluemix I created an app, a Cloudant NoSQL DB, a dashDB and an Embeddable Reporting service. In dashDB I created a table with a couple of columns and some simple data. Next I configured the Embeddable Reporting service and pointed it to the Cloudant DB for its own storage and dashDB for reporting data. Next I open Report Studio and create a chart mapping in some data:
When I play the report page, I get an indication that I have not supplied data:
However if I create a different report and ask for a List ...
The list appears just fine ...
I am at a loss to understand why my chart will not appear but my list will. I will be happy to amend and update my question with any relevant information anyone may need.
Imagine a vertical column chart. Now imagine data of the form:
Dallas 10
New York 30
San Francisco 50
We can easily imagine the cities on the X-Axis and the values on the Y-Axis. This is easy enough. But now imagine that our X-Axis rows in our data are not unique ... for example:
West-Region 10
East-Region 30
West-Region 20
What then should the "value" of the West-Region column be? The column names should be unique and hence we can't have two columns with the same name. Should the value of the West-Region be 30 (the sum) or 15 (the average) or something else?
And that is where the problem comes in. When we define a column in a chart, there is no defined Aggregate Function. What we need to do is define how we want values to be aggregated together. If we select the column and select its properties, we can find an Aggregate Function option. We can choose a function such as Average.
Once defined, the chart will show up correctly because it can now properly handle aggregation. Now, this might seem strange especially if we know for certain that there is never a need for aggregation because values are unique ... but apparently, these are the rules (for better or worse) and, once set, charts now show: