I have a postgres data-source in Grafana that's normalized which restricts my graph-visualization legend to show only the ID (hash) of my record. I want to make this human-readable but the id -> name mapping is in a different datasource/postgres database.
Grafana supports templating-variables which I think could allow me to load my id -> naming reference data but there isn't clear documentation on how to access the label_values as a reference-table within the postgres driver's query editor.
Is there a way to configure the template variable to load reference data (id -> name) & leverage it to translate my metric/legend ids within the grafana postgres driver?
For Example (pseudo-grafana postgres query editor):
SELECT
$__timeGroupAlias(start,$__interval),
animal_names.__value AS metric,
count(dog.chewed_bones) AS “# bones chewed“
FROM animals.dog dog
JOIN $TEMPLATE_VAR_REF_DATA animal_names ON dog.id = animal_names.__text
WHERE $__timeFilter(start_time)
GROUP BY 1,2
ORDER BY 1,2
Closest answer I found is here but doesn't get into details:
johnymachine's comment # https://github.com/grafana/grafana/issues/1032
I realized the github comment meant use a jsonb aggregate function as a variable like in the following solution:
Dashboard Variable (Type Query): select jsonb_object_agg(id,name) from animal_names;
Grafana Postgres Pseudo-Query:
SELECT
$__timeGroupAlias(start,$__interval),
animal_names::jsonb ->> dog.id::text AS metric,
count(dog.chewed_bones) AS “# bones chewed“
FROM animals.dog
WHERE $__timeFilter(start_time)
Related
I have an azure pipeline that moves data from one point to another in parquet files. I need to join some data from a Postgresql database that is in an AWS tenancy by a unique ID. I am using a dataflow to create the unique ID I need from two separate columns using a concatenate. I am trying to create where clause e.g.
select * from tablename where unique_id in ('id1','id2',id3'...)
I can do a lookup query to the database, but I can't figure out how to create the list of IDs in a parameter that I can use in the select statement out of the dataflow output. I tried using a set variable and was going to put that into a for-each, but the set variable doesn't like the output of the dataflow (object instead of array). "The variable 'xxx' of type 'Array' cannot be initialized or updated with value of type 'Object'. The variable 'xxx' only supports values of types 'Array'." I've used a flatten to try to transform to array, but I think the sync operation is putting it back into JSON?
What a workable approach to getting the IDs into a string that I can put into a lookup query?
Some notes:
The parquet file has a small number of unique IDs compared to the total unique IDs in the database.
If this were an azure postgresql I could just use a join in the dataflow to do the join, but the generic postgresql driver isn't available in dataflows. I can't copy the entire database over to Azure just to do the join and I need the dataflow in Azure for non-technical reasons.
Edit:
For clarity sake, I am trying to replace local python code that does the following:
query = "select * from mytable where id_number in "
df = pd.read_parquet("input_file.parquet")
df['id_number'] = df.country_code + df.id
df_other_data = pd.read_sql(conn, query + str(tuple(df.id_number))
I'd like to replace this locally executing code with ADF. In the ADF process, I have to replace the transformation of the IDs which seems easy enough if a couple of different ways. Once I have the IDs in the proper format in a column in a dataset, I can't figure out how to query a database that isn't supported by Data Flow and restrict it to only the IDs I need so I don't bring down the entire database.
Due to variables of ADF only can store simple type. So we can define an Array type paramter in ADF and set default value. Paramters of ADF support any type of elements including complex JSON structure.
For example:
Define a json array:
[{"name": "Steve","id": "001","tt_1": 0,"tt_2": 4,"tt3_": 1},{"name": "Tom","id": "002","tt_1": 10,"tt_2": 8,"tt3_": 1}]
Define an Array type paramter and set its default value:
So we will not get any error.
I’m trying to suss out how to format my key/value pair dashboard variable. I’ve got a variable whose definitions are:
sensor_list = 4431,8298,11041,13781
sensor_kv = 4431 : Storage,8298 : Stairs,11041 : Closet,13781 : Attic
However, I can't seem to use it effectively for queries and dashboard formatting with InfluxDB. For example, I've got a panel whose query is this:
SELECT last("battery_ok") FROM "autogen"."Acurite-Tower" WHERE ("id" =~ /^$sensor_list$/) AND $timeFilter GROUP BY time($__interval) fill(null)
That works, but if I replace it with the KV, I can't get the value:
SELECT last("battery_ok") FROM "autogen"."Acurite-Tower" WHERE ("id" =~ /^$sensor_kv$/) AND $timeFilter GROUP BY time($__interval) fill(null)
^ that comes back with no data.
I'm also at a loss as to how to access the value of the KV pair in, say, the template values for a repeating panel. ${sensor_kv:text} returns the word "All" but ${sensor_kv:value} actually causes a straight up error: "Error: Variable format value not found"
My goal here is twofold:
To use the key side of the kv map as the ID to query from in the DB
To use the value side as the label of the stat panel and also as the alias of the measurement if I'm querying in a graph
I’ve read the formatting docs and all they mention are lists; there are no key/value examples on there, and certainly none that do this. It’s clearly a new-ish feature (here is the GH issue where its implementation is merged) so I’m hoping there’s just a doc miss somewhere.
In PR that you linked there is a tiny comment that key/value pair has to contain spaces.
So when you're defining a pairs in Values separated by comma it should be like
key1 : value1, key2 : value2
These will not work
key1:value1, key2:value2
key1 :value1, key2 :value2
key1: value1, key2: value2
Let's say that name of the custom variable is var1
Then you can access the key by ${var1} ,$var1, ${var1:text} or [[var1:text]]
(some datasources will be satisfied with $var1 - some will understand only ${var1:text})
And you can access the value by ${var1:value} [[var1:value]]
Tested in Grafana 8.4.7
I realise this might not be all the information you're after, but hope it will be useful. I came across this question when trying to implement something similar myself (also using InfluxDB), and I have managed to access both keys and values in a query
My query looks like this:
SELECT
"Foo.${VariableName:text}.Bar.${VariableName:value}"
FROM "db"
WHERE (filters, filters) AND $timeFilter GROUP BY "bas"
So as you see, my use case was a bit different from what you're trying to achieve, but it demonstrates that it's basically possible to access both the key and the value in a query.
Key/values are working with some timeseries DB where it makes sense, e.g. MySQL https://grafana.com/docs/grafana/latest/datasources/mysql/:
Another option is a query that can create a key/value variable. The query should return two columns that are named __text and __value. The __text column value should be unique (if it is not unique then the first value is used). The options in the dropdown will have a text and value that allows you to have a friendly name as text and an id as the value.
But that's not a case for InfluxDB: https://grafana.com/docs/grafana/latest/datasources/influxdb/ InfluxDB can't return key=>value result - it returns only timeseries (that's not a key=>value) or only values or only keys.
Workarounds:
1.) Use supported DB (MySQL, PostgreSQL) just to have correct key=>value results. You really don't need to create table for that, just combination of SELECT, UNION, ... and you will get desired result.
2.) Use hidden variable which will be "translating" value to key, which will be used then in the query. E.g. https://community.grafana.com/t/how-to-alias-a-template-variable-value/10929/3
Of course everything has pros and cons, for example multi value variable values may not work as expecting.
In grafana documentation there is information about global variables, but when i try using them in my query (I am using mysql database), they don't get translated. Why is that?
An example:
For query
SELECT
'$__dashboard'
Gives in generated query:
SELECT
'$__dashboard'
But when I do select on variable defined by myself in the editor it works just fine.
I have a simple Custom variable called route with e.g. this value:
/foo/bar,/foo/baz,/foo/baz/foo
I'm trying to map these values to some more understandable values, e.g. Custom route_names:
bar,baz,foo
Searching on google resulted in people doing nested variables, but whatever I try in Grafana 5.3.4, I can't get it to work. If I do a Query variable and use -- Grafana -- as source, I don't know what to put in the query field. route.* didn't do anything, $route neither.
What is the correct way of selecting a value from one variable and map it to the other? I.e. What is the query language being used when selecting -- Grafana -- as datasource?
As a side note, I have two datasources at the moment, my actual data source where I get my graph data from and -- Grafana --.
There are correct answers on the first floor. solve "key/value pairs" by SELECT 'txt1' AS __text, 'value1' AS __value UNION SELECT 'txt2' AS __text, 'value2' AS __value
This is not possible with Custom template variables (unless smth changed in recent Grafana versions). It can be done with variables coming from mysql, postgres and clickhouse datasource queries. See examples in https://community.grafana.com/t/key-value-style-for-custom-template-variable-configuration-and-usage/3109 thread. Can't tell about this feature support in other datasource types.
OrientDB manual shows how to obtain CLUSTER information using the console.
For example, to obtain a list of Clusters, i would do:
orientdb> LIST CLUSTERS
But, i cannot find how to obtain the same information using SQL.
Is there a way to obtain a list of all the Clusters using SQL?
(I'm using OrientDB 2.2.29).
You can use something like:
select name as NAME, clusterIds as ID from (select expand(classes) from metadata:schema)