How to encode table based data in Vega-Lite? - encoding

First of all, it is hard to describe what I exactly mean by "table based data", because in some way all the input data for vega is "table-ish", but this example should make it clear:
Most (if not all) of the Vega-Lite examples for multi line charts use data like,
"data": {
"values": [
{"id": 0, "symbol": "A", "value": 4},
{"id": 1, "symbol": "A", "value": 2},
{"id": 0, "symbol": "B", "value": 3},
{"id": 1, "symbol": "B", "value": 8}
]
}
which is simple to color the lines of A and B with an ecoding like this,
"mark": "line",
"encoding": {
"x": {"field": "id", "type": "quantitative"},
"y": {"field": "value", "type": "quantitative"},
"color": {"field": "symbol", "type": "nominal"}
}
But what if I want to produce the same result with a table based form of data like this,
"data": {
"values": [
{"id": 0, "A": 4, "B": 3},
{"id": 1, "A": 2, "B": 8}
]
}
1. How can I encode table based data into one colored multi line chart?
A basic encoding could be to create line charts for every field and layer them on top of each other like this,
"encoding": {
"x": {"field": "id", "type": "quantitative"}
},
"layer": [
{
"mark": "line",
"encoding": {
"y": {"field": "A", "type": "quantitative"}
}
},
{
"mark": "line",
"encoding": {
"y": {"field": "B", "type": "quantitative"}
}
}
]
But with this I don't know how to color the lines differently or how to create a legend.
2. Is this type of input data idiomatic to the way vega/vega-lite is designed?

The data that vega-lite works with is often known as "long-form" or "column-oriented" data. The type of data you're asking about is often known as "wide-form" or "row-oriented" data. This is discussed briefly in the documentation for Altair, a Python wrapper for vega-lite: https://altair-viz.github.io/user_guide/data.html#long-form-vs-wide-form-data
In the current release of Vega-Lite (v2.X) your only option is to modify the data source to be column-oriented with an external tool. This will change in the v3.0 release of Vega-Lite, which adds the Fold transform which is designed to convert row-oriented data to column-oriented within a chart specification.
So, in Vega-Lite 3, you could use the fold transform like this (vega editor link):
{
"data": {"values": [{"id": 0, "A": 4, "B": 3}, {"id": 1, "A": 2, "B": 8}]},
"transform": [{"fold": ["A", "B"]}],
"mark": "line",
"encoding": {
"x": {"field": "id", "type": "quantitative"},
"y": {"field": "value", "type": "quantitative"},
"color": {"field": "key", "type": "nominal"}
}
}

Another solution (a bit tedious) is to use layer and create n layers for n columns
{
"$schema": "https://vega.github.io/schema/vega-lite/v3.json",
"data": {"url": "data/seattle-weather.csv", "format": {"type": "csv"}},
"layer": [{
"mark": {"type": "line", "color": "orange"},
"encoding": {
"x": {"timeUnit": "yearmonthdate", "field": "date", "type": "temporal"},
"y": {"field": "temp_max", "type": "quantitative"}
}
}, {
"mark": {"type": "line", "color": "red"},
"encoding": {
"x": {"timeUnit": "yearmonthdate", "field": "date", "type": "temporal"},
"y": {"field": "temp_min", "type": "quantitative"}
}
}]
}
Future support for layer repeat (https://github.com/vega/vega-lite/issues/1274) may make this a more reasonable solution.

Related

How to create different record for every update in FIWARE-STH Comet?

I just started using FIWARE and I'm having trouble with the data I want to persist on MongoDB. I want to use STH Comet in minimal mode. I am using a Python script that creates my context data with a specific id and then updates certain attributes of this data. While I can't see any problem when I observe the JSON data from this script, and HTTP requests but MongoDB keeps only the last updated version of this data in the related collection. In addition, the "sth" labeled collections does not show up. However I can see the "credate" and "moddate" metadatas in attributes. But I need different records for every update.
Here is the first POST request that I send to Orion to create context data:
{"id": "urn:ngsi-ld:entity:001",
"type": "Log",
"a": {"type": "Datetime", "value": "17/09/2021"},
"b": {"type": "Datetime", "value": "12:00:18.0"},
"c": {"type": "Integer", "value": 49.51},
"d": {"type": "Integer", "value": 175.35},
"e": {"type": "Integer", "value": 24.25},
"f": {"type": "Integer", "value": 999.1},
"g": {"type": "Integer", "value": 85.87},
"h": {"type": "Integer", "value": -0.01},
"i": {"type": "Integer", "value": 37.41},
"j": {"type": "Integer", "value": 60.65}}
Here is what I get in MongoDB collection, attrNames has been made dummy for the question:
{
"_id": {
"id": "urn:ngsi-ld:entity:001",
"type": "Log",
"servicePath": "/"
},
"attrNames": [
"a",
"b",
"c",
"d",
"e",
"f",
"g",
"h",
"i",
"j"
],
"attrs": {
"a": {
"type": "Datetime",
"creDate": 1645514416,
"modDate": 1645514416,
"value": "17/09/2021",
"mdNames": []
},
"b": {
"value": "12:00:21.0",
"type": "Datetime",
"mdNames": [],
"creDate": 1645514416,
"modDate": 1645514597
},
"c": {
"value": 666.47,
"type": "Integer",
"mdNames": [],
"creDate": 1645514416,
"modDate": 1645514640
},
"d": {
"value": 175.55,
"type": "Integer",
"mdNames": [],
"creDate": 1645514416,
"modDate": 1645514597
},
"e": {
"value": 24.27,
"type": "Integer",
"mdNames": [],
"creDate": 1645514416,
"modDate": 1645514597
},
"f": {
"value": 999.28,
"type": "Integer",
"mdNames": [],
"creDate": 1645514416,
"modDate": 1645514597
},
"g": {
"type": "Integer",
"creDate": 1645514416,
"modDate": 1645514416,
"value": 85.87,
"mdNames": []
},
"h": {
"type": "Integer",
"creDate": 1645514416,
"modDate": 1645514416,
"value": -0.01,
"mdNames": []
},
"i": {
"type": "Integer",
"creDate": 1645514416,
"modDate": 1645514416,
"value": 37.41,
"mdNames": []
},
"j": {
"value": 1111.47,
"type": "Integer",
"mdNames": [],
"creDate": 1645514416,
"modDate": 1645514640
}
},
"creDate": 1645514416,
"modDate": 1645514640,
"lastCorrelator": "679e1cba-93b0-11ec-be0f-0242ac120102"
This is the data output I can obtain with STH Comet Query:
{
"contextResponses": [
{
"contextElement": {
"attributes": [
{
"name": "c",
"values": [
{
"_id": "621ca977165813000740898e",
"recvTime": "2022-02-28T10:52:39.012Z",
"attrName": "c",
"attrType": "Integer",
"attrValue": 49.47
},
{
"_id": "621ca97d165813000740899e",
"recvTime": "2022-02-28T10:52:45.581Z",
"attrName": "c",
"attrType": "Integer",
"attrValue": 49.61
},
{
"_id": "621ca98316581300074089a7",
"recvTime": "2022-02-28T10:52:51.239Z",
"attrName": "c",
"attrType": "Integer",
"attrValue": 49.46
},
{
"_id": "621ca98916581300074089ac",
"recvTime": "2022-02-28T10:52:57.662Z",
"attrName": "c",
"attrType": "Integer",
"attrValue": 49.57
},
{
"_id": "621ca98d16581300074089b9",
"recvTime": "2022-02-28T10:53:01.852Z",
"attrName": "c",
"attrType": "Integer",
"attrValue": 49.47
},
{
"_id": "621ca99316581300074089c7",
"recvTime": "2022-02-28T10:53:07.242Z",
"attrName": "c",
"attrType": "Integer",
"attrValue": 49.41
},
{
"_id": "621ca9a116581300074089d2",
"recvTime": "2022-02-28T10:53:21.305Z",
"attrName": "c",
"attrType": "Integer",
"attrValue": 49.51
}
]
}
],
"id": "urn:ngsi-ld:entity:001",
"isPattern": false,
"type": "Log"
},
"statusCode": {
"code": "200",
"reasonPhrase": "OK"
}
}
]
}
As you can see, I can obtain the MongoDB _id in the returned query, but the problem is, I cannot find the location of these seperate datas in related MongoDB database.
I've gone through all the docs but either couldn't find the solution or am I missing something in the architectural logic. How should I go about the issue?
Thanks!
To put it simply, I realized that the problem I was experiencing was due to the parameters that I did not change in the config.js file of the STH Comet I set up on the remote server.
So, I changed the host parameter (from "localhost" to the "server's own IP") in config.server part and the URI parameter (from "localhost:27017" to "serversIP:27017") in the config.database part in the file.
The system is now working as expected.

how to make an indented tree in vega

I'm trying to create an indented tree e.g. as in https://observablehq.com/#d3/indented-tree
I think that what this example does which I can't replicate in vega is encapsulated in this code:
root = { let i = 0; return d3.hierarchy(data).eachBefore(d => d.index = i++); }
eachBefore is a pre-order traversal on the output of d3.hierarchy.
Is there any way to get this result from (upstream) vega, or is this a feature request for an index output from the tree transform? (/something similar, or else a custom transform)
By the way, I think it may be easy to turn the specific tree layout example into an indented tree because the id happens to give the same 'index' ordering (I think), but think we need to use eachBefore where the data isn't so conveniently ordered.
Thanks for any suggestions!
Declan
Update
I made a change in vega-hierarchy described here:
https://github.com/declann/vega/commit/a651ff36cd3f0897054aa1b236f82e701db62432
Now I can use pre_traversal_id from tree transform output to do indented trees, e.g.:
indented tree in (custom) vega-editor, with tree output including pre_traversal_id field
Modified spec: https://gist.github.com/declann/91fd150ae04016e5890a30295fa58a07
Not sure if this help, but, when I enter at vega.github.io/vega/examples/tree-layout I played with the controls and (after change the settings to: layout:tidy - links:orthogonal - separation:true) I got a similar result you shown in the observablehq page:
Open the Chart in the Vega Editor
Code:
{
"$schema": "https://vega.github.io/schema/vega/v5.json",
"description": "An example of Cartesian layouts for a node-link diagram of hierarchical data.",
"width": 600,
"height": 1600,
"padding": 5,
"signals": [
{
"name": "labels", "value": true
},
{
"name": "layout", "value": "tidy"
},
{
"name": "links", "value": "orthogonal"
}
],
"data": [
{
"name": "tree",
"url": "data/flare.json",
"transform": [
{
"type": "stratify",
"key": "id",
"parentKey": "parent"
},
{
"type": "tree",
"method": {"signal": "layout", "value": "tidy"},
"size": [{"signal": "height"}, {"signal": "width - 100"}],
"separation": true,
"as": ["y", "x", "depth", "children"]
}
]
},
{
"name": "links",
"source": "tree",
"transform": [
{ "type": "treelinks" },
{
"type": "linkpath",
"orient": "horizontal",
"shape": {"signal": "links"}
}
]
}
],
"scales": [
{
"name": "color",
"type": "linear",
"range": {"scheme": "greys"},
"domain": {"data": "tree", "field": "depth"},
"zero": true
}
],
"marks": [
{
"type": "path",
"from": {"data": "links"},
"encode": {
"update": {
"path": {"field": "path"},
"stroke": {"value": "#828282"}
}
}
},
{
"type": "symbol",
"from": {"data": "tree"},
"encode": {
"enter": {
"size": {"value": 25}
},
"update": {
"x": {"field": "x"},
"y": {"field": "y"},
"fill": {"field": "depth"}
}
}
},
{
"type": "text",
"from": {"data": "tree"},
"encode": {
"enter": {
"text": {"field": "name"},
"baseline": {"value": "middle"}
},
"update": {
"x": {"field": "x"},
"y": {"field": "y"},
"dx": {"signal": "datum.children ? -7 : 7"},
"align": {"signal": "datum.children ? 'right' : 'left'"}
}
}
}
]
}
I believe this is possible by adding expression transforms for x (based on tree_node depth) and y (based on the tree node id)
The gist of it is to transform x and y after the tree transform
{"type": "stratify", "key": "id", "parentKey": "parent"},
{
"type": "tree",
"method": {"signal": "layout"},
"size": [{"signal": "height"}, {"signal": "width - 100"}],
"separation": {"signal": "separation"},
"as": ["_", "_", "depth", "children"]
},
{"type": "formula", "expr": "datum.depth * 20", "as": "x"},
{"type": "formula", "expr": "datum.id * 14", "as": "y"}
Here's an example that modifies the Vega Tree layout example

How to create Grouped Bar Chart in Vegalite?

Name | Value 1 | Value 2
BTC | 1 | 2
ETH | 1 | 2
to this:
Tried to used this as an example: https://vega.github.io/vega-lite/examples/bar_grouped.html,
but I can't make it work.
Can someone please point me to the right direction? Thank you in advance.
Instead of using column provided in your question, You can simply use layers and keep the x axis as common and provide value1 and value2 in y axis of each layer respectively and simply provide some offset to show it as a grouped bar chart. Below is the basic spec configuration and editor:
{
"$schema": "https://vega.github.io/schema/vega-lite/v5.json",
"description": "A simple bar chart with embedded data.",
"title": "My chart",
"width": 200,
"data": {
"values": [
{"name": "BTH", "value1": 28, "value2": 24, "legendTitle": "value1"},
{"name": "ETH", "value1": 55, "value2": 25, "legendTitle": "value2"}
]
},
"encoding": {
"x": {"field": "name", "type": "nominal", "axis": {"labelAngle": 0}}
},
"layer": [
{
"mark": {"type": "bar", "xOffset": -20, "size": 30, "color": "skyblue"},
"encoding": {
"y": {
"field": "value1",
"type": "quantitative",
"axis": {"title": null, "ticks": false}
}
}
},
{
"mark": {"type": "bar", "size": 30, "xOffset": 18, "color": "orange"},
"encoding": {
"y": {
"field": "value2",
"type": "quantitative",
"axis": {"title": null, "ticks": false}
}
}
},
{
"mark": {"type": "text"},
"encoding": {
"fill": {
"field": "legendTitle",
"scale": {"range": ["skyBlue", "orange"]},
"legend": {"title": null, "symbolType": "square", "orient": "bottom"}
}
}
}
]
}

Vegas (Scala/Spark/Vega) color every data point

Vegas("A scatterplot").
withDataFrame(neuronnet_activation_df).
mark(Point).
encodeX("s", Quantitative).
encodeY("d", Quantitative).
encodeColor(field="feature_0_prediction",scale=Scale(rangeNominals=List("#c41f01", "#00c610"))).
show
Is there away to plot each point with a specifiy RGB or aRGB value?
I have the colors already computed, so I do not need to use ranges, also the color range is not linear for my data.
I'm not certain how this maps onto the Vegas syntax, but in Vega-Lite you can do this by passing color codes as data, and setting the color scale to null. For example:
{
"data": {
"values": [
{"s": 1, "d": 1, "color": "#c41f01"},
{"s": 3, "d": 3, "color": "#00c610"}
]
},
"mark": {"type": "circle", "size": 200},
"encoding": {
"color": {"type": "nominal", "field": "color", "scale": null},
"x": {"type": "quantitative", "field": "s"},
"y": {"type": "quantitative", "field": "d"}
},
"$schema": "https://vega.github.io/schema/vega-lite/v2.6.0.json"
}

VEGA CHART individual bar color

i whant to make every bar in chart - in different collor. please help.
so default chart like described in docs:
https://vega.github.io/vega/examples/bar-chart/
for example - i whant to make like: blue, red, yellow, green, blue, blue.
please help! thanks!
Using the Vega bar chart example you linked to:
First, create a scale to define the colors:
{
"name": "colors",
"type": "ordinal",
"domain": {"data": "table", "field": "category"},
"range": {"scheme": "category10"}
}
The value for the "scheme" can be a built-in scheme from Vega, or a custom scheme.
Then, you need to specify the color for the fill on the mark's update property set, specifying the name of the scheme:
"update": {
"fill": {"scale": "colors", "field": "category"}
},
You can define a categorical color scale and use it to change the color of the bar mark. Check out some of the other examples that use categorical color scales.
I had the same problem and with a little help from https://vega.github.io/vega-lite/examples/bar_color_disabled_scale.html I got to the following - the idea is to encode the color in the data:
{
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"description": "A simple bar chart with embedded data.",
"data": {
"values": [
{"a": "A", "b": 28, "c": "black"},
{"a": "B", "b": 55, "c": "black"},
{"a": "C", "b": 43, "c": "black"},
{"a": "D", "b": 91, "c": "red"}
]
},
"mark": "bar",
"encoding": {
"x": {"field": "a", "type": "ordinal"},
"y": {"field": "b", "type": "quantitative"},
"color": {"field": "c", "type": "nominal", "scale": null}
}
}
alternatively this also works and keeps the legend if you need it:
{
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"description": "A simple bar chart with embedded data.",
"data": {
"values": [
{"a": "A", "b": 28},
{"a": "B", "b": 55},
{"a": "C", "b": 43},
{"a": "D", "b": 91}
]
},
"mark": "bar",
"encoding": {
"x": {"field": "a", "type": "ordinal", "sort": ["C", "A", "B", "D"]},
"y": {"field": "b", "type": "quantitative"},
"color": {"field": "a", "type": "nominal", "sort": ["C", "A", "B", "D"], "scale": {"range": ["red", "black", "black", "black", "black"]}}
}
}
For bar charts
{
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"description": "A bar chart that directly encodes color names in the data.",
"data": {
"values": [
{
"color": "yellow",
"b": 28
},
{
"color": "green",
"b": 55
},
{
"color": "blue",
"b": 43
}
]
},
"mark": "bar",
"encoding": {
"x": {
"field": "color",
"type": "nominal"
},
"y": {
"field": "b",
"type": "quantitative"
},
"color": {
"field": "color",
"type": "nominal",
"scale": null
}
}
}
For Pie charts
{
"$schema": "https://vega.github.io/schema/vega/v5.json",
"width": 200,
"height": 200,
"autosize": "none",
"signals": [
{
"name": "startAngle", "value": 0,
"bind": {"input": "range", "min": 0, "max": 6.29, "step": 0.01}
},
{
"name": "endAngle", "value": 6.29,
"bind": {"input": "range", "min": 0, "max": 6.29, "step": 0.01}
},
{
"name": "padAngle", "value": 0,
"bind": {"input": "range", "min": 0, "max": 0.1}
},
{
"name": "innerRadius", "value": 0,
"bind": {"input": "range", "min": 0, "max": 90, "step": 1}
},
{
"name": "cornerRadius", "value": 0,
"bind": {"input": "range", "min": 0, "max": 10, "step": 0.5}
},
{
"name": "sort", "value": false,
"bind": {"input": "checkbox"}
}
],
"data": [
{
"name": "table",
"values": [
{"field": 10,"color": "blue"},
{"field": 3,"color": "green"},
{"field": 7,"color": "red"},
{"field": 8,"color":"orange"},
{"field": 15,"color":"yellow"}
],
"transform": [
{
"type": "pie",
"field": "field",
"startAngle": {"signal": "startAngle"},
"endAngle": {"signal": "endAngle"},
"sort": {"signal": "sort"}
}
]
}
],
"scales": [
{
"name": "color",
"type": "ordinal",
"domain": {"data": "table", "field": "color"},
"range": {"scheme": "category20"}
}
],
"marks": [
{
"type": "arc",
"from": {"data": "table"},
"encode": {
"enter": {
"fill": { "field": "color",
"type": "nominal",
"scale": null},
"x": {"signal": "width / 2"},
"y": {"signal": "height / 2"}
},
"update": {
"startAngle": {"field": "startAngle"},
"endAngle": {"field": "endAngle"},
"padAngle": {"signal": "padAngle"},
"innerRadius": {"signal": "innerRadius"},
"outerRadius": {"signal": "width / 2"},
"cornerRadius": {"signal": "cornerRadius"}
}
}
}
]
}