D3 Multi Line Graph with Dots - charts

I am new to D3.js. I love it but I am having real trouble figuring out the best approach to structuring data.
I would ideally like to create a simple multiline graph that has over points over the selected points. Firstly I have the multiple lines created but trying to add the points has stumped me, and I think it has to do with the structure of my data.
Here is my working fiddle. I'm not sure if I should be trying to use d3.nest to re-arrange the data
I have a json object that I am retrieving from a google form which is all nice and smooth. This is what it looks like:
var data = [{
"description": "Global warming is a serious and pressing problem. We should begin taking steps now even if this involves significant costs",
"year2013": 40,
"year2012": 36,
"year2011": 41,
"year2010": 46,
"year2009": 48,
"year2008": 60,
"year2006": 68,
}, {
"description": "The problem of global warming should be addressed, but its effects will be gradual, so we can deal with the problem gradually by taking steps that are low in cost",
"year2013": 44,
"year2012": 45,
"year2011": 40,
"year2010": 40,
"year2009": 39,
"year2008": 32,
"year2006": 24,
}, {
"description": "Until we are sure that global warming is really a problem, we should not take any steps that would have economic costs",
"year2013": 16,
"year2012": 18,
"year2011": 19,
"year2010": 13,
"year2009": 13,
"year2008": 8,
"year2006": 7,
}, {
"description": "Don't know / refused",
"year2013": 1,
"year2012": 1,
"year2011": 1,
"year2010": 1,
"year2009": 1,
"year2008": 0,
"year2006": 1,
}]
Any help would be appreciated, I have been at it for days.
Cheers!

First - I would flatten your data
data = [
{date:"2011",type: "line0", amount:20}
...
]
Then nest your data by type
nested = d3.nest()
.key( (d) -> return d.type )
.entries(data)
Then append your line groups
# Line Groups
groups = container.selectAll('g.full-line')
.data(nested, (d) -> return d.key )
# ENTER
groups.enter().append('svg:g')
.attr( 'class', (d,i) -> "full-line#{i}" )
# EXIT
d3.transition(groups.exit()).remove()
# TRANSITION
d3.transition(groups)
Then append your chart lines
# Individual Lines
lines = groups.selectAll('.line').data (d)-> [d.values]
# ENTER
lines.enter().append("svg:path")
.attr("class","line")
.attr("d", d3.svg.line()
.interpolate(interpolate)
.defined(defined)
.x( (d,i) -> return xScale(d,i) )
.y( (d,i) -> return yScale(d,i) ) )
# EXIT
d3.transition( groups.exit().selectAll('.line') )
.attr("d",
d3.svg.line()
.interpolate(interpolate)
.defined(defined)
.x( (d,i) -> return xScale(d,i) )
.y( (d,i) -> return yScale(d,i) ) )
# TRANSITION
d3.transition(lines)
.attr("d",
d3.svg.line()
.interpolate(interpolate)
.defined(defined)
.x( (d,i) -> return xScale(d,i) )
.y( (d,i) -> return yScale(d,i) ) )

Thanks
I ended up using something similar.
/* Transform Data */
data = data.map(function (d) {
return {
country: d.country,
date: new Date(d.year.toString()),
value: d.value
};
});
/* Nest Data */
data = d3.nest().key(function (d) {
return d.country;
}).entries(data);`

Related

Understanding layout rules in PySimpleGUI

I am trying to build a GUI with PySimpleGUI for the first time and am struggling with the layout reuse rules.
The overall aim of the script will be to make entering data into a CSV table easier for users. For this purpose, the users will be able to enter data into a form, submit the data, and decide if they want to add another data set or exit.
I have tried to work with nested functions but am constantly breaking the script because I am supposed to use a new layout for each window. I have defined 3 different windows with their own layouts so far:
window1 = main window where data can be added
window2 = window with error notification
window3 = window opening after SUBMIT, asking users if they want to continue
In addition, I am trying to call the function for window1 again if users decide to add more data (repeatGUI()), but this is not permitted.
I am aware that there are other issues in the script, too, but I would mainly appreciate some input on the layout-reuse issue. What is the proper way of opening an input window multiple times?
DRAFT SCRIPT:
import csv
import PySimpleGUI as sg
sg.main_get_debug_data()
myfile="C:\\######\\File.csv"
counter=0
def buildGUI(): # defining GUI frame, title, fields and buttons
sg.ChangeLookAndFeel('GreenTan')
window1 = sg.FlexForm('MY DB', size = (900,700), resizable=True)
layout1 = [
[sg.Text('Please enter a new factoid:')],
[sg.Text('Factoid ID', size=(15, 1), key="-FACTOID-")],
[sg.Text('Person ID', size=(15, 1), key="-PERSON-")],
[sg.Text('Exact Date', size=(10, 1)), sg.InputText()],
[sg.Text('Event Before', size=(10, 1)), sg.InputText()],
[sg.Text('Event After', size=(10, 1)), sg.InputText()],
[sg.Text('Event Start', size=(10, 1)), sg.InputText()],
[sg.Text('Event End', size=(10, 1)), sg.InputText()],
[sg.Text('Event Type', size=(15, 1)), sg.InputText()],
[sg.Text('Person Name', size=(15, 1)), sg.InputText()],
[sg.Text('Person Title', size=(15, 1)), sg.InputText()],
[sg.Text('Person Function', size=(15, 1)), sg.InputText()],
[sg.Text('Place Name', size=(15, 1)), sg.InputText()],
[sg.Text('Institution Name', size=(15, 1)), sg.InputText()],
[sg.Text('Related Persons', size=(15, 1)), sg.InputText()],
[sg.Text('Alternative Names', size=(15, 1)), sg.InputText()],
[sg.Text('Source Quote', size=(10, 1)), sg.InputText()],
[sg.Text('Additional Info', size=(10, 1)), sg.InputText()],
[sg.Text('Comment', size=(10, 1)), sg.InputText()],
[sg.Text('Source', size=(10, 1)), sg.InputText()],
[sg.Text('Source URL', size=(10, 1)), sg.InputText()],
[sg.Text('Info Dump', size=(10, 1)), sg.InputText()],
[sg.Text(size=(70,1), key='-MESSAGE1-')],
[sg.Submit(), sg.Button('Clear Input')]
]
while True: # The Event Loop
event1, values1 = window1.Layout(layout1).Read()
print(layout1)
def startGUI(event1, values1): # start GUI and get data
# interact with user to get input
if event1 == 'Submit': # user clicked "Submit"
def submitGUI(values1): # submitting new data via GUI
fact_id=("#") # place holder: to be calculated later
pers_id=("#") # place holder: to be calculated later
entries=[fact_id, pers_id]
for v in values1.values():
entries.append(v)
try:
with open(myfile, 'a', encoding="utf-8") as f:
w=csv.writer(f)
w.writerow(entries) # write list items to CSV file
f.close()
try:
window3 = sg.FlexForm('NEW DATA?', size = (500,100))
layout3 = [
[sg.Text("Your input has been saved! Do you want to add a new factoid?")],
[sg.Text(size=(70,1), key='-MESSAGE2-')],
[sg.Button("YES"), sg.Button("NO"), sg.Exit()]
]
while True:
event3, values3 = window3.Layout(layout3).Read()
if event3 == 'YES': # user clicked YES
window1.close()
try:
repeatGUI() # this breaks layout rules!
except:
print("Not allowed!")
pass
elif event3 == 'NO':
window3['-MESSAGE2-'].update("See you again soon!")
window1.close()
elif event3 in (sg.WINDOW_CLOSE_ATTEMPTED_EVENT, 'Exit'):
window3.close()
except:
pass
except PermissionError:
window2 = sg.FlexForm('DB ERROR', size = (500,100))
layout2 = [
[sg.Text("Someone else is using the file! Please try again later.")],
[sg.Exit()]
]
event2, values2 = window2.Layout(layout2).Read()
if event2 in (sg.WINDOW_CLOSE_ATTEMPTED_EVENT, 'Exit'):
window2.close() # user clicked "Exit"
submitGUI(values1)
elif event1 == 'Clear Input': # user clicked "Cancel"
window1.Refresh()
elif event1 == sg.WINDOW_CLOSE_ATTEMPTED_EVENT: # user closed window
window1.close() # AttributeError: module 'PySimpleGUI' has no attribute 'WIN_CLOSED'
startGUI(event1, values1)
buildGUI()
def repeatGUI():
counter+=1
print(counter)
buildGUI()
Following way show that you use the variable layout1 as layout of sg.Window again and again, none of element in layout1 initialized after 1st time using.
while True: # The Event Loop
event1, values1 = window1.Layout(layout1).Read()
...
Following code preferred,
window1 = sg.Window('Title1', layout1, finalize=True)
while True:
event1, values1 = window1.read()
...
or
def make_window1():
layout = [
[sg.Text('Please enter a new factoid:')],
....
]
return sg.Window('Title1', layout, finalized=True)
window1 = make_window1()
while True:
event1, values1 = window1.read()
...
Based on #jason-yang 's answer, I was able to fix my layout issue and revised the whole script to
a) define layouts outside the function that starts the GUI
and
b) get rid of the attempt to open the same input window several times.
Having a persistent window is much more suitable in my case, so I am using a "Clear" button to allow new data input:
elif event1 == 'Clear to add new data': # user clicked "Clear"
window1['-MESSAGE1-'].update("Ready for your next data set!")
# update all input fields
window1['-DATE-'].update('')
window1['-BEFORE-'].update('')
window1['-AFTER-'].update('')
window1['-START-'].update('')
window1['-END-'].update('')
window1['-EVENT-'].update('')
window1['-PERSNAME-'].update('')

Getting Import Error quite randomly when using plotly express and having multiple graphs on one page

Relatively new to Dash, and this is a problem that has been vexing me for months now. I am making a multi-page app that shows some basic data trends using cards, and graphs embedded within cardbody. 30% of the time, the app works well without any errors and the other 70% it throws either one of the following:
ImportError: cannot import name 'ValidatorCache' from partially initialized module 'plotly.validator_cache' (most likely due to a circular import)
OR
ImportError: cannot import name 'Layout' from partially initialized module 'plotly.graph_objects' (most likely due to a circular import)
Both these appear quite randomly and I usually refresh the app to make them go away. But obviously I am doing something wrong. I have a set of dropdowns that trigger callbacks on graphs. I have been wracking my head about this. Any help/leads would be appreciated. The only pattern I see in the errors is they seem to emerge when the plotly express graphs are being called in the callbacks.
What am I doing wrong? I have searched all over online for help but nothing yet.
Sharing with some relevant snippets of code (this may be too long and many parts not important to the question, but to give you a general idea of what I have been working towards)
import dash
import dash_bootstrap_components as dbc
from dash.dependencies import Input, Output, State
import dash_core_components as dcc
import dash_html_components as html
import plotly.graph_objs as go
from plotly.subplots import make_subplots
import plotly.express as px
card_content1_1 = [
dbc.CardHeader([dbc.Row([html.H5("SALES VOLUME TREND", className = "font-weight-bold text-success"),
dbc.Button(
html.I(className="fa fa-window-maximize"),
color="success",
id="sales_maximize",
className="ml-auto",
# href="www.cogitaas.com"
)
])]),
dbc.CardBody(
[dcc.Graph(
id='sales_graph',
figure={},
style={'height':'30vh'}
# className="mt-5"
)])]
card_stacked_discount = [
dbc.CardHeader([dbc.Row([html.H5("VOLUMES UNDER DIFFERENT DISCOUNT LEVELS", className="font-weight-bold text-info text-center"),
dbc.Button(
html.I(className="fa fa-window-maximize"),
color="info",
id="discount_maximize",
className="ml-auto",
# href="www.cogitaas.com"
)
])]),
dbc.CardBody(
[dcc.Dropdown(
id = 'stacked_discount_dropdown',
options =stacked_discount_options,
value=stacked_discount_options[0].get('value'),
style={'color':'black'},
# multi=True
),
dcc.Graph(
id='stacked_discount_graph',
figure={},
style={'height':'30vh'}
)])]
cards = html.Div(
[
dbc.Row(
[
dbc.Col(dbc.Card(card_content1_1, color="success", outline=True,
style={'height':'auto'}), width=8),
],
className="mb-4",
),
dbc.Row(
[
dbc.Col(dbc.Card(card_stacked_discount, color="info", outline=True), width=8),
dbc.Col(dbc.Card([
dbc.Row([
dbc.Col(dbc.Card(disc_sub_title, color="info", inverse=True)),
]),
html.Br(),
dbc.Row([
dbc.Col(dbc.Card(disc_sub_card1, color="info", outline=True)),
]),
]), width=4)
],
className="mb-4",
),
]
)
tab1_content = dbc.Card(
dbc.CardBody(
[cards,]
),
className="mt-3",
)
tabs = dbc.Tabs(dbc.Tab(tab1_content, label="Data", label_style={'color':'blue'}, tab_style={"margin-left":"auto"}),])
content = html.Div([
html.Div([tabs]),
],id="page-content")
app.layout = html.Div([dcc.Location(id="url"), content])
#app.callback(
dash.dependencies.Output('sales_graph', 'figure'),
[dash.dependencies.Input('platform-dropdown', 'value'),
dash.dependencies.Input('signature-dropdown', 'value'),
dash.dependencies.Input('franchise-dropdown', 'value'),
dash.dependencies.Input('sales_maximize', 'n_clicks'),
dash.dependencies.Input('time-dropdown', 'value'),
])
def update_sales_graph(plat, sign, fran, maximize, time_per):
print(str(time_per)+"Test")
time_ax=[]
if isinstance(time_per,str):
time_ax.append(time_per)
time_per=time_ax
if (time_per==None) or ('Full Period' in (time_per)):
dff = df[(df.Platform==plat) & (df.Signature==sign) & (df.Franchise==fran)]
elif ('YTD' in time_per):
dff = df[(df.Platform == plat) & (df.Signature == sign) & (df.Franchise == fran) & (df.year==2020)]
else:
dff = df[(df.Platform==plat) & (df.Signature==sign) & (df.Franchise==fran) & (df.Qtr_yr.isin(time_per))]
fig = px.area(dff, x='Date', y='Qty_Orig', color_discrete_sequence=px.colors.qualitative.Dark2)
fig.add_trace(go.Scatter(x=dff['Date'], y=dff['Outliers'], mode = 'markers', name='Outliers',
line=dict(color='darkblue')))
fig.add_trace(go.Scatter(x=dff['Date'], y=dff['bestfit'], name='Long Term Trend',
line=dict(color='darkblue')))
fig.update_layout(font_family="Rockwell",
title={'text': fran + " Volume Trend",
'y': 0.99,
# 'x': 0.15,
# 'xanchor': 'auto',
'yanchor': 'top'
},
legend=dict(
orientation="h",
# y=-.15, yanchor="bottom", x=0.5, xanchor="center"
),
yaxis_visible=False, yaxis_showticklabels=False,
xaxis_title=None,
margin=dict(l=0, r=0, t=0, b=0, pad=0),
plot_bgcolor='White',
paper_bgcolor='White',
)
fig.update_xaxes(showgrid=False, zeroline=True)
fig.update_yaxes(showgrid=False, zeroline=True)
changed_id = [p['prop_id'] for p in dash.callback_context.triggered][0]
if 'maximize' in changed_id:
fig.show()
return fig
#app.callback(
dash.dependencies.Output('stacked_discount_graph', 'figure'),
[dash.dependencies.Input('platform-dropdown', 'value'),
dash.dependencies.Input('signature-dropdown', 'value'),
dash.dependencies.Input('franchise-dropdown', 'value'),
dash.dependencies.Input('discount_maximize', 'n_clicks'),
dash.dependencies.Input('stacked_discount_dropdown', 'value'),
dash.dependencies.Input('time-dropdown', 'value'),
])
def stacked_discount(plat, sign, fran, maximize, sales_days, time_per):
time_ax=[]
if isinstance(time_per,str):
time_ax.append(time_per)
time_per=time_ax
# else:
# time_per=list(time_per)
if (time_per==None) or ('Full Period' in (time_per)):
df_promo = df_promo_vol[(df_promo_vol.Platform==plat) & (df_promo_vol.Signature==sign) & (df_promo_vol.Franchise==fran)]
elif ('YTD' in time_per):
df_promo = df_promo_vol[(df_promo_vol.Platform == plat) & (df_promo_vol.Signature == sign) & (df_promo_vol.Franchise == fran) & (df_promo_vol.Year==2020)]
else:
df_promo = df_promo_vol[(df_promo_vol.Platform==plat) & (df_promo_vol.Signature==sign) & (df_promo_vol.Franchise==fran) & (df_promo_vol.Qtr_yr.isin(time_per))]
color_discrete_map = {
"0 - 10": "orange",
"10 - 15": "green",
"15 - 20": "blue",
"20 - 25": "goldenrod",
"25 - 30": "magenta",
"30 - 35": "red",
"35 - 40": "aqua",
"40 - 45": "violet",
"45 - 50": "brown",
"50 + ": "black"
}
category_orders = {'disc_range': ['0 - 10', '10 - 15', '15 - 20', '20 - 25', '25 - 30', '30 - 35', '35 - 40',
'40 - 45', '45 - 50', '50 + ']}
if (sales_days == None) or (sales_days == 'sales_act'):
fig = px.bar(df_promo, x='start', y='units_shipped', color='disc_range',
color_discrete_map=color_discrete_map,
category_orders=category_orders,
)
else:
fig = px.bar(df_promo, x='start', y='Date', color="disc_range",
color_discrete_map=color_discrete_map,
category_orders=category_orders,
)
fig.update_layout(font_family="Rockwell",
title={'text': fran + " Sales Decomposition",
'y': 0.99,
'x': 0.1,
# 'xanchor': 'auto',
'yanchor': 'top'
},
legend=dict(
orientation="h",
# y=-.15, yanchor="bottom", x=0.5, xanchor="center"
),
# yaxis_visible=False, yaxis_showticklabels=False,
xaxis_title=None,
margin=dict(l=0, r=0, t=30, b=30, pad=0),
plot_bgcolor='White',
paper_bgcolor='White',
)
fig.update_xaxes(showgrid=False, zeroline=True)
fig.update_yaxes(showgrid=False, zeroline=True)
changed_id = [p['prop_id'] for p in dash.callback_context.triggered][0]
if 'maximize' in changed_id:
fig.show()
return fig
Well, it appears I may have stumbled on to an answer. I was using the pretty much the same inputs for multiple callbacks and that could have been causing some interference with the sequencing of inputs. Once I integrated the code into one callback with multiple outputs, the problem seems to have disappeared.
Was dealing with this same issue where everything in my app worked fine, then I made an entirely separate section & callback that started throwing those circular import errors.
Was reluctant to re-arrange my (rightfully) separated callbacks to be just a single one and found you can fix the issue by just simply importing what the script says it's failing to get. In my case, plotly was trying to import the ValidatorCache and Layout so adding these to the top cleared the issue and now my app works as expected. Hope this helps someone experiencing a similar issue.
from plotly.graph_objects import Layout
from plotly.validator_cache import ValidatorCache

Ionic BLE Manufacturer specific Data

Using #ionic-native/ble I'm able to scan and discover a BLE device which has manufacturer specific data.
According to the lib (https://github.com/don/cordova-plugin-ble-central#ios-1) here is the way to get this data
const mfgData = new Uint8Array(device.advertising.kCBAdvDataManufacturerData);
console.log('Manufacturer Data: ', mfgData);
const hex = Buffer.from(mfgData).toString('hex');
console.log(hex);
The encode to hex result being 2604 0504 386 55c0b
What I don't understand is the proper way to use this result to decode the manufacturer (company) id, which is supposed to be "0x0426"
You can try the following :
const toHexString = bytes =>
bytes.reduce((str, byte) => str + byte.toString(16).padStart(2, '0'), '');
console.log(toHexString(new Uint8Array([0, 1, 2, 42, 100, 101, 102, 255])))

How to generate a random values between two hexcode in flutter

I am trying to generate a list of hexcode between two values so I can use then to generate a range of colors.
What I am trying to achieve is this.
List.generate(36, (i) => generateRandomCode(0xFF0587D8, 0xFF0345B5))
generateRandomCode(min, max) {
// implementation here
}
How do I generate this int in the function generateRandomCode?
Generator:
int generateRandomCode(int minValue, int maxValue) {
return Random().nextInt((maxValue - minValue).abs() + 1) + min(minValue, maxValue);
}
Usage:
final list = List<int>.generate(36, (i) => generateRandomCode(0xFF0587D8, 0xFF0345B5));
print(list);
Result:
/flutter ( 6592): [4278422613, 4278508577, 4278489065, 4278486019, 4278499653, 4278480654, 4278464106, 4278474805, 4278462976, 4278549386, 4278537465, 4278418510, 4278496777, 4278405225, 4278411018, 4278412393, 4278461314, 4278538568, 4278549901, 4278510124, 4278492024, 4278530862, 4278517728, 4278425917, 4278442865, 4278497051, 4278430858, 4278497227, 4278462764, 4278412600, 4278448684, 4278422213, 4278464891, 4278473256, 4278543371, 4278476016]
You can still convert it to list of colors
final colorList = list.map((hex) => Color(hex)).toList();
print(colorList);
Result:
I/flutter ( 6592): [Color(0xff04c366), Color(0xff03d608), Color(0xff03a34a), Color(0xff048eac), Color(0xff03924a), Color(0xff03f0ba), Color(0xff052271), Color(0xff03ef8a), Color(0xff0582e0), Color(0xff0551ae), Color(0xff0402b3), Color(0xff0552be), Color(0xff050553), Color(0xff04c39f), Color(0xff053f88), Color(0xff04b6b8), Color(0xff05299f), Color(0xff03f1a7), Color(0xff03ca2f), Color(0xff04a864), Color(0xff04ee66), Color(0xff0358ce), Color(0xff03b741), Color(0xff046785), Color(0xff04ef11), Color(0xff04e618), Color(0xff03ff8a), Color(0xff03dc97), Color(0xff04353e), Color(0xff04cff6), Color(0xff03bfa4), Color(0xff049ca3), Color(0xff04bbac), Color(0xff03c5d3), Color(0xff05730b), Color(0xff036c8f)]
Use a List Generate Constructor
new List<int>.generate(3, (int index) => index * index);
Link to the source : Link generator
List<int> createList(int min, int max) {
return List.generate(max - min + 1, (i) => min + i);
}
Result:
print(">>>>>>>>>> ${createList(10, 22).join(", ")}");
// >>>>>>>>>> 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22

leafletR map doesn't load in shiny on start

I have the following little piece of code (more less as described HERE) - I want to control the number of points to be shown by a slider in shiny. You can see that the initial map is loaded after a little while (watch console output), but it will only show up after you used the slider once.
But I'd like the map to show up after it is created during launch of the shiny app - any hints how to do that?
## app.R ##
library(shiny)
library(shinydashboard)
library(httr)
library(leafletR)
data(quakes)
# dest_dir=tempdir()
dest_dir="foo_map"
dest_file = paste(dest_dir,"quakes","quakes.html",sep="\\")
dat = quakes
createMapHTML <- function(inputFreq=1) {
q.dat <- toGeoJSON(data=dat[seq(from = 1, to = nrow(dat), by=inputFreq), ],
dest=dest_dir, name="quakes")
sty <- styleSingle(col="darkblue", fill="darkblue", rad=6)
# create map
q.map <- leaflet(data=q.dat, dest=dest_dir, size = c(1200, 600), incl.data=TRUE,
base.map=list("osm"), style=sty, popup="*", controls="all")
}
# createMapHTML()
runApp(list(
ui = dashboardPage(
dashboardHeader(title = "quakes"),
dashboardSidebar(
sliderInput("slider", "#observations frequency:", 1, 100, 1)
),
dashboardBody(
htmlOutput("inc")
)
),
server = function(input, output, session) {
createMap <- reactive({
createMapHTML(input$slider)
return(includeHTML(dest_file))
})
output$inc<-renderUI({ createMap() })
}
))
so the bottleneck with the leafletR package is the conversion to GeoJson. Additionally the "includeHTML & htmlOutput" workaround for embedding the html out is flaky..
To avoid both I just switched to the leaflet packackage:
## app.R ##
library(shiny)
library(shinydashboard)
library(leaflet)
data(quakes)
dat = quakes
runApp(list(
ui = dashboardPage(
dashboardHeader(title = "quakes"),
dashboardSidebar(
sliderInput("slider", "#observations frequency:", 1, 100, 1)
),
dashboardBody(
leafletOutput("map", height = 600)
)
),
server = function(input, output) {
output$map <- renderLeaflet({
map <- leaflet() %>% addTiles()
map %>% addCircles(data=dat[seq(from = 1, to = nrow(dat), by=input$slider), ], #input$slider
lat = ~lat, lng = ~long, fillOpacity = 1.0)
})
}
))