Multiple data series and multiple data tooltip with HighMaps - highmaps

I would like to display multiple data for the same state in the tool tip. how should i define the data and format the tool tip accordingly?
In the example the defined data is
var data = [['in-tn', 1], ['in-ap', 2]]
and the tooltip is
Random data
Tamilnadu : 1
I would like to define my data as
var data = [['in-tn', Rank1 , Value1 , 'Station ID1', Value2],
['in-tn', Rank2 , Value2 , 'Station ID2', Value3],
['in-ap', Rank3 , Value3 , 'Station ID3', Value5],
['in-ap', Rank4 , Value4 , 'Station ID4', Value6]]
and get the tool tip when hoverd on 'in-tn' as
TamiNadu
Station 1 , Rank = 1, Value1 = 13, Value 2 = 189.76<br>
Station 10, Rank = 2, Value1 = 23, Value 2 = 156.45<br>
My fiddle with data series but without tooltip
//data for which the Highmaps to be generated with all data in tooltip which matches the state code
var data = [['in-tn', 1 , 13 , 'Station 1', 189.76],
['in-tn', 2 , 23 , 'Station 10', 156.45],

First you need to define the structure of your data for that the Highcharts would "know" how to parse it, so to do that please use the series.keys parameter, just like that:
series: [{
data: data,
name: 'Random data',
keys: ['hc-key', 'rank', 'value', 'stationId', 'value2'],
states: {
hover: {
color: '#FFC300'
}
},
dataLabels: {
enabled: true,
format: '{point.name}'
}
}]
Then you have to define your own tooltip format. You can achieve it using tooltip.pointFormat:
tooltip: {
pointFormat: '<span>{point.stationId}, Rank: <b>{point.rank}</b>, Value 1: <b>{point.value}</b>, Value 2: <b>{point.value2}</b></span>'
}
Live example: https://jsfiddle.net/BlackLabel/1oudhgq3/
API Reference:
https://api.highcharts.com/highmaps/tooltip.pointFormat
https://api.highcharts.com/highcharts/series.line.keys

Related

PySpark: Explode schema columns does not match with underlying nested schema

I am using pyspark in combination with Azure-Synapse. I am reading multiple nested JSON with the same structure in a dataframe using the below sample:
{
"AmountOfOrders": 2,
"TotalEarnings": 1800,
"OrderDetails": [
{
"OrderNumber": 1,
"OrderDate": "2022-7-06",
"OrderLine": [
{
"LineNumber": 1,
"Product": "Laptop",
"Price": 1000
},
{
"LineNumber": 2,
"Product": "Tablet",
"Price": 500
},
{
"LineNumber": 3,
"Product": "Mobilephone",
"Price": 300
}
]
},
{
"OrderNumber": 2,
"OrderDate": "2022-7-06",
"OrderLine": [
{
"LineNumber": 1,
"Product": "Printer",
"Price": 100,
"Discount": 0
},
{
"LineNumber": 2,
"Product": "Paper",
"Price": 50,
"Discount": 0
},
{
"LineNumber": 3,
"Product": "Toner",
"Price": 30,
"Discount": 0
}
]
}
]
}
I am trying to get the the LineNumbers of Ordernumber 1 in a separate dataframe using a generic function which extract the array and Struct of the dataframe. Using the code below:
def read_nested_structure(df,excludeList,messageType,coll):
display(df.limit(10))
print('read_nested_structure')
cols =[]
match = 0
match_field = ""
print(df.schema[coll].dataType.fields)
for field in df.schema[coll].dataType.fields:
for c in excludeList:
if c == field.name:
print('Match = ' + field.name)
match = 1
if match == 0:
# cols.append(coll)
cols.append(col(coll + "." + field.name).alias(field.name))
match = 0
# cols.append(coll)
print(cols)
df = df.select(cols)
return df
def read_nested_structure_2(df,excludeList,messageType):
cols =[]
match = 0
for coll in df.schema.names:
if isinstance(df.schema[coll].dataType, ArrayType):
print( coll + "-- : Array")
df = df.withColumn(coll, explode(coll).alias(coll))
cols.append(coll)
elif isinstance(df.schema[coll].dataType, StructType):
if messageType == 'Header':
for field in df.schema[coll].dataType.fields:
cols.append(col(coll + "." + field.name).alias(coll + "_" + field.name))
elif messageType == 'Content':
print('Struct - Content')
for field in df.schema[coll].dataType.fields:
cols.append(col(coll + "." + field.name).alias(field.name))
else:
for c in excludeList:
if c == coll:
match = 1
if match == 0:
cols.append(coll)
df = df.select(cols)
return df
df = spark.read.load(datalakelocation + '/test.json', format='json')
df = unpack_to_content_dataframe_simple_2(df,exclude)
df = df.filter(df.OrderNumber == 1)
df = unpack_to_content_dataframe_simple_2(df,exclude)
display(df.limit(10))
which result in the following dataframe:
as you can see the yellow marked attribute is added to the dataframe which is not part of OrderNumber 1. How can I filter a row in the dataframe which results in a update schema ( in this case without the Discount attribute)?
I have used read_nested_structure_2() function in the following way to get the same results as yours. The code I used to get this result using read_nested_structure_2() is as follows:
x = read_nested_structure_2(df,[],'Header')
y = read_nested_structure_2(x,[],'Content')
y = y.filter(y.OrderNumber == 1)
z = read_nested_structure_2(y,[],'Header')
final = read_nested_structure_2(z,[],'Content')
display(final)
The output of after using this code is:
The column Discount will be created even if it is specified for one Product in the entire input JSON. In order to remove this column, we have to do it separately to get another dataframe without Discount (only if it is invalid).
You are going to use the same function to extract data from StructType or ArrayType, it is not recommended to write code to remove fields (say Discount) having all null values, in the same function. Doing so would complicate the code.
Instead, we can write another function which does this work for us. This function should remove a column where all of its values are null. The following is the function that can be used to do this.
def exclude_fields_that_dont_exist(filtered_df):
cols=[]
#iterate through columns
for column in filtered_df.columns:
#null_count is the count of null values in a column
null_count = filtered_df.filter(filtered_df[column].isNull()).count()
#check if null_count equals the total column value count
#if they are equal, those columns are not required (Like Discount)
if(filtered_df.select(column).count() != null_count):
cols.append(column)
#return dataframe with required columns
return filtered_df.select(*cols)
When you use this function on the filtered dataframe (final in my case), then you get a resulting dataframe as shown below:
mydf = exclude_fields_that_dont_exist(final)
# removes columns from a dataframe that have all null values.
display(mydf)
NOTE:
For example, for OrderNumber=1, the product Laptop has a 10% discount and the rest of the products (for the same order number) don't have a discount value (in the JSON).
The function needs to include the Discount column since it is a required information.
To avoid using more loops inside a function, you can also consider replacing all the null values with 0 since a Product with no Discount specified (null value) is same as a Product with Discount value as 0 (If this is feasible, then you can use fill() or fillna() functions to fill null values with any desired value)

Use dimension when creating measure with Case statement in Looker

I am trying to create a measure conditionally based on a dimension.
My dimensions:
dimension_group: date {
hidden: yes
type: time
timeframes: [
raw,
date,
week,
month,
quarter,
year
]
convert_tz: no
datatype: date
sql: ${TABLE}.date ;;
}
dimension: status {
type: string
sql: CASE
WHEN UPPER(${TABLE}.status) ='APPROVED' THEN 'Approved'
WHEN UPPER(${TABLE}.status) ='PENDING' THEN 'Pending'
END;;
}
My Measures:
measure: xyz {
type: sum
value_format: "$#,##0.00"
sql: ${TABLE}.xyz ;;
}
measure: abc {
type: sum
value_format: "$#,##0.00"
sql: ${TABLE}.abc ;;
}
Measure with conditions:
measure: conditional {
type: number
value_format: "$#,##0.00"
sql: CASE WHEN ${status} = 'Pending' THEN ${xyz}
ELSE ${abc}
END;;
}
On my Explore, when I select date and conditional.  I keep getting the error:
ERROR: column "table.status" must appear in the GROUP BY clause or be used in an aggregate function
I understand what the error is. I am just not sure how to fix this. How do I resolve this error? I need all the dimensions and measures.
You can create a dimension conditional:
dimension: conditional {
type: number
value_format: "$#,##0.00"
sql: CASE WHEN ${status} = 'Pending' THEN ${xyz}
ELSE ${abc}
END;;
}
And then create a measure sum_conditional on that dimension:
measure: sum_conditional {
type: sum
value_format: "$#,##0.00"
sql: ${conditional};;
}

Filter data dynamically based on a hierarchical user id in cube.js

I have two tables called Writers and Publications where there is a foreign key publications.writer_id = writers.id
Writers
id (int)
parent_id (int)
role (varchar)
name (varchar)
path (ltree)
1
ADMIN
Firstname Lastname
2
1
EDITOR
Anon Anon
1.2
3
2
WEB EDITOR
Maisy Tickles
1.2.3
4
2
WEB EDITOR
Jack Beanstalk
1.2.4
5
3
WEB PROOFREADER
Sunny Ray
1.2.3.5
Publications
id (int)
writer_id (FK)
publication_name (varchar)
word_length (int)
published (datetime)
1
2
My First Magazine
6000
2019-09-09 09:00:00
2
2
My Second Magazine
6000
2019-09-16 09:00:00
3
3
My First Article
1000
2019-09-23 09:00:00
4
4
My First Article
1500
2019-09-23 09:00:00
5
4
My Second Article
600
2019-10-01 09:00:00
6
5
My First Piece
600
2020-10-01 09:00:00
I want to do a proof of concept in cube.js Developer Playground to show various charts. Is it possible to filter dynamically based on the user_id so that they can only access content that is equal to or in their subtree i.e.
If an ADMIN/EDITOR is using, they can see all the publications
If the WEB EDITOR (writers.id=4) is using the application, they can only see their own articles (publications.id in (4,5))
If the WEB EDITOR (writers.id=3) is using the application, they can see their publication and the WEB PROOFREADER's one (publications.id in (3,6))
The WEB PROOFREADER should only see their publication (publications.id=6)
These are the models I have set up so far
cube(`Writers`, {
sql: `SELECT * FROM public.writers`,
preAggregations: {},
joins: {
Publications:{
sql: `${CUBE}.id = ${Publication}.writer_id`,
relationship: `hasMany`
}
},
measures: {
count: {
type: `count`,
drillMembers: [id, name, created]
}
},
dimensions: {
id: {
sql: `id`,
type: `number`,
primaryKey: true
},
role: {
sql: `role`,
type: `string`,
},
};
cube(`Publications`, {
sql: `SELECT * FROM public.publications`,
preAggregations: {},
joins: {
Writer:{
sql: `${CUBE}.writer_id = ${Writers}.id`,
relationship: `hasOne`
}
},
measures: {
count: {
type: `count`,
drillMembers: [id, name, created]
}
},
dimensions: {
id: {
sql: `id`,
type: `number`,
primaryKey: true
},
wordLength: {
sql: `word_length`,
type: `number`,
},
};
I know there are filters and segments but these appear to be static. Is there any way to pass a writer_id to filter the relevant data dynamically? (I have no previous knowledge of JS)
I think these recipes can help you:
https://cube.dev/docs/recipes/role-based-access
https://cube.dev/docs/recipes/column-based-access
https://cube.dev/docs/recipes/passing-dynamic-parameters-in-a-query
I think you can query the cube from your front-end application with a filter of id.

Postgres json select

Let's say that I have an incoming JSON message to my app:
{
line_nbr : 1,
value: 10
},
{
line_nbr : 2,
value: 30
},
]
Is it possible to perform the following selection in postgres :
SELECT JsonObject.value, qty from table_x where id in JsonObjects.line_nbr
In other words join on the incoming JSON object
with your_js as (
select (value->>'line_nbr')::int as line_nbr
from jsonb_array_elements('JsonObjects'::jsonb) as je
)
select line_nbr, qty
from table_x
join your_js on line_nbr = table_x.id
check here for detail

One2many field issue Odoo 10.0

I have this very weird issue with One2many field.
First let me explain you the scenario...
I have a One2many field in sale.order.line, below code will explain the structure better
class testModule(models.Model):
_name = 'test.module'
name = fields.Char()
class testModule2(models.Model):
_name = 'test.module2'
location_id = fields.Many2one('test.module')
field1 = fields.Char()
field2 = fields.Many2one('sale.order.line')
class testModule3(models.Model):
_inherit = 'sale.order.line'
test_location = fields.One2many('test.module2', 'field2')
CASE 1:
Now what is happening is that when i create a new sales order, i select the partner_id and then add a sale.order.line and inside this line i add the One2many field test_location and then i save.
CASE 2:
Create new sales order, select partner_id then add sale.order.line and inside the sale.order.line add the test_location line [close the sales order line window]. Now after the entry before hitting save i change a field say partner_id and then click save.
CASE 3:
this case is same as case 2 but with the addition that i again change the partner_id field [changes made total 2 times first of case2 and then now], then i click on save.
RESULTS
CASE 1 works fine.
CASE 2 has a issue of
odoo.sql_db: bad query: INSERT INTO "test_module2" ("id", "field2", "field1", "location_id", "create_uid", "write_uid", "create_date", "write_date") VALUES(nextval('test_module2_id_seq'), 27, 'asd', ARRAY[1, '1'], 1, 1, (now() at time zone 'UTC'), (now() at time zone 'UTC')) RETURNING id
ProgrammingError: column "location_id" is of type integer but expression is of type integer[]
LINE 1: ...VALUES(nextval('test_module2_id_seq'), 27, 'asd', ARRAY[1, '...
now for this case i put a debugger on create/write method of sale.order.line to see waht the values are getting passed..
values = {u'product_uom': 1, u'sequence': 0, u'price_unit': 885, u'product_uom_qty': 1, u'qty_invoiced': 0, u'procurement_ids': [[5]], u'qty_delivered': 0, u'qty_to_invoice': 0, u'qty_delivered_updateable': False, u'customer_lead': 0, u'analytic_tag_ids': [[5]], u'state': u'draft', u'tax_id': [[5]], u'test_location': [[5], [0, 0, {u'field1': u'asd', u'location_id': [1, u'1']}]], 'order_id': 20, u'price_subtotal': 885, u'discount': 0, u'layout_category_id': False, u'product_id': 29, u'price_total': 885, u'invoice_status': u'no', u'name': u'[CARD] Graphics Card', u'invoice_lines': [[5]]}
in the above values location_id is getting passed like u'location_id': [1, u'1']}]] which is not correct...so for this i correct the issue in code and the update the values and pass that...
CASE 3
if the user changes the field say 2 or more than 2 times then the values are
values = {u'invoice_lines': [[5]], u'procurement_ids': [[5]], u'tax_id': [[5]], u'test_location': [[5], [1, 7, {u'field1': u'asd', u'location_id': False}]], u'analytic_tag_ids': [[5]]}
here
u'location_id': False
MULTIPLE CASE
if the user does case 1 the on the same record does case 2 or case 3 then sometimes the line will be saved as field2 = Null or False in the database other values like location_id and field1 will have data but not field2
NOTE: THIS HAPPENS WITH ANY FIELD NOT ONLY PARTNER_ID FIELD ON HEADER LEVEL OF SALE ORDER
I tried debugging myself but couldn't find the reason why this is happening .