How to parameterize a column for aggregation in Power BI desktop? - group-by

I have users who would like to be able to modify what columns a table aggregates by. My issue is that I seem unable to do this in Power BI. I basically want to be able to do the following in SQL:
SELECT
<OrgLevel1>,
<OrgLevel2>,
SUM([Revenue])
FROM [Data]
GROUP BY
<OrgLevel1>,
<OrgLevel2>
;
where the user can change <OrgLevel1> and/or <OrgLevel2> to be any of { "(All)", [Department], [Product] }.
The issue may be related to this post: https://community.powerbi.com/t5/Desktop/Calculated-Column-Table-Change-Dynamically-According-to-Slicer/m-p/655991#M314800
Here's a link to a workbook that illustrates this issue, TestParameterizeGroupby.pbix (hosted by Google Drive). I've also included field definitions below with screenshots. Thanks for any help.
TestParameterizeGroupby.pbix
Link: TestParameterizeGroupby.pbix (hosted by Google Drive)
Problem
[Org Level 1] and [Org Level 2] fields are not recalculating from the users' selection. Only the default values are shown.
Expected result in table
"Org Level 1", "Org Level 2", "Revenue"
"(All)", "(All)", 28
Note
The purpose is to have parameterizable organization level fields so that the report user can aggregate by all, department, product, or both in either order.
Table and column definitions
'Data' = DATATABLE(
"Department",
STRING,
"Product",
STRING,
"Revenue",
DOUBLE,
{
{"DeptA", "ProdX", 5.0},
{"DeptA", "ProdY", 6.0},
{"DeptB", "ProdX", 10.0},
{"DeptB", "ProdY", 7.0}
}
)
'Data'[Org Level 1] = SWITCH(
'Org Level 1 Parameter'[Org Level 1 Parameter Value],
0,
"(All)",
1,
[Department],
2,
[Product]
)
// Problem: [Org Level 1] and [Org Level 2] fields are not recalculating from the users' selection. Only the default values are shown.
'Org Level 1' = DATATABLE(
"Org Level 1",
STRING,
"Org Level 1 Parameter",
INTEGER,
{
{"(0) (All)", 0},
{"(1) Department", 1},
{"(2) Product", 2}
}
)
'Org Level 1 Parameter'[Org Level 1 Parameter] = GENERATESERIES(0, 2, 1)
'Org Level 1 Parameter'[Org Level 1 Parameter Value] = SELECTEDVALUE('Org Level 1 Parameter'[Org Level 1 Parameter], 1)
Table 'Org Level 1' has a 1-1 relationship with 'Org Level 1 Parameter' on column [Org Level 1 Parameter].
The user selects the value for 'Data'[Org Level 1] by selecting the value for 'Org Level 1'[Org Level 1].
Tables and columns for [Org Level 2] are defined in the same way as [Org Level 1].
Screenshots
Report view:
Data view:
Model view:
Cross-reference to post in Power BI forum:
Power BI Forum: How to parameterize a column for aggregation

One solution to this is to add two list values parameters and use their values in Power Query M code to modify the database query. Lets assume that you have a table Data with columns Department, Product and Revenue. For simplicity I will add one more column, named Dummy Column, with all rows having the same value (e.g. null). I will explain why later in this post. So the table looks like this:
Then in your report specify a query when adding this table to your model (lets assume we will import it, but in general you can do this in DirectQuery too):
Now if you look the M code you will see the above query there:
Source = Sql.Database(".", "StackOverflow", [Query=" select ....
Now define couple of parameters, that the end-user can use to select how the data should be aggregated. Lets name them Level 1 and Level 2:
The value of a parameter can be used in M by parameter name, and & is used to concatenate strings. So if there is a parameter Name with value Samuel, the expression "Hello, " & Name & "!" will be evaluated as Hello, Samuel!. The idea is to check the value of our parameters and modify the database query accordingly.
In the select part, we will replace the name of the field selected, or we will put '' (empty string) in case of <All> (I surrounded parameter values with brackets to be more easily to distinguish parameter values from database field names). So the expression should look like:
"select " & (if #"Level 1" = "<Department>" then "Department" else ..." (and so on)
Because there is a space in our parameter's name, we need to surround it with #" and ", so Level1 can be referenced simply as Level1 in the code, but Level 1 becomes #"Level 1".
The group by part is a bit trickier. We should add a comma between field names, add or not field name, or even omit the group by at all (in case both parameters are set to <All>). To simplify this, I added one dummy column, with all rows having the same value (e.g. null) and always group by this column. This way building the group by clause is way more simpler - in case the parameter value is not <All>, we should add , fieldname. So the code could look like this:
"group by DummyColumn" & (if #"Level 1" = "<Department>" then ", Department" else ..." (and so on)
So the final M code is this:
let
Source = Sql.Database(".", "StackOverflow", [Query="select#(lf) " & (if #"Level 1" = "<Department>" then "Department" else if #"Level 1" = "<Product>" then "Product" else "''") & " as [Org Level 1]#(lf) , " & (if #"Level 2" = "<Department>" then "Department" else if #"Level 2" = "<Product>" then "Product" else "''") & " as [Org Level 2]#(lf) , SUM(Revenue) as Revenue#(lf)from Data#(lf)group by DummyColumn" & (if #"Level 1" = "<Department>" then ", Department" else if #"Level 1" = "<Product>" then ", Product" else "") & (if #"Level 2" = "<Department>" then ", Department" else if #"Level 2" = "<Product>" then ", Product" else "")])
in
Source
Now the end-user can change parameter values, by clicking Edit Queries -> Edit Parameters:
And select how to group the data:
By default, Power BI Desktop will warn you first time, when particular query is executed:
If you want to turn this off, go to File -> Options and settings -> Options -> (GLOBAL) Security and make sure Require user approval for new native database queries is not selected:
When the end-user changes parameter values, the data will change too, e.g.:
Or:
And so on...
This trick works well in Power BI Desktop when every user has its own copy of the .pbix file. However, if you publish it, first changing parameter values is not very convenient (you must go to datasat's settings) and more important, changing parameter values affects all users, which are looking at this report. You can also use it to modify Table.Group statements generated by Power Query Editor, in case you want to aggregate the data in Power BI, but changing the database query is easier and more flexible.
If you want to enable this scenario for concurrent multi-users scenarios for published reports, you can use slicers and What-if parameters. Unfortunately, What-if parameters can be numeric (you can't define the list of values there), so you can use measures to "decode" the int value of the parameter and write some DAX code to perform different aggregations accordingly. It is more work, but if it is needed, it can be made too.

Related

Is it possible to iterate through a single API call to a SharePoint list using Power Query?

Strange use case here. I have a list with approximately 18000 rows. I have specific lookup columns that show up as expected when I use the SharePoint Online list connector in Power BI. This method is horribly slow though. I stumbled upon a guide by a Power BI user Hoosier BI that outlines a way to get list items using REST API. When I use a REST API call, however, results return but the same columns I want are suspiciously absent.
I can hit the list in a web browser and see that when using https://mysharepointurl/mysharepointsite/_api/web/lists/GetByID('mylistid')/Items
all 18000 rows are accounted for but the columns I want are not there. This changes when I call the item number specifically and view field values as text
https://mysharepointurl/mysharepointsite/_api/web/lists/GetByID('mylistid')/Items(1)/FieldValuesAsText
All columns are available and expandable, including the lookup columns that were previously missing.
If using Power Query, is there a way to incrementally iterate through each item in my list and return results based on ViewFieldsAsText call?
If not, am I missing something simple here? I've taken a look at the list in SharePoint, changed the default view to include all columns I want, made sure those columns are indexed, and more that I can't recall off the top of my head.
Thoughts?
For reference, this is the M query I was using to retrieve all rows initially:
sitename = "", // if a subsite use "Site/SubSite"
listname = "BigList",
baseurl = "https:///sites/"
& sitename
& "/_api/web/lists/GetByTitle('"
& listname
& "')/",
itemcount = Json.Document(
Web.Contents(baseurl & "ItemCount", [Headers = [Accept = "application/json"]])
)[value],
skiplist = List.Numbers(0, Number.RoundUp(itemcount / 5000), 5000),
#"Converted to Table" = Table.FromList(
skiplist,
Splitter.SplitByNothing(),
null,
null,
ExtraValues.Error
),
#"Renamed Columns" = Table.RenameColumns(#"Converted to Table", {{"Column1", "Skip"}}),
#"Changed Type" = Table.TransformColumnTypes(#"Renamed Columns", {{"Skip", type text}}),
fieldselect = "&$top=5000", // all fields with no expansion
//fieldselect = "&$top=5000&$select = Id,Title,Person,Date", // list desired fields (no expansion)
//fieldselect = "&$top=5000&$select=Id,Title,Choice,LookupColumn/Title,LookupColumn/Project,LookupColumn/ProjectStatus,Date,Person/LastName,Person/FirstName,Person/EMail&$expand=LookupColumn,Person",
Custom1 = Table.AddColumn(
#"Changed Type",
"Items",
each Json.Document(
Web.Contents(
baseurl & "/items?$skipToken=Paged=TRUE%26p_ID=" & [Skip] & fieldselect,
[Headers = [Accept = "application/json"]]
)
)
),
#"Expanded Items" = Table.ExpandRecordColumn(Custom1, "Items", {"value"}, {"value"}),
#"Expanded value" = Table.ExpandListColumn(#"Expanded Items", "value")
in
#"Expanded value"```

Accessing the ATTR field troubleshooting in Tableau?

I am working on Tableau server but I believe the problem I am facing does not correspond to tableau server specifically.
I am using two data sources ds1 and ds2 which are joined using the dimension Id . ds1 has a field city and ds2 has a field district. There is only 1 city corresponding to each Id but there can be multiple district corresponding to an Id .
I have created a calculated field Points in ds2 which is described in the code segment.
I have researched from different sites and blogs (including tableau support). I came to a close possible reason behind this and I might be wrong . The ATTR function which works on row level and identify if a row is unique then it outputs the dimension otherwise it outputs '*' . I think when I joined those two tables the district dimension in ds2 might have the '*' instead of actual district values, so it might not be able to compare the conditions in if statements of Point .
//Point//
IF [city] == "Delhi"
AND [district] == "Dist1"
AND [district] == "Dist2"
THEN "100 Section"
ELSEIF [city] == "Mumbai"
AND [district] == "Dist11"
AND [district] == "Dist12"
THEN "200 Section"
ELSE "Other Section"
END
When I insert data which satisfy the conditions in calculated field, it is going in Other section of the Point .I want it to go in desired section.
For instance
Id = 19
city = Delhi
district = Dist1
district = Dist2
district = Dist3
It should go in 100 Section but it is going in Other Section . What modifications should I do or add to make the Point work properly ?

libreoffice base create a list filtered by another list's value

I have a table of provinces and a table of cities with ProvienceID. In a form, I want to create a list of cities filtered by selected value of provience list.
How can I do that?
I can create both lists but Cities list shows all cities from all provinces but i want to show only cities from the province that I have selected in Provinces list.
I have another table "Users" with "CityID" and "ProvinceID" that my form edits it and I need to save selected values of Province and City Lists in it, not only show it in the form.
Create two example tables named "Provinces" and "Cities".
ProvinceID Name
~~~~~~~~~~ ~~~~
0 South
1 North
2 Large Midwest
3 Southeast
4 West
CityID Name ProvinceID
~~~~~~ ~~~~ ~~~~~~~~~~
0 Big City 2
1 Very Big City 2
2 Rural Village 1
3 Mountain Heights 0
4 Coastal Plains 4
5 Metropolis 2
Create a query called "ProvinceNames":
SELECT "Name" AS "Province"
FROM "Provinces"
ORDER BY "Province" ASC
Create a query called "Province of City":
SELECT "Provinces"."Name" AS "Province", "Cities"."Name" AS "City"
FROM "Cities", "Provinces" WHERE "Cities"."ProvinceID" = "Provinces"."ProvinceID"
ORDER BY "Province" ASC, "City" ASC
In the form, create a table control based on the query "ProvinceNames".
Using the Form Navigator (or the Form Wizard), create a subform for query "Province of City".
Right-click on subform and choose Properties. Under Data tab:
Link master fields "Province"
Link slave fields "Province"
Create a table control for the subform as well. Now, the cities shown in the subform control depend on the province selected in the main form control.
EDIT:
Here is an example using a filter table to store the current value of the list box. Create two more tables named "Users" and "FilterCriteria".
UserID Name ProvinceID CityID
~~~~~~ ~~~~~~~ ~~~~~~~~~~ ~~~~~~
0 Person1 1 2
1 Person2 2 0
RecordID ProvinceID CityID
~~~~~~~~ ~~~~~~~~~~ ~~~~~~
the only 0 0
We'll also need two Basic macros which can be stored in the document or in My Macros. Go to Tools -> Macros -> Organize Macros -> LibreOffice Basic.
Sub ReadProvince (oEvent as Object)
forms = ThisComponent.getDrawPage().getForms()
mainForm = forms.getByName("MainForm")
cityForm = forms.getByName("CityForm")
listboxProvince = mainForm.getByName("listboxProvince")
listboxCity = cityForm.getByName("listboxCity")
selectedItemID = listboxProvince.SelectedValue
If IsEmpty(selectedItemID) Then
selectedItemID = 0
End If
conn = mainForm.ActiveConnection
stmt = conn.createStatement()
strSQL = "UPDATE ""FilterCriteria"" SET ""ProvinceID"" = " & selectedItemID & _
"WHERE ""RecordID"" = 'the only'"
stmt.executeUpdate(strSQL)
listboxCity.refresh()
lCityCol = mainForm.findColumn("CityID")
currentCityID = mainForm.getInt(lCityCol)
cityForm.updateInt(cityForm.findColumn("CityID"), currentCityID)
listboxCity.refresh()
End Sub
Sub CityChanged (oEvent as Object)
listboxCity = oEvent.Source.Model
cityForm = listboxCity.getParent()
mainForm = cityForm.getParent().getByName("MainForm")
lCityCol = mainForm.findColumn("CityID")
selectedItemID = listboxCity.SelectedValue
If IsEmpty(selectedItemID) Then
selectedItemID = 0
End If
mainForm.updateInt(lCityCol, selectedItemID)
End Sub
Now we need to set up the form like this. In this example, I used two top-level forms instead of a subform. ProvinceID and CityID text boxes are not required but may be helpful in case something goes wrong.
To start creating this form, use the form wizard to create a new form and add all fields from the Users table.
Now, in the Form Navigator, create a form called "CityForm". Content type is SQL command, and Content is:
SELECT "RecordID", "ProvinceID", "CityID" FROM "FilterCriteria"
WHERE "RecordID" = 'the only'
Next, create the "listboxProvince" list box under MainForm. Data Field is "ProvinceID", and List content is the following Sql.
SELECT "Name", "ProvinceID" FROM "Provinces" ORDER BY "Name" ASC
Finally, create the "listboxCity" list box under CityForm. Data Field is "CityID", and List content is the following Sql.
SELECT "Name", "CityID" FROM "Cities" WHERE "ProvinceID" = (
SELECT "ProvinceID" FROM "FilterCriteria"
WHERE "RecordID" = 'the only')
Macros are linked under the Events tab of each control.
Assign "After record change" of the MainForm to ReadProvince().
Assign "Changed" of listboxProvince to ReadProvince().
Assign "Changed" of listboxCity control to CityChanged().
The result allows us to select the Province to filter the list of Cities. Provinces and Cities that are selected are saved in the Users table.
There is another approach which may be better that I have not had time to explore. Instead of the "FilterCriteria" table, apply a filter to the Cities list. The relevant code in ReadProvince() would look something like this.
cityForm.Filter = "ProvinceID=" & selectedItemID
cityForm.ApplyFilter = True
cityForm.reload()
cityForm.absolute(0)
Whatever approach is taken, a complete solution requires complex macro programming. To make it easier, you may decide to use a simpler solution that is not as powerful. For more information, there is a tutorial at https://forum.openoffice.org/en/forum/viewtopic.php?t=46470.
EDIT 2
A solution that requires fewer queries is at https://ask.libreoffice.org/en/question/143186/how-to-use-user-selected-value-from-combobox1-in-combobox2-select-statement/?answer=143231#post-id-143231. The second list box is based on a list of values instead of an SQL query.

Can I report values from one field using another as a reference point in the same table?

My report returns a default of;
"My cat is Fat and Lazy"
Field_1 Field_2
======== ========
1 Sleek
2 Athletic
However I want to replace 'Fat' with 'Sleek' and 'Lazy' with 'Athletic'
So the final string will read ;
"My cat is Sleek and Athletic"
My question is this, can I make the report pick up (via a formula) that if Field_1 = 1 to replace Fat with 'Sleek'?
Field_1 is static but Field_2 is dynamic and the values will change on depending on the data.
Depending on how you're pulling the data you could do this in a couple of ways:
In the SQL add a CASE statement:
CASE WHEN field_1 = 1 THEN 'sleek' ELSE field_2 END AS modified_field_2
In a formula field (Crystal syntax used in example):
if { table_name.field_1 } = 1 then 'sleek' else { table_name.field_2 }
Edit:
Please see the following link for an example of how to join a table to itself (a self-join) in order to "link" the two records to get the full sentence: http://sqlfiddle.com/#!2/59566/16

MDX: Calculated Dimensions?

I don't know if this is possible or not, or if my limited knowledge of MDX is pushing me in the wrong direction...
The cube I'm dealing with has two different dimensions for dates, [Statement Dates] and [Premium Dates]. Further in the hierarchy of each looks like this:
[Statement Dates].[Statement Year].[2008]
[Payment Dates].[Payment Year].[2008]
For the business logic I'm implementing I need to do something like:
select
({ [Measures].[Commission] }) on columns,
({[Products].[Product Category]}) on rows
from [Cube]
where
(
IIF( [Products].[Product Category].CurrentMember.Name = "Some Category",
[Statement Dates].[Statement Year].[2008],
[Payment Dates].[Payment Year].[2008] )
)
So I need it to discriminate what dimension to use for filtering the year based on what product category is being used.
This parses ok and the query runs, but the numbers seem to suggest that the IIF statement is always returning false.
Because the WHERE clause gets evaluated first the .CurrentMember function in the IIF will only be seeing "All Product Cateogories". In which case the [Products].[Product Category].CurrentMember.Name will never be equal to "Some Category" as the only product category in context is "All Product Cateogories".
One possible work around would be to do a calculation like the following:
WITH MEMBER Measures.Commission2 as
SUM(
{[Products].[Product Category].[Product Category].Members}
, IIF( [Products].[Product Category].CurrentMember.Name = "Some Category"
, ([Statement Dates].[Statement Year].[2008],[Measures].[Commission])
, ([Payment Dates].[Payment Year].[2008].[Measures].[Commission]) )
)
select
({ [Measures].[Commission2] }) on columns
, ({[Products].[Product Category]}) on rows
from [Cube]
You could also do a scoped assignement in the calculation script in your cube to do this sort of thing.