Merge queries matching on dates and leave null if no match found - merge

I have got two tables below (examples):
Table SFID
Sales Force ID
Type
Name
Assistant
From
To
123
ABC
Store A
Ben
01/04/2020
30/04/2020
123
ABC
Store A
Jen
01/05/2020
31/05/2020
123
ABC
Store A
Ben
01/06/2020
21/06/2020
126
DEF
Store B
Tim
01/04/2020
30/04/2020
126
DEF
Store B
Tim
01/04/2020
null
and
Table Activity
Transaction ID
Date
Sales Force ID
1
03/05/2020
123
2
03/06/2020
200
3
01/01/2021
123
4
02/01/2021
126
I want my end result to be
Transaction ID
Date
Sales Force ID
Type
Name
Assistant
1
03/05/2020
123
ABC
Store A
Jen
2
03/06/2020
200
null
null
null
3
01/01/2021
123
null
null
null
4
02/01/2021
126
DEF
Store B
Tim
To do this, the best solution was the one posted in here with some modifications (allow both To and From to be null). However, only the row on transaction ID 2 disappears because that Sales Force had already had assistant entries (they get wiped out on the filtered row step). I also tried the solution presented in here but it takes ages to load.
I would like to know if there was a way to guarantee all transactions appear without having to introduce lines to table SFID for periods they don't have assistants and without making the query a really slow one.
This is my code:
let
Source = Source,
#"Merged Queries" = Table.NestedJoin(Source,{"Sales Force ID"},SFID,{"SFID"},"SFID",JoinKind.LeftOuter),
#"Expanded SFID" = Table.ExpandTableColumn(#"Merged Queries", "SFID", {"Type", "Name", "Assistant", "From", "To"}, {"Type", "Name", "Assistant", "From", "To"}),
#"Changed Type" = Table.TransformColumnTypes(#"SFID",{{"Date", type date}, {"From", type date}, {"To", type date}}),
FilteredRows = Table.SelectRows(#"Changed Type", each ([Date] >= [From] and [Date] <= [To]) or ([Date] >= [From] and [To] = null)or ([From] = null and [To] = null)),
#"Removed Columns" = Table.RemoveColumns(FilteredRows,{"From", "To"})
in
#"Removed Columns"

So after almost 2 weeks with no response and an unjustified downgrade, I found a solution!
I created a query that would basically produce this table with the above mentioned code.
Transaction ID
Date
Assistant
1
01/01/2021
Jen
4
02/01/2021
Tim
let
Source = Source,
#"Merged Queries" = Table.NestedJoin(Source,{"Sales Force ID"},SFID,{"SFID"},"SFID",JoinKind.LeftOuter),
#"Expanded SFID" = Table.ExpandTableColumn(#"Merged Queries", "SFID", {"Type", "Name", "Assistant", "From", "To"}, {"Type", "Name", "Assistant", "From", "To"}),
#"Changed Type" = Table.TransformColumnTypes(#"Expanded SFID",{{"Date", type date}, {"From", type date}, {"To", type date}}),
FilteredRows = Table.SelectRows(#"Changed Type", each ([Date] >= [From] and [Date] <= [To]) or ([Date] >= [From] and [To] = null)),
#"Removed Columns" = Table.RemoveColumns(FilteredRows,{"From", "To"})
in
#"Removed Columns"
And left-joined it on my initial version of Table Activity
I kept the information of type Type and Name in a separate Query (as they don't change) and then again left-joined it on the Table Activity.
Final query looks like this:
let
Source = Source,
#"Merged Queries1" = Table.NestedJoin(Source,{"Sales Force Code"},Info,{"SFID"},"Info",JoinKind.LeftOuter),
#"Expanded Info" = Table.ExpandTableColumn(#"Merged Queries1", "Info", {"Type", "Name"}, {"Type", "Name"}),
#"Merged Queries" = Table.NestedJoin(#"Expanded Info",{"ID"},IDvsAssistant,{"ID"},"IDvsAssistant",JoinKind.LeftOuter),
#"Expanded IDvsAssistant" = Table.ExpandTableColumn(#"Merged Queries", "IDvsAssistant", {"Assistant"}, {"Assistant"})
in
#"Expanded IDvsAssistant"

Related

Merging tables by date range in PowerBI

I have 2 tables:
Table 1: enrollment
student_id enrollment_date
1000 4/26/2022
1001 11/15/2022
1002 6/6/2022
1003 9/6/2022
1001 9/14/2022
1004 9/21/2022
1002 11/13/2022
1005 12/27/2022
Table 2: promotion
promo_id promotion promo_start promo_end
1 30% Promo 9/5/2022 9/26/2022
2 Christmas Promotion 12/25/2022 12/31/2022
3 11.11 Promotion 11/11/2022 11/15/2022
What I want to achieve is to merge 2 tables together based on date range defined in promotion table. If the enrollment_date did not fall into any promotion period, then just leave it as null. Below is the expected output:
Any advise or help will be greatly appreciated!
Here you go.
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("RYxBCoAgEEWvIgPtIv0jBq27QHtp0a4WJaj3J0lz/u495o33BFVHI1kzqC2GO6i+YhftNBvmn3j+kPbRE8vZesYr5ftI9UW+wlMsWHPLK1lIbaUGJkDKbjUgdSHX6v0F", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [promo_id = _t, promotion = _t, promo_start = _t, promo_end = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"promo_id", Int64.Type}, {"promotion", type text}, {"promo_start", type text}, {"promo_end", type text}}),
#"Changed Type with Locale" = Table.TransformColumnTypes(#"Changed Type", {{"promo_end", type date},{"promo_start", type date}}, "en-US")
in
#"Changed Type with Locale"
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("dc7LCcAwDAPQVYrPAVnKp2SW0P3XKA0lrQ/x9SHJYxjd/ZhnyQrUIJfsSpO4iARrMC1riKm8pGPX18ESqPxI3Cw9X+Rg9TNB52vXDQ==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [student_id = _t, enrollment_date = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"student_id", Int64.Type}, {"enrollment_date", type text}}),
#"Changed Type with Locale" = Table.TransformColumnTypes(#"Changed Type", {{"enrollment_date", type date}}, "en-US"),
#"Added Custom" = Table.AddColumn(#"Changed Type with Locale", "promo_id", (x)=> Table.SelectRows(promotion, (y)=> x[enrollment_date] >= y[promo_start] and x[enrollment_date] <= y[promo_end]){0}?[promo_id]?)
in
#"Added Custom"

Group by with multiple conditions Power Query Power BI

I'm new to Power Bi.
I want to calculate Outstanding Jira Tickets and group the tickets yearly+Monthly. I did it using excel.
There are 7 columns(ticketID,CreatedDate,ResolvedDate,Createdyear,CreatedMonth,ResolvedYear and ResolvedMonth). What I want to do is calculate outstanding tickets for each year and month(EX: for 2021 Jan, How many outstanding tickets?).
The calculation mechanism is explained below.
To calculate Outstanding ticekts for eachyear(X) monthly(Y) wise,below logic is used.
A = All not resolved tickets(tickets with null value for resolvedDate)
B = Resolved null and CreatedYear = X and CreatedMonth >Y
C=ResolvedYear= X and ResolvedMonth>Yand CreatedYear<X
D= ResolvedYear = X and ResolvedMonth>Y and Createdyear= X and CreatedMonth<=Y
Outstanding tickets for Year(X) AND Month(Y)= A-B+C+D
Example:
Lets say if i want to calculate outstanding ticekts for 2022 May, then i need to find using below conditions.
A= All not resolved tickets(tickets with null value for resolvedDate)
B=Resolved null and CreatedYear 2022 and createdMonth>May
C=ResolvedYear 2022 and ResolvedMonth>May and CreatedYear<2022
D= ResolvedYear 2022 and ResolvedMonth>May and Createdyear 2022 and CreatedMonth<=May
Outstanding tickets for 2022 May= A-B+C+D
Please check the excel file using the attached link.
Corresponding excel query:
Used Excel Query=
=COUNTIF(ResolvedDate,"")-COUNTIFS(ResolvedDate,"",Created_Year,G2,Created_Month,CONCAT(">",MONTH(DATEVALUE(H2&" 1"))))+COUNTIFS(Resolved_Year,G2,Resolved_Month,CONCAT(">",MONTH(DATEVALUE(H2&" 1"))),Created_Year,CONCAT("<",G2))+COUNTIFS(Resolved_Year,G2,Resolved_Month,CONCAT(">",MONTH(DATEVALUE(H2&" 1"))),Created_Year,G2,Created_Month,CONCAT("<=",MONTH(DATEVALUE(H2&" 1"))))
What i tried with PowerBI:
So now i want to do this in PowerBI,
i get the data from excel and groupby using year and month using count rows, my idea is to change the groupby code in applied steps.
= Table.Group(#"Changed Type", {"CreatedYear", "CreatedMonth"}, {{"Count", each Table.RowCount(_), Int64.Type}})
but the problem is I don't know how to get summation and subtraction like i did it in powerBI.
Can someone please provide detailed instructions on creating columns and calculations or measures with formulas?
Assuming Table1 looks like
You can use this code for Table1
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Created", type date}, {"Resolved", type date}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each #date(2021,1,1)),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "Custom.1", each Date.EndOfMonth([Custom])),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom1",{"Custom", "Custom.1"}),
#"Added Custom2" = Table.AddColumn(#"Removed Columns", "Create.Year", each Date.Year([Created])),
#"Added Custom3" = Table.AddColumn(#"Added Custom2", "Create.Month", each Date.Month([Created])),
#"Added Custom4" = Table.AddColumn(#"Added Custom3", "Resolve.Year", each Date.Year([Resolved])),
#"Added Custom5" = Table.AddColumn(#"Added Custom4", "Resolve.Month", each Date.Month([Resolved]))
in #"Added Custom5"
and get
Assuming Table2 looks like
you can use this code for Table2
let Source = Excel.CurrentWorkbook(){[Name="Table2"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Year", Int64.Type}, {"Month", type text}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "MonthNumber", each Date.Month(Date.FromText([Month]&"-01-2020"))),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "a",(i)=>Table.RowCount(Table.SelectRows(Table1, each [Resolved]=null))),
#"Added Custom2" = Table.AddColumn(#"Added Custom1", "b",(i)=>Table.RowCount(Table.SelectRows(Table1, each [Resolved]=null and i[Year]=[Create.Year] and [Create.Month]> i[MonthNumber]))),
#"Added Custom3" = Table.AddColumn(#"Added Custom2", "c",(i)=>Table.RowCount(Table.SelectRows(Table1, each [Resolve.Year]=i[Year] and [Resolve.Month]>i[MonthNumber] and [Create.Year]< i[Year]))),
#"Added Custom4" = Table.AddColumn(#"Added Custom3", "d",(i)=>Table.RowCount(Table.SelectRows(Table1, each [Resolve.Year]=i[Year] and [Resolve.Month]>i[MonthNumber] and [Create.Year]= i[Year] and [Create.Month]<=i[MonthNumber]))),
#"Added Custom5" = Table.AddColumn(#"Added Custom4", "Outstanding", each [a]-[b]+[c]+[d]),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom5",{"MonthNumber", "a", "b", "c", "d"})
in #"Removed Columns"
and get

Power Query - creating a tracker for multiple studies

New to Power Query and trying to learn how I can use it more to optimize workflows.
Here, I have multiple tables (1 for each study), and am trying to create a tracker that can identify the subject, whether or not they are participating in a specific study, and other data (e.g., study ID). Each table has:
Unique subject identifier (which could be used to link them across studies)
Study-specific identifier
Other misc info (dates of test1, test2, etc)
I am trying to make something that groups the columns of all my tables by the unique subject identifier, and then could have columns for "Study1", "Study 2", "Study 3" with a simple Y/N to show whether or not they are participating in that study. If Y, then also show the study-specific identifier. However, I am stuck.
I've appended all 3 tables into 1 master table - which results in duplicates as a participant can be in multiple studies. When I try the "Group By" function, I group by "Unique subject identifier" with new columns Study1, Study2, Study3, etc and operation "All rows". When I expand these however, it creates multiple duplicate rows, which defeats the purpose of the "Group by" function.
Would appreciate any suggestions you may have.
Example
Each table has some variation of these columns:
Unique participant identifier
Unique Study identifier
Date of Test 1, etc
A
100
01-Apr-2022
B
101
02-Apr-2022
C
102
03-Apr-2022
Let's say Participant A is on study 1 only. Participant B is on Study 2 (study ID 201) and 3 (study ID 301). Participant C is on all 3 (study IDs 102, 202, and 302 respectively).
I am trying to make a table that will show:
Unique participant identifier
Study 1
Study 1 Identifier
Study 2
Study 2 Identifier
Study 3
Study 3 Identifier
A
Y
100
B
Y
101
Y
201
Y
301
C
Y
102
Y
302
Alongside the dates of tests (not shown, but same concept). These tables are updated as we go along, so power query would draw these data from the tables to create a "live" tracker.
The current code I have in the advanced editor is:
let
Source = Table.Combine({STUDY1, STUDY2, STUDY3}),
#"Grouped Rows" = Table.Group(#"Removed Columns", {"Unique participant identifier"}, {{"STUDY12", each _, type table}, {"STUDY22", each _, type table}, {"STUDY32", each _, type table}}),
#"Expanded STUDY12" = Table.ExpandTableColumn(#"Grouped Rows", "STUDY12", {"STUDY1-Study ID", "STUDY1-ABC Study ID", "STUDY1"}, {"STUDY12.STUDY1-Study ID", "STUDY12.STUDY1-ABC Study ID", "STUDY12.STUDY1"}),
#"Expanded STUDY22" = Table.ExpandTableColumn(#"Expanded STUDY12", "STUDY22", {"STUDY2-Study ID", "STUDY2-OC Study ID", "STUDY2"}, {"STUDY22.STUDY2-Study ID", "STUDY22.STUDY2-ABC Study ID", "STUDY22.STUDY2"}),
#"Expanded STUDY32" = Table.ExpandTableColumn(#"Expanded STUDY22", "STUDY32", {"STUDY3-Study ID", "STUDY3"}, {"STUDY32.STUDY3-Study ID", "STUDY32.STUDY3"}),
#"Reordered Columns" = Table.ReorderColumns(#"Expanded STUDY32",{" Unique participant identifier ", "STUDY12.STUDY1-Study ID", "STUDY12.STUDY1", "STUDY12.STUDY1-ABC Study ID", "STUDY22.STUDY2", "STUDY22.STUDY2-Study ID", "STUDY22.STUDY2-ABC Study ID", "STUDY32.STUDY3", "STUDY32.STUDY3-Study ID"})
in
#"Reordered Columns"
Done via the "Group by" function. This creates multiple rows for each participant though.
Here's one method. Your sample data is not that great. You also dont mention how to divide the Unique Study identifier into buckets. I assume it is just the integer portion of the number divided 100. This is a bit long to attempt to make the steps obvious
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Unique participant identifier", type text}, {"Unique Study identifier", Int64.Type}, {"Date of Test", type date}}),
// there needs to be some way to identify groups of studies. I am using the hundreds place to find that
#"Added Custom" = Table.AddColumn(#"Changed Type", "StudyNumber", each Number.RoundDown([Unique Study identifier]/100)),
// first table -- studies only
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Date of Test"}),
#"Pivoted Column" = Table.Pivot(Table.TransformColumnTypes(#"Removed Columns", {{"StudyNumber", type text}}, "en-US"), List.Distinct(Table.TransformColumnTypes(#"Removed Columns", {{"StudyNumber", type text}}, "en-US")[StudyNumber]), "StudyNumber", "Unique Study identifier", List.Sum),
// second tabe -- study dates only
#"Removed Columns2" = Table.RemoveColumns(#"Added Custom",{"Unique Study identifier"}),
#"Pivoted Column1" = Table.Pivot(Table.TransformColumnTypes(#"Removed Columns2", {{"StudyNumber", type text}}, "en-US"), List.Distinct(Table.TransformColumnTypes(#"Removed Columns2", {{"StudyNumber", type text}}, "en-US")[StudyNumber]), "StudyNumber", "Date of Test"),
RenamedColumns = Table.TransformColumnNames(#"Pivoted Column1", each if _="Unique participant identifier" then _ else _&" Date"),
#"Removed Columns1" = Table.RemoveColumns(RenamedColumns,{"Unique participant identifier"}),
// combine the tables
combined = Table.FromColumns(Table.ToColumns(#"Pivoted Column")&Table.ToColumns( #"Removed Columns1"),Table.ColumnNames(#"Pivoted Column") &Table.ColumnNames(#"Removed Columns1"))
in combined
code version for yes/no
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Unique participant identifier", type text}, {"Unique Study identifier", Int64.Type}, {"Date of Test", type date}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "StudyNumber", each Number.RoundDown([Unique Study identifier]/100)),
// first table -- studies only
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Date of Test"}),
#"Rounded Off" = Table.TransformColumns(#"Removed Columns",{{"StudyNumber", each Text.From(_) &" Identifier"}}),
Part1 = Table.Pivot(Table.TransformColumnTypes(#"Rounded Off", {{"StudyNumber", type text}}, "en-US"), List.Distinct(Table.TransformColumnTypes(#"Rounded Off", {{"StudyNumber", type text}}, "en-US")[StudyNumber]), "StudyNumber", "Unique Study identifier", List.Sum),
// 2nd table -- YN only
#"Change to Y" = Table.TransformColumns(#"Removed Columns",{{"Unique Study identifier",each "Y" }}),
#"Rounded Off1" = Table.TransformColumns(#"Change to Y",{{"StudyNumber", each "Study " &Text.From(_)}}),
Pivot = Table.Pivot(Table.TransformColumnTypes(#"Rounded Off1", {{"StudyNumber", type text}}, "en-US"), List.Distinct(Table.TransformColumnTypes(#"Rounded Off1", {{"StudyNumber", type text}}, "en-US")[StudyNumber]), "StudyNumber", "Unique Study identifier"),
Part2 = Table.RemoveColumns(Pivot,{"Unique participant identifier"}),
// 3rd table -- study dates only
#"Removed Columns2" = Table.RemoveColumns(#"Added Custom",{"Unique Study identifier"}),
#"Pivoted Column" = Table.Pivot(Table.TransformColumnTypes(#"Removed Columns2", {{"StudyNumber", type text}}, "en-US"), List.Distinct(Table.TransformColumnTypes(#"Removed Columns2", {{"StudyNumber", type text}}, "en-US")[StudyNumber]), "StudyNumber", "Date of Test"),
RenamedColumns3 = Table.TransformColumnNames(#"Pivoted Column", each if _="Unique participant identifier" then _ else _&" Date"),
Part3 = Table.RemoveColumns(RenamedColumns3,{"Unique participant identifier"}),
// combine the tables
combined = Table.FromColumns(Table.ToColumns(Part1)&Table.ToColumns( Part2)&Table.ToColumns( Part3),Table.ColumnNames(Part1) &Table.ColumnNames(Part2)&Table.ColumnNames(Part3))
in combined

How to create a measure that show the count on previous month (PowerBI)

Hi everyone,
I want to create a column in the matrix table to show the count of OK for previous month, I named the measure as Prev Month OK count as shown in the screenshot above. However, the output is not what I want based on what I tried. This is my formula:
Prev Month OK count = CALCULATE([OK count], DATEADD('Aggregate'[Intake],-1,MONTH))
Even I change the MONTH parameter to QUARTER in DATEADD, it doesn't work as well, there is only a value in the Total which I'm not sure what does the value means.
The screenshot above is the expected output that I want. Take note that [OK count] is also a measure.
Any help or advise will be greatly appreciated!
Screenshot for Date Column
You can do it in Power Query and create a new table using this following code-
let your table name is your_old_table_name
let you have these 2 column there - date and value
Now create a new table **your_new_table_name" as below-
let
Source = your_old_table_name,
#"Grouped Rows" = Table.Group(Source, {"date"}, {{"_sum", each List.Sum([value]), type nullable number}}),
#"Sorted Rows" = Table.Sort(#"Grouped Rows",{{"date", Order.Ascending}}),
#"Added Index" = Table.AddIndexColumn(#"Sorted Rows", "Index", 1, 1, Int64.Type),
#"Added Custom" = Table.AddColumn(#"Added Index", "Custom", each [Index] + 1),
#"Merged Queries" = Table.NestedJoin(#"Added Custom", {"Index"}, #"Added Custom", {"Custom"}, "Added Custom", JoinKind.LeftOuter),
#"Expanded Added Custom" = Table.ExpandTableColumn(#"Merged Queries", "Added Custom", {"_sum"}, {"Added Custom._sum"}),
#"Removed Columns" = Table.RemoveColumns(#"Expanded Added Custom",{"Index", "Custom"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Added Custom._sum", "previous_date_sum"}})
in
#"Renamed Columns"
You will have an output similar as below-

Show Visual Studio Online Work Item Cycle Times

Ideally, I would like to calculate the cycle time of the whole value stream represented by Product Backlog Items in Visual Studio Online via Power BI. (I would then love to get time per status, i.e. how long was it sat in the "New" state, or how long was it stuck in the "Committed" state.)
To start with, I'm interested in using the calculated value that represents the time between the created date and the closed date of the product backlog item. After this, I'm keen on getting the distribution of values found.
product backlog item time between created timestamp to closed timestamp
Would be a starting point, but this shows "queued duration minutes by title", of course.
Most other attempts result in:
Can’t determine relationships between the fields
Is there a way to get some indication of cycle times?
Assuming each rows has a created date and closed date, you could use the power query duration function to get the difference between the two dates.
https://msdn.microsoft.com/en-us/library/mt296613.aspx
Update: After looking into it (see my comments below) I realize that it takes some Olympic-level power query skills to build the query you need. So I created it for you :). The below works in Power BI desktop, but doesn't refresh in PowerBI.com for some reason (I'm checking into that). To make this work, you need to put your VSO account and the Query ID in the ConnectionInfo record.
Recommend setting up an alternate credential rather than using BASIC auth.
You can get a QueryID by saving a shared query to your VSO and then using curl to retrieve the id using the query:
//Replace YOURACCOUNT and YOURPROJECT with correct values below
curl -u username:password https://YOURACCOUNT.visualstudio.com/DefaultCollection/YOURPROJECT/_apis/wit/queries?$depth=1&api-version=1.0
Then you can use the following query (create a blank query and paste it into the advanced editor in Power BI Desktop). You might need to adjust some of the steps starting at "workitems =" since your query might have different fields than mine did. If you return created, resolved and closed dates, the duration calculations I used should work for you. You'll set the authentication to "Basic" when challenged in Power BI Desktop and supply the required credentials.
let
//TODO: replace YOURACCOUNT and YOURQUERYID below
ConnectionInfo = [account = "YOURACCOUNT.visualstudio.com", queryID="YOURQUERYID"],
account = Record.FieldValues(ConnectionInfo){0},
rootQuery = let
queryID = Record.FieldValues(ConnectionInfo){1},
query = "https://" & account & "/DefaultCollection/PowerBIClients/_apis/wit/wiql/" & queryID,
Source = Json.Document(Web.Contents(query))
in
Source,
#"INT-columns" = let
Source = rootQuery,
columns = Source[columns],
#"Converted to Table" = Table.FromList(columns, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Expanded Column1" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"referenceName", "name", "url"}, {"referenceName", "name", "url"})
in
#"Expanded Column1",
#"RAW-workitems" = let
Source = rootQuery,
workItems = Source[workItems],
#"Converted to Table" = Table.FromList(workItems, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Expanded Column1" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"id", "url"}, {"id", "url"})
in
#"Expanded Column1",
#"INT-columnClause" = let
Source = #"INT-columns",
colNames = Table.RemoveColumns(Source,{"name", "url"}),
list = Table.ToList(colNames),
joined = Text.Combine(list, ",")
in
joined,
#"INT-workitemsToGet" = let
Source = #"RAW-workitems",
col = Table.RemoveColumns(Source,{"url"}),
#"Changed Type" = Table.TransformColumnTypes(col,{{"id", type text}}),
list = Table.ToList(#"Changed Type"),
l = List.Count(list),
limits = List.Generate(()=>0, each _ < l, each _ + 100),
#"Converted to Table" = Table.FromList(limits, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Renamed Columns" = Table.RenameColumns(#"Converted to Table",{{"Column1", "iterations"}}),
#"Added Custom" = Table.AddColumn(#"Renamed Columns", "itemsToGet", each List.Range(list, [iterations], 100)),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "itemsToGetString", each Text.Combine([itemsToGet], ",")),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom1",{"iterations", "itemsToGet"})
in
#"Removed Columns",
#"INT-workitemRequests" = let
Source = #"INT-workitemsToGet",
#"Added Custom" = Table.AddColumn(Source, "requests", each "https://" & account & "/DefaultCollection/_apis/wit/workitems?ids=" & [itemsToGetString] & "&fields=" & #"INT-columnClause"),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"itemsToGetString"})
in
#"Removed Columns",
workitems = let
#"INT-workitemsRequests (2)" = let
Source = #"INT-workitemRequests",
results = Table.AddColumn(Source, "Results", each Json.Document(Web.Contents([requests]))),
out = 1
in
results,
#"Removed Columns" = Table.RemoveColumns(#"INT-workitemsRequests (2)",{"requests"}),
#"Expanded Results" = Table.ExpandRecordColumn(#"Removed Columns", "Results", {"count", "value"}, {"Results.count", "Results.value"}),
#"Expanded Results.value" = Table.ExpandListColumn(#"Expanded Results", "Results.value"),
#"Expanded Results.value1" = Table.ExpandRecordColumn(#"Expanded Results.value", "Results.value", {"id", "fields", "url"}, {"Results.value.id", "Results.value.fields", "Results.value.url"}),
#"Expanded Results.value.fields" = Table.ExpandRecordColumn(#"Expanded Results.value1", "Results.value.fields", {"System.Id", "System.WorkItemType", "System.State", "System.AssignedTo", "System.CreatedDate", "System.Title", "Microsoft.VSTS.Common.ResolvedDate", "Microsoft.VSTS.Common.ClosedDate"}, {"System.Id", "System.WorkItemType", "System.State", "System.AssignedTo", "System.CreatedDate", "System.Title", "Microsoft.VSTS.Common.ResolvedDate", "Microsoft.VSTS.Common.ClosedDate"}),
#"Removed Columns1" = Table.RemoveColumns(#"Expanded Results.value.fields",{"Results.count", "Results.value.id"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns1",{{"Results.value.url", "WorkItemUrl"}}),
#"Changed Type" = Table.TransformColumnTypes(#"Renamed Columns",{{"Microsoft.VSTS.Common.ResolvedDate", type datetime}, {"Microsoft.VSTS.Common.ClosedDate", type datetime}, {"System.CreatedDate", type datetime}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "DurationToResolved", each Duration.TotalDays([Microsoft.VSTS.Common.ResolvedDate] - [System.CreatedDate])),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "DurationToClosed", each Duration.TotalDays([Microsoft.VSTS.Common.ClosedDate] - [System.CreatedDate])),
#"Changed Type1" = Table.TransformColumnTypes(#"Added Custom1",{{"DurationToResolved", type number}, {"DurationToClosed", type number}})
in
#"Changed Type1"
in workitems