How to loop through a list in a M query? - azure-devops

I have a query in PowerQuery editor, which brings all projects in an Azure DevOps organization:
let
Source = OData.Feed("https://analytics.dev.azure.com/someorg/_odata/v3.0-preview/Projects", null, [Implementation="2.0", OmitValues=ODataOmitValues.Nulls, ODataVersion=4]),
ProjectName = Source[ProjectName]
in
ProjectName
Also, I have this OData query which brings back all tasks for a single project:
let
Source = OData.Feed ("https://analytics.dev.azure.com/someorg/{project}/_odata/v3.0-preview/WorkItems?"
&"$filter=WorkItemType eq 'User Story' "
&"and State ne 'Closed' "
&"and startswith(Area/AreaPath,'{areapath}') "
&"&$select=WorkItemId,Title,WorkItemType,State,Priority,Severity,TagNames,AreaSK "
&"&$expand=AssignedTo($select=UserName),Iteration($select=IterationPath),Area($select=AreaPath), "
&"Links( "
&"$filter=LinkTypeName eq 'Related' "
&"and TargetWorkItem/WorkItemType eq 'User Story'; "
&"$select=LinkTypeName; "
&"$expand=TargetWorkItem($select=WorkItemType,WorkItemId,Title,State) "
&") "
,null, [Implementation="2.0",OmitValues = ODataOmitValues.Nulls,ODataVersion = 4])
in
Source
I need to run the second query once per each project in the first list, and append them in a single table, replacing {project} for each element. Is this possible in PowerQuery?
Thanks in advance;

Create a custom function where result will be your table
In your case something like:
query name is GetDataForProject
(project) =>
let
Source = OData.Feed ("https://analytics.dev.azure.com/someorg/{" & project & "}/_odata/v3.0-preview/WorkItems?"
&"$filter=WorkItemType eq 'User Story' "
&"and State ne 'Closed' "
&"and startswith(Area/AreaPath,'{areapath}') "
&"&$select=WorkItemId,Title,WorkItemType,State,Priority,Severity,TagNames,AreaSK "
&"&$expand=AssignedTo($select=UserName),Iteration($select=IterationPath),Area($select=AreaPath), "
&"Links( "
&"$filter=LinkTypeName eq 'Related' "
&"and TargetWorkItem/WorkItemType eq 'User Story'; "
&"$select=LinkTypeName; "
&"$expand=TargetWorkItem($select=WorkItemType,WorkItemId,Title,State) "
&") "
,null, [Implementation="2.0",OmitValues = ODataOmitValues.Nulls,ODataVersion = 4])
in
Source
Convert your list into table and add new column where you use your created function, something like:
let
Source = Table.FromList(YourListOfProjects)
,InvokedFunction = Table.AddColumn(Source, "GetDataForProject", each GetDataForProject([Column1]))
in
InvokedFunction
You can then expand tables in new created column:
In result you get appended tables.

Related

Why is the Azure DevOps PowerBI burndown a repeating topic and so hard to create?

Given this odata string (right from MS docs), and I don't want a single team. I want a slicer for all teams.
"let Source = OData.Feed ("https://analytics.dev.azure.com/{organization}/{project}/_odata/v3.0-preview/WorkItemSnapshot? " &"$apply=filter( " &"WorkItemType eq 'User Story' " &"and (Teams/any(x:x/TeamName eq '{teamname}) or Teams/any(x:x/TeamName eq '{teamname}) or Teams/any(x:x/TeamName eq '{teamname}) " &"and StateCategory ne 'Completed' " &"and DateValue ge Iteration/StartDate " &"and DateValue le Iteration/EndDate " &"and Iteration/StartDate le now() " &"and Iteration/EndDate ge now() " &") " &"/groupby( " &"(DateValue,State,WorkItemType,Priority,Area/AreaPath,Iteration/IterationPath), " &"aggregate($count as Count, StoryPoints with sum as TotalStoryPoints) " &") " ,null, [Implementation="2.0",OmitValues = ODataOmitValues.Nulls,ODataVersion = 4]) in Source"
Why despite scouring the internet and this board for a month, I cannot replicate the simplest of Agile charts, the BurnDown filtered by a slicer for all TeamName(s) (see odata below), not Team, not TeamSK... using this odata string? I can get the total for all Teams easy.
= OData.Feed("https://analytics.dev.azure.com/(insert org)/(insert project)/_odata/v4.0-preview/Teams?&$select=TeamName,TeamSK,ProjectSK", null, [Implementation="2.0"])
'''
Basically replicate the BurnDown widget in ADO.

Create Sprint Burndown chart cross-org from Azure DevOps in Power BI

I have the query below to construct Sprint Burndown cross-org in Azure DevOps. It works fine and smooth.
OData.Feed ("https://analytics.dev.azure.com/" & organization & "/" & projectName & "/_odata/v3.0-preview/WorkItemSnapshot?"
&"$apply=filter(WorkItemType eq 'User Story' and StateCategory ne 'Completed' and DateValue ge Iteration/StartDate and DateValue le Iteration/EndDate and Iteration/StartDate ge 2020-01-01Z)"
&"/groupby( "
&"(DateValue,State,WorkItemType,Area/AreaPath,Iteration/IterationPath,Iteration/StartDate,Iteration/EndDate), "
&"aggregate($count as Count, StoryPoints with sum as TotalStoryPoints) "
&") ",
[#"Authorization" = "Basic " & apiKey]
, [Implementation="2.0",OmitValues = ODataOmitValues.Nulls,ODataVersion = 4])
However, I wanted to query the Teams under each of the project as well but it is not as straightforward as adding Teams/TeamName in the groupby section compared to Iteration and Area.
Below is my attempt that fails:
OData.Feed ("https://analytics.dev.azure.com/" & organization & "/" & projectName & "/_odata/v3.0-preview/WorkItemSnapshot?"
&"$apply=filter(WorkItemType eq 'User Story' and StateCategory ne 'Completed' and DateValue ge Iteration/StartDate and DateValue le Iteration/EndDate and Iteration/StartDate ge 2020-01-01Z)"
&"/groupby( "
&"(DateValue,State,WorkItemType,Teams/TeamName,Area/AreaPath,Iteration/IterationPath,Iteration/StartDate,Iteration/EndDate), "
&"aggregate($count as Count, StoryPoints with sum as TotalStoryPoints) "
&") ",
[#"Authorization" = "Basic " & apiKey]
, [Implementation="2.0",OmitValues = ODataOmitValues.Nulls,ODataVersion = 4])
I found out the more effective way is to use another OData endpoint WorkItemBoardSnapshot where I can do server side groupby which is far more efficient than doing it in Power BI.
Here is some reference:
OData.Feed ("https://analytics.dev.azure.com/" & organization & "/" & projectName &"/_odata/v3.0-preview/WorkItemBoardSnapshot?"
&"$apply=filter(WorkItemType eq 'User Story' and StateCategory ne 'Completed' and DateValue ge Iteration/StartDate and DateValue le Iteration/EndDate and Iteration/StartDate ge 2020-01-01Z)"
&"/groupby("
&"(DateValue,WorkItemType,Team/TeamName,Iteration/IterationPath,Iteration/StartDate,Iteration/EndDate),"
&"aggregate($count as Count, StoryPoints with sum as TotalStoryPoints)"
&")",
[#"Authorization" = "Basic " & apiKey]
, [Implementation="2.0",OmitValues = ODataOmitValues.Nulls,ODataVersion = 4])
The diff is changing WorkItemSnapshot into WorkItemBoardSnapshot and notice that board snapshot is using Team instead of Teams.

Access list of object array with alias in jpa

i have written a native query in jpa which fetch list of object array
list = entityManager.createNativeQuery("select emp.card Card,emp.code Code,emp.emp_name EmpName,ln.BAL Balance, ln.cum_intr CumIntr, ln.install Install,ln.install_no InstallNo, "
+ " led.BAL ledgBAL,led.cum_intr ledgCum,led.install ledgInstall,led.install_no ledgInstallNo "
+ " from LOANS ln "
+ " inner join "
+ " LOAN_LEDGER led "
+ " on (led.EMPLOYEES_ID = ln.EMPLOYEES_ID and ln.LOAN_UID= led.LOAN_UID and (ln.bal!= led.BAL or ln.cum_intr!= led.cum_intr or ln.INSTALL != led.INSTALL )) "
+ " inner join EMPLOYEES emp "
+ " on ln.employees_id = emp.id "
+ " where led.ledger_month in "
+ " ( 201312 )").getResultList();
I assigned alias for each field.this is my code to access this object array.
Iterator it = list.iterator();
while (it.hasNext()) {
Object[] result = (Object[]) it.next();
System.err.println("Result Arrayyyyy "+result[0]);
}
Now i want to access this result array using alias . How can I do it?

JDBC select query with join and like fails

I am trying to search the data in two tables by getting the last four numbers of a SSN as input from the user.
I have a JDBC query that is as follows:
`String sql = "SELECT N.NUMBER,N.LAST_NAME,N.FIRST_NAME,M.SSN,M.SEX,M.BIRTH_DATE"
+"FROM" +" G.M_NAME_A N, G.M_ID_A M"
+"WHERE" + "N.NUMBER = M.NUMBER"
+"AND" + "M.SSN like 'l4ssn' ";'
It fails with either "FROM Keyword not found where expected" or "Invalid Column index".
Please help me format the query.
yes, it should fail. Check your SQL, you're missing a space in front of "FROM", and ditto most anywhere you're concatenating strings.
1、Change the SQL as follows. Space is added around FROM, WHERE and AND,If you follow what you write sql, sql stitching is so out:
String sql = "SELECT N.NUMBER,N.LAST_NAME,N.FIRST_NAME,M.SSN,M.SEX,M.BIRTH_DATEFROM G.M_NAME_A N, G.M_ID_A MWHEREN.NUMBER = M.NUMBERAND M.SSN like 'l4ssn' ";
String sql = "SELECT N.NUMBER,N.LAST_NAME,N.FIRST_NAME,M.SSN,M.SEX,M.BIRTH_DATE"
+" FROM G.M_NAME_A N, G.M_ID_A M"
+" WHERE N.NUMBER = M.NUMBER"
-- modify +" AND" + "M.SSN like 'l4ssn' ";'
+" AND M.SSN like '%l4ssn%' ";
Change the SQL as follows. Space is added around FROM, WHERE and AND
String sql = "SELECT N.NUMBER,N.LAST_NAME,N.FIRST_NAME,M.SSN,M.SEX,M.BIRTH_DATE"
+" FROM " +" G.M_NAME_A N, G.M_ID_A M"
+" WHERE " + "N.NUMBER = M.NUMBER"
+" AND " + "M.SSN like 'l4ssn' ";
Your query will look for M.SSN which is 14ssn it's just like saying M.SSN = '14ssn'. If you want to use query something similar to 14ssn use M.SSN like '%14ssn%' instead
Just on side note, use prepared statement instead of using the query parameter in your sql string.
http://docs.oracle.com/javase/tutorial/jdbc/basics/prepared.html
Try this query
String sql=" SELECT N.NUMBER,N.LAST_NAME,N.FIRST_NAME,M.SSN,M.SEX,M.BIRTH_DATE "+
" FROM G.M_NAME_A N,G.M_ID_A M "+
" WHERE N.NUMBER = M.NUMBER "+
" AND M.SSN like 'l4ssn' ";
The question is resolved using the following query and PreparedStatement:
String l4ssn=("%"+l4ssn);
String sql = "SELECT N.NUMBER,N.LAST_NAME,N.FIRST_NAME,M.SSN,M.SEX,M.BIRTH_DATE"
+" FROM " +" G.M_NAME_A N, G.M_ID_A M"
+" WHERE " + "N.NUMBER = M.NUMBER"
+" AND " + "M.SSN like ?";
PreparedStatement pstmt = new PreparedStatement(sql);
pstmt.setString(1,l4ssn);

ERROR: duplicate key value violates unique constraint "xak1fact_dim_relationship"

I am getting the below error while deleting some rows and updating the table based on a condition from java. My database is PostgreSQL 8.4. Below is the error:
ERROR: duplicate key value violates unique
constraint "xak1fact_dim_relationship"
The code cuasing this issue is below:
/**
* Commits job. This does the following:
* <OL>
* <LI> cancel previous datamart states </LI>
* <LI> drop diabled derived objects </LI>
* <LI> flip mirror relationships for objects with next states </LI>
* <LI> advance rolloff_state from start1 to complete1, from start2 to complete </LI>
* <LI> set 1/0 internal states to 0/-1 </LI>
* <LI> remove header objects with no letter rows </LI>
* <LI> mark mirror rel as OS if children are missing (e.g., semantic w/o agg build) </LI>
* <LI> mark mirror rel as OS if int-map not in sync with dim base (e.g., int-map SQL w/o semantic) </LI>
* </OL>
*/
protected void CommitJobUnprotected()
throws SQLException
{
if (_repl.epiCenterCon== null)
throw makeReplacementError(0);
boolean oldAutoCommitStatus = _repl.epiCenterCon.getAutoCommit();
try
{
_repl.epiCenterCon.setAutoCommit(false);
Statement stmt = null;
boolean committed = false;
synchronized (SqlRepl.metaChangeLock)
{
try
{
stmt = _repl.epiCenterCon.createStatement();
// update internal states for fact_dim_relationship
metaExec(stmt, "DELETE from fact_dim_relationship WHERE internal_state = -1 AND " +
" EXISTS (SELECT 1 FROM fact_dim_relationship WHERE internal_state = 1)",
" SELECT 1 from fact_dim_relationship WHERE internal_state = -1 AND " +
" EXISTS (SELECT 1 FROM fact_dim_relationship WHERE internal_state = 1)"); /*1*/
metaExec( stmt, "UPDATE fact_dim_relationship SET internal_state = internal_state - 1 " +
" WHERE EXISTS (SELECT 1 FROM fact_dim_relationship inner1 " +
" WHERE inner1.internal_state = 1 " +
" AND inner1.fact_tbl_key = fact_dim_relationship.fact_tbl_key " +
" AND inner1.dim_base_key = fact_dim_relationship.dim_base_key ) ",
" SELECT 1 FROM fact_dim_relationship " +
" WHERE EXISTS (SELECT 1 FROM fact_dim_relationship inner1 " +
" WHERE inner1.internal_state = 1 " +
" AND inner1.fact_tbl_key = fact_dim_relationship.fact_tbl_key " +
" AND inner1.dim_base_key = fact_dim_relationship.dim_base_key ) "); /*5*/
System.out.println("Update done on fact_dim_relationship");
_repl.doDrop(SqlReplLogger.DB_META, stmt, "fact_agg", "SELECT fact_agg_key FROM fact_agg f WHERE " +
" NOT EXISTS (SELECT 1 FROM fact_agg_letter l WHERE " +
" f.fact_agg_key = l.fact_agg_key) "); /*6*/
_repl.doDrop(SqlReplLogger.DB_META, stmt, "dim_base_agg", "SELECT dim_base_agg_key FROM dim_base_agg d WHERE " +
" NOT EXISTS (SELECT 1 FROM dim_base_agg_letter l WHERE " +
" d.dim_base_agg_key = l.dim_base_agg_key) "); /*6*/
CheckOutOfSync(stmt, "fact_agg", null); /*7*/
CheckOutOfSync(stmt, "dim_base_agg", null); /*7*/
metaExec( stmt, " update mirror_relationship set relation_to_current = 'Out Of Sync' " +
" where dim_col_intmap_key is not null " +
" and relation_to_current = 'One Back' " +
" and not exists ( " +
" select 1 " +
" from mirror_relationship m2, dim_col_view c, dim_col_intmap i " +
" where m2.dim_base_key = c.dim_base_key " +
" and c.dim_col_key = i.dim_col_key " +
" and i.dim_col_intmap_key = mirror_relationship.dim_col_intmap_key " +
" and m2.relation_to_current = 'One Back') ",
" SELECT 1 FROM mirror_relationship " +
" where dim_col_intmap_key is not null " +
" and relation_to_current = 'One Back' " +
" and not exists ( " +
" select 1 " +
" from mirror_relationship m2, dim_col_view c, dim_col_intmap i " +
" where m2.dim_base_key = c.dim_base_key " +
" and c.dim_col_key = i.dim_col_key " +
" and i.dim_col_intmap_key = mirror_relationship.dim_col_intmap_key " +
" and m2.relation_to_current = 'One Back') "); /*8*/
// clean out the tables used by mombuilder, aggbuilder, and semantics
metaExec( stmt, "delete from relation_intermediary", "select 1 from relation_intermediary" );
_repl.epiCenterCon.commit();
committed = true;
}
finally
{
safeMetaRollbackIfNeeded( committed );
_repl.safeClose( null, stmt );
}
} // end synchronized block
}
finally
{
_repl.epiCenterCon.setAutoCommit(oldAutoCommitStatus);
}
}
The first delete statement ran well, but while running the update it is throwing the above exception....! We support the SQLServer, Oracle and DB2, and the same code runs fine with other DBs. By the way we run these statements in a READ_COMMITTED transaction level and we are setting the autocommit off if anything fails in between we safely rolls back. If i run the above code with autocommit true the code works fine! But we should not do so. I am suspecting the Multi version concurrency control feature of PostgreSQL, am i wrongly setting the Isolation level? Please help me as early as possible. I can provide what ever the info you want.
If it is only this particular set of queries, use SET CONSTRAINT:
BEGIN;
SET CONSTRAINT = xak1fact_dim_relationship DEFERRED;
-- Do your SQL
COMMIT;
If this is a very common case, you can change your database schema to you can, change your database schema to support INITIALLY DEFERRED.