VSTS Analytics request blocked due to exceeding usage of resource AnalyticsBlockingResource - azure-devops

I have a PowerBI mashup that performs 4 queries against different VSTS projects on our tenant via the VSTS Analytics module. I have setup each query as a specific analytics views. Each view returns < 200 records and are simple "Get Story Backlog items" for a single team for today only.
I am frequently getting a message like the following:
An error occurred in the ‘DS BI WorkItems’ query. Error: Request was
blocked due to exceeding usage of resource 'AnalyticsBlockingResource'
in namespace 'User'. For more information on why your request was
blocked, see the topic "Rate limits" on the Microsoft Web site
(https://go.microsoft.com/fwlink/?LinkId=823950). Details:
DataSourceKind=Visual Studio Team Services
ActivityId=a6ac93f3-549c-4eb0-b64e-2b38e18ae7ee
Url=https://vrmobility.analytics.visualstudio.com/_odata/v2.0-preview/WorkItems?$filter=((ProjectSK%20eq%208e25983d-a154-4b53-915f-1394b34e5338)%20and%20((ProjectSK%20eq%208e25983d-a154-4b53-915f-1394b34e5338%20and%20Teams/any(t:t/TeamSK%20eq%2019afa381-35ca-47db-9060-51baa5d0485e))))%20and%20Processes/any(b:(b/BacklogName%20eq%20'Stories')%20and%20((b/ProjectSK%20eq%208e25983d-a154-4b53-915f-1394b34e5338%20and%20(b/TeamSK%20eq%2019afa381-35ca-47db-9060-51baa5d0485e))))&$select=LeadTimeDays,CycleTimeDays,CompletedDate,StateCategory,ParentWorkItemId,ActivatedDate,Activity,VRAgile_ActualCompletionIteration,VRAgile_ActualUatIteration,BusinessValue,VRAgile_ChangeAreaOwnerTeam,ChangedDate,ClosedDate,CompletedWork,VRAgile_CompletionTargetConfidence,CreatedDate,FinishDate,FoundIn,WorkItemId,VRAgile_IncludedinVersion,IntegrationBuild,OriginalEstimate,VRAgile_PlannedCompletionIteration,VRAgile_PlannedUATIteration,Priority,Reason,VRAgile_ReleaseQuality,RemainingWork,vrmobility_VRAgile_RequestedBy,VRAgile_RequestedDept,R...
error=Record
I have checked the page and looked at the Usage page on our VSTS tenant but during these times my user is not indicated as blocked and VSTS user interface works normally.
The issue goes away after a few minutes but it will then return after a couple of changes made in PowerBI (like adding a new column, changing data type etc) because this automatically refreshes all 4 queries again and this seems to trigger this unacceptable usage.
It is really frustrating as I can't continue working on the report and have to go and do something else for 5 minutes really impacting my flow.
Any ideas on cause, solution/workarounds? It feels to me like an overly sensitive VSTS limit on the VSTS Analytic service

Azure DevOps Services limits the resources individuals can consume and the number of requests they can make to certain commands. When these limits are exceeded, subsequent requests may be either delayed or blocked.
See the below link...
https://learn.microsoft.com/en-us/azure/devops/integrate/concepts/rate-limits?view=azure-devops

Related

Can't access to TFS because of exceeding usage of resource 'Concurrency' in namespace 'VSID'

When I browse to our TFS via Source Control Explorer, an error occurs:
Request was blocked due to exceeding usage of resource 'Concurrency'
in namespace 'VSID'. For more information on why your request was
blocked, see the topic "Rate limits" on the Microsoft Web site
I think what you encountered is related to a sudden event on our side. Users visiting Azure DevOps will see a 429 throttled requests like Request was blocked due to exceeding usage of resource ... And as I know this issue has been resolved, please check it.
More details about this event you can check this link. The issue was fully mitigated from 7/4/2020. Our engineers will be investigating this further to learn from and reduce the risk of potential recurrences. We apologize for the impact this had on our customers.
I had the same error in the logs for one of our apps that day but now it is working again. Thanks.

Firestore: "Exceeded quota for veryifying passwords"?

Hi I got this error in one of my ETE tests which exercises login functionality and start up behavior for my angular app.
The appears to be triggered by logging in using
await this.angularFireAuth.auth.signInWithEmailAndPassword(uname, pw);
where angularFireAuth is an injected instance of AngularFireAuthfrom '#angular/fire/auth';
I checked the Firestore quotas here but I can't find a reference to a quota for verifying passwords. Can anybody point me to what the quota is?
The console error reported looks like this:
zone-evergreen.js:659 Unhandled Promise rejection: Exceeded quota for verifying passwords. ; Zone: ProxyZone ; Task: Promise.then ; Value: u
The problem resolves after a few minutes and then test runs as expected.
I have found the message you are receiving being handled in this github thread.
Here are some of the important comments from the thread:
For the error you are facing "Exceeded quota for verifying passwords", this usually happens when one sends requests for verifying passwords or password login requests too many times at once (more than 20 requests per second per IP address or 25 requests per 10 min per account). When we get a huge amount of requests in a short period of time, the limit is applied automatically to protect our servers.
This is an internal quota (regardless of pricing plans) enforced by Firebase Authentication to prevent abuse when making authentication requests, for this reason the quota can change without notice.
In order to avoid triggering this alert, you can use a different IP address or
back off the number of requests per minute to something like 10-20, to avoid triggering the automated abuse detection.
If you are sending too many requests in a short period of time from the same IP address, then there is an expectation that you will get throttled at some point. This may prevent you from getting successful integration tests but there is a security benefit that comes with that. The easier it is for you to test, the easier it is for malicious scripts to be written too against your project. We have similar integration tests in other firebase auth libraries (client and admin) and we try to work with the limit.
If you have a legitimate need to increase the limit, then you can file a bug with support and make a case for that. You could even file for a feature request to whitelist calls from certain IP addresses.

Detailed access logs in Azure CDN

Amazon and Google both provide detailed access logs for HTTP requests served by their CDN platforms. As far as I can see there is no such thing in any of Microsoft's CDN solutions (Microsoft, Akamai, Verizon Standard/Premium). There are diagnostic logs and reports for top resources, but I need the individual requests or at least the total number of requests per URL and day.
I have read Azure CDN file download statistics, but it is very old, so something may have changed.
Is it possible to get the access logs we need from a CDN in Azure using a method I have missed, or is this still a dead end?
It can be done now, at least if using "Azure CDN from Microsoft". You need to create a Log Analytics workspace (if you don't have one already), go to Diagnostics settings on your CDN profile (not endpoint) and route the raw logs to that workspace.
You can also put it in storage, or feed it into an event hub, but I found making reports easiest via log analytics - you can easily create a chart of hit/miss ratio per CDN Point-of-Presence with this:
AzureDiagnostics
| where Category == "AzureCdnAccessLog"
| summarize request_count = count(), totalResponseBytes = sum(toint(responseBytes_s)) by pop_s, cacheStatus_s, sentToOriginShield_b
| order by request_count desc
Read the whole of that second link - you may need to re-save endpoints not created recently.
Here's that query used to diagnose that misses from Cape Town (and Johannesburg) were being sent to an origin shield PoP in London before actually being pulled from the origin by the London PoP.

Azure Resource Group has no usage or billing

I've got a trial Azure subscription running with a couple of cloud services projects, a VM, a web app etc all up and running for a couple of weeks now.
Usage is light but resources are being consumed. i.e. stuffs been done.
My problem is the breakdown of charges re: resource groups. Perhaps I misunderstand but I thought that since every resource is put in a group and only one group by default then whatever is billed for the period should essentially be the sum of the charges for each of the resource groups?
The docs say you can get a consolidated view of resource group costs like below:
But the reality is quite different. From this summary on the portal home screen:
I can drill in to find the $48.32 expenditure:
I can drill into that and get cost by resource:
But when I want to get the cost by resource groups the info is either missing or incomplete. For these resource groups:
only one, containing a VM, has any billing data but only for compute. No data charges or storage charges.
And all the rest just have no usage or billing.
So this is kinda useless to me since we still have to dig around to find out where the difference went.
The documentation and some of the build 2015 videos suggest you can get a comprehensive view of your costs by resource group.
"how much is the marketing dept costing us"
"You can clarify billing for your organization by viewing the rolled-up costs for the entire group."
But that doesn't seem to be the case at all. Out of $48.32 of expenditure (albeit comped) only $8.43 is accounted for in resource group costings.
So have I not set something up correctly or does the portal not in fact support this?
Can I get this info by cross referencing (costs by resource IDS) with (resource group member resources id) via Powershell?
How do i get the complete operating cost (compute, storage, data) for a given resource group?

What are some best practices for basic workflow using biztalk 2009?

I'm working on a Biztalk 2009 proof of concept project and would like to setup a basic approval workflow.
Request Vacation.
Step 1. Request Vacation.
Step 2. Route Request to Manager.
Step 3. Manager Approve / Denies Vacation.
Step 4. Update Vacation Calender / Notify Originator of Approval.
Outside of Biztalk I'll have to provide an interface for requesting vacation and approving vacation.
How do I get the list of items pending approval for my manager?
I'm confused about if the list of pending items should be maintained / queried from Biztalk, or if I need to build a separate set of tables to manage what is awaiting approval.
Any help or suggestions would be greatly appreciated!
At it's core, BizTalk is a messaging bus (okay, not really, but it's a useful abstraction). It doesn't actually store data on its own. In point of fact, I'm not sure based just on this scenario that BizTalk is your best bet. If you have SharePoint, using normal WorkFlows through SharePoint might work better for you.
That said, if the BizTalk decision has been made, I'd do the following:
Interface for requesting / approving
The Approval interface would hook into a data store (probably a SQL DB) to get a list of the "waiting approval" items.
Once the status of the item is set to "Approved" or "Denied" have SQL push data to BizTalk to kick off the rest of the process.
To your specific quesiton: yes, you'll need some place outside of BizTalk to store the requests, and the list of "pending" requests would be better generated from that store.