We have a lot of users that use SharePoint and also we use only application token to manage the data. So we send lots of requests and we are facing throttling. So the questions are:
Are there any possibilities to increase the requests limit?
How can I see my current quota/resource usage?
You can not request a limit increase, and Microsoft does not provide any specific information regarding the algorithm they use to decide when or why to begin throttling.
The best option you have is to apply incremental back-off logic to account for throttled calls:
https://learn.microsoft.com/en-us/sharepoint/dev/solution-guidance/handle-sharepoint-online-throttling-by-using-exponential-back-off
You have to contact Microsoft support and open a ticket in order to get information regarding which service principals and users are being throttled. An engineer will have to pull the info from their database.
They can not give you specific quota/resource information. The will only tell you how many calls were throttled for a given principal.
Related
I am planning to create an application that will send out email to each member of a sharepoint group using JSOM and REST API but I am not sure what possible limit I can encounter doing it? I know that SharePoint will block you if you hit certain number of network transaction. I was wondering anybody know any limit I hit like for example calling rest api for a specific number of times in a certain time period will give you some kind of timeout.
Writing about it, I am thinking that doing it on Batch might be a better idea.
SPO rate limit has not been exposed. To avoid getting blocked, please take a reference of below materials:
https://learn.microsoft.com/en-us/sharepoint/dev/general-development/how-to-avoid-getting-throttled-or-blocked-in-sharepoint-online
https://learn.microsoft.com/en-us/sharepoint/dev/solution-guidance/handle-sharepoint-online-throttling-by-using-exponential-back-off
Background
Given a symbol (i.e. cashtag), I need to retrieve twits from 2015 up to 2019 in order to perform sentiment analysis for a research project.
Problem
The public API is limited to the last 30 messages, as explained in https://api.stocktwits.com/developers/docs/api#streams-symbols-docs.
https://api.stocktwits.com/api/2/streams/symbol/NVDA.json
I tried the same request in an authenticated manner, but I face the same limitation.
https://api.stocktwits.com/api/2/streams/symbol/NVDA.json?access_token=<access_token>
Question
Is there a way to work around the limitation ?
Yes, there is. You should contact their support team, to get Partner level access to download historical data.
"There are some endpoints and data that are not available in the public Stocktwits API, including the full firehose data stream of our content. You may also have an application that requires higher usage limits (rate, message or other limits) than the public API. These additional capabilities are highlighted in the API documentation. If your application requires access to these capabilities, please contact our team to discuss."
https://api.stocktwits.com/developers/contact
With Microsoft Graph, I can set a subscription on a resource. In my case an event. I am going to be using an admin authenticated account to access multiple calendars.
Is there a way to set a subscription to get notifications on all the calendars the admin can see?
If not, is there a way to send in a block of subscriptions with a single request? Because we are limited to how many requests we can specify in a specific timeframe. (I'm not sure what the limit is) but if I have 500 calendars I need to set subscriptions on so I get notifications of changes, how are you supposed to do this and not get hit by the request per timeframe limit?
Currently, there isn't a way to send multiple subscription creation requests in the same HTTP REST call. Every different resource for which a subscription is being created would have its own HTTP call into the Graph REST API.
You can recommend a "batching" feature (so multiple REST requests can be processed in the same HTTP call to the Graph API) on UserVoice: https://officespdev.uservoice.com/
There is also a consideration that, in my experience, the number of simultaneous subscriptions allowed is around 20, so 500 subscriptions might be out of the question. The best advice I've been given on the subject is to loop through all the objects one at a time to refresh them in sequence. The throttling that follows is a different issue altogether.
When a 429/"Unknown Error" comes back (ie throttling), it comes with a retry-after header which should be observed. I might point out that throttling, for me, is still a huge issue.
Does anyone know if there is a way to increase the quota limit of 10 queries when batching calls to the core reporting API?
This question/answer mentions the limit of 10: How can I combine/speed up multiple API calls to improve performance?
If I try to add more than 10 queries to the batch only the first ten are processed, each one after that contains a 403 quota exceeded error.
Is there a pay option? Would love to speed up the process of reporting on GA data for a bunch of URLs. I looked in my Google Developer's Console under the Analytics API where there is an option to increase the per-user limit and a link to request additional quota but I don't need total quota to increase, only allowed batch requests.
Thanks!
Quota is the number of requests you are allowed to make to a Google API without requesting permission to access more. Most of the Google APIs have a free quota, a number of requests Google lets you make without asking for permission to make more request. There are project based quotas and user based quotas.
Unless it says other wise APIs Quotas are projects based not user based.
User quota example
Per-user limit 10 requests/second/user
Some Quotas are user based, a user is normally the person that has authenticated the request. Every request sent to google contains information about who is making the request in the form of the IP address where the request came from. If you have your code running on a server the IP address is the same all the time so Google sees it as the same user. You can get around his by adding a random Quotauser to your request this will identify the request based upon different users.
If you send to many requests to fast from the same user you will see the following error.
userRateLimitExceeded The request failed because a per-user rate limit
has been reached.
The best way to get around this is to use QuotaUser in all of your requests, and identify different users to Google. Or just send a random number every time should also work.
Answer: You can't apply for an extension of the flood protection user rate limit. But you can get around it by using QuotaUser.
more info on quotas can be found on Google developers console APIs
I am trying to fetch the reportstats from our account. I need to make async calls because otherwise I would get and error that the data is to old.
When I create multiple requests I will get the error: "There have been too many calls from this ad-account. Wait a bit and try again."
I have only made about 30 request in a small time because of the way the async reports work. Is there a better way to fetch te reporting data? And if there is not is there a way to see the request score that is mentioned in the documentation?
And an other question will be, is there a difference in the amount of request when your app is on development access?
Thanks in advance,
Jorik
First point, according to access level docs here there is heavy rate limiting on the apps that are in development stage.
Second, To fetch reports there are multiple endpoints that, such as ad account wise reports, campaign wise reports, ad wise reports, here is a link to the docs for Insights API
available params are :
act_AD_ACCOUNT_ID/insights
CAMPAIGN_ID/insights
ADSET_ID/insights
AD_ID/insights
Lastly, about rate limiting in marketing api. It is done as a sliding window method which means there is no actual track of number of requests per day or something, its just that a lot of requests in short amount of time is not allowed.
two things you can do are,
first see the response of api and if the response is ratelimit error, stop the request.
second, use batch requests
Here is a gist from troubleshooting guide on limits
Troubleshooting
Timeouts
The most common issues causing failure at this endpoint are too many requests and time outs:
On /GET or synchronous requests, you can get out-of-memory or timeout errors.
On /POST or asynchronous requests, you can possibly get timeout errors. For asynchronous requests, it can take up to an hour to complete a request including retry attempts. For example if you make a query that tries to fetch large volume of data for many ad level objects.
Recommendations
There is no explicit limit for when a query will fail. When it times out, try to break down the query into smaller queries by putting in filters like date range.
Unique metrics are time consuming to compute. Try to query unique metrics in a separate call to improve performance of non-unique metrics.
Rate Limiting
The Facebook Insights API utilizes rate limiting to ensure an optimal reporting experience for all of our partners. For more information and suggestions, see our Insights API Limits & Best Practices.