I have been using a self-hosted github runner for a few months now. It was working well for all along until at one point it stopped connecting to github.
I tried to run the connectivity check using ./config.sh --check and I got the following message in the generated log file.
Http response body: {"message":"API rate limit exceeded for x.x.x.x. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)","documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}
It seems the runner has exceeded the API rate limit. I am not sure know how that happened because it supposed to be 60 API requests per hour and the runner was basically idling for some days before this happened.
The documentation URL mentioned in the error message above does contain information on how to increase the rate limit but it does not mention anything about how to do that when using self-hosted runners. Can someone point me in the right direction?
Related
I am encountering a API rate limit exceeded error on Jenkins. I am unsure why I would be exceeding that many requests and how I can fix it. I have included a screenshot of the exact error.
https://i.stack.imgur.com/PLpbM.png
We have spent several weeks trying to fix an issue that occurs in the customer's production environment and does not occur in our test environment.
After several analyses, we have found that this error occurs only when one condition is met: processing times greater than 3600 seconds in the API.
The situation is the following:
SAP is connected to a server with Windows Server 2016 and IIS 10.0 where we have an API that is responsible for interacting with a DB use by an external system.
The process that we execute sends data from SAP to the API and this, with the data it receives from SAP and the data it obtains from the DB of the external system, performs a processing and a subsequent update in the DB.
This process finishes without problems when the processing time in the API is less than 3600 seconds.
On the other hand, when the processing time is greater than 3600 seconds, the API generates the response correctly, and the server tries to return the response to SAP, but it is not possible.
Below I show an example of a server log entry when it tries to return a response after more than 3600 seconds of API processing. As you can see, a 995 error occurs: (I have censored some parts)
Any idea where the error could come from?
We have compared IIS configurations in Production and Test. We have also reviewed the parameters of the SAP system in Production and Test and we have not found anything either.
I remain at your disposal to provide any type of additional information that may be useful for solving the problem.
UPDATE 1 - 02/09/2022
After enabling FRT (Failed Request Tracing) on IIS for 200 response codes, looking at the event log of the request that is causing the error, we have seen this event at the end:
Any information about what could be causing this error? ErrorCode="The I/O operation has been aborted because of either a thread exit or an application request. (0x800703e3)"
UPDATE 2 - 02/09/2022
Comparing configurations from customer's environment and our test environment:
There is a Firewall between SAP Server and IIS Server with the default idle timeout configured for TCP (3600 seconds). This is not happening in Test Environment because there is no Firewall.
Establishing a Firewall policy specifying a custom idle timeout for this service (7200 seconds) the problem will be solved.
sc-win32 status 995, the I/O operation has been aborted because of
either a thread exit or an application request.
Please check the setting of minBytesPerSecond configuration parameter in IIS. The default "minBytesPerSecond" is 240.
Specifies the minimum throughput rate, in bytes, that HTTP.sys
enforces when it sends a response to the client. The minBytesPerSecond
attribute prevents malicious or malfunctioning software clients from
using resources by holding a connection open with minimal data. If the
throughput rate is lower than the minBytesPerSecond setting, the
connection is terminated.
In our dev environment we have lots of repos, lots of builds and lots of buildservers, and most of the time things work just like they should - however, we are seeing an increase in builds that fail because of timeouts.
These timeouts are not happening because we are getting close to the limit, but because something "gets stuck/blocked" in the pipeline and it stays on that step until timeout kills the build.
To better debug why that happens, we need to be able to query what builds fails because of this timeout, so we for instance can see, if it is a particular build server or agent that has this problem.
We can not find anything in the API that would give us the timeout error, but we can see that the UI is able to deduct it somehow:
So far we have narrowed it down to query all builds with completed status (through this API), but we get no completion reason, and buildtimes are never exact the same as the timeout of the build defintion, so "guessing" it from the execution plan will also be a bit shaky.
How can we filter our builds down to only the builds that have timed out?
We can use the below API to get details for a build.
Note: do not add timelineId, we should list all info
GET https://dev.azure.com/{organization}/{project}/_apis/build/builds/{buildId}/timeline?api-version=6.1-preview.2
If the build is canceled because of the timeout setting, we can get the message: The job running on agent Hosted Agent ran longer than the maximum time of xxx minutes. For more information, see https://go.microsoft.com/fwlink/?linkid=2077134
By the way, we can use the API Builds - List to filter all failed build. if the build is canceled due to a timeout setting. the result is failed instead of cancel.
I am using Web IDE and want to see the log by clicking on the arrow.I can only see an empty "Untitled" page. The Node.js app is running normally. Live edit is switch off.
After some minutes:
Gateway Timeout
The proxy server did not receive a timely response from the upstream server.
Reference #1.45bf1402.1511018717.3dddb8b
I'm not for sure what Web IDE you are referring to. The only one I'm aware of is the DevOps (which works for me below):
It seems to me like this error that you posted would indicate a temporary outage. Is it still an issue?
In any case, I would advise opening a support ticket if you encounter this issue again (more details about your account would help). I think the Bluemix proxy will time out requests if they take too long.
I have a build for an Ionic project and its E2E testing with SauceLabs. The build is timing out after 49 min 17 sec(50 min). All of my jobs are running well and logging output frequently at least every 1-2 min. The timeout is happening consistently at 50 min.
My build goes meets all the requirements as mentioned here to not suffer a time out. Also, there is no timeout for the build as mentioned in the docs. So the build shouldn't timeout as it is happening in the case. Any resolutions for this Issue?
Here are some of the logs:
https://travis-ci.org/magician03/moodlemobile2/builds/241500777
https://travis-ci.org/magician03/moodlemobile2/builds/241414546
https://travis-ci.org/magician03/moodlemobile2/builds/241401570
Your build ends with this message:
The job exceeded the maximum time limit for jobs, and has been
terminated.
It is the expected behaviour. Exists a limit of 50 minutes as explained here and here:
Build Timeouts #
It is very common for test suites or build scripts to hang. Travis CI
has specific time limits for each job, and will stop the build and add
an error message to the build log in the following situations:
A job produces no log output for 10 minutes
A job on travis-ci.org takes longer than 50 minutes
A job running on OS X infrastructure takes longer than 50 minutes - (applies to travis-ci.org or travis-ci.com)
A job on Linux infrastructure on travis-ci.com takes longer than 120 minutes
Some common reasons why builds might hang:
Waiting for keyboard input or another kind of human interaction
Concurrency issues (deadlocks, livelocks and so on) Installation of
native extensions that take very long time to compile There is no
timeout for a build; a build will run as long as all the jobs do as
long as each job does not timeout.
Your build doesn't complete before for a specific issue in your build.
I would ask another question focused in your code and language node_jsand no in this limit.
I develop native apps so I can not help on this topic but I found this ticket:
It seems that they updated Node.js to 6.X, tested it using Travis-ci, it failed and currently they don't use Travis-ci, so I would ask directly to MoodleHQ in their forums.
jleyva Juan Leyva added a comment - 03/Nov/16 6:05 PM Dani, can you
enable in your Travis account your moodlemobile2 repository so we can
see if Travis is working with the new dependencies? I already changed
the tracker fields so Travis is aware of the branch (but it requires
first you to enable you forked moodlemobile2 repo)
jleyva Juan Leyva added a comment - 03/Nov/16 7:31 PM Builds are
failing: https://travis-ci.org/dpalou/moodlemobile2/builds/172896611
Protractor or Jasmine or whatever is not working with this dependency
set
You can also check related issues and compare, this configuration works using:
node_modules/.bin/protractor e2e-tests/protractor.conf.js --directConnect
in protractor-conf.js change chromeOnly to directConnect