Rundeck job returns "curl: (52) Empty reply from server" if the jobs takes 2 minutes or more.
Also part of the error "SSH command execution error: NonZeroResultCode: Remote command failed with exit status 52"
We have a rundeck job (curl cmd) that executes on a Linux 7 server that for the majority of the time returns a "success" for the completion of the job.
However sometimes it will return "failed" if the job runs 2 minutes or more.
A value for "5m" has been added to the job "Timeout" on the configuration.
The job actually is successful but rundeck still returns the "failed" status.
When comparing the successful against failed logs I notice some extra steps in the failed log which pushes it past the 2 minutes time.
Any ideas why the failed status as mentioned above?
Thanks,
Ken
It's a problem with your curl command against the web server, usually happens when you call using HTTP against an HTTPS based service. take a look at this. Another possibility is a network problem.
Related
i have a problem with my rundeck, in automatisation process i need to perform some long task with Rundeck such as Database Refresh from PRODUCTION to TEST Database.
This case cause problems sometimes.
I've no timeout set on rundeck job, i use script that launch action on a distant server trough SSH.
If a step took long time (Drop database, import database with datapump), Rundeck generate an error
Failed: NonZeroResultCode: Remote command failed with exit status -1
after a certain period of time.
It's look like a timeout, but i don't know where this timeout occured and i can't find it.
But the DROP or the IMPDB still continue his job and if you wait for it, the action has successfully completed but rundeck marked the step as failed, i put some workaround such as error handler to by pass but it's not very clean.
Do you have any idea were that timeout is set ?
In my opininon, after a certain period of time, if rundeck did not receive an answer from the distant server, it cut off and mark as failed with the error code Failed: NonZeroResultCode: Remote command failed with exit status -1
You can increase the ssh timeout globally at the framework.properties file.
Stop the Rundeck server, then add the following lines on the /etc/rundeck/framework.properties file:
framework.ssh-connection-timeout=0
framework.ssh-command-timeout=0
Then start the Rundeck service.
The timeout values are in milliseconds, 0 is indefinite.
More info here.
we've been using Laravel / Guzzle to handle all our jobs for over a year but over the past couple of weeks we've seen a complete failure down to 2 errors:
cURL error 6: Could not resolve host
cURL error 6: getaddrinfo() thread failed to start
Our jobs run on the hour and every job fails with the same error, once it starts it will continue to fail all jobs.
Restarting the server fix the issue for a few days but it's just come back.
Anything to check first? Pretty puzzled.
I have a CircleCI config in which I am running machine executor, starting up my application using docker-compose, then running E2E tests in Cypress, that are not in docker, against this app. Cypress basically spins up a headless browser and tries to hit the specified url.
Now, no matter what my first test always fails. I have created 10 tests that just hit the root url and click a button. I have those tests run first. First one always fails.
The error is
CypressError: cy.visit() failed trying to load:
http://localhost:3000/
Which basically means there was no response or a 4-500
I though the app might not be ready yet, so I added a sleep before starting tests. I set that to 9 minutes (10 minutes times out on CircleCI). First test failed. I ratcheted that down to 2 minutes. First test fails.
Again to be clear, the first 10 tests are the same, so it’s not test specific.
Update
I have crossposted this to the CircleCI forum.
I think your server isn’t ready.
Before run cypress test, you have to wait for server.
If so, don't use sleep.
You can use wait-on or start-server-and-test.
You can check this doc. https://docs.cypress.io/guides/guides/continuous-integration.html#Boot-your-server
So TJ Jiang was close with his answer. Unfortunately wait-on doesn't work with CircleCI.
So I wrote my own,
timeout 540 bash -c 'while [ $(curl -s -o /dev/null -w %{http_code} localhost:3000) != "200" ]; do sleep 15; done' || false
This pings the url every 15 seconds for 9 minutes till it receives a http status code of 200 or it exits.
To be specific I wrote a Makefile target that uses that line above, I don't know how just pasting that in config.yml would work.
So running this as a step prior to running the cypress tests does in fact make it so all 5 identical tests pass.
So now it pings for about 3 1/2 minutes then goes through. I had tested with sleep up to 9 minutes and always the same result. So I don't know why this works better than sleep but it clearly does.
I had the same issue. Solved it by server/route/wait. I first start the server, then create a rout alias, wait for it to call the Api endpoint (which triggers the app to fill the dom) then do the rest (cy.get/click).
Example before:
cy.get('myCssSelector')
After server/route/wait:
cy.server()
cy.route('GET', '*/project/*').as('getProject')
cy.wait('#getProject')
.then(resp => {
expect(resp.status).to.eq(201)
})
cy.get('myCssSelector')
I'm trying to create a Workload Scheduler job that executes a curl command.
In Steps I've selected Start a program for the step and RP_CLOUD as the Agent(it's the only option). I pasted my curl command into Program.
Now when I try to enable the job I get a popup saying <b>AWSUI4177E</b><br />AWSUI4177E Unable to update the Process.<br /><i>AWSUI4299E An internal error has occurred: AWSPRE001E The user "paul.carron#anaeko.com.5c81ed484ccf4c54aa9e348e" cannot create a job of type "executable" on the "RP_CLOUD" workstation. Download and install a Workload Automation Agent on a different machine.</i>.
The curl statement works when executd in my Terminal. What am I doing wrong?
There are some security constrains on running jobs on the agents provided by the infrastructure.
I see two options:
Use the restful job type (since you are invoking a curl command)
Install an agent
When halting service in Process Admin console - using Halt service button - I get following message after a while:
Halting of the Service timed out. Most probably, the Process Server failed to halt the Service within short time period.
From what I can see, it happens only when said service has few (say in tenths)steps, when it has more (hundreds) steps, it works well.
Can somebody see cause of this and tell me what to do? Thanks.
If we fail to halt a hung or long running (infinite loop) service within a short duration, process admin console does not allow to halt it.
This is the default behavior of how halt will work. Only way to restart the server to reinstate the BPM env.