Spring cloud performance tunning - spring-cloud

Question

I'm doing a performance test against a Spring Cloud application. When number of concurrent users exceeds 150, it starts to give "Forwarding error"
{"timestamp":1458685370986,"status":500,"error":"Internal Server Error","exception":"com.netflix.zuul.exception.ZuulException","message":"Forwarding error"}
Which parameter I should adjust to get rid of the error?

Answer 1

You should post your logs for the error, without that we can only guess what the exact error is. As Forwarding error reported by ZuulExcetption is a generic error.
See this link for the RibbonRoutingFilter.forward() method which actually reports this error. I'm adding the code here for the backup.
private HttpResponse forward(RestClient restClient, String service, Verb verb, String uri, Boolean retryable,
MultiValueMap<String, String> headers, MultiValueMap<String, String> params,
InputStream requestEntity) throws Exception {
Map<String, Object> info = this.helper.debug(verb.verb(), uri, headers, params,
requestEntity);
RibbonCommand command = new RibbonCommand(service, restClient, verb, uri, retryable,
headers, params, requestEntity);
try {
HttpResponse response = command.execute();
this.helper.appendDebug(info, response.getStatus(),
revertHeaders(response.getHeaders()));
return response;
}
catch (HystrixRuntimeException ex) {
info.put("status", "500");
if (ex.getFallbackException() != null
&& ex.getFallbackException().getCause() != null
&& ex.getFallbackException().getCause() instanceof ClientException) {
ClientException cause = (ClientException) ex.getFallbackException()
.getCause();
throw new ZuulException(cause, "Forwarding error", 500, cause
.getErrorType().toString());
}
throw new ZuulException(ex, "Forwarding error", 500, ex.getFailureType()
.toString());
}
}
As you can see that only viable place where the error can be generated is in command.execute(), where command is an instance of HystrixCommand. Here is a link for the execute() method in HystrixCommand.
Below is the code for backup.
public R execute() {
try {
return queue().get();
} catch (Exception e) {
throw decomposeException(e);
}
}
Here the queue() is a Future instance
Most common error that can occur with the Future is a timeout exception. Since here Future instance queue() is not bound by any timetout value, it can go on waiting for ever.
However most of the time API which make use of Future have a thread monitoring the time they take and they interrupt it after a certain period of time. Same is done by Ribbon.
If yours indeed is a timeout issue then an easy solution is to increase Ribbon timeout value by using following property.
ribbon.ReadTimeout=10000
//or
<client-name>.ribbon.ReadTimeout=10000
Time out majorly can occur if the tomcat server which hosts the service which is proxied by the Zuul has too much load. It's whole thread pool is exhausted thus resulting in the next requests having to wait for long time.
This can probably be alleviated by change the number of threads that your service tomcat has by using following property.
server.tomcat.max-threads=0
By default it's set to 0, which leaves it to the embedded server's default. In tomcat's case it's 200. See the reference maxThreads property in tomcat.
Note: To increase the thread pool size we have to make sure that the machine has that capacity to provide resources if that many threads were to be in execution simultaneously.