Citrus integration tests fail randomly - citrus-framework

Question

Following up on How to access variables in a (citrus) static-response-adapter
Test layout:
Citrus -> SUT
SUT <-> MockServerOne
SUT <-> MockServerTwo
Citrus <- SUT
Applying the suggestions from the post above, I now see randomly occuring exceptions.
The test code:
#Test
#CitrusTest
public void testRequestNok() throws Exception {
setDefaultTestVariables();
http(builder -> builder
.client("fooClient")
.send()
.post("/foo/bar")
.payload(new ClassPathResource("foo/foo-sample-request.xml"))
.messageType(MessageType.XML)
.contentType("application/xml")
.accept("application/xml")
.fork(true));
applyBehavior(new MockOneServerBehavior());
applyBehavior(new MockTwoServerBehavior("bar/bar-error-response.xml"));
http(builder -> builder.client("fooClient").receive().response(HttpStatus.BAD_GATEWAY));
}
where the behaviors look like this
public class MockOneServerBehavior extends AbstractTestBehavior {
public void apply() {
http(builder -> builder.server("mockServerOne")
.receive()
.post());
http(builder -> builder.server("mockServerOne")
.send()
.response()
.payload(new ClasspathResource("foo/bar.xml"));
}
}
The test fails (roughly in 5% of all cases) and so far I have seen two different kinds of exceptions:
A ConcurrentModificationException:
2017-03-02 11:00:54,836 main [DEBUG ] citrus.channel.ChannelConsumer - Receiving message from: mockServerOne.inbound
2017-03-02 11:00:54,831 qtp1185618138-20 [DEBUG ] http.servlet.CitrusDispatcherServlet - Could not complete request
org.springframework.messaging.MessageDeliveryException: failed to send Message to channel 'mockServerOne.inbound'; nested exception is java.util.ConcurrentModificationException
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:449) ~[spring-integration-core-4.3.7.RELEASE.jar:4.3.7.RELEASE]
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:373) ~[spring-integration-core-4.3.7.RELEASE.jar:4.3.7.RELEASE]
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115) ~[spring-messaging-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.messaging.core.GenericMessagingTemplate.doSendAndReceive(GenericMessagingTemplate.java:150) ~[spring-messaging-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.messaging.core.GenericMessagingTemplate.doSendAndReceive(GenericMessagingTemplate.java:45) ~[spring-messaging-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.messaging.core.AbstractMessagingTemplate.sendAndReceive(AbstractMessagingTemplate.java:42) ~[spring-messaging-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.integration.core.MessagingTemplate.sendAndReceive(MessagingTemplate.java:97) ~[spring-integration-core-4.3.7.RELEASE.jar:4.3.7.RELEASE]
at com.consol.citrus.channel.ChannelSyncProducer.send(ChannelSyncProducer.java:76) ~[citrus-core-2.7.jar:?]
at com.consol.citrus.channel.ChannelEndpointAdapter.handleMessageInternal(ChannelEndpointAdapter.java:68) ~[citrus-core-2.7.jar:?]
at com.consol.citrus.endpoint.AbstractEndpointAdapter.handleMessage(AbstractEndpointAdapter.java:55) ~[citrus-core-2.7.jar:?]
at com.consol.citrus.http.controller.HttpMessageController.handleRequestInternal(HttpMessageController.java:137) ~[citrus-http-2.7.jar:?]
at com.consol.citrus.http.controller.HttpMessageController.handlePostRequest(HttpMessageController.java:63) ~[citrus-http-2.7.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_112]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[spring-web-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) ~[spring-web-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:116) ~[spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) ~[spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) ~[spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963) ~[spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897) ~[spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) [spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872) [spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) [javax.servlet-api-3.1.0.jar:3.1.0]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) [spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [javax.servlet-api-3.1.0.jar:3.1.0]
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841) [jetty-servlet-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634) [jetty-servlet-9.4.1.v20170120.jar:9.4.1.v20170120]
at com.consol.citrus.http.servlet.GzipServletFilter.doFilterInternal(GzipServletFilter.java:43) [citrus-http-2.7.jar:?]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1621) [jetty-servlet-9.4.1.v20170120.jar:9.4.1.v20170120]
at com.consol.citrus.http.servlet.RequestCachingServletFilter.doFilterInternal(RequestCachingServletFilter.java:41) [citrus-http-2.7.jar:?]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1621) [jetty-servlet-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:541) [jetty-servlet-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188) [jetty-server-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1228) [jetty-server-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168) [jetty-server-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:481) [jetty-servlet-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166) [jetty-server-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1130) [jetty-server-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [jetty-server-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213) [jetty-server-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:118) [jetty-server-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) [jetty-server-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.server.Server.handle(Server.java:564) [jetty-server-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) [jetty-server-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) [jetty-server-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279) [jetty-io-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:112) [jetty-io-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124) [jetty-io-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:672) [jetty-util-9.4.1.v20170120.jar:9.4.1.v20170120]
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:590) [jetty-util-9.4.1.v20170120.jar:9.4.1.v20170120]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Caused by: java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437) ~[?:1.8.0_112]
at java.util.HashMap$EntryIterator.next(HashMap.java:1471) ~[?:1.8.0_112]
at java.util.HashMap$EntryIterator.next(HashMap.java:1469) ~[?:1.8.0_112]
at java.util.AbstractMap.toString(AbstractMap.java:554) ~[?:1.8.0_112]
at java.util.Formatter$FormatSpecifier.printString(Formatter.java:2886) ~[?:1.8.0_112]
at java.util.Formatter$FormatSpecifier.print(Formatter.java:2763) ~[?:1.8.0_112]
at java.util.Formatter.format(Formatter.java:2520) ~[?:1.8.0_112]
at java.util.Formatter.format(Formatter.java:2455) ~[?:1.8.0_112]
at java.lang.String.format(String.java:2940) ~[?:1.8.0_112]
at com.consol.citrus.message.DefaultMessage.toString(DefaultMessage.java:108) ~[citrus-core-2.7.jar:?]
at java.lang.String.valueOf(String.java:2994) ~[?:1.8.0_112]
at java.lang.StringBuilder.append(StringBuilder.java:131) ~[?:1.8.0_112]
at org.springframework.messaging.support.GenericMessage.toString(GenericMessage.java:111) ~[spring-messaging-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at java.lang.String.valueOf(String.java:2994) ~[?:1.8.0_112]
at java.lang.StringBuilder.append(StringBuilder.java:131) ~[?:1.8.0_112]
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:430) ~[spring-integration-core-4.3.7.RELEASE.jar:4.3.7.RELEASE]
... 56 more
and a problem with TypeConversion, which leads to a XMLParseException:
2017-03-02 11:11:59,543 main [DEBUG ] citrus.validation.MessageValidatorRegistry - Found 3 message validators for message type: XML
2017-03-02 11:11:59,543 main [DEBUG ] validation.xml.DomXmlMessageValidator - Start message validation ...
2017-03-02 11:11:59,543 main [DEBUG ] validation.xml.DomXmlMessageValidator - Start XML message validation
2017-03-02 11:11:59,544 main [WARN ] citrus.util.TypeConversionUtils - Using object toString representation: Failed to convert value of type 'java.io.ByteArrayInputStream' to required type 'java.lang.String'; nested exception is java.lang.IllegalStateException: Cannot convert value of type 'java.io.ByteArrayInputStream' to required type 'java.lang.String': no matching editors or conversion strategy found
2017-03-02 11:11:59,544 main [WARN ] citrus.util.TypeConversionUtils - Using object toString representation: Failed to convert value of type 'java.io.ByteArrayInputStream' to required type 'java.lang.String'; nested exception is java.lang.IllegalStateException: Cannot convert value of type 'java.io.ByteArrayInputStream' to required type 'java.lang.String': no matching editors or conversion strategy found
[Fatal Error] :1:1: Content is not allowed in prolog.
2017-03-02 11:11:59,553 main [INFO ] consol.citrus.Citrus -
2017-03-02 11:11:59,553 main [ERROR ] consol.citrus.Citrus - TEST FAILED FooTest.testBar <foo.bar.baz> Nested exception is:
com.consol.citrus.exceptions.CitrusRuntimeException: org.w3c.dom.ls.LSException: Content is not allowed in prolog.
at com.consol.citrus.validation.xml.DomXmlMessageValidator.validateMessagePayload(DomXmlMessageValidator.java:116) ~[citrus-core-2.7.jar:?]
at com.consol.citrus.validation.xml.DomXmlMessageValidator.validateMessagePayload(DomXmlMessageValidator.java:68) ~[citrus-core-2.7.jar:?]
at com.consol.citrus.validation.AbstractMessageValidator.validateMessage(AbstractMessageValidator.java:65) ~[citrus-core-2.7.jar:?]
at com.consol.citrus.validation.AbstractMessageValidator.validateMessage(AbstractMessageValidator.java:50) ~[citrus-core-2.7.jar:?]
at com.consol.citrus.actions.ReceiveMessageAction.validateMessage(ReceiveMessageAction.java:216) ~[citrus-core-2.7.jar:?]
at com.consol.citrus.actions.ReceiveMessageAction.doExecute(ReceiveMessageAction.java:133) ~[citrus-core-2.7.jar:?]
at com.consol.citrus.actions.AbstractTestAction.execute(AbstractTestAction.java:42) ~[citrus-core-2.7.jar:?]
at com.consol.citrus.dsl.actions.DelegatingTestAction.doExecute(DelegatingTestAction.java:54) ~[citrus-java-dsl-2.7.jar:?]
at com.consol.citrus.actions.AbstractTestAction.execute(AbstractTestAction.java:42) ~[citrus-core-2.7.jar:?]
at com.consol.citrus.TestCase.executeAction(TestCase.java:229) [citrus-core-2.7.jar:?]
at com.consol.citrus.dsl.runner.DefaultTestRunner.run(DefaultTestRunner.java:198) ~[citrus-java-dsl-2.7.jar:?]
at com.consol.citrus.dsl.runner.DefaultTestRunner.http(DefaultTestRunner.java:526) [citrus-java-dsl-2.7.jar:?]
at com.consol.citrus.dsl.testng.TestNGCitrusTestRunner.http(TestNGCitrusTestRunner.java:360) ~[citrus-java-dsl-2.7.jar:?]
at foo.bar.baz.FooTest.testFoo(FooTest.java:92) ~[integrationTest/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_112]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:216) ~[spring-core-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at com.consol.citrus.dsl.testng.TestNGCitrusTest.invokeTestMethod(TestNGCitrusTest.java:139) ~[citrus-java-dsl-2.7.jar:?]
at com.consol.citrus.dsl.testng.TestNGCitrusTest.run(TestNGCitrusTest.java:109) [citrus-java-dsl-2.7.jar:?]
at com.consol.citrus.dsl.testng.TestNGCitrusTest.run(TestNGCitrusTest.java:58) [citrus-java-dsl-2.7.jar:?]
at org.testng.internal.MethodInvocationHelper.invokeHookable(MethodInvocationHelper.java:209) [testng-6.9.10.jar:?]
at org.testng.internal.Invoker.invokeMethod(Invoker.java:639) [testng-6.9.10.jar:?]
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:820) [testng-6.9.10.jar:?]
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1128) [testng-6.9.10.jar:?]
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129) [testng-6.9.10.jar:?]
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112) [testng-6.9.10.jar:?]
at org.testng.TestRunner.privateRun(TestRunner.java:782) [testng-6.9.10.jar:?]
at org.testng.TestRunner.run(TestRunner.java:632) [testng-6.9.10.jar:?]
at org.testng.SuiteRunner.runTest(SuiteRunner.java:366) [testng-6.9.10.jar:?]
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:361) [testng-6.9.10.jar:?]
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:319) [testng-6.9.10.jar:?]
at org.testng.SuiteRunner.run(SuiteRunner.java:268) [testng-6.9.10.jar:?]
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) [testng-6.9.10.jar:?]
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86) [testng-6.9.10.jar:?]
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1244) [testng-6.9.10.jar:?]
at org.testng.TestNG.runSuitesLocally(TestNG.java:1169) [testng-6.9.10.jar:?]
at org.testng.TestNG.run(TestNG.java:1064) [testng-6.9.10.jar:?]
at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:72) [testng-plugin.jar:?]
at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:127) [testng-plugin.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_112]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147) [idea_rt.jar:?]
Caused by: org.w3c.dom.ls.LSException: Content is not allowed in prolog.
at org.apache.xerces.parsers.DOMParserImpl.parse(Unknown Source) ~[xercesImpl-2.11.0.jar:?]
at com.consol.citrus.util.XMLUtils.parseMessagePayload(XMLUtils.java:356) ~[citrus-core-2.7.jar:?]
at com.consol.citrus.validation.xml.DomXmlMessageValidator.validateXMLSchema(DomXmlMessageValidator.java:148) ~[citrus-core-2.7.jar:?]
at com.consol.citrus.validation.xml.DomXmlMessageValidator.validateMessagePayload(DomXmlMessageValidator.java:91) ~[citrus-core-2.7.jar:?]
... 45 more
Caused by: org.apache.xerces.xni.parser.XMLParseException: Content is not allowed in prolog.
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) ~[xercesImpl-2.11.0.jar:?]
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) ~[xercesImpl-2.11.0.jar:?]
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) ~[xercesImpl-2.11.0.jar:?]
at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source) ~[xercesImpl-2.11.0.jar:?]
at org.apache.xerces.impl.XMLDocumentScannerImpl$PrologDispatcher.dispatch(Unknown Source) ~[xercesImpl-2.11.0.jar:?]
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source) ~[xercesImpl-2.11.0.jar:?]
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) ~[xercesImpl-2.11.0.jar:?]
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) ~[xercesImpl-2.11.0.jar:?]
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) ~[xercesImpl-2.11.0.jar:?]
at org.apache.xerces.parsers.DOMParserImpl.parse(Unknown Source) ~[xercesImpl-2.11.0.jar:?]
at com.consol.citrus.util.XMLUtils.parseMessagePayload(XMLUtils.java:356) ~[citrus-core-2.7.jar:?]
at com.consol.citrus.validation.xml.DomXmlMessageValidator.validateXMLSchema(DomXmlMessageValidator.java:148) ~[citrus-core-2.7.jar:?]
at com.consol.citrus.validation.xml.DomXmlMessageValidator.validateMessagePayload(DomXmlMessageValidator.java:91) ~[citrus-core-2.7.jar:?]
... 45 more
In case of the XMLParseException, I have verified on HTTP level that the response is identical to when the test actually passes.
Both exceptions look like threading issues to me.
I would be grateful for any feedback or alternative approach I might try.

Answer 1

Indeed this has been an issue located in the default message implementation when converting its payload to a String for logging purpose. The issue has been fixed with the mentioned github commit:
https://github.com/christophd/citrus/commit/2c0e70c4db9af102c2544ad60fb9447afb8a7b9c
Mentioned behavior is fixed since Citrus 2.7.2.