Attachment missing in MTOM response from Citrus SOAP server simulation - soap

I have built a sample Citrus testcase to simulate a SOAP server that responds with an MTOM attachment.
runner.soap(action -> action.server("simulationServer")
.receive()
...[validation etc]
);
runner.soap(action -> action.server("simulationServer")
.send()
.name("get-response")
.mtomEnabled(Boolean.TRUE)
.attachment("myAttachment", "application/octet-stream", new ClassPathResource("testfiles/myAttachment.pdf"))
.payload("<getResponse xmlns:xmime=\"http://www.w3.org/2005/05/xmlmime\">\n" +
" <document>\n" +
" <contentElements>\n" +
" <contentElement xmime:contentType=\"application/pdf\">cid:myAttachment</contentElement>\n" +
" </contentElements>\n" +
" <id>Test</id>\n" +
" </document>\n" +
"</getResponse>\n")
);
When I run this test and call the Citrus simulation with SoapUI, I see the contents of myAttachment.pdf in the debug logs. So at least it looks like Citrus tries to send the attachment.
However, in SoapUI I do not get an attachment. There is a XOP element in the SOAP response, but no attachment. The RAW view of SoapUI of the Citrus response looks like this.
HTTP/1.1 200 OK
Date: Tue, 16 Jan 2018 15:30:36 GMT
Accept: text/xml, text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
SOAPAction: ""
Content-Type: Multipart/Related; boundary="----=_Part_0_382348859.1516116636524"; type="application/xop+xml"; start-info="text/xml"
Transfer-Encoding: chunked
Server: Jetty(9.4.6.v20170531)
------=_Part_0_382348859.1516116636524
Content-Type: application/xop+xml; charset=utf-8; type="text/xml"
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Header/><SOAP-ENV:Body><getResponse xmlns:xmime="http://www.w3.org/2005/05/xmlmime">
<document>
<contentElements>
<contentElement xmime:contentType="application/pdf"><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:myAttachment"/></contentElement>
</contentElements>
<id>Test</id>
</document>
</getResponse></SOAP-ENV:Body></SOAP-ENV:Envelope>
------=_Part_0_382348859.1516116636524--
In an MTOM response with attachment the attachment starts where this RAW view ends. It should continue like this
------=_Part_0_382348859.1516116636524-- [last line from above]
Content-Type: application/pdf
Content-Transfer-Encoding: binary
Content-ID: <myAttachment>
%PDF-1.4... [PDF content]
I am using Citrus 2.7.2 release.
Update
Still no success on this. Wireshark shows the same picture as SoapUI: the attachment is missing in the response.
However, when I debug into the code on the server (Citrus) side, I see the attachment in the response message until I get lost somewhere in a MessageSendingTemplate. Same on the console log. The message has the attachment.
There is an inline MTOM variant in the Citrus documentation, but I can't find a way to set this mtom-inline on the attachment in Java config.
Any hints, where to set a breakpoint to find where the attachment get lost? Or any other suggestions/examples on the Citrus side?

The setMtomInline field sits on the SoapAttachment interface. I am not sure if I got the setup right - but seems to work for inlined attachements - fails for soap attachements / multipart. The SoapUI Mock does not show any attachements when receiving requests from following testcase.
SoapAttachment soapAttachment = new SoapAttachment();
soapAttachment.setMtomInline(false);
soapAttachment.setContentResourcePath("log4j.xml");
soapAttachment.setContentType("application/octet-stream");
soapAttachment.setContentId("FILE");
SoapMessage soapMessage = new SoapMessage();
soapMessage.mtomEnabled(true);
soapMessage.soapAction("/HelloService/sayHello");
soapMessage.setPayload(
"<ht:HelloRequest " +
"xmlns:ht=\"http://citrusframework.org/schemas/samples/HelloMtomService\" " +
"xmlns:xop=\"http://www.w3.org/2004/08/xop/include\" >\n" +
" <ht:Message>Hei .. citrus does stream mtom</ht:Message>\n" +
" <ht:Data><xop:Include href=\"cid:FILE\"/></ht:Data>\n" +
"</ht:HelloRequest>");
soapMessage.addAttachment(soapAttachment);
runner.soap(action -> {
action.client("helloMtomSoapuiClient")
.send()
.soapAction("/HelloService/sayHello")
.message(soapMessage);
});
If I do the same for MtomInline set to true, I see the attachement as base64 encoded content text in the ht:Data node.
SoapAttachment soapAttachment = new SoapAttachment();
soapAttachment.setContentResourcePath("log4j.xml");
soapAttachment.setMtomInline(true);
soapAttachment.setContentType("application/xml");
soapAttachment.setContentId("MyAttachement");
soapAttachment.setEncodingType("base64Binary");
runner.soap(action -> {
action.client("helloMtomSoapuiClient")
.send()
.soapAction("/HelloService/sayHello")
.mtomEnabled(true)
.payload("<ht:HelloRequest xmlns:ht=\"http://citrusframework.org/schemas/samples/HelloMtomService\">\n" +
" <ht:Message>Hei .. citrus does mtom</ht:Message>\n" +
" <ht:Data>cid:MyAttachement</ht:Data>\n" +
"</ht:HelloRequest>")
.attachment(soapAttachment);
});
Either soapUI or citrus swallows the attachement. Some help or working JavaDSL sample would be nice.

It was actually a bug that will be fixed in Citrus 2.7.4 release. See https://github.com/christophd/citrus/issues/328
The inline MTOM variant with XML config works for me in the current release.
<ws:send endpoint="simulationServer" mtom-enabled="true">
<message>
<resource file="testfiles/simulation/get-response.xml" />
</message>
<ws:attachment content-id="myAttachment" content-type="application/octet-stream" mtom-inline="true" encoding-type="base64Binary">
<ws:resource file="classpath:testfiles/myAttachment.pdf"/>
</ws:attachment>
</ws:send>

Related

Karate API - Getting 500 error while hitting SOAP service in batch

I am automating 3 APIs (Rest_1, Rest_2 and Soap_3 services) using Karate API. Basically Rest_1 output will be input to Rest_2 and Rest_2 output will be input to Soap_3. Created 3 feature files for each API and one master feature file that calls these 3 features. Rest_1-->Rest_2 flow is working as expected, Rest_2-->Soap_3 request XML is generating as expected but the same request is not hitting the Soap_3 service and getting the response 500.
When I execute Soap_3 feature file alone it is giving me the expected response. But in batch it is throwing 500 error.
If I take Rest_2-->Soap_3 generated XML and run it in SoapUI manually it is working fine.
Request your help in this issue. Thanks in advance !
Below is my Soap_3 service feature file:
Feature: Get PolicyDetails
Background:
* configure headers = {Content-Type: 'application/soap+xml; charset=utf-8'}
* configure logPrettyResponse = true
* configure ssl = true
* configure ssl = 'TLSv1.2'
* header Authorization = call read('classpath:resources/common/basic_auth.js') {username:'test',password:'test'}
* url 'https://soap_3apiurl'
Scenario: get the PolicyInfo
* configure charset = null
# getting submission id from preious feature file output
* call read('classpath:resources/dynamic/previous.feature'){'submissionID':'#(submissionID)'}
* xml req = read ('classpath:resources/common/RetrivePolicyDetails.xml')
* karate.set ('req/soapenv:Envelope/soapenv:Body/ns2:retrieveSubmission/ns2:aRequest/SubmissionID',submissionID)
Given request req
When soap action 'https://soap3apiurl'
Then status 200
And print response
**Console log:**
16:43:30.562 [ForkJoinPool-1-worker-1] DEBUG com.intuit.karate - response time in milliseconds: 58.18
1 < 500
1 < Accept-Encoding:
1 < Authorization: Basic
1 < Cneonction: close
1 < Content-Type: application/soap+xml; charset=UTF-8
1 < Cookie: NSC
1 < Date:
1 < Host: soap3APIurl.com
1 < Set-Cookie:
1 < Transfer-Encoding: chunked
1 < X-Forwarded-For: 10.00.00.1, 10.00.00.20
1 < X-Forwarded-Host:
1 < X-Forwarded-Server:
1 < X-dynaTrace: FW1;10000008;-1100030439;601946;6;-11000009;60194;1
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body>
<soapenv:Fault>
<faultcode xmlns:soap11Env="http://schemas.xmlsoap.org/soap/envelope/">soap11Env:Server</faultcode>
<faultstring>Error while building message</faultstring>
</soapenv:Fault>
</soapenv:Body>
</soapenv:Envelope>
16:43:30.565 [ForkJoinPool-1-worker-1] ERROR com.intuit.karate - status code was: 500, expected: 200
Do not leave a space after the read function reference:
* xml req = read('classpath:resources/common/RetrivePolicyDetails.xml')
This is explained here: https://github.com/intuit/karate/tree/develop/karate-core#locator-lookup
If this is not the issue, please follow this process: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
* configure headers = {Cookie : null}
Adding above line resolved my issue. Thanks.

Perl: Connecting to Walmart API to load items Unsupported Media type error

We are trying using the marketplace.walmartapis.com api to bulk list items and it is returning unsupported media type for xml item feed
We are using Perl and are able to perform other commands to update inventory and pricing, check orders, lookup feeds and skus etc. The token SHOULD be good, as we ran two calls back to back using the same token and the first call (item inventory) worked as expected before trying to send the xml file. It is only when we try to send the xml batch file that we run into the problem.
We assume the issue is with the header/authentication and have tried many different combinations. But obviously we are missing something.
What is being sent:
POST https://marketplace.walmartapis.com/v3/feeds?feedType=item HTTP/1.1
Headers:
Authorization: Basic MDc4......
Content-Length: 2277
Accept: application/xml
Content-Type: multipart/formdata
Host: marketplace.walmartapis.com
WM_QOS.CORRELATION_ID: TB123456V32
WM_SEC.ACCESS_TOKEN: eyJra...
WM_SVC.NAME: Walmart Marketplace
Boundary: 1234ran4321dom5678boundary
formdata payload:
--1234ran4321dom5678boundary
Content-Disposition: formdata; name="xml"
Content-length: 2151
<?xml version="1.0" encoding="UTF-8"?>
<MPItemFeed xmlns="http://walmart.com/">
<MPItemFeedHeader>
<version>3.1</version>
<requestBatchId>......</requestBatchId>
<feedDate>2019-02-18T19:45:17</feedDate>
<mart>WALMART_US</mart>
</MPItemFeedHeader>
<MPItem>
<processMode>CREATE</processMode>
<sku>....</sku>
<productIdentifiers>
<productIdentifier>
<productIdType>UPC</productIdType>
<productId>..........</productId>
</productIdentifier>
</productIdentifiers>
<MPProduct>
<productName>.....................</productName>
<category>
<SportAndRecreation>
<SportAndRecreationOther>
<shortDescription>........................</shortDescription>
<keyFeatures>
<keyFeaturesValue>I...............</keyFeaturesValue>
</keyFeatures>
<brand>............</brand>
<manufacturer>.............</manufacturer>
<manufacturerPartNumber>.............</manufacturerPartNumber>
<modelNumber>..</modelNumber>
<mainImageUrl>............</mainImageUrl>
<count>1</count>
<isProp65WarningRequired>No</isProp65WarningRequired>
<sportsLeague>
<sportsLeagueValue>....</sportsLeagueValue>
</sportsLeague>
<keywords>................</keywords>
<isMemorabilia>......</isMemorabilia>
<isCollectible>...........</isCollectible>
</SportAndRecreationOther>
</SportAndRecreation>
</category>
</MPProduct>
</MPOffer>
<price>21.95</price>
<StartDate>2019-02-23T19:45:17</StartDate>
<EndDate>2019-04-19T19:45:17</EndDate>
<ShippingWeight>
<measure>2</measure>
<unit>lb</unit>
</ShippingWeight>
<ProductTaxCode>2038345</ProductTaxCode>
</MPOffer>
</MPItem>
</MPItemFeed>
--1234ran4321dom5678boundary--
back from Walmart.pm
$HASH = {
"error" => {
"category" => "DATA",
"causes" => {},
"code" => "UNSUPPORTED_MEDIA_TYPE.GMP_GATEWAY_API",
"errorIdentifiers" => {},
"info" => "Unsupported Media Type.",
"severity" => "ERROR"
},
"xmlns:ns2" => "http://walmart.com/"
};
Any pointers in the right direction would be appreciated.
Thanks for everyone that looked. The problem seems to be solved
It was one minor typo
multipart/formdata needed to be multipart/form-data
With that change now we are getting a feed id that shows up in the Seller area

Azure media service job creation fails using rest api

Trying to consume Azure media service rest api. (following the tutorial : https://learn.microsoft.com/en-us/azure/media-services/media-services-rest-get-started)
Everything works fine until the point I try to create a Job. Sending the same request as in example (except asset id and token) and getting response :
Parsing request content failed due to: Make sure to only use property names that are defined by the type
Request:
POST https://wamsdubclus001rest-hs.cloudapp.net/api/Jobs HTTP/1.1
Connection: Keep-Alive
Content-Type: application/json
Accept: application/json; odata=verbose
Accept-Charset: UTF-8
Authorization: Bearer token -> here i send real token
DataServiceVersion: 1.0;NetFx
MaxDataServiceVersion: 3.0;NetFx
x-ms-version: 2.11
Content-Length: 458
Host: wamsdubclus001rest-hs.cloudapp.net
{
"Name":"TestJob",
"InputMediaAssets":[
{
"__metadata":{
"uri":"https://wamsdubclus001rest-hs.cloudapp.net/api/Assets('nb%3Acid%3AUUID%3A5168b52a-68ed-4df1-bac8-0648ce734ff6')"
}
}
],
"Tasks":[
{
"Configuration":"Adaptive Streaming",
"MediaProcessorId":"nb:mpid:UUID:ff4df607-d419-42f0-bc17-a481b1331e56",
"TaskBody":"<?xml version=\"1.0\" encoding=\"utf-8\"?><taskBody><inputAsset>JobInputAsset(0)</inputAsset> <outputAsset>JobOutputAsset(0)</outputAsset></taskBody>"
}
]
}
Response:
{
"error":{
"code":"",
"message":{
"lang":"en-US",
"value":"Parsing request content failed due to: Make sure to only use property names that are defined by the type"
}
}
}
It seems to be related with __metadata property. when I follow instruction from here : Creating Job from REST API returns a request property name error, the error changes:
"error":{
"code":"",
"message":{
"lang":"en-US",
"value":"Invalid input asset reference in TaskBody - "
}
}
}
Cant figure out whats wrong, thanks
Let me check on this, but it could be a couple issues that I have run into in the past.
First. Set both the Accept and Content-Type headers to:
"application/json; odata=verbose"
Next, double check that you are actually using the long underscore character on the metadata property. I've had issues where that was sending the wrong underscore character and it didn't match the property name.
Let me know if either of those helps.
It seems the issue was about "Content-Type". As I am using .net Core it was not easy to set the Conent-type as "application/json; odata=verbose".
1) Tried with RestSharp - dosnt support it, it cuts "odata=verbose" part out
2) Tried with Systsem.Net.Http.HttpClient -> Possible but difficult.
To add it as "Accept" :
MediaTypeWithQualityHeaderValue mtqhv;
MediaTypeWithQualityHeaderValue.TryParse("application/json;odata=verbose", out mtqhv);
client.DefaultRequestHeaders.Accept.Add(mtqhv);//ACCEPT header
To add it as "Content-Type" :
request.Content = new StringContent(content,
System.Text.Encoding.UTF8); //CONTENT-TYPE header -> default type will be text/html
request.Content.Headers.Clear(); // need to clear it - it will fail otherwise
request.Content.Headers.TryAddWithoutValidation("Content-Type","application/json;odata=verbose");

POST request on arduino with ESP8266 using WifiESP library

I am attempting to make RESTful POST request using the WifiESP library (https://github.com/bportaluri/WiFiEsp). I'm able to successfully make the request with curl, but consistently get an error using the Arduino and ESP. I suspect the problem is related to the manual formatting of the POST request the library requires, but I don't see anything wrong. Here my sanitized code:
if (client.connect(server, 80)) {
Serial.println("Connected to server");
// Make a HTTP request
String content = "{'JSON_key': 2.5}"; // some arbitrary JSON
client.println("POST /some/uri HTTP/1.1");
client.println("Host: http://things.ubidots.com");
client.println("Accept: */*");
client.println("Content-Length: " + sizeof(content));
client.println("Content-Type: application/json");
client.println();
client.println(content);
}
The error I get (via serial monitor) is this:
Connected to server
[WiFiEsp] Data packet send error (2)
[WiFiEsp] Failed to write to socket 3
[WiFiEsp] Disconnecting 3
My successful curl requests looks like this:
curl -X POST -H "Content-Type: application/json" -d 'Some JSON' http://things.ubidots.com/some/uri
After some experimentation, here is the solution to the multiple problems.
The JSON object was not correctly formatted. Single quotes were not accepted, so I needed to escape the double quotes.
The host does not need "http://" in a POST request; POST is a HTTP method.
The sizeof() method returns the size, in bytes, of the variable in memory rather than the length of the string. It needs to be replaced by .length().
Appending an integer to a string requires a cast.
This is the corrected code:
if (client.connect(server, 80)) {
Serial.println("Connected to server");
// Make the HTTP request
int value = 2.5; // an arbitrary value for testing
String content = "{\"JSON_key\": " + String(value) + "}";
client.println("POST /some/uri HTTP/1.1");
client.println("Host: things.ubidots.com");
client.println("Accept: */*");
client.println("Content-Length: " + String(content.length()));
client.println("Content-Type: application/json");
client.println();
client.println(content);
}
The code explained by Troy D is right and it's working .I think the error in posting the data to the server is due to this line
client.println("Content-Length: " + sizeof(content));
and the correct way is
client.println("Content-Length: " + String(content.length()));
Now coming to this error
Connected to server
[WiFiEsp] Data packet send error (2)
[WiFiEsp] Failed to write to socket 3
[WiFiEsp] Disconnecting 3
This is the error of library you can ignore it.
The problem with "Data packet send error (2)", "Failed to write to socket 3" and "Disconnecting 3" is not a problem within the WifiEsp library as far as I can see, believe it's more likely to be within the AT firmware. By default the http headers contain a "Connection: close" parameter which in normal cases should be correct. However with this bug the server will get disconnected before the reply is received on the client side and any response from the server will be identified as garbage data. Using the value "Connection: keep-alive" as a workaround will make it possible to receive the acceptance from the server in a proper way.
I'm running my Arduino + ESP8266-07 against a MVC based Web Api that I created on one of my servers and in the controllers Post-method I use a single string as return value, the value I return if everything is ok is simply one of the strings that WifiEsp keeps track of (It will still include the http status code in the response header that it returns)
public async Task<string> Post([FromBody]JObject payload)
{
//Code to handle the data received, in my case I log unit ip, macaddress, datetime and sensordata into a db with entity framework
return "SEND OK";
}
So in your Arduino code try following instead:
String PostHeader = "POST http://" + server + ":" + String(port) + "/api/values HTTP/1.1\r\n";
PostHeader += "Connection: keep-alive\r\n";
PostHeader += "Content-Type: application/json; charset=utf-8\r\n";
PostHeader += "Host: " + server + ":" + String(port) + "\r\n";
PostHeader += "Content-Length: " + String(jsonString.length()) + "\r\n\r\n";
PostHeader += jsonString;
client.connect(server.c_str(), port);
client.println(PostHeader);
client.stop();
In the file debug.h located in the library source code you could alter a define and get more output to your serial console. Open the file and change
#define _ESPLOGLEVEL_ 3
to
#define _ESPLOGLEVEL_ 4
Save the file and recompile/deploy your source code to your Arduino and you will get extensive information about all AT commands the library sends and what the library receives in return.

Use of charter encoding with content type in JAX-RS 2.0 - Jersey

I am developing JAX-RS 2.0 client with Jersey implementation. REST API returns response with content type as 'application/json; UTF-8'
At client side, Jersey cannot recognize this content-type and throws error like
Unable to parse "Content-Type" header value: "application/json; UTF-8"
org.glassfish.jersey.message.internal.HeaderValueException: Unable to parse "Content-Type" header value: "application/json; UTF-8"
at org.glassfish.jersey.message.internal.InboundMessageContext.exception(InboundMessageContext.java:318)
at org.glassfish.jersey.message.internal.InboundMessageContext.singleHeader(InboundMessageContext.java:313)
at org.glassfish.jersey.message.internal.InboundMessageContext.getMediaType(InboundMessageContext.java:427)
If content type is "application/json" same code could process the response. I tried to search over google and some posts say that such character encoding along with content type is supported in Jersey but I didn't find it working.
Following is code I am trying with.
Client clientSaml = ClientBuilder.newBuilder().sslContext(context)
.hostnameVerifier(new HostnameVerifier() {
#Override
public boolean verify(String arg0, SSLSession arg1) {
// TODO Auto-generated method`enter code here` stub
return true;
}
}).build();
clientSaml.register(new SAMLAuthenticator(token));
clientSaml.register(new LoggingFilter());
WebTarget target = clientSaml
.target(URL_OF_API);
Response resp = target.request(MediaType.APPLICATION_JSON).get();
System.out.println(resp.getMediaType().toString());
Does anybody had this issue before and what could be possible solution of this ?? Does any other implementation of JAX-RS support it ?
You forgot charset=:
Content-Type: application/json; charset=UTF-8
See Section 14.17 Content-Type of HTTP 1.1.
I did workaround of this issue using ClientResponseFilter
Thanks to hint suggested in Jersey and #FormParam not working when charset is specified in the Content-Type