ColdFusion 10 RESTful Web Service unable to receive gzip'd (binary) body - rest

I've written a CF10 RESTful web service that accepts a POST from a third party. The POST comes in with successfully with proper headers showing that it's content-type is application/json and content-encoding is gzip.
The body however comes in like this
??VJ.-.??M-?LQ?R22?0W?Q??Os-????b??????_?ZT??175 ????T?E???r??KKJ??3??S]A?#u??%??`?f??FJ???`?
The issue is that in the receiving function the cfargument for body is set to type="string" when really what it should be is type="binary". Unfortunately setting the type to binary causes the call to fail. The calling server receives the following:
Notification response HTTP/1.1 500 Internal Server Error
Content-Length: 41 Content-Type: text/plain Server: Microsoft-IIS/7.5
CF_TOMCAT_REUSE_THIS_CONNECTION: FALSE X-Powered-By: ASP.NET Date:
Thu, 30 May 2013 19:53:17 GMT Connection: close
{"Message":"Variable BODY is undefined."}
I have no control of the third party call to my REST endpoint.
Does anyone have any ideas that would allow my REST endpoint to receive this gzipped (binary) body? Alternatively, does anyone know how to convert the string representation back into a gzip'd binary that can then be inflated and I can then recover the json packet?
My code looks like this:
<cffunction name="trigger" access="remote" returntype="string" httpmethod="POST">
<cfargument name="body" type="any" >
<cfargument name="Length" type="String" restArgsource="Header" restargname="Content-Length" >
<cfargument name="Type" type="String" restArgsource="Header" restargname="Content-Type">
<cfargument name="Encoding" type="String" restArgsource="Header" restargname="Content-Encoding" >
<cfset var result = "HTTP/1.1 200 OK" >
<!--- Do some processing here --->
<cfreturn result>
</cffunction>
Thank you all in advance.

Just to add some additional input here, I did submit a bug request as attempting to change the web.xml configuration file did not initially work for me. I had added additional entries into configuration file for the GZIP decoding and instead what needed to be done was to add to the existing entries the parameter value for the GZIP encoding filters.
The original-unchanged file in web.xml should show entries such as this, found under the servlet named CFRestServlet:
<init-param>
<param-name>com.sun.jersey.spi.container.ContainerRequestFilters</param-name>
<param-value>coldfusion.rest.servlet.CFUriConnegFilter;coldfusion.rest.servlet.CFRequestFilter</param-value>
</init-param>
<init-param>
<param-name>com.sun.jersey.spi.container.ContainerResponseFilters</param-name>
<param-value>coldfusion.rest.servlet.CFResponseFilter</param-value>
</init-param>
In order to enable the gzip filters, update these two parameters to have them to appear this way instead:
<init-param>
<param-name>com.sun.jersey.spi.container.ContainerRequestFilters</param-name>
<param-value>coldfusion.rest.servlet.CFUriConnegFilter;com.sun.jersey.api.container.filter.GZIPContentEncodingFilter;coldfusion.rest.servlet.CFRequestFilter</param-value>
</init-param>
<init-param>
<param-name>com.sun.jersey.spi.container.ContainerResponseFilters</param-name>
<param-value>coldfusion.rest.servlet.CFResponseFilter;com.sun.jersey.api.container.filter.GZIPContentEncodingFilter</param-value>
</init-param>
As you'll see, you simply need to add the "com.sun.jersey.api.container.filter.GZIPContentEncodingFilter" as another value to the semi-colon separated list of values. Once you've made the changes, restart your CF server and then your REST based services will properly and automatically decode any GZip encoded data they receive.
For reference, my bug request can be read here: https://bugbase.adobe.com/index.cfm?event=bug&id=3694176
Thanks.

If I understand your problem correctly, this can be done with GZIPContentEncodingFilter
In short CF's rest support is built upon JERSEY APIs. When a request comes to a rest endpoint (defined in the URL), a series of jersey filters execute before invoking the actual function. This is where this filter would execute and handle GZIP compression for you.
You may configure only the request filter (In case the library is not expecting a gzipped response back). Just add following in web.xml and hopefully will be done.
<init-param>
<param-name>com.sun.jersey.spi.container.ContainerRequestFilters</param-name>
<param-value>com.sun.jersey.api.container.filter.GZIPContentEncodingFilter</param-value>
</init-param>
CF already has one filter added for handling *.json & *.xml requests.
HTH,
Chandan Kumar

GZip encoding support added in CF11: https://wikidocs.adobe.com/wiki/display/coldfusionen/RESTful+Web+Services+in+ColdFusion#RESTfulWebServicesinColdFusion-SupportforGZipencoding

Related

What is a working minimal example of POST /channelgroups/_bulkUpdate in Mirth REST API?

In the Mirth REST API found on EG https://localhost:8443/api/, under Channel Groups, there's a test section for bulkUpdate, which has two parameters: channelGroups, and removedChannelGroupIds. By default I set override to true.
Despite passing channelGroups and the ID list in various formats (EG as a channelGroup XML object, list of channels, list of channelGroups etc) Mirth rejects those formats and I have no idea what format Mirth is after. Leaving either field blank also fails. Does anyone have an example of a minimal working dataset for both fields that will return success from Mirth when calling bulkUpdate?
Mirth version is 3.6.0, but ideally it should also work in 3.5.1 for legacy devices.
This was answered on the user forum https://www.mirthcorp.com/community/forums/showthread.php?t=218606
That route actually expects multipart/form-data. So for example you could send a request with "Content-Type: multipart/form-data; boundary=abc123" and a payload like:
--abc123
Content-Type: application/xml; charset=utf8
Content-Disposition: form-data; name="channelGroups"
<set>
<channelGroup version="3.6.1">
<id>56a61dfb-58df-4286-8100-5ccab05364ba</id>
<name>Group 1</name>
<revision>1</revision>
<lastModified>
<time>1537550138646</time>
<timezone>UTC</timezone>
</lastModified>
<description></description>
<channels/>
</channelGroup>
</set>
--abc123
Content-Type: application/xml; charset=utf8
Content-Disposition: form-data; name="removedChannelGroupIds"
<set/>
--abc123--
As of mirth 3.6, it is not possible to call this API function from the SwaggerUI.
It also seems to be failing to create the channels within the channel group.

Passing encoded XML content in SOAP envelop request in Java application

I'm developing a Java application where I need to make a web service request using SOAP protocol. In that SOAP envelop request, one of the XML attribute/property called content will hold the XML file itself like below:-
<n1:envelope>
<message>
<id>67872894892424</id>
</message>
<sender agency="GS1" scheme="tGLK">ur.0</sender>
<recipients>
<id agency="GS1" scheme="tGLK">test</id>
</recipients>
<content encoding="XML" extension="A" format="ATTP">
------
----
</content></n1:envelope>
I'm new to it and requires some assistance in clarifying my doubts. I'm using JDK1.8 and Apache CXF. How can I do marshaling/unmarshalling for this particular content attribute? Normal SOAP envelops request is easy to do it but not sure how to pass an encoded XML itself in SOAP request. Is there any reference links? Thanks
Your case is an example of including XML inside XML. Yes, XML could be included inside an SOAP(XML), there are two ways, though both means almost same thing.
Using CDATA encoding
<hello><![CDATA[<El><E2><E3 attr="D1">Text</E3></E2></El>]]></hello>
Converting the XML into text by replacing, < with <, " with &quote; and > with >
<hello><El><E2><E3 attr="D1">Text</E3></E2></El></hello>
To
<hello>>El<>E2<>E3 attr="D1"<Text>/E3<>/E2<>/El<</hello>
Hence, to include the XML inside SOAP, while adding the XML element, you need to follow either approach while marshaling/unmarshaling.

How to send REST call in WSO2ESB with "Date" transport header

I'm trying to call a REST service from within WSO2ESB. The request needs to contain an authentication code, which will get computed based on some values including the Date transport header, which must also be part of the transport headers.
<syn:property name="Date" value="Mi, 1 Mrz 2015 11:00:00 MEZ" scope="transport" />
<syn:property name="X-Auth-Code" value="SomeCodeBasedOnDateHeader" scope="transport" />
<syn:send>
<syn:endpoint>
<syn:http uri-template="http://localhost:8280/rest/resourceA/{uri.var.resA}/resourceB/{uri.var.resB}" method="POST" />
</syn:endpoint>
</syn:send>
But when trying to send the request, the Date transport header will get removed by WSO2 ESB (The REST service will not get any Date header). Is there any chance to include the Date Header?
You can preserve the date by adding the following property to
D:\stack\wso2am-2.1.0\repository\conf\nhttp.properties
D:\stack\wso2am-2.1.0\repository\conf\passthru-http.properties
http.headers.preserve=Date
It seems that thoses headers are removed in both, NIO and Passthrough http transports :
Connection
Transfer-Encoding
Date
Content-Length
Keep-Alive
Server
User-Agent
You can preserve Server and User-Agent headers, setting http.server.preserve=true or http.user.agent.preserve=true in ESB_HOME/repository/conf/nhttp.properties or passthru-http.properties depending on which transport is configured in axis2, but it looks like there is no property to preserve Date header : you may be required to write your own http transport sender and configure your custom class in ESB_HOME/repository/conf/axis2/axis2.xml : <transportSender name="http" class="sss">

Changing DynamicChannelBuffer in Netty to String and back to ChannelBuffer

My web server is written in Scala using Twitter's Finagle library, which in turn relies on Netty. As such, the request content is returned as a DynamicChannelBuffer. If I upload an image to the server using curl from the Terminal like this:
curl -T "abc.jpg" http://127.0.0.1:8080/test/image
Then I can read, and forward the image to a backend webserver using a SOAP packet that looks like this:
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Header>
<AuthHeader xmlns="http://www.testtesttest.co.za/">
<LogonID>testtesttest</LogonID>
<Password>testtesttest</Password>
</AuthHeader>
</soap:Header>
<soap:Body>
<uploadFile xmlns="http://www.testtesttest.co.za/">
<FileDetails>
<FileName>image.jpg</FileName>
<FileContents>
{(Base64.encode(request.getContent())).toString(UTF_8)
</FileContents>
</FileDetails>
</uploadFile>
</soap:Body>
</soap:Envelope>
In the example above, the code: (Base64.encode(request.getContent())).toString(UTF_8) converts the request content to a base 64 encoded string.
The problem is that I need to read the image content from Multipart Http request that is sent from a PhoneGap mobile app. PhoneGap gives me no option, to send only the image, and insists in doing the file upload as a multipart request.
To break the multipart request apart, I change the request.getContent() result into a string using toString(UTF_8), and then getting the image data part by splitting the http multipart message into it's separate chunks:
var requestParts = request.content.toString(UTF_8).split("\\Q--*****org.apache.cordova.formBoundary\\E")
val imageParts = requestParts(3).split("\\n\\s*\\n")
val imageHeader = imageParts(0)
val imageBody = imageParts(1)
This is crappy, I know (I'll improve later), but does the trick for now. imageBody now has the image content as a string.
Now, if I put the imageBody back into the SOAP packet, I have to encode it again using:
val encoder = new BASE64Encoder();
val encodedImage = encoder.encode(imageBody)
At this point the image is just garble. It's size looks right, but I'm messing something up with the string conversion or encoding. For the first example, I'm using Netty's encoder, but for the second example I'm using the standard java encoder. The reason is that Netty's encoder can only encode objects of type ChannelBuffer.
I don't want to say this too loud, but I've been struggling with this for more than a day. Any help here will be GREATLY appreciated.
So this works:
image --> [curl] ------> post1 --> [your code] --> soap msg 1 --> [back-end]
This does not:
image --> [phonegap] --> post2 --> [your code] --> soap msg 2 --> [back-end]
To solve this type of problem reliably you need to understand which encoding is used in each step.
Assuming you can use the same image, can you check the raw encoded content in post1 and post2 and infer which encoding is being used? Then when you understand that, log the content in your code as you decode and recode the message. That way you can ensure it's the same in soap msg1 and soap msg2.

SOAP Action WSDL

I'm trying to implement a client for National Rail Enquiries' SOAP Service (http://www.livedepartureboards.co.uk/ldbws/).
I stick the WSDL (http://realtime.nationalrail.co.uk/ldbws/wsdl.aspx) into http://soapclient.com/soaptest.html, but I get back the error message "Unable to handle request without a valid action parameter. Please supply a valid soap action."; what on earth should the action be?
Thanks,
Stewart
edit:
I just used soapclient.com as a quick example. In my software, I send the following XML; I still get that I'm missing an action.
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://www.w3.org/2003/05/soap-envelope" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:tns="http://thalesgroup.com/RTTI/2008-02-20/ldb/" xmlns:soap12="http://schemas.xmlsoap.org/wsdl/soap12/" xmlns:ldbt2="http://thalesgroup.com/RTTI/2008-02-20/ldb/types" xmlns:ldbt="http://thalesgroup.com/RTTI/2007-10-10/ldb/types" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns:ct="http://thalesgroup.com/RTTI/2007-10-10/ldb/commontypes" >
<SOAP-ENV:Body>
<ldbt2:GetDepartureBoardRequest xmlns:ldbt2="http://thalesgroup.com/RTTI/2008-02-20/ldb/" >
<ldbt2:numRows>5</ldbt2:numRows>
<ldbt2:crs>WAT</ldbt2:crs>
<ldbt2:filterCrs>GLD</ldbt2:filterCrs>
<ldbt2:filterType>to</ldbt2:filterType>
<ldbt2:timeOffset>0</ldbt2:timeOffset>
</ldbt2:GetDepartureBoardRequest>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
If its a SOAP 1.1 service then you will also need to include a SOAPAction HTTP header field:
http://www.w3.org/TR/2000/NOTE-SOAP-20000508/#_Toc478383528
I have come across exactly the same problem when trying to write a client for the National Rail SOAP service with Perl.
The problem was caused because the Perl module that I'm using 'SOAP::Lite' inserts a '#' in the SOAPAction header ...
SOAPAction: "http://thalesgroup.com/RTTI/2008-02-20/ldb/#GetDepartureBoard"
This is not interpreted correctly by .NET servers. I found this out from Example 3-19 in O'Reilly's Programming Web Services with SOAP . The solution was given below in section 3-20, namely you need to explicitly specify the format of the header with the 'on_action' method.
print SOAP::Lite
-> uri('urn:Example1')
-> on_action(sub{sprintf '%s/%s', #_ })
-> proxy('http://localhost:8080/helloworld/example1.asmx')
-> sayHello($name)
-> result . "\n\n";
My guess is that soapclient.com is using SOAP::Lite behind the scenes and so are hitting the same problem when talking to National Rail.
The solution is to write your own client so that you have control over the format of the SOAPAction header ... but you've probably done that already.
SOAPAction is required in SOAP 1.1 but can be empty ("").
See https://www.w3.org/TR/2000/NOTE-SOAP-20000508/#_Toc478383528
"The header field value of empty string ("") means that the intent of the SOAP message is provided by the HTTP Request-URI."
Try setting SOAPAction=""
When soapAction is missing in the SOAP 1.2 request (and many clients do not set it, even when it is specified in WSDL), some app servers (eg. jboss) infer the "actual" soapAction from {xsd:import namespace}+{wsdl:operation name}.
So, to make the inferred "actual" soapAction match the expected soapAction, you can set the expected soapAction to {xsd:import namespace}+{wsdl:operation name} in your WS definition (#WebMethod(action=...) for Java EE)
Eg. for a typical Java EE case, this helps (not the Stewart's case, National Rail WS has 'soapAction' set):
#WebMethod(action = "http://packagename.of.your.webservice.class.com/methodName")
If you cannot change the server, you will have to force client to fill soapAction.
I've just spent a while trying to get this to work an have a written a Ruby gem that accesses the API. You can read more on it's project page.
This is working code in Ruby:
require 'savon'
client = Savon::Client.new do
wsdl.document = "http://realtime.nationalrail.co.uk/LDBWS/wsdl.aspx"
end
response = client.request 'http://thalesgroup.com/RTTI/2012-01-13/ldb/GetDepartureBoard' do
namespaces = {
"xmlns:soap" => "http://schemas.xmlsoap.org/soap/envelope/",
"xmlns:xsi" => "http://www.w3.org/2001/XMLSchema-instance",
"xmlns:xsd" => "http://www.w3.org/2001/XMLSchema"
}
soap.xml do |xml|
xml.soap(:Envelope, namespaces) do |xml|
xml.soap(:Header) do |xml|
xml.AccessToken do |xml|
xml.TokenValue('ENTER YOUR TOKEN HERE')
end
end
xml.soap(:Body) do |xml|
xml.GetDepartureBoardRequest(xmlns: "http://thalesgroup.com/RTTI/2012-01-13/ldb/types") do |xml|
xml.numRows(10)
xml.crs("BHM")
xml.filterCrs("BHM")
xml.filterType("to")
end
end
end
end
end
p response.body
Hope that's helpful for someone!
We put together Web Services on Windows Server and were trying to connect with PHP on Apache. We got the same error. The issue ended up being different versions of the Soap client on the different servers. Matching the SOAP versions in the options on both servers solved the issue in our case.
the service have 4 operations:
1. GetServiceDetails
2. GetArrivalBoard
3. GetDepartureBoard
4. GetArrivalDepartureBoard
I have solved this problem, in Java Code, adding:
MimeHeaders headers = message.getMimeHeaders();
headers.addHeader("SOAPAction", endpointURL);