ColdFusion CFHTTP connection to remote web service - soap

Question

I have successfully connected to a remote webservice using SOAPUI (www.soapui.org). But, I am having trouble calling it successfully from CF9.2.
Here is my entire CFC function. There are dynamic vars but I've tested the output in the soapUI interface and it works:
<cffunction name="getOrganisation" access="remote" returnType="any" output="true">
<cfargument name="iPageNumber" type="any" required="false" default="0">
<cfargument name="iPageSize" type="any" required="false" default="0">
<cfargument name="bCurrentNamesOnly" type="boolean" required="false" default="1">
<cfargument name="bExcludeNotRtos" type="boolean" required="false" default="0">
<cfargument name="bExcludeRtoWithoutActiveRegistration" type="boolean" required="false" default="0">
<cfargument name="sFilter" type="any" required="false" default="">
<cfargument name="bIncludeCode" type="boolean" required="false" default="1">
<cfargument name="sRegistrationManagers" type="any" required="false" default="">
<cfargument name="sClassificationFilters" type="any" required="false" default="">
<cfargument name="sScheme" type="any" required="false" default="">
<cfset var endpoint = "https://ws.staging.training.gov.au/Deewr.Tga.WebServices/OrganisationService.svc/Organisation">
<cfsavecontent variable="soapBody">
<cfoutput>
<soapenv:Envelope
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:ser="http://training.gov.au/services/"
xmlns:arr="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
<soapenv:Header/>
<soapenv:Body>
<ser:Search>
<ser:request>
<ser:PageNumber>#arguments.iPageNumber#</ser:PageNumber>
<ser:PageSize>#arguments.iPageSize#</ser:PageSize>
<ser:ClassificationFilters>
<ser:ClassificationFilter>
<ser:Scheme>#arguments.sScheme#</ser:Scheme>
<ser:Values>
<cfif len(arguments.sClassificationFilters)>
<cfloop list="#arguments.sClassificationFilters#" index="item">
<arr:string>#item#</arr:string>
</cfloop>
</cfif>
</ser:Values>
</ser:ClassificationFilter>
</ser:ClassificationFilters>
<ser:CurrentNamesOnly>#arguments.bCurrentNamesOnly#</ser:CurrentNamesOnly>
<ser:ExcludeNotRtos>#arguments.bExcludeNotRtos#</ser:ExcludeNotRtos>
<ser:ExcludeRtoWithoutActiveRegistration>#arguments.bExcludeRtoWithoutActiveRegistration#</ser:ExcludeRtoWithoutActiveRegistration>
<ser:Filter>#arguments.sFilter#</ser:Filter>
<ser:IncludeCode>#arguments.bIncludeCode#</ser:IncludeCode>
<ser:RegistrationManagers>
<cfif len(arguments.sRegistrationManagers)>
<cfloop list="#arguments.sRegistrationManagers#" index="item">
<arr:string>#item#</arr:string>
</cfloop>
</cfif>
</ser:RegistrationManagers>
</ser:request>
</ser:Search>
</soapenv:Body>
</soapenv:Envelope>
</cfoutput>
</cfsavecontent>
<cfhttp
url="#endpoint#"
method="post"
username="#variables.username#"
password="#variables.password#">
<cfhttpparam type="header" name="accept-encoding" value="no-compression" />
<cfhttpparam type="xml" value="#trim(soapBody)#"/>
</cfhttp>
<cfdump var="#cfhttp.FileContent#"><cfabort>
<cfreturn cfhttp.FileContent>
</cffunction>
Running this, I get the error:
An error occurred when verifying security for the message.
Following is the complete return xml
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<s:Fault>
<faultcode xmlns:a="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">a:InvalidSecurity</faultcode>
<faultstring xml:lang="en-AU">An error occurred when verifying security for the message.</faultstring>
</s:Fault>
</s:Body>
</s:Envelope>
So, it appears to be an authorisation issue.
Here is the SoapUI request screen:
So, how do I construct the cfhttp, or maybe cfinvoke, to emulate the soapUI call?
EDIT
SOAP Request XML
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ser="http://training.gov.au/services/" xmlns:arr="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
<soapenv:Header/>
<soapenv:Body>
<ser:Search>
<ser:request>
<ser:PageNumber>0</ser:PageNumber>
<ser:PageSize>0</ser:PageSize>
<ser:ClassificationFilters>
<ser:ClassificationFilter>
<ser:Scheme></ser:Scheme>
<ser:Values>
<arr:string></arr:string>
</ser:Values>
</ser:ClassificationFilter>
</ser:ClassificationFilters>
<ser:CurrentNamesOnly>true</ser:CurrentNamesOnly>
<ser:ExcludeNotRtos>0</ser:ExcludeNotRtos>
<ser:ExcludeRtoWithoutActiveRegistration>0</ser:ExcludeRtoWithoutActiveRegistration>
<ser:Filter></ser:Filter>
<ser:IncludeCode>1</ser:IncludeCode>
<ser:RegistrationManagers>
<arr:string></arr:string>
</ser:RegistrationManagers>
</ser:request>
</ser:Search>
</soapenv:Body>
</soapenv:Envelope>
EDIT 2
Further information:
Here is the url directly to the service: https://ws.staging.training.gov.au/Deewr.Tga.WebServices/OrganisationService.svc
Below is a screenshot of some authentication related information from the docs

Answer 1

I'm thinking that you need to post the username and password in cfhttpparam tags, not as attributes in the cfhttp tag.

Answer 2

SOAPUI is a fantastic tool, and one I've bee using a lot recently when debugging SOAP requests or transferring them from static .wsdl files into a ColdFusion component.
The first thing to look at is the error message itself:
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<s:Fault>
<faultcode xmlns:a="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">a:InvalidSecurity</faultcode>
<faultstring xml:lang="en-AU">An error occurred when verifying security for the message. </faultstring>
</s:Fault>
</s:Body>
</s:Envelope>
The faultcode node contains a link to the OASIS security namespace document:
http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd
Viewing that in the browser, you can see what security values are required and you can clarify the naming conventions. So, we can confirm from this that you do need to send through the following with the following format:
Username
Password
Without having access to the request you are testing in SOAPUI, we can also see that the Username and Password values are visible in the property window (your screen grab from the interface, above).
Are these values set within the XML of the request in SOAPUI? Perhaps in the SOAP request header, like so:
<soapenv:Header>
<authInfo xsi:type="soap:authentication">
<Username xsi:type="xsd:string">?</Username>
<Password xsi:type="xsd:string">?</Password>
</authInfo>
</soapenv:Header>
If that is the case, you will also need to include this header in your ColdFusion component when building the soapBody variable.
Also, can you view the URL of the SOAP request directly in the browser to see what variables it expects?
https://ws.staging.training.gov.au/Deewr.Tga.WebServices/OrganisationService.svc/Organisation
The overall structure of your CFC and soapBody looks good, so without having access to the file you are running in SOAPUI to see it running, debugging it and providing answers would be a little tricky.
If you can rule out all of the possibilities mentioned above and still have issues, let me know.

Answer 3

Yes, you were not originally passing a Oasis security header. The answers here so far probably aren't going to work however. Oasis standard can allow for many an varied combinations of user/password/nonce/digest/created and encoding attributes.
As you may or may not have found, the Axis 1 library (CF9 and lower user Axis 1 (versions 1.1-1.4); CF10 also has Axis 2 (Axis version 1.6)). And thus you must POST (cfhttp/http) to pass and receive an Oasis header. The Axis 1 library will cry about trying to understand the Oasis node in the header area.**
Since you are manually posting, you can simply construct your Oasis header in your header much like the example fro Matt Gifford.
Depending on what exactly the receiving party is expecting, you may will have something like this:
<wsse:Security soap:mustUnderstand="true" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns:soap="http://www.w3.org/2003/05/soap-envelope">
<wsse:UsernameToken wsu:Id="UsernameToken-15">
<wsse:Username>
<!-- Removed-->
</wsse:Username>
<wsse:Password>
<!-- Removed-->
</wsse:Password>
<wsse:Nonce>
<!-- Removed-->
</wsse:Nonce>
<wsu:Created>2012-07-11T02:02:48.410Z</wsu:Created>
</wsse:UsernameToken>
</wsse:Security>
or this (from early cladding method pre-output):
<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" soapenv:mustUnderstand="1">
<wsse:UsernameToken xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" >
<wsse:Username>#Arguments.szUserName#</wsse:Username>
<wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0##PasswordText">#Arguments.szPassword#</wsse:Password>
</wsse:UsernameToken>
</wsse:Security>
or this (final output):
<wsse:Security soapenv:mustUnderstand="1"xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
<wsse:UsernameToken wsu:Id="UsernameToken-974900"xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
<wsse:Username>SuperJellyMan</wsse:Username>
<wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">jellybeanboom</wsse:Password>
<wsse:Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary">SmVsbHlCZWFuQm9vbTk3NDkwMA==</wsse:Nonce>
<wsu:Created>2012-07-26T17:00:34Z</wsu:Created>
</wsse:UsernameToken>
</wsse:Security>
And so on and so forth. I have seen people invoke java libraries to do this more dynamically. I simply built a few simple methods to build my nonce and format my xml time stamp. But basically, it is pretty simple to build some kind of function or other helper methods to clad the values required in xml with the proper Oasis attributes and xsd references and then insert that in your header of your request of the manual post.
The tricky part is knowing exactly what they want in the Oasis security header if they do not tell you. The xsd simply shows what is acceptable according to Oasis standards, but not necessarily what they are using (and in what combination). If you can get that from them, you can pretty easily put that together from the Oasis docs.
Usually, the combination is User/Pwd (first example). If a Nonce is used, very often a Password Digest is also used (the password gets a ##PasswordDigest instead of ##PasswordText) and is an encoded hash of the created time and password (see Oasis docs for more explanation).
But, the last project I worked on had a plain text pwd plus a nonce with no digest. Not exactly logical since the nonce is supposed to make a fake request more difficult and use a non-clear text password which needs to be decoded by receiver and compared to validate... So it is totally up to the service on what they want/use/need.
You should be able to paste your concocted Oasis header into SoapUI and pretty quickly figure out what they might need/want from their responses to same.
If they have specified user/password (as text), you might find one of these examples (probably the second one) workable and not passing the nonce and created elements.
** Here is the Axis 1 fault for fun reference when Oasis is used with CF9 or less using actual SOAP methods (web service stub, rather than manual post):**
The fault returned when invoking the web service operation is: AxisFault faultCode: {http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd}InvalidSecurity faultSubcode: faultString: An error occurred when verifying security for the message. faultActor: faultNode:
faultDetail: {http://xml.apache.org/axis/}stackTrace:An error occurred when verifying security for the message. at org.apache.axis.message.SOAPFaultBuilder.createFault(SOAPFaultBuilder.java:221) at org.apache.axis.message.SOAPFaultBuilder.endElement(SOAPFaultBuilder.java:128) at org.apache.axis.encoding.DeserializationContext.endElement(DeserializationContext.java:1087)
at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source) at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source) at org.apache.xerces.parsers.XML11Conf...
[EDIT]
Here is a bit more info as I am working on another webservice that is similar.
Your [SOAP] xml with the cannot possibly be working. I use soapUI as well and unless I add in a security header values in their proper spot the service will return exactly the response you are receiving ("InvalidSecurity") so not very sure this xml is what you should be trying to replicate. I have also successfully built a simple cf based script version to build the header which properly generates the header and adds it to a createObject() invoked javastubbed webservice:
doc = xmlNew();
doc['Security'] = XmlElemNew(doc,'http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd', 'Security');
doc.Security['UsernameToken'] = XmlElemNew(doc, 'UsernameToken');
doc.Security.UsernameToken['Username'] = XmlElemNew(doc, 'Username');
doc.Security.UsernameToken.username.XmlText = "TESTERDUDE" ;
doc.Security.UsernameToken.username.XmlAttributes["xsi:type"] = "xsd:string";
doc.Security.UsernameToken['Password'] = XmlElemNew(doc, 'Password');
doc.Security.UsernameToken.password.XmlText = "YetAnotherPassword";
doc.Security.UsernameToken.password.XmlAttributes["xsi:type"] = "xsd:string";
doc.Security.UsernameToken.password.XmlAttributes["Type"] = "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0##PasswordDigest";
doc.Security.UsernameToken['Nonce'] = XmlElemNew(doc, 'Nonce');
doc.Security.UsernameToken.nonce.XmlText = "tKUH8ab3Rokm4t6IAlgcdg9yaEw="; // This would be generated if needed
doc.Security.UsernameToken.nonce.XmlAttributes["xsi:type"] = "xsd:string";
doc.Security.UsernameToken['Created'] = XmlElemNew(doc, "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd", 'Created');
doc.Security.UsernameToken.created.XmlText = "2010-08-10T10:52:42Z";
doc.Security.UsernameToken.created.XmlAttributes["xsi:type"] = "xsd:string";
addSOAPRequestHeader(ws, "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd", "Security", doc);
This properly generates [taken from the server side xml using getSoapRequest() which grabs the entire xml request] :
<soapenv:Header>
<Security soapenv:actor="" soapenv:mustUnderstand="0" xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:ns1="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
<UsernameToken>
<Username xsi:type="xsd:string">TESTERDUDE</Username>
<Password xsi:type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordDigest">My Password is my Voice</Password>
<Nonce xsi:type="xsd:string">tKUH8ab3Rokm4t6IAlgcdg9yaEw=</Nonce>
<Created xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns:ns2="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xsi:type="xsd:string">2010-08-10T10:52:42Z</Created>
</UsernameToken>
</Security>
</soapenv:Header>