JAX-WS Metro, how to intercept correct encrypted/signed message with invalid characters / signature mismatch - soap

Question

My question is quite related to this one
I have spend weeks of headaches to try and fight it, but there doesn't seem to exist a solution worthy of mention, apart from the solution to the above question, which is a terrible workaround, but there really seem to exist nothing else around.
We are trying to communicate with a legacy system that has an established and running web service, with certain WS-Security constraints declared in its WSDL. We cannot change anything on the server, we just have to do as it bids. We also have a third party client implementation that actually works and communicates with the server, so we know that the communication works - using THAT specific client. Now, we want to make our own.
The above WS-Security policy includes encryption and signing. There were following scenarios of what to do:
write our own code to encrypt/decrypt and sign/verify
use one of the ready JAX-WS implementations to do the above for us
The second option of course is what we tried to do. Then we branch into following:
Metro/WSIT
Apache CXF
Everybody on the web suggests the latter option (which I tried too) - but for the time being I went with the first one (especially since we do not have any integration with Spring to take advantage of CXF's good integration with it)
After struggling with a bit of ambiguous documentation and various wizards (NetBeans), we came to a solution that contained very little custom code, a configuration file with some keystores, and the usual generated code from wsimport utility.
Some time passed, it included dumping the XML SOAP requests and responses, comparing the failing ones that we produce to the successful ones from the 3rd-party client. Lots of pain, with no results - the messages were different variously, but the core logic and structure was okay - then - you couldn't actually compare the encrypted parts. After some time I ended up with a client that sent something, and actually received something back, but failed to decrypt the response.
Actually it was decrypted alright, but the signature digest verification was failing. It is worth to mention that the original XML message contained a "&" character, as well as multiple newlines. I.e. the payload of the SOAP message was not syntactically correct XML. Anyway.
It seems that this digest verification is deeply rooted inside Metro/WSIT stack and there was absolutely no way I could find to actually intercept and correct that digest - or actually the contents upon which this digest was calculated - obviously - the problem was that some special characters were translated or canonicalized either after or before the digest calculation, and we (rather the underlying implementation that I tried to use to keep my hands clean) did something different from what the server side of the web service did.
Even the Metro tubes (nice name, but horrendously scarce documentation - it seems that nobody uses Metro/WSIT these days - or, should I say, nobody uses SOAP, or SOAP with this level of security? - when I tried Apache CXF, the generated SOAP messages were deceptively similar) and their way of intercepting messages didn't seem to help - when trying to get the raw contents of the message, no provided methods (Packet.getMessage().writeTo... - and other variations) could actually bypass the digest verification thing - because they ALL tried to read the contents the StAX way, streaming etc. (invoking StreamingPayLoadDigester.accept that invariably failed)
But hope would die last, and I would try again and again to find some obscure undocumented magic to make my thing work. Okay, i was about to call it a day and dig hard into java encryption - until I found the above question, that is. Actually it "exploits" a log message that gets printed from deep within the Metro code (actually from wssx-impl I think) with the canonicalized decrypted message, before throwing the digest mismatch exception. Thankfully, this message gets printed using java.util.logging, and this can be intercepted in various ways - e.g. to send it in some kind of synchronized queue, to be consumed by my client. Ugh. If somebody has a better idea, please write your thoughts.
Thank you all.

Answer 1

Finally I resorted to rebuilding Metro/WSIT version 2.1.1 found on GitHub, commenting a single line in WS-SX Implementation project (ws-sx\wssx-impl...\StreamingPayloadDigester.java:145)
if (!Arrays.equals(originalDigest, calculatedDigest)) {
XMLSignatureException xe = new XMLSignatureException(LogStringsMessages.WSS_1717_ERROR_PAYLOAD_VERIFICATION());
logger.log(Level.WARNING, LogStringsMessages.WSS_1717_ERROR_PAYLOAD_VERIFICATION()); //,xe);
// bypass throwing exception
// throw new WebServiceException(xe);
}
It could have been done in a better way, introducing a flag, for instance.
The order of the projects, starting from the smallest one where I did the change, to the one I include into my own project as Metro implementation is approximately as follows:
WS-SX Implementation is referenced in ->
WS-Security Project is referenced in ->
Metro Web Services Interoperability Technology Implementation Bundle (wsit-impl) is referenced in ->
Metro Web Serrvices Runtime non-OSGi Bundle (webservices-rt) included in my client