403 Forbidden for SharePoint version APIs BUT they work at design time AND all other REST calls work - rest

I have a strange problem whereby any REST APIs relating to SharePoint versions work when testing at design time but generate a 403 Forbidden error at runtime. What is also odd is that all other REST API calls work fine at both design time and runtime and all the parameters are identical to the ones that don't work (headers etc.) and I've done a cut and paste on everything, but still anything relating to versions isn't working, although that might just be a red herring?
The app can successfully delete files and overwrite them, check in and out, etc. so seemingly there are no permission issues. I have also tried checking a file out before reading the version information just in case (as check out is forced before any actions can be carried out on this site) but that didn't work either.
This is an example of one of the calls that is causing the error:
https://mycompany.SharePoint.com/sites/{SiteName}/_api/web/GetFileByServerRelativeUrl('/sites/{SiteName}/Shared%20Documents/{FilenameAndPath}')/version

Look at this post:
https://sharepoint/_api/web/folders/getbyurl('Documents')/files/getbyurl('myfile.docx')/versions?$filter=VersionLabel eq '2.0'

Check the Accept and Content-Type headers in the OnBeforeRequest REST API callback.
I have found that when testing the IDE will send for instance "application/json" for both, but at runtime, the platform adds ";utf-8" to the values of these headers. The requests are then often rejected without a proper error specification/declaration.

Related

Sophos Intelix Dynamic Analysis - job disappears (404) before it returns a report

I'm experiencing an issue using Sophos Labs Intelix Dynamic file analysis: https://api.labs.sophos.com/doc/analysis/file/dynamic.html
For some files submitted, the API returns 202 for several minutes, then returns 404, as if the job is suddenly lost. There is no explanation or report provided. I did not experience this issue until this morning when suddenly it happened for 7 different files, although we have not made any code changes.
Why is this happening and what are we supposed to do to handle it?
Some example dynamic analysis jobIDs which now return 404: 10c37c488e86b6871c3f6224ff5273e1, bc900e7bfb8bb25e1f8f127dc25ab8fe, 48cda26c2e3cb54d0b3f4b79ac1ddb7e
Also, is this the correct forum for support requests? I am also trying to go through https://support.sophos.com/support/s/sophos-technical-support but they don't seem to have any knowledge or understanding about Intelix API. My credentials for the API don't appear to be linked to any kind of support portal?
Thanks!

JAX-WS Metro, how to intercept correct encrypted/signed message with invalid characters / signature mismatch

My question is quite related to this one
I have spend weeks of headaches to try and fight it, but there doesn't seem to exist a solution worthy of mention, apart from the solution to the above question, which is a terrible workaround, but there really seem to exist nothing else around.
We are trying to communicate with a legacy system that has an established and running web service, with certain WS-Security constraints declared in its WSDL. We cannot change anything on the server, we just have to do as it bids. We also have a third party client implementation that actually works and communicates with the server, so we know that the communication works - using THAT specific client. Now, we want to make our own.
The above WS-Security policy includes encryption and signing. There were following scenarios of what to do:
write our own code to encrypt/decrypt and sign/verify
use one of the ready JAX-WS implementations to do the above for us
The second option of course is what we tried to do. Then we branch into following:
Metro/WSIT
Apache CXF
Everybody on the web suggests the latter option (which I tried too) - but for the time being I went with the first one (especially since we do not have any integration with Spring to take advantage of CXF's good integration with it)
After struggling with a bit of ambiguous documentation and various wizards (NetBeans), we came to a solution that contained very little custom code, a configuration file with some keystores, and the usual generated code from wsimport utility.
Some time passed, it included dumping the XML SOAP requests and responses, comparing the failing ones that we produce to the successful ones from the 3rd-party client. Lots of pain, with no results - the messages were different variously, but the core logic and structure was okay - then - you couldn't actually compare the encrypted parts. After some time I ended up with a client that sent something, and actually received something back, but failed to decrypt the response.
Actually it was decrypted alright, but the signature digest verification was failing. It is worth to mention that the original XML message contained a "&" character, as well as multiple newlines. I.e. the payload of the SOAP message was not syntactically correct XML. Anyway.
It seems that this digest verification is deeply rooted inside Metro/WSIT stack and there was absolutely no way I could find to actually intercept and correct that digest - or actually the contents upon which this digest was calculated - obviously - the problem was that some special characters were translated or canonicalized either after or before the digest calculation, and we (rather the underlying implementation that I tried to use to keep my hands clean) did something different from what the server side of the web service did.
Even the Metro tubes (nice name, but horrendously scarce documentation - it seems that nobody uses Metro/WSIT these days - or, should I say, nobody uses SOAP, or SOAP with this level of security? - when I tried Apache CXF, the generated SOAP messages were deceptively similar) and their way of intercepting messages didn't seem to help - when trying to get the raw contents of the message, no provided methods (Packet.getMessage().writeTo... - and other variations) could actually bypass the digest verification thing - because they ALL tried to read the contents the StAX way, streaming etc. (invoking StreamingPayLoadDigester.accept that invariably failed)
But hope would die last, and I would try again and again to find some obscure undocumented magic to make my thing work. Okay, i was about to call it a day and dig hard into java encryption - until I found the above question, that is. Actually it "exploits" a log message that gets printed from deep within the Metro code (actually from wssx-impl I think) with the canonicalized decrypted message, before throwing the digest mismatch exception. Thankfully, this message gets printed using java.util.logging, and this can be intercepted in various ways - e.g. to send it in some kind of synchronized queue, to be consumed by my client. Ugh. If somebody has a better idea, please write your thoughts.
Thank you all.
Finally I resorted to rebuilding Metro/WSIT version 2.1.1 found on GitHub, commenting a single line in WS-SX Implementation project (ws-sx\wssx-impl...\StreamingPayloadDigester.java:145)
if (!Arrays.equals(originalDigest, calculatedDigest)) {
XMLSignatureException xe = new XMLSignatureException(LogStringsMessages.WSS_1717_ERROR_PAYLOAD_VERIFICATION());
logger.log(Level.WARNING, LogStringsMessages.WSS_1717_ERROR_PAYLOAD_VERIFICATION()); //,xe);
// bypass throwing exception
// throw new WebServiceException(xe);
}
It could have been done in a better way, introducing a flag, for instance.
The order of the projects, starting from the smallest one where I did the change, to the one I include into my own project as Metro implementation is approximately as follows:
WS-SX Implementation is referenced in ->
WS-Security Project is referenced in ->
Metro Web Services Interoperability Technology Implementation Bundle (wsit-impl) is referenced in ->
Metro Web Serrvices Runtime non-OSGi Bundle (webservices-rt) included in my client

Using (Re)Captcha without any server code?

Is it posible to use the recaptcha API without any server side code? I keep getting a CORS error. Then I found this saying Recaptcha doesn't allow requests directly from browsers.
I also tried creating a php proxy on my server (although defeating the purpose of this question) but keep getting an error that the challenge field is not valid even though I'm sending the correct value.
I know that client-only captchas are inherently unsafe, but this is a low-consequences app and I need to put something in there.

Blackberry ksoap2 request issues

First time posting a question. I'm trying to call some SOAP webservices from inside a blackberry app using the ksoap2 library. I've successfully managed to get a response from the one service, which uses an HTTP url, but now that I'm trying to get response from a (different) HTTPS url, I've run up against a brick wall.
The response dump I'm getting has the following fault message:
"An error occurred while routing the message for element value : (country option I specified in my request). Keep-Alive and Close may not be set using this property. Parameter name: value."
The weird thing is that using Oxygen XML's SOAP tools with the XML request dump works just fine. Any ideas where to start looking? This has taken up a full day already.
Update:
Responding to your comment below - it turns out the double quoting is part of the SOAP spec. Some servers are more relaxed in their implementation, and will work without the quotes.
ksoap2 doesn't force the quotes onto your actions - you may want to patch your ksoap2 library to ensure the quotes are always there.
ymmv
Original:
I don't think this is a SOAP related problem, nor with BlackBerry.
I think the problem lies on the server side, since that error string is not a common error (just google it to see no hits on the whole internet other than this question).
Looks like this is a job for the network guy on the server side to tell you what he's seeing on his end.
Only other thing I can think of is to make the call using HTTP instead of HTTPS. You can then use some network sniffer to see what the difference between the messages is. Alternatively, install an SSL proxy with something like "Charles" and sniff the packets like that.

REST Zend Framework, How to routing module based versioning & Api Key

I am building a RESTful API services with ZF 1.10.8 as am newbie its a little bit confusing when dealing with ZF routing.
I need to have versioning, api_key, and response format in url, something like:
/:version/:response_format/:api_key/:controller ...
/1.0/json/1234567890/articles/
The version is module based with the latest version as default
How to get this done?
Versioning is really not as simple as putting /v1/ in the URI.
In fact, that makes the API non-REST.
To do REST properly, every resource (thing the client wants to access) has one and only one URI.
The URI stays the same for v1 & v2 & v2; what changes is how you present that resource to the client.
How do you know which version they want? They set it as a request header.
How do you know which format (json,xml,html,wml,etc) they want it in? They set it as a request header.
How do you know which language they want it in? Request header.
The thing to remember is that the URI they are requesting stays the same.
Because each resource only has 1 URI, you never want a method name in the URI.
This is bad:
- /edit/place/43
Instead, you should use the proper HTTP methods
- to create a place, do an HTTP POST to /place
- to view place 43, do an HTTP GET to /place/43
- to update place 43, do an HTTP PUT to /place/43
- to delete place 43, do an HTTP DELETE to /place/43
When returning the response to the client, you should also include the URIs of all related bits of data the client might want to retrieve next. One of the principles of REST is that once the client has connected, it can find all the URIs it needs within the API itself. It only needs to know one URI to get into the system, and from that point on, all required URIs are provided in responses. This has the benefit of allowing you to change your URIs at will, since the client should never be paying attention to what they are... just using them as needed (i.e. the client knows what the URI points to, but not where it points).
Lastly, keep in mind that you don't want to be sending success/error markers as xml or json. They should be sent back as HTTP response codes. There's a code for creation, and one for deletion, and one for updating, etc.
Here are some fantastic articles on REST in general, and doing REST with the Zend Framework in particular:
http://blog.steveklabnik.com/2011/07/03/nobody-understands-rest-or-http.html
http://timelessrepo.com/haters-gonna-hateoas
http://martinfowler.com/articles/richardsonMaturityModel.html
http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven
http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm#sec_5_2_1_1
http://www.techchorus.net/create-restful-applications-using-zend-framework
http://www.techchorus.net/create-restful-applications-using-zend-framework-part-ii-using-http-response-code
http://weierophinney.net/matthew/archives/233-Responding-to-Different-Content-Types-in-RESTful-ZF-Apps.html
http://www.enrise.com/2010/12/rest-style-context-switching/
http://www.enrise.com/2011/01/rest-style-context-switching-part-2/
http://www.informit.com/articles/article.aspx?p=1566460
http://www.chrisdanielson.com/tag/zend_rest_controller/
http://barelyenough.org/blog/2008/05/versioning-rest-web-services/
I particularly recommend the article at weierophinney.net, for implementation details.
This is just an idea, but I would avoid making the code know anything at all about the version. (Other than what its current version number is.) Instead, I would make the /:version/ part of your URI the base in your rewrite scheme.
So instead of the base being something like: "http://www.example.com/"
It would be: "http://www.example.com/1.0/"
In this way you can simply have different branches of your source control on the server separately and your web server can determine which version to route the URI to. Then your code doesn't need any knowledge of how to handle different versions and your code base doesn't get polluted with large switch statements to do different things based on version.
To make it a little safer, you can require requests to contain the version number in the header. Then your code can just check if the version number in the header matches the version number of the code it's being routed to and throw an error if they don't match.
For example: Sending a GET to http://www.example.com/2.0/ with a version number in the header of 1.0 would throw a "wrong version" error. Your code would only need to know that header_version != current_version, so it shouldn't need to change as you release new versions.