We have got issues in our AEM application for cross site scripting. We decided to check for any scripts before submitting a request. How do we check if there is any script available in the SOAP request at the server side(Java). Is this the correct solution for avoiding cross site scripting issue?
This is a pretty broad question, and we can't provide any implementation details since we don't know any of your architecture or implementation details. However, there are some general XSS things to keep in mind:
If you are "checking for scripts" only in the browser, using JS, before submitting a form that will not solve anything. People can easily bypass this by simply issuing the HTTP request that the form would have made from any other tool (e.g. curl, PostMan, etc.). You need to check for bad data on the server side while processing the request that the Form is submitting.
As far as how to do this sort of thing on the CQ server side: Adobe has some recommendation that you should read through:
AEM 6.1
AEM 5.6
The PDF "cheat sheet" link on those pages will probably be most helpful.
There are different ways to mitigate the XSS risk. White-listing the data to let only known good data through, black-listing the data to block out any known bad data, encoding the data to prevent scripts from being treated as HTML. For an excellent read on what to do pay attention to the OWASP recommendations
Check out XSSAPI , you can use methods in this api to prevent XSS security risks.
On the other hand, you could probably start using sightly which provides automatic contextual XSS protection.
Related
We are implementing application security in our website. Its a REST based application, so i will have to validate the whole request payload, rather than each attribute. This payload need to be validated against all type of attacks (SQL,XSS etc). While browsing i found people are using ESAPI for web security.
I am confused between ESAPI.validator().getValidXXX, ESAPI.encoder() Java API's of ESAPI library. What is the difference between these two and when to use which API. I would also like to know in what cases we might use both API's
As per my understanding i could encode an input to form a valid html using both API's
Eg:
ESAPI.encoder().encodeForHTML(input);
ESAPI.validator().getValidSafeHTML(context, input, maxLength, allowNull).
For XSS attacks, I have made code changes to strip-of html tags using java pettern&matcher, but i would like to achieve the same using ESAPI. Can someone help me how to achieve it.
Or
Are there any new java plugins developed for websecurity similar to ESAPI which i did not come accross. I have found https://jsoup.org/, but it solves only XSS attacks, i am looking for a library which provides API's for several attacks (SQL injection/XSS)
ESAPI.encoder().encodeForHTML(input);
You use this when you're sending input to a browser, so that the data you're sending gets escaped for HTML. This can get tricky, because you have to know if that exact data is for example, being passed to javascript before it is being rendered into HTML. Or if it's being used as part of an HTML attribute.
We use:
ESAPI.validator().getValidSafeHTML(context, input, maxLength, allowNull).
when we want to get "safe" HTML from a client, that is backed by an antisamy policy file that describes exactly what kinds of HTML tags and HTML attributes we will accept from the user. The default is deny, so you have to explicitly tell policy file, if you will accept:
text
You need to specify that you want the "a" tag, and that you will allow an "href" attribute, and you can even specify further rules against the content within the text fields and tag attributes.
You only need "getValidSafeHTML" if your application needs to accept HTML content from the user... which is usually specious in most corporate applications. (Myspace used to allow this, and the result was the Samy worm.)
Generally, you use the validator API when content is coming into your application, and the encoder API when you direct content back to a user or a backend interpreter. AntiSamy isn't supported anymore, so if you need a "safe HTML" solution, use OWASP's HTML Sanitizer.
Are there any new java plugins developed for websecurity similar to
ESAPI which i did not come accross. I have found https://jsoup.org/,
but it solves only XSS attacks, i am looking for a library which
provides API's for several attacks (SQL injection/XSS)
The only other one that attempts a similar amount of security is HDIV. Here is an answer that compares HDIV to ESAPI by an HDIV developer.
*DISCLAIMER: I am an ESAPI developer, and OWASP member.
Sidenote: I discourage the use of Jsoup, because by default it mutates incoming data, constructing "best guess" (invalid) parse trees, and doesn't allow you fine-grained control of that behavior... meaning, if there's an instance where you want to override and mandate a particular kind of policy, Jsoup asserts that it is always smarter than you are... and that's simply not the case.
I'm finding it hard to determine best practice for detecting duplicate form submissions. I'm using the latest SpringBoot, Thymeleaf and Spring-Security and the out-of the-box CSRF functionality all appears to be working.
The design of the application is such that submit buttons get disabled via JavaScript onclick, successful POSTs result in a redirect (POST->Redirect->Get pattern) and I had (seemingly wrongly) thought that the CSRF protection would provide the server-side protection for anything that slipped through the JavaScript.
For some reason my dodgy Logitech G500 mouse (which has started double-clicking everything) has managed to highlight a problem with the application. Somehow it has defeated the JavaScript and it has revealed that there is no protection on the server for duplicate form submissions - i.e. the form got processed twice. I'll have a look into the JavaScript later, but I don't want to rely upon this to protect the server so I want to be able to detect it at the server.
Given how much Spring does (including the CSRF protection) I was somewhat surprised and have done a lot of Googling. From what I can tell, there used to be something in the old Spring framework (references to AbstractFormController.handleInvalidSubmit) but that no longer exists now. I've also seen references to RequestMappingHandlerAdapter and settings such as synchronizeSession and sessionForm, but I don't really understand them yet. There are also a load of custom solutions that people have produced, including a HandlerInterceptorAdapter with associated tag library and a cache that performs some custom processing.
So my questions are:
Why doesn't the CSRF protection prevent this?
What sort of support is built in to detect and handle duplicate form submission?
If a custom solution is necessary, do you have any advice for best practice? In particular, the second click will get rejected
and if I display an error page the user might never see the handling
of the first click and thus not realise it was actually processed
directly.
I have read this: Duplicate form submission in Spring , including the Synchronizer piece from 2009 but of course it's quite old and some of those things are no longer valid.
Thanks
Marcus
The problem
We are building an SOA with a RESTful approach to the services. Once the systems are in production we will have many clients consuming the interface including internal and 3rd party systems.
We would like to be able to consume and echo in the response information provided by the client application such as: -
Session Id - Could be a Java EE session id or anything client specific, this is useful for the support team and debugging client issues to trace them through all our systems.
Transaction Id - A unique identifier for the request that we can echo back to the client to aid the client in request/response correlation if they invoke the service asynchronously or if we implement an 202 Accepted style long running process.
Potential solutions
So sticking to the RESTful constraints would suggest we need to utilise HTTP to implement this and there are several options we could implement.
Pragma header - implement extension-pragmas for transaction-id, session-id, etc. This seems like purist of solutions as it utilises a standard HTTP header although I would be concerned it became a dumping ground for everything we can't be bothered to think about properly.
X-My-Header - custom headers for each field we require. May be stripped by proxies, not core HTTP so feels anti-rest
In query string or XML/JSON representations - Add the fields to all our resources. Because it's an operational parameter it feels like it should be provided as metadata rather than on a resource.
Cookies - use Cookie and Set-Cookie to hold custom key-values; useful in the case of session ids as most implementations use cookies already. Would have to re-send every time to support client side correlation which kind of defeats the point of using a cookie.
The answer
Is there any precedent for this? Are we mad? Is there something obvious I am missing in all my research? Does no-one actually care how they will support their services once they are deployed? Should I just shut up and go away?
I'm hoping someone can help.
P.S. sorry if this is a bit of an essay, the advice did say "be specific"....
Oh, this is a pain. I've been there too.
Well, the idea with metadata for transactions, sessions etc. is a good idea. For logging, at least.
The problem is to setup something that is compliant with various corporations policies and SOA infrastructure.
There is a tradeof between best design and maximum interopability in the case of HTTP.
The safe path is to encode the metadata in the message itself. Not very nice, and such a solution ends up looking a bit like SOAP where you have an envelope with headers for all messages.
I ended up using an X-header for information such as transaction id. However, as you mentioned, proxies/b2b-gateways etc. might strip headers, it's not obvious that you can retreive them with all appointed development frameworks, COTS applications etc. So if you do like this, you should avoid make the metadata mandatory to get a solution running - just "nice to have".
Cookies are nothing but pain. They might be annoying or sometimes even useful with browser interaction, but in a SOA scenario, it will be bad idea. Many things can go wrong and it's a pain to debug cross organisations.
I would also avoid using query strings along with POST or PUT data. It's possible according to the HTTP specs. but not when it comes to implementation in random framework.
You can use a GUID and let the client generate it and pass it as part of any request that starts the workflow/business process. This GUID can be used to correlate across multiple components participating in the workflow.
For a SaaS startup I'm involved in, I am building both a RESTful web API and a couple of client apps on different platforms that consume it. I think I've got the API figured out, but now I'm turning to the clients. As I've been reading about REST, I see that a key part of REST is discovery, but there seems to be a lot of debate between two different interpretations of what discovery really means:
Developer discovery: The developer hard-codes copious amounts of API details into the client, such as resource URI's, query parameters, supported HTTP methods, and other details that they've discovered through browsing the docs and experimenting with the API's responses. This type of discovery IMHO necessitates cool linkage and the API versioning question, and leads to hard coupling of the client code to the API. Not much better than if using a well-documented collection of RPC's it seems.
Runtime discovery - The client app itself is able to figure out everything it needs with little or no out-of-band information (presumably, only a knowledge of the media types the API deals with.) Links can be hot. But to make the API very efficient, a lot of link templating for query parameters seems to be needed, which makes out-of-band info creep back in. There are possibly other difficulties I haven't thought of yet since I haven't gotten to that point in development. But I do like the idea of loose coupling.
Runtime discovery seems to be the holy grail of REST, but I'm seeing precious little discussion about how to implement such a client. Almost all REST sources I've found seem to assume Developer discovery. Anyone know of some Runtime discovery resources? Best practices? Examples or libraries with real code? I'm working in PHP (Zend Framework) for one client. Objective-C (iOS) for the other.
Is Runtime discovery a realistic goal, given the present set of tools and knowledge in the developer community? I can write my client to treat all of the URI's in an opaque manner, but how to do this most efficiently is a question, especially over low-bandwidth connections. Anyway, URI's are only part of the equation. What about link templating in the Runtime context? How about communicating what methods are supported, aside from making a lot of OPTIONS requests?
This is definitely a tough nut to crack. At Google, we've implemented our Discovery Service that all our new APIs are built against. The TL;DR version is we generate a JSON Schema-like spec that our clients can parse - many of them dynamically.
That results means easier SDK upgrades for the developer and easy/better maintenance for us.
By no means the perfect solution, but many of our devs seem to like.
See link for more details (and make sure to watch the vid.)
Fascinating. What you are describing is basically the HATEOAS principle. What is HATEOAS you ask? Read this: http://en.wikipedia.org/wiki/HATEOAS
In layman's terms, HATEOAS means link following. This approach decouples your client from specific URL's and gives you the flexibility to change your API without breaking anyone.
You did your home work and you got to the heart of it: runtime discovery is holy grail. Don't chase it.
UDDI tells a poignant story of runtime discovery: http://en.wikipedia.org/wiki/Universal_Description_Discovery_and_Integration
One of the requirements that should be satisfied before you can call an API 'RESTful' is that it should be possible to write a generic client application on top of that API. With the generic client, a user should be able to access all the API's functionality. A generic client is a client application that does not assume that any resource has a specific structure beyond the structure that is defined by the media type. For example, a web browser is a generic client that knows how to interpret HTML, including HTML forms etc.
Now, suppose we have a HTTP/JSON API for a web shop and we want to build a HTML/CSS/JavaScript client that gives our customers an excellent user experience. Would it be a realistic option to let that client be a generic client application? No. We want to provide a specific look-and-feel for every specific data element and every specific application state. We don't want to include all knowledge about these presentation-specifics in the API, on the contrary, the client should define the look and feel and the API should only carry the data. This implies that the client has hard-coded coupling of specific resource elements to specific layouts and user interactions.
Is this the end of HATEOAS and thus the end of REST? Yes and no.
Yes, because if we hard-code knowledge about the API into the client, we loose the benefit of HATEOAS: server-side changes may break the client.
No, for two reasons:
Being "RESTful" is a property of the API, not of the client. As long as it is possible, in theory, to build a generic client that offers all capabilities of the API, the API can be called RESTful. The fact that clients don't obey the rules, is not the API's fault. The fact that a generic client would have a lousy user experience is not an issue. Why is it important to know that it is possible to have a generic client, if we don't actually have that generic client? This brings me to the second reason:
A RESTful API offers clients the option to choose how generic they want to be, i.e. how resilient to server-side changes they want to be. Clients which need to provide a great user experience may still be resilient to URI changes, to changes in default values and more. Clients doing batch jobs without user interaction may be resilient to other kinds of changes.
If you are interested in practical examples, checkout my JAREST paper. The last section is about HATEOAS. You will see that with JAREST, even highly interactive and visually attractive clients can be quite resilient to server-side changes, though not 100%.
I think the important point about HATEOAS is not that it is some holy grail client-side, but that it isolates the client from URI changes - it is assumed you are using known (or developer discovered custom) Link Relations that will allow the system to know which link for an object is the editable form. The important point is to use a media type that is hypermedia aware (e.g. HTML, XHTML, etc).
You write:
To make the API very efficient, a lot of link templating for query parameters seems to be needed, which makes out-of-band info creep back in.
If that link template is supplied in the previous request, then there is no out-of-band information. For example a HTML search form uses link templating (/search?q=%#) to generate a URL (/search?q=hateoas), but nothing is known by the client (the web browser) other than how to use HTML forms and GET.
I am in the process of scoping the development of an iPhone app for a client. Among other things, the app will allow users to browse through and place orders on specific (tangible) products.
The client has a website that currently does a similar thing and due to their limited budget and the fact that the website runs on a third-party proprietary platform which they have no control over, we are investigating possible alternatives to building a web service.
On the website, user registration and authentication, as well as order placing is done through POST requests via secure HTTP. The response is always a formatted HTML page which will contain strings indicating whether the request was successful or not, and if there was an error, what the error is etc.
So provided I can replicate the POST requests on the phone, and parse the HTML responses to read the results of each request, do you think this is an acceptable alternative to building a web service to handle this?
Apart from the possibility of pages changing (which we can manage) and the fact that I will probably have to download and parse a relatively large HTML response, are there any other drawbacks to this solution and is there anything else that I might be missing?
Many thanks in advance for your thoughts.
Cheers,
Rog
You could create an intermediary server that will communicate with the client server, and on it expose some REST web services with json (small overhead and easy to handle) responses that will be consumed by the iPhone app.
So, you're going to parse HTML and formulate POSTs off a third-party server, and pray that they don't even so much as rename a form field.
Your question is in two parts:
Do I think that a miracle is an acceptable solution? I don't.
Do I think that aside from the fact a miracle is required, are there any other drawbacks? None that I can think of.
You didn't ask, but this is a terrible course of action. Two suggestions.
I spy an assumption that the providers of the third-party platform aren't interested in enabling third-party applications by providing an API. They have a very good business reason for this, which is that it promotes platform lock-in. Reach out to their support department and have a talk with them.
You have to sell the client on building an intermediary web service. To at least try to mitigate the damage that changes on this third-party platform can do to your app, I recommend that you build and operate a proxy that receives requests from your applications, and proxies them over to the third-party platform. You should build into this client-server protocol a means for returning "we are in maintenance mode, go away" messages to apps, for that inevitable day when the third-party server changes something that breaks your app (they swapped the billing and shipping address pages, for instance) and you have to rush through an update through Apple to deal with it.
The proxy could be written in something more flexible and easy to bash stuff out in, such as PHP, Python, Perl, or Ruby. It could be hosted at Amazon in a micro instance.
p.s. This question is inappropriately tagged objective C.
HTML is the worst because of parsing (1-2secs per page), memory, and changes, but you already know that. Check in advance that ALL the data you need is exposed on the HTML.
If you use an intermediary server you are moving work elsewhere and you have another server to maintain. I would only do that if memory is an issue. Check How To Choose The Best XML Parser for Your iPhone Project for memory/performance/xpath support. libxml2 is a good option, but it depends on your needs. And maybe you'll want to check ASIHTTPRequest features before using the SDK.
I think utilising the web language of JSON would contribute to the diminishing of the parsing time. By building a REST service that, when sent a GET request, returns the correct information for easy sorting, you could then display the output a lot faster than that of parsing straight HTML.
I prefer JSON over XML, but everyone has their personal preference. You should look at a few very good libraries that are built specifically for parsing purposes of both XML and JSON.
For XML I recommend using the inbuilt libxml parser. Albeit, this can sometimes deem very difficult to use. A simple Google search will bring up a heap of results that relate specifically to what parser should be used depending on what task is to be completed.
As for a JSON parser, I recommend SBJSON. I am currently using it one of the biggest projects I have undertaken and it is definitely working perfectly for my use.
If you need a good way to connect to a RESTful web service, you should try LRResty.
Don't go for a parsing solution on the iPhone for 4 reasons:
Server can change their design and break your application (AppStore submition is long) + They can also detect that the request are sent from an application based on user agent which you have to update the application to change it.
Some of the requests might be made thru Javascript so you not only have to parse (X)HTML but also Javascript request (which can be in the form of XMLHttpRequest, but don't have to)
Long term evolution of the mobile market : maybe your client want (or will want) an application for android, Blackberry, Bada OS (Samsung), Symbian (Nokia/ OVIStore), Java Mobile or Windows Phone 7?
Of course network traffic, Memory and CPU needed to parse HTML (look the time it takes to the browser to do it?)
Regarding the traffic, if the application will not have a huge traffic you can home-host your proxy. Or you can find some provider to host it for you. I guess you won't need more than a couple of Megabytes of storage but maybe traffic. For less than 100€/year you can find some with unlimited traffic (like OVH Pro plan or Infomaniak). But if you want to go Java have a look at Google App Engine : you pay only if your traffic is important and if your application generate many CPU Cycles. If not : you don't have to pay. And it's hosted on Google server : reliable.
If the client is open, you could consider the paypal API.