In the classic "RESTful Web Services" book (O'Reilly, ISBN 978-0-596-52926-0) it says on page 251 "Some firewalls block HTTP PUT and DELETE but not POST."
Is this still true?
If it's true I have to allow overloaded POST to substitute for DELETE.
Firewalls blocking HTTP PUT/DELETE are typically blocking incoming connections (to servers behind the firewall). Assuming you have controls over the firewall protecting your application, you shouldn't need to worry about it.
Also, firewalls can only block PUT/DELETE if they are performing deep inspection on the network traffic. Encryption will prevent firewalls from analyzing the URL, so if you're using HTTPS (you are protecting your data with SSL, right?) clients accessing your web service will be able to use any of the standard four HTTP verbs.
Some 7 layer firewalls could analyze traffic to this degree. But I'm not sure how many places would configure them as such. You might check on serverfault.com to see how popular such a config might be (you could also always check with your IT staff)
I would not worry about overloading a POST to support a DELETE request.
HTML 4.0 and XHTML 1.0 only support GET and POST requests (via ) so it is commonplace to tunnel a PUT/DELETE via a hidden form field which is read by the server and dispathced appropriately. This technique preserves compatibility across browsers and allows you to ignore any firewall issues.
Ruby On Rails and .NET both handle RESTful requests in this fashion.
As an aside GET, POST, PUT & DELETE requests are fully supported through the XMLHttpRequest request object at present. XHTML 2.0 's officially supports GET, POST, PUT & DELETE as well.
You can configure a firewall to whatever you want ( at least in theory ) so don't be surprised if some sys admins do block HTTP PUT/DELETE.
The danger of HTTP PUT/DELETE is concerning some mis-configured servers: PUT replaces documents (and DELETE deletes them ;-) on the target server. So some sys admins decide up right to block PUT in case a crack is opened somewhere.
Of course we are talking about Firewalls acting at "layer 7" and not just at the IP layer ;-)
Related
The scenario being when many different clients are requesting the same resource from a reverse proxy server, is the reverse proxy forwarding the client request (to the server with the resource) for each client request? If so is that by default or does it depend on the configuration (the request headers: etags, if-none-match, .ect) between the proxy and server with the resource. Thanks
This question might be more appropriate over in ServerFault. That being said, it highly depends on the reverse proxy setup. It's likely both the reverse-proxy software being used (there are many, many types, from nginx, to custom c++, to even Apache) as well as the configuration set for each. If you're trying to write a reverse proxy, syndicating requests to the resource is a tempting, but dangerous idea. It works very well for public, static pages, but you have to be careful not to open just one connection to, for example, facebook.com, and send the result to two different people (showing one of them the other's personal data). For that reason, it should only be attempted when the resource marks, through appropriate headers that it is safe to cache.
This is my scenario:
So,
Requests via encrypted HTTPS go to Apache like: https://server1/MyPerlApp
If the user is not logged in, they get a redirect to some login page (in the server1), and Apache doesn't proxy the request to Server2
When the user logged in - IS authenticated - then Apache forwards all requests that are coming to https://server1/MyPerlApp to http://server2:5000
Question1: Is this possible? (Asking, because I don't know Apache enough deeply, and this is not an simple:
ProxyPass /MyPerlApp http://server2:5000/
because I need need authenticate the user at server1 and set ProxyPass Only if authenticated.
Since Apache is quite flexible I assume the answer is yes for the above (but confirmation and details is very welcomed) - so here are my main specific questions:
How will my Plack application know what user is authenticated at the Apache level (i.e. on the 1st server)?
what is an easy way to deliver some of the user info to the perl app on the server2? e.g. with Apache's mod_rewrite what appends an user=username parameter to each query,
can Apache can set some HTTP headers that my perl app should read?
is there an easy and recommenced way?
I'm looking for how to avoid authentication routines in my starman/perl app, maily because:
the user need to log into server1 anyway (for other tasks in his workflow)
if he is already logged in, authentication in my app is not needed (avoid unnecessary double login)
but I still need to know which users are logged in (via Apache at server1)
There is already similar questions, but:
https://stackoverflow.com/q/12561830/734304 (no answer)
https://stackoverflow.com/q/11907797/734304 (no answer)
Apache reverse proxy with basic authentication (similar, but the backend is in the same server and same apache)
[I think you asked four questions here. Some of them overlap. I will try to answer as many as I can, then edit your question to make it a bit clearer. It might be helpful to post your current Apache httpd.conf so people can see how you are handling access and authentication currently. That way you might get better suggestions on how to integrate the proxied application(s) with your Apache instance.]
Setting up a front-end that can handle "Web Site Single Sign On" requires some planning and configuration but it is worth the effort. To make this easier, you'll want to use Apache-2.4. You probably are using this version, but Apache has become something of a workhorse, such that some sites update it much less frequently than in the past. Apache 2.4 includes mod_session and mod_auth_form which make it possible to set up form-based "web portal Single Sign On" sorts of tools with Apache for sites with multiple back-end application servers (often running on separate machine ports or sockets) combined under one outward facing set of URL/URIs. This pattern of use was so widespread with Apache that the 2.4 release added features to make it easier to do.
You asked about an "easy recommended" way to do what you have described. Well, you are on the right track. Apache's httpd is really useful for this kind of authentication/authorization and "user login" sort of application - so much so that it's become a staple tool for what you are trying to do.
You asked how to "deliver the user information" to the back-end server. You do that in the same way you handle state in any web application: with sessions and cookies. Session information contains key/value pairs encoded as an application/x-www-form-urlencodedstring. You can also create an HTTP_SESSION environment value that you back-end application can read from. Your Plack/Starman application has to be able to handle sessions and cookies (i.e. it has to be "session aware") if you want to use them there of course. Look at Plack::Middleware::Session for ideas on how to approach this.
For sure, setting up authentication with mod_auth_form is more complicated than Basic authentication. But with form based logins javascript can be used (judiciously), client applications can store form information locally for quick logins; as well, forms are flexible and can gather more data and pass more information to the user and some of the complexity (redirection after authentication) can be handled by Apache. Since they are just an HTML <form>, you can start simply and make them more elaborate as your site grows. That said you can have an Apache Reverse Proxy simply provide Basic Auth for your back-end.
Without seeing more details about your installation I can't say how/why you might need mod_rewrite per se, but Rewrite directives can play nicely with ProxyPass. Of course throughout your site you'd want to check for authentication and session information and redirect users to a login form where/when necessary. Using mod_auth_form makes this easier to implement at the cost of a somewhat more complicated configuration. As for the reverse prosy itself, you'd use ProxyPass in the normal way to pass requests to your back end:
ProxyPass /app http://[starmanhost]:3000/
Then you need configure or tweak your current Apache system to have Session On and require authentication for the URLs in question (unless the entire / requires authentication) in the standard Apache way:
<Location /app>
AuthType Basic
Session On
SessionCookieName session path=/
...
require valid-user
</Location>
etc. As the Apache docs point out (and you'll want to read mod_session, mod_proxy among others), you can pass session information around for use by back-end applications.
If the SessionHeader directive is used to define an HTTP request
header, the session, encoded as a application/x-www-form-urlencoded
string, will be made available to the application.
From mod_session documentation Integrating Sessions with External Applications.
For privacy/security you'll want to use mod_session_crypto and SSL if that's possible. As you note you will not need encryption to be "end to end" (i.e. HTTPS from client to outward facing front-end and between the reverse proxy and back-end applications) but if outside connections are https:// and you keep session information on the server (using mod_session_dbd as another response noted) using encrypted storage, you can avoid obvious threats inherent in sharing user session information across servers. The best part of this is you can add these layers one by one without having to modify your back-end applications extensively. This is the advantage of creating a solid "WebSSO server" front-end to handle logins.
Note that I've been using the term WebSSO here a bit loosely. Strictly speaking, WebSSO (and SSO) are much broader and more encompassing concepts with their own standards tracks and technologies (there are a couple Apache projects focused on this). This is why I tend to call the approach you are trying "Web Site SSO". Support for a wide range of authentication, programming language modules, proxying, and rewriting makes Apache's httpd the "swiss army knife/duct tape" of choice for handling logins and sessions in this way.
Your rational for doing this is sound, since you can avoid extra logins and confusing users (and their browsers). As well, by decoupling the authentication steps from your application and dedicating that task to Apache, you make it easier for developers to write back-end applications. Your question is very general though. I think you can start to try out some of the suggestions that begin to appear here and if you run into problems you can follow up with more specific questions focused on your implementation.
Get the Apache bits working correctly first (Session On; ProxyPass, <Location /app>) and make sure the right information is getting created, stored and passed on by the front-end. This will be very useful for lots of things going forward. Apache gurus can help here. Once you have the proper session information being passed to your back-end you can ask questions about how to access and use it in in your perl code with starman and plack. There may be missing or rough bits in tools and documentation but lots of sites want to do what you have described so these things will appear and continue to improve. Good luck.
References
A Gentle Introduction to Plack Sessions
Deploy Catalyst Applications with Starman and Apache
Using Apache mod_auth_form
Authentication in Apache2.4 using mod_auth_form and mod_dbd
Reverse proxying behind Apache
Apache's mod_session looks to be the component you are missing. Since the proxy is the gateway to the applications in the back-end, it can handle the authentication on the HTTP layer and pass back sessions as needed to the Perl script using the proxy entry.
Exposing the user information to the Perl application can happen in a few ways.
mod_session_dbd - is a module to store session information in a database. This could then be shared with the back-end server hosting the Perl application.
mod_session_cookie - is a module to store session information in a cookie on the browser of the client. Session variables would be stored in the cookie and the Perl application would retrieve them.
But, cookies or putting session variables in the URL open up security concerns. Cookies and headers can be modified.
mod_proxy should pass the session variables back to the applications in the form html.
http://httpd.apache.org/docs/trunk/mod/mod_session.html
We want to use a web service in our app which obviously requires to call a URL. It's not HTTPS, just plain old HTTP, using NSURLConnection.
The problem is: This web service is VERY expensive and every thousand calls costs us real money. The fear is that someone could figure out which URL we call and then misuse that, letting the costs explode. There is no way for us to track if a call to that web service was legitimate.
We're calculating based on how many apps we sell, multiplied by an assumption of how often that app will be used per user in average. We have some good statistics on which we base our assumptions.
Are there known ways of figuring out which URL an app is calling on the Internet to retrieve information?
You could easily use a network sniffer while the phone is on WiFi to figure out this information. It sounds like it is actually critical that you use SSL with some sort of secure token in the URL.
If this is not an option perhaps you can provide your own proxy service that would use SSL and security tokens? Proxy also grants the ability to throttle requests and block users known to be malicious. Throttling puts an upper bound on the expense each user can incur within a given time interval. Another benefit of a proxy is that it allows one to gather statistics and measure the costs incurred by different users facilitating malicious user detection and business planning. Proxy could also save you some money if the service behind it is stateless by adding a cache that would remove a lot of expensive calls.
If the Web service is not encrypted, it would be trivial to use a proxy to intercept the Web requests made by the phone. If the expensive Web service does not offer at least some form of basic authentication, I would seriously reconsider including its URL in a public app.
Using plain URLs is a sure way of letting script kiddies run you out of business. If there is no way for you to track if a call to the expensive web service was legitimate, set up your own web service that fronts the real web service to make sure that your own web service can verify the legitimacy of the call before forwarding the request to the real web service.
Yes, there's plenty of ways to do this. For one example, hook up the iPhone to a wifi network, in which the router has a transparent proxy. Examine the proxy's logs. You'll see all URLs. Depends how determined your users are, but this is rather easy.
Ignoring the fact that people who jailbreak their devices could possibly look at your application, I believe it is possible to examine traffic like any other device (laptop, tablet, etc.) if someone was sniffing traffic over a WiFi hotspot using applications such as WireShark. However, I doubt there would be much risk of this over a cellular 3G network.
Good question.
As many have said, yes, it's easy to figure out the urls your app requests.
Note about HTTPS:
But since you are using HTTPS you are okay because over HTTPs the domain will be obscured to the IP address, and people cannot see the URL query string parameters. For example, if your URL was https://somewebsite.com?uid=mylogin&pass=mypass, they definitely won't be able to see "uid=mylogin&pass=mypass", and they probably can only see the IP address, not the domain name itself. (see https://serverfault.com/questions/186445/can-an-attacker-sniff-data-in-a-url-over-https)
Sidenote:
Might be safe to assume that Apple performs some sort of HTTP request diagnostics when they review your app -- which would make sense because it's in their best interest to try and figure out what your app does from many angles.
So I want to make my own Internet Filter. Don't worry, i'm not asking for a tutorial. I'm just wondering if Fiddler would make a good backbone for it. I'm a little worried because it seemed that there's a few things Fiddler can't always pick up - or that there are workarounds. So, my question:
Would Fiddler grab all web data? i.e, chats, emails, websites, etc.
Are there any known workarounds?
Any other reasons not to use it?
Thanks!
I think you mean FiddlerCore rather than Fiddler. Fiddler(Core) is a web proxy meaning it captures HTTP/HTTPS traffic; it won't capture traffic that uses other protocols (e.g. IRC, etc). To capture traffic from other protocols, you'll need a lower-level interception point (e.g. a Windows Firewall filter) which will capture everything, but it will not be able to decrypt HTTPS traffic, and parsing / modifying the traffic will prove MUCH harder.
I have an otherwise working iPhone program. A recent change means that it generates some very long URLs (over 4000 characters sometimes) which I know isn't a great idea and I know how to fix -- that's not what I'm asking here.
The curious thing is that when I make the connection using a 3G network (Vodafone UK) I get this HTTP "414 Request-URI Too Long" error but when I connect using my local WiFi connection it works just fine.
Why would I get different results using different types of network? Could they be routing requests to different servers depending on where the connection originates? Or is there something else at stake here?
The corollary questions relate to how common this is. Is it likely to happen whenever I use a cell network or just some networks?
I would suspect that your 3G requests are being passed through some proxy which doesn't fancy 4000 char long URLs, and returns an HTTP 414 error.
I suspect the Vodafone connection is going through a proxy and/or gateway that can't handle the extra-long URL, and that your 414 Request-URI Too Long is coming from it.
Some wireless operators - including Vodafone UK, I believe - deploy inline proxies that transparently intercept your HTTP requests for purposes of optimization. Some of these proxies are based on software like the Squid proxy cache, which can have problems with very long URLs. As a result, your requests might not even be making it to your server.
To work around this issue, you can try sending your HTTP requests to the server on a non-standard TCP port. Generally speaking, these proxies are only configured to perform HTTP processing on port 80. Thus, if you can send your traffic on a different port, it might make it through unscathed.