In this question, I am partially referring to this one.
I am generating an upload URI with an authenticated request on my server, using the gcloud-package for Node. This is done with the createResumableUpload-method of a file. The actual upload will be done in a browser, which will not have the same origin.
Currently, my PUT-requests are cancelled because this header is missing, while OPTIONS-requests work fine.
I found three solutions of which none work.
On number 8 of the troubleshooting list of the documentation about CORS on Google Cloud Storage, it is recommended to set the origin to * (wildcard), using the XML CORS API. While this is outdated, as the API has switched to JSON, it still won't set the header afterwards. I also dislike having to set this to a wildcard, pretty insecure.
Set CORS-option when generating the upload URI to * and the actual origin, these are both ignored
Set CORS as a query string parameter, this is also ignored.
Am I missing something here, or is this still not fixed after two years?
This is still not supported, unfortunately. Resumable uploads are logically considered to be a single operation, which is assumed to involve one remote entity. As I understand it, only the first Origin header will be respected.
You could workaround this in a couple of ways. The best way would probably be to have your server sign a URL and allow the client to start the upload themselves. Alternately, when starting the upload, you could try having your server provide the Origin header that the clients will use, keeping it consistent through the entire operation.
Thanks to Brandon Yarbrough's answer, I could fix my situation.
Turns out, there's a pretty easy solution around this. Set the origin-header using a request-interceptor, and supply it as an option to the createResumableUpload-method of a file in a bucket.
You can now finish your uploads from a browser.
Related
I run an self-hosted instance of BigBlueButton and signed up for Xirsys TURN server services because we need to serve clients behind (pretty restrictive) firewalls. Before I had been running my own instance of coturn, but as this led to problems recently, I thought I will got someone who does this for a living a try.
Now the configuration in BBB is explained here:
https://docs.bigbluebutton.org/2.2/setup-turn-server.html
Yet so far I completely failed to match the parameters I receive from Xirsys with what I have to put into the /usr/share/bbb-web/WEB-INF/classes/spring/turn-stun-servers.xml file in the place of the <turn.example.com> and <secret_value>.
Did anyone ever make this work? I did try and find a tutorial but also failed.
bbb_web, is returning this the turn uris. passwords to the html5 client, that the client is using in sip.js
so you can either get bbb-web to send valid username/passwords is same method is used, or modify the html5 client to make a Xirsys api call, to get access to the turn candidates.
Would need to look at api docs. twilio has a similar service.
regards,
Stephen
not the most elegant solution but the easiest one for me:
modify the final bbb js bundle to load the stunturn info from a fixed url in
e.g.
/usr/share/meteor/bundle/programs/web.browser/f30716b2b57e2862c4db2325 b7aac63f4622842b.js
the minified part should then look somewhat like:
const r=Meteor.settings.public.media,i='https://<yourbbburl>/html5client/stunturn.json',a=r.cacheStunTurnServers,s=r.fallbackStunServer;
and put either the static credentials or generated ones in a file stunturn.json besides the js bundle.
I know that the title is not that correct, but i don't know how to name this problem...
Currently I'm trying to design my first REST-API for a conversion-service. Therefore the user has an input file which is given to the server and gets back the converted file.
The current problem I've got is, that the converted file should be accessed with a simple GET /conversionservice/my/url. However it is not possible to upload the input file within GET-Request. A POST would be necessary (am I right?), but POST isn't cacheable.
Now my question is, what's the right way to design this? I know that it could be possible to upload the input file before to the server and then access it with my GET-Request, but those input files could be everything!
Thanks for your help :)
A POST request is actually needed for a file upload. The fact that it is not cachable should not bother the service because how could any intermediaries (the browser, the server, proxy etc) know about the content of the file. If you need cachability, you would have to implement it yourself probably with a hash (md5, sha1 etc) of the uploaded file. This would keep you from having to perform the actual conversion twice, but you would have to hash each file that was uploaded which would slow you down for a "cache miss".
The only other way I could think of to solve the problem would be to require the user to pass in an accessible url to the file in the query string, then you could handle GET requests, but your users would have to make the file accessible over the internet. This would allow caching but limit the usability.
Perhaps a hybrid approach would be possible where you accepted a POST for a file upload and a GET for a url, this would increase the complexity of the service but maximize usability.
Also, you should look into what caches you are interested in leveraging as a lot of them have limits on the size of a cache entry meaning if the file is sufficiently large it would not cache anyway.
In the end, I would advise you to stick to the standards already established. Accept the POST request for the file upload and if you are interested in speeding up the user experience maybe make the upload persist, this would allow the user to upload a file once and download it in many different formats.
You sequence of events can be as follows:
Upload your file/files using POST. For immediate response, you can return required information using your own headers. (It should return document key to access the file for future use.)
Then you can use GET for further operations using the above mentioned document key as a query string.
I've just converted a PageKit (mod_perl) application to Plack. This means that I now need some way to enforce the POST_MAX/MAX_BODY that Apache2::Request would have previously handled. The easiest way to do this would probably be just to put nginx in front of the app, but the app is already sitting behind HAProxy and I don't see how to do this with HAProxy.
So, my question is how I might go about enforcing a maximum body size in Plack::Middleware without reading the entire body of the request first?
Specifically I'm concerned with file uploads. Checking size via Plack::Request::Upload is too late, since the entire body would have been read at this point. The app will be deployed via Starman, so psgix.streaming should be true.
I got a response from Tatsuhiko Miyagawa via Twitter. He says, "if you deploy with Starman it's too late even with the middleware because the buffering is on. I'd do it with nginx".
This answers my particular question as I'm dealing with a Starman deployment.
He also noted that "rejecting a bigger upload before reading it on the backend could cause issues in general"
I have a large byte file (log file) that I want to upload to server using PUT request. The reason I choose PUT is simply because I can use it to create a new resource or update an existing resource.
My problem is how to handle situation when server or Network disruption happens during PUT request.
That is say I have a huge file, during the transfer of which, Network failure happens. When the network resumes, I dont want to start the entire upload. How would I handle this?
I am using JAX-RS API with RESTeasy implementation.
Some people are using the Content-Range Header to achieve this but many people (like Mark Nottingham) state that this is not legal for requests. Please read the comments to this answer.
Besides there is no support from JAX-RS for this scenario.
If you really have the repeating problem of broken PUT requests I would simply let the client slice the files:
PUT /logs/{id}/1
PUT /logs/{id}/2
PUT /logs/{id}/3
GET /logs/{id} would then return the aggregation of all successful submitted slices.
I've spent the last 5 months developing a gwt app, and it's now become time for third party people to start using it. In preparation for this one of them has set up my app behind a reverse proxy, and this immediately resulted in problems with the browser's same origin policy. I guess there's a problem in the response headers, but I can't seem to rewrite them in any way to make the problem go away. I've tried this
response.setHeader("Server", request.getRemoteAddress());
in some sort of naive attempt to mimic the behaviour I want. Didn't work (to the surprise of no-one).
Anyone knowing anything about this will most likely snicker and shake their heads when reading this, and I do not blame them. I would snicker too, if it was me... I know nothing at all about this, and that naturally makes this problem awfully hard to solve. Any help at all will be greatly appreciated.
How can I get the header rewrite to work and get away from the SOP issues I'm dealing with?
Edit: The exact problem I'm getting is a pop-up saying:
"SmartClient can't directly contact
URL
'https://localhost/app/resource?action='doStuffs'"
due to browser same-origin policy.
Remove the host and port number (even
if localhost) to avoid this problem,
or use XJSONDataSource protocol (which
allows cross-site calls), or use the
server-side HttpProxy included with
SmartClient Server."
But I shouldn't need the smartclient HttpProxy, since I have a proxy on top of the server, should I? I've gotten no indications that this could be a serialisation problem, but maybe this message is hiding the real issue...
Solution
chris_l and saret both helped to find the solution, but since I can only mark one I marked the answer from chris_l. Readers are encouraged to bump them both up, they really came through for me here. The solution was quite simple, just remove any absolute paths to your server and use only relative ones, that did the trick for me. Thanks guys!
The SOP (for AJAX requests) applies, when the URL of the HTML page, and the URL of the AJAX requests differ in their "origin". The origin includes host, port and protocol.
So if the page is http://www.example.com/index.html, your AJAX request must also point to something under http://www.example.com. For the SOP, it doesn't matter, if there is a reverse proxy - just make sure, that the URL - as it appears to the browser (including port and protocol) - isn't different. The URL you use internally is irrelevant - but don't use that internal URL in your GWT app!
Note: The solution in the special case of SmartClient turned out to be using relative URLs (instead of absolute URLs to the same origin). Since relative URLs aren't an SOP requirement in browsers, I'd say that's a bug in SmartClient.
What issue are you having exactly?
Having previously had to write a reverseproxy for a GWT app I can't remember hitting any SOP issues, one thing you need to do though is make sure response headers and uri's are rewritten to the reverseproxies url - this includes ajax callback urls.
One issue I hit (which you might also experience) when running behind a reverseproxy was with the serialization policy of GWT server.
Fixing this required writing an implementation of RemoteServiceServlet. While this was in early/mid 2009, it seems the issue still exists.
Seems like others have hit this as well - see this for further details (the answer by Michele Renda in particular)