Any way to get the size of a (remote, http) file without actually downloading it?
All other similar questions seem to revolve around grabbing the expectedContentLength from the NSURLResponse object in didReceiveResponse: but I don't want to download the file, I just want to know how big it is.
Try to make HEAD request, it returns only headers with Content-Length header included.
~$ curl -I http://google.com/
HTTP/1.1 301 Moved Permanently
Location: http://www.google.com/
Content-Length: 219
...
Related
After setting up my API to upload files, I realised that there is a special case where you want to upload a picture (jpg), you defined the binary support at the API, but you get the following error:
The request signature we calculated does not match the signature you
provided. Check your AWS Secret Access Key and signing method.
Consult the service documentation for details.
The Canonical String for this request should have been
'PUT /test/vi-dummy-bucket/testImg2.jpg
content-type:application/x-www-form-urlencoded
host:qhweyos7z2.execute-api.us-west-1.amazonaws.com
x-amz-date:20170808T154441Z
x-amz-security-token: // security token string no quotes
content-type;host;x-amz-date;x-amz-security-token 5fa90f0 ...'
The String-to-Sign should have been
'AWS4-HMAC-SHA256\n20170808T154441Z
20170808/us-west-1/execute-api/aws4_request
f7a38fa ...'
The strange thing is that uploading simple text files works with the exact same api call, then only thing I have to change is
Content-Type 'text/plain'
and write a text in the raw portion of the request.
Not sure if this is a Content-Type issue or a Request Body Issue, if I leave everything in the working state (text/plain & text in the body) and just change the body to binary and set the image, I get the above error.
My API gateway is in us-west-1 region
My S3 bucket is in us-east-1 region
And the request I am using is:
PUT /test/vi-dummy-bucket/testImg2.jpg HTTP/1.1
Host: qhwe7z2.execute-api.us-west-1.amazonaws.com
Content-Type: application/x-www-form-urlencoded
X-Amz-Security-Token: FQoDYX ...
X-Amz-Date: 20170808T154441Z
Authorization: AWS4-HMAC-SHA256
Credential=ASIAJICO6JFTJWN7A/20170808/us-west-1/execute-
api/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-
security-token,
Signature=6a792 ... Cache-Control: no-cache
Postman-Token: e9d1f730-f50b-7e27-70cc-c15a138d8cc6
(Binary Image)
This is another version of the request (same error):
PUT /test/vi-dummy-bucket/testImg2.jpg HTTP/1.1
Content-Type: image/jpeg
x-amz-security-token: FQoDY ...
x-amz-date: 20170808T190134Z
Authorization: AWS4-HMAC-SHA256
Credential=ASIAIZSP5YKVLJ3GVVQA/20170808/us-west-1/execute-
api/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-
security-token,
Signature=b2324 ...
Host: qhos7z2.execute-api.us-west-1.amazonaws.com
Connection: close
User-Agent: Paw/3.1.2 (Macintosh; OS X/10.12.6) GCDHTTPRequest
Content-Length: 823236
--- UPDATE ---
After implementing the sigV4 sigining manually using the generated SDK, the signature is no longer an issue.
The only problem left, is that the generated SDK only accepts a string as the "body", so I have to convert the file to a binary string. Then it passes correctly and a file is created in S3, but the size is now double and its not viewable, as if the binary string wasn't converted back to the binary file. So frustrating...
BTW, I've already tried PASSTHROUGH and CONVERT_TO_BINARY.
Updated: It looks like this may be related to a known error in Postman. For reference here is a related SO question: AWS Signature Error using Postman to access the AWS API Gateway when posting a binary
and here is the bug report for Postman: https://github.com/postmanlabs/postman-app-support/issues/3232
Does the request work if you use an alternate rest client and/or a command line utility like curl or httpie?
If you configured the binary support you should probably set the Content-Type to match the binary content you're sending.
From what you've posted you're sending the binary content with Content-Type application/x-www-form-urlencoded but if the body is actually a binary jpeg file I'd expect that you should be sending Content-Type image/jpeg
I have HAproxy is behind an AWS ELB. As soon as i remove the ELB, i can get the custom error page. but, with ELB in the front of Haproxy, i get HTTP/1.1 504 GATEWAY_TIMEOUT Content-Length: 0 Connection: keep-alive.
Can anyone tell me what is going on please? Thanks
errorfile:
HTTP/1.0 403 Forbidden
Cache-Control: no-cache
Connection: close
Content-Type: text/html
<html><body><h1>403 Forbidden</h1>
Request forbidden by administrative rules.
</body></html>
Me and a coworker just had the same problem. After receiving the timeouts, and reading amazon definition for this type of http code, i got in my head my error file was "malformed". After a lot of trys, we managed to discover that there is something funny with the CL-RF (new lines) on the error file "header".
I downloaded HaProxy default file from their git (https://raw.githubusercontent.com/haproxy/haproxy/60220bbc4b6b3c4279d3c96232cf2c2461ecc55e/examples/errorfiles/503.http) and when you open it on vi(m) it has a ^M (CR) sign on the headers(everything before the body, including the empty line separating them). If you cant download it, you could just write it (just the top part) on wordpad or something like it (dos) and then send it to you unix machine.
So i wrote my on file using their header and now everything works fine.
Cheers.
Problem
I'm testing a wsdl application with a remote server, so I can't have any influence on the requesting mechanism.
While testing with it, I get this log messages.
DEBUG:spyne.protocol.soap.soap11:ValueError: Deserializing from unicode strings
with encoding declaration is not supported by lxml.
DEBUG:spyne.protocol.xml:Validated ? False
40.50.60.70 - - [14/Jul/2016 16:58:42] "POST /?WSDL HTTP/1.1" 500 480
So the problem seems to be, that my remote server doing it's requests in a utf-8 encoded string, but I'm not able to reproduce it on my local machine. There I'm doing this with curl, which has a charset=utf-8 encoding in it's header and a input xml, wich has encoding='UTF-8' in it's`-tag too.
$ curl -X POST -H "Content-Type: text/xml; charset=utf-8" -d #request.xml
http://localhost:8000 > response.xml
Approaches
I tried to set an event_listener at the process which builds the wsdl, to maybe get a chance to manipulate the input string with ctx.in_string or something else.
MyService.event_manager.add_listener('wsdl_document_built', _on_wsdl_document_built)
MyService.event_manager.add_listener('document_built', _on_document_built)
application.event_manager.add_listener('wsdl_document_built', _on_wsdl_document_built)
application.event_manager.add_listener('document_built', _on_document_built)
But there was no execution of my added functions.
Also i searched unseccessful the docs for a type of configuration, to pre decode it.
So my question is how to force some kind of encoding of the incoming request xml?
I'll start with the log that I am receiving below:
Dec.15.11.56-Rf: Incoming Request URL: /
Dec.15.11.56-Rf: SECURE GET Path: / From: mlocal.cldeals.com Rewritten: www.cldeals.com
Dec.15.11.56-Rf: Received 302 Found [text/html; charset=UTF-8] response for /
Dec.15.11.56-Rf: Sending 302 text/html; charset=UTF-8 response for /
Dec.15.11.56-Rf: Stats. Total: 0.52088702, Upstream: 0.48212701, Processing: 0.00105600, ProcessingOther: 0.04037500
Basically, when I go to mlocal.cldeals.com, it loads fine. If I click on another page, say mlocal.cldeals.com/products, that loads fine as well. The issue seems to be when I go to the account page and try to switch back to the homepage, maybe some type of security issue? When I try to switch back to mlocal.cldeals.com, the home page, it boots me off and sends me to www.cldeals.com. Is there something I can add to force this from not happening? Additionally, is this just a local server issue that would go away when I launch it on Moovweb's server? Any help is greatly appreciated.
Thank you.
It looks like the backend response to https://www.cldeals.com is a 302 to http://www.cldeals.com:80/. Not sure why that is the case (see note below *)
curl -v -o /dev/null https://www.cldeals.com
This response contains a hardcoded Location header and your project is passing along the response as is, which is why you are being booted off your local server.
Because the Location header value has a port specified, you'll need to modify your config.json to include this line in the mapping:
{
"host_map": [
"$.cldeals.com => www.cldeals.com",
"$.cldeals.com => www.cldeals.com:80"
]
}
This way, the SDK knows to rewrite that specific host:port value... (By default all HTTP requests go through port 80, so that information isn't really necessary)
*This is might be bug in the backend implementation because once you log in, you should be in HTTPS mode until you log out. (I can see some pages with personal information being transmitted over plain HTTP)
I am using curl for sending a POST HTTP message to my server. At Server side I am opening a socket and reading the data by using following code
recv(socket_Fd, (void *)ucBuffer, (size_t)((sizeof(ucBuffer) - 1)), NULL);
I am able to get the header of the POST message but in message body I am getting only one line , rest are missing.
Data I am receiving at server end.
POST /info HTTP/1.1
User-Agent: curl/7.22.0 (i686-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
Host: 192.168.0.57:10000
Accept: */*
Content-Length: 356
Content-Type: application/x-www-form-urlencoded
Bhupesh Bhargava
In message header it's showing right content length but message body is missing. Any idea where I am doing wrong.
curl command I am using
curl --data-binary #/home/bhupesh/data_save2 http://192.168.0.57:10000/info
The curl command seems to be OK and if we go by the documentation here, the following should be true.
Data is posted in a similar manner as --data-ascii does, except that
newlines are preserved and conversions are never done.
So, this leaves us with the fact that there should be a problem in your Server implementation. It is not quite sure how you are getting the received stuff at the server, but you should be careful about sequencing what you receive by yourself. Here is an example how you could do that.