Problem
I'm testing a wsdl application with a remote server, so I can't have any influence on the requesting mechanism.
While testing with it, I get this log messages.
DEBUG:spyne.protocol.soap.soap11:ValueError: Deserializing from unicode strings
with encoding declaration is not supported by lxml.
DEBUG:spyne.protocol.xml:Validated ? False
40.50.60.70 - - [14/Jul/2016 16:58:42] "POST /?WSDL HTTP/1.1" 500 480
So the problem seems to be, that my remote server doing it's requests in a utf-8 encoded string, but I'm not able to reproduce it on my local machine. There I'm doing this with curl, which has a charset=utf-8 encoding in it's header and a input xml, wich has encoding='UTF-8' in it's`-tag too.
$ curl -X POST -H "Content-Type: text/xml; charset=utf-8" -d #request.xml
http://localhost:8000 > response.xml
Approaches
I tried to set an event_listener at the process which builds the wsdl, to maybe get a chance to manipulate the input string with ctx.in_string or something else.
MyService.event_manager.add_listener('wsdl_document_built', _on_wsdl_document_built)
MyService.event_manager.add_listener('document_built', _on_document_built)
application.event_manager.add_listener('wsdl_document_built', _on_wsdl_document_built)
application.event_manager.add_listener('document_built', _on_document_built)
But there was no execution of my added functions.
Also i searched unseccessful the docs for a type of configuration, to pre decode it.
So my question is how to force some kind of encoding of the incoming request xml?
Related
After setting up my API to upload files, I realised that there is a special case where you want to upload a picture (jpg), you defined the binary support at the API, but you get the following error:
The request signature we calculated does not match the signature you
provided. Check your AWS Secret Access Key and signing method.
Consult the service documentation for details.
The Canonical String for this request should have been
'PUT /test/vi-dummy-bucket/testImg2.jpg
content-type:application/x-www-form-urlencoded
host:qhweyos7z2.execute-api.us-west-1.amazonaws.com
x-amz-date:20170808T154441Z
x-amz-security-token: // security token string no quotes
content-type;host;x-amz-date;x-amz-security-token 5fa90f0 ...'
The String-to-Sign should have been
'AWS4-HMAC-SHA256\n20170808T154441Z
20170808/us-west-1/execute-api/aws4_request
f7a38fa ...'
The strange thing is that uploading simple text files works with the exact same api call, then only thing I have to change is
Content-Type 'text/plain'
and write a text in the raw portion of the request.
Not sure if this is a Content-Type issue or a Request Body Issue, if I leave everything in the working state (text/plain & text in the body) and just change the body to binary and set the image, I get the above error.
My API gateway is in us-west-1 region
My S3 bucket is in us-east-1 region
And the request I am using is:
PUT /test/vi-dummy-bucket/testImg2.jpg HTTP/1.1
Host: qhwe7z2.execute-api.us-west-1.amazonaws.com
Content-Type: application/x-www-form-urlencoded
X-Amz-Security-Token: FQoDYX ...
X-Amz-Date: 20170808T154441Z
Authorization: AWS4-HMAC-SHA256
Credential=ASIAJICO6JFTJWN7A/20170808/us-west-1/execute-
api/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-
security-token,
Signature=6a792 ... Cache-Control: no-cache
Postman-Token: e9d1f730-f50b-7e27-70cc-c15a138d8cc6
(Binary Image)
This is another version of the request (same error):
PUT /test/vi-dummy-bucket/testImg2.jpg HTTP/1.1
Content-Type: image/jpeg
x-amz-security-token: FQoDY ...
x-amz-date: 20170808T190134Z
Authorization: AWS4-HMAC-SHA256
Credential=ASIAIZSP5YKVLJ3GVVQA/20170808/us-west-1/execute-
api/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-
security-token,
Signature=b2324 ...
Host: qhos7z2.execute-api.us-west-1.amazonaws.com
Connection: close
User-Agent: Paw/3.1.2 (Macintosh; OS X/10.12.6) GCDHTTPRequest
Content-Length: 823236
--- UPDATE ---
After implementing the sigV4 sigining manually using the generated SDK, the signature is no longer an issue.
The only problem left, is that the generated SDK only accepts a string as the "body", so I have to convert the file to a binary string. Then it passes correctly and a file is created in S3, but the size is now double and its not viewable, as if the binary string wasn't converted back to the binary file. So frustrating...
BTW, I've already tried PASSTHROUGH and CONVERT_TO_BINARY.
Updated: It looks like this may be related to a known error in Postman. For reference here is a related SO question: AWS Signature Error using Postman to access the AWS API Gateway when posting a binary
and here is the bug report for Postman: https://github.com/postmanlabs/postman-app-support/issues/3232
Does the request work if you use an alternate rest client and/or a command line utility like curl or httpie?
If you configured the binary support you should probably set the Content-Type to match the binary content you're sending.
From what you've posted you're sending the binary content with Content-Type application/x-www-form-urlencoded but if the body is actually a binary jpeg file I'd expect that you should be sending Content-Type image/jpeg
I have been reading multiple articles on how to sniff and subsequently use the data obtained to interact with closed source apis recently.
I am concentrated on the tinder api since it seemed to me ample research had been done already on it, hence it would be easy to learn from.
http://ttcubicle.blogspot.com/2015/03/reverse-engineering-tinders-api.html
http://ec2-52-42-144-243.us-west-2.compute.amazonaws.com/tinder/
Althrough I managed to sniff the authentification between the tinder app on my phone and the server through fiddler, I am not able to actually simulate that login using curl on the command line
----- Below is the request send from my phone to the server --- for obvious reasons I changed my actual data ----
POST https://api.gotinder.com/auth HTTP/1.1
platform: android
User-Agent: Tinder Android Version 6.5.1
os-version: 23
Facebook-ID: 10151935000326599
Accept-Language: en
app-version: 1955
Content-Type: application/json; charset=utf-8
Content-Length: 257
Host: api.gotinder.com
Connection: Keep-Alive
Accept-Encoding: gzip
X-Auth-Token: ccXX9a-4a99c-4e32-8154-9b21asf5eec
{"facebook_token":"EAfasfasfasfN6solZAh8M3kwxsP1JzF6OBDocdNUEyxd8tsVCN6kWZA6fArZB0T5dZArmdVvKAXUuQZCOtoVZBPasfzUMz9RfFoSpEifEVm7bAIspEerbLKRgW3DCpHHuxVyZApr1koAHhIjCGtxUZAAZAtDvTTbayrkF","facebook_id":"111111119","locale":"en"}
My knowledge regarding POST / Headers and all of these things is still a bit shacky (thats why I am trying to re-enact) but from what I understand that next step should be to send a POST request with curl that sends the X-Auth-Token in the header and facebook_token and so on in the data part.
something like this:
curl -H "Content-Type: application/json" -H "X-Auth-Token: cc5555a-499c-4e32-8154-9b25555ec" -d '{"facebook_token":"EAAGasdpsBAEzbJDJdcHXLjKpDjN6solZAh8M3kwxsP1JzF6OBDocdNUEyxd8tsVCN6kWZA6fArZB0T5dZArmdVvKAXUuQZCOtoVZBPZBMTUJzUMz9RfFoSpEifEVm7bAIspEerbLKRgW3DCpHHuxVyZApr1koAHhIjCGtxUZAAZA555TTbayrkF","facebook_id":"101519555326599","locale":"en"}' https://api.gotinder.com/auth
However, no matter how I change the parameters around, I always get Errorcodes 500 or 401 thrown back at me. The maximum I can get is the server telling me that it excpects a facebook_token (which is obviously send in the data section)
Does anyone has experience with this sort of problem ?
Thank you
I just discovered phyton and... holyshit i am in love!
payloadauth = {"facebook_token":"EAA xxxxxx
header1 = { 'platform': 'android','User-Agent': 'Tinder, 'X-Auth-Token':'cblabla
with requests.Session() as c:
response = c.post('https://api.gotinder.com/auth',data=payloadauth)
response = c.get('https://api.gotinder.com/recs/core?locale=en', headers=header1)
print(response.json())
four lines of code...
I'm using a web service to receive json encoded data. It was working fine. Recently I have enabled gzip compression in my web server. After that I'm getting this error in my sencha touch application.But it's working fine when I checked the url via web browser. Any idea?
net::ERR_CONTENT_DECODING_FAILED
That error is thrown by the browser; not Sencha.
The request should contain the correct header: Accept-Encoding: gzip, deflate
The response should contain the header telling the client which compression scheme is being used: Content-Encoding: gzip
See http://en.wikipedia.org/wiki/HTTP_compression.
Also check that the content is actually compressed using the specified scheme, see the following thread: Error 330 (net::ERR_CONTENT_DECODING_FAILED):
I am using curl for sending a POST HTTP message to my server. At Server side I am opening a socket and reading the data by using following code
recv(socket_Fd, (void *)ucBuffer, (size_t)((sizeof(ucBuffer) - 1)), NULL);
I am able to get the header of the POST message but in message body I am getting only one line , rest are missing.
Data I am receiving at server end.
POST /info HTTP/1.1
User-Agent: curl/7.22.0 (i686-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
Host: 192.168.0.57:10000
Accept: */*
Content-Length: 356
Content-Type: application/x-www-form-urlencoded
Bhupesh Bhargava
In message header it's showing right content length but message body is missing. Any idea where I am doing wrong.
curl command I am using
curl --data-binary #/home/bhupesh/data_save2 http://192.168.0.57:10000/info
The curl command seems to be OK and if we go by the documentation here, the following should be true.
Data is posted in a similar manner as --data-ascii does, except that
newlines are preserved and conversions are never done.
So, this leaves us with the fact that there should be a problem in your Server implementation. It is not quite sure how you are getting the received stuff at the server, but you should be careful about sequencing what you receive by yourself. Here is an example how you could do that.
Any way to get the size of a (remote, http) file without actually downloading it?
All other similar questions seem to revolve around grabbing the expectedContentLength from the NSURLResponse object in didReceiveResponse: but I don't want to download the file, I just want to know how big it is.
Try to make HEAD request, it returns only headers with Content-Length header included.
~$ curl -I http://google.com/
HTTP/1.1 301 Moved Permanently
Location: http://www.google.com/
Content-Length: 219
...