I use Vert.x HttpServer created like this:
vertx.createHttpServer(new HttpServerOptions().setDecompressionSupported(true));
My client can send two type of messages to this server:
message compressed by gzip (java.util.zip.GZIPOutputStream is used for compression)
message compressed by snappy (org.xerial.snappy.SnappyFramedOutputStream is used for compression)
I expect that incoming message will be decompressed automatically by underlying netty server. It is working for gzip messages, however it is not working for snappy messages. Snappy messages are not decompressed automatically.
I send following HTTP headers with gzip messages:
Content-Encoding: gzip
Content-Type: application/gzip
I send following HTTP headers with snappy messages:
Content-Encoding: x-snappy-framed
Content-Type: application/x-snappy-framed
Is it possible to enable snappy decompression for HTTP server in Vert.x 4.2.5?
If not, is it possible to extend Vert.x so netty's SnappyFrameDecoder will be used?
Thank you very much for your help.
In 4.2.5, it is not possible. There is a draft pull request that should be released in 4.3.0: https://github.com/eclipse-vertx/vert.x/pull/4262
Related
I'm trying to simulate a TCP protocol while using UDP.
I want my server to be able to send chunks of the file the client requested while waiting for acks from the client.
My idea is to send chunk then recv (listen for ack) but that makes it so I can only send chunk nº2 after receiving ack nº1.
Should I just send all of them in a loop and then wait for all the acks in a loop with timeouts?
Or is there way to have recv in the background with a timeout and if it receives an ack it stops the setsockopt timeout feature, and if the timeout occurs it interrupts the current script to resend that chunk?
Edit:
My bad didn't mention it was using C.
OS is linux.
Server-Side Libraries:
packet-format.h
arpa/inet.h
limits.h
netinet/in.h
stdbool.h
stddef.h
stdio.h
stdlib.h
string.h
sys/socket.h
unistd.h
Client-Side Libraries:
packet-format.h
limits.h
netdb.h
stdbool.h
stddef.h
stdio.h
stdlib.h
string.h
unistd.h
I'm unable to send both, JSON body data and a video/image file together in one request on JMeter. It gives a 401 unauthorized, even though I have added the Authorization header in the HTTP Header Manager. The same request with headers works fine on Postman.
I've already referred to this guide here with no luck.
https://www.blazemeter.com/blog/testing-advanced-rest-api-file-uploads-jmeter/
Content-Type: multipart/form-data
{
"email": "xyz#gmail.com",
"password": "xyz",
"file": ${__FileToString("/Downloads/SampleVideo_1280x720_1mb.mp4")}
}
SAMPLER RESULT:
Thread Name: Thread Group 1-1
Sample Start: 2019-09-30 14:26:25 IST
Load time: 2922
Connect Time: 838
Latency: 2922
Size in bytes: 202
Sent bytes:1056397
Headers size in bytes: 202
Body size in bytes: 0
Sample Count: 1
Error Count: 1
Data type ("text"|"bin"|""):
Response code: 401
Response message: Unauthorized
HTTPSampleResult fields:
ContentType:
DataEncoding: null
REQUEST:
POST data:
--o3F8APyqP080W3wk0N_-96jzl11Bfsa
Content-Disposition: form-data; name="file"; filename="SampleVideo_1280x720_1mb.mp4"
Content-Type: video/mp4
Content-Transfer-Encoding: binary
<actual file content, not shown here>
--o3F8APyqP080W3wk0N_-96jzl11Bfsa--
[no cookies]
JMeter should be doing what real browser (or other application using your API endpoint) is doing.
If the browser (or the application) sends 2 requests - JMeter must be sending 2 requests as well. If the browser (or the application) sends 1 request - JMeter must be sending 1 request. In case of 1 request my expectation is that you should be sending application/json as the value of the Content-Type header like it's done in the guide you're referring.
--boundary
Content-Type: application/json; charset=UTF-8
JSON Metadata
--boundary
Content-Type: file MIME type
File content
--boundary--
The answer is quite simple: just record the request using JMeter's HTTP(S) Test Script Recorder and JMeter will come up with proper configuration of the HTTP Request sampler(s), HTTP Header Manager, etc. All you will need to do is to implement parameterization and correlation so the script could be replayed successfully.
One important bit: during the recording you need to have SampleVideo_1280x720_1mb.mp4 file in JMeter's "bin" folder as modern browsers don't return the full path hence JMeter will be able to properly capture the file upload request only if the file lives in its "bin" folder. More information: Recording File Uploads with JMeter
To post json body and attachment you can add your "email" and "password" in Parameters tab of JMeter HTTPS request and file in Files Upload tab.
After setting up my API to upload files, I realised that there is a special case where you want to upload a picture (jpg), you defined the binary support at the API, but you get the following error:
The request signature we calculated does not match the signature you
provided. Check your AWS Secret Access Key and signing method.
Consult the service documentation for details.
The Canonical String for this request should have been
'PUT /test/vi-dummy-bucket/testImg2.jpg
content-type:application/x-www-form-urlencoded
host:qhweyos7z2.execute-api.us-west-1.amazonaws.com
x-amz-date:20170808T154441Z
x-amz-security-token: // security token string no quotes
content-type;host;x-amz-date;x-amz-security-token 5fa90f0 ...'
The String-to-Sign should have been
'AWS4-HMAC-SHA256\n20170808T154441Z
20170808/us-west-1/execute-api/aws4_request
f7a38fa ...'
The strange thing is that uploading simple text files works with the exact same api call, then only thing I have to change is
Content-Type 'text/plain'
and write a text in the raw portion of the request.
Not sure if this is a Content-Type issue or a Request Body Issue, if I leave everything in the working state (text/plain & text in the body) and just change the body to binary and set the image, I get the above error.
My API gateway is in us-west-1 region
My S3 bucket is in us-east-1 region
And the request I am using is:
PUT /test/vi-dummy-bucket/testImg2.jpg HTTP/1.1
Host: qhwe7z2.execute-api.us-west-1.amazonaws.com
Content-Type: application/x-www-form-urlencoded
X-Amz-Security-Token: FQoDYX ...
X-Amz-Date: 20170808T154441Z
Authorization: AWS4-HMAC-SHA256
Credential=ASIAJICO6JFTJWN7A/20170808/us-west-1/execute-
api/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-
security-token,
Signature=6a792 ... Cache-Control: no-cache
Postman-Token: e9d1f730-f50b-7e27-70cc-c15a138d8cc6
(Binary Image)
This is another version of the request (same error):
PUT /test/vi-dummy-bucket/testImg2.jpg HTTP/1.1
Content-Type: image/jpeg
x-amz-security-token: FQoDY ...
x-amz-date: 20170808T190134Z
Authorization: AWS4-HMAC-SHA256
Credential=ASIAIZSP5YKVLJ3GVVQA/20170808/us-west-1/execute-
api/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-
security-token,
Signature=b2324 ...
Host: qhos7z2.execute-api.us-west-1.amazonaws.com
Connection: close
User-Agent: Paw/3.1.2 (Macintosh; OS X/10.12.6) GCDHTTPRequest
Content-Length: 823236
--- UPDATE ---
After implementing the sigV4 sigining manually using the generated SDK, the signature is no longer an issue.
The only problem left, is that the generated SDK only accepts a string as the "body", so I have to convert the file to a binary string. Then it passes correctly and a file is created in S3, but the size is now double and its not viewable, as if the binary string wasn't converted back to the binary file. So frustrating...
BTW, I've already tried PASSTHROUGH and CONVERT_TO_BINARY.
Updated: It looks like this may be related to a known error in Postman. For reference here is a related SO question: AWS Signature Error using Postman to access the AWS API Gateway when posting a binary
and here is the bug report for Postman: https://github.com/postmanlabs/postman-app-support/issues/3232
Does the request work if you use an alternate rest client and/or a command line utility like curl or httpie?
If you configured the binary support you should probably set the Content-Type to match the binary content you're sending.
From what you've posted you're sending the binary content with Content-Type application/x-www-form-urlencoded but if the body is actually a binary jpeg file I'd expect that you should be sending Content-Type image/jpeg
Problem
I'm testing a wsdl application with a remote server, so I can't have any influence on the requesting mechanism.
While testing with it, I get this log messages.
DEBUG:spyne.protocol.soap.soap11:ValueError: Deserializing from unicode strings
with encoding declaration is not supported by lxml.
DEBUG:spyne.protocol.xml:Validated ? False
40.50.60.70 - - [14/Jul/2016 16:58:42] "POST /?WSDL HTTP/1.1" 500 480
So the problem seems to be, that my remote server doing it's requests in a utf-8 encoded string, but I'm not able to reproduce it on my local machine. There I'm doing this with curl, which has a charset=utf-8 encoding in it's header and a input xml, wich has encoding='UTF-8' in it's`-tag too.
$ curl -X POST -H "Content-Type: text/xml; charset=utf-8" -d #request.xml
http://localhost:8000 > response.xml
Approaches
I tried to set an event_listener at the process which builds the wsdl, to maybe get a chance to manipulate the input string with ctx.in_string or something else.
MyService.event_manager.add_listener('wsdl_document_built', _on_wsdl_document_built)
MyService.event_manager.add_listener('document_built', _on_document_built)
application.event_manager.add_listener('wsdl_document_built', _on_wsdl_document_built)
application.event_manager.add_listener('document_built', _on_document_built)
But there was no execution of my added functions.
Also i searched unseccessful the docs for a type of configuration, to pre decode it.
So my question is how to force some kind of encoding of the incoming request xml?
I'm using a web service to receive json encoded data. It was working fine. Recently I have enabled gzip compression in my web server. After that I'm getting this error in my sencha touch application.But it's working fine when I checked the url via web browser. Any idea?
net::ERR_CONTENT_DECODING_FAILED
That error is thrown by the browser; not Sencha.
The request should contain the correct header: Accept-Encoding: gzip, deflate
The response should contain the header telling the client which compression scheme is being used: Content-Encoding: gzip
See http://en.wikipedia.org/wiki/HTTP_compression.
Also check that the content is actually compressed using the specified scheme, see the following thread: Error 330 (net::ERR_CONTENT_DECODING_FAILED):