How to decode response with zlib in fiddler - fiddler

I know there is Response body is encoded. Click to decode but it dosen't work.
The response I get is encoded by zlib not gzip and there is no Content-Encoding: gzip in response header.
Now I can save the response body to a file and then decode it by Python, but I really want to see the pretty content just in fiddler.
What should I do?

I don't have enough rep. to comment, but I improved PaleNeutron's snippet.
Fix request decoding
Added logic to check for the zlib header so we don't double decode and create an empty request/response
In the FiddlerScript tab:
public static ContextAction("Force Decode (zlib)")
function AddEncoding(oSessions: Fiddler.Session[]){
for (var x:int = 0; x < oSessions.Length; x++){
if (oSessions[x].oRequest.headers["Content-Encoding"]=="zlib"){
oSessions[x].oRequest.headers["Content-Encoding"]="deflate";
oSessions[x].utilDecodeRequest();
}
if (oSessions[x].oResponse.headers["Content-Encoding"]=="zlib"){
oSessions[x].oResponse.headers["Content-Encoding"]="deflate";
oSessions[x].utilDecodeResponse();
}
}
UI.actUpdateInspector(true,true);
}

You can add or edit headers in Fiddler. Go to the Headers inspector and then select the Unlock for Editing option from the Edit menu. Then right-click on the headers and choose Add or Edit Header in the context menu.
Thus you can add/modify Content-Encoding.
You can also do that in FiddlerScript.

Add this to fiddler script
public static ContextAction("force decode with deflate")
function AddEncoding(oSessions: Fiddler.Session[]){
for (var x:int = 0; x < oSessions.Length; x++){
oSessions[x].utilDecodeRequest();
oSessions[x].oResponse.headers["Content-Encoding"]="deflate"
oSessions[x].utilDecodeResponse();
}
UI.actUpdateInspector(true,true);
}
And force decode with deflate choice will appear in the right-click menu.

Related

Prevent URL-encoding of values when adding headers to TRESTClient

I'm working on a Delphi REST client for a public API that requires an HMAC256/Base64 signed string to be added to the headers of the request to authenticate. I've spent hours trying to figure out why it's not working, so I compared the raw request from my Delphi client to that of a working C# library (using Wireshark).
It turns out my request matches perfectly the request generated by the working C# library, except that Delphi's REST client is URL-encoding the values added to the request's header, therefore invalidating the carefully crafted signature.
This is how I'm adding the signature string to the header:
RESTRequest1.Params.AddHeader('SIGNATURE', FSignature);
The signature string may have slashes, plus signs, and/or equal signs that are being URL-encoded when they shouldn't. For example when the value of the signature string is...
FSignature = '8A1BgACL9kB6P/kXuPdm99s05whfkrOUnEziEtU+0OY=';
...then the request should should output raw headers like...
GET /resource HTTP/1.1
User-Agent: Embarcadero URI Client/1.0
Connection: Keep-Alive
<snip>
SIGNATURE: 8A1BgACL9kB6P/kXuPdm99s05whfkrOUnEziEtU+0OY=
<snip>
...but instead Wireshark shows this as the real value being sent...
SIGNATURE: 8A1BgACL9kB6P%2FkXuPdm99s05whfkrOUnEziEtU%2B0OY%3D
Is there a way to prevent the URL-encoding of values when using AddHeader? Or maybe another way to add raw headers to a TRESTClient request?
PS: I already tried both TRESTRequest.Params.AddHeader and TRESTClient.AddParameter with TRESTRequestParameterKind.pkHTTPHEADER as the Kind parameter. Both resulted in URL-encoded values.
PS2: Using Delphi RAD Studio 10.3.
You should include poDoNotEncode in the Options property of the TRESTRequestParameter.
This can be done using:
RESTClient1.AddParameter('SIGNATURE', FSignature, pkHTTPHEADER, [poDoNotEncode]);
or by using:
RESTClient1.Params.AddHeader('SIGNATURE', FSignature).Options := [poDoNotEncode];

vertx-web httpserver messy code when response a Chinese word

I met an issue when I was learning Vert.x-Web, below code will return a messy code for Chinese words, any one can help?
HttpServer server = vertx.createHttpServer();
server.requestHandler(request -> {
// This handler gets called for each request that arrives on the server
HttpServerResponse response = request.response();
response.putHeader("content-type", "text/plain charset='utf-8'");
// Write to the response and end it
response.end("Hello World!中文");
});
server.listen(8080);
I just found the reason, I think actually vert.x support UTF-8 encoding, but we need to make sure all the html files and related files including css, js, and font files all match UTF-8 format while saving it. we can use notepad open the file and check if it is UTF-8 format, if not, use "Save As..." to save it as UTF-8 format.

Play WS - check compression headers

I enabled gzip compression for all the responses in my web service (Play 2.4) by following those instructions. Easy to set up, and I can see it works like a charm having checked with curl and wireshark that the responses are sent compressed.
Now I want to be a good developer and add an integration test to make sure no one breaks HTTP compression next week. Here's where the fun begins! My test looks like this:
"use HTTP compression" in {
forAll(endPoints) { endPoint =>
val response = await(
WS.url(Localhost + port + "/api" + endPoint).withHeaders("Accept-Encoding" -> "gzip").get()
)
response.header("Content-Encoding") mustBe Some("gzip")
}
}
However, the test fails as WS's response headers don't include content enconding information, and the body is returned as plain text, uncompressed.
[info] - should use HTTP compression *** FAILED ***
[info] forAll failed, because:
[info] at index 0, None was not equal to Some("gzip") (ApplicationSpec.scala:566)
Checking the traffic in wireshark when running this test I can clearly see the server is returning a gzip-encoded response, so it looks like WS is somehow transparently decompressing the response and stripping the content-encoding headers? Is there a way I can get the plain, compressed response with full headers so I can check whether the response is compressed or not?
I don't think you can do that. If I'm not mistaken , the problem here is that Netty return the content already uncompressed, so the header is removed also.
There is a configuration in AsyncHTTPClient to set that (setKeepEncoding), but unfortunately this only works in version 2.0 and newer, and Play 2.4 WS lib uses version 1.9.x.
Either way, the client Play gives you is already configured, and I don't know if you are able to tweak it. But you can create a new client to emulate that behavior:
// Converted from Java code: I have never worked with those APi's in Scala
val cfg = new AsyncHttpClientConfig.Builder().addResponseFilter(new ResponseFilter {
override def filter[T](ctx: FilterContext[T]): FilterContext[T] = {
val headers = ctx.getRequest.getHeaders
if (headers.containsKey("Accept-Encoding")) {
ctx.getResponseHeaders.getHeaders.put("Content-Encoding", List("gzip"))
}
ctx
}
}).build()
val client: NingWSClient = NingWSClient(cfg)
client.url("...") // (...)
Again, this is just emulating the result you need. Also, probably a more clever logic than just add gzip as Content-Encoding (ex: put the first algorithm requested in "Accepts Encoding") is advised.
Turns out we can't really use Play-WS for this specific test because it already returns the content uncompressed and stripped of the header (see #Salem's insightful answer), so there's no way to check whether the response is compressed.
However it's easy enough to write a test that checks for HTTP compression using standard Java classes. All we care about is whether the server answers in (valid) GZIP form when sending a request with Accept-Encoding: gzip. Here's what I ended up with:
forAll(endPoints) { endPoint =>
val url = new URL(Localhost + port + "/api/" + endPoint)
val connection = url.openConnection().asInstanceOf[HttpURLConnection]
connection.setRequestProperty("Accept-Encoding", "gzip")
Try {
new GZIPInputStream(connection.getInputStream)
} must be a 'success
}

MediaElementAudioSource outputs zeroes due to CORS access restrictions for

I try get frequency from element audio with src is a url
var aud = document.getElementById("audio-player");
var canvas, ctx, source, context, analyser, fbc_array;
function initMp3Player(){
try {
context = new (window.AudioContext || window.webkitAudioContext)();
} catch(e) {
throw new Error('The Web Audio API is unavailable');
}
analyser = context.createAnalyser(); // AnalyserNode method
analyser.smoothingTimeConstant = 0.6;
analyser.fftSize = 512;
canvas = document.getElementById('canvas_up');
ctx = canvas.getContext('2d');
source = context.createMediaElementSource(aud);
source.crossOrigin = 'anonymous';
source.connect(analyser);
analyser.connect(context.destination);
frameLooper();
}
function frameLooper(){
window.requestAnimationFrame(frameLooper);
fbc_array = new Uint8Array(analyser.frequencyBinCount);
analyser.getByteFrequencyData(fbc_array);
console.log(fbc_array);
var gradient = ctx.createLinearGradient(0,0,0,300);
gradient.addColorStop(1,'#000000');
gradient.addColorStop(0.65,'#000000');
gradient.addColorStop(0.55,'#FF0000');
gradient.addColorStop(0.25,'#FFCC00');
gradient.addColorStop(0,'#ffffff');
if(fbc_array != null){
ctx.clearRect(0, 0, canvas.width, canvas.height);
}
ctx.fillStyle = gradient; // Color of the bars
for (var i = 0; i < (fbc_array.length); i++ ){
var value = -(fbc_array[i]/4);
ctx.fillRect(i*5,canvas.height,4,value*2);
}
}
window.addEventListener("load", initMp3Player, false);
and HTML:
<audio id="audio-player"><source src="" type="audio/mpeg"></audio>
but I receive error:
MediaElementAudioSource outputs zeroes due to CORS access restrictions for ...
I searched very much but i receive a good answer and detail. I'm not really good english, so very super if answers have demo ... thanks
I just find this problem, and mad with the Message:MediaElementAudioSource outputs zeroes due to CORS access restrictions for. But it's just a message, i can still hear the audio.
And I googled lots of this, think this link will be helpful:http://www.codingforums.com/javascript-programming/342454-audio-api-js.html
The createMediaElementSource method should create an object that uses the MediaElementAudioSourceNode interface. Such objects are subject to Cross-Origin Resource Sharing (CORS) restrictions based on the latest draft of the Web Audio API spec. (Note that this restriction doesn't appear to be in the outdated W3C version of the spec.) According to the spec, silence should be played when CORS restrictions block access to a resource, which would explain the "outputs zeroes" message; presumably, zero is equivalent to no sound.
To lift the restriction, the owner of the page at
http://morebassradio.no-ip.org:8214/;stream/1 would need to configure
their server to output an Access-Control-Allow-Origin header with
either a list of domains (including yours) or the * value to lift it
for all domains. Given that this stream appears to already be
unrestricted, public-facing content, maybe you can convince the owners
to output that header. You can test whether the header is being sent
by pressing Ctrl+Shift+Q in Firefox to open the Network panel, loading
the stream through the address bar, and then inspecting the headers
associated with that HTTP request in the Network panel.
Note that they can't use a meta element here since the audio stream
is, obviously, not an HTML document; that technique only works for
HTML and XHTML documents.
(While you're messing with Firefox panels, you may want to make sure
Security errors and warnings are enabled (by clicking the Security
button or its arrow) in the Console panel (Ctrl+Shift+K). I'm not sure
if there's a corresponding CORS message in Firefox like in Chrome, but
there might be. I wasted a bunch of time wondering why a page wasn't
working one day while troubleshooting a similar technology, Content
Security Policy (CSP), only to find that I had the relevant Firefox
messages hidden.)
You shouldn't need to mess with the crossorigin property/attribute
unless you set crossorigin = "use-credentials" (JavaScript) or
crossorigin="use-credentials" (HTML) somewhere, but you probably
didn't do that because that part of the HTML spec isn't finalized yet,
and it would almost certainly cause your content to "break" after
doing so since credentials would be required at that point.
I'm not familiar with the Web Audio API, so I wasn't able to figure
out how to output a MediaElementAudioSourceNode and trigger an error
message for my own troubleshooting. If I use createMediaElementSource
with an HTMLMediaElement (HTMLAudioElement), the result doesn't seem
to be a MediaElementAudioSourceNode based on testing using the
instanceof operator even though the spec says it should be if I'm
reading it right.
Then in my situation, i get the HTTP response Header:
HTTP/1.1 206 Partial Content
Date: Thu, 02 Jun 2016 06:50:43 GMT
Content-Type: audio/mpeg
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: X-Log, X-Reqid
Access-Control-Max-Age: 2592000
Content-Disposition: inline; filename="653ab5685893b4bf.mp3"
Content-Transfer-Encoding: binary
Last-Modified: Mon, 16 May 2016 02:00:05 GMT
Server: nginx
Cache-Control: public, max-age=31536000
ETag: "FpGQqtcf_s2Ce8W_4Mv6ZqSVkVTK"
X-Log: mc.g;IO:2/304
X-Reqid: 71cAAFQgUBiJMVQU
X-Qiniu-Zone: 0
Content-Range: bytes 0-1219327/1219328
Content-Length: 1219328
Age: 1
X-Via: 1.1 xinxiazai211:88 (Cdn Cache Server V2.0), 1.1 hn13:8 (Cdn Cache Server V2.0)
Connection: keep-alive
Note that "Access-Control-Allow-Origin: *", i think this just the right thing, but i still get the message. Hope it help you.
This is correct. You can't access media from a different domain in Web Audio without CORS enabled on the media server (and making the appropriate CORS request.) This is to prevent cross-domain information attacks.
I was running into this problem when I would develop my application by opening the index.html file in my browser. A server was required in order to use the audio files I needed.
I installed the Live Server extension on Visual Studio Code - one of many ways to solve this.

Scala Playframework send file

I have a string of data, which I get from data in my database. I want to send it to the user, but without creating a local copy of the file, something like
Ok(MyString).as("file/csv")
But it is not working. How can I do it?
You can do this by using chunked with an Enumerator. I've also used withHeaders to specify the content type and disposition of the Result to "attachment", so that the client will interpret it as a file to download (rather than opening in the browser itself).
import play.api.libs.iteratee.Enumerator
val myString: String = ??? // the String you want to send as a file
Ok.chunked(Enumerator(myString.getBytes("UTF-8")).andThen(Enumerator.eof))
.withHeaders(
"Content-Type" -> "text/csv",
"Content-Disposition" -> "attachment; filename=mystring.csv"
)
This might not compile right away, depending on the types you're getting from the database.
Come to think of it, this should also work (without the Enumerator):
Ok(myString).withHeaders( /* headers from above */ )