Docker API returns 200 OK then 400 BAD REQUEST - rest

I am writing an API client for Docker. I understood from the documentation that the API is Restful/HTTP, yet if you connect to the local daemon you have to do it over the exposed unix socket.
It all seems to work, I open a socket, send an HTTP request (which respects the specification), I receive the expected response, but also a 400 BAD REQUEST response follows immediately.
Here is the request:
GET /info HTTP/1.1
Host: localhost
Accept: application/json
And here is what I get:
HTTP/1.1 200 OK
Api-Version: 1.30
Content-Type: application/json
Docker-Experimental: false
Ostype: linux
Server: Docker/17.06.1-ce (linux)
Date: Thu, 01 Feb 2018 18:53:18 GMT
Transfer-Encoding: chunked
892
{"ID":"6MGE:35TO:BI..." ...}
0
HTTP/1.1 400 Bad Request
Content-Type: text/plain; charset=utf-8
Connection: close
400 Bad Request
First, I figured that there is a bug on my side and I am somehow sending 2 requests, but I enabled debugging and followed the logs with sudo journalctl -fu docker.service and there is exactly one request received... at least one is logged, the GET /info. I've also debugged the code and 1 single request is sent.
Any hint is greatly appreciated!
Edit: here is the client's code:
final StringBuilder hdrs = new StringBuilder();
for(final Map.Entry<String, String> header : headers) {
hdrs.append(header.getKey() + ": " + header.getValue())
.append("\r\n");
}
final String request = String.format(
this.template(), method, home, hdrs, this.readContent(content)
);
final UnixSocketChannel channel = UnixSocketChannel.open(
new UnixSocketAddress(this.path)
);
final PrintWriter writer = new PrintWriter(
Channels.newOutputStream(channel)
);
writer.print(request);
writer.flush();
final InputStreamReader reader = new InputStreamReader(
Channels.newInputStream(channel)
);
CharBuffer result = CharBuffer.allocate(1024);
reader.read(result);
result.flip();
System.out.println("read from server: " + result.toString());

It seems like you have an extra CRLF between headers and body.
private String template() {
final StringBuilder message = new StringBuilder();
message
.append("%s %s HTTP/1.1\r\n")
.append("Host: localhost").append("\r\n")
.append("%s")
.append("\r\n").append("\r\n") //one of these is superfluous, as each header line ends with "\r\n" itself
.append("%s");
return message.toString();
}
Remove one append("\r\n") after headers and see what happens.

Fixed. Initially, I thought the problem was with the line endings (that they should have been \n instead of \r\n). Turns out, the 400 BAD REQUEST occured because the Connection: close header was missing, while the Request made was being closed right after receiving the response.
More details here.

Related

Wrong Content-Type in response in IErrorHandler

I would like to send response to my service in JSON format. I catch my custom errors in my custom behavior:
void IErrorHandler.ProvideFault(Exception error, MessageVersion version, ref Message fault)
{
XDocument errorMsg = XDocument.Parse("<errorMessage>" + error.Message + "</errorMessage>");
var jsonWriter = new JsonErrorBodyWriter(errorMsg);
fault = Message.CreateMessage(version, null, jsonWriter);
fault.Properties.Add(WebBodyFormatMessageProperty.Name, new WebBodyFormatMessageProperty(WebContentFormat.Json));
HttpResponseMessageProperty prop = new HttpResponseMessageProperty();
prop.StatusCode = HttpStatusCode.Unauthorized;
prop.Headers.Add("Content-Type", "application/json; charset=utf-8");
prop.Headers[HttpRequestHeader.ContentType] = "application/json; charset=utf-8";
--Tried different ways to achieve this
fault.Properties.Add(HttpResponseMessageProperty.Name, prop);
}
But I get wrong content-type in response. And also I couldn't manage to write any custom header like :
prop.Headers.Add("Test", "Value");
Reponse:
HTTP/1.1 401 Unauthorized
Content-Type: application/xml; charset=utf-8
Server: Microsoft-IIS/10.0
X-Powered-By: ASP.NET
Date: Wed, 30 Sep 2020 08:41:15 GMT
Content-Length: 37
{"description":"Autorization Failed"}
What is wrong in my code?

Downloading data from REST API response but getting a response code of -1

REST API details from documentation:
GET /REST/sql_snapshot/2003-03-01.sql.gz HTTP/1.1
Host: some.api.net
Authorization: Basic qpow3i12o3
The response shown below omits the message body, which contains binary compressed SQL data.
HTTP/1.1 200 OK
Date: Wed, 05 Mar 2003 10:19:46 GMT
Server: Apache/1.3.22 (Unix) (Red-Hat/Linux)Content-Type: application/octet-stream
I'm getting a response code of -1 from below.
URL url = new URL("https://some.api.net/REST/sql_snapshot/2003-03-01.sql.gz");
HttpsURLConnection connection = (HttpsURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Content-Type", "application/octet-stream");
connection.setDoOutput(true);
connection.connect();

Unexpected character in payload

My Citrus test sends a (travel)request to some REST API. The response is handled as follows:
http()
.client("http://localhost:18082/cases")
.send()
.post()
.accept("application/json; charset=UTF-8")
.contentType("application/json")
//.payload(new ClassPathResource("templates/travelrequest.json"));
.payload(
"{ "+
"\"definition\": \"travelrequest.xml\", "+
"\"name\": \"travelrequest\" "+
"} "
);
Although response code 500 is received, this is what I expect. In Wireshark I captured the following package:
Host: localhost:18082
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_121)
Accept-Encoding: gzip,deflate
{ "definition": "travelrequest.xml", "name": "travelrequest" } HTTP/1.1 500 Internal Server Error
Server: spray-can/1.3.3
Date: Thu, 13 Apr 2017 15:33:37 GMT
When I move the payload to a template, the receive part of my test now looks like this:
http()
.client("http://localhost:18082/cases")
.send()
.post()
.accept("application/json; charset=UTF-8")
.contentType("application/json")
.payload(new ClassPathResource("templates/travelrequest.json"));
//.payload(
// "{ "+
// "\"definition\": \"travelrequest.xml\", "+
// "\"name\": \"travelrequest\" "+
// "} "
//);
The template resource contains this text:
{
"definition": "travelrequest.xml",
"name": "travelrequest"
}
When I run this test, I receive a different response code: 400. In Wireshark I captured the following package:
Host: localhost:18082
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_121)
Accept-Encoding: gzip,deflate
?{
"definition": "travelrequest.xml",
"name": "travelrequest"
}HTTP/1.1 400 Bad Request
Server: spray-can/1.3.3
Date: Thu, 13 Apr 2017 15:36:15 GMT
Do notice that the request starts with an unexpected questionmark. This questionmark is not visible in the Citrus output:
17:36:13,629 DEBUG client.HttpClient| Sending HTTP message to: 'http://localhost:18082/cases'
17:36:13,629 DEBUG client.HttpClient| Message to send:
{
"definition": "travelrequest.xml",
"name": "travelrequest"
}
17:36:13,630 DEBUG ingClientInterceptor| Sending Http request message
Do notice the space directly before the opening bracket.
Is this some special character? Why is it added to the payload? Is there a logical explanation?
Cheers,
Ed
Seems to be an encoding issue. Citrus by default uses UTF-8 encoding when reading the file content. Maybe the file uses some other encoding and the first character is the result of this difference.
Please check the file encoding. You can tell Citrus to use some other encoding by setting the System property
citrus.file.encoding=UTF-8
You can also add this property to the citrus-application.properties file as described here: http://www.citrusframework.org/reference/html/configuration.html#citrus-application-properties

Malformed Session ID Cookie

Set-Cookie: SESSIONID=836cfc64b5856712b040a0b1b3bf4237; Secure; HttpOnly
Set-Cookie: Watson-DPAT=gBJQjAG%2FYflxpHKCwJVswQPEBuUmikj38zzFm8UZNTbOERxbeXS4WVxBIT5JetJBkeO1RT16PNz6%2BI17oFrEqvxjny%2FifZRorvBxXVzDmFkRpfRLxxj6ZNvFCvuRL1DtfW3nL8Ne1QDwpuKQmNt8%2BD9vFk7bGjlaziHT0ZFhNffWJT7FRCWbuJAyjKd%2BQui2WTIl6B8KglPi6GG1buh5UPDE%2Bc8OvrqyAfJRfYOApRdx7kHhtHdxIV7g%2FzNExXhafScqxi4cWEa5Kg9YGcypr8SIO%2FD7WOq0KyERHUDkbZatH53CCii25it5XD0plnt3cVc4bWs8tXkMT82V9DwCYULto64L%2BgNh30iTpyv72xOIfHeZTt2KISfhXMy6z86ueaJZzNd4nS6rwc7s2E8ldxwYLXrCU996xsLmzPYbGSzaeFLplG7c%2BCxzTlAll5fn8eMMbGn30W%2BrXLNtcaJ3lRK2nvzQCim1GhMdqoOvOcSvPWiJoVBrF8lc75eGSr8C%2Fovq20fOk3NDw4f0UPfBEGZYuAtXjonU7QdRhSgLRXxKyGvcYHEWeWUOQ2kvtI2m%2FRD%2BMRx9384p1v6uu8XfaU16IqoidV0Vew3MLPW4fxcOWRqnWKy0iIYbIJrWVcigloIy%2FNxgO7oHW7aacgH1u8IluAURz5AiE1Bej4l%2FjAI91IUTEssbg6fsXd3AqmlkixDglDJBgTEtMoXhXVyDjvJSaVUqdFTokP3YcRNhlTzqDQ3vG8txTLzECsyQHZ7DgWp%2B98P3zjvtad9xB%2BDzXhF4CaUB7ve99bWO5FO1DU3KRhx7pEAKGselDCoxTOkjIhEMMJbeQrbC1QWJ4uR9KlBPBdIbShd3; path=/speech-to-text/api; secure; HttpOnly
The request to create the Speech to Text session works.
{
"recognize": "https://stream.watsonplatform.net/speech-to-text/api/v1/sessions/836cfc64b5856712b040a0b1b3bf4237/recognize",
"recognizeWS": "wss://stream.watsonplatform.net/speech-to-text/api/v1/sessions/836cfc64b5856712b040a0b1b3bf4237/recognize",
"observe_result": "https://stream.watsonplatform.net/speech-to-text/api/v1/sessions/836cfc64b5856712b040a0b1b3bf4237/observe_result",
"session_id": "836cfc64b5856712b040a0b1b3bf4237",
"new_session_uri": "https://stream.watsonplatform.net/speech-to-text/api/v1/sessions/836cfc64b5856712b040a0b1b3bf4237"
}
Then I try to get the status of the session to make sure the state is "initialized" but I get a "Malformed Session ID Cookie" error.
GET /speech-to-text/api/v1/sessions/836cfc64b5856712b040a0b1b3bf4237/recognize HTTP/1.1\x0d
Content-Length: 0\x0d
Accept-Encoding: gzip\x0d
Authorization: Basic OGIyMTk0MDYtYWYzYS00YTFhLWExYmMtZDA3ZjNlNTY2Y2JmOm1lYmpBaG1ndkhMSw==\x0d
Cookie: SESSIONID=836cfc64b5856712b040a0b1b3bf4237; Watson-DPAT=gBJQjAG%2FYflxpHKCwJVswQPEBuUmikj38zzFm8UZNTbOERxbeXS4WVxBIT5JetJBkeO1RT16PNz6%2BI17oFrEqvxjny%2FifZRorvBxXVzDmFkRpfRLxxj6ZNvFCvuRL1DtfW3nL8Ne1QDwpuKQmNt8%2BD9vFk7bGjlaziHT0ZFhNffWJT7FRCWbuJAyjKd%2BQui2WTIl6B8KglPi6GG1buh5UPDE%2Bc8OvrqyAfJRfYOApRdx7kHhtHdxIV7g%2FzNExXhafScqxi4cWEa5Kg9YGcypr8SIO%2FD7WOq0KyERHUDkbZatH53CCii25it5XD0plnt3cVc4bWs8tXkMT82V9DwCYULto64L%2BgNh30iTpyv72xOIfHeZTt2KISfhXMy6z86ueaJZzNd4nS6rwc7s2E8ldxwYLXrCU996xsLmzPYbGSzaeFLplG7c%2BCxzTlAll5fn8eMMbGn30W%2BrXLNtcaJ3lRK2nvzQCim1GhMdqoOvOcSvPWiJoVBrF8lc75eGSr8C%2Fovq20fOk3NDw4f0UPfBEGZYuAtXjonU7QdRhSgLRXxKyGvcYHEWeWUOQ2kvtI2m%2FRD%2BMRx9384p1v6uu8XfaU16IqoidV0Vew3MLPW4fxcOWRqnWKy0iIYbIJrWVcigloIy%2FNxgO7oHW7aacgH1u8IluAURz5AiE1Bej4l%2FjAI91IUTEssbg6fsXd3AqmlkixDglDJBgTEtMoXhXVyDjvJSaVUqdFTokP3YcRNhlTzqDQ3vG8txTLzECsyQHZ7DgWp%2B98P3zjvtad9xB%2BDzXhF4CaUB7ve99bWO5FO1DU3KRhx7pEAKGselDCoxTOkjIhEMMJbeQrbC1QWJ4uR9KlBPBdIbShd3\x0d
User-Agent: Mojolicious (Perl)\x0d
Host: stream.watsonplatform.net\x0d
HTTP/1.1 400 Bad Request\x0d
X-Backside-Transport: FAIL FAIL\x0d
Connection: Keep-Alive\x0d
Transfer-Encoding: chunked\x0d
X-Error-Cause: Zuul Error: Malformed Session ID Cookie\x0d
Content-Type: application/json\x0d
Date: Wed, 01 Jun 2016 19:41:26 GMT\x0d
Server: -\x0d
X-Global-Transaction-ID: 237895544\x0d
X-DP-Watson-Tran-ID: stream-dp01-c0182762-b9fe-4533-acab-7fbeb02b63dd\x0d
The code is using a single instance of Mojo::UserAgent so the cookies are maintained on each request.
when using sessions you receive a SESSIONID cookie when creating the session, that cookie you need to send back to the service on every call you make after creating the session. Please note that the value of that cookie does not equal "session_id": "836cfc64b5856712b040a0b1b3bf4237", it is a longer alphanumeric string.
btw. why are you using sessions? what is your use case? maybe you could benefit from sessionless calls (simpler) or websockets (better for live use cases)
Dani
Using a trailing slash is the cause of this error. The interesting part is that the "start session" POST request with the trailing slash URL will succeed and return the correct JSON data. The next request to get the session status will fail. Not really a code problem. I also demonstrated the issue with curl.
my $ua = Mojo::UserAgent->new();
$ua->proxy->detect();
$ua->inactivity_timeout(0);
# THIS URL WORKS - no trailing slash
my $start_session_url = "https://${watson_username}:${watson_password}\#stream.watsonplatform.net/speech-to-text/api/v1/sessions";
# THIS URL DOES NOT WORK - with trailing slash
# my $start_session_url = "https://${watson_username}:${watson_password}\#stream.watsonplatform.net/speech-to-text/api/v1/sessions/";
my $session_tx = $ua->post($start_session_url);
my $response;
my $recognize_url;
if($response = $session_tx->success) {
print Dumper($response->json);
$recognize_url = $response->json->{recognize};
} else {
die "Failure to start session";
}
$recognize_url =~ s/https:\/\//https:\/\/${watson_username}:${watson_password}\#/;
# Malformed Cookie error happens here
my $status_tx = $ua->get($recognize_url);
if($response = $status_tx->success) {
print Dumper($response->json);
} else {
die "Failure to get session status";
}

Luasocket custom headers, 404 turns to 301

My previous question was about fetching page title in lua using the socket.http module. The question lies here. Previously, youtube pages led me to a 404 error page. Based on MattJ's help, I put up custom HOST header for the request. This is what I did and what was the result:
Code
header = { host= "youtube.com" }
local result,b,c,h = http.request{ url = "http://www.youtube.com/watch?v=_eT40eV7OiI", headers = header }
print ( result, b, c, h )
for k,v in pairs(c) do print(k,v) end
Result
1 301 table: 0047D430 HTTP/1.1 301 Moved Permanently
x-content-type-options nosniff
content-length 0
expires Tue, 27 Apr 1971 19:44:06 EST
cache-control no-cache
connection close
location http://www.youtube.com/watch?v=_eT40eV7OiI
content-type text/html; charset=utf-8
date Sat, 28 Apr 2012 04:26:21 GMT
server wiseguy/0.6.11
As far as I was able to understand from this, the error is basically because of X-Content-Type-Options valued nosniff. Reading its documentation, I got to know that the only defined value, "nosniff", prevents Internet Explorer from MIME-sniffing a response away from the declared content-type.
Please help me so that I can use custom proxy and fetch the youtube(and some other sites, as mentioned in the previous question) title from their body. Here is the complete LUA file I currently have:
local http = require "socket.http"
http.PROXY="http://<proxy address here>:8080"
header = { host= "youtube.com" }
local result,b,c,h = http.request{ url = "http://www.youtube.com/watch?v=_eT40eV7OiI", headers = header }
print ( result, b, c, h )
for k,v in pairs(c) do print(k,v) end
I believe this line should be changed:
header = { host= "youtube.com" }
To:
header = { host= "www.youtube.com" }
After that, works for me.
The solution is to install luasec and to use ssl.https module to do the request.
Answered here by Paul Kulchenko!
Example:
-- luasec version 0.4.2
require("ssl")
require("https")
-- ssl.https.request(...)