Malformed Session ID Cookie - perl

Set-Cookie: SESSIONID=836cfc64b5856712b040a0b1b3bf4237; Secure; HttpOnly
Set-Cookie: Watson-DPAT=gBJQjAG%2FYflxpHKCwJVswQPEBuUmikj38zzFm8UZNTbOERxbeXS4WVxBIT5JetJBkeO1RT16PNz6%2BI17oFrEqvxjny%2FifZRorvBxXVzDmFkRpfRLxxj6ZNvFCvuRL1DtfW3nL8Ne1QDwpuKQmNt8%2BD9vFk7bGjlaziHT0ZFhNffWJT7FRCWbuJAyjKd%2BQui2WTIl6B8KglPi6GG1buh5UPDE%2Bc8OvrqyAfJRfYOApRdx7kHhtHdxIV7g%2FzNExXhafScqxi4cWEa5Kg9YGcypr8SIO%2FD7WOq0KyERHUDkbZatH53CCii25it5XD0plnt3cVc4bWs8tXkMT82V9DwCYULto64L%2BgNh30iTpyv72xOIfHeZTt2KISfhXMy6z86ueaJZzNd4nS6rwc7s2E8ldxwYLXrCU996xsLmzPYbGSzaeFLplG7c%2BCxzTlAll5fn8eMMbGn30W%2BrXLNtcaJ3lRK2nvzQCim1GhMdqoOvOcSvPWiJoVBrF8lc75eGSr8C%2Fovq20fOk3NDw4f0UPfBEGZYuAtXjonU7QdRhSgLRXxKyGvcYHEWeWUOQ2kvtI2m%2FRD%2BMRx9384p1v6uu8XfaU16IqoidV0Vew3MLPW4fxcOWRqnWKy0iIYbIJrWVcigloIy%2FNxgO7oHW7aacgH1u8IluAURz5AiE1Bej4l%2FjAI91IUTEssbg6fsXd3AqmlkixDglDJBgTEtMoXhXVyDjvJSaVUqdFTokP3YcRNhlTzqDQ3vG8txTLzECsyQHZ7DgWp%2B98P3zjvtad9xB%2BDzXhF4CaUB7ve99bWO5FO1DU3KRhx7pEAKGselDCoxTOkjIhEMMJbeQrbC1QWJ4uR9KlBPBdIbShd3; path=/speech-to-text/api; secure; HttpOnly
The request to create the Speech to Text session works.
{
"recognize": "https://stream.watsonplatform.net/speech-to-text/api/v1/sessions/836cfc64b5856712b040a0b1b3bf4237/recognize",
"recognizeWS": "wss://stream.watsonplatform.net/speech-to-text/api/v1/sessions/836cfc64b5856712b040a0b1b3bf4237/recognize",
"observe_result": "https://stream.watsonplatform.net/speech-to-text/api/v1/sessions/836cfc64b5856712b040a0b1b3bf4237/observe_result",
"session_id": "836cfc64b5856712b040a0b1b3bf4237",
"new_session_uri": "https://stream.watsonplatform.net/speech-to-text/api/v1/sessions/836cfc64b5856712b040a0b1b3bf4237"
}
Then I try to get the status of the session to make sure the state is "initialized" but I get a "Malformed Session ID Cookie" error.
GET /speech-to-text/api/v1/sessions/836cfc64b5856712b040a0b1b3bf4237/recognize HTTP/1.1\x0d
Content-Length: 0\x0d
Accept-Encoding: gzip\x0d
Authorization: Basic OGIyMTk0MDYtYWYzYS00YTFhLWExYmMtZDA3ZjNlNTY2Y2JmOm1lYmpBaG1ndkhMSw==\x0d
Cookie: SESSIONID=836cfc64b5856712b040a0b1b3bf4237; Watson-DPAT=gBJQjAG%2FYflxpHKCwJVswQPEBuUmikj38zzFm8UZNTbOERxbeXS4WVxBIT5JetJBkeO1RT16PNz6%2BI17oFrEqvxjny%2FifZRorvBxXVzDmFkRpfRLxxj6ZNvFCvuRL1DtfW3nL8Ne1QDwpuKQmNt8%2BD9vFk7bGjlaziHT0ZFhNffWJT7FRCWbuJAyjKd%2BQui2WTIl6B8KglPi6GG1buh5UPDE%2Bc8OvrqyAfJRfYOApRdx7kHhtHdxIV7g%2FzNExXhafScqxi4cWEa5Kg9YGcypr8SIO%2FD7WOq0KyERHUDkbZatH53CCii25it5XD0plnt3cVc4bWs8tXkMT82V9DwCYULto64L%2BgNh30iTpyv72xOIfHeZTt2KISfhXMy6z86ueaJZzNd4nS6rwc7s2E8ldxwYLXrCU996xsLmzPYbGSzaeFLplG7c%2BCxzTlAll5fn8eMMbGn30W%2BrXLNtcaJ3lRK2nvzQCim1GhMdqoOvOcSvPWiJoVBrF8lc75eGSr8C%2Fovq20fOk3NDw4f0UPfBEGZYuAtXjonU7QdRhSgLRXxKyGvcYHEWeWUOQ2kvtI2m%2FRD%2BMRx9384p1v6uu8XfaU16IqoidV0Vew3MLPW4fxcOWRqnWKy0iIYbIJrWVcigloIy%2FNxgO7oHW7aacgH1u8IluAURz5AiE1Bej4l%2FjAI91IUTEssbg6fsXd3AqmlkixDglDJBgTEtMoXhXVyDjvJSaVUqdFTokP3YcRNhlTzqDQ3vG8txTLzECsyQHZ7DgWp%2B98P3zjvtad9xB%2BDzXhF4CaUB7ve99bWO5FO1DU3KRhx7pEAKGselDCoxTOkjIhEMMJbeQrbC1QWJ4uR9KlBPBdIbShd3\x0d
User-Agent: Mojolicious (Perl)\x0d
Host: stream.watsonplatform.net\x0d
HTTP/1.1 400 Bad Request\x0d
X-Backside-Transport: FAIL FAIL\x0d
Connection: Keep-Alive\x0d
Transfer-Encoding: chunked\x0d
X-Error-Cause: Zuul Error: Malformed Session ID Cookie\x0d
Content-Type: application/json\x0d
Date: Wed, 01 Jun 2016 19:41:26 GMT\x0d
Server: -\x0d
X-Global-Transaction-ID: 237895544\x0d
X-DP-Watson-Tran-ID: stream-dp01-c0182762-b9fe-4533-acab-7fbeb02b63dd\x0d
The code is using a single instance of Mojo::UserAgent so the cookies are maintained on each request.

when using sessions you receive a SESSIONID cookie when creating the session, that cookie you need to send back to the service on every call you make after creating the session. Please note that the value of that cookie does not equal "session_id": "836cfc64b5856712b040a0b1b3bf4237", it is a longer alphanumeric string.
btw. why are you using sessions? what is your use case? maybe you could benefit from sessionless calls (simpler) or websockets (better for live use cases)
Dani

Using a trailing slash is the cause of this error. The interesting part is that the "start session" POST request with the trailing slash URL will succeed and return the correct JSON data. The next request to get the session status will fail. Not really a code problem. I also demonstrated the issue with curl.
my $ua = Mojo::UserAgent->new();
$ua->proxy->detect();
$ua->inactivity_timeout(0);
# THIS URL WORKS - no trailing slash
my $start_session_url = "https://${watson_username}:${watson_password}\#stream.watsonplatform.net/speech-to-text/api/v1/sessions";
# THIS URL DOES NOT WORK - with trailing slash
# my $start_session_url = "https://${watson_username}:${watson_password}\#stream.watsonplatform.net/speech-to-text/api/v1/sessions/";
my $session_tx = $ua->post($start_session_url);
my $response;
my $recognize_url;
if($response = $session_tx->success) {
print Dumper($response->json);
$recognize_url = $response->json->{recognize};
} else {
die "Failure to start session";
}
$recognize_url =~ s/https:\/\//https:\/\/${watson_username}:${watson_password}\#/;
# Malformed Cookie error happens here
my $status_tx = $ua->get($recognize_url);
if($response = $status_tx->success) {
print Dumper($response->json);
} else {
die "Failure to get session status";
}

Related

Docker API returns 200 OK then 400 BAD REQUEST

I am writing an API client for Docker. I understood from the documentation that the API is Restful/HTTP, yet if you connect to the local daemon you have to do it over the exposed unix socket.
It all seems to work, I open a socket, send an HTTP request (which respects the specification), I receive the expected response, but also a 400 BAD REQUEST response follows immediately.
Here is the request:
GET /info HTTP/1.1
Host: localhost
Accept: application/json
And here is what I get:
HTTP/1.1 200 OK
Api-Version: 1.30
Content-Type: application/json
Docker-Experimental: false
Ostype: linux
Server: Docker/17.06.1-ce (linux)
Date: Thu, 01 Feb 2018 18:53:18 GMT
Transfer-Encoding: chunked
892
{"ID":"6MGE:35TO:BI..." ...}
0
HTTP/1.1 400 Bad Request
Content-Type: text/plain; charset=utf-8
Connection: close
400 Bad Request
First, I figured that there is a bug on my side and I am somehow sending 2 requests, but I enabled debugging and followed the logs with sudo journalctl -fu docker.service and there is exactly one request received... at least one is logged, the GET /info. I've also debugged the code and 1 single request is sent.
Any hint is greatly appreciated!
Edit: here is the client's code:
final StringBuilder hdrs = new StringBuilder();
for(final Map.Entry<String, String> header : headers) {
hdrs.append(header.getKey() + ": " + header.getValue())
.append("\r\n");
}
final String request = String.format(
this.template(), method, home, hdrs, this.readContent(content)
);
final UnixSocketChannel channel = UnixSocketChannel.open(
new UnixSocketAddress(this.path)
);
final PrintWriter writer = new PrintWriter(
Channels.newOutputStream(channel)
);
writer.print(request);
writer.flush();
final InputStreamReader reader = new InputStreamReader(
Channels.newInputStream(channel)
);
CharBuffer result = CharBuffer.allocate(1024);
reader.read(result);
result.flip();
System.out.println("read from server: " + result.toString());
It seems like you have an extra CRLF between headers and body.
private String template() {
final StringBuilder message = new StringBuilder();
message
.append("%s %s HTTP/1.1\r\n")
.append("Host: localhost").append("\r\n")
.append("%s")
.append("\r\n").append("\r\n") //one of these is superfluous, as each header line ends with "\r\n" itself
.append("%s");
return message.toString();
}
Remove one append("\r\n") after headers and see what happens.
Fixed. Initially, I thought the problem was with the line endings (that they should have been \n instead of \r\n). Turns out, the 400 BAD REQUEST occured because the Connection: close header was missing, while the Request made was being closed right after receiving the response.
More details here.

Unexpected character in payload

My Citrus test sends a (travel)request to some REST API. The response is handled as follows:
http()
.client("http://localhost:18082/cases")
.send()
.post()
.accept("application/json; charset=UTF-8")
.contentType("application/json")
//.payload(new ClassPathResource("templates/travelrequest.json"));
.payload(
"{ "+
"\"definition\": \"travelrequest.xml\", "+
"\"name\": \"travelrequest\" "+
"} "
);
Although response code 500 is received, this is what I expect. In Wireshark I captured the following package:
Host: localhost:18082
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_121)
Accept-Encoding: gzip,deflate
{ "definition": "travelrequest.xml", "name": "travelrequest" } HTTP/1.1 500 Internal Server Error
Server: spray-can/1.3.3
Date: Thu, 13 Apr 2017 15:33:37 GMT
When I move the payload to a template, the receive part of my test now looks like this:
http()
.client("http://localhost:18082/cases")
.send()
.post()
.accept("application/json; charset=UTF-8")
.contentType("application/json")
.payload(new ClassPathResource("templates/travelrequest.json"));
//.payload(
// "{ "+
// "\"definition\": \"travelrequest.xml\", "+
// "\"name\": \"travelrequest\" "+
// "} "
//);
The template resource contains this text:
{
"definition": "travelrequest.xml",
"name": "travelrequest"
}
When I run this test, I receive a different response code: 400. In Wireshark I captured the following package:
Host: localhost:18082
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_121)
Accept-Encoding: gzip,deflate
?{
"definition": "travelrequest.xml",
"name": "travelrequest"
}HTTP/1.1 400 Bad Request
Server: spray-can/1.3.3
Date: Thu, 13 Apr 2017 15:36:15 GMT
Do notice that the request starts with an unexpected questionmark. This questionmark is not visible in the Citrus output:
17:36:13,629 DEBUG client.HttpClient| Sending HTTP message to: 'http://localhost:18082/cases'
17:36:13,629 DEBUG client.HttpClient| Message to send:
{
"definition": "travelrequest.xml",
"name": "travelrequest"
}
17:36:13,630 DEBUG ingClientInterceptor| Sending Http request message
Do notice the space directly before the opening bracket.
Is this some special character? Why is it added to the payload? Is there a logical explanation?
Cheers,
Ed
Seems to be an encoding issue. Citrus by default uses UTF-8 encoding when reading the file content. Maybe the file uses some other encoding and the first character is the result of this difference.
Please check the file encoding. You can tell Citrus to use some other encoding by setting the System property
citrus.file.encoding=UTF-8
You can also add this property to the citrus-application.properties file as described here: http://www.citrusframework.org/reference/html/configuration.html#citrus-application-properties

Google oAuth2 unauthorized_client by refresh_token

i'm trying to use an refresh token from the oAuth2 web redirect auth in my console perl script. The client id is the same and correct client id i used in my javascript an i checked it 5 times that it ist the same as i have in my google API's Console.
The client secret is checked two and it is correct.
The refresh token was created with approval_prompt=force&access_type=offline
Here is my perl sample code i use:
# -----------------------------------------------------------------------------------
my $CLIENT_ID = 'XXXXX.apps.googleusercontent.com';
my $CLIENT_SECRET = 'YYYYYYYYYYY';
# -----------------------------------------------------------------------------------
# TESTING
my $refresh_token = '1/is_5_minutes_old';
# -----------------------------------------------------------------------------------
my $string = '';
$string .= 'grant_type=refresh_token';
$string .= '&client_id=' . $CLIENT_ID;
$string .= '&client_secret=' . $CLIENT_SECRET;
$string .= '&refresh_token=' . $refresh_token;
$ua = LWP::UserAgent->new;
my $req =
HTTP::Request->new( POST => 'https://accounts.google.com/o/oauth2/token' );
$req->content_type('application/x-www-form-urlencoded');
$req->content($string);
print $string . "\n";
my $res = $ua->request($req);
print $res->as_string;
The response of it:
HTTP/1.1 400 Bad Request
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Connection: close
Date: Mon, 02 Sep 2013 10:50:26 GMT
Pragma: no-cache
Server: GSE
Content-Type: application/json
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Alternate-Protocol: 443:quic
Client-Date: Mon, 02 Sep 2013 10:50:26 GMT
Client-Peer: 74.125.136.84:443
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /C=US/O=Google Inc/CN=Google Internet Authority G2
Client-SSL-Cert-Subject: /C=US/ST=California/L=Mountain View/O=Google Inc/CN=accounts.google.com
Client-SSL-Cipher: RC4-SHA
Client-SSL-Warning: Peer certificate not verified
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
{
"error" : "unauthorized_client"
}
i hope you have an idea to help.
greatings
Invalid client usually means that the client ID and client secret don't match, or there is a typo in one of them (though you mention you've double checked this!). Nothing in your code looks wrong.
When you retrieve the refresh token, could you try putting the access token that comes along with it into the tokeinfo endpoint and making sure the values for client ID there match the ones you've configured with: https://www.googleapis.com/oauth2/v1/tokeninfo?access_token=
It might be worth dumping out the request to make sure there isn't a mistake in that (e.g. too short a content-length header or similar).
Dumping the request is key. To help , here is one I baked earlier..
==POST==
https://accounts.google.com/o/oauth2/token
refresh_token=1/_PEzU2m71wertwertwerJUtrtrytrytryf3trytryoCo
&client_id=612222222225
&client_secret=Q7334534543534yKLu
&grant_type=refresh_token
Are you using the short form of the client id, ie. just the number?

Luasocket custom headers, 404 turns to 301

My previous question was about fetching page title in lua using the socket.http module. The question lies here. Previously, youtube pages led me to a 404 error page. Based on MattJ's help, I put up custom HOST header for the request. This is what I did and what was the result:
Code
header = { host= "youtube.com" }
local result,b,c,h = http.request{ url = "http://www.youtube.com/watch?v=_eT40eV7OiI", headers = header }
print ( result, b, c, h )
for k,v in pairs(c) do print(k,v) end
Result
1 301 table: 0047D430 HTTP/1.1 301 Moved Permanently
x-content-type-options nosniff
content-length 0
expires Tue, 27 Apr 1971 19:44:06 EST
cache-control no-cache
connection close
location http://www.youtube.com/watch?v=_eT40eV7OiI
content-type text/html; charset=utf-8
date Sat, 28 Apr 2012 04:26:21 GMT
server wiseguy/0.6.11
As far as I was able to understand from this, the error is basically because of X-Content-Type-Options valued nosniff. Reading its documentation, I got to know that the only defined value, "nosniff", prevents Internet Explorer from MIME-sniffing a response away from the declared content-type.
Please help me so that I can use custom proxy and fetch the youtube(and some other sites, as mentioned in the previous question) title from their body. Here is the complete LUA file I currently have:
local http = require "socket.http"
http.PROXY="http://<proxy address here>:8080"
header = { host= "youtube.com" }
local result,b,c,h = http.request{ url = "http://www.youtube.com/watch?v=_eT40eV7OiI", headers = header }
print ( result, b, c, h )
for k,v in pairs(c) do print(k,v) end
I believe this line should be changed:
header = { host= "youtube.com" }
To:
header = { host= "www.youtube.com" }
After that, works for me.
The solution is to install luasec and to use ssl.https module to do the request.
Answered here by Paul Kulchenko!
Example:
-- luasec version 0.4.2
require("ssl")
require("https")
-- ssl.https.request(...)

adding a response header to 302 response using perl

i am trying to write a perl page that returns an http 302 response to a different location and adds a custom header to that response.
so my desired http response should be something like this:
HTTP/1.1 302 Moved
Date: Sun, 15 Apr 2012 10:59:02 GMT
Server: Apache
Location: http://www.google.com
Content-Length: 396
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1
CUSTOM_HEADER: CUSTOM_VALUE
i tried using CGI:
#!/bin/perl
use strict;
use APR::Request::Apache2;
my $r = shift;
$r->content_type('text/html; charset=utf-8');
$r->headers_out()->add("CUSTOM_HEADER", "CUSTOM_VALUE");
$r->headers_out()->add("Location", "http://www.google.com");
$r->status(302);
and i do get 302 response to google but no CUSTOM_HEADER. once i change the status to 200 by $r->status(200); i do get the CUSTOM_HEADER.
so whats up with this behavior ? how can i add my header to the 302 response ?
Use $r->err_headers_out->set or $r->err_headers_out->add
my $r = shift;
$r->content_type('text/html; charset=utf-8');
$r->err_headers_out->set(Location => "http://www.google.com");
$r->status(302);
You should use err_headers_out(). These will be printed even on errors and redirects.