Unexpected character in payload - citrus-framework

My Citrus test sends a (travel)request to some REST API. The response is handled as follows:
http()
.client("http://localhost:18082/cases")
.send()
.post()
.accept("application/json; charset=UTF-8")
.contentType("application/json")
//.payload(new ClassPathResource("templates/travelrequest.json"));
.payload(
"{ "+
"\"definition\": \"travelrequest.xml\", "+
"\"name\": \"travelrequest\" "+
"} "
);
Although response code 500 is received, this is what I expect. In Wireshark I captured the following package:
Host: localhost:18082
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_121)
Accept-Encoding: gzip,deflate
{ "definition": "travelrequest.xml", "name": "travelrequest" } HTTP/1.1 500 Internal Server Error
Server: spray-can/1.3.3
Date: Thu, 13 Apr 2017 15:33:37 GMT
When I move the payload to a template, the receive part of my test now looks like this:
http()
.client("http://localhost:18082/cases")
.send()
.post()
.accept("application/json; charset=UTF-8")
.contentType("application/json")
.payload(new ClassPathResource("templates/travelrequest.json"));
//.payload(
// "{ "+
// "\"definition\": \"travelrequest.xml\", "+
// "\"name\": \"travelrequest\" "+
// "} "
//);
The template resource contains this text:
{
"definition": "travelrequest.xml",
"name": "travelrequest"
}
When I run this test, I receive a different response code: 400. In Wireshark I captured the following package:
Host: localhost:18082
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_121)
Accept-Encoding: gzip,deflate
?{
"definition": "travelrequest.xml",
"name": "travelrequest"
}HTTP/1.1 400 Bad Request
Server: spray-can/1.3.3
Date: Thu, 13 Apr 2017 15:36:15 GMT
Do notice that the request starts with an unexpected questionmark. This questionmark is not visible in the Citrus output:
17:36:13,629 DEBUG client.HttpClient| Sending HTTP message to: 'http://localhost:18082/cases'
17:36:13,629 DEBUG client.HttpClient| Message to send:
{
"definition": "travelrequest.xml",
"name": "travelrequest"
}
17:36:13,630 DEBUG ingClientInterceptor| Sending Http request message
Do notice the space directly before the opening bracket.
Is this some special character? Why is it added to the payload? Is there a logical explanation?
Cheers,
Ed

Seems to be an encoding issue. Citrus by default uses UTF-8 encoding when reading the file content. Maybe the file uses some other encoding and the first character is the result of this difference.
Please check the file encoding. You can tell Citrus to use some other encoding by setting the System property
citrus.file.encoding=UTF-8
You can also add this property to the citrus-application.properties file as described here: http://www.citrusframework.org/reference/html/configuration.html#citrus-application-properties

Related

RESTHeart Error When Trying To Load CSV Data

I try to load csv data by using:
given().
contentType("text/csv").
body("num,name,val\n1,dror,x\n2,amir,y").
when().
log ().
all ().
post("csv?db=restheart&coll=poi&id=0").
then().log().ifError()
I get status 400:
"message": "Content-Type request header must be 'text/csv'"
although the header seems right:
Content-Type=text/csv; charset=ISO-8859-1
full log following:
Request path: http://localhost:8080/csv?db=restheart&coll=poi&id=0
Proxy: <none>
Request params: <none>
Query params: <none>
Form params: <none>
Path params: <none>
Multiparts: <none>
Headers: Accept=*/*
Content-Type=text/csv; charset=ISO-8859-1
Cookies: <none>
Body:
num,name,val
1,dror,x
2,amir,y
HTTP/1.1 400 Bad Request
Content-Encoding: gzip
X-Powered-By: restheart.org
Auth-Token-Location: /tokens/admin
Authentication-Info: nextnonce="moqjKcH05wkNMTU4NTk1NDY2OTEwNe2jU1Xuay1TK5O7rrmyiDA=",qop="auth",rspauth="d31b66e99f7dbb1c339b17741f9380c5",cnonce="f1843d31ec429268",nc=00000001
Access-Control-Expose-Headers: Location, ETag, X-Powered-By, Auth-Token, Auth-Token-Valid-Until, Auth-Token-Location
Date: Fri, 03 Apr 2020 22:57:49 GMT
Auth-Token: 16p0go30vzwa3ju0h839zn4wfnipcy914g56b5t0xkjh9yotxa
Connection: keep-alive
Access-Control-Allow-Origin: *
Auth-Token-Valid-Until: 2020-04-03T23:12:49.109466Z
Access-Control-Allow-Credentials: true
Content-Type: application/json
Content-Length: 142
{
"http status code": 400,
"http status description": "Bad Request",
"message": "Content-Type request header must be 'text/csv'",
"_links": {
"self": {
"href": "/csv"
}
}
}
Any idea what is going wrong?
I checked the code of the CsvLoader service (that handles the request) and it checks Content-Type to be exactly text/csv, so I think the problem is with the additional charset substring ; charset=ISO-8859-1
Force the Content-Type header to be exactly just text-cvs.
I also fixed this bug (see commit 47fa0821). Fix will be included in upcoming v5.0 release.

Downloading data from REST API response but getting a response code of -1

REST API details from documentation:
GET /REST/sql_snapshot/2003-03-01.sql.gz HTTP/1.1
Host: some.api.net
Authorization: Basic qpow3i12o3
The response shown below omits the message body, which contains binary compressed SQL data.
HTTP/1.1 200 OK
Date: Wed, 05 Mar 2003 10:19:46 GMT
Server: Apache/1.3.22 (Unix) (Red-Hat/Linux)Content-Type: application/octet-stream
I'm getting a response code of -1 from below.
URL url = new URL("https://some.api.net/REST/sql_snapshot/2003-03-01.sql.gz");
HttpsURLConnection connection = (HttpsURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Content-Type", "application/octet-stream");
connection.setDoOutput(true);
connection.connect();

Docker API returns 200 OK then 400 BAD REQUEST

I am writing an API client for Docker. I understood from the documentation that the API is Restful/HTTP, yet if you connect to the local daemon you have to do it over the exposed unix socket.
It all seems to work, I open a socket, send an HTTP request (which respects the specification), I receive the expected response, but also a 400 BAD REQUEST response follows immediately.
Here is the request:
GET /info HTTP/1.1
Host: localhost
Accept: application/json
And here is what I get:
HTTP/1.1 200 OK
Api-Version: 1.30
Content-Type: application/json
Docker-Experimental: false
Ostype: linux
Server: Docker/17.06.1-ce (linux)
Date: Thu, 01 Feb 2018 18:53:18 GMT
Transfer-Encoding: chunked
892
{"ID":"6MGE:35TO:BI..." ...}
0
HTTP/1.1 400 Bad Request
Content-Type: text/plain; charset=utf-8
Connection: close
400 Bad Request
First, I figured that there is a bug on my side and I am somehow sending 2 requests, but I enabled debugging and followed the logs with sudo journalctl -fu docker.service and there is exactly one request received... at least one is logged, the GET /info. I've also debugged the code and 1 single request is sent.
Any hint is greatly appreciated!
Edit: here is the client's code:
final StringBuilder hdrs = new StringBuilder();
for(final Map.Entry<String, String> header : headers) {
hdrs.append(header.getKey() + ": " + header.getValue())
.append("\r\n");
}
final String request = String.format(
this.template(), method, home, hdrs, this.readContent(content)
);
final UnixSocketChannel channel = UnixSocketChannel.open(
new UnixSocketAddress(this.path)
);
final PrintWriter writer = new PrintWriter(
Channels.newOutputStream(channel)
);
writer.print(request);
writer.flush();
final InputStreamReader reader = new InputStreamReader(
Channels.newInputStream(channel)
);
CharBuffer result = CharBuffer.allocate(1024);
reader.read(result);
result.flip();
System.out.println("read from server: " + result.toString());
It seems like you have an extra CRLF between headers and body.
private String template() {
final StringBuilder message = new StringBuilder();
message
.append("%s %s HTTP/1.1\r\n")
.append("Host: localhost").append("\r\n")
.append("%s")
.append("\r\n").append("\r\n") //one of these is superfluous, as each header line ends with "\r\n" itself
.append("%s");
return message.toString();
}
Remove one append("\r\n") after headers and see what happens.
Fixed. Initially, I thought the problem was with the line endings (that they should have been \n instead of \r\n). Turns out, the 400 BAD REQUEST occured because the Connection: close header was missing, while the Request made was being closed right after receiving the response.
More details here.

can't get web page source from url in Swift

I'm currently using SwiftHTTP for getting source of url address. I am using 'get method' for getting source code of url address which is
do {
let opt = try HTTP.GET(self.my_url_address!)
opt.start { response in
if let err = response.error {
print("error: \(err.localizedDescription)")
return
}
print(response.description)
}
} catch let error {
print("got an error creating the request: \(error)")
}
after this code run I got this output in Xcode output screen
URL: http://myweburl.com/detay/swat-under-siege.html
Status Code: 200
Headers: Content-Type: text/html
Connection: keep-alive
CF-RAY: 38391215a60e2726-FRA
Set-Cookie: ASPSESSIONIDSABBBSDT=HPKKPJGCDLKMDMILNGHPCAGD; path=/
Date: Mon, 24 Jul 2017 18:51:24 GMT
Vary: Accept-Encoding
X-Powered-By: ASP.NET Transfer-Encoding: Identity
Server: cloudflare-nginx
Content-Encoding: gzip
Cache-Control: private
The status code is 200 but the output is not the source code of url. How can I fix this?
Response is correct. I've tried requesting the website (the real one) and it works:
print(response.data.base64EncodedString())
If you decode the BASE64 data, it will render valid HTML code.
The issue seems related to encoding. After checking the website's head tag, it states that the charset is windows-1254
String(data: response.data, encoding: .windowsCP1254) // works. latin1, etc.
Your issue is similar to SWIFT: NSURLSession convert data to String

Failed to parse SDKResponse from http_response

I implemented a webhook to conform to the Actions on Google Conversation Protocol. However, when I simulate an interaction through the Web Simulator, I get the following error:
{
"response": “action name isn’t responding right now. Try again soon.\n",
"audioResponse": "...",
"debugInfo": {
"sharedDebugInfo": [
{
"name": "ExecutionResponse",
"debugInfo": "Failed to parse SDKResponse from http_response: 'HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nTransfer-Encoding: chunked\r\nConnection: keep-alive\r\nDate: Sun, 11 Dec 2016 22:54:50 GMT\r\nServer: Server-Software/1.0\r\nGoogle-Assistant-API-Version: v1\r\nVary: Accept-Encoding\r\nContent-Encoding: gzip\r\nX-Cache: Miss from CDN\r\nVia: 1.1 cdn.example.net (CDN)\r\n\r\n{\"conversation_token\":null,\"expect_user_response\":false,\"expected_inputs\":[],\"final_response\":{\"speech_response\":{\"ssml\":null,\"text_to_speech\":\"Hello!\"}}}'"
}
]
}
}
The block inside the debugInfo field is:
Failed to parse SDKResponse from http_response: 'HTTP/1.1 200 OK
Content-Type: application/json
Transfer-Encoding: chunked
Connection: keep-alive
Date: Sun, 11 Dec 2016 22:54:50 GMT
Server: Server-Software/1.0
Google-Assistant-API-Version: v1
Vary: Accept-Encoding
Content-Encoding: gzip
X-Cache: Miss from CDN
Via: 1.1 cdn.example.net (CDN)
{\"conversation_token\":null,\"expect_user_response\":false,\"expected_inputs\":[],\"final_response\":{\"speech_response\":{\"ssml\":null,\"text_to_speech\":\"Hello!\"}}}'
Fields that have null values or empty arrays should be excluded. For example, since expect_user_response is false, the expected_inputs field should not be present in the output. Use the following instead:
{
"expect_user_response": false,
"final_response": {
"speech_response": {
"text_to_speech": "Hello!"
}
}
}