REST::Client | unable to parse GET response JSON object - rest

Question

This is the first time i am writing a Perl client to consume a REST service. I am using REST::Client and JSON perl module. The web service returns data in JSON format. The problem is when I try to use from_json or decode_json method on client->responseContent() method, I am getting an error saying
"malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "HTTP/1.1 200 \r\nCon..."
The web service is ofcourse a stable one and works fine with other languages REST clients.
After debugging the issue I found that client->responseContent() not only contains the JSON data but also the header information hence from_json is unable to parse it. Below is the snippet of the code:
my $url = "/data";
my $client = REST::Client->new();
$client->setHost($host);
my $headers = {Accept => 'application/json'};
$client->GET($url, $headers);
my $response = from_json($client->responseContent());
Not able to figure out this thing from two days now :-(
Here is the dump of "$client->{_res}->dump"
Fri Feb 23 09:38:35 2018: HTTP/0.9 200 EOF
Client-Date: Fri, 23 Feb 2018 09:38:35 GMT
Client-Peer: 45.32.84.105:8282
Client-Response-Num: 1
HTTP/1.1 200 \r
Content-Type: application/json;charset=UTF-8\r
Transfer-Encoding: chunked\r
Date: Fri, 23 Feb 2018 09:38:33 GMT\r
Connection: close\r
\r
2000\r
[{"REGION":"AP","REMARK":null,"STATUS":"PROD","UPDATED_TIME":null,"UPDATED_BY":null,"ROUTE_ID":1,"ROUTE_ID_VER":20150310,"USER_ROUTE_LOGIC":"|CAPTIVE|","USER_DEST":null,"USER_ORDSIZE_TYPE":null,"MIN_USER_ORDSIZE_VAL":0,"MAX_USER_ORDSIZE_VAL":100,"TAG_775":"|1|","CROSS_CURRENCY":"|Y|N|","TAG_12703":"|PB-CS|","COUNTRY":"|AU|HK|ID|IN|JP|KR|MY|SG|","TAG_12207":...
(+ 423449 more bytes not shown)
Even when the transfer encoding is not chunked, I am getting the same issue;
Fri Feb 23 10:40:20 2018: HTTP/1.1 200 ^M
Content-Type: application/json;charset=utf-8^M
Content-Length: 1618^M
Date: Fri, 23 Feb 2018 10:40:20 GMT^M
Connection: close^M
^M
{ "data":[ {
"REGION" : "AP",
"REMARK" : "",
"STATUS" : "PROD",
"UPDATED_TIME" : "",
"UPDATED_BY" : "",
Ultimately, solved using curl command for now (Actual code snippet below):
my $command = "curl '$url'";
my $rules = qx/$command/;

Answer 1

I have a similar issue that I am working through. I suspect (Have not verified yet) that the root cause is that the server is not passing the Encoding-Content Response Header, therefore LWP::UserAgent and HTTP::Response are not attempting to return the decoded string. I am going to try to isolate where this decision is being made and see what options are available and request a patch.
Perl REST::Client - Garbage data in response