Catalyst::Controller::REST returns 415 unsupported media type - perl

I want to write restful api via Catalyst, and use for this [Catalyst::Controller::REST][1].
I have written that code.
package addressbook::Controller::REST;
use strict;
use warnings;
use base qw(Catalyst::Controller::REST);
sub user :Local :ActionCLass('REST') :Args(1){
my( $self, $c, $id ) = #_;
$c->stash( id => $id );
}
# Get user
sub user_GET {
my ( $self, $c ) = #_;
my $user = $c->model('DB::User')->find( { id => $c->stash->{id} } );
if ( $user ){
$self->status_ok($c, entity => { firstname => $user->firstname } );
}
else {
$self->status_not_found($c, message => 'No matching user');
}
}
__PACKAGE__->config(default => 'text/x-json');
1;
Then i run the server, go to localhost:3000/rest/user/1 (i have user by that id) and get
Cannot find a Content-Type supported by your client.
I try to set PACKAGE->config application/json, text/xml, text/html, text/x-yaml ... but it's not help.
Any ideas?
Thanks.

As implemented, Catalyst Action REST does content negotiation on the request to determine the serialization method to be used. The default setting is only a fallback and normally your request in the real world will contain a content type.
Documentation on the supported content types and how to map in new deserializers can be found here:
enter link description here. Note also that recent versions remove built in support for YAML
which would have been the default response to text/html if you just requested the url in your browser.
Use curl or test from a real browser using Javascript
curl -H "Content-Type: application/json" http://localhost:3000/rest/user/1
Also check your installed version of Catalyst which will appear on the info line as you start up the server.

Related

How to access session data from test?

The Mojolicious framework states next:
Any aspect of the application (helpers, plugins, routes, etc.) can be introspected from Test::Mojo through the application object.
But when helper, for example, $c->current_user deals with session it fails.
The session data is not available and I can not access it from test:
$t->app->session # {}
Thus $t->app->current_user fails too.
How to access session data from test?
UPD The test
use Mojo::Base -strict;
use Mojolicious::Lite;
use Test::More;
use Test::Mojo;
get '/set_session' => sub {
my $c = shift;
$c->session->{ user_id } = 1;
$c->render( text => $c->session->{ user_id } );
};
get '/get_session' => sub {
my $c = shift;
$c->render( text => $c->session->{ user_id } );
};
my $t = Test::Mojo->new;
$t->get_ok( '/set_session' )->status_is(200);
is $t->app->session->{ user_id }, 1, 'Session available from test script';
$t->get_ok( '/get_session' )->status_is(200)
->content_is( 1 );
done_testing();
UPD test result
ok 1 - GET /set_session
ok 2 - 200 OK
not ok 3 - Session available from test script
# Failed test 'Session available from test script'
# at t/session.t line 22.
# got: undef
# expected: '1'
ok 4 - GET /get_session
ok 5 - 200 OK
ok 6 - exact match for content
1..6
# Looks like you failed 1 test of 6.
UPD
It seems that Mojo::Test object should save session object in addition to the request and response objects from the previous transaction
To test helpers in context of last request I write next role:
package Test::Mojo::Role::Helper;
use Mojo::Base -role;
sub helper {
my( $t ) = #_;
$t->tx->req->cookies( #{ $t->tx->res->cookies } );
$t->app->build_controller( $t->tx );
}
1;
Then use it as next:
use Test::Mojo;
my $t = Test::Mojo->with_roles( '+Helper' )->new( 'MyApp' );
$t->post_ok( '/login', json => { contact => $user_c, password => $user_p } )
->status_is( 200 );
is $t->helper->uid, 1, "Authorized user has identificator";
is $t->helper->role, 'user', "Authorized user has 'user' privilege";
UPD More robust solution
package Test::Mojo::Role::Helper;
use Mojo::Base -role;
my $req_context; # Here is controller object
sub helper { $req_context }
sub hook_context {
my( $t ) = #_;
$t->app->hook( after_dispatch => sub{ $req_context = shift });
$t;
}
1;
The testing is same with next small difference. When application is constructed we should hook to after_dispatch event:
my $t = Test::Mojo
->with_roles( '+Helper' )
->new( 'App' )
->hook_context;
The Test::Mojo class does not give you direct access to the session contents. The test class represents a client of your Mojolicious application, and the client does not have direct access to the session cookie either (well, it's just base64-encoded JSON so it's not exactly secret, but still …).
The “proper” way to test the session is to check that the app behaves correctly regarding the session, not just to check that the session was set to some value. That's effectively what your /get_session endpoint does. Of course you shouldn't just add such an endpoint for testing, but consider how the session fits into your requirements. E.g. as a BDD-style scenario:
Feature: the secret page
there is a secret page that should be only visible to logged-in users.
Background:
Given a user "test:test123"
Given a new client
Scenario: users cannot see the page when they are not logged in
When I visit the /secret page
Then I get a 404 response
Scenario: users can see the page after logging in
Given I log in as "test:test123"
When I visit the /secret page
Then I see "this is the secret"
The $t->app->session does not contain the session because the session data is loaded into the controller's stash. This only exists for the duration of the request. In particular app->session is merely a helper that delegates to the current controller, not a primary method of the application.
If you really need to peek into the session cookie, this might be the least insane way to do it, short of inflating a controller object:
my ($session) = grep { $_->name eq $t->app->sessions->cookie_name } $t->ua->cookie_jar->all->#*;
$session = $session->value =~ s/--[^-]+$//r; # strip signature
$session =~ tr/-/=/;
$session = $t->app->sessions->deserialize->(Mojo::Util::b64_decode $session);

How do I do a chunked transfer-encoding upload with WWW:Mechanize?

I'm attempting to use a particular web service, and I can successfully perform the upload with the following command:
curl -X POST --header "Transfer-Encoding: chunked" -d #Downloads/file.pdf https://some.webservice/upload
I get back a json response indicate success.
However, I'm unable to figure out how to do the same with WWW::Mechanize.
$mech->post("https://" . $server . "/upload", Content_Type => 'multipart/form-data', Content => [upID => $upid, name => $dlfile, userID => 0, userK => 0, file_0 => [$dlfile]]);
This receives a similar json response with a big fat error message in it.
Do I need to explicitly set the Transfer-Encoding header first? Is there some other trick to it? Google's not shedding much light on this, Perlmonks neither, and the documentation's a little obtuse.
You can do it using HTTP::Request::StreamingUpload
my $starttime = time();
my $req = HTTP::Request::StreamingUpload->new(
POST => $url,
path => $file,
headers => HTTP::Headers->new(
'Transfer-Encoding' => 'chunked'
),
);
my $gen = $req->content;
die unless ref($gen) eq "CODE";
my $total = 0;
$req->content(sub {
my $chunk = &$gen();
$total += length($chunk);
print "\r$total / $size bytes ("
. int($total/$size*100)
. "%) sent, "
. int($total/1000/(time()-$starttime+1))
. " k / sec ";
return $chunk;
});
my $resp = $ua->request($req);
print "\n";
unless ($resp->is_success) {
die "Failed uploading the file: ", $resp->status_line;
}
my $con = $resp->content;
return $con;
Do you really need WWW::Mechanize? It is a subclass of LWP::UserAgent with additional functionality that gives browser-like functionality like filling in and submitting forms, clicking links, a page history with a "back" operation etc. If you don't need all of that then you may as well use LWP::UserAgent directly
Either way, the post method is inherited unchanged from LWP::UserAgent, and it's fine to use it directly as you have done
The way to send a chunked POST is to set the Content to a reference to a subroutine. The subroutine must return the next chunk of data each time it is called, and finally ann empty string or undef when there is no more to send
Is the data supposed to be a JSON string?
It's easiest to write a factory subroutine that returns a closure, like this
sub make_callback {
my ($data) = shift;
sub { substr($data, 0, 512, "") }
}
Then you can call post like this
my $payload = to_json(...);
$mech->post(
"https://$server/upload",
Content_Type => 'multipart/form-data',
Content => make_callback($payload)
);
Please be aware that all of this is untested

Upload media to twitter using perl

Has anyone successfully uploaded media to Twitter, ie posted a tweet with an image using Perl?
I would like to upload images from my blog without doing it all manually.
update_with_media(status, media[]) is deprecated, and crashes. Twitter says to use plain update(), passing a media id. However it is first necessary to upload the media to get the id, and I can find no code examples.
Any experience in this area?
Cheers,
Peter
I thought I'd add an update here even though the thread is quite old. I too was looking for an answer to how to use Perl to upload media to twitter.
Net::Twitter is perfectly capable of sending PNG images up to Twitter, though the documentation is poor. The OP is correct that update_with_media is deprecated and crashed for us.
You do need to use the $nt->upload AND $nt->update methods combined. You upload the RAW PNG file encoded with base64, I could not get the RAW PNG file upload to work but no issues when base64 encoded. The upload method returns, on success, a JSON structure which has the media_ids in. These id's are then passed with the $nt->update method. Here's some actual code
use Net::Twitter;
use File::Slurp;
use MIME::Base64;
use Data::Dumper;
my $nt = Net::Twitter->new(
ssl => 1,
traits => [qw/API::RESTv1_1/],
consumer_key => $config->{twitter}{api_key},
consumer_secret => $config->{twitter}{api_secret},
access_token => $config->{twitter}{access_token},
access_token_secret => $config->{twitter}{access_token_secret},
);
my $filename = "<somelink to a PNG file>";
my $file_contents = read_file ($filename , binmode => ':raw');
my $status = $nt->upload(encode_base64($file_contents));
print "SendTweet: status = ".Dumper($status);
my $status2;
eval {
$status2 = $nt->update({status => $s , media_ids => $status->{media_id} });
print "status2 = ".Dumper($status2);
};
if ($#)
{
print "Error: $#\n";
}
The code is pulled directly from our working test code so should work. This code is purely proof of concept so you will need to add in all your twitter authentication etc.
We have only done PNG files but I see no reason why video etc shouldn't work fine as we simply followed the Twitter docs.
Rob
In the end I used readpipe with twurl. Had I known about twurl in the first place I likely would not have bothered with Net::Twitter! twurl works great, and returns a full json string to tell you what went wrong, if anything.
Pre-requisite: you need to get the Oauth keys for your twitter account. (See here - https://developer.twitter.com/en/docs/basics/authentication/guides/access-tokens.html). There are some step by step online exmaples elsewhere also that can help.
Here's the code I ended up with.
First call the tweet module (and these are not valid keys by the way - just insert yours)
use Jimtweet;
my $tweet=Jimtweet->new();
$tweet->consumer_key('KvfevhjwkKJvinvalidkeycvhejwkKJVCwe');
$tweet->consumer_secret('KvfevhjwkKJvcvnvalidkeyhejwkKJVCwe');
$tweet->oauth_token('KvfevhjwkKJvcvhenvalidkeyjwkKJVCwKvfevhjwkKJvcvhejwkKJVCweVU');
$tweet->oauth_token_secret('KvfevhjwkKJvcvhejwnvalidkeykKJVCwe');
my $res = $tweet->update_status($message, $ENV{DOCUMENT_ROOT}.$li);
Jimtweet is a free module I found online (I forget where), but I had to modify it to do the image upload. It follows below.
$message is the text status message to send
$li contains the local path to the file I want to upload. This is a file local on the server. $ENV{DOCUMENT_ROOT} contains the server path to the public HTML files on my website.
$res contains a JSON reply from twitter you can look at if you need to.
If you want to use this, cut & paste everything from 'package Jimtweet;' and on into a file called Jimtweet.pm which the above code should use. If your perl installation can't find the module try adding the line use lib "/home/your/path/to/jimtweet/directory;" before the use Jimtweet; line.
Twitter requires an image to be uploaded, it then returns a media_id, then you include the media_id in a regular message you want to post. See Jimtweet package below:
package Jimtweet;
#####JimTweet 0.1 By James Januszka 2010
#####Email:jimjanuszka#gmail.com
#####Twitter:#jamesjanuszka
#####Modifications by John Bell to include image upload. Twitter: #BellUkcbajr
use strict;
use warnings;
use LWP;
use HTTP::Headers;
use URI::Encode qw(uri_encode);
use URI::Escape qw(uri_escape);
use Digest::HMAC_SHA1;
####Constructor####
sub new {
my $self={};
$self->{OAUTH_VERSION}=uri_escape("1.0");
$self->{OAUTH_SIGNATURE_METHOD}=uri_escape("HMAC-SHA1");
$self->{OAUTH_TIMESTAMP}=undef;
$self->{OAUTH_NONCE}=undef;
$self->{AGENT}="jimtweet/0.1";
#####################
#Use this for status updates
$self->{URLx}="https://api.twitter.com/1.1/statuses/update.json";
#####################
#Use this for image upload
$self->{URL}="https://upload.twitter.com/1.1/media/upload.json";
$self->{BROWSER}=LWP::UserAgent->new(agent =>$self->{AGENT});
$self->{CONSUMER_KEY}=undef;
$self->{CONSUMER_SECRET}=undef;
$self->{OAUTH_TOKEN}=undef;
$self->{OAUTH_TOKEN_SECRET}=undef;
$self->{STATUS}=undef;
$self->{MEDIAurl}=undef;
bless($self);
return $self;
}
sub consumer_key{
my $self=shift;
if (#_) { $self->{CONSUMER_KEY}=uri_escape(shift) }
return $self->{CONSUMER_KEY};
}
sub consumer_secret{
my $self = shift;
if (#_) { $self->{CONSUMER_SECRET}=uri_escape(shift) }
return $self->{CONSUMER_SECRET};
}
sub oauth_token{
my $self = shift;
if (#_) { $self->{OAUTH_TOKEN}=uri_escape(shift) }
return $self->{OAUTH_TOKEN};
}
sub oauth_token_secret{
my $self = shift;
if (#_) { $self->{OAUTH_TOKEN_SECRET}=uri_escape(shift) }
return $self->{OAUTH_TOKEN_SECRET};
}
sub update_status(#){
sleep(2);
my $self = shift;
if (#_) { $self->{STATUS}=uri_escape(shift) }
if (#_) { $self->{MEDIAurl}=shift }
#Got parameters. Now create the POST to upload an image
my $seconds = time();
$self->{OAUTH_TIMESTAMP}=uri_escape($seconds);
$self->{OAUTH_NONCE}=$self->{OAUTH_TIMESTAMP};
#####################
#Use this for uploads. Parameters have to be in alphabetical order!
my $query=qq(oauth_consumer_key=$self->{CONSUMER_KEY}&oauth_nonce=$self->{OAUTH_NONCE}&oauth_signature_method=$self->{OAUTH_SIGNATURE_METHOD}&oauth_timestamp=$self->{OAUTH_TIMESTAMP}&oauth_token=$self->{OAUTH_TOKEN}&oauth_version=$self->{OAUTH_VERSION});
my $sig="POST&";
$sig .=uri_encode($self->{URL},1);
$sig .="&";
$sig .=uri_encode($query,1);
my $sig_key=$self->{CONSUMER_SECRET};
$sig_key .="&";
$sig_key .=$self->{OAUTH_TOKEN_SECRET};
my $hmac = Digest::HMAC_SHA1->new($sig_key);
$hmac->add($sig);
my $oauth_signature_base64=$hmac->b64digest;
$oauth_signature_base64 .="=";
my $utf8_oauth_signature_base64=uri_escape($oauth_signature_base64);
my $header=qq(OAuth oauth_nonce="$self->{OAUTH_NONCE}", oauth_signature_method="$self->{OAUTH_SIGNATURE_METHOD}", oauth_timestamp="$self->{OAUTH_TIMESTAMP}", oauth_consumer_key="$self->{CONSUMER_KEY}", oauth_token="$self->{OAUTH_TOKEN}", oauth_signature="$utf8_oauth_signature_base64", oauth_version="$self->{OAUTH_VERSION}");
my $res = $self->{BROWSER}->post(
$self->{URL},
'Authorization' => $header,
'content-type' => 'form-data',
'Content' => [ media => ["$self->{MEDIAurl}"] ]
);
use JSON;
my $response = decode_json ($res->content);
my $media_id = $response->{'media_id'};
$seconds = time();
$self->{OAUTH_TIMESTAMP}=uri_escape($seconds);
$self->{OAUTH_NONCE}=$self->{OAUTH_TIMESTAMP}
my $queryx=qq(media_ids=$media_id&oauth_consumer_key=$self->{CONSUMER_KEY}&oauth_nonce=$self->{OAUTH_NONCE}&oauth_signature_method=$self->{OAUTH_SIGNATURE_METHOD}&oauth_timestamp=$self->{OAUTH_TIMESTAMP}&oauth_token=$self->{OAUTH_TOKEN}&oauth_version=$self->{OAUTH_VERSION}&status=$self->{STATUS});
my $sigx="POST&";
$sigx .=uri_encode($self->{URLx},1);
$sigx .="&";
$sigx .=uri_encode($queryx,1);
my $hmacx = Digest::HMAC_SHA1->new($sig_key);
$hmacx->add($sigx);
my $oauth_signature_base64x=$hmacx->b64digest;
$oauth_signature_base64x .="=";
my $utf8_oauth_signature_base64x=uri_escape($oauth_signature_base64x);
my $headerx=qq(OAuth oauth_nonce="$self->{OAUTH_NONCE}", oauth_signature_method="$self->{OAUTH_SIGNATURE_METHOD}", oauth_timestamp="$self->{OAUTH_TIMESTAMP}", oauth_consumer_key="$self->{CONSUMER_KEY}", oauth_token="$self->{OAUTH_TOKEN}", oauth_signature="$utf8_oauth_signature_base64x", oauth_version="$self->{OAUTH_VERSION}");
#And done generating content. Now to POST to twitter.
$res = $self->{BROWSER}->post(
$self->{URLx},
'Authorization' => $headerx,
'content-type' => 'application/x-www-form-urlencoded',
'Content' => qq(media_ids=$media_id&status=$self->{STATUS})
);
return $res;
}
####Footer####
1; #so the require or use succeeds

Perl Mojolicious EventSource gzip output

I have a working Mojolicious server that provides data with HTML5 EventSource.
Works well but I would like the data to be encoded in gzip format.
Sending compressed data with a write gives a CONTENT_DECODING_FAILED in Chrome Dev tools.
Using the suggested method with "hook after_render" does not seem to work with event-stream.
How can I send gzip encoded data using EventSource and Mojolicious ?
use Mojolicious::Lite;
use Mojo::Redis;
use IO::Compress::Gzip 'gzip';
my $redis = Mojo::Redis->new;
get 'radar_events' => sub {
my $c = shift;
$c->render_later;
$c->inactivity_timeout(300);
$c->res->headers->content_type('text/event-stream');
$c->res->headers->cache_control('no_cache');
$c->res->headers->content_encoding('gzip');
$c->res->headers->header( 'Access-Control-Allow-Origin' => '*' );
my $id = Mojo::IOLoop->recurring(
5 => sub {
$c->delay(
sub {
my $delay = shift;
$redis->get( 'radar', $delay->begin );
},
sub {
my ( $delay, $jstring ) = #_;
my $buf = "event:rupdate\ndata: ".$jstring."\n\n";
gzip \$jstring => \my $buf;
$c->write($buf);
}
);
}
);
$c->on( finish => sub { Mojo::IOLoop->remove($id) } );
};
app->start;
I think it is not possible the way you do it. Content-Encoding is the encoding of whole body, which includes the event:rupdate\n and data:.... messages. Also, each event message is a single line (see the specification), so you cannot transfer binary data (like compressed data) this way.
I don't know if the browser support Content-Encoding with event streams. But if they do, you would need to provide a single gzip stream, starting with the begin of the body and only ending once your are done. And since gzip buffers data to achieve better compression you would need to explicitly flush the gzip object after each event you've added.

Posting to a WEB API from Perl results in a null value in [FromBody]

We have created a WEB API (in .NET framework 4.0) and gave the endpoint info to one of our clients. They created a program in Perl that posts to our endpoint.
Every post they have made so far arrives into our endpoint as null. When we initially started programming, we had that same issue in JQuery when posting by means of $.ajax. We solved it by adding a '=' at the beginning of the post data.
The Perl code they have submitted is the following:
sub _postPackages {
my ($self,$dataToSend) = #_;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
$ua->agent("integrationapp/1.0 ");
# Create a request
my $req = HTTP::Request->new(POST => $self->{postAddress} );
$req->content_type("application/json;charset=utf-8");
$req->content($dataToSend->{data});
#print Data::Dumper->Dump([$req]);
# Pass request to the user agent and get a response back
my $res = $ua->request($req);
where postAddress is our endpoint and $dataToSend is the message data. Is it possible that they need to add the '=' at the beginning of the $dataToSend message.
Any help will be greatly appreciated.
This is a bit of pseudo code here..
But I'm guessing you want to do something like this:
# some post sub
my ($self, $data) = #_;
my $ua = $self->get_user_agent();
my $json_xs = $self->get_json_xs();
my $json_encoded = $json_xs->utf8->encode($data);
$self->set_post_data($json_encoded);
$self->set_api_call();
my $response_body = $ua->post(
$self->get_api_call(),
'Content' => $self->get_post_data(),
'Content-type' => "application/json;charset=UTF-8"
);
print STDERR "POSTING NEW RESOURCE: " . Dumper($self);