mojolicious helper storing an elasticsearch connection - perl

i'm experimenting with elasticsearch within mojolicious.
I'm reasonably new at both.
I wanted to create a helper to store the ES connection and I was hoping to pass the helper configuration relating to ES (for example the node info, trace_on file etc).
If I write the following very simple helper, it works;
has elasticsearch => sub {
return Search::Elasticsearch->new( nodes => '192.168.56.21:9200', trace_to => ['File','/tmp/elasticsearch.log'] );
};
and then in startup
$self->helper(es => sub { $self->app->elasticsearch() });
however if I try to extend that to take config - like the following -
it fails. I get an error "cannot find index on package" when the application calls $self->es->index
has elasticsearch => sub {
my $config = shift;
my $params->{nodes} = '192.168.56.21:' . $config->{port};
$params->{trace_to} = $config->{trace_to} if $config->{trace_to};
my $es = Search::Elasticsearch->new( $params );
return $es;
};
and in startup
$self->helper(es => sub { $self->app->elasticsearch($self->config->{es}) });
I assume I'm simply misunderstanding helpers or config or both - can someone enlighten me?
Just fyi, in a separate controller file I use the helper as follows;
$self->es->index(
index => $self->_create_index_name($index),
type => 'crawl_data',
id => $esid,
body => {
content => encode_json $data,
}
);
that works fine if I create the helper using the simple (1st) form above.
I hope this is sufficient info? please let me know if anything else is required?

First of all, has and helper are not the same. has is a lazily built instance attribute. The only argument to an attribute constructor is the instance. For an app, it would look like:
package MyApp;
has elasticsearch => sub {
my $app = shift;
Search::ElasticSearch->new($app->config->{es});
};
sub startup {
my $app = shift;
...
}
This instance is then persistent for the life of the application after first use. I'm not sure if S::ES has any reconnect-on-drop logic, so you might need to think about it a permanent object is really what you want.
In contrast a helper is just a method, available to the app, all controllers and all templates (in the latter case, as a function). The first argument to a helper is a controller instance, whether the current one or a new one, depending on context. Therefore you need to build your helper like:
has (elasticsearch => sub {
my ($c, $config) = #_;
$config ||= $c->app->config->{es};
Search::ElasticSearch->new($config);
});
This mechanism will build the instance on demand and can accept pass-in arguments, perhaps for optional configuration override as I have shown in that example.
I hope this answers your questions.

Related

How to fetch values that are hard coded in a Perl subroutine?

I have a perl code like this:
use constant OPERATING_MODE_MAIN_ADMIN => 'super_admin';
use constant OPERATING_MODE_ADMIN => 'admin';
use constant OPERATING_MODE_USER => 'user';
sub system_details
{
return {
operating_modes => {
values => [OPERATING_MODE_MAIN_ADMIN, OPERATING_MODE_ADMIN, OPERATING_MODE_USER],
help => {
'super_admin' => 'The system displays the settings for super admin',
'admin' => 'The system displays settings for normal admin',
'user' => 'No settings are displayed. Only user level pages.'
}
},
log_level => {
values => [qw(FATAL ERROR WARN INFO DEBUG TRACE)],
help => "http://search.cpan.org/~mschilli/Log-Log4perl-1.49/lib/Log/Log4perl.pm#Log_Levels"
},
};
}
How will I access the "value" fields and "help" fields of each key from another subroutine? Suppose I want the values of operating_mode alone or log_level alone?
The system_details() returns a hashref, which has two keys with values being hashrefs. So you can dereference the sub's return and assign into a hash, and then extract what you need
my %sys = %{ system_details() };
my #loglevel_vals = #{ $sys{log_level}->{values} };
my $help_msg = $sys{log_level}->{help};
The #loglevel_vals array contains FATAL, ERROR etc, while $help_msg has the message string.
This makes an extra copy of a hash while one can work with a reference, as in doimen's answer
my $sys = system_details();
my #loglevel_vals = #{ $sys->{log_level}->{values} };
But as the purpose is to interrogate the data in another sub it also makes sense to work with a local copy, what is generally safer (against accidentally changing data in the caller).
There are modules that help with deciphering complex data structures, by displaying them. This helps devising ways to work with data. Often quoted is Data::Dumper, which also does more than show data. Some of the others are meant to simply display the data. A couple of nice ones are Data::Dump and Data::Printer.
my $sys = system_details;
my $log_level = $sys->{'log_level'};
my #values = #{ $log_level->{'values'} };
my $help = $log_level->{'help'};
If you need to introspect the type of structure stored in help (for example help in operating_mode is a hash, but in log_level it is a string), use the ref builtin func.

How to deploy a test database for DBIx Class

I have Mojolicious app with a test suite using Test::Class::Moose. Using DBIx::Class to interact with my database, is there a way to setup an in-memory database that I can add fixture data to?
I'd like to use an in memory database because it'll mean that the application will have less setup configuration. I do know how to setup an actual SQLite database for testing but managing that for testing along with a mySQL database for production doesn't sound like easy management (eg "oh no, forgot to rebuild the testing database").
Loading data from fixtures seems ideal, that way you have more control over what is actually in the database. Example, you find a bug with a row that contains certain data, add a row like that to your fixture file and test until successful.
Now how to actually set up an in-memory database using DBIx? :)
Right now this is how I'm building the schema for my Test::Class::Moose suite:
has cats => (
is => 'ro',
isa => 'Test::Mojo::Cats',
default => sub { return Test::Mojo::Cats->new(); }
);
has schema => (
is => 'ro',
lazy => 1,
builder => '_build_schema_and_populate',
);
sub _build_schema_and_populate {
my $test = shift;
my $config = $test->cats->app->config();
my $schema = Cat::Database::Schema->connect(
$config->{db_dsn},
$config->{db_user},
$config->{db_pass},
{
HandleError => DBIx::Error->HandleError,
unsafe => 1
}
);
require DBIx::Class::DeploymentHandler;
my $dh = DBIx::Class::DeploymentHandler->new({
schema => $schema,
sql_translator_args => { add_drop_table => 0 },
schema_version => 3,
});
$dh->prepare_install;
$dh->install;
my $json = read_file $config->{fixture_file};
my $fixtures = JSON::decode_json($json);
$schema->resultset($_)->populate($fixtures->{$_}) for keys %{$fixtures};
return $schema;
}
Where my config specifies dbi:SQLite:dbname=:memory: as the database dsn.
When running the test suite, the tables don't seem to be loaded, as I get errors stating the table does not exist, eg Can't locate object method "id" via package "no such table: cats"
Is there some extra setup that I'm not doing when wanting to deploy to an in-memory database?
Thanks
PS:
Doing the following works in a single script, I don't know if I'm doing something that Test::Class::Moose or Mojo doesn't like with the above
#!/usr/bin/perl
use Cats::Database::Schema;
use File::Slurp;
use JSON;
use Data::Dumper;
my $schema = Cats::Database::Schema->connect(
'dbi:SQLite:dbname=:memory:', '', ''
);
my $json = read_file('../t/fixtures.json');
my $fixtures = JSON::decode_json($json);
$schema->deploy();
$schema->resultset($_)->populate($fixtures->{$_}) for keys %{$fixtures};
# returns fixture data fine
# warn Dumper($schema->resultset('User')->search({}));
I believe I figured it out
The way I use the DBIx schema in the app is to instantiate it within a base controller which all sub controllers inherit. No matter how I built and populated the in memory database in the Test::Class::Moose object, it would not be using the instance specified there, instead it would be using the one specified in the base controller.
the solution was to move the schema construction up one level (from controller to the app root) as an attribute, allowing me to override it in Test Mojo to use the in memory db.

Join attempt throwing exceptions

I'm sure I'm overlooking something glaringly obvious and I apologize for the newbie question, but I've spent several hours back and forth through documentation for DBIx::Class and Catalyst and am not finding the answer I need...
What I'm trying to do is automate creation of sub-menus based on the contents of my database. I have three tables in the database to do so: maps (in which sub-menu items are found), menus (contains names of top-level menus), maps_menus (assigns maps to top-level menus). I've written a subroutine to return a hash of resultsets, with the plan of using a Template Toolkit nested loop to build the top-level and sub-menus.
Basically, for each top-level menu in menus, I'm trying to run the following query and (eventually) build a sub-menu based on the result:
select * FROM maps JOIN maps_menus ON maps.id_maps = maps_menus.id_maps WHERE maps_menus.id_menus = (current id_menus);
Here is the subroutine, located in lib/MyApp/Schema/ResultSet/Menus.pm
# Build a hash of hashes for menu generation
sub build_menu {
my ($self, $maps, $maps_menus) = #_;
my %menus;
while (my $row = $self->next) {
my $id = $row->get_column('id_menus');
my $name = $row->get_column('name');
my $sub = $maps_menus->search(
{ 'id_maps' => $id },
{ join => 'maps',
'+select' => ['maps.id_maps'],
'+as' => ['id_maps'],
'+select' => ['maps.name'],
'+as' => ['name'],
'+select' => ['maps.map_file'],
'+as' => ['map_file']
}
);
$menus{$name} = $sub;
# See if it worked...
print STDERR "$name\n";
while (my $m = $sub->next) {
my $m_id = $m->get_column('id_maps');
my $m_name = $m->get_column('name');
my $m_file = $m->get_column('map_file');
print STDERR "\t$m_id, $m_name, $m_file\n";
}
}
return \%menus;
}
I am calling this from lib/MyApp/Controller/Maps.pm thusly...
$c->stash(menus => [$c->model('DB::Menus')->build_menu($c->model('DB::Map'), $c->model('DB::MapsMenus'))]);
When I attempt to pull up the page, I get all sorts of exceptions, the top-most of which is:
[error] No such relationship maps on MapsMenus at /home/catalyst/perl5/lib/perl5/DBIx/Class/Schema.pm line 1078
Which, as far as I can tell, originates from the call to $sub->next. I take this as meaning I'm doing my query incorrectly and not getting the results I think I should be. However, I'm not sure what I'm missing.
I found the following lines, defining the relationship to maps, in lib/MyApp/Schema/Result/MapsMenus.pm
__PACKAGE__->belongs_to(
"id_map",
"MyApp::Schema::Result::Map",
{ id_maps => "id_maps" },
{ is_deferrable => 1, on_delete => "CASCADE", on_update => "CASCADE" },
);
...and in lib/MyApp/Schema/Result/Map.pm
__PACKAGE__->has_many(
"maps_menuses",
"MyApp::Schema::Result::MapsMenus",
{ "foreign.id_maps" => "self.id_maps" },
{ cascade_copy => 0, cascade_delete => 0 },
);
No idea why it's calling it "maps_menuses" -- that was generated by Catalyst. Could that be the problem?
Any help would be greatly appreciated!
I'd suggest using prefetch of the two relationships which form the many-to-many relationship helper and maybe using HashRefInflator if you don't need access to the row objects.
Note that Catalyst doesn't generate a DBIC (which is btw the official abbreviation for DBIx::Class, DBIx is a whole namespace) schema, SQL::Translator or DBIx::Class::Schema::Loader do. Looks at the docs of the module you've used to find out how to influence its naming.
Also feel free to change the names if they don't fit you.

Route to static file in Mojo

I have small app based on mojolicious. And I have index.html in public dir. I want to have route to this file when user asks for '/'.
I wrote two solution, but I don't like them.
First solution - add simple controller.
sub stratup {
//...
$r->get('/')->to('general#index_html');
//...
}
package MyPackage::General;
use Mojo::Base 'Mojolicious::Controller';
use strict;
use warnings;
sub index_html {
my $self = shift;
$self->render_static('index.html');
return;
}
1;
Second solution - add hook
sub startup {
my $self = shift;
$self->hook(before_dispatch => sub {
my $self = shift;
if ($self->req->url eq '/') {
$self->req->url( Mojo::URL->new('/index.html') );
}
});
What I want:
$r->get('/')->to('/index.html');
or something like that.
P.S. I know, than usualy nginx/apache do it, but I use morbo to run code.
You want:
$r->get('...')->to(cb => sub {
my $c = shift;
$c->reply->static('index.html')
});
(As long as you're after Mojolicous 5.45 2014-09-26)
By far the simplest way is
get "/" => "index";
I'll dig this up from the graveyard, why not.
I found myself similarly trying to serve a static html file in a docker container that I had using to serve both a Mojolicious REST API and a Vue.js front end. After searching around and piecing sporadic information together, this is what seems to work for me.
** disclaimer: I have not fully tested this with Vue routing and other aspects as yet.
My directory structure:
/app
/app/script
/app/modules/ui
/app/modules/ui/dist
From the command line the app directory, using morbo to test:
morbo script/ui.pl
ui.pl script
#!/usr/bin/env perl
use Mojolicious::Lite -signatures;
use Mojo::File qw(curfile);
use v5.25;
my $app = app;
my $static = $app->static;
push #{$static->paths}, curfile->dirname->sibling('modules/ui/dist')->to_string;
any '/' => sub {
my $c = shift;
my $content = $static->file("/index.html")->slurp;
$c->render(text => $content);
};
$app->start;
Using a combo of information from https://metacpan.org/pod/Mojolicious::Static and basic routing information at https://docs.mojolicious.org/Mojolicious/Lite, I could get the vue.js index page to render as expected.
** UPDATED A DAY LATER **
As it turns out, there is an easier way, though not clearly documented. If you place the static files inside your public folder, you can use the default helpers included with Mojolicious to render the files. The documentation refers to it here, https://docs.mojolicious.org/Mojolicious/Guides/Rendering#Serving-static-files, but it's not very clear on how to make it happen.
I tooled around some, but it took browsing the code of Controller.pm of for Mojolicious to sort it out. This section of the POD led me to determine how to get the reply object:
=head2 helpers
my $helpers = $c->helpers;
Return a proxy object containing the current controller object and on which helpers provided by /app can be called. This includes all helpers from Mojolicious::Plugin::DefaultHelpers and Mojolicious::Plugin::TagHelpers.
# Make sure to use the "title" helper and not the controller method
$c->helpers->title('Welcome!');
# Use a nested helper instead of the "reply" controller method
$c->helpers->reply->not_found;
Based on this, I can drop my files into the public folder:
/app/public/index.html
Then modify my controller to match:
# https://docs.mojolicious.org/Mojolicious/Guides/Rendering#Serving-static-files
any '/' => sub {
my $c = shift;
$c->helpers->reply->static('index.html');
};

Mojolicious custom sessions

I am trying to use database sessions with Mojolicious instead of the builtin ones that are working with signed cookies.
In the startup subroutine I have something like:
my $dbh = DBI->connect(
$config->{database}->{dsn},
$config->{database}->{user},
$config->{database}->{password},
);
my $session = MojoX::Session->new(
store => [dbi => {dbh => $dbh}], # use MojoX::Session::Store::Dbi
transport => 'cookie', # this is by default
ip_match => 1
);
(ref($self))->attr( 'session' => sub {
return $session;
} );
And I want to use the session object like $self->session or $self->app->session in controllers.
Unfortunately it doesn't work - it uses previous sessions (from different browsers).
This drives me crazy - all I tried today was to make this work - I've read all the documentation available, also the source of MojoX::Session and all its related classes, tried in about 923847293847239847 ways to make it work, but nothing seems to do it.
PS: I created the session table in the db.
Do you know what I should do in order to be able to use DB sessions with Mojolicious?
You can connect MojoX::Session to the application as a plugin in a startup function.
use Mojolicious::Plugin::Session;
[...]
sub startup {
my $self = shift;
[...]
$self->plugin( session => {
stash_key => 'mojox-session',
store => [dbi => {dbh => $dbh}], # use MojoX::Session::Store::Dbi
transport => 'cookie',
ip_match => 1
});
[...]
Afterwards, you'll have access to session through stash key 'mojox-session' in controllers.
For example:
$self->stash('mojox-session')->data('something');
You can use whatever sort of session backend you like. Just create your own controller base class derived from Mojolicious::Controller and override session(), like so:
package NiceController;
use Mojo::Base 'Mojolicious::Controller';
sub session { # custom code here }
1;
then in startup(), set your controller class as the default:
$self->controller_class('NiceController');
Finally, make sure your application controllers inherit from NiceController instead of Mojolicious::Controller
It's probably a good idea to make your overridden session() function work just like the built-in one, to avoid future confusion.
-xyz
The $app->session method is reserved for using the built-in sessions.
You should take a look at the Mojolicious helpers and you probably want to use a different method name to avoid conflict.