How to deploy a test database for DBIx Class - perl

I have Mojolicious app with a test suite using Test::Class::Moose. Using DBIx::Class to interact with my database, is there a way to setup an in-memory database that I can add fixture data to?
I'd like to use an in memory database because it'll mean that the application will have less setup configuration. I do know how to setup an actual SQLite database for testing but managing that for testing along with a mySQL database for production doesn't sound like easy management (eg "oh no, forgot to rebuild the testing database").
Loading data from fixtures seems ideal, that way you have more control over what is actually in the database. Example, you find a bug with a row that contains certain data, add a row like that to your fixture file and test until successful.
Now how to actually set up an in-memory database using DBIx? :)
Right now this is how I'm building the schema for my Test::Class::Moose suite:
has cats => (
is => 'ro',
isa => 'Test::Mojo::Cats',
default => sub { return Test::Mojo::Cats->new(); }
);
has schema => (
is => 'ro',
lazy => 1,
builder => '_build_schema_and_populate',
);
sub _build_schema_and_populate {
my $test = shift;
my $config = $test->cats->app->config();
my $schema = Cat::Database::Schema->connect(
$config->{db_dsn},
$config->{db_user},
$config->{db_pass},
{
HandleError => DBIx::Error->HandleError,
unsafe => 1
}
);
require DBIx::Class::DeploymentHandler;
my $dh = DBIx::Class::DeploymentHandler->new({
schema => $schema,
sql_translator_args => { add_drop_table => 0 },
schema_version => 3,
});
$dh->prepare_install;
$dh->install;
my $json = read_file $config->{fixture_file};
my $fixtures = JSON::decode_json($json);
$schema->resultset($_)->populate($fixtures->{$_}) for keys %{$fixtures};
return $schema;
}
Where my config specifies dbi:SQLite:dbname=:memory: as the database dsn.
When running the test suite, the tables don't seem to be loaded, as I get errors stating the table does not exist, eg Can't locate object method "id" via package "no such table: cats"
Is there some extra setup that I'm not doing when wanting to deploy to an in-memory database?
Thanks
PS:
Doing the following works in a single script, I don't know if I'm doing something that Test::Class::Moose or Mojo doesn't like with the above
#!/usr/bin/perl
use Cats::Database::Schema;
use File::Slurp;
use JSON;
use Data::Dumper;
my $schema = Cats::Database::Schema->connect(
'dbi:SQLite:dbname=:memory:', '', ''
);
my $json = read_file('../t/fixtures.json');
my $fixtures = JSON::decode_json($json);
$schema->deploy();
$schema->resultset($_)->populate($fixtures->{$_}) for keys %{$fixtures};
# returns fixture data fine
# warn Dumper($schema->resultset('User')->search({}));

I believe I figured it out
The way I use the DBIx schema in the app is to instantiate it within a base controller which all sub controllers inherit. No matter how I built and populated the in memory database in the Test::Class::Moose object, it would not be using the instance specified there, instead it would be using the one specified in the base controller.
the solution was to move the schema construction up one level (from controller to the app root) as an attribute, allowing me to override it in Test Mojo to use the in memory db.

Related

Perl Mojolicious Model

I am creating a
Mojolicious
application but I can't understand the documentation for creating a model.
Could someone provide an example of how to create a model, run a raw SQL query and get the results in a controller to pass to the view?
I am thinking of something like this:
Model
package LS::Model::Dt;
use Mojo::Base;
use DBI;
# Here is what I don't understand
# Do I need to create a subroutine that connects to the database like this?
sub connect_db {
my $user = 'user_sql';
my $pass = 'pass_sql';
my $connection = "dbi:Sybase:server=db.sql-srv.com;database=Adventure";
my $dbh = DBI->connect($connection, $user, $pass) or die 'Cannot connect';
}
sub queries{
my $query_selectall = "select * from foo";
my $all_query = $dbh->selectall_arrayref($query_selectall, {Slice => {}});
}
Controller
package LS::Controller::Home;
use Mojo::Base 'Mojolicious::Controller';
use LS::Model::Dt
sub home {
my $self = shift;
# Somehow get the query results here
my $query_res = #somehow get the query results here
$self->render(res=>$query_res);
}
1;
Later edit: I have managed somehow via this tutorial:
http://oliverguenther.de/2014/04/applications-with-mojolicious-part-four-database-schemas-with-dbixclass/
Github code for when you are now sure where the author is creating a file is available here:
https://github.com/oliverguenther/Moblo
If you encounter problems also check this:
Mojolicious Deploying database schema

How does DBIx::Class::Storage::DBI's connect_info work?

The docs for connect_info:
connect_info
This method is normally called by "connection" in DBIx::Class::Schema,
which encapsulates its argument list in an arrayref before passing
them here.
The argument list may contain:
The same 4-element argument set one would normally pass to "connect"
in DBI, optionally followed by extra attributes recognized by
DBIx::Class:
$connect_info_args = [ $dsn, $user, $password, \%dbi_attributes?, \%extra_attributes? ];
A single code reference which returns a
connected DBI database handle optionally followed by extra attributes
recognized by DBIx::Class:
$connect_info_args = [ sub { DBI->connect (...) }, \%extra_attributes? ];
A single hashref with all the attributes and the dsn/user/password
mixed together:
$connect_info_args = [{
dsn => $dsn,
user => $user,
password => $pass,
%dbi_attributes,
%extra_attributes,
}];
$connect_info_args = [{
dbh_maker => sub { DBI->connect (...) },
%dbi_attributes,
%extra_attributes,
}];
This is particularly useful
for Catalyst based applications, allowing the following config
(Config::General style):
<Model::DB>
schema_class App::DB
<connect_info>
dsn dbi:mysql:database=test
user testuser
password TestPass
AutoCommit 1
</connect_info>
</Model::DB>
The dsn/user/password combination can be substituted by the dbh_maker key
whose value is a coderef that returns a connected DBI database handle
Please note that the DBI docs recommend that you always explicitly set
AutoCommit to either 0 or 1. DBIx::Class further recommends that it be
set to 1, and that you perform transactions via our "txn_do" in
DBIx::Class::Schema method. DBIx::Class will set it to 1 if you do not
do explicitly set it to zero. This is the default for most DBDs. See
"DBIx::Class and AutoCommit" for details.
What is this? Is it a method called internally, or a global? And, if it's a method called internally why is it being sent either a dbh maker, or four arguments? What determines what it is sent? It's listed as being a method. What is $connect_info_args?
Here is how I got it to work
Schema Class
You've got to use the storage that's done like this (and you can find it in the docs)
package MyDBIC::Schema;
__PACKAGE__->storage_type("DBIx::Class::Storage::DBI::mysql::MySubClass")
Storage Class
package DBIx::Class::Storage::DBI::mysql::MySubClass;
use mro 'c3';
use base 'DBIx::Class::Storage::DBI::mysql';
sub connect_info {
my $self = shift;
my $retval = $self->next::method([{
username => _username(),
password => $password,
dsn => "my:dsn:"
})
$retval;
};
I found a rough example of how to do it burred here. Talk about shitty docs..
connect_info appears to be broke. Things tried.
Subclassing a Storage with a custom connect_info. It gets called, but nothing it returns does anything useful. Calling ->connect->storage->ensure_connected; on a schema results in error.
Wrapping connect_info with Moose::around(). Fails to wrap. The method 'connect_info' was not found in the inheritance hierarchy.
Calling $self->connect_info() from BUILD... DBIx::Class::Storage::DBI::connect_info gets called but no matter what I hand to it, the sub only gets $self, the first argument.
Calling __PACKAGE__->connect_info({}) or __PACKAGE__->connect_info([{}]) results in both arguments being sent to the DBIx::Class::Storage::DBI::connect_info but the lack of a $self results in Class::XSAccessor: invalid instance method invocant: no hash ref supplied at when the sub tries to write to the accessor.

mojolicious helper storing an elasticsearch connection

i'm experimenting with elasticsearch within mojolicious.
I'm reasonably new at both.
I wanted to create a helper to store the ES connection and I was hoping to pass the helper configuration relating to ES (for example the node info, trace_on file etc).
If I write the following very simple helper, it works;
has elasticsearch => sub {
return Search::Elasticsearch->new( nodes => '192.168.56.21:9200', trace_to => ['File','/tmp/elasticsearch.log'] );
};
and then in startup
$self->helper(es => sub { $self->app->elasticsearch() });
however if I try to extend that to take config - like the following -
it fails. I get an error "cannot find index on package" when the application calls $self->es->index
has elasticsearch => sub {
my $config = shift;
my $params->{nodes} = '192.168.56.21:' . $config->{port};
$params->{trace_to} = $config->{trace_to} if $config->{trace_to};
my $es = Search::Elasticsearch->new( $params );
return $es;
};
and in startup
$self->helper(es => sub { $self->app->elasticsearch($self->config->{es}) });
I assume I'm simply misunderstanding helpers or config or both - can someone enlighten me?
Just fyi, in a separate controller file I use the helper as follows;
$self->es->index(
index => $self->_create_index_name($index),
type => 'crawl_data',
id => $esid,
body => {
content => encode_json $data,
}
);
that works fine if I create the helper using the simple (1st) form above.
I hope this is sufficient info? please let me know if anything else is required?
First of all, has and helper are not the same. has is a lazily built instance attribute. The only argument to an attribute constructor is the instance. For an app, it would look like:
package MyApp;
has elasticsearch => sub {
my $app = shift;
Search::ElasticSearch->new($app->config->{es});
};
sub startup {
my $app = shift;
...
}
This instance is then persistent for the life of the application after first use. I'm not sure if S::ES has any reconnect-on-drop logic, so you might need to think about it a permanent object is really what you want.
In contrast a helper is just a method, available to the app, all controllers and all templates (in the latter case, as a function). The first argument to a helper is a controller instance, whether the current one or a new one, depending on context. Therefore you need to build your helper like:
has (elasticsearch => sub {
my ($c, $config) = #_;
$config ||= $c->app->config->{es};
Search::ElasticSearch->new($config);
});
This mechanism will build the instance on demand and can accept pass-in arguments, perhaps for optional configuration override as I have shown in that example.
I hope this answers your questions.

Join attempt throwing exceptions

I'm sure I'm overlooking something glaringly obvious and I apologize for the newbie question, but I've spent several hours back and forth through documentation for DBIx::Class and Catalyst and am not finding the answer I need...
What I'm trying to do is automate creation of sub-menus based on the contents of my database. I have three tables in the database to do so: maps (in which sub-menu items are found), menus (contains names of top-level menus), maps_menus (assigns maps to top-level menus). I've written a subroutine to return a hash of resultsets, with the plan of using a Template Toolkit nested loop to build the top-level and sub-menus.
Basically, for each top-level menu in menus, I'm trying to run the following query and (eventually) build a sub-menu based on the result:
select * FROM maps JOIN maps_menus ON maps.id_maps = maps_menus.id_maps WHERE maps_menus.id_menus = (current id_menus);
Here is the subroutine, located in lib/MyApp/Schema/ResultSet/Menus.pm
# Build a hash of hashes for menu generation
sub build_menu {
my ($self, $maps, $maps_menus) = #_;
my %menus;
while (my $row = $self->next) {
my $id = $row->get_column('id_menus');
my $name = $row->get_column('name');
my $sub = $maps_menus->search(
{ 'id_maps' => $id },
{ join => 'maps',
'+select' => ['maps.id_maps'],
'+as' => ['id_maps'],
'+select' => ['maps.name'],
'+as' => ['name'],
'+select' => ['maps.map_file'],
'+as' => ['map_file']
}
);
$menus{$name} = $sub;
# See if it worked...
print STDERR "$name\n";
while (my $m = $sub->next) {
my $m_id = $m->get_column('id_maps');
my $m_name = $m->get_column('name');
my $m_file = $m->get_column('map_file');
print STDERR "\t$m_id, $m_name, $m_file\n";
}
}
return \%menus;
}
I am calling this from lib/MyApp/Controller/Maps.pm thusly...
$c->stash(menus => [$c->model('DB::Menus')->build_menu($c->model('DB::Map'), $c->model('DB::MapsMenus'))]);
When I attempt to pull up the page, I get all sorts of exceptions, the top-most of which is:
[error] No such relationship maps on MapsMenus at /home/catalyst/perl5/lib/perl5/DBIx/Class/Schema.pm line 1078
Which, as far as I can tell, originates from the call to $sub->next. I take this as meaning I'm doing my query incorrectly and not getting the results I think I should be. However, I'm not sure what I'm missing.
I found the following lines, defining the relationship to maps, in lib/MyApp/Schema/Result/MapsMenus.pm
__PACKAGE__->belongs_to(
"id_map",
"MyApp::Schema::Result::Map",
{ id_maps => "id_maps" },
{ is_deferrable => 1, on_delete => "CASCADE", on_update => "CASCADE" },
);
...and in lib/MyApp/Schema/Result/Map.pm
__PACKAGE__->has_many(
"maps_menuses",
"MyApp::Schema::Result::MapsMenus",
{ "foreign.id_maps" => "self.id_maps" },
{ cascade_copy => 0, cascade_delete => 0 },
);
No idea why it's calling it "maps_menuses" -- that was generated by Catalyst. Could that be the problem?
Any help would be greatly appreciated!
I'd suggest using prefetch of the two relationships which form the many-to-many relationship helper and maybe using HashRefInflator if you don't need access to the row objects.
Note that Catalyst doesn't generate a DBIC (which is btw the official abbreviation for DBIx::Class, DBIx is a whole namespace) schema, SQL::Translator or DBIx::Class::Schema::Loader do. Looks at the docs of the module you've used to find out how to influence its naming.
Also feel free to change the names if they don't fit you.

Mojolicious custom sessions

I am trying to use database sessions with Mojolicious instead of the builtin ones that are working with signed cookies.
In the startup subroutine I have something like:
my $dbh = DBI->connect(
$config->{database}->{dsn},
$config->{database}->{user},
$config->{database}->{password},
);
my $session = MojoX::Session->new(
store => [dbi => {dbh => $dbh}], # use MojoX::Session::Store::Dbi
transport => 'cookie', # this is by default
ip_match => 1
);
(ref($self))->attr( 'session' => sub {
return $session;
} );
And I want to use the session object like $self->session or $self->app->session in controllers.
Unfortunately it doesn't work - it uses previous sessions (from different browsers).
This drives me crazy - all I tried today was to make this work - I've read all the documentation available, also the source of MojoX::Session and all its related classes, tried in about 923847293847239847 ways to make it work, but nothing seems to do it.
PS: I created the session table in the db.
Do you know what I should do in order to be able to use DB sessions with Mojolicious?
You can connect MojoX::Session to the application as a plugin in a startup function.
use Mojolicious::Plugin::Session;
[...]
sub startup {
my $self = shift;
[...]
$self->plugin( session => {
stash_key => 'mojox-session',
store => [dbi => {dbh => $dbh}], # use MojoX::Session::Store::Dbi
transport => 'cookie',
ip_match => 1
});
[...]
Afterwards, you'll have access to session through stash key 'mojox-session' in controllers.
For example:
$self->stash('mojox-session')->data('something');
You can use whatever sort of session backend you like. Just create your own controller base class derived from Mojolicious::Controller and override session(), like so:
package NiceController;
use Mojo::Base 'Mojolicious::Controller';
sub session { # custom code here }
1;
then in startup(), set your controller class as the default:
$self->controller_class('NiceController');
Finally, make sure your application controllers inherit from NiceController instead of Mojolicious::Controller
It's probably a good idea to make your overridden session() function work just like the built-in one, to avoid future confusion.
-xyz
The $app->session method is reserved for using the built-in sessions.
You should take a look at the Mojolicious helpers and you probably want to use a different method name to avoid conflict.