DBIx::something that can restore session variables on reconnection? - perl

Ordinary DBI::db handler will lost all database session settings that was made using $dbh->do('SET variable_name=value').
Is there any DBIx::* class/package or so that provides method like "set_session" to set session variables and can restore this variables after detection of connection lost (connection timeout in 90% of real cases) ?
It may looks like this:
# inside the user code:
$dbh->set(variable => 'string', yet_another_variable => 42)
# inside the DBIx::* package:
sub reconnect {
# ...
while (my ($var, $val) = each %{$self->saved_vars}) {
$self->dbh->do("SET $var=?", {}, $val)
}
# ...
}

DBI supports something called Callbacks. I can't link to this bit of the doc as the section is quite long, so here it is verbatim.
A more common application for callbacks is setting connection state
only when a new connection is made (by connect() or connect_cached()).
Adding a callback to the connected method (when using connect) or via
connect_cached.connected (when useing connect_cached()>) makes this
easy. The connected() method is a no-op by default (unless you
subclass the DBI and change it). The DBI calls it to indicate that a
new connection has been made and the connection attributes have all
been set. You can give it a bit of added functionality by applying a
callback to it. For example, to make sure that MySQL understands your
application's ANSI-compliant SQL, set it up like so:
my $dbh = DBI->connect($dsn, $username, $auth, {
Callbacks => {
connected => sub {
shift->do(q{
SET SESSION sql_mode='ansi,strict_trans_tables,no_auto_value_on_zero';
});
return;
},
}
});
This is your exact use-case I believe. Do this instead of running your own code after you've connected.

Related

Solved: DBI cached statements gone and CGI::Session is stucked

I'm using Apache2.2(worker)/mod_perl 2.0.4/Apache::DBI/CGI::Session and Firebird RDBMS.
I also wrote CGI::Session::Driver::firebird.pm to work with Firebird RDBMS.
DB connection is pooled by Apache::DBI and give connection handle to CGI::Session {Handle=>$dbh}.
Number of DB connection is equals to number of worker processes.
I posted Programming with Apache::DBI and firebird. Get Stucked httpd on exception 3 month ago.
I found a reason of that issue, and want to know how to fix it.
$dbh = DBI->connect("dbi:Firebird:db=$DBSERVER:/home/cdbs/xxnet.fdb;
ib_charset=UTF8;ib_dialect=3",$DBUSER,$DBPASS,{
AutoCommit=>1,
LongReadLen=>8192,
RaiseError=>1
});
my $session = new CGI::Session('dbi:firebird',$sessid,{Handle=>$dbh});
my $ses_p1 = $session->param('p1');
eval { $dbh->begin_work()
my $sql = "SELECT * FROM SAMPLETABLE"
my $st = $dbh->prepare($sql);
$st->execute();
while (my $R = $st->fetchrow_hashref()) {
...
}
$st->finish();
}; warn $# if $#;
if ($#) {
$dbh->rollback();
}else{
$dbh->commit();
}
$session->flush();
When an sql error is occured, an eval block catches exception and rollback transaction.
After that, CGI::Session does not retrieve session object no more.
Because prepare_cached statement failes at CGI::Session::DBI.pm.
CGI::Session::DBI.pm use prepare_cached($sql,undef,3). '3' is safest way of using cached statement, but it never find broken statement at this situation.
How to fix this ?
raise request to change CGI::Session::DBI.pm to use prepare() statement ?
write store(),retrieve(),traverse() function in firebird.pm to use prepare() statement ?
It may other prepare_cached() going to fail after catch exception...
1) I add die statement on CGI::Session->errstr()
I got an error "new(): failed: load(): couldn't retrieve data: retrieve(): $sth->execute failed with error message"
2) I flush session object after session->load()
if $session is valid, changes are stored to DB.
3) I replace begin_work() to {AutoCommit}=0
results are same. I can use $dbh normally after catching exception and rollback, BUT new CGI::Session returns error.
------------------------------------------ added 2017/07/26 18:47 JST
Please give me your suggestion.
Thank you.
There are various things you could try before request changes to CGI::Session::Driver::DBI.pm ...
First, change the way new CGI::Session is called in order to diagnose if the problem happens when the session is created or loaded:
my $session = CGI::Session->new('dbi:firebird',$sessid,{Handle=>$dbh}) or die CGI::Session->errstr();
The methods param or delete stores changes to the session inside $session handle, not in DB. flush stores in DB the changes made inside the session handle. Use $session->flush() only after a session->param set/update or a session delete:
$session->param('p1','someParamValue');
$session->flush() or die 'Unable to update session storage!';
# OR
$session->delete();
$session->flush() or die 'Unable to update session storage!';
The method flush does not destroy $session handle (you still can call $session->param('p1') after the flush). In some cases mod_perl caches $session causing problems to the next attempt to load that same session. In those cases it needs to be destroyed when it's not needed anymore:
undef($session)
The last thing i can suggest is avoid using begin_work method, control the transaction behavior with AutoCommit instead (because the DBD::Firebird documentation says that's the way transactions should be controlled) and commit inside the eval block:
eval {
# Setting AutoCommit to 0 enables transaction behavior
$dbh->{AutoCommit} = 0;
my $sql = "SELECT * FROM SAMPLETABLE"
my $st = $dbh->prepare($sql);
$st->execute();
while (my $R = $st->fetchrow_hashref()) {
...
}
$st->finish();
$dbh->commit();
};
if ($#) {
warn "Tansaction aborted! $#";
$dbh->rollback();
}
# Remember to set AutoCommit to 1 after the eval
$dbh->{AutoCommit} = 1;
You said you wrote your own session driver for Firebird... You should see how the CGI/Driver/sqlite.pm or CGI/Driver/mysql.pm are made, maybe you need to write some fetching method you are missing...
Hope this helps!!

How does DBIx::Class::Storage::DBI's connect_info work?

The docs for connect_info:
connect_info
This method is normally called by "connection" in DBIx::Class::Schema,
which encapsulates its argument list in an arrayref before passing
them here.
The argument list may contain:
The same 4-element argument set one would normally pass to "connect"
in DBI, optionally followed by extra attributes recognized by
DBIx::Class:
$connect_info_args = [ $dsn, $user, $password, \%dbi_attributes?, \%extra_attributes? ];
A single code reference which returns a
connected DBI database handle optionally followed by extra attributes
recognized by DBIx::Class:
$connect_info_args = [ sub { DBI->connect (...) }, \%extra_attributes? ];
A single hashref with all the attributes and the dsn/user/password
mixed together:
$connect_info_args = [{
dsn => $dsn,
user => $user,
password => $pass,
%dbi_attributes,
%extra_attributes,
}];
$connect_info_args = [{
dbh_maker => sub { DBI->connect (...) },
%dbi_attributes,
%extra_attributes,
}];
This is particularly useful
for Catalyst based applications, allowing the following config
(Config::General style):
<Model::DB>
schema_class App::DB
<connect_info>
dsn dbi:mysql:database=test
user testuser
password TestPass
AutoCommit 1
</connect_info>
</Model::DB>
The dsn/user/password combination can be substituted by the dbh_maker key
whose value is a coderef that returns a connected DBI database handle
Please note that the DBI docs recommend that you always explicitly set
AutoCommit to either 0 or 1. DBIx::Class further recommends that it be
set to 1, and that you perform transactions via our "txn_do" in
DBIx::Class::Schema method. DBIx::Class will set it to 1 if you do not
do explicitly set it to zero. This is the default for most DBDs. See
"DBIx::Class and AutoCommit" for details.
What is this? Is it a method called internally, or a global? And, if it's a method called internally why is it being sent either a dbh maker, or four arguments? What determines what it is sent? It's listed as being a method. What is $connect_info_args?
Here is how I got it to work
Schema Class
You've got to use the storage that's done like this (and you can find it in the docs)
package MyDBIC::Schema;
__PACKAGE__->storage_type("DBIx::Class::Storage::DBI::mysql::MySubClass")
Storage Class
package DBIx::Class::Storage::DBI::mysql::MySubClass;
use mro 'c3';
use base 'DBIx::Class::Storage::DBI::mysql';
sub connect_info {
my $self = shift;
my $retval = $self->next::method([{
username => _username(),
password => $password,
dsn => "my:dsn:"
})
$retval;
};
I found a rough example of how to do it burred here. Talk about shitty docs..
connect_info appears to be broke. Things tried.
Subclassing a Storage with a custom connect_info. It gets called, but nothing it returns does anything useful. Calling ->connect->storage->ensure_connected; on a schema results in error.
Wrapping connect_info with Moose::around(). Fails to wrap. The method 'connect_info' was not found in the inheritance hierarchy.
Calling $self->connect_info() from BUILD... DBIx::Class::Storage::DBI::connect_info gets called but no matter what I hand to it, the sub only gets $self, the first argument.
Calling __PACKAGE__->connect_info({}) or __PACKAGE__->connect_info([{}]) results in both arguments being sent to the DBIx::Class::Storage::DBI::connect_info but the lack of a $self results in Class::XSAccessor: invalid instance method invocant: no hash ref supplied at when the sub tries to write to the accessor.

How to deploy a test database for DBIx Class

I have Mojolicious app with a test suite using Test::Class::Moose. Using DBIx::Class to interact with my database, is there a way to setup an in-memory database that I can add fixture data to?
I'd like to use an in memory database because it'll mean that the application will have less setup configuration. I do know how to setup an actual SQLite database for testing but managing that for testing along with a mySQL database for production doesn't sound like easy management (eg "oh no, forgot to rebuild the testing database").
Loading data from fixtures seems ideal, that way you have more control over what is actually in the database. Example, you find a bug with a row that contains certain data, add a row like that to your fixture file and test until successful.
Now how to actually set up an in-memory database using DBIx? :)
Right now this is how I'm building the schema for my Test::Class::Moose suite:
has cats => (
is => 'ro',
isa => 'Test::Mojo::Cats',
default => sub { return Test::Mojo::Cats->new(); }
);
has schema => (
is => 'ro',
lazy => 1,
builder => '_build_schema_and_populate',
);
sub _build_schema_and_populate {
my $test = shift;
my $config = $test->cats->app->config();
my $schema = Cat::Database::Schema->connect(
$config->{db_dsn},
$config->{db_user},
$config->{db_pass},
{
HandleError => DBIx::Error->HandleError,
unsafe => 1
}
);
require DBIx::Class::DeploymentHandler;
my $dh = DBIx::Class::DeploymentHandler->new({
schema => $schema,
sql_translator_args => { add_drop_table => 0 },
schema_version => 3,
});
$dh->prepare_install;
$dh->install;
my $json = read_file $config->{fixture_file};
my $fixtures = JSON::decode_json($json);
$schema->resultset($_)->populate($fixtures->{$_}) for keys %{$fixtures};
return $schema;
}
Where my config specifies dbi:SQLite:dbname=:memory: as the database dsn.
When running the test suite, the tables don't seem to be loaded, as I get errors stating the table does not exist, eg Can't locate object method "id" via package "no such table: cats"
Is there some extra setup that I'm not doing when wanting to deploy to an in-memory database?
Thanks
PS:
Doing the following works in a single script, I don't know if I'm doing something that Test::Class::Moose or Mojo doesn't like with the above
#!/usr/bin/perl
use Cats::Database::Schema;
use File::Slurp;
use JSON;
use Data::Dumper;
my $schema = Cats::Database::Schema->connect(
'dbi:SQLite:dbname=:memory:', '', ''
);
my $json = read_file('../t/fixtures.json');
my $fixtures = JSON::decode_json($json);
$schema->deploy();
$schema->resultset($_)->populate($fixtures->{$_}) for keys %{$fixtures};
# returns fixture data fine
# warn Dumper($schema->resultset('User')->search({}));
I believe I figured it out
The way I use the DBIx schema in the app is to instantiate it within a base controller which all sub controllers inherit. No matter how I built and populated the in memory database in the Test::Class::Moose object, it would not be using the instance specified there, instead it would be using the one specified in the base controller.
the solution was to move the schema construction up one level (from controller to the app root) as an attribute, allowing me to override it in Test Mojo to use the in memory db.

mojolicious helper storing an elasticsearch connection

i'm experimenting with elasticsearch within mojolicious.
I'm reasonably new at both.
I wanted to create a helper to store the ES connection and I was hoping to pass the helper configuration relating to ES (for example the node info, trace_on file etc).
If I write the following very simple helper, it works;
has elasticsearch => sub {
return Search::Elasticsearch->new( nodes => '192.168.56.21:9200', trace_to => ['File','/tmp/elasticsearch.log'] );
};
and then in startup
$self->helper(es => sub { $self->app->elasticsearch() });
however if I try to extend that to take config - like the following -
it fails. I get an error "cannot find index on package" when the application calls $self->es->index
has elasticsearch => sub {
my $config = shift;
my $params->{nodes} = '192.168.56.21:' . $config->{port};
$params->{trace_to} = $config->{trace_to} if $config->{trace_to};
my $es = Search::Elasticsearch->new( $params );
return $es;
};
and in startup
$self->helper(es => sub { $self->app->elasticsearch($self->config->{es}) });
I assume I'm simply misunderstanding helpers or config or both - can someone enlighten me?
Just fyi, in a separate controller file I use the helper as follows;
$self->es->index(
index => $self->_create_index_name($index),
type => 'crawl_data',
id => $esid,
body => {
content => encode_json $data,
}
);
that works fine if I create the helper using the simple (1st) form above.
I hope this is sufficient info? please let me know if anything else is required?
First of all, has and helper are not the same. has is a lazily built instance attribute. The only argument to an attribute constructor is the instance. For an app, it would look like:
package MyApp;
has elasticsearch => sub {
my $app = shift;
Search::ElasticSearch->new($app->config->{es});
};
sub startup {
my $app = shift;
...
}
This instance is then persistent for the life of the application after first use. I'm not sure if S::ES has any reconnect-on-drop logic, so you might need to think about it a permanent object is really what you want.
In contrast a helper is just a method, available to the app, all controllers and all templates (in the latter case, as a function). The first argument to a helper is a controller instance, whether the current one or a new one, depending on context. Therefore you need to build your helper like:
has (elasticsearch => sub {
my ($c, $config) = #_;
$config ||= $c->app->config->{es};
Search::ElasticSearch->new($config);
});
This mechanism will build the instance on demand and can accept pass-in arguments, perhaps for optional configuration override as I have shown in that example.
I hope this answers your questions.

Mojolicious custom sessions

I am trying to use database sessions with Mojolicious instead of the builtin ones that are working with signed cookies.
In the startup subroutine I have something like:
my $dbh = DBI->connect(
$config->{database}->{dsn},
$config->{database}->{user},
$config->{database}->{password},
);
my $session = MojoX::Session->new(
store => [dbi => {dbh => $dbh}], # use MojoX::Session::Store::Dbi
transport => 'cookie', # this is by default
ip_match => 1
);
(ref($self))->attr( 'session' => sub {
return $session;
} );
And I want to use the session object like $self->session or $self->app->session in controllers.
Unfortunately it doesn't work - it uses previous sessions (from different browsers).
This drives me crazy - all I tried today was to make this work - I've read all the documentation available, also the source of MojoX::Session and all its related classes, tried in about 923847293847239847 ways to make it work, but nothing seems to do it.
PS: I created the session table in the db.
Do you know what I should do in order to be able to use DB sessions with Mojolicious?
You can connect MojoX::Session to the application as a plugin in a startup function.
use Mojolicious::Plugin::Session;
[...]
sub startup {
my $self = shift;
[...]
$self->plugin( session => {
stash_key => 'mojox-session',
store => [dbi => {dbh => $dbh}], # use MojoX::Session::Store::Dbi
transport => 'cookie',
ip_match => 1
});
[...]
Afterwards, you'll have access to session through stash key 'mojox-session' in controllers.
For example:
$self->stash('mojox-session')->data('something');
You can use whatever sort of session backend you like. Just create your own controller base class derived from Mojolicious::Controller and override session(), like so:
package NiceController;
use Mojo::Base 'Mojolicious::Controller';
sub session { # custom code here }
1;
then in startup(), set your controller class as the default:
$self->controller_class('NiceController');
Finally, make sure your application controllers inherit from NiceController instead of Mojolicious::Controller
It's probably a good idea to make your overridden session() function work just like the built-in one, to avoid future confusion.
-xyz
The $app->session method is reserved for using the built-in sessions.
You should take a look at the Mojolicious helpers and you probably want to use a different method name to avoid conflict.