In perl how to check whether db con object is able to access data base or not ?
== when fcgi is running and db got disconnected/shutdown fcgi db con object will fail to connect to database and the error will get only while binding a query or executing a query .. How to detect dbcon obj is proper before binding or executing a query .. ?
I assume you're talking about a DBI connection object. All DBI handles have a ping() method which will check whether or not the connection is still active.
The documentation says this:
ping
$rc = $dbh->ping;
Attempts to determine, in a reasonably efficient way, if the database server is still running and the connection to it is still working. Individual drivers should implement this function in the most suitable manner for their database engine.
The current default implementation always returns true without actually doing anything. Actually, it returns "0 but true" which is true but zero. That way you can tell if the return value is genuine or just the default. Drivers should override this method with one that does the right thing for their type of database.
Few applications would have direct use for this method. See the specialized Apache::DBI module for one example usage.
I think that's what you want.
Related
I have the following configuration in application.conf for an in-memory database HSQLDB:
db {
inmemory {
jndiName = jndiInMemory
driver = org.hsqldb.jdbc.JDBCDriver
url = "jdbc:hsqldb:mem:inmemory"
}
}
And connect in a controller with the following statements
val database = Database.forName("jndiInMemory")
val session = database.createSession
val conn = session.conn
// JDBC statements
Problem is that when the code runs several times, I get an exception in session.conn:
HikariPool-34 - Connection is not available, request timed out after
30000ms.
Since I'm using JNDI, I figured that the connections are reused. Do I have to drop the session after I finish using it? How to fix this code?
Hard to tell without looking at the actual code but in general: when you create a database connection at the start of the application, you usually reuse it until application ends - then you should close connection.
If you spawn a new connection every time you do query, without ending previous ones you will run at the connection limit pretty fast.
Easy to use pattern is: create a session in the beginning and then use dependency injection to pass it to wherever you need to run it.
BTW, I noticed that for some configurations e.g. Slick create connection statically (as in: stores them as static class properties). So, you need to create a handler, that closes session when application exits. It run ok... until you start it several times over in SBT, which by default uses the same JVM to run itself and spawned application. In such cases it is better to run things as fork. For tests I use Test / fork := true, for run I use sbt-revolver, though I am not sure how that would play out with Play.
I have some automated tests that I run in order to test a MongoDB-related library. In order to do that, I start a Mongo server with a temporary data directory and on an ephemeral port, connect to it, and run some tests.
This leads to a race condition, obviously. So in my first version of these tests, I paused for a fixed amount of time and waited to make sure mongod had time to start before the tests began.
This was frustrating (and inefficient), so I decided to monitor the standard output and wait for a line on mongod's standard output stream matching the regular expression:
/\[initandlisten\] waiting for connections/
This got it working. So good, then I prepared to circle back and try to find a more robust way to do it. I recalled that a Java library called "embedmongo" ran MongoDB-based tests, and figured it must solve the problem. And it does this (GitHub):
protected String successMessage() {
return "waiting for connections on port";
}
... and uses that to figure out whether the process has started correctly.
So, are we right? Is examining the mongod process output log (is it ever internationalized? could the wording of the message ever change?) the very best way to do this? Or is there something more robust that we're both missing?
What we do in a similar scenario is:
Try to connect to the configured port (simply new Socket(host, port)) in a loop until it works (10 ms delay) - this ensures, that the mongo client, which starts an internal monitoring thread, does not throw exceptions due to "connection refused"
Connect to the mongodb and query something. This is important, as all mongo client objects are lazy init. (Simple listDatabaseNames() on the client is enough, but make sure to actually read the result.)
All the time, check the process for not being terminated.
I just wrote a small untilMongod command which does just that, which you can use in bash scripting: https://github.com/FGM/untilMongod
Includes a bash + Node.JS example use case.
I have a class inheriting from DbContext implementing a code-first Entity Framework object.
I would like to have a unit test that exercises the model builder in this dbcontext -- this would be useful for detecting things like 'key not defined on entity' errors.
But I may not have an actual database available during unit testing time. Is there a way to exercise this code without actually trying to make a database connection with the context. I tried something innocuous like:
var ctx = new MyDbContext("Data Source=(local);Initial Catalog=Dummy;<.......>");
var foo = ctx.GetValidationErrors(); //triggers DBModelBuilder, should throw if my DBModel is goofed up
This does technically work. However this takes a very long time to run -- if I pause and inspect the call stack it is triggering a call to System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin
Eventually it times out, swallows the connection error and finishes my test.
Is there any way to do this without the connection attempt?
The solution here is to just kind of go with the problem and use the lowest possible connection timeout. I'm using this connection string:
Server=localhost; Database=tempdb; Integrated Security=true;Connection Timeout=1;ConnectRetryCount=0
It still triggers the problem, but with a one second timeout and no automatic retries (for only this one test in the system) it's totally acceptable.
And if the developer has a local db installed, it will accidentally work even faster.
My mod_perl2-based intranet app uses DBI->connect_cached() which is supposedly overridden by Apache::DBI's version of the same. It has normally worked quite well, but just recently we started having an issue on our testing server--which had only two users connected--whereby our app would sometimes, but not always, die when trying to reload a page with 'FATAL: sorry, too many clients already' connecting to our postgres 9.0 backend, despite all of them being <IDLE> if I look at the stats in pgadmin3.
The backend is separate from our development and production backends, but they're all configured with max_connections = 100. Likewise the httpd services are all separate, but configured with
StartServers 8
MinSpareServers 5
MaxSpareServers 20
ServerLimit 99
MaxClients 99
MaxRequestsPerChild 4000
....
PerlModule Apache::DBI
I had been under the impression that I shouldn't call disconnect() on my database handles if I wanted them to actually benefit from caching. Was I wrong about that? If not, I guess I'll ask about the above error separately. Just wanted to make sure it wasn't this setup...
Apache::DBI's docs say:
When loading the DBI module (do not confuse this with the Apache::DBI
module) it checks if the environment variable 'MOD_PERL' has been set
and if the module Apache::DBI has been loaded. In this case every
connect request will be forwarded to the Apache::DBI module.
....
There is no need to remove the disconnect statements from your code.
They won't do anything because the Apache::DBI module overloads the
disconnect method.
If you are developing new code that is strictly for use in mod_perl,
you may choose to use DBI->connect_cached() instead, but consider
adding an automatic rollback after each request, as described above.
So I guess for my mod_perl2-only app, I don't need Apache::DBI because Apache::DBI's devs recommend using DBI->connect_cached. And I don't need disconnect statements.
But then DBI's docs say:
Note that the behaviour of [ connect_cached ] differs in several
respects from the behaviour of persistent connections implemented by
Apache::DBI. However, if Apache::DBI is loaded then connect_cached
will use it.
This makes it sound like Apache::DBI will actually affect connect_cached, in that instead of getting DBI->connect_cached behaviour when I call that, I'll get Apache::DBI->connect behaviour. And Apache::DBI's docs recommend against that.
UPDATE: I've set the first 5 parameters in the above config all to 1, and my app is still using up more and more connections as I hit its pages. This I don't understand at all--it should only have one process, and that one process should be re-using its connection.
Unless you plan on dropping Apache::DBI, the answer is a firm no, because Apache::DBI's override really does nothing:
# overload disconnect
{
package Apache::DBI::db;
no strict;
#ISA=qw(DBI::db);
use strict;
sub disconnect {
my $prefix = "$$ Apache::DBI ";
Apache::DBI::debug(2, "$prefix disconnect (overloaded)");
1;
}
;
}
I'm currently reading Effective Perl Programming (2nd edition). I have come across a piece of code which was described as being poorly written, but I don't yet understand what's so bad about it, or how it should be improved. It would be great if someone could explain the matter to me.
Here's the code in question:
sub sum_values_per_key {
my ( $class, $dsn, $user, $password, $parameters ) = #_;
my %results;
my $dbh =
DBI->connect( $dsn, $user, $password, $parameters );
my $sth = $dbh->prepare(
'select key, calculate(value) from my_table');
$sth->execute();
# ... fill %results ...
$sth->finish();
$dbh->disconnect();
return \%results;
}
The example comes from the chapter on testing your code (p. 324/325). The sentence that has left me wondering about how to improve the code is the following:
Since the code was poorly written and accesses DBI directly, you'll have to create a fake DBI object to stand in for the real thing.
I have probably not understood a lot of what the book has so far been trying to teach me, or I have skipped the section relevant for understanding what's bad practice about the above code... Well, thanks in advance for your help!
Since the chapter is about testing, consider this:
When testing your function, you are also (implicitly) testing DBI. This is why it's bad.
Good testing always only checks one functionality. To guarantee this, it would be required
to not use DBI directly, but use a mock object instead. This way, if your test fails, you
know it's your function and not something else in another module (like DBI in your example).
I think what Brian was trying to say by "poorly written" is that you do not have a separation between business logic and data access code (and database connection mechanics, while at it).
A correct approach to writing functions is that a function (or method) should do one thing, not 3 things at once.
As a result of this big lump of functionality, when testing, you have to test ALL THREE at the same time, which is difficult (see discussion of using "test SQLite DB" in those paragraphs). Or, as an alternative, do what the chapter was devoted to, and mock the DBI object to test the business logic by pretending that the data access AND DB setup worked a certain way.
But mocking a complicated-behaving object like DBI is very and very complicated to do right.
What if the database is not accessible? What if there's blocking? What if your query has a syntax error? What if the DB connection times out when executing the query? What if...
Good test code tests ALL those error situations and more.
A more correct approach (pattern) for the code would be:
my $dbh = set_up_dbh();
my $query = qq[select key, calculate(value) from my_table];
my $data = retrieve_data($dbh, $query);
# Now, we don't need to test setting up database connection AND data retrieval
my $calc_results = calculate_results($data);
This way, to test the logic in calculate_results (e.g. summing the data), you merely need to mock DATA passed to it, which is very easy (in many cases, you just store several sets of test data in some test config); as opposed to mocking the behavior of a complicated DBI object used to retrieve the data.
There is nothing wrong with using DBI by itself.
The clue is in the fact that this is the testing chapter. I assume the issue being pointed out is that the function opens and closes a database connection itself. It should instead expect a database handle as a parameter and just run queries on it, leaving any concerns about opening and closing a database connection to its caller. That will make the job of the function more narrow, so it makes the function more flexible.
That in turn also makes the function easier to test: just pass it a mock object as a database handle. As it is currently written, you need at least to redefine DBI::connect to test it, which isn’t hard, but is definitely messy.
A method called sum_values_per_key should be interested in summing the values of some keys, not fetching the data to be summed.
It does not meet the S (Single responsibility principle) of SOLID programming. http://en.wikipedia.org/wiki/Solid_%28object-oriented_design%29
This means that it is both:
Not reusable if you wish to use different source data.
Difficult to test in an environment without a database connection.
1) Suppose you have a dozen objects each with a dozen methods like this. Twenty of those methods will be called during the execution of the main program. You now have made 20 DB connections where you only need one.
2) Suppose you are not happy with original DBI and extended it with My::DBI. You now have to rewrite 144 functions in 12 files.
(Apache::DBI might be an example here).
3) You have to carry 3 positional parameters in each call to those 144 functions. Human brain works well with about 7 objects at a time; you have just waisted almost half that space. This makes code less maintainable.