loading DB driver in Global.beforeStart - postgresql

I want to implement some DB cleanup at each startup (full schema deletion and recreation while in dev-enviroment).
I'm doing it in Global.beforeStart. And because it's literally before start I need to load DB drivers myself.
The code is:
#Override
public void beforeStart(Application app){
System.out.println("IN beforeStart");
try{
Class.forName("org.postgresql.Driver");
System.out.println("org.postgresql.Driver LOADED");
} catch (ClassNotFoundException cnfe){
System.out.println("NOT LOADED org.postgresql.Driver");
cnfe.printStackTrace();
}
ServerConfig config = new ServerConfig();
config.setName("pgtest");
DataSourceConfig postgresDb = new DataSourceConfig ();
postgresDb.setDriver("org.postgresql.Driver");
postgresDb.setUsername("postgres");
postgresDb.setPassword("postgrespassword");
postgresDb.setUrl("postgres://postgres:postgrespassword#localhost:5432/TotoIntegration2");
config.setDataSourceConfig(postgresDb);
config.setDefaultServer(true);
EbeanServer server = EbeanServerFactory.create(config);
SqlQuery countTables = Ebean.createSqlQuery("select count(*) from pg_stat_user_tables;");
Integer numTables = countTables.findUnique().getInteger("count");
System.out.println("numTables = " + numTables);
if(numTables>2){
DbHelper.cleanSchema();
}
System.out.println("beforeStart EXECUTED");
//DbHelper.cleanSchema();
}
Class.forName("org.postgresql.Driver") passed without exceptions, but then I'm getting:
com.avaje.ebeaninternal.server.lib.sql.DataSourceException: java.sql.SQLException: No suitable driver found for postgres
on the line EbeanServer server = EbeanServerFactory.create(config);
Why?

Use onStart instead, it's performed right after beforeStart but it's natural candidate for operating on database (in production mode it doesn't wait for first request), javadoc for them:
/**
* Executed before any plugin - you can set-up your database schema here, for instance.
*/
public void beforeStart(Application app) {
}
/**
* Executed after all plugins, including the database set-up with Evolutions and the EBean wrapper.
* This is a good place to execute some of your application code to create entries, for instance.
*/
public void onStart(Application app) {
}
Note, that you don't need include DB config additionally here, you can use your models here the same way as you do in controller.

Related

Detecting when the database session gets refreshed on a Spring Boot 2 application

I'm trying to execute the following SQL statement every time the Database Session gets refreshed. I have a Spring Boot 2.0.1.RELEASE with JPA application and a PostgreSQL Database.
select set_config('SOME KEY', 'SOME VALUE', false);
As the PostgreSQL documentation states the is_local parameter is used to indicate that this configuration value will apply just for the current transaction -if true- or will be attached to the session (as I require) -if false-
The problem is that I'm not aware when Hibernate/Hikari are refreshing the db session, so, in practice, the application start failing when it has a couple of minutes running, as you can imagine...
My approach -that is not working yet- is to implement a EmptyInterceptor, for that I have added a DatabaseCustomizer class to inject my hibernate.session_factory.interceptor properly in a way that Spring can fill out all my #Autowires
DatabaseInterceptor.class
#Component
public class DatabaseInterceptor extends EmptyInterceptor {
#Autowired
private ApplicationContext context;
#Override
public void afterTransactionBegin(Transaction tx) {
PersistenceService pc = context.getBean(PersistenceService.class);
try {
pc.addPostgresConfig("SOME KEY", "SOME VALUE");
System.out.println("Config added...");
} catch (Exception e) {
e.printStackTrace();
}
}
}
DatabaseCustomizer.class
#Component
public class DatabaseCustomizer implements HibernatePropertiesCustomizer {
#Autowired
private DatabaseInterceptor databaseInterceptor;
#Override
public void customize(Map<String, Object> hibernateProperties) {
hibernateProperties.put("hibernate.session_factory.interceptor", databaseInterceptor);
}
}
Obviously, there is a problem with this approach because when I #Override the afterTransactionBegin method to start another transaction I get an Infinite loop.
I tried to look something inside that Transaction tx that could help to be sure that this transaction is not being generated by my own addPostgresConfig but there is not much on it.
Is there something else I could try to achieve this?
Thanks in advance,

UnitTest FluentNhibernate using PostgreSQLConfiguration

When setting up our new architecture I followed a guide which used NHibernate with MsSql2008 configuration.
We are not using MsSql2008, instead using Postgresql. The configuration for this all works great and it saves to the database etc.
I am trying to write a unit test to test the UoW but I can't get the InMemory configuration to work.
The guide that I followed used this following Provider:
public class InMemoryNHibernateConfigurationProvider : NHibernateConfigurationProvider
{
public override Configuration GetDatabaseConfiguration()
{
var databaseDriver = SQLiteConfiguration.Standard.InMemory().ShowSql();
return CreateCoreDatabaseConfiguration(databaseDriver);
}
public static void InitialiseDatabase(Configuration configuration, ISession session)
{
new SchemaExport(configuration).Execute(true, true, false, session.Connection, Console.Out);
}
}
My standard (Non UnitTest) configuration looks like this:
public abstract class NHibernateConfigurationProvider : INHibernateConfigurationProvider
{
public abstract Configuration GetDatabaseConfiguration();
public Configuration CreateCoreDatabaseConfiguration(
IPersistenceConfigurer databaseDriver,
Action<Configuration> databaseBuilder = null)
{
var fluentConfiguration =
Fluently.Configure()
.Database(databaseDriver)
.Mappings(m => m.AutoMappings.Add(AutoMap.AssemblyOf<Organisation>(new DefaultMappingConfiguration())
//.Conventions.AddFromAssemblyOf<IdGenerationConvention>()
.UseOverridesFromAssemblyOf<OrganisationMappingOverride>()));
if (databaseBuilder != null)
{
fluentConfiguration.ExposeConfiguration(databaseBuilder);
}
return fluentConfiguration.BuildConfiguration();
}
}
public class PostgreSQLServerNHibernateConfigurationProvider : NHibernateConfigurationProvider
{
private static readonly string NpgsqlConnectionString = ConfigurationManager.ConnectionStrings["ProdDBConnection"].ConnectionString;
public override Configuration GetDatabaseConfiguration()
{
return CreateCoreDatabaseConfiguration(
PostgreSQLConfiguration.Standard.ConnectionString(NpgsqlConnectionString).
Dialect("NHibernate.Dialect.PostgreSQL82Dialect").ShowSql(),
BuildDatabase);
}
....... // Other Methods etc
}
How do I write a InMemoryConfigurationProvider that tests using PostgresqlConfiguration instead of SqlLiteCOnfiguration. PostgresqlConfiguration does not have an InMemory option.
Do I implement a configuration that creates another database and just drop it on teardown? Or is there perhaps another way of doing it?
Using sqlite works really well and although it does have some differences to SQL-server which we use they are so minor it doesn't matter for testing purposes.
With that said, this is how we setup the tests:
All test-cases where we want to write/read from db extend the SqLiteTestBaseclass. That way they all get access to a session created by the basesetup method, and can setup the daos / repositories as needed.
Using this approach we also always get a fresh new db for each test-case.
Update:
After trying this out a bit more I actually found that you have to modify it a bit to use InMemory (we had previously used sqlite backed by a file on disk instead). So the updated (complete) setup looks like this:
private Configuration _savedConfig;
[SetUp]
public void BaseSetup()
{
FluentConfiguration configuration =
Fluently.Configure()
.Database(SQLiteConfiguration.Standard
.InMemory)
.ExposeConfiguration(
x => x.SetInterceptor(new MultiTenancyInterceptor(ff)))
.Mappings(m => m.FluentMappings.AddFromAssemblyOf<IRepository>())
.Mappings(m => m.FluentMappings.ExportTo("c:\\temp\\mapping"))
.ExposeConfiguration(x => _savedConfig = x) //save the nhibernate configuration for use when creating the schema, in order to be able to use the same connection
.ExposeConfiguration(x => ConfigureEnvers(x))
.ExposeConfiguration(x => ConfigureListeners(x));
ISessionFactory sessionFactory;
try
{
sessionFactory = configuration.BuildSessionFactory();
}
catch (Exception ex)
{
Console.WriteLine(ex.StackTrace);
throw;
}
_session = sessionFactory.OpenSession();
BuildSchema(_savedConfig, _session);
}
private void BuildSchema(Configuration config, ISession session)
{
new SchemaExport(config)
.Execute(false, true, false, session.Connection, null);
}
The reason why you have to jump through all these hoops in order to use the in-memory version of Sqlite is due to the db being tied to the connection. You have to use the same connection that creates the db to populate the schema, thus we have to save the Configuration object so that we can export the schema later when we've created the connection.
See this blogpost for some more details: http://www.tigraine.at/2009/05/29/fluent-nhibernate-gotchas-when-testing-with-an-in-memory-database/
N.B: This only shows the setup of the db. We have some code which also populates the db with standard values (users, customers, masterdata etc) but I've omitted that for brevity.

Grails integration tests with managed Mongodb

I'm currently using mongoDB and I wanted to be available to run integration and funcional tests on any machine (currently in dedicated build server and in a future CI server).
The main problem is that I have to be able to check mongodb installation (and if not present, install it), start a mongodb instance on startup and shut it down once the process has finished.
There's an already developed question here Embedded MongoDB when running integration tests that suggests installing a gradle or maven plugin.
This gradle plugin https://github.com/sourcemuse/GradleMongoPlugin/ can do this, but I will have to manage my dependencies with it, already tried. The problem with this approach is not gradle itself, but when tried this I've lost all the benefits from my IDE (STS, intellij).
Did anyone managed to do this?
If someone configured gradle with a grails project withour losing the grails perspective, I will appreciate that help too!
Thanks!
Trygve.
I have recently created a grails plugin for this purpose. https://github.com/grails-plugins/grails-embedded-mongodb
Currently it is in snapshot, however I plan to publish a release this week
I've had good results using an in-memory Mongo server for integration tests. It runs fast and doesn't require starting up a separate Mongo server or dealing with special grails or maven config. This means that the tests can run equally well with any JUnit test runner, i.e. within any IDE or build system. No extra setup required.
In-memory Mongo example
I have also used the "flapdoodle" embedded mongo server for testing. It uses a different approach in that it downloads and executes a separate process for a real Mongo instance. I have found that this mechanism has more moving parts and seems to be overkill for me when all I really want to do is verify that my app works correctly with a mongo server.
Better answer late than never -
Unfortunately I found that Fongo does not address all of my requirements quite well - most notably, $eval is not implemented so that you cannot run integration tests with migration tools such as Mongeez.
I settled for EmbedMongo, which I am using in my Spock/Geb integration tests via JUnit ExternalResource rules. Even though Gary is right when he says that a real managed DB comes with many more moving parts, but I found that I'd rather take that risk than rely on a mock implementation. So far it worked quite well, give or take an unclean database shutdown during test suite teardown, which fortunately does not impact the tests. You would use the rules as follows:
#Integration(applicationClass = Application)
#TestFor(SomeGrailsArtifact) // this will inject grailsApplication
class SomeGrailsArtifactFunctionalSpec extends Specification {
#Shared #ClassRule
EmbedMongoRule embedMongoRule = new EmbedMongoRule(grailsApplication)
#Rule
ResetDatabaseRule resetDatabaseRule = new ResetDatabaseRule(embedMongoRule.db)
...
For the sake of completeness, these are the rule implementations:
EmbedMongoRule.groovy
import org.junit.rules.ExternalResource
import com.mongodb.MongoClient
import com.mongodb.MongoException
import de.flapdoodle.embed.mongo.MongodProcess
import de.flapdoodle.embed.mongo.MongodStarter
import de.flapdoodle.embed.mongo.config.IMongodConfig
import de.flapdoodle.embed.mongo.config.MongodConfigBuilder
import de.flapdoodle.embed.mongo.config.Net
import de.flapdoodle.embed.mongo.distribution.Version
import de.flapdoodle.embed.process.runtime.Network
/**
* Rule for {#code EmbedMongo}, a managed full-fledged MongoDB. The first time
* this rule is used, it will download the current production MongoDB release,
* spin it up before tests and tear it down afterwards.
*
* #author Michael Jess
*
*/
public class EmbedMongoRule extends ExternalResource {
private def mongoConfig
private def mongodExecutable
public EmbedMongoRule(grailsApplication) {
if(!grailsApplication) {
throw new IllegalArgumentException(
"Got null grailsApplication; have you forgotten to supply it to the rule?\n" +
"\n" +
"#Integration(applicationClass = Application)\n" +
"#TestFor(MyGrailsArtifact)\n // will inject grailsApplication" +
"class MyGrailsArtifactSpec extends ... {\n" +
"\n" +
"\t..." +
"\t#Shared #ClassRule EmbedMongoRule embedMongoRule = new EmbedMongoRule(grailsApplication)\n" +
"\t...\n" +
"}")
}
mongoConfig = grailsApplication.config.grails.mongodb
}
#Override
protected void before() throws Throwable {
try {
MongodStarter starter = MongodStarter.getDefaultInstance()
IMongodConfig mongodConfig = new MongodConfigBuilder()
.version(Version.Main.PRODUCTION)
.net(new Net(mongoConfig.port, Network.localhostIsIPv6()))
.build()
mongodExecutable = starter.prepare(mongodConfig)
MongodProcess mongod = mongodExecutable.start()
} catch (IOException e) {
throw new IllegalStateException("Unable to start embedded mongo", e)
}
}
#Override
protected void after() {
mongodExecutable.stop()
}
/**
* Returns a new {#code DB} for the managed database.
*
* #return A new DB
* #throws IllegalStateException If an {#code UnknownHostException}
* or a {#code MongoException} occurs
*/
public def getDb() {
try {
return new MongoClient(mongoConfig.host, mongoConfig.port).getDB(mongoConfig.databaseName)
} catch (UnknownHostException | MongoException e) {
throw new IllegalStateException("Unable to retrieve MongoClient", e)
}
}
}
ResetDatabaseRule.groovy - currently not working since GORM ignores the grails.mongodb.databaseName parameter as of org.grails.plugins:mongodb:4.0.0 (grails 3.x)
import org.junit.rules.ExternalResource
/**
* Rule that will clear whatever Mongo {#code DB} is provided.
* More specifically, all non-system collections are dropped from the database.
*
* #author Michael Jess
*
*/
public class ResetDatabaseRule extends ExternalResource {
/**
* Prefix identifying system tables
*/
private static final String SYSTEM_TABLE_PREFIX = "system"
private def db
/**
* Create a new database reset rule for the specified datastore.
*
* #param getDb Closure returning a reference to the {#link DB} instance
* to reset.
*/
ResetDatabaseRule(db) {
this.db = db
}
#Override
protected void before() throws Throwable {
db.collectionNames
.findAll { !it.startsWith(SYSTEM_TABLE_PREFIX) }
.each { db.getCollection(it).drop() }
}
}

Entity Framework MigrateDatabaseToLatestVersion giving error

I am attempting to use Entity Framework code based migrations with my web site. I currently have a solution with multiple projects in it. There is a Web API project which I want to initialize the database and another project called the DataLayer project. I have enabled migrations in the DataLayer project and created an initial migration that I am hoping will be used to create the database if it does not exist.
Here is the configuration I got when I enabled migrations
public sealed class Configuration : DbMigrationsConfiguration<Harris.ResidentPortal.DataLayer.ResidentPortalContext>
{
public Configuration()
{
AutomaticMigrationsEnabled = false;
}
protected override void Seed(Harris.ResidentPortal.DataLayer.ResidentPortalContext context)
{
// This method will be called after migrating to the latest version.
// You can use the DbSet<T>.AddOrUpdate() helper extension method
// to avoid creating duplicate seed data. E.g.
//
// context.People.AddOrUpdate(
// p => p.FullName,
// new Person { FullName = "Andrew Peters" },
// new Person { FullName = "Brice Lambson" },
// new Person { FullName = "Rowan Miller" }
// );
//
}
}
The only change I made to this after it was created was to change it from internal to public so the WebAPI could see it and use it in it's databaseinitializer. Below is the code in the code in the Application_Start that I am using to try to initialize the database
Database.SetInitializer(new MigrateDatabaseToLatestVersion<ResidentPortalContext, Configuration>());
new ResidentPortalUnitOfWork().Context.Users.ToList();
If I run this whether or not a database exists I get the following error
Directory lookup for the file "C:\Users\Dave\Documents\Visual Studio 2012\Projects\ResidentPortal\Harris.ResidentPortal.WebApi\App_Data\Harris.ResidentPortal.DataLayer.ResidentPortalContext.mdf" failed with the operating system error 2(The system cannot find the file specified.).
CREATE DATABASE failed. Some file names listed could not be created. Check related errors.
It seems like it is looking in the totally wrong place for the database. It seems to have something to do with this particular way I am initializing the database because if I change the code to the following.
Database.SetInitializer(new DropCreateDatabaseAlways<ResidentPortalContext>());
new ResidentPortalUnitOfWork().Context.Users.ToList();
The database will get correctly created where it needs to go.
I am at a loss for what is causing it. Could it be that I need to add something else to the configuration class or does it have to do with the fact that all my migration information is in the DataLayer project but I am calling this from the WebAPI project?
I have figured out how to create a dynamic connection string for this process. You need to first add this line into your EntityFramework entry on Web or App.Config instead of the line that gets put there by default.
<defaultConnectionFactory type="<Namespace>.<ConnectionStringFacotry>, <Assembly>"/>
This tells the program you have your own factory that will return a DbConnection. Below is the code I used to make my own factory. Part of this is a hack to get by the fact that a bunch of programmers work on the same set of code but some of us use SQL Express while others use full blown SQL Server. But this will give you an example to go by for what you need.
public sealed class ResidentPortalConnectionStringFactory: IDbConnectionFactory
{
public DbConnection CreateConnection(string nameOrConnectionString)
{
SqlConnectionStringBuilder builder = new SqlConnectionStringBuilder(ConfigurationManager.ConnectionStrings["PortalDatabase"].ConnectionString);
//save off the original catalog
string originalCatalog = builder.InitialCatalog;
//we're going to connect to the master db in case the database doesn't exist yet
builder.InitialCatalog = "master";
string masterConnectionString = builder.ToString();
//attempt to connect to the master db on the source specified in the config file
using (SqlConnection conn = new SqlConnection(masterConnectionString))
{
try
{
conn.Open();
}
catch
{
//if we can't connect, then append on \SQLEXPRESS to the data source
builder.DataSource = builder.DataSource + "\\SQLEXPRESS";
}
finally
{
conn.Close();
}
}
//set the connection string back to the original database instead of the master db
builder.InitialCatalog = originalCatalog;
DbConnection temp = SqlClientFactory.Instance.CreateConnection();
temp.ConnectionString = builder.ToString();
return temp;
}
}
Once I did that I coudl run this code in my Global.asax with no issues
Database.SetInitializer(new MigrateDatabaseToLatestVersion<ResidentPortalContext, Configuration>());
using (ResidentPortalUnitOfWork temp = new ResidentPortalUnitOfWork())
{
temp.Context.Database.Initialize(true);
}

Reset Embedded H2 database periodically

I'm setting up a new version of my application in a demo server and would love to find a way of resetting the database daily. I guess I can always have a cron job executing drop and create queries but I'm looking for a cleaner approach. I tried using a special persistence unit with drop-create approach but it doesn't work as the system connects and disconnects from the server frequently (on demand).
Is there a better approach?
H2 supports a special SQL statement to drop all objects:
DROP ALL OBJECTS [DELETE FILES]
If you don't want to drop all tables, you might want to use truncate table:
TRUNCATE TABLE
As this response is the first Google result for "reset H2 database", I post my solution below :
After each JUnit #tests :
Disable integrity constraint
List all tables in the (default) PUBLIC schema
Truncate all tables
List all sequences in the (default) PUBLIC schema
Reset all sequences
Reenable the constraints.
#After
public void tearDown() {
try {
clearDatabase();
} catch (Exception e) {
Fail.fail(e.getMessage());
}
}
public void clearDatabase() throws SQLException {
Connection c = datasource.getConnection();
Statement s = c.createStatement();
// Disable FK
s.execute("SET REFERENTIAL_INTEGRITY FALSE");
// Find all tables and truncate them
Set<String> tables = new HashSet<String>();
ResultSet rs = s.executeQuery("SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA='PUBLIC'");
while (rs.next()) {
tables.add(rs.getString(1));
}
rs.close();
for (String table : tables) {
s.executeUpdate("TRUNCATE TABLE " + table);
}
// Idem for sequences
Set<String> sequences = new HashSet<String>();
rs = s.executeQuery("SELECT SEQUENCE_NAME FROM INFORMATION_SCHEMA.SEQUENCES WHERE SEQUENCE_SCHEMA='PUBLIC'");
while (rs.next()) {
sequences.add(rs.getString(1));
}
rs.close();
for (String seq : sequences) {
s.executeUpdate("ALTER SEQUENCE " + seq + " RESTART WITH 1");
}
// Enable FK
s.execute("SET REFERENTIAL_INTEGRITY TRUE");
s.close();
c.close();
}
The other solution would be to recreatethe database at the begining of each tests. But that might be too long in case of big DB.
Thre is special syntax in Spring for database manipulation within unit tests
#Sql(scripts = "classpath:drop_all.sql", executionPhase = Sql.ExecutionPhase.AFTER_TEST_METHOD)
#Sql(scripts = {"classpath:create.sql", "classpath:init.sql"}, executionPhase = Sql.ExecutionPhase.BEFORE_TEST_METHOD)
public class UnitTest {}
In this example we execute drop_all.sql script (where we dropp all required tables) after every test method.
In this example we execute create.sql script (where we create all required tables) and init.sql script (where we init all required tables before each test method.
The command: SHUTDOWN
You can execute it using
RunScript.execute(jdbc_url, user, password, "classpath:shutdown.sql", "UTF8", false);
I do run it every time when the Suite of tests is finished using #AfterClass
If you are using spring boot see this stackoverflow question
Setup your data source. I don't have any special close on exit.
datasource:
driverClassName: org.h2.Driver
url: "jdbc:h2:mem:psptrx"
Spring boot #DirtiesContext annotation
#DirtiesContext(classMode = DirtiesContext.ClassMode.BEFORE_EACH_TEST_METHOD)
Use #Before to initialise on each test case.
The #DirtiesContext will cause the h2 context to be dropped between each test.
you can write in the application.properties the following code to reset your tables which are loaded by JPA:
spring.jpa.hibernate.ddl-auto=create