I have a Quarkus project that uses a postgresql datasource. In production, we create the necessary schemas on the db manually before.
When I run quarkusDev mode and use the devservices, I therefor would like to run an init script on the testcontainer to create the schemas before liquibase does its migrations, which otherwise will fail.
I tried this without success
quarkus.datasource.jdbc.url=jdbc:tc:postgresql:13:///quarkus?TC_INITSCRIPT=testcontainer/schema-init.sql
quarkus.datasource.jdbc.driver=org.testcontainers.jdbc.ContainerDatabaseDriver
Nothing got picked up by the postgres testcontainer.
How can I run a init script on a datasource testcontainer with quarkus?
As stated here: https://quarkus.io/guides/databases-dev-services#database-vendor-specific-configuration
specific properties supported by the Testcontainers JDBC driver such as TC_INITSCRIPT, TC_INITFUNCTION, TC_DAEMON, TC_TMPFS are not supported.
So this simply does not work
Related
I am trying to initialize a database for my project which is based on using apache airflow. I am not too familiar with what happened but I changed my value from airflow.cfg file to sql_alchemy_conn =postgresql+psycopg2:////Users/gabeuy/airflow/airflow.db. Then when I saved the changes and ran the command airflow db init, the error occurred and did not allow me to run the db.
I tried looking up different ways to change it, ensured that I had Postgres and psycopg installed but it still resulted in an error when I ran the command. I was expecting it to run so that I could access the airflow db local host with the DAGs. error occured
Your sql_alchemy_conn is pointing to a local file path (indicating a SQLite DB), but the protocol is indicating a PostgreSQL DB. The error is telling you it's missing a password, which is required by PostgreSQL.
For PostgreSQL, the expected URL format is:
postgresql+psycopg2://<user>:<password>#<host>/<db>
And for a SQLite DB, the expected URL format is:
sqlite:////<path/to/airflow.db>
A SQLite DB is convenient for testing purposes. A SQLite DB is stored as a single file on your computer which makes it easy to set up (airflow db init will generate the file if it doesn't exist). A PostgreSQL DB takes a bit more work to set up, but is generally advised for a production scenario.
For more information about Airflow database configuration, see: https://airflow.apache.org/docs/apache-airflow/stable/howto/set-up-database.html.
And for more information about airflow db CLI commands, see: https://airflow.apache.org/docs/apache-airflow/stable/cli-and-env-variables-ref.html#db.
I'm trying to use the testcontainers-scala-postgresql to spin up some tests using Testcontainers, PostgreSQL and Scala. I want to run an init script during container startup to create and populate the table.
However, the com.dimafeng.testcontainers.PostgreSQLContainer type doesn't contain the withInitScript method, which is present in the Java version.
Is there any other way I can configure the execution of an init script during startup?
You could use PostgreSQLContainer.Def, where are an ability to pass commonJdbcParams with initScriptPath:
val initScriptParam = JdbcDatabaseContainer.CommonParams(initScriptPath = Option("init_script.sql"))
val postgresqlContainer: PostgreSQLContainer = PostgreSQLContainer.Def(commonJdbcParams = initScriptParam).createContainer()
Postgres image has the option to init the database on start, here's the docs on DockerHub (look for the /docker-entrypoint-initdb.d).
In short, *.sql, *.sql.gz, or *.sh scripts under /docker-entrypoint-initdb.d are executed on startup.
In the Java version you can use the withCopyFileToContainer method, I think something similar should be in Scala right?
So something like:
new PostgreSQLContainer<>("postgres:14-alpine")
.withCopyFileToContainer(
MountableFile.forClasspathResource("/schema.sql"),
"/docker-entrypoint-initdb.d/"
);
And if you have more files, they will be executed in lexicographic order.
When I want to use a Mysql database in a Springboot app, I am able to create it on start via a string in properties similar to this:
spring.datasource.jdbc-url=jdbc:mysql://localhost:3306/testDb?createDatabaseIfNotExist=true&autoReconnect=true&useSSL=false&allowPublicKeyRetrieval=true
Nevertheless PostgreSQL seems to ignore this:
spring.datasource.jdbc-url=jdbc:postgresql://localhost:5432/testdb?createDatabaseIfNotExist=true&autoReconnect=true&useSSL=false&allowPublicKeyRetrieval=true
Is there a way to create a PostgreSQL db on start of a SpringBoot app before Flyway attempts to initiate tables?
Postgres doesn't support creating a database on demand via the JDBC URL. You can learn about what configuration is possible in the documentation.
I am using a PostgreSQL TestContainer to test Liquibase schema migration in Spring Boot. I don't have any respositories. I am wondering if I can see/access the contents of the TestContainer, and test the schema migration.
Yes, you can access the Docker container spawned by Testcontainers like any other Docker container. Using the JUnit 5 extension or the JUnit 4 rule for Testcontainers will however shut down the container after your tests.
You can use the coontainer re-usability feature for Testcontainers (in alpha state since 1.12.3) for ensuring your containers are up- and running after your tests finish.
As Testcontainers will launch the container on an ephemeral port, simply execute docker ps and check to which local port the container port is mapped. E.g.:
b0df4733babb postgres:9.6.12 "docker-entrypoint.s…" 19 seconds ago Up 18 seconds 0.0.0.0:32778->5432/tcp inspiring_dewdney
You can now connect to your db on localhost:32778 with e.g. PgAdmin or the database view of IntelliJ IDEA and check your database tables.
The credentials for the access are those you specify in your test:
static PostgreSQLContainer postgreSQLContainer = (PostgreSQLContainer) new PostgreSQLContainer()
.withDatabaseName("differentDatabaseName")
.withUsername("duke")
.withPassword("s3cret")
.withReuse(true);
As a workaround you could also put in a breakpoint at the end of your test, debug the test, and quickly check your database.
UPDATE: If you want to verify the validity of your schema, you can use a Hibernate feature for this:
spring.jpa.hibernate.ddl-auto=validate
This will validate that your Java entity setup matches the underlying database schema on application startup. You can also add this to your production application.properties file as your application won't start if there is a mismatch (e.g. missing table, column).
For this to work in a test, you need to either use #DataJpaTest or use #SpringBootTest to the whole application context connect to your local container.
Find more information here.
I have been able to successfully instantiate a connection to PostgreSQL using the reactive driver and config as per the micronaut docs.
See:
https://docs.micronaut.io/snapshot/guide/index.html#postgresSupport
I have been trying to figure out a way to use flyway to run db migrations but in order to instantiate a flyway bean I need a Datasource which doesn't seem to be possible to get using the reactive driver.
You don't need a datasource bean to instantiate flyway. You can provide it with the url, username, and password instead.
The micronaut configuration can be injected and read to get the configuration values. io.micronaut.configuration.postgres.reactive.PgPoolConfiguration
From their docs:
// Create the Flyway instance and point it to the database
Flyway flyway = Flyway.configure().dataSource("jdbc:h2:file:./target/foobar", "sa", null).load();
// Start the migration
flyway.migrate();