We are making mongodb connection with X509.
We have added below propertied in application.properties.
spring.data.mongodb.uri
spring.data.mongodb.host
But for keystore location and password we do not want to send using command line.
if we send using command line it works
but after removing it fails to start.
We set the property in SprintApplication class using #POstConstruct but that did not solved the issue.
Inject properties such as password using the jvm option or environment value.
Enter at run java
java -Dspring.data.mongodb.password=your_password ...
or Set MONGODB_PASSWORD to an environment variable.
spring.data.mongodb.password=${MONGODB_PASSWORD}
You can also encrypt using jasypt.
see https://www.jasypt.org
Related
We have a Golang app built with Gorm that connects with PostgreSQL 14 (hosted in the same machine) with an SSL mode of verify-full. The certificates were issued by ZeroSSL, and generated with the help of Acme.sh. In the PostgreSQL configuration, we have set the following certificate-related variables:
ssl = on
ssl_cert_file = '/etc/ssl/certs/our-certificate.pem'
ssl_key_file = '/etc/ssl/private/our-certificate.key'
The Golang app and the PostgreSQL server are hosted in an Ubuntu 20.04 machine.
Our Golang app depends on a bunch of environment variables for, among others, database passwords and usernames. When we pass those variables to our app directly, i.e.:
DB_HOST=dbhost DB_NAME=dbname DB_USERNAME=username DB_PASSWORD=123456789 [...] ./go-app
Everything works well.
On the other hand, we also tried exporting those variables first. Exporting the variables is done by putting the variables inside an env file first, then doing the following:
set -a
source env-file.env
set +a
The environment variables are exported now at this point, and we can echo the variables from the terminal.
Unfortunately, when our Golang app reaches the point where it would attempt to connect with the PostgreSQL server, we would get the following error message:
error="failed to connect to `host=dbhost user=username database=dbname`: failed SASL auth (FATAL: password authentication failed for user \"username\" (SQLSTATE 28P01))"
I have already logged the contents of the exported database-related environment variables to the console, and they were of the same values as the variables directly passed to the process.
Why does this happen? It's a weird behaviour for me, but I believe I am simply lacking some necessary information.
I am fairly new to Eclipse Ditto and have just started using it for my project.
I am trying to connect Cloud hosted mongodb instance to ditto.
Following the documentation I know that I need to add some variables and pass them to docker-compose. The problem is that I do not know what should be the values of these variables as there are no examples.
Are all these variables necessary or will just the URI work?
This is my current .env file config
MONGO_DB_URI=mongodb+srv://username:pass#IP
MONGO_DB_READ_PREFERENCE=primary
MONGO_DB_WRITE_CONCERN=majority
The command I am using to start ditto is
docker-compose --env-file .env up
I have removed mongodb service from docker-compose.yml
Nice to hear that you started using Ditto in your project.
You need to set the following env variables to connect to your Cloud hosted MongoDB.
MONGO_DB_URI: Connection string to MongoDB
For more detailes see: https://docs.mongodb.com/manual/reference/connection-string/
If you have a ReplicaSet your MongoDB URI should look like this: mongodb://[username:password#]mongodb0.example.com:27017,mongodb1.example.com:27017,mongodb2.example.com:27017/?replicaSet=myRepl
I assume you also need to enable SSL to connect to your MongoDB.
To do so set this env var.
MONGO_DB_SSL_ENABLED: true
If you want to use a specific Ditto version you can set the following env var
DITTO_VERSION= e.g. 2.1.0-M3
If you use .env as file name you can start Ditto with:
docker-compose up
The other options for pool size, read preference and write concern aren't necessary as there are default values in place.
Is it possible to give postgresql testcontainer a custom postgresql.conf file via config?
I have included maven dependency
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>postgresql</artifactId>
<version>1.10.6</version>
</dependency>
And using 'Database containers launched via JDBC URL scheme' for DB url
As such have the setting in my Spring Boot app as:
datasource:
url: jdbc:tc:postgresql:10-alpine:///databasename
driver-class-name: org.testcontainers.jdbc.ContainerDatabaseDriver
I need to have a custom setting in postgresql.conf.
Is there a way of pushing postgresql.conf to the docker container started by testcontainers?
EDIT 1
Thanks #vilkg I did know about the TC_INITSCRIPT script option and SET function however:
I am wanting a custom setting such as my.key
ALTER system does not work for your own settings eg: ALTER SYSTEM SET my.key = 'jehe'; get error Could not execute the SQL command.
Message returned: `ERROR: unrecognized configuration parameter "my.key"
I had previously try SET and ALTER DATABASE as below
SET my.key = 'new value 8'; -- sets for current session
ALTER DATABASE test SET my.key = 'new value 8'; -- sets for subsequent sessions
select current_setting('my.key');
PROBLEM IS
when testcontainer starts postgres container and I pass it an init script to run
url: jdbc:tc:postgresql:10-alpine:///databasename?TC_INITSCRIPT=init_pg.sql
and I can include the above SQL its happy..
I know setting of that secret.key is working correctly in this script because it will fail on the line select current_setting('my.key'); if other two are commented out
I also know that runing it against db name test is correct eg: 'ALTER DATABASE test' because if I use a different name it fails
Testcontainers automatically connects the app to db named test
So with all of the above I believe the DB is setup nicely and all should be good
BUT
When I use 'current_setting('my.key')' within application code it fails
If you want to continue launching Postgres container using JDBC URL scheme, test containers can execute init script for you. The script must be on the classpath, referenced as follows:
jdbc:tc:postgis:9.6://hostname/databasename?TC_INITSCRIPT=somepath/init.sql
ALTER SYSTEM SET command was introduced in postgres 9.4, so you could use it in your init script.
Another option would be to start postgres container using database containers objects and use withCopyFileToContainer() method. Example:
JdbcDatabaseContainer<?> postgisContainer = new PostgisContainerProvider()
.newInstance()
.withDatabaseName( POSTGRES_DATABASE_NAME )
.withUsername( POSTGRES_CREDENTIALS )
.withPassword( POSTGRES_CREDENTIALS )
.withCopyFileToContainer(MountableFile.forClasspathResource("postgresql.conf"), "/var/lib/postgresql/data"));
EDIT:
If none of the above works, you can reconfigure Postgres command and pass your custom configuration keys. All you need is extending PostgreSQLContainer and overriding configure() method.
#Override
protected void configure()
{
setCommand( "postgres -c $your_config_key=$your_config_value" );
}
we have to create our database and the connection’s user. This is done by using environment variables from the Docker image. To change postgres.conf we can use DockerFIle where we will replace the existing postgres.conf by the new configuration.
#ClassRule
public static GenericContainer postgresql= new GenericContainer(
new ImageFromDockerfile("postgresql:10-alpine")
.withDockerfileFromBuilder(dockerfileBuilder -> {
dockerfileBuilder.from("myPostgresql:10-alpine")
// root password is mandatory
.env("PG_ROOT_PASSWORD", "root_password")
.env("PG_DATABASE", "postgres")
.env("PG_USER", "postgres")
.env("PG_PASSWORD", "postgres")
})
Next, we have to create a database schema and populate the database. From the image documentation, the directory /docker-entrypoint-initdb.d is scanned at startup and all files with .sh, .sql et .sql.gz extension are executed. So, we just have to put our files schema.sql and data.sql in this directory. Somme parameteres can be set by sql requesting running database
.withFileFromClasspath("a_schema.sql", "db/pg/schema.sql")
.withFileFromClasspath("b_data.sql", "db/pg/data.sql"))
We have a Windows EC2 instance on which we are running a custom command line application (C# console app using NpgSQL) to connect to a PostgreSQL RDS instance. Based on the instructions here:
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_PostgreSQL.html#PostgreSQL.Concepts.General.SSL
we created a new DB parameter group with rds.force_ssl set to 1 and rebooted our RDS instance. We also downloaded and imported to Windows the pem file referenced on the page.
I was able to connect to the RDS instance from my Windows EC2 instance via pgAdmin by specifying SSL mode as Verify-Full. Our command-line application reads connection strings from a file and they look like this now that I've added the sslmode parameter:
Server=OurInstanceAddress;Port=5432;SearchPath='$user,public,topology';Database=OurDatabase;User Id=username;Password=mypassword;sslmode=verify-full;
Using this connection string failed with the error referenced at the bottom of the page:
FATAL: no pg_hba.conf entry for host "host.ip", user "someuser", database "postgres", SSL off
I tried adding the sslrootcert parameter, but I'm not sure if I'm dealing with it properly. I tried using the example (sslrootcert=rds-ssl-ca-cert.pem) and I tried using the name of the pem that I downloaded. I feel like there is something about the path information that I'm giving to the sslrootcert parameter that isn't right, especially in a Windows environment. I've tried using the name, I've tried using the following paths:
- sslrootcert=C:\keys\rds-combined-ca-bundle.pem - single backslash
- sslrootcert=C:\\\keys\\\rds-combined-ca-bundle.pem - double backslash
- sslrootcert=C:/keys/rds-combined-ca-bundle.pem - Linux style backslash
All of these produced the same error mentioned above.
Any insight would be appreciated.
I solved it using the environment variables instead for specifiying cert paths in connection url
-DPGSSLROOTCERT=/certs/root.crt
-DPGSSLKEY=/certs/amazon-postgresql.key
-PGSSLCERT=/certs/amazon-postgresql.crt
Although I'm in cygwin. There are some hints in the documentation when using windows here https://www.postgresql.org/docs/9.0/static/libpq-ssl.html
I am starting the jboss_3.2.7 with the linux user jbs using jboss with RUNASIS option, but it is not working while the entire system[linux] restart. Which is starting the jboss as root user.
I added the jboss service in chkconfig option of linux for starting the jboss on linux restart.
In the jboss service file (/etc/init.d) modified the user to RUNASIS
define the user under which jboss will run, or use RUNASIS to run as the current user
JBOSSUS=${JBOSSUS:-"RUNASIS"}
You are using quite old JBoss version and I personally never see it. But I think it should be very similar to newer ones.
Please try to put your user after when defining these variable:
JBOSSUS=jbs
The other solution is to setting these variable before executing running script:
export JBOSSUS=jbs; /etc/init.d/jboss start
Update
I have just downloaded JBoss 3.2.7 and I checked the jboss_init_redhat.sh script (I hope you use these one as a templete for your starting script).
In the file jboss_init_redhat.sh you can find such lines:
#define the user under which jboss will run, or use RUNASIS
#to run as the current user
JBOSSUS=${JBOSSUS:-"jboss"}
These line defines the new user name. It checks if the variable JBOSSUS is set up and if not is uses jboss user as default name.
The second interesting part of these script:
if [ "$JBOSSUS" = "RUNASIS" ]; then
SUBIT=""
else
SUBIT="su - $JBOSSUS -c "
fi
You should know one thing: when you run automatically any script from init scripts it is always run as a root user. Thats why in the script should be command which change the effective user to someone else. And here you have these part of the script.
It first checked if your username is RUNASIS and if it is yes - do nothing. In another case it run JBoss as a another user by using su command.
In your case it should be sufficient to change JBOSSUS variable definition to something like that:
JBOSSUS=jbs
After that you can start these script as a root user and it should run JVM with JBoss with jbs user.