At the moment, I'm hardcoding several variables like resource names and ports. I would like to move them out of my code.
What are recommended means of implementing a central configuration outside the actual code? Like a file maybe. So that, while the production and development are using same git repository, the configurations would be seperate. I am using Play 2 Framework on Scala.
I would suggest using the Typesafe Config library. It loads and parses files that can be a mix of .properties style, JSON, or extended JSON (called HOCON - "Human-Optimized Config Object Notation"), and is the configuration style used by Play 2 itself (and Akka, Spray, and a quickly growing list of other libraries).
In Play's standard application.conf file you can include files like so:
include "file:///..."
In this file you could override what properties you need to.
Additionally, (as documented in the excellent play docs), one can specify conf files during app startup like so:
Using -Dconfig.file
You can also specify another local configuration file not packaged into the application artifacts:
$ start -Dconfig.file=/opt/conf/prod.conf
Using -Dconfig.url
You can also specify a configuration file to be loaded from any URL:
$ start -Dconfig.url=http://conf.mycompany.com/conf/prod.conf
Note that you can always reference the original configuration file in a new prod.conf file using the include directive, such as:
include "application.conf"
key.to.override=blah
Configuration is likely a mean of taste, but Typesafe Config is one of the common libraries to use in scala/play ecosystem (e.g. it is used in akka).
Related
I am new in Akka and want to set up different application.conf files per environment (test, staging, prod, etc.).
I read the documentation from Akka but I couldn't find a way to do it.
May you know any proper way to have multiple configuration files?
Akka uses the Typesafe Config Library.
The convenience method ConfigFactory.load() loads the following (first-listed are higher priority):
system properties
application.conf (all resources on classpath with this name)
application.json (all resources on classpath with this name)
application.properties (all resources on classpath with this name)
reference.conf (all resources on classpath with this name)
So we usually create an application.conf file in /resources folder.
For different environments, we can create environment specific files like
development.conf
production.conf
Using includes, include application.conf at the beginning and specify environment specific configurations which will override the application.conf.
To use a specific environment force a different config source (e.g. from command line -Dconfig.resource=environment.conf)
Start your application using below command:
$ sbt run -Dconfig.resource=development.conf
Here are couple ways to do it.
Add tags with in a file itself like
prod-config : { YOUR_CONFIGURATION_FOR_PROD }
test-config : { YOUR_CONFIGURATION_TEST_ENV }
and so-on
Make separate files for the each environment and rename it once after deploying them to specific environment. Say from application-prod.conf to application.conf
A possibility is have a single variable in application.conf which would be "confFile", based on an actual env like
confFile=${?CONF_FILE_NAME}
Then in your code, you load the correspondind file like this
val configFile = ConfigFactory.load().getString("confFile")
val appConf = ConfigFactory.load(configFile)
The options for this aren't so much related to Akka as it is to the config library. Akka uses the default config resolution unless you give it a Config instance.
With no specific user code to manually select a specific file you have a few options. I think the most useful are the system properties config.resource to choose a file on the class path (inside the app jar for example), config.file to use a file from the file system.
Of course you may still have reasons to write your own code to select files as the other answers suggested.
More details in the config library docs: https://github.com/lightbend/config#standard-behavior
I am building a Scala Play 2.4 application which uses the typesafe activator.
I would like to run my tests 2 times with a different configuration file for each run.
How can I specify alternative config files, or override the config settings?
I currently run tests with the command "./activator test"
You can create different configuration files for different environments/purposes. For example, I have three configuration files for local testing, alpha deployment, and production deployment as in this project https://github.com/luongbalinh/play-mongo
You can specify the configuration for running as follows:
activator run -Dconfig.resource=application.conf
where application.conf is the configuration you want to use.
You can create different configuration files for different environments. To specify the configuration to use it with activator run, use the following command:
activator "run -Dconfig.resource=application.conf"
where the application.conf is the desired configuration. Without the quotes it did not work for me. This is using the same configuration parameters as you use when going into production mode as described here:
https://www.playframework.com/documentation/2.5.x/ProductionConfiguration#Specifying-an-alternate-configuration-file
Important to know is also that config.resource tries to locate the configuration within the conf/ folder, so no need to specify that as well. For full paths not among the resources, use config.file. Further reading is also in the above link.
The quotes need to be used because you do not want to send the -D to activator, but to the run command. Using the quotes, the activator's JVM gets no -D argument but it interprets "run -Dconfig.file=application.conf" and sets the config.file property accordingly, also in the activator's JVM.
This was already discussed here: Activator : Play Framework 2.3.x : run vs. start
Since all the above are partially incorrect, here is my hard wrought knowledge from the last weekend.
Use include "application.conf" not include "application" (which Akka does)
Configs must be named .conf or Play will discard them silently
You probably want -Dconfig.file=<file>.conf so you're not classpath dependent
Make sure your provide the full file path (e.g. /opt/configs/prod.conf)
Example
Here is an example of this we run:
#prod.conf
include "application"
akka.remote.hostname = "prod.blah.com"
# Example of passing in S3 keys
s3.awsAccessKeyId="YOUR_KEY"
s3.awsSecretAccessKey="YOUR_SECRET_KEY"
And just pass it in like so:
activator -Dconfig.file=/var/lib/jenkins/jenkins.conf test
of if you fancy SBT:
sbt -Dconfig.file=/var/lib/jenkins/jenkins.conf test
Dev Environment
Also note it's easy to make a developer.conf file as well, to keep all your passwords/local ports, and then set a .gitignore so dev's don't accidentally check them in.
The below command works with Play 2.5
$ activator -Dconfig.resource=jenkins.conf run
https://www.playframework.com/documentation/2.5.x/ProductionConfiguration
Is it possible to use environmental variable substitution when including another configuration file?
I would like to have something like that:
include "${HOME}/.foo/credentials.conf"
Configuration documentation mentions locating resources and include substitution but not together.
This works:
include "/home/me/.foo/credentials.conf"
and my HOME is correctly set.
But all attempts to make include "${HOME}/.foo/credentials.conf" so far failed
Background:
I deliberately want to keep credentials and other sensitive data out of our code base but have them available for local dev environments for testing. I am aware of more sophisticated solutions using external storage like hinted here Playframework 2 - Storing your credentials and we use something similar for live and preview environments but these are not suitable for local dev setup.
An alternative is to include credentials file to code base after all but use git ignore to prevent pushing it, but it is fragile solution and risk is someone will eventually push it and compromise credentials.
TBH I'm not even able to include file with absolute path /home/me... anyway approach which will work for you is just using alternative conf file as described in the same doc:
In file /home/me/.foo/credentials.conf you need to include application.conf - Play will fallback it to the file in classpath (this which is under VCS):
include "application.conf"
myCredentials.user="Espinosa"
myCredentials.password="fooBar123"
then run/start your app with this config file locally:
play -Dconfig.file=${HOME}/.foo/credentials.conf ~run
and that's it.
Note: of course it's easier to setup this addition in your IDE (i.e. IntelliJ: Run > Edit configurations) or write a shell script containing this command
For my Play 2.2/Scala application (built with SBT), I would like to deploy different configuration files depending on the environment I'm deploying to (e.g. to couple a deployment with a particular database server). How does one create different variants of the application's configuration file (conf/application.conf) for different deployment targets? Hopefully variants can be generated from a base version?
What I'm used to from .NET is to have a base configuration file (Web.config), which undergoes a certain transformation depending on the profile one is deploying (e.g. Production). Does one use a similar technique in the Play/Scala world?
Alternative configuration files are covered in Play's documentation quite well in section Specifying alternative configuration file.
In short - in application.conf you place default configuration of your app, and additionally you need to create additional files for you environment(s) ie. life.conf, dev.conf etc. In these files you first need to include application.conf (which will read whole default configuration) and next just overwrite only parts which have to be changed - ie. DB credentials, it could be dev.conf:
include "application.conf"
db.default.driver=org.h2.Driver
db.default.url="jdbc:h2:mem:alternative-database-for-dev-testing"
db.default.user=developer
db.default.password="developerpass"
So finally you start your application (after dist) as
./start -Dconfig.resource=dev.conf
or with the Play console
play -Dconfig.resource=dev.conf run
Several tips:
It's good idea to do not place your 'life' DB credentials in default application.conf file, if some dev will forget to include his dev.conf he won't damage the production DB, instead you should put it in prod.conf.
Also these additional configs shouldn't be placed in any VCS (ie. git) repository - creating them directly on target machine (and ignoring in repository) give you sure, that people who shouldn't know the life database credentials won't see it.
It's also possible to use remote alternative config file, which can be useful ie. when you deploying several instances of the same app ie. on several hosts in the cloud.
Each dev can has own config file ie dev_aknuds1.conf, dev_biesior.conf etc, so you can ignore them with one pattern dev_*.conf in repo.
Finally you can just create a shell script (unix) or bat file (Windows) to start using choosen config file like start_dev.sh, run_dev.sh etc. so you won't need to write -Dconfig.resource=... each time
I've had a look around but it's not very clear to me how I can configure a set of environment specific variables for my Play framework application.
As an example, I would like to use an in memory database like h2 for local development but when I move to production or my pre-production environment I would like to be connecting to a postgres database.
How do I configure my app so that it will use the variables relevant to the environment it is being deployed to? This is a Scala Play app.
One option (as documented in the excellent play docs), is to specify conf files during app startup.
Using -Dconfig.resource will search for an alternative configuration file in the application classpath (you usually provide these alternative configuration files into your application conf/ directory before packaging). Play will look into conf/ so you don’t have to add conf/.
$ /path/to/bin/<project-name> -Dconfig.resource=prod.conf
Using -Dconfig.file you can specify an environment specific configuration file not packaged into the application artifacts:
$ start -Dconfig.file=/opt/conf/prod.conf
Using -Dconfig.url you can also specify a configuration file to be loaded from any URL:
$ start -Dconfig.url=http://conf.mycompany.com/conf/prod.conf
Note that you can always reference the original configuration file in a new prod.conf file using the include directive, such as:
include "application.conf"
key.to.override=blah
You can have Puppet or some similar tools to generate the needed parameters in environment.conf and place in a dedicated directory.
Then in application.conf, at the end of the file, have this:
include "file:///[your directory...]/environment.conf"
to override any testing or local values (e.g.DB parameteres) listed above
You can use different configuration files by overriding onLoadConfig method of the Global object like:
object Global extends GlobalSettings {
override def onLoadConfig(config: Configuration, path: File, classloader: ClassLoader, mode: Mode.Mode): Configuration = {
val fileName = s"application.${mode.toString.toLowerCase}.conf"
config ++ Configuration(ConfigFactory.load(fileName))
}
}
This way you have 'application.test.conf' for test mode and 'application.dev.conf' for develop mode whereas you can use another config file in production via '-Dconfig.file' parameter.