Adding Property in Scala Environment Properties
val sysProps = System.getProperties
sysProps.setProperty("current.date.time", LocalDateTime.now().toString())
i'm able to save this property.
I tried accessing this property(current.date.time) in log4j.properties like below
log4j.appender.file.File=C:/Users/vsami/Desktop/Demo_${current.date.time}.log
log4j.appender.file.File=C:/Users/vsami/Desktop/Demo_${env:current.date.time}.log
Log file is getting generated in above location like Demo_.log, Expected :- Demo_2019/11/27T13:21:00.log
Above implementation is not helping me in accessing variable from environment properties and generate log file with expected naming convention.
JVM has properties that can be passed via -D parameter at VM boot. -Dprop=value.
These properties can be read via System.getProperties API call. See docs for more info.
Environment variables are not specified on JVM boot and managed independently from VM by your boot environment (it can be shell, bash etc). You cannot change environment variables in already running VM. These variables can be read via System.getenv()
$ is a look up operator in log4j and can be used to resolve env variables with env: prefix or Main Arguments Lookup with prefix main:.
You could use main:current.date.time and initialise your value as following
MainMapLookup.setMainArguments(Array("--current.date.time", LocalDateTime.now().toString()));
Make sure that MainMapLookup is called before logging is initialised.
Related
I am using ASP.NET Core 5.0 and I have a Web API app deployed to internal cloud where few settings like DB are controlled via environment variables on the host cloud. In my Startup.cs I have the below code
string projectDbConnection = Configuration.GetSection("ProjectDatabaseSettings").GetValue<string>("PROJECT_DB_CONNECTION");
string projectDbName = Configuration.GetSection("ProjectDatabaseSettings").GetValue<string>("PROJECT_DB_NAME");
Here as I understand, when running locally in IIS Express it looks for appsettings.<Environment>.json and they take precedence over appsettings.json values.
But this app is always connecting to the wrong DB when I deployed to Cloud where I mentioned the PROJECT_DB_CONNECTION & PROJECT_DB_NAME as Environment variables for the app.
To make the app read from the Environment variables I had to change the above Code in Startup.cs as
string projectDbConnection = Configuration.GetValue<string>("PROJECT_DB_CONNECTION");
string projectDbName = Configuration.GetValue<string>("PROJECT_DB_NAME");
I am unable to understand the difference between the GetSection.GetValue and just GetValue and why I should use Configuration.GetValue() to direct app to read from Env variables.
what am I missing and when should we use what?
Naming of environment variables
There is kind of a naming convention in the environment variables for nested appsettings to env vars, see naming of environment variables.
Each element in the hierarchy is separated by a double underscore.
In your case it would work if you name the env variable: ProjectDatabaseSettings__PROJECT_DB_CONNECTION.
Config Order
Regarding to Microsoft Documentation there is a order in which the config sources are checked.
ChainedConfigurationProvider : Adds an existing IConfiguration as a source. In the default configuration case, adds the host configuration and setting it as the first source for the app configuration.
appsettings.json using the JSON configuration provider.
appsettings.Environment.json using the JSON configuration provider. For example, appsettings.Production.json and appsettings.Development.json.
App secrets when the app runs in the Development environment.
Environment variables using the Environment Variables configuration provider.
Command-line arguments using the Command-line configuration provider.
The usecase
This is useful when you are developing local using appsettings.json, but run in a cluster or cloud in production where it is more convenient to use environment variables (f.e.: in kubernetes environment variables are set via config maps).
We're developing several Lagom-based Scala micro-services. They are configured using variable replacement in application.conf, eg.
mysql = {
url = "jdbc:mysql://"${?ENV_MYSQL_DATABASE_URL}
During development, we set these variables as Java System Properties via a env.sbt file that calls System.setProperty("ENV_MYSQL_DATABASE_URL", url). This is working fine.
Now I want to deploy this in a container to my local Docker installation. We are using the SbtReactiveAppPlugin to build the Docker image from build.sbt and simply run sbt Docker/publishLocal. This works as expected, a Docker image is created and I can fire it up.
However, passing in environment variables using the standard docker or docker-compose mechanisms does not seem to work. While I can see that the environment variables are set correctly inside the Docker container (verified using env on a bash and also by doing log.debug("ENV_MYSQL_DATABASE_URL via env: " + sys.env("ENV_MYSQL_DATABASE_URL")) inside the service), they are not used by the application.conf and not available in the configuration system. The values are empty/unset (verified through configuration.getString("ENV_MYSQL_DATABASE_URL").toString() and the exceptions thrown by the mysql system and other systems).
The only way I've gotten it to work was by fudging this into the JAVA_OPTS via JAVA_OPTS=-D ENV_MYSQL_DATABASE_URL=..... However, this seems like a hack, and doesn't appear to scale very well with dozens of environment parameters.
Am I missing something, is there a way to easily use the environment variables inside the Lagom application and application.conf?
Thanks!
I've used Lightbend config to configure Lagom services via environment variables in docker containers for many years, so know that it can be done and has been pretty straightforward in my experience.
With that in mind, when you say that they're not used by application.conf, do you mean that they're unset? Note that unless you're passing a very specific option as a Java property, configuration.getString("ENV_MYSQL_DATABASE_URL") will not read from an environment variable, so checking that will not tell you anything about whether mysql.url is affected by the environment variable. configuration.getString("mysql.url") will give you a better idea of what's going on.
I suspect that in fact your Docker image is being built with the dev-mode properties hardcoded in, and since Java system properties take precedence over everything else, they're shadowing the environment variable.
You may find it useful to structure your application.conf along these lines:
mysql_database_url = "..." # Some reasonable default default for dev-mode
mysql_database_url = ${?ENV_MYSQL_DATABASE_URL}
mysql {
url = "jdbc://"${mysql_database_url}
}
In this case, you have a reasonable default for a developer (probably including in the docs some instructions for running MySQL in a way compatible with that configuration). The default can then be overridden via setting a Java property (e.g. JAVA_OPTS=-Dmysql_database_url) or by setting the ENV_MYSQL_DATABASE_URL environment variable.
While I agree with the answer provided by Levi Ramsey, I would suggest you to use typesafe's config to load the your config
My project setup looks as follows:
I would like to dynamically fill in application.conf values.
These values should be read from the correct properties file (${env}.props.properties). The correct properties file depends on the property env which is given with a run or build command (Denv=xxx).
application.conf
key=${my.property.value.read.from.props.properties.file}
key2=...
Thanks in advance!
You can tell Typesafe Config to load a different config file altogether by specifying flag -Dconfig.resource=your.file.properties as you run your application. If the config file is not a bundled resource you can use -Dconfig.file=/path/to/your.file.properties instead. (You can also specify an URL with -Dconfig.url; see https://github.com/typesafehub/config#user-content-standard-behavior for more info)
Doing this will skip loading application.conf altogether so remember to set Play!-specific properties in your own properties-file.
You can try Typesafe ConfigFactory.invalidateCaches to invalid config entries. As api doc says; first make the changes then call above api, followed by load() (one solution would be to have a scheduler that calls it every x interval).
Disclaimer - I haven't tried it myself
https://lightbend.github.io/config/latest/api/com/typesafe/config/ConfigFactory.html#invalidateCaches--
I have REST API, that is developed using Scala and Spray framework. I am able to execute and launch my Api from localhost. The API is connected to the database. The IP Address(localhost) and port of Database is read from the "application.conf" file under the resources.
Everything works fine till I start using Docker. In Docker I have :
1. One Docker container of Rest API
2. One Docker container of Database.
The IP address of Database changes for each docker instance, therefore I need to update my "application.conf" file. Although I can use the hostname of Db instance that remains the same.
My issue is : Can I have two "application.conf" files , one for localhost and one for Docker instance? IS there a way to change the "application.conf" file at the run time.
P.s I am using "sbt run" to run the application and as per documentation it does not support java system properties or environment variables
Yes, you can choose the config at runtime. spray & akka use the typesafe config library which allows setting single settings or the whole configuration using JVM properties.
From the documentation of config:
For applications using application.{conf,json,properties}, system
properties can be used to force a different config source:
config.resource specifies a resource name - not a basename, i.e. application.conf not application
config.file specifies a filesystem path, again it should include the extension, not be a basename
config.url specifies a URL
These system properties specify a replacement for
application.{conf,json,properties}, not an addition. They only
affect apps using the default ConfigFactory.load() configuration. In
the replacement config file, you can use include "application" to
include the original default config file; after the include statement
you could go on to override certain settings.
I'm trying to modify the config of a running Wildfly (inside Docker) using the JBoss CLI. One thing I need to do is
data-source add --connection-url=jdbc:mysql://${DB_PORT_3306_TCP_ADDR}:3306/xplore (...)
I need the resulting config in standalone.xml to literally contain
<connection-url>jdbc:mysql://${DB_PORT_3306_TCP_ADDR}:3306/xplore</connection-url>
DB_PORT_3306_TCP_ADDR is a variable that will be resolved by Wildfly when it's rebooted. It does not exist when I'm CLI-ing.
The problem is that CLI attempts to resolve it (and fails) during my CLI session (instead of injecting it as a plain string), resulting in output like errors and even
<connection-url>$</connection-url>
I've tried to work around this by adding "..." and '...', escaping the $ to \$, removing the brackets, and defining a variable containing my string in the script and inserting that variable (still gets resolved). I've looked up the docs on this, and while there is a command resolve-expression(), there is no inverse, no setting for it, nothing. It doesn't seem possible to prevent the resolution from happening.
How can I insert an unresolved expression into my config via the CLI?
The answer was to issue this command:
data-source add --connection-url=jdbc:mysql://${DB_PORT_3306_TCP_ADDR:}:3306/xplore (...)
With a : after ADDR. This obviously solves the resolution error since it will now default to an empty value, and somehow it actually causes the string value itself to be written. It also turns out it wasn't the CLI that was resolving my variable, but the server itself.
The real solution though was to stop the session, modify the server settings and disable variable resolution entirely, then start a new session.