Load application.conf from folder in deployed Scala app - scala

I have an application that loads configuration from application.conf using ConfigFactory: lazy val myConfig = ConfigFactory.load(pathToConfig)
The application.conf is initially located in src/main/resources
When I deploy my application I want it to load the config from APP_HOME/conf/application.conf
To do so, I excluded the application.conf from the resource folder when building the Rmp and I have added my APP_HOME/conf to the class path.
jar {
exclude '*.conf'
}
and
startScripts {
classpath += files('src/main/resources')
doLast {
def windowsScriptFile = file getWindowsScript()
def unixScriptFile = file getUnixScript()
println('unix script is ' + unixScriptFile.text)
windowsScriptFile.text = windowsScriptFile.text.replace('%APP_HOME%\\lib\\resources', '%APP_HOME%\\conf')
unixScriptFile.text = unixScriptFile.text.replace('\$APP_HOME/lib/resources', '\$APP_HOME/conf')
println('after unix script is ' + unixScriptFile.text)
}
}
The odd thing is that when I modify the $APP_HOME/conf/application.conf and restart the app, the changes are not picked up: ie the old configuration is still being used
Any idea what might cause this or how I can print where the config is being loaded from would be helpful

With many attempts, I got it to work by calling lazy val myConfig = ConfigFactory.load() without specifying the conf file name or path.
Although it solved my issue I still don't understand why calling load with the file name or file path didn't work

Related

Merging configurations for spark using typesafe library and extraJavaOptions

I'm trying to merge 2 config file (or create a config file based on a single reference file) using
lazy val finalConfig:
Option(System.getProperty("user.resource"))
.map(ConfigFactory.load)
.map(_.withFallback(ConfigFactory.load(System.getProperty("config.resource"))).resolve())
.getOrElse(ConfigFactory.load(System.getProperty("config.resource")))
I'm defining my java variable inside spark using spark-submit ....... --conf spark.driver.extraJavaOptions=-Dconfig.resource=./reference.conf,-Duser.resource=./user.conf ...
My goal is to be able to point a file that is not inside my jar to be used by System.getProperty("..") in my code. I changed the folder for testing (cd ..) and keep getting the same error so I guess spark doesn't care about my java arguments..?
Is there a way to point to a file (or even 2 files in my case) so that they can be merged?
I also tried to include the reference.conf file but not the user.conf file: it recognizes the reference.conf but not the user.conf that i gave with --conf spark.driver.extraJavaOptions=-Duser.resource=./user.conf .
Is there a way to do that? Thanks if you can help
I don't see you doing ConfigFactory.parseFile to loaded a file containing properties.
Typesafe automatically read any .properties file in the class path, all -D parameters passed in to the JVM and then merges them.
I am reading an external property file which is not part of the jar as following. The file "application.conf" is placed on the same directory where the jar is kept.
val applicationRootPath = System.getProperty("user.dir")
val config = Try {
ConfigFactory.parseFile(new File(applicationRootPath + "/" + "application.conf"))
}.getOrElse(ConfigFactory.empty())
appConfig = config.withFallback(ConfigFactory.load()).resolve
ConfigFactory.load() already contains all the properties present on the properties files in the class path and -d parameters. I am giving priority to my external "application.conf" and falling back on default values. For matching keys "application.conf" take precedence over other sources.

read a scala conf file from a maven jar

I would like to read two scala conf file from a maven jar named myconfiguration1.conf and myconfiguration2.conf:
driver= oracle.jdbc.driver.OracleDriver
user = myUser
password = pass
url = myUrl
table = my_table
before generate my jar, i used a line like this:
val myconfig1 = ConfigFactory.parseFile(new File("./my-project/src/main/resources/myconfiguration1.conf"))
val myconfig2 = ConfigFactory.parseFile(new File("./my-project/src/main/resources/myconfiguration2.conf"))
Inside my jar, i have a tree like this
my-project-0.0.1.jar
|___ myconfiguration1.conf
|___ myconfiguration2.conf
Do you have any idea since the two lines don't work with the jar file.
Seems you are using typesafe config to load the conf file, you can directly use ConfigFactory.load("myconfiguration1.conf") and ConfigFactory.load("myconfiguration2.conf") to load the classpath configuration file.
public static load(java.lang.String resourceBasename)

Pick the right config file

I have created two files in my src/main/resources folder
application-dev.conf
which contains
dev {
oracle {
host = "foo"
}
}
and
application-qa.conf
which contains
qa {
oracle {
host = "bar"
}
}
I read this configuration with the following code
val env = args.lift(0).getOrElse("dev")
val parsedConfig = ConfigFactory.parseFile(new File(s"src/main/resources/application-${env}.conf"))
val conf = ConfigFactory.load(parsedConfig)
val config = conf.getConfig(env)
println(config.getString("oracle.host"))
Everything works great at development time and I am able to read the right configuration file based on the environment specified. If I don't specify anything then development is chosen.
However now when I package my jar file as an assembly using sbt assembly and now try to run my application from the command line java -jar ./target/MyApplication.jar
I get an error
Exception in thread "main" com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'dev'
at com.typesafe.config.impl.SimpleConfig.findKeyOrNull(SimpleConfig.java:152)
at com.typesafe.config.impl.SimpleConfig.findOrNull(SimpleConfig.java:170)
at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:184)
at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:189)
My objective is that I should have multiple configuration files and I should be able to choose between them at dev time and also when the application is packaged as a jar.
Based on the suggestion below I modified my code to
val env = args.lift(0).getOrElse("dev")
val parsedConfig = ConfigFactory.parseFile(new File(getClass.getResource(s"/application-${env}.conf").toURI))
val conf = ConfigFactory.load(parsedConfig)
val config = conf.getConfig(env)
println(config.getString("oracle.host"))
works in dev, but when I try to run my assembly it throws an exception
Exception in thread "main" java.lang.IllegalArgumentException: URI is not hierarchical
at java.io.File.<init>(File.java:418)
The config file is not in src/main/resources in your assembly. It usually is at the root of the classpath (unless you have configured the plugin to package it somewhere else).
Try using like below.
ConfigFactory.load(s"application-${env}.conf")
This loads the config file from the classpath.

how to get values from addition conf files included in application.conf in scala

Hi I am trying to create additional conf files and include them in application.conf but i am unable to fetch my values I am definitely making mistakes here is my files
application.conf
include "DirectUserWriteMongoActor"
akka {
loggers = ["akka.event.slf4j.Slf4jLogger"]
loglevel = "DEBUG"
}
DirectUserWriteMongoActor.conf
akka {
actor{
my-dispatcher {
throughput = 10
}
}
}
i want to get the throughput value in my code but i am getting badpath exception Here is my code
val config = ConfigFactory.load("application/DirectUserWriteMongoActor")
println("throughput is "+config.getString("akka.actor.my-dispatcher.throughput"))
All you should need to do here is to call ConfigFactory.load(). When you do that, as long as it can find application.conf on the classpath, it will load it and then properly include the other conf file DirectUserWriteMongoActor due to your use of the include directive.

No configuration setting found for key typesafe config

Im trying to implement a configuration tool typesafehub/config
im using this code
val conf = ConfigFactory.load()
val url = conf.getString("add.prefix") + id + "/?" + conf.getString("add.token")
And the location of the property file is /src/main/resources/application.conf
But for some reason i'm receiving
com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'add'
File content
add {
token = "access_token=6235uhC9kG05ulDtG8DJDA"
prefix = "https://graph.facebook.com/v2.2/"
limit = "&limit=250"
comments="?pretty=0&limit=250&access_token=69kG05ulDtG8DJDA&filter=stream"
feed="/feed?limit=200&access_token=623501EuhC9kG05ulDtG8DJDA&pretty=0"
}
Everything looks configured correctly ?? do i missed something .
thanks,
miki
The error message is telling you that whatever configuration got read, it didn't include a top level setting named add. The ConfigFactory.load function will attempt to load the configuration from a variety of places. By default it will look for a file named application with a suffix of .conf or .json. It looks for that file as a Java resource on your class path. However, various system properties will override this default behavior.
So, it is likely that what you missed is one of these:
Is it possible that src/main/resources is not on your class path?
Are the config.file, config.resource or config.url properties set?
Is your application.conf file empty?
Do you have an application.conf that would be found earlier in your class path?
Is the key: add defined in the application.conf?
Are you using an IDE or sbt?
I had a similar problem while using Eclipse. It simply did not find the application.conf file at first and later on failed to notice edits.
However, once I ran my program via sbt, all worked just fine, including Eclipse. So, I added 'main/resources' to the libraries (Project -> Properties -> Java Build Path -> Libraries", "add class folder"). That might help you as well.
Place your application.conf in the src folder and it should work
I ran into this issue inside a Specs2 test that was driven by SBT. It turned out that the issue was caused by https://github.com/etorreborre/specs2/issues/556. In that case, the Thread's contextClassLoader wasn't using the correct classloader. If you run into a similar error, there are other versions of ConfigFactory.load() that allow you to pass the current class's ClassLoader instead. If you're using Specs2 and you're seeing this issue, use a version <= 3.8.6 or >= 4.0.1.
Check you path. In my case I got the same issue, having application.conf placed in src/main/resources/configuration/common/application.conf
Incorrect:
val conf = ConfigFactory.load(s"/configuration/common/application.conf")
Correct
val conf = ConfigFactory.load(s"configuration/common/application.conf")
it turned out to be a silly mistake i made.
Following that, i does not matter if you use ":" or "=" in .conf file.
Getting the value from example:
server{
proc {
max = "600"
}
}
conf.getString("server.proc.max")
Even you can have the following conf:
proc {
max = "600"
}
proc {
main = "60000"
}
conf.getString("proc.max") //prints 600
conf.getString("proc.min") //prints 60000
I ran into this doing a getString on an integer in my configuration file.
I ran into exactly the same problem and the solution was to replace = with : in the application.conf. Try with the following content in your application.conf:
add {
token: "access_token=6235uhC9kG05ulDtG8DJDA"
prefix: "https://graph.facebook.com/v2.2/"
limit: "&limit=250"
comments: "?pretty=0&limit=250&access_token=69kG05ulDtG8DJDA&filter=stream"
feed: "/feed?limit=200&access_token=623501EuhC9kG05ulDtG8DJDA&pretty=0"
}
Strangely, IntelliJ doesn't detect any formatting or syntax error when using = for me.
in my case it was a stupid mistake,
i m change file name from "application.config" to "application.conf" and its works .
If the application.conf is not getting discovered, you could add this to build.sbt:
unmanagedSourceDirectories in Compile += baseDirectory.value / "main/resources"
Please don't use this to include any custom path. Follow the guidelines and best-practices
As mentioned by others, make sure the application.conf is place in: src/main/resources.
I placed the file there error went away.
Looking at these examples helped me as well:
https://github.com/lightbend/config/tree/main/examples/scala
Use ConfigFactory.parseFile for other locations