if multiple XML configuration files exist in CLASSPATH, so what will happen? Just pick out a file randomly and ignore others, or the latter configuration overrides the former?
I just tried, does not support multiple configuration files
Related
My project have multiple spark applications (each with its own spark session) and all of them refers to same log4j.properties file. I want to specify different log4j.properties file (log4j_1.properties) for some of the jobs. I tried below but it is still taking configuration from original file (log4j.properties) only.
Not sure what I am missing here?
conf.set("spark.driver.extraJavaOptions", "-Dlog4j.configuration=file:log4j_1.properties")
Note:
I can see above option set on SparkUI environment tab
Both log4j_1.properties and log4j.properties are there in classpath
Try
-Dlog4j.configuration=log4j_1.properties
I have a Maven project written in Scala and deployed as a .jar. The project has the typical structure:
project-root
|__src
|__main
| |__scala
| |__resources
|
|__test
|__scala
|__resources
I have many hardcoded values in the application that I'd like to extract to a configuration file.
I could just create a text config file anywhere, read it from the code and use the parameters, but I'm looking for a best practice/approach. I'd like to take this things into account:
Is there a way such that I can change config values and not have to rebuild the .jar? That would be very practical.
Where would be the best location for the file? E. g. create a project-root/src/config directory and put the file there?
Any remarkable reasons about choosing among different formats? I'd go with XML, but I've seen quite a bit of YAML and HOCON around...
If you want to be able to change configuration values without building the jar, you will want your config external to the artifact. There are ways to modify the contents of the built jar but I don't think that is what you are wanting to do. If you want to have config you can change on the fly, keep it out of your jar and reference it on the Classpath at runtime.
If you want to keep it on your artifact, and rebuild when you change values, keep the configuration file in src/main/resources. Maven will automatically package it into your jar for you at the root of the archive.
As for format, that's probably personal preference. Reading YAML usually requires another library such as snakeyml to parse effectively, so if you are trying to keep your library light on dependencies, maybe look at .config or .properties files instead. Otherwise, XML and YAML are always nice for complex configuration.
I come from the C# .NET world and as I build my Scala project I noticed how many different ways there are to do things in the open source community. Anyways in .Net C# project we have the Web.config or the App.config file which is very useful to specify app keys that are used globally and it is also helpful for transformation for different environments and such.
What would be the equivalent for a Scala SBT project? Do you know a place to store your connection strings and other app settings?
There is no universal configuration file name. The basic way is to use Properties, see e.g. how to read properties file in scala, and call the file your-application-name.properties. But there is a popular library called simply Config which I'd recommend; as https://github.com/typesafehub/config#standard-behavior says, its standard names for config files are
The convenience method ConfigFactory.load() loads the following (first-listed are higher priority):
system properties
application.conf (all resources on classpath with this name)
application.json (all resources on classpath with this name)
application.properties (all resources on classpath with this name)
reference.conf (all resources on classpath with this name)
The idea is that libraries and frameworks should ship with a reference.conf in their jar. Applications should provide an application.conf , or if they want to create multiple configurations in a single JVM, they could use ConfigFactory.load("myapp") to load their own myapp.conf . (Applications can provide a reference.conf also if they want, but you may not find it necessary to separate it from application.conf .)
As #ashalynd's answer says, in order for any such library to see your configuration files, they should go into src/{main,test}/resources.
I think you are talking about application.conf. It can be found in /src/main/resources or in /src/test/resources (affecting main and test settings, respectively). If application.conf is not present in /src/test/resources, the main application.conf will be used for the tests.
I have a custom configuration file for my database, and I want to externalize it when my Play project is packaged into a war.
I tried javaOptions in build.sbt with the -DConfig.file java command, but my file is not detected...
my line : javaOptions in run += "-Dconfig.file=conf/database.conf"
and my deployed war on a Tomcat server is :
/mywar/META-INF/MANIFEST.MF
/mywar/WEB-INF/lib/myplayproject.jar (with other dependencies)
I put my conf file in lib directory : /lib/conf/database.conf
Where am I doing wrong ?
(based on this link : http://www.scala-sbt.org/0.12.3/docs/Detailed-Topics/Forking.html)
If you deploy your war on Tomcat, I don't see how sbt is involved, which may explain why your option is ignored.
Play is not really meant to be deployed as a war. I think the easiest thing for you would be to manually load the config file in your code via the typesafe config library.
When deploying to a container you aren't in control of the JVM process so you can't specify command line options to set system properties.
Your easiest approach here is probably to just rename your config file to reference.conf or application.conf. Those two names are auto-loaded by the configuration library from the classpath.
You could also edit reference.conf or application.conf to add include "database.conf" if you want to keep a separate file.
Otherwise you could manually load a config file, e.g. with ConfigFactory.parseResource to search the classpath.
In addition to including classpath resources you can include files and urls out of the box. Use the include url("http://example.com") or include file("foo.conf") syntax. By hand (in code) ConfigFactory can of course also parse urls and files.
If that isn't flexible enough, it's also possible to customize includes in two ways.
One is to create a custom ConfigIncluder http://typesafehub.github.io/config/latest/api/com/typesafe/config/ConfigIncluder.html
The other is to create a custom URL protocol in the usual Java way, see Creating custom URI scheme using URI class and http://www.cooljeff.co.uk/2009/12/12/custom-url-protocols-and-multiple-classloaders/ for example. Then use include url("myproto:foobar")
What is the recommended way to have jars from my $MULE_HOME/apps/app-name/lib/ directory take precedence over jars in $MULE_HOME/lib/opt directory?
MES 3.2.1's opt directory has mail-1.4.3.jar and I need mail-1.4.4.jar, the latter of which is in my application's lib directory. It appears, however, that the order in which these are loaded is inconsistent or fixed with Mule's libraries coming first.
I have hacked a solution by replacing mail-1.4.3 with mail-1.4.4 in $MULE_HOME/lib/opt, but would like a more robust way of doing this so I don't have to make the same change in all my Mule instances.
Thank you for your time!
You can embed your own JARs in your application's lib directory and use the loader.override property of the mule-deploy.properties deployment descriptor, documented here.
If you want to learn more about classloading in Mule 3, turn to this page.
In your case, your deployment descriptor should look like:
loader.override=-javax.mail