I have read the documentation concerning falling back to environment variables at https://github.com/typesafehub/config/blob/master/HOCON.md#substitution-fallback-to-environment-variables. My understanding was that it would pickup any envars. So for instance, if from the shell I was able to do echo $HOSTNAME and see a non-empty response then HOCON should do that as well.
In my application.conf I have a line
akka.remote.netty.tcp.hostname = ${HOSTNAME}
However, my app is not happy with this and fails to start with.
/conf/application.conf: 9: Could not resolve substitution to a value: ${HOSTNAME}
Is this a user issue? A shell issue? I am able to login as the user and echo $HOSTNAME
Tagging this scala and akka since that userbase probably has the most exposure to HOCON
The reason for HOCON not picking up the envar is that my app runs as a linux service (Centos 6.5) which clears away most environment variables.
See https://unix.stackexchange.com/questions/44370/how-to-make-unix-service-see-environment-variables for a relevant description of the issue
this is a shot in the dark, but are you using an older version of typesafe-config? maybe its a newer-ish feature? the feature seems to be advertised as you describe, but if you are pulling in typesafe-config as a transient dependency (say from akka), maybe you are getting an older version.
what happens if you remove the substitution in your .conf file (so parsing is successful) and then print out the contents of ConfigFactory.systemEnvironment()? for reference: http://typesafehub.github.io/config/latest/api/com/typesafe/config/ConfigFactory.html#systemEnvironment--
HOSTNAME isn't an environment variable. It's a bash internal variable. See https://superuser.com/questions/132489/hostname-environment-variable-on-linux for more details.
Related
Hi I'm spinning out Pantheon's Localdev for testing and I'm getting this warning message on the logs
You have a lot of keys!
Any ideas what is causing this warning?
I think this is related to lando. This thread on Github suggests:
To disable the warning you can run lando config you will see a maxKeyWarning setting. This value can be overridden in a lando global config file, i.e. set the value that matches your number of used keys..
As documented in the SSH keys config page, the keys key is meant to be used in combination with a user-space Lando override file. This file is .gitignored and is meant to provide user-specific overrides, eg you would not put ALL the keys of ALL the users here, just yours.
We're developing several Lagom-based Scala micro-services. They are configured using variable replacement in application.conf, eg.
mysql = {
url = "jdbc:mysql://"${?ENV_MYSQL_DATABASE_URL}
During development, we set these variables as Java System Properties via a env.sbt file that calls System.setProperty("ENV_MYSQL_DATABASE_URL", url). This is working fine.
Now I want to deploy this in a container to my local Docker installation. We are using the SbtReactiveAppPlugin to build the Docker image from build.sbt and simply run sbt Docker/publishLocal. This works as expected, a Docker image is created and I can fire it up.
However, passing in environment variables using the standard docker or docker-compose mechanisms does not seem to work. While I can see that the environment variables are set correctly inside the Docker container (verified using env on a bash and also by doing log.debug("ENV_MYSQL_DATABASE_URL via env: " + sys.env("ENV_MYSQL_DATABASE_URL")) inside the service), they are not used by the application.conf and not available in the configuration system. The values are empty/unset (verified through configuration.getString("ENV_MYSQL_DATABASE_URL").toString() and the exceptions thrown by the mysql system and other systems).
The only way I've gotten it to work was by fudging this into the JAVA_OPTS via JAVA_OPTS=-D ENV_MYSQL_DATABASE_URL=..... However, this seems like a hack, and doesn't appear to scale very well with dozens of environment parameters.
Am I missing something, is there a way to easily use the environment variables inside the Lagom application and application.conf?
Thanks!
I've used Lightbend config to configure Lagom services via environment variables in docker containers for many years, so know that it can be done and has been pretty straightforward in my experience.
With that in mind, when you say that they're not used by application.conf, do you mean that they're unset? Note that unless you're passing a very specific option as a Java property, configuration.getString("ENV_MYSQL_DATABASE_URL") will not read from an environment variable, so checking that will not tell you anything about whether mysql.url is affected by the environment variable. configuration.getString("mysql.url") will give you a better idea of what's going on.
I suspect that in fact your Docker image is being built with the dev-mode properties hardcoded in, and since Java system properties take precedence over everything else, they're shadowing the environment variable.
You may find it useful to structure your application.conf along these lines:
mysql_database_url = "..." # Some reasonable default default for dev-mode
mysql_database_url = ${?ENV_MYSQL_DATABASE_URL}
mysql {
url = "jdbc://"${mysql_database_url}
}
In this case, you have a reasonable default for a developer (probably including in the docs some instructions for running MySQL in a way compatible with that configuration). The default can then be overridden via setting a Java property (e.g. JAVA_OPTS=-Dmysql_database_url) or by setting the ENV_MYSQL_DATABASE_URL environment variable.
While I agree with the answer provided by Levi Ramsey, I would suggest you to use typesafe's config to load the your config
BitBake fails for me because it can't find https://www.example.com.
My computer is an x86-64 running native Xubuntu 18.04. Network connection is via DSL. I'm using the latest versions of the OpenEmbedded/Yocto toolchain.
This is the response I get when I run BitBake:
$ bitbake -k core-image-sato
WARNING: Host distribution "ubuntu-18.04" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
ERROR: OE-core's config sanity checker detected a potential misconfiguration.
Either fix the cause of this error or at your own risk disable the checker (see sanity.conf).
Following is the list of potential problems / advisories:
Fetcher failure for URL: 'https://www.example.com/'. URL https://www.example.com/ doesn't work.
Please ensure your host's network is configured correctly,
or set BB_NO_NETWORK = "1" to disable network access if
all required sources are on local disk.
Summary: There was 1 WARNING message shown.
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
The networking issue, the reason why I can't access www.example.com, is a question for the SuperUser forum. My question here is, why does BitBake rely on the existence of www.example.com? What is it about that website that is so vital to BitBake's operation? Why does BitBake post an Error if it cannot find https://www.example.com?
At this time, I don't wish to set BB_NO_NETWORK = "1". I would rather understand and resolve the root cause of the problem first.
Modifying poky.conf didn't work for me (and from what I read, modifying anything under Poky is a no-no for a long term solution).
Modifying /conf/local.conf was the only solution that worked for me. Simply add one of the two options:
#check connectivity using google
CONNECTIVITY_CHECK_URIS = "https://www.google.com/"
#skip connectivity checks
CONNECTIVITY_CHECK_URIS = ""
This solution was originally found here.
For me, this appears to be a problem with my ISP (CenturyLink) not correctly resolving www.example.com. If I try to navigate to https://www.example.com in the browser address bar I just get taken to the ISP's "this is not a valid address" page.
Technically speaking, this isn't supposed to happen, but for whatever reason it does. I was able to work around this temporarily by modifying the CONNECTIVITY_CHECK_URIS in poky/meta-poky/conf/distro/poky.conf to something that actually resolves:
# The CONNECTIVITY_CHECK_URI's are used to test whether we can succesfully
# fetch from the network (and warn you if not). To disable the test set
# the variable to be empty.
# Git example url: git://git.yoctoproject.org/yocto-firewall-test;protocol=git;rev=master
CONNECTIVITY_CHECK_URIS ?= "https://www.google.com/"
See this commit for more insight and discussion on the addition of the www.example.com check. Not sure what the best long-term fix is, but the change above allowed me to build successfully.
If you want to resolve this issue without modifying poky.conf or local.conf or any of the files for that matter, just do:
$touch conf/sanity.conf
It is clearly written in meta/conf/sanity.conf that:
Expert users can confirm their sanity with "touch conf/sanity.conf"
If you don't want to execute this command on every session or build, you can comment out the line INHERIT += "sanity" from meta/conf/sanity.conf, so the file looks something like this:
Had same issue with Bell ISP when accessing example.com gave DNS error.
Solved by switching ISP's DNS IP to Google's DNS (to avoid making changes to configs):
https://developers.google.com/speed/public-dns/docs/using
I have created Azure Batch pool with Linux Machine and specified Application Package for the Pool.
My command line is
command='python $AZ_BATCH_APP_PACKAGE_scriptv1_1/tasks/XXX/get_XXXXX_data.py',
python3: can't open file '$AZ_BATCH_APP_PACKAGE_scriptv1_1/tasks/XXX/get_XXXXX_data.py':
[Errno 2] No such file or directory
when i connect to node and look at working directory non of the Application Package files are present there.
How do i make sure that files from Application Package are available in working directory or I can invoke/execute files under Application Package from command line ?
Make sure that your async operation have proper await in place before you start using the package in your code.
Also please share your design \ pseudo-code scenario and how you are approaching it as a design?
Further to add:
Seems like this one is pool level package.
The error seems like that the application env variable is either incorrectly used or there is some other user level issue. Please checkout linmk below and specially the section where use of env variable is mentioned.
This seems like user level issue because In case of downloading the package resource, if there will be an error it will be visible to you via exception handler or at the tool level is you are using batch explorer \ Batch-labs or code level exception handling.
https://learn.microsoft.com/en-us/azure/batch/batch-application-packages
Reason \ Rationale:
If the pool level or the task application has error, an error-list will come back if there was an error in the application package then it will be returned as the UserError or and AppPackageError which will be visible in the exception handle of the code.
Key you can always RDP into your node and checkout the package availability: information here: https://learn.microsoft.com/en-us/azure/batch/batch-api-basics#connecting-to-compute-nodes
I once created a small sample to help peeps around so this resource might help you to checkeout the use here.
Hope rest helps.
On Linux, the application package with version string is formatted as:
AZ_BATCH_APP_PACKAGE_{0}_{1}
On Windows it is formatted as:
AZ_BATCH_APP_PACKAGE_APPLICATIONID#version
Where 0 is the application name and 1 is the version.
$AZ_BATCH_APP_PACKAGE_scriptv1_1 will take you to the root folder where the application was unzipped.
Does this "exact" path exist in that location?
tasks/XXX/get_XXXXX_data.py
You can see more information here:
https://learn.microsoft.com/en-us/azure/batch/batch-application-packages
Edit: Just saw this question: "or can I invoke/execute files under Application Package from command line"
Yes you can invoke and execute files from the application package directory with the environment variable above.
If you type env on the node you will see the environment variables that have been set.
I'm trying to modify the config of a running Wildfly (inside Docker) using the JBoss CLI. One thing I need to do is
data-source add --connection-url=jdbc:mysql://${DB_PORT_3306_TCP_ADDR}:3306/xplore (...)
I need the resulting config in standalone.xml to literally contain
<connection-url>jdbc:mysql://${DB_PORT_3306_TCP_ADDR}:3306/xplore</connection-url>
DB_PORT_3306_TCP_ADDR is a variable that will be resolved by Wildfly when it's rebooted. It does not exist when I'm CLI-ing.
The problem is that CLI attempts to resolve it (and fails) during my CLI session (instead of injecting it as a plain string), resulting in output like errors and even
<connection-url>$</connection-url>
I've tried to work around this by adding "..." and '...', escaping the $ to \$, removing the brackets, and defining a variable containing my string in the script and inserting that variable (still gets resolved). I've looked up the docs on this, and while there is a command resolve-expression(), there is no inverse, no setting for it, nothing. It doesn't seem possible to prevent the resolution from happening.
How can I insert an unresolved expression into my config via the CLI?
The answer was to issue this command:
data-source add --connection-url=jdbc:mysql://${DB_PORT_3306_TCP_ADDR:}:3306/xplore (...)
With a : after ADDR. This obviously solves the resolution error since it will now default to an empty value, and somehow it actually causes the string value itself to be written. It also turns out it wasn't the CLI that was resolving my variable, but the server itself.
The real solution though was to stop the session, modify the server settings and disable variable resolution entirely, then start a new session.