Accessing environment variable value from service fabric environment xml file - azure-service-fabric

I have a scenario where I need to read an environment variable from the machine where my Service Fabric application is deployed.
More specifically in my cloud.xml (environment file) I want StorageConnectionString to use the value from one of the environment variable that is set on the machine by some other external tool.
<?xml version="1.0" encoding="utf-8"?>
<Application xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Name="fabric:/AppFabricName.ServiceFabric" xmlns="http://schemas.microsoft.com/2011/01/fabric">
<Parameters>
<Parameter Name="StorageConnectionString" Value="%ENVVARIABLE%" />
</Parameters>
</Application>
Is the above valid ? Did not work when I tried even though running SET on cmd prompt did show that variable exists.

It's not straight forward, but here's a way:
Specify a setup entry point (a new executable or script) to read the environment variable and write it to a file in the application's work directory.
When the main entry point comes up, read the file from the work directory.

If you are passing the data via configuration files(or parameters) at deployment and accessing it via code using Context.CodePackageActivationContext.GetConfigurationPackage‌​Object("Config") , the right way should be using <Parameter Name="StorageConnectionString" Value="[ENVVARIABLE]" /> (note that it is encolsed by [ ]) and have a configuration file with a variable called ENVVARIABLE and set the overrides to your service. You should not need to set environment variables, environment variables is other thing.
Please take a look at this SO question to check if will solve the problem.
If your plan is getting an environment variable already set in the machine, unfortunately SF does not support Environment Variable Replacement using tokens like %ENVVARIABLE%.
even though running SET on cmd prompt did show that variable exists.
The other process that changes the environment variable, if it just use the default set, it will set their own process environment variables, to be available to other processes, it has to update environment variable on Machine or User scope(assuming they run on same user), this is why you can't see from command running SET variablename
...
Assuming the variable is set correctly and it is a machine scope variable...
When the Environment Variable is set before the process start, I would suggest you setup an EntryPoint Script to set it at startup, like you would do to a guest executable.
See a NodeJs example here
You could maybe also do write to file and read from your app as Loek suggested, but I think it would be too complicate for too litle.
In case the environment variable is set after the process started, or if it changes later, you should get it from within your application directly instead, you could just use:
System.Environment.GetEnvironmentVariable(variableName,EnvironmentVariableTarget.Machine);
And you pass the variable name via config.
In both cases your user\process must have permission to read Environment Variable from Machine Scope

Related

Using environment variables to configure Docker deployment of Lagom Scala application

We're developing several Lagom-based Scala micro-services. They are configured using variable replacement in application.conf, eg.
mysql = {
url = "jdbc:mysql://"${?ENV_MYSQL_DATABASE_URL}
During development, we set these variables as Java System Properties via a env.sbt file that calls System.setProperty("ENV_MYSQL_DATABASE_URL", url). This is working fine.
Now I want to deploy this in a container to my local Docker installation. We are using the SbtReactiveAppPlugin to build the Docker image from build.sbt and simply run sbt Docker/publishLocal. This works as expected, a Docker image is created and I can fire it up.
However, passing in environment variables using the standard docker or docker-compose mechanisms does not seem to work. While I can see that the environment variables are set correctly inside the Docker container (verified using env on a bash and also by doing log.debug("ENV_MYSQL_DATABASE_URL via env: " + sys.env("ENV_MYSQL_DATABASE_URL")) inside the service), they are not used by the application.conf and not available in the configuration system. The values are empty/unset (verified through configuration.getString("ENV_MYSQL_DATABASE_URL").toString() and the exceptions thrown by the mysql system and other systems).
The only way I've gotten it to work was by fudging this into the JAVA_OPTS via JAVA_OPTS=-D ENV_MYSQL_DATABASE_URL=..... However, this seems like a hack, and doesn't appear to scale very well with dozens of environment parameters.
Am I missing something, is there a way to easily use the environment variables inside the Lagom application and application.conf?
Thanks!
I've used Lightbend config to configure Lagom services via environment variables in docker containers for many years, so know that it can be done and has been pretty straightforward in my experience.
With that in mind, when you say that they're not used by application.conf, do you mean that they're unset? Note that unless you're passing a very specific option as a Java property, configuration.getString("ENV_MYSQL_DATABASE_URL") will not read from an environment variable, so checking that will not tell you anything about whether mysql.url is affected by the environment variable. configuration.getString("mysql.url") will give you a better idea of what's going on.
I suspect that in fact your Docker image is being built with the dev-mode properties hardcoded in, and since Java system properties take precedence over everything else, they're shadowing the environment variable.
You may find it useful to structure your application.conf along these lines:
mysql_database_url = "..." # Some reasonable default default for dev-mode
mysql_database_url = ${?ENV_MYSQL_DATABASE_URL}
mysql {
url = "jdbc://"${mysql_database_url}
}
In this case, you have a reasonable default for a developer (probably including in the docs some instructions for running MySQL in a way compatible with that configuration). The default can then be overridden via setting a Java property (e.g. JAVA_OPTS=-Dmysql_database_url) or by setting the ENV_MYSQL_DATABASE_URL environment variable.
While I agree with the answer provided by Levi Ramsey, I would suggest you to use typesafe's config to load the your config

Failed to read environment Variables in Scala using $ symbol

Adding Property in Scala Environment Properties
val sysProps = System.getProperties
sysProps.setProperty("current.date.time", LocalDateTime.now().toString())
i'm able to save this property.
I tried accessing this property(current.date.time) in log4j.properties like below
log4j.appender.file.File=C:/Users/vsami/Desktop/Demo_${current.date.time}.log
log4j.appender.file.File=C:/Users/vsami/Desktop/Demo_${env:current.date.time}.log
Log file is getting generated in above location like Demo_.log, Expected :- Demo_2019/11/27T13:21:00.log
Above implementation is not helping me in accessing variable from environment properties and generate log file with expected naming convention.
JVM has properties that can be passed via -D parameter at VM boot. -Dprop=value.
These properties can be read via System.getProperties API call. See docs for more info.
Environment variables are not specified on JVM boot and managed independently from VM by your boot environment (it can be shell, bash etc). You cannot change environment variables in already running VM. These variables can be read via System.getenv()
$ is a look up operator in log4j and can be used to resolve env variables with env: prefix or Main Arguments Lookup with prefix main:.
You could use main:current.date.time and initialise your value as following
MainMapLookup.setMainArguments(Array("--current.date.time", LocalDateTime.now().toString()));
Make sure that MainMapLookup is called before logging is initialised.

JBoss 7.1.1 changing JNDI binding in runtime

In JBoss 7.1.1 in standalone mode all JNDI bindings are configured in standalone.xml file in jboss:domain:naming:1.1 subsystem. According to documentation standalone.xml cannot be modified when server is running. I've tried to use JBoss CLI but I don't know how to write/modify resource.
How to change value in JNDI without restarting jboss?
Should help you: https://docs.jboss.org/author/display/AS71/JNDI+Reference
Topic - Binding entries to JNDI:
An example standalone.xml might look like:
<subsystem xmlns="urn:jboss:domain:naming:1.1" >
<bindings>
<simple name="java:global/a" value="100" type="int" />
<object-factory name="java:global/b" module="com.acme" class="org.acme.MyObjectFactory" />
<lookup name="java:global/c" lookup="java:global/b" />
</bindings>
</subsystem>
To add these entries via the CLI:
/subsystem=naming/binding=java\:global\/mybinding:add(binding-type=simple, type=long, value=1000)
To see all all options that are taken by the add command (this can
actually be used to get the description of any CLI command):
/subsystem=naming/binding=*:read-operation-description(name=add)
Have not tried, but i hope this helps!
UPDATE - with tested examples:
Add JDNI name binding java:global/a:
/subsystem=naming/binding=java\:global\/a:add(value=10,binding-type=simple,type=java.lang.Integer)
Read existing JDNI name binding java:global/a:
/subsystem=naming/binding=java\:global\/a:read-resource(include-defaults=true)
Modify JDNI name binding value java:global/a:
/subsystem=naming/binding=java\:global\/a:write-attribute(name=value, value=20)
Remove JDNI name binding java:global/a:
/subsystem=naming/binding=java\:global\/a:remove()
Executing command directly from shell:
./jboss-cli.sh --connect --command="/subsystem=naming/binding=java\:global\/a:read-resource(include-defaults=true)"
The question has a lot of views so I'll answer to it. Inspired by #mik response I've figured out that to change value of some JNDI key e.g. java:jboss/api/key to newApiKey run JBoss CLI and execute:
connect
/subsystem=naming/binding=java\:jboss\/api\/key/:write-attribute(name=value,value=newApiKey)
The change will be immediately visible on server and also stored (updated) in standalone.xmlso it won't get lost after server restart.
I was looking exactly for how to add or modify a JNDI binding at runtime, but I needed to to do this in a Wildfly 9 domain (cluster) configuration (not standalone), which is pretty much the same configuration as JBoss 7. However, I couldn't figure out a way to effectively apply changes without restarting all servers.
To start with, enter the JBoss command line interface and connect to your server domain controller:
./jboss-cli.sh
connect
First, you need to find which profile is active on the server group, so as, on the server root /, enter the following commands:
cd server-group=
ls
Afterwards, you should enter the only server group shown in the listing command (ls) by typing cd {{your_server_group_name}}, then type ls again and look for an entry named profile to check which one is active. Let's consider full-ha as an active profile for our example.
Next, go back to the root configuration folder / by typing cd .. and enter the following commands to navigate and view all JNDI bindings available with their current values:
cd profile=full-ha/subsystem=naming/binding=
:read-resource(recursive=true)
By doing this, you'll be able to see all available JNDI bindings and their attributes, if you want to list only binding names, type ls instead of the last command.
In order to modify a binding, type cd and the name of the binding listed in the previous command. Let's suppose you want to change the value of a binding named java:/webservice.url, then you should enter
cd java\:\/webservice.url
Notice that is necessary to quote some characters in your binding name such as : (colon) and / (slash) with a backslash (\).
To modify an attribute within this binding you should use the :write-attribute command. In this example, let's suppose you want to modify (or add) an attribute named "value" with its content as "this is a value":
:write-attribute(name=value,value="this is a value")
So as to apply this change, you'll need to restart all servers in the cluster by typing the following command:
/server-group={{server-group-name}}:restart-servers
If you want to know more commands to add or remove JNDI bindings check this jboss-cli snippets page
This configuration has been tested successfully in Wildfly 9.0.1

when I add "<Runtime executionContext="elevated" />" to my web role definition, and attempt to deploy, it fails to start

When I add "" to my ServiceDefinition.csdef, it fails to start when publishing:
<WebRole name="xxx" vmsize="Small" enableNativeCodeExecution="true">
<Runtime executionContext="elevated" />
<Sites>...
Everything works with it gone. Need to add it to specify machine key in Azure SDK 1.3, as described here: http://msdn.microsoft.com/en-us/library/gg494983.aspx
Anyone else run into this?
Would running under admin privileges cause other code to break? When I RDP in, the error I find when I run on localhost is not related to this, but it's code that works when this runtime line is removed.
I just checked the Service Definition Schema and the Runtime
element does not exist inside WebRole.
Actually it does.
<WebRole name="<web-role-name>" vmsize="[ExtraSmall|Small|Medium|Large|ExtraLarge]" enableNativeCodeExecution="[true|false]">
...
<Runtime executionContext="[limited|elevated]">
I just checked the Service Definition Schema and the Runtime element does not exist inside WebRole.
If you would like to have Full Trust, you should go to yor CloudService project properties and set it there. But i really don't know if you need that to specify the machine key.
Best regards

no-eclipse tomcat env variable equivalent

I can set an environment variable with Eclipse when I click on the Tomcat Server and then Open Launch configuration -> Environment -> Variable, Value. What is no-gui, no-eclipse textual-xml equivalent of this variable setting?
Context Parameters
See: Context Parameters section of The Context Container page.
You can configure named values that will be made visible to the web application as servlet context initialization parameters by nesting <Parameter> elements inside this element. For example, you can create an initialization parameter like this:
<Context ...>
...
<Parameter name="companyName" value="My Company, Incorporated"
override="false"/>
...
</Context>
This is equivalent to the inclusion of the following element in the web application deployment descriptor (/WEB-INF/web.xml):
<context-param>
<param-name>companyName</param-name>
<param-value>My Company, Incorporated</param-value>
</context-param>
but does not require modification of the deployment descriptor to customize this value.
If you're using gnu-linux operating system you can edit the catalina.sh script to export you environment variable. First line of the script (after comments, of course) should be
export variableName=variableValue
The credit is for Lucas that told me how to do.
If you meant arguments passed to the launching of your servlet context, see the Answer by miku.
Environment Entries
If you meant the Environment Entries feature, use <Environment> rather than <Parameter> seen in that other Answer.
Here is an example where I set a flag signaling if I am running the web app for development, testing, acceptance, education, or production.
<?xml version="1.0" encoding="UTF-8"?>
<Context>
<!-- Domain: DEV, TEST, ACPT, ED, PROD -->
<Environment name = "work.basil.example.deployment-mode"
description = "Signals whether to run this web-app with development, testing, acceptance, education, or production settings."
value = "DEV"
type = "java.lang.String"
override = "false"
/>
</Context>
You have your choice of a few places to place such XML. For this example, we would want to externalize the file, keeping it outside the WAR file containing our web-app. By externalizing, we can deploy our WAR to any of the dev, test, acpt, ed, or prod servers without needing special builds or manual editing. So for this purpose I like to use the feature of Tomcat allowing for an XML file named the same as the context. The file is place in the conf folder, in which you create nested folders named after the Tomcat engine and the host. For example, on my development machine that would be my_tomcat_base_folder/conf/Catalina/localhost/example.xml where example is the name of my context, Catalina is the engine name, and localhost is the host name (not using a domain name while in development).
The syntax of the above XML is Tomcat-specific. For use in Servlet-standard places, such as the /WEB-INF/web.xml file, the equivalent as documented in the Servlet 4 spec section 14.4.21 would be:
<env-entry>
<env-entry-name>work.basil.example.deployment-mode</env-entry-name>
<description>Signals whether to run this web-app with development, testing, acceptance, education, or production settings.</description>
<env-entry-value>DEV</env-entry-value>
<env-entry-type>java.lang.String</env-entry-type>
</env-entry>