Thorntail: dynamic values on project_defaults.yml - kubernetes

Working with Thorntail/Kubernetes, is it possible to use dynamic values on project_defaults.yml? For example:
thorntail:
ejb3:
thread-pools:
default:
max-threads: ${my.variable}
Where my.variable will be configured on the dashboard of a specific module.
The idea is to replace standalone.xml values without performing a new deploy every time I need to change the value.

That is in theory possible, because Thorntail is based on WildFly, which allows changing some configuration values without restart. Your example refers to a WildFly configuration value, so it might be possible. Those configuration values in Thorntail that don't come from WildFly don't allow for this at all.
You can use the management fraction to let Thorntail expose the WildFly management interface. Then, you can use the WildFly management client (ModelControllerClient) to issue management operations. However, I'd discourage you from doing this, as Thorntail isn't really designed for it.
(Also, Thorntail has reached its end of life. See here for more: https://thorntail.io/posts/the-end-of-an-era/)

We can do that by configuring the variable as an environment variable. Using the previous example:
thorntail:
ejb3:
thread-pools:
default:
max-threads: ${env.my_variable:default_value}
And then we configure my_variable on the dashboard of the desired module.
It is important to notice that we should define a default value, because if that thorntail property is numeric and the environment variable is not defined on the module, we will face a parsing problem since the parser will interpret the variable name as a number. This is a real example:
Kubernets module:
RCNT_MAX_THREADS is defined with 12
project_defaults.yml:
thorntail:
ejb3:
thread-pools:
default:
max-threads: ${env.RCNT_MAX_THREADS:10}
This will cause the application to use max-threads with the value 12. If we don't define RCNT_MAX_THREADS on the module, the value 10 will be used instead.
There is also a very useful option to track what values are really being used: https://docs.thorntail.io/2.5.0.Final/#the-usage-txt-file_thorntail

Related

Externalize configuration Akka

I am new in Akka and I faced the problem below.
I want to externalize the configuration in my App. More specifically, I have some variables that are different per each environment. So I think that I can have specific environment variables (secrets, etc) for each environment.
But what I can do with some variables (non-secrets) which are different per each environment?
What is the difference between, dev.properties, application.conf, deploy.json files?
What is the proper way to load variables from those files?
There's a few options:
Environment variables and using the support for substitution (there is also support for having default in the file and only use the environment vars if they are set). - https://github.com/lightbend/config#optional-system-or-env-variable-overrides
System properties, if you set a system property when you start the JVM, and that system property matches a path in the config file, it overrides the setting
You can point to an alternative application.conf file using a system property - https://github.com/lightbend/config#standard-behavior
If that is not enough you could also do completely custom logic around selecting logic by programmatically creating a Config instance and passing that to the ActorSystem when you create it.
The dev.properties and deploy.json is AFAIK not related to Akka, unless something specifically done in your application.

How can I create a transient domain in libvirt?

How can I create a transient domain using libvirt? (Using QEMU/KVM as back-end)
The documentation discusses the difference between transient and persistent domains at this link: http://wiki.libvirt.org/page/VM_lifecycle#Transient_guest_domains_vs_Persistent_guest_domains
Still, I haven't found any concrete example on how to create one.
The only pointer I found is in this email: https://www.redhat.com/archives/libvirt-users/2011-August/msg00057.html, where the maintainer suggests to add the <transient/> tag in the <disk> field of the XML's description.
When I tried, I got this disappointing answer: "libvirtError: unsupported configuration: transient disks not supported yet".
Is this feature really "not supported yet", or am I missing something? The documentation makes me think that this should be supported.
Any answer related to the C or Python binding, virsh, or virt-manager will be highly appreciated!
Using virsh
If you are using virsh, than there are commands:
define -- This command takes an XML file as it's parameter and makes the domain known to libvirt (you can reference that domain by using its name or UUID).
start -- This command takes the domain name or UUID as its parameter and starts (boots) the domain.
create -- This command takes an XML file as it's parameter and creates (starts) the domain with settings described in that file. Depending on whether the domain is known to libvirt (previously defined with that UUID) it may result in two things:
if it is already defined, the known domain is marked as started, it is persistent domain, but it is started with the settings supplied and not those it was defined with).
in case it is not defined, the domain started is now a transient domain (it disappears when it is destroyed, shuts down, etc.).
undefine -- This command takes a domain name or UUID (or ID if it's started) and makes it unknown to libvirt, but if that domain is running it doesn't destroy it, just marks it transient.
C functions
In C, the APIs that virsh is using for these commands are:
define -- virDomainDefineXML
start -- virDomainCreate
create -- virDomainCreateXML
undefine -- virDomainUndefine
Notes:
The names may be a little bit confusing, but due to backward compatibility it is kept from Xen times.
Most of those mention commands have parameters which may alter the behavior, these may cause using different C functions for the purpose.

Is it possible to manage ldap contexts in jboss?

In the same way that JBoss can manage jdbc data sources can it manage LDAP contexts using JNDI?
Yes, this is just a Tomcat custom <Resource>, see the Tomcat documentation.
In brief, just define a and provide your own implementation of an LdapContextFactory. JBoss will call your getObjectInstance() method with a first parameter that you can cast to a Reference, traverse all its RefAddrs, and get your option names via RefAddr.getType(), and their values via RefAddr.getContent(). Then you use whatever you have passed there to create your LdapContext and return it.

How to handle environment-specific application configuration organization-wide?

Problem
Your organization has many separate applications, some of which interact with each other (to form "systems"). You need to deploy these applications to separate environments to facilitate staged testing (for example, DEV, QA, UAT, PROD). A given application needs to be configured slightly differently in each environment (each environment has a separate database, for example). You want this re-configuration to be handled by some sort of automated mechanism so that your release managers don't have to manually configure each application every time it is deployed to a different environment.
Desired Features
I would like to design an organization-wide configuration solution with the following properties (ideally):
Supports "one click" deployments (only the environment needs to be specified, and no manual re-configuration during/after deployment should be necessary).
There should be a single "system of record" where a shared environment-dependent property is specified (such as a database connection string that is shared by many applications).
Supports re-configuration of deployed applications (in the event that an environment-specific property needs to change), ideally without requiring a re-deployment of the application.
Allows an application to be run on the same machine, but in different environments (run a PROD instance and a DEV instance simultaneously).
Possible Solutions
I see two basic directions in which a solution could go:
Make all applications "environment aware". You would pass the environment name (DEV, QA, etc) at the command line to the app, and then the app is "smart" enough to figure out the environment-specific configuration values at run-time. The app could fetch the values from flat files deployed along with the app, or from a central configuration service.
Applications are not "smart" as they are in #1, and simply fetch configuration by property name from config files deployed with the app. The values of these properties are injected into the config files at deploy-time by the install program/script. That install script takes the environment name and fetches all relevant configuration values from a central configuration service.
Question
How would/have you achieved a configuration solution that solves these problems and supports these desired features? Am I on target with the two possible solutions? Do you have a preference between those solutions? Also, please feel free to tell me that I'm thinking about the problem all wrong. Any feedback would be greatly appreciated.
We've all run into these kinds of things, particularly in large organizations. I think it's most important to manage your own expectations first, and also ask whether it's really necessary to tell every system and subsystem on a given box to "change to DEV mode" or "change to PROD mode". My personal recommendation is as follows:
Make individual boxes responsible for a different stage - i.e. "this is a DEV box", and "this is a PROD box".
Collect as much of the configuration that differs from box to box in one location, even if it requires soft links or scripts that collect the information to then print out.
A. This way, you can easily "dump this box's configuration" in two places and see what differs, for example after a new deployment.
B. You can also make configuration changes separate from software changes, at least to some degree, which is a good way to root out bugs that happen at release time.
Then have everything base its configuration on something/somewhere that is not baked-in or hard-coded - just make sure to collect and document it in that one location. It almost doesn't matter what the mechanism is, which is a good thing, because some systems just don't want to be forced to use some mechanisms or others.
Sorry if this is too general an answer - the question was very general. I've worked in several large software-based organizations before, and this seemed to be the best approach. Using a standalone server as "one unit of deployment" is the most realistic scenario (though sometimes its expensive), since applications affect each other, and no matter how careful you are, you destabilize a whole system when you move any given gear or cog.
The alternative gets very complex very quickly. You need to start rewriting the applications that you have control over in order to have them accept a "DEV" switch, and you end up adding layers of kludge to the ones you don't have control over. Usually, the ones you don't have control over at least base their properties on something defined on a system-wide level, unless they are "calling the mothership for instructions".
It's easier to redirect people to a remote location and have them "use DEV" vs "use PROD" than it is to "make this machine run like DEV" vs "make this machine run like PROD". And if you're mixing things up, like having a DEV task run together on the same box as a PROD task, then that's not a realistic scenario anyways: I guarantee that eventually you will be granting illegal DEV-only access to somebody on PROD, and you'll have a DEV task wipe out a PROD database.
Hope this helps. Let me know if you'd like to discuss more specifics involved.
I personally prefer solution 2 (the app should know itself, by its configuration, what environment it is running in). With solution 1 (pass the environment name as a startup parameter) the danger of using the wrong environment specifier is much too high. Accessing the TEST database from PROD code and vice versa may cause mayhem, if the two installed code bases are not of the same version, as is often the case.
My current project uses solution 1, but I don't like that. A previous project I worked on used a variation of solution 2: The build process generated one setup file for every environment, making sure that they contained the same code base but appropriate configuration paramters. That worked like a charm, but I know it contradicts the paradigm that the "exact same build files must be deployed everywhere".
I think I have asked a related, self-answered, question, before I read this one : How to organize code so that we can move and update it without having to edit the location of the configuration file? . So, on that basis, I provide an answer here. I don't like the idea of "smart" application (solution 1 here) for such a simple task as finding environment settings. It seems a complicated framework for something that should be simple. The idea of an install script (solution 2 here) is powerful, but it is useful to allow the user to change the content of the config file, but would it allow to change the location of this config file? What is this "central configuration service", where is it located? My answer is that I would go with option 2, if the goal is to set the content of the configuration file, but I feel that the issue of the location of this configuration file remains unanswered here.
If you're using JSON to store/transmit configuration (or can use JSON in your pre-deploy process to output to some other format) you can annotate key/property names for environment/context-specific values with arbitrary or environment-specific suffixes, and then dynamically prefer/discriminate them at build/deploy/run/render -time, while leaving un-annotated properties alone.
We have used this to avoid duplicating entire configuration files (with the associated problems well known) AND to reduce repetition. The technique is also perfect for internationalization (i18n) -- even within the same file, if desired.
Example, snippet of pre-processed JSON config:
var config = {
'ver': '1.0',
'help': {
'BLURB': 'This pre-production environment is not supported. Contact Development Team with questions.',
'PHONE': '808-867-5309',
'EMAIL': 'coder.jen#lostnumber.com'
},
'help#www.productionwebsite.com': {
'BLURB': 'Please contact Customer Service Center',
'BLURB#fr': 'S\'il vous plaît communiquer avec notre Centre de service à la clientèle',
'BLURB#de': 'Bitte kontaktieren Sie unseren Kundendienst!!1!',
'PHONE': '1-800-CUS-TOMR',
'EMAIL': 'customer.service#productionwebsite.com'
},
}
... and post-processed (in this case, at render time) given dynamic, browser-environment-known location.hostname='www.productionwebsite.com' and navigator.language of 'de'):
prefer(config,['www.productionwebsite.com','de']); // prefer(obj,string|Array<string>)
JSON.stringify(config); // {
'ver': '1.0',
'help': {
'BLURB': 'Bitte kontaktieren Sie unseren Kundendienst!!1!',
'PHONE': '1-800-CUS-TOMR',
'EMAIL': 'customer.service#productionwebsite.com'
}
}
If a non-annotated ('base') property has no competing annotated property, it is left alone (presumably global across environments) otherwise its value is replaced by an annotated value, if the suffix matches one of the inputs to the preference/discrimination function. Annotated properties that do not match are dropped entirely.
You can mix and match this behaviour to annotate configuration to achieve distinctions of global, default, specific that are (assuming you're sensible) readable with zero/minimal duplication.
The single, recursive prefer() function (as we're calling it, lacking the need or desire to make an entire project/framework out of it) we've developed so far (see jsFiddle, with inline docs) goes a bit further than this simple example, and (explained in greater detail here) handles deeply-nested configuration objects, as well as preferential ordering and (if you need to stay flat) combination of suffixes.
The function relies on JS ability to reference object properties as strings, dynamically, and tolerate # and & delimiters in property names which are not valid in dot-notation syntax but consequently (help) prevent developers from breaking this technique by accidentally referring to pre-processed/annotated attributes in code (unless they, non-conventionally don't prefer to use dot-notation.)
We have yet to have this break anything for us, nor have we been schooled on any fundamental flaws of this technique, beyond irresponsible/unintended usage or investment/fondness for existing frameworks/techniques that pre-exist. We have also not profiled it for performance (we only tend to run this once per build/session, etc.) so in your own usage, YMMV.
Most configurations transmitted client-side of course would not want to contain sensitive pre-production values, so one could (should!) use the same function to generate a production-only version (with no annotations) in pre-deploy, while still enjoying a SINGLE configuration file upstream in your process.
Further, if you're doing this for i18n, you may not want the entire wad going over the wire, so could process it server-side (cached or live, etc.) or pre-process it in build/deploy by splitting into separate files, but STILL enjoying a single source of truth as early in your workflow as possible.
We have not explored implementing the same function in Java (or C#, PERL, etc.) assuming it's even possible (with some exotic reflection maybe?) but a build environment that includes NodeJS could farm that step out easily.
Well if it suits your needs and you have no problem of storing the connection strings in the source control repository, you could create files like:
appsettings.dev.json
appsettings.qa.json
appsettings.staging.json
And choose the right one in the deployment script and rename it to the actual appsettings.json, which is then read by your app.

Global launch configuration in Eclipse?

This seems like a simple thing, but I can't find an answer in the existing questions:
How do you add a global argument to all your present and existing run or debug configurations? In my case, I need a VM argument, but I see that this could be useful for runline arguments as well.
Basically, every time I create a unit test I need to create a configuration (or run, which creates one), and then manually edit each one with the same VM argument. This seems silly for such a good tool.
This is not true. You can add the VM arguments to the JRE definition. This is exactly what it is for. I use it myself so that assertions are enabled and heap is 1024mb on every run, even future ones.
Ouch: 7-years bug, asking for running configuration template, precisely for that kind or reason.
This thread proposes an interesting workaround, based on duplicating a fake configuration based on string substitution:
You can define variables in Window->Preferences->Run/Debug->String Substitution. For example you can define a projectName_log4j variable with the
correct -Dlog4j.configuration=... value.
In a run configuration you can use ${projectName_log4j} and you don't have to remember the real value.
You can define a project-specific "empty" run configuration.
Set the project and the arguments fields in this configuration but not the main class. If you have to create a new run configuration for this project select this one and use 'Duplicate' from its popup-menu to copy this configuration.
You have to simply set the main class and the program arguments.
Also you can combine both solutions: use a variable and define an "empty"
run configuration which use this variable. The great advantage in this case
is when you begin to use a different log4j config file you have to change
only the variable declaration.
Not ideal, but it may alleviate your process.