I'm building a Dash app using the basic authentication dash_auth. Unfortunately, this requires to hardcode a dictionary of usernames and passwords. This is not a huge problem since the app is only for in-house use.
Now, we would like to deploy this to Heroku by automatically tracking one branch of the GitHub repo because this seems most convenient. The problem is that this would require us to put the hardcoded passwords in the Github repository as well.
This post suggested using environment variables for tokens and client keys but how should I do this for dictionaries of passwords?
I'm open to alternative solutions as well.
Nothing really changes when doing this with a dictionary. You just need to parse the JSON string into a Python data structure.
In your application, instead of hard-coding the dictionary as shown in the documentation:
VALID_USERNAME_PASSWORD_PAIRS = {
'hello': 'world'
}
pull it in from the environment, e.g. something like this:
import json
import os
VALID_USERNAME_PASSWORD_PAIRS = json.loads(os.getenv("VALID_USERNAME_PASSWORD_PAIRS"))
Then set your usernames as Heroku config vars:
heroku config:set VALID_USERNAME_PASSWORD_PAIRS='{"hello": "world"}'
The single quotes here should avoid most issues with special characters being interpreted by your shell.
For local development you can set a VALID_USERNAME_PASSWORD_PAIRS environment variable, e.g. via a .env file if you are using tooling that understands that.
Another option for local development would be to hard-code just a default value into your script by adding a default argument:
VALID_USERNAME_PASSWORD_PAIRS = json.loads(
os.getenv("VALID_USERNAME_PASSWORD_PAIRS", default='{"local": "default"}')
)
Note that we give default a string here, not a dict, since we're passing the result into json.loads().
Be careful with this last option since you could accidentally publish the code without setting the environment variable, in which case the local default credentials would work.
Spring cloud config server supports reading property files with name ${spring.application.name}.properties. However I have 2 properties files in my application.
a.properties
b.properties
Can I get the config server to read both these properties files?
Rename your properties files in git or file system where your config server is looking at.
a.properties -> <your_application_name>.properties
a.properties -> <your_application_name>-<profile-name>.properties
For example, if your application name is test and you are running your application on dev profile, below two properties will be used together.
test.properties
test-dev.properties
Also you can specify additional profiles in bootstrap.properties of your config client to retrieve more properties files like below. For example,
spring:
profiles: dev
cloud:
config:
uri: http://yourconfigserver.com:8888
profile: dev,dev-db,dev-mq
If you specify like above, below all files will be used together.
test.properties
test-dev.properties
test-dev-db.prpoerties
test-dev-mq.properties
Note that the provided answer assumes your property files address different execution profiles. If they dont, i.e., your properties are split into different files for some other reason, e.g., maintenance purposes, divided by business/functional domain, or any other reason that suits your needs, then, by defining a profile for each such file, you are just "abusing" the profile feature, for achieving your goal (multiple property files per app).
You could then ask "OK, so what is the problem with that?". The problem is that you restrain yourself from various possibilities that you would otherwise have. If you actually want to customize your application configuration by profile you will have to create pseudo, sub, profiles for that since the file name is already a profile. Example:
Your application configuration could be customized by different profiles, which you use inside your springboot application (e.g. in #Profile() annotation), let them be dev, uat, prod. You can boot your application setting different profiles as active, e.g. 'dev' vs 'uat', and get the group of properties that you desire. For your a.properties b.properties and c.properties file, if different file names were supported, you would have a-dev.properties b-dev.properties and c-dev.properties files vs a-uat.properties b-uat.properties and c-uat.properties files for 'dev' and 'uat' profile.
Nevertheless, with the provided solution, you already have defined 3 profiles for each file: appname-a.properties appname-b.properties, and appname-c.properties: a, b, and c. Now imagine you have to create a different profile for each... profile(! it already shows something goes wrong here)! you would end up with a lot of profile permutations (which would get worse as files increase): The files would be appname-a-dev.properties, appname-b-dev.properties, app-c-dev.properties vs appname-a-uat.properties, appname-b-uat.properties, app-c-uat.properties, but the profiles would have been increased from ['dev', ' uat'] to ['a-dev', 'b-dev', 'c-dev', 'a-uat', 'b-uat', 'c-uat'] !!!
Even worse, how are you going to cope with all these profiles inside your code and more specifically your #Profile() annotations? Will you clutter the code space with "artificial" profiles just because you want to add one or two more different property files? It should have been sufficient to define your dev or uat profiles, where applicable, and define somewhere else the applicable property file names (which could then be further supported by profile, without any other configuration action), just as it happens in the externalized properties configuration for individual springboot apps
For argument completeness, I will just add here that if you want to switch to .yml property files one day, with the provided profile-based naming solution, you also loose the ability to define different "yaml document sections per profile" inside the same .yml file (Yes, in .yml you can have one property file yet define multiple logical yml documents inside, which its usually done for customizing the properties for different profiles, while having all related properties in one place). You loose the ability because you have already used the profile in the file name (appname-profile.yml)
I have issued a pull request with a minor fix for spring-cloud-config-server 1.4.x, which allows defining additionally supported file names (appart from "application[-profile]" and "{appname}[-profile]", that are currently supported) by providing a spring.cloud.congif.server.searchNames environment property - analogous to spring.config.name for springboot apps. I hope it gets reviewed and accepted.
I came across the same requirement lately with a little more constraint that I am not allowed to play around the environment profiles. So I wasn't allowed to do as the accepted answer. I'm sharing how I did it as an alternative to those who might have same case as me.
In my application, I have properties such as:
appxyz-data-soures.properties
appxyz-data-soures-staging.properties
appxyz-data-soures-production.properties
appxyz-interfaces.properties
appxyz-interfaces-staging.properties
appxyz-interfaces-production.properties
appxyz-feature.properties
appxyz-feature-staging.properties
appxyz-feature-production.properties
application.properties // for my use, contains local properties only
bootstrap.properties // for my use, contains management properties only
In my application, I have these particular properties set that allow me to achieve what I needed. But note I have the rest of needed config as well (enable cloud config, actuator refresh, eureka service discovery and so on) - just highlighting these for emphasis:
spring.application.name=appxyz
spring.cloud.config.name=appxyz-data-soures,appxyz-interfaces,appxyz-feature
You can observe that I didn't want to play around my application name but instead I used it as prefix for my config property files.
In my configuration server I configured in application.yml to capture pattern: 'appxyz-*':
spring:
cloud:
config:
server:
git:
uri: <git repo default>
repos:
appxyz:
pattern: 'appxyz-*'
uri: <another git repo if you have 1 repo per app>
private-key: ${git.appxyz.pk}
strict-host-key-checking: false
ignore-local-ssh-settings: true
private-key: ${git.default.pk}
In my Git repository I have the following. No application.properties and bootstrap because I didn't want those to be published and overridden/refreshed externally but you can do if you want.
appxyz-data-soures.properties
appxyz-data-soures-staging.properties
appxyz-data-soures-production.properties
appxyz-interfaces.properties
appxyz-interfaces-staging.properties
appxyz-interfaces-production.properties
appxyz-feature.properties
appxyz-feature-staging.properties
appxyz-feature-production.properties
It will be the pattern matching pattern: 'appxyz-*' that will capture and return the matching files from my git repository. The profile will also apply and fetch the correct property file accordingly. The prioritization of value is also preserved.
Furthermore, if you wish to add more file in your application (say appxyz-circuit-breaker.properties), we only need to do:
Add the name pattern in the spring.cloud.config.name=...,appxyz-circuit-breaker
The add the copies of the file locally and also externally (in the git repo.
No need to add/modify more or restart your configuration server later on. For new application, it's like a one time registration thing to add an entry under the repos of application.yml.
Hope it helps in one way or another!
In your application bootstrap.properties, you have to specify like below:
spring.application.name=a,b
I want to run spring xd with Oracle(11g) which i already have in my environment. Currently my first concern is the jobs UI (my database has existing data of job executions that were performed by spring-batch and i simply want to display the details of those executions).
i'm using spring-xd-1.0.0.M5. I followed the instructions in the reference guide and i changed application.yml to have the following:
spring:
datasource:
url: jdbc:oracle:oci:MY_USERNAME/MYPWD#//orarmydomain.com:1521/myservice
username: MY_USERNAME
password: MYPWD
driverClassName: oracle.jdbc.OracleDriver
profiles:
active: default,oracle
i modified also batch-jdbc.properties to have the database configuration similar to the above.
Yet, when i start xd-singlnode.bat (or either xd-admin.bat) it seems like it ignores my oracle configuration and still uses the default hsqldb.
what am i doing wrong?
Thanks
The likely reason is that we did not upgrade the windows .bat scripts to take advantage of the property overriding via xd-config.yml. If you go into the unix script for xd-singlenode you will see that when java is invoked there there is an option
-Dspring.config.location=$XD_CONFIG
you can for now hardcode your location of that file, use file: as the prefix.
Also, The UI right now is very primitive, you will not be able to see many details about the job execution. There are however many job related commands you can execute in the shell and there is only one gap regarding step execution information as compared to what is available via spring-batch-admin.
The issue to watch for this is https://jira.springsource.org/browse/XD-1209 and it is schedule for the next milestone release.
Let me know how it goes, thanks!
Cheers,
Mark
I am using Confluence 4.2.5 (build 3284) with CAS SSO connected to my LDAP server and would like to be able to call synchroniseUserDirectories() from the LDAP server when a user changes their password so that the change is instantaneous.
The way it works now is that users have to wait for the Confluence to run it's periodic LDAP synchronization which can be disconcerting for them.
I have tried using the XML-RPC interface to call changeUserPassword() (as an administrator) but it doesn't work. The operation raises an exception "Error changing password for user ...". I presume that that is because the user is defined in the LDAP but I can't tell for sure because the exception message wasn't clear about the cause.
Here is example code that I would like to be able to use. It doesn't work.
#!/usr/bin/env python
import xmlrpclib
url = 'https://docs.example.com'
admin_user = 'frobisher'
admin_pass = 'supersecretstuff'
username = 'bigbob'
new_password = 'bigbobsbigsecret'
server = xmlrpclib.ServerProxy(url + '/rpc/xmlrpc')
token = server.confluence2.login(admin_user, admin_pass)
# CITATION: https://developer.atlassian.com/display/CONFDEV/Remote+Confluence+Methods
# this doesn't exist but would be my preferred approach.
# It raises a NoSuchMethodException exception.
server.confluence2.synchroniseUserDirectories(token)
# this throws a general exception, because of the LDAP? The message
# wasn't clear about the source of the problem.
#server.confluence2.changeUserPassword(token,
# username,
# password)
server.confluence2.logout(token)
Is there any way to do this using SOAP or REST? I was concerned about REST because it sounds like it is still a prototype.
If none of those approaches will work, can it be done with a simple plugin considering that this must be a push operation from the LDAP server to the Confluence server? I have no experience writing plugins but I do some java work occasionally.
Any hints would be greatly appreciated.
The short answer is "no". The ability to synchronise remote user directories is not exposed as a remote operation in Confluence.
The long answer is "yes", you can write a plugin to do this. If you're already familiar with java, then perhaps the best answer is to just show you some source code I've written that performs a similar function: https://bitbucket.org/jaysee00/confluence-user-sync-api This plugin gives you SOAP, XML-RPC and JSON-RPC methods to force an individual user account to be synced in to Confluence from a remote directory.
That might suit your purposes as-is, but I imagine it would be possible to edit the source of this plugin and change it to synchronise an entire directory, too.
I'm working on a CherryPy application based on what I found on that BitBucket repository.
As in this example, there is two config files, server.cfg (aka "global") and app.cfg.
Both config files are loaded in the serve.py file :
# Update the global settings for the HTTP server and engine
cherrypy.config.update(os.path.join(self.conf_path, "server.cfg"))
# ...
# Our application
from webapp.app import Twiseless
webapp = Twiseless()
# Let's mount the application so that CherryPy can serve it
app = cherrypy.tree.mount(webapp, '/', os.path.join(self.conf_path, "app.cfg"))
Now, I'd like to add the Database configuration.
My first thought was to add it in the server.cfg (is this the best place? or should it be located in app.cfg ?).
But if I add the Database configuration in the server.cfg, I don't know how to access it.
Using :
cherrypy.request.app.config['Database']
Works only if the [Database] parameter is in the app.cfg.
I tried to print cherrypy.request.app.config, and it shows me only the values defined in app.cfg, nothing in server.cfg.
So I have two related question :
Is it best to put the database connection in the server.cfg or app.cfg file
How to access server.cfg configuration (aka global) in my code
Thanks for your help! :)
Put it in the app config. A good question to help you decide where to put such things is, "if I mounted an unrelated blog app at /blogs on the same server, would I want it to share that config?" If so, put it in server config. If not, put it in app config.
Note also that the global config isn't sectioned, so you can't stick a [Database] section in there anyway. Only the app config allows sections. If you wanted to stick database settings in the global config anyway, you'd have to consider config entry names like "database_port" instead. You would then access it directly by that name: cherrypy.config.get("database_port").