I'm setting up my Quarkus app to run in the cloud, but I couldn't find how to handle encrypted properties in the configuration file.
In my Spring Boot apps I know that I can prefix a property with some tags so it'll be decrypted before usage:
password={cipher}{key:alias}<encrypted-text>
Is there any Quarkus AWS plugin that handles such syntax?
Or any way that I can access the configuration properties before usage so I decrypt them manually?
This is not supported in Quarkus. There are some prototype to support something similar in the future but they are not complete yet. Please follow: https://github.com/quarkusio/quarkus/issues/7442
The recommendation is to Vault: https://quarkus.io/guides/vault
If you want to access configuration before usage, you can implement an interceptor: https://smallrye.io/docs/smallrye-config/main/interceptors/interceptors.html
Related
The answer on many forums is to simply "create the key". However, this also requires further maintenance because then the flow.xml.gz file cannot be easily migrated between deployments. I do not experience this on previous versions (v1.12.0 & 1.13.0).
Any thoughts on mitigating this issue?
Error:
ERROR [main] o.a.nifi.properties.NiFiPropertiesLoader Clustered Configuration Found: Shared Sensitive Properties Key [nifi.sensitive.props.key] required for cluster nodes
ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.lang.IllegalArgumentException: There was an issue decrypting protected properties
Similar question: Docker - Nifi : 1.14.0 - Startup failure - Caused by: org.apache.nifi.properties.SensitivePropertyProtectionException
Create the key.
Don't let NiFi generate a random one, supply it from your deployment code along with all the other settings that go into nifi.properties. If you have the same key, you can copy/migrate the flow.xml.gz and share it within clusters.
This also works with an encrypted key if you provide the decryption hex key in bootstrap.conf when deploying.
The latest NiFi version has support for Hashicorp vaults. That might allow you to obtain the correct keys at runtime and share them among cluster nodes.
If you want to work without a key, you will need to use NiFi 1.13.2 or older. From the admin guide:
Starting with version 1.14.0, NiFi requires a value for 'nifi.sensitive.props.key' in nifi.properties.
The following command can be used to read an existing flow.xml.gz configuration and set a new sensitive properties key in nifi.properties:
$ ./bin/nifi.sh set-sensitive-properties-key [sensitivePropertiesKey]
The minimum required length for a new sensitive properties key is 12 characters.
Ignore this error.
Create a new deployment, either import the flow file via the GUI or copy and paste the XML flow file, then restart the deployment.
In my testing, I have not seen any evidence that the sensitive key property is required.
I'm building a multitenant application and I'm using Keycloak for authentication and authorization.
Foreach each tenant, the idea is to have a dedicated Keycloak realm. Each tenant will have exactly the same roles and clients.
I have tried to export one existing realm, use it as template and import it for new tenant. Problem: I'm facing database constraint violation due to internal id.
Question: Is there an elegant way to achieve this, having a template to create a new realm ?
Be sure that the feature for uploading script is enabled. For a deployment with a docker-compose just add this:
command: -Dkeycloak.profile.feature.upload_scripts=enabled
Export your realm (the one to be used as model)
Remove all line containing "id:" and "_id:"
Search and replace template realm name by the new realm name
In Keycloak UI admin console, Add new realm, provide the file and that is all.
You can use the cleaned exported file as template.
Can't comment due to rep,
but I'd like to add to #Youssouf Maiga's answer,
that you should also modify any fields that contain values under "authenticationFlowBindingOverrides":
Replace any entries that have values assigned under "direct_grant" or "browser"
i.e
"authenticationFlowBindingOverrides": {
"direct_grant": "f5d1wb45e-27eb-4466-937439-9cc8a615ad65e",
"browser": "5b23141a1c-7af8d-410e-a9b451f-0eec12039c72e9"
},
replaced with
"authenticationFlowBindingOverrides": {},
I tried cloning my realm based on this and got an error saying:
"Unable to resolve auth flow binding override for: direct_grant" when importing the modified realm export.
Keycloak version 16.1.1
What you could do is configure everything using the Keycloak Terraform provider. That way you only have to define the configuration once, in code, and then apply it using Terraform. See for the documentation: https://registry.terraform.io/providers/mrparkers/keycloak/latest/docs
An advantage of this is that you can put your code in an SCM tool (e.g. git), so you can track your changes, and go back to a previous version if necessary.
We use Kafka, Kafka connect and Schema-registry in our stack. Version is 2.8.1(Confluent 6.2.1).
We use Kafka connect's configs(key.converter and value.converter) with value: io.confluent.connect.avro.AvroConverter.
It registers a new schema for topics automatically. But there's an issue, AvroConverter doesn't specify subject-level compatibility for a new schema
and the error appears when we are trying to get config for the schema via REST API /config: Subject 'schema-value' does not have subject-level compatibility configured
If we specify the request parameter defaultToGlobal then global compatibility is returned. But it doesn't work for us because we cannot specify it in the request. We are using 3rd party UI: AKHQ.
How can I specify subject-level compatibility when registering a new schema via AvroConverter?
Last I checked, the only properties that can be provided to any of the Avro serializer configs that affect the Registry HTTP client are the url, whether to auto-register, and whether to use the latest schema version.
There's no property (or even method call) that sets either the subject level or global config during schema registration
You're welcome to check out the source code to verify this
But it doesn't work for us because we cannot specify it in the request. We are using 3rd party UI: AKHQ
Doesn't sound like a Connect problem. Create a PR for AKHQ project to fix the request
As of 2021-10-26, I used akhq 0.18.0 jar and confluent-6.2.0, the schema registry in akhq is working fine.
Note: I also used confluent-6.2.1, seeing exactly the same error. So, you may want to switch back to 6.2.0 to give a try.
P.S: using all only for my local dev env, VirtualBox, Ubuntu.
#OneCricketeer is correct.
There is no possibility to specify subject-level compatibility in AvroConverter unfortunately.
I see only two solutions:
Override AvroConverter to add property and functionality to send an additional request to API /config/{subject} after registering the schema.
Contribute to AKHQ to support defaultToGlobal parameter. But in this case, we also need to backport schema-registry RestClient. Github issue
The second solution is more preferable till the user would specify the compatibility level in the settings of the converter. Without this setting in the native AvroConverter, we have to use the custom converter for every client who writes a schema. And it makes a lot of effort.
For me, it looks strange why the client cannot set up the compatibility at the moment of registering the schema and has to use a different request for it.
I want to include a properties file of environment variables to better integrate between environments in AWS deployment of WSO2 Identity Server. I could put all the environment variables in line in the wso2server.sh, but it would be better to inject a properties file that has all the variables I need.
I am trying to include:
-Ddeployment.conf="$CARBON_HOME/repository/conf/etc/dev-env.properties" \
in the wso2server.sh where my dev-env.properties has variables that I want to include in the xml configurations. An example being the usr-mgt.xml connection string:
<Property name="ConnectionURL">${user.mgt.connection.url}</Property>
I could do -Duser.mgt.connection.url="connection-string" \ but I have about 20 properties that I currently want to set this way and would prefer to keep them all in one file instead of in line environment variables. I found this Medium article
describing something like what I am looking for but I'm not sure it's exactly what I want and it was unclear how to implement this.
Do I need to write a Java Util class to read these environment variables from the properties file or is there a simpler way to do this? And if I need a utils class what would that look like?
As far as I know, this feature will support from the WSO2 products which are based on Carbon Kernek 5 and onwards. But at the moment most of the WSO2 products are based on kernel version 4.0. Therefore I think you can't get this done with existing WSO2 products.
I wish to use consul strictly as a config source.
I am using spring-cloud-consul-config to get my config.
I am using git2consul to load files into consul and read them.
As per the spring cloud documentation I have added the following to my build.gradle
compile ("org.springframework.cloud:spring-cloud-starter-consul-config")
and have the following in my application.properties
spring.application.name=test-service
spring.cloud.consul.config.enabled=true
spring.cloud.consul.enabled=true
spring.cloud.consul.config.format=FILES
The problem I am facing is that the expected properties are not being loaded into the ConfigurationProperties beans. On further debugging in the ConsulPropertySourceLocator::locate(Environment environment) method, I see that the this.properties object is still loaded with KEY_VALUE enum.
This led me to ConsulConfigBootstrapConfiguration class, where the ConsulConfigProperties bean is being instantiated using a constructor.
Is this the problem or do I have something wrong in my setup.
If someone has a working setup of git2consul and spring cloud config, please can you point me to it for reference.
These values that you have in application.properties
spring.application.name=test-service
spring.cloud.consul.config.enabled=true
spring.cloud.consul.enabled=true
spring.cloud.consul.config.format=FILES
need to be in bootstrap.properties.