I understand that the config server needs to be up to bootstrap any of its clients.
Is there any way we can embed both Spring config Server and client in the same application so that each application could protect its sensitive information easily? If it is possible, I will use server for reading properties from the git and client for the decryption.
This flow works with me:
Step 1: Use only Config Server dependency:
pom.xml
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config-server</artifactId>
</dependency>
Step 2: Enable Config Server via#EnableConfigServer annotation.
Step 3: In bootstrap.yml file (not application.yml), we will configure:
spring:
cloud:
config:
server:
# This flag indicates that the server should configure itself from its own remote repository
bootstrap: true
git:
uri: https://github.com/your-git-account/your-config-repository
username: user
password: secret
searchPaths: foo,bar*
timeout: 10
# prefix string to avoid conflicting with context/server path of application
prefix: config
Step 4: Remove all configurations related to Config Server in application.yml file.
If I understand you correctly, all that you'd need to do is add the #EnableConfigServer annotation and set spring.cloud.config.server.bootstrap=true. For detail see Embedding the Config Server and spring-cloud-config issue 100.
Related
Good Afternoon. I'm trying to use Swagger in Quarkus and locally it works great for me, however when I upload it to the production environment I'm using Ingress-Nginx as a reverse proxy in a Kubernetes cluster and I'm running into a problem, I don't allows you to view the Swagger interface:
Postman Local:
Swaager Local:
Postman Kubernetes Environment with Ingress-Nginx:
Swaager-Ui in Kubernetes Environment with Ingress-Nginx:
My application.properties:
quarkus.datasource.db-kind=oracle
quarkus.datasource.jdbc.driver=oracle.jdbc.driver.OracleDriver
#quarkus.datasource.jdbc.driver=io.opentracing.contrib.jdbc.TracingDriver
quarkus.datasource.jdbc.url=jdbc:oracle:thin:#xxxxxxxxxxxx:1522/IVR
quarkus.datasource.username=${USERNAME_CONNECTION_BD:xxxxxxxx}
quarkus.datasource.password=${PASSWORD_CONNECTION_BD:xxxxxxxx.}
quarkus.http.port=${PORT:8082}
quarkus.http.ssl-port=${PORT-SSl:8083}
# Send output to a trace.log file under the /tmp directory
quarkus.log.file.path=/tmp/trace.log
quarkus.log.console.format=%d{HH:mm:ss} %-5p [%c{2.}] (%t) %s%e%n
# Configure a named handler that logs to console
quarkus.log.handler.console."STRUCTURED_LOGGING".format=%e%n
# Configure a named handler that logs to file
quarkus.log.handler.file."STRUCTURED_LOGGING_FILE".enable=true
quarkus.log.handler.file."STRUCTURED_LOGGING_FILE".format=%e%n
# Configure the category and link the two named handlers to it
quarkus.log.category."io.quarkus.category".level=INFO
quarkus.log.category."io.quarkus.category".handlers=STRUCTURED_LOGGING,STRUCTURED_LOGGING_FILE
quarkus.ssl.native=true
quarkus.http.ssl.certificate.key-store-file=${UBICATION_CERTIFICATE_SSL:srvdevrma1.jks}
quarkus.http.ssl.certificate.key-store-file-type=${TYPE_CERTIFICATE_SSL:JKS}
quarkus.http.ssl.certificate.key-store-password=${PASSWORD_CERTIFICATE_SSL:xxxxxxx}
quarkus.http.ssl.certificate.key-store-key-alias=${ALIAS_CERTIFICATE_SSL:xxxxxxxxx}
quarkus.native.add-all-charsets=true
quarkus.swagger-ui.path=/api/FindPukCodeBS/swagger-ui
quarkus.smallrye-openapi.path=/api/FindPukCodeBS/swagger
mp.openapi.extensions.smallrye.info.title=FindPukCodeBS
%dev.mp.openapi.extensions.smallrye.info.title=FindPukCodeBS
%test.mp.openapi.extensions.smallrye.info.title=FindPukCodeBS
mp.openapi.extensions.smallrye.info.version=1.0.1
mp.openapi.extensions.smallrye.info.description=Servicio que consulta el codigo puk asociado a una ICCID (SIMCARD)
mp.openapi.extensions.smallrye.info.termsOfService=Your terms here
mp.openapi.extensions.smallrye.info.contact.email=xxxxxxxxxxxxxxxxxxxx.com
mp.openapi.extensions.smallrye.info.contact.name=xxxxxxxxxxxxxxxxxx#telefonica.com
mp.openapi.extensions.smallrye.info.contact.url=http://exampleurl.com/contact
mp.openapi.extensions.smallrye.info.license.name=Apache 2.0
mp.openapi.extensions.smallrye.info.license.url=https://www.apache.org/licenses/LICENSE-2.0.html
What can be done in these cases?
The Swagger-UI is included by default only in dev mode.
To enable it on your application, you must set this parameter:
quarkus.swagger-ui.always-include=true
This parameter is build time, so you can't change it on your deploy. You must set it into your application.properties.
Reference
https://quarkus.io/guides/all-config#quarkus-swagger-ui_quarkus-swagger-ui-swagger-ui
I am using spring-cloud-config-server with SVN as repository. When I start the cloud-config-server and client[microservice], configuration value is picked up properly.
After changing a configuration value and SVN commit, I am firing refresh POST call, URL: http://localhost:8080/actuator/refresh [8080 is client port]. The updated value in SVN is not getting refreshed.
It is known that config-server stores the SVN data locally.[In my case, folder location - /tmp/config-repo-5393789580706388886] The strange thing here is that, the committed change in SVN is updated locally, once the 'refresh' REST call is triggered. But it is just that application is not picking it up.
Spring boot version - 2.1.3.
Cloud version - Greenwich Release.
Config-Server Details:
Maven dependency:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config-server</artifactId>
</dependency>
<dependency>
<groupId>org.tmatesoft.svnkit</groupId>
<artifactId>svnkit</artifactId>
</dependency>
application.yml:
spring:
application:
name: spring-cloud-config-server
profiles:
active: subversion
cloud:
config:
server:
svn:
uri: https://svn.*****.com/repos/****/microservices
username: #####
password: #####
default-label: source
server:
port: 8888
config-client[microservice] details:
Maven dependency:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-config</artifactId>
</dependency>
application.yml:
spring:
application:
name: [client-app-name]
cloud:
config:
uri: http://localhost:8888
management:
endpoints:
web:
exposure:
include: "*"
endpoint:
health:
show-details: ALWAYS
security:
enabled: false
server:
port: 8080
While starting the config-client microservice, able to see that it is fetching the values from the expected SVN location via config-server.
c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at : http://localhost:8888
c.c.c.ConfigServicePropertySourceLocator : Located environment: name=[client-app-name], profiles=[default], label=null, version=10718, state=null
b.c.PropertySourceBootstrapConfiguration : Located property source: CompositePropertySource {name='configService', propertySources=[MapPropertySource {name='configClient'}, MapPropertySource {name='https://svn.*****.com/repos/****/microservices/source/[client-app-name].properties'}]}
While invoking the test REST call to get the configuration value from config client, I am getting the value from SVN.
After doing the change in the file , [client-app-name].properties and committing to SVN. While doing the REST call http://localhost:8080/actuator/refresh, getting the following as response, which is as expected.
[]-bash-4.2$ curl -X POST http://localhost:9000/actuator/refresh
["config.client.version","configValue"]-bash-4.2$
At the same time, getting the following message from logs as expected.
c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at : http://localhost:8888
c.c.c.ConfigServicePropertySourceLocator : Located environment: name=[client-app-name], profiles=[default], label=null, version=10718, state=null
b.c.PropertySourceBootstrapConfiguration : Located property source: CompositePropertySource {name='configService', propertySources=[MapPropertySource {name='configClient'}, MapPropertySource {name='https://svn.*****.com/repos/****/microservices/source/[client-app-name].properties'}]}
The REST call to the config-client , to fetch the updated configuration just returning the previous configuration value.
If I restart the client, it is picking up the updated latest configuration. Also, the SVN change in updated locally[In my case, folder location - /tmp/config-repo-5393789580706388886]
I am really not able to find out the mistake I did. Any inputs to resolve this would be very much helpful. Thanks a lot.
Did you set #RefreshScope annotation to your Component?
For example, TestProperties is the #ConfigurationProperites that have configValue.
#Bean
public TestComponent(TestProperties properties) {
return new TestComponent(properties.getConfigValue());
}
If you are using raw configValue like above example, you have to annotate it.
#Bean
public TestComponent(TestProperties properties) {
return new TestComponent(properties);
}
If you are using properties as it is, it would be refreshed.
I'm trying to configure SSL for Kafka Connect REST API (2.11-2.1.0).
The problem
I tried two configurations (worker config):
with listeners.https. prefix
listeners=https://localhost:9000
listeners.https.ssl.keystore.location=/mypath/keystore.jks
listeners.https.ssl.keystore.password=mypassword
listeners.https.ssl.key.password=mypassword
and without listeners.https. prefix
listeners=https://localhost:9000
ssl.keystore.location=/mypath/keystore.jks
ssl.keystore.password=mypassword
ssl.key.password=mypassword
Both configurations starts OK, and show following exception when trying to connect to https://localhost:9000 :
javax.net.ssl.SSLHandshakeException: no cipher suites in common
In log, I see that SslContextFactory was created with any keystore, but with ciphers:
210824 ssl.SslContextFactory:350 DEBUG: Selected Protocols [TLSv1.2, TLSv1.1, TLSv1] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
210824 ssl.SslContextFactory:351 DEBUG: Selected Ciphers [TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, ...]
210824 component.AbstractLifeCycle:177 DEBUG: STARTED #10431ms SslContextFactory#42f8285e[provider=null,keyStore=null,trustStore=null]
What I did
As I know that password from keystore is absolutely correct, I digged into source code, and started to debug.
Finally, I find out that neither plain ssl.* nor prefixed listeners.https.ssl.* configurations are not taken into account, and it turns that there is not possibility to configure SSL for Kafka Connect REST API currently.
Call sequence is:
RestServer.createConnector
SSLUtils.createSslContextFactory
AbstractConfig.valuesWithPrefixAllOrNothing
Last method is the reason of troubles.
If we have listeners.https. properties, they cannot be returned, because they filtered out at line 254 (since WorkerConfig contains no properties with the prefix).
Otherwise, if we have unprefixed ssl. properties, they also not returned, because values field contains only known properties from the same WorkerConfig (values are result of ConfigDef.parse).
Am I missing something, and has anyone successfully configured SSL for kafka connect rest api ?
Try export KAFKA_OPTS=-Djava.security.auth.login.config=/apps/kafka/conf/kafka/kf_jaas.conf where kf_jaas.conf contains ZooKeeper client authentication
I haven't test Connect REST API, but KafkaTemplate send and recieves messages with ssl.
From your configuration i may assume two problems:
you not specified the truststore (for certificate chain check)
you used absolute path, but spring keystore-location interprets as
relative to /webapp
I tried test application from examples:
https://memorynotfound.com/spring-kafka-and-spring-boot-configuration-example/
and
https://gist.github.com/itzg/e3ebfd7aec220bf0522e23a65b1296c8
Tested with springboot 2.0.4.RELEASE, used kafka library
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
</dependency>
and this my application.properties content:
spring.application.name=my-stream-app
spring.kafka.bootstrap-servers=localhost:9093
spring.kafka.ssl.truststore-location=kafka.server.truststore.jks
spring.kafka.ssl.truststore-password=123456
spring.kafka.ssl.keystore-location=kafka.server.keystore.jks
spring.kafka.ssl.keystore-password=123456
spring.kafka.ssl.key-password=123456
spring.kafka.properties.security.protocol=SSL
spring.kafka.consumer.group-id=properties test-consumer-group
app.topic.foo=test
fragment of kafka server configuration:
listeners=SSL://localhost:9093
ssl.truststore.location=/home/legioner/kafka.server.truststore.jks
ssl.truststore.password=123456
ssl.keystore.location=/home/legioner/kafka.server.keystore.jks
ssl.keystore.password=123456
ssl.key.password=123456
I have a ConfigServer, very basic:
#EnableConfigServer
#SpringBootApplication
public class ConfigServerApplication {
public static void main(String[] args) {
SpringApplication.run(ConfigServerApplication.class, args);
}
}
I'm using spring-cloud-config-server:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config-server</artifactId>
</dependency>
And I expect it to work the same when deployed to Pivotal Web Services as when I run it locally.
I deployed my configs to a public server with encrypted keys:
spring:
cloud:
config:
server:
git:
uri: https://mypublic.domain/gitbasedconfig
And in my bootstrap.yml, application.yml I have a property with the key:
encrypt:
key: my.super.secret.symmetric.key
This all works locally:
curl http://localhost:8888/myservice/default
responds with all of my encrypted passwords decrypted properly.
When I deploy the same artifact to PWS with the following manifest.yml:
---
applications:
- name: myservice
memory: 384M
disk: 384M
buildpack: java_buildpack
path: target/myservice.jar
env:
ENCRYPT_KEY: my.super.secret.symmetric.key
If I deploy with or without the env->ENCRYPT_KEY neither work. When I call the service, all of my encrypted keys are returned as
invalid.my.key.name: "<n/a>",
In the PWS logs I can see this:
Fri May 20 2016 13:26:21 GMT-0500 (CDT) [APP] OUT {"timeMillis":1463768781279,"thread":"http-nio-8080-exec-4","level":"WARN","loggerName":"org.springframework.cloud.config.server.encryption.CipherEnvironmentEncryptor","message":"Cannot decrypt key: my.key.name (class java.lang.IllegalArgumentException: Unable to initialize due to invalid secret key)","endOfBatch":false,"loggerFqcn":"org.apache.commons.logging.impl.SLF4JLocationAwareLog","contextMap":[],"source":{"class":"org.springframework.cloud.config.server.encryption.CipherEnvironmentEncryptor","method":"decrypt","file":"CipherEnvironmentEncryptor.java","line":81}}
When I look at the http://myservice.on.pws/env I can see that there are values for encrypt.key in both application.yml, bootstrap.yml and I can also see the environment value. These are all the same value.
Why are my encrypted values not being decrypted properly when I'm providing the symmetric key value in both the properties files and/or the environment? Is there some other property that I need to add to make this work on PWS? The non-encrypted values are working properly within the same configs, so everything is wired properly. It's just the encrypted values that are not working.
I think that Spencergibb and Vinicius Carvalho were both correct.
The Java Cryptopgraphy Extensions can't be distributed with the standard java buildpack.
The Pivotal Support site provided a possible solution which is to fork the javabuildpack and update it to include the proper permissions for JCE. The deploy the application with the custom buildpack. One caveat is that you/I won't get the automatic updates.
https://support.run.pivotal.io/entries/76559625-How-do-I-use-the-JCE-Unlimited-Strength-policy-with-my-Java-app-
I have read the cookbook regarding deploying my symfony2 app to production environment. I find that it works great in dev mode, but the prod mode first wouldn't allow signing in (said bad credentials though I signed in with those very credentials in dev mode), and later after an extra run of clearing and warming up the prod cache, I just get http500 from my prod route.
I had a look in the config files and wonder if this has anything to do with it:
config_dev.php:
imports:
- { resource: config.yml }
framework:
router: { resource: "%kernel.root_dir%/config/routing_dev.yml" }
profiler: { only_exceptions: false }
web_profiler:
toolbar: true
intercept_redirects: false
monolog:
handlers:
main:
type: stream
path: %kernel.logs_dir%/%kernel.environment%.log
level: debug
firephp:
type: firephp
level: info
assetic:
use_controller: true
config_prod:
imports:
- { resource: config.yml }
#doctrine:
# orm:
# metadata_cache_driver: apc
# result_cache_driver: apc
# query_cache_driver: apc
monolog:
handlers:
main:
type: fingers_crossed
action_level: error
handler: nested
nested:
type: stream
path: %kernel.logs_dir%/%kernel.environment%.log
level: debug
I also noticed that there is a routing_dev.php but no routing_prod, the prod encironment works great however on my localhost so... ?
In your production environment when you run the app/console cache:warmup command you need to make sure you run it like this: app/console cache:warmup --env=prod --no-debug Also, remember that the command will warmup the cache as the current user, so all files will be owned by the current user and not the web server user (eg: www-data). That is probably why you get a 500 server error. After you warmup the cache run this: chown -R www-data.www-data app/cache/prod (be sure to replace www-data with your web server user.
Make sure your parameters.ini file has any proper configs in place since its common for this file to not be checked in to whatever code repository you might be using. Or (and I've even done this) its possible to simply forget to put parameters from dev into the prod parmeters.ini file.
You'll also need to look in your app/logs/prod.log to see what happens when you attempt to login.