How do I retrieve multiple AWS Security Secrets? - spring-cloud

Using spring-cloud-starter-aws-secrets-manager-config to retrieve AWS secrets at application start up.
After defining multiple secrets in AWS secrets manager I cannot see how I can define multiple mappings to map those secrets.
bootstrap.yml
aws:
secretsmanager:
prefix: /secret
defaultContext: context-name
profileSeparator: _
failFast: true
name: service-name
enabled: true
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.9.RELEASE</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>demo</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>Finchley.SR4</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-aws-secrets-manager-config</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
secrets defined in AWS as
/secret/context-name/val1 [foo:bar]
/secret/context-name/val2 [wibble:wombat]
The underlying code in AwsSecretsManagerPropertySource seems to look for the actual secret at the following
/secret/context-name
/secret/context-name_
/secret/service-name
/secret_service-name_
only, so never finds the secret at /secret/context-name/X
Is this expected behaviour ? if so, how would I define multiple secrets in bootstrap.yml ?

you can define your custom(not read by spring by default) secret name via
spring:
config:
import: optional:aws-secretsmanager:{secret arn}
or via
spring:
config:
import: optional:aws-secretsmanager:{/secret/shortName}
when spring boot application boots up, it usually read and load secrets in the environment from these secrets
/prefix/application
/prefix/{your-application-name}
in addition with what you defined via spring.config.import
you can read more about it from here

For me below configuration in bootstrap.yml worked
spring:
application:
name: AwsSecretManager
aws:
secretsmanager:
prefix: /secret
defaultContext: service
profileSeparator: _
failFast: true
name: AwsSecretManager
enabled: true
cloud:
aws:
stack:
auto: false
region:
use-default-aws-region-chain: true
credentials:
use-default-aws-credentials-chain: true
Properties defined in AWS as below
Secret name: /secret/AwsSecretManager_dev
Here dev denotes the profile for which the properties are defined.
Note:
The configuration defined in bootstarp.yml is just a sample configuration for only the Secret manager service and does not make use of AWS stack. The properties will vary based on requirement.

Related

Can't import Spring Cloud dependencies for enabling Eureka client

I work on app in which I try to register Eureka client on server but stuck on the first step where I need dependency which make enable #EnableEurekaClient annotation.
I followed cloud.spring.io instructions which did not work:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-eureka</artifactId>
</dependency>
II) I then found some latest stack & tried (as follows) which did not work as well:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
I wonder, as all folder suddenly disappear in case I put above dependency in the pom.xml (I use Intellij)
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-
4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.5.3</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.eurekaproject.learn</groupId>
<artifactId>microservice-app</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>microservice-app</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>11</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-eureka</artifactId>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<excludes>
<exclude>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</project>
application.yml
spring:
application:
name: microservice-app
datasource:
url: jdbc:mysql://localhost:3306/microservice-app
username: root
password: 12345
jpa:
hibernate:
ddl-auto: none
server:
port: 8092
eureka:
client:
service-url:
defaultZone: http://localhost:8081/eureka
info:
app:
name: ${spring.application.name}
You need to add the Spring Cloud BOM to your pom.xml as follows:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>2020.0.3</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
Once this is added, all Spring Cloud dependencies including spring-cloud-starter-netflix-eureka-client will resolve, and you should be able to use annotations such as #EnableDiscoveryClient for activating the Eureka discovery client.

Failed to determine a suitable driver class - Spring Boot- PostgreSQL

I am trying to connect a Postgres DB to my Spring Boot aplication, but I get the following error:
Failed to configure a DataSource: 'url' attribute is not specified and no embedded datasource could be configured.
Reason: Failed to determine a suitable driver class
Action:
Consider the following:
If you want an embedded database (H2, HSQL or Derby), please put it on the classpath.
If you have database settings to be loaded from a particular profile you may need to activate it (no profiles are currently active).
application.yaml
datasource:
driver-class-name: org.postgresql.Driver
url: jdbc:postgresql://localhost:5432/my_db
username: postgres
password: pswrd
# JPA properties
jpa:
hibernate:
ddl-auto: update # When you launch the application for the first time - switch "none" at "create"
show-sql: true
database: my_db
database-platform: org.hibernate.dialect.PostgreSQLDialect
open-in-view: false
generate-ddl: true
Project structure:
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.5.2</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>PostgresApp</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>PostgresApp</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>11</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
You appear to be missing the spring prefix from all of the properties in your application.yaml. It should be the following:
spring:
datasource:
driver-class-name: org.postgresql.Driver
url: jdbc:postgresql://localhost:5432/my_db
username: postgres
password: pswrd
# JPA properties
jpa:
hibernate:
ddl-auto: update # When you launch the application for the first time - switch "none" at "create"
show-sql: true
database: my_db
database-platform: org.hibernate.dialect.PostgreSQLDialect
open-in-view: false
generate-ddl: true

Spring Cloud Config Server OAuth2 authorization

I have a config server and spring boot microservices running with config server requiring basic authentication user/pw. I have a Keycloak server I want to use for authentication and authorization of the microservice to the config server.
I have added the keycloak spring boot adapter to the config server so now the config server requires an OAuth2 token and have verified with Postman that it works as expected.
This is my microservice pom
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.3.7.RELEASE</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>demo-3</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>demo-3</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>11</java.version>
<spring-cloud.version>Hoxton.SR9</spring-cloud.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-config</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-oauth2</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-oauth2-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
this is my bootstrap.yml
spring:
application:
name: appAuthzRestSpringBoot
cloud:
config:
uri: ${SOA_CONFIG_URI}
fail-fast: true
security:
oauth2:
client:
registration:
keycloak:
authorization-grant-type: client_credentials
client-id: app-authz-rest-springboot
client-secret: secret
provider:
keycloak:
token-uri: http://localhost:8080/auth/realms/spring-boot-quickstart/protocol/openid-connect/token
The bootstrap call to the config server does not have an OAuth2 bearer token.
What am I missing?

Camel Spring Boot Actuator Not Working In Container

I'm using this Camel example as a reference and changed the Initializer to extend SpringBootServletInitializer so that I can deploy to JBoss EAP 6.4.
When running this as a SpringBoot application:
The Spring Actuator endpoints like /info work. /health even returns the health of the Camel Context, however Camel Actuator endpoints like /routes return http 404.
When deploying to JBoss:
None of the actuator endpoints work. They all return http 404.
The problem is not unique to the demo, all of our Containerized Camel Spring Boot Applications running in JBoss have this same problem, and to be clear, these applications are entirely functional otherwise.
bootstrap.yml
camel:
springboot:
name: Demo
health:
check:
routes:
enabled: true
indicator:
enabled: true
endpoints:
actuator:
enabled: true
health:
enabled: true
management:
security:
enabled: false
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>war</packaging>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.8.RELEASE</version>
<relativePath/>
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-spring-boot-starter</artifactId>
<version>2.20.0</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-stream-starter</artifactId>
<version>2.20.0</version>
</dependency>
</dependencies>
</project>
Use below in Application.yml file.
camel:
springboot:
health.indicator.enabled:true
management:
security:
enabled:false

Feign client building URL using app name instead of hostname

In a nutshell:
feign client tries to call http://MyApp/endpoint instead of http://10.0.1.24:8080/endpoint
I have an application with instances in AWS registering to an Eureka server and a feign client on the same Eureka server trying to communicate with the application (server is also client of itself if you will).
This is an sample of the /eureka/apps output on the server:
<application>
<name>JSAPI</name>
<instance>
<hostName>10.0.1.24</hostName>
<app>JSAPI</app>
<ipAddr>10.0.1.24</ipAddr>
<status>UP</status>
<overriddenstatus>UNKNOWN</overriddenstatus>
<port enabled="true">8080</port>
<securePort enabled="false">443</securePort>
<countryId>1</countryId>
<dataCenterInfo class="com.netflix.appinfo.AmazonInfo">
<name>Amazon</name>
<metadata>
<public-ipv4>52.91.157.255</public-ipv4>
<accountId>831427253318</accountId>
<local-hostname>10.0.1.24</local-hostname>
<public-hostname>ec2-52-91-157-255.compute-1.amazonaws.com</public-hostname>
<instance-id>i-1f86e68c</instance-id>
<local-ipv4>10.0.1.24</local-ipv4>
<instance-type>m4.large</instance-type>
<vpc-id>vpc-3c5e155b</vpc-id>
<ami-id>ami-ee5737f9</ami-id>
<mac>0e:3c:e0:fa:3f:1c</mac>
<availability-zone>us-east-1a</availability-zone>
</metadata>
</dataCenterInfo>
<leaseInfo>
<renewalIntervalInSecs>30</renewalIntervalInSecs>
<durationInSecs>90</durationInSecs>
<registrationTimestamp>1478146401223</registrationTimestamp>
<lastRenewalTimestamp>1478152383740</lastRenewalTimestamp>
<evictionTimestamp>0</evictionTimestamp>
<serviceUpTimestamp>1478146378932</serviceUpTimestamp>
</leaseInfo>
<metadata class="java.util.Collections$EmptyMap"/>
<homePageUrl>http://10.0.1.24:8080/</homePageUrl>
<statusPageUrl>http://10.0.1.24:8080/info</statusPageUrl>
<healthCheckUrl>http://10.0.1.24:8080/health</healthCheckUrl>
<vipAddress>JSAPI</vipAddress>
<secureVipAddress>JSAPI</secureVipAddress>
<isCoordinatingDiscoveryServer>false</isCoordinatingDiscoveryServer>
<lastUpdatedTimestamp>1478146401223</lastUpdatedTimestamp>
<lastDirtyTimestamp>1478110734306</lastDirtyTimestamp>
<actionType>ADDED</actionType>
</instance>...
The client code is like so:
#FeignClient("JSAPI")
public interface JsapiCustomerClient {
#RequestMapping(method = RequestMethod.POST, value = "/customers")
public void createCustomer(#Valid #RequestBody CustomerConfig customer);
}
I get the following error when trying to execute the method:
Connection refused executing POST http://JSAPI/customers
I expected it to try and call POST http://10.0.1.24:8080/customers
Any clue? This works on my local, fails once in the cloud.
Thanks
EDIT: Adding pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.knetikcloud</groupId>
<artifactId>clustermanager</artifactId>
<version>0.0.1-SNAPSHOT</version>
</parent>
<artifactId>clustermanager-server</artifactId>
<name>clustermanager-server</name>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-eureka-server</artifactId>
</dependency>
<dependency>
<groupId>com.knetikcloud</groupId>
<artifactId>clustermanager-core</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config-server</artifactId>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-oauth2</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-feign</artifactId>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.11.45</version>
</dependency>
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-acm</artifactId>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-mongodb</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<!-- Packages the application to run in Elastic Beanstalk -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptor>assembly/beanstalk.xml</descriptor>
<finalName>${project.name}</finalName>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<description>A Eureka server paired with JSAPI apps specific status endpoints. The server is used to manage instance and customer status within a JSAPI cluster.</description>
</project>
Could you post your eureka-client settings?
I believe you would need to set #FeignClient's name (JSAPI in this case) to a mapping in application.yml (or .properties) and not to the service name.
I recently blogged about Service registration and discovery using Spring Cloud, Eureka, Ribbon, Feign, ... but here is an extract of the accompanying source code:
#FeignClient(name = ActorsClient.ACTORS_SERVIDE_ID)
public interface ActorsClient {
String ACTORS_SERVIDE_ID = "the-demo-registration-api-1";
String ACTORS_ENDPOINT = "/actors";
String ACTOR_BY_ID_ENDPOINT = "/actors/{id}";
...
Demo Service 2’s application.yml
...
eureka:
client:
registerWithEureka: true
fetchRegistry: true
serviceUrl:
defaultZone: http://localhost:8000/eureka/
instance:
hostname: ${hostName}
statusPageUrlPath: ${management.context-path}/info
healthCheckUrlPath: ${management.context-path}/health
preferIpAddress: true
metadataMap:
instanceId: ${spring.application.name}:${server.port}
...
the-demo-registration-api-1:
ribbon:
# Eureka vipAddress of the target service
DeploymentContextBasedVipAddresses: demo-registration-api-1
#listOfServers: localhost:${SERVER.PORT}
NIWSServerListClassName: com.netflix.niws.loadbalancer.DiscoveryEnabledNIWSServerList
# Interval to refresh the server list from the source (ms)
ServerListRefreshInterval: 30000
...
You could see the-demo-registration-api-1 is a key in yml file mapped to a registered service with name: demo-registration-api-1
You can specify eureka.instance.preferIpAddress in your Eureka configuration then the ip instead of the hostname will be used.
If you want to use hostnames you can specify eureka.instance.hostname: ${vcap.application.uris[0]} for example from environment.
More info:
http://cloud.spring.io/spring-cloud-static/Brixton.SR6/#_eureka_metadata_for_instances_and_clients