Problem
Does anyone know how to configure bootstrap.yml to tell Spring Cloud Vault to go to the correct path for k2 v2 and not try other paths first?
Details
I can successfully connect to my Vault, running k2 v2, but Spring Cloud will always try to connect to paths in the vault that don't exist, throwing a 403 on startup.
Status 403 Forbidden [secret/application]: permission denied; nested exception is org.springframework.web.client.HttpClientErrorException$Forbidden: 403 Forbidden
The above path, secret/application, doesn't exist because k2 v2 puts data in the path. For example: secret/data/application.
This isn't a show-stopper because Spring Cloud Vault does check other paths, including the correct one that has the data item in the path, but the fact a meaningless 403 is thrown during startup is like a splinter in my mind.
Ultimately, it does try the correct k2 v2 path
2019-03-18 12:22:46.611 INFO 77685 --- [ restartedMain] b.c.PropertySourceBootstrapConfiguration : Located property source: CompositePropertySource {name='vault', propertySources=[LeaseAwareVaultPropertySource {name='secret/data/my-app'}
My configuration
spring.cloud.vault:
kv:
enabled: true
backend: secret
profile-separator: '/'
default-context: my-app
application-name: my-app
host: localhost
port: 8200
scheme: http
authentication: TOKEN
token: my-crazy-long-token-string
Thanks for your help!
Add the following lines in your bootstrap.yml, this disables the generic backend
spring.cloud.vault:
generic:
enabled: false
for more information https://cloud.spring.io/spring-cloud-vault/reference/html/#vault.config.backends.generic
In addition to the accepted answer it's important to turn off (or just remove) fail-fast option:
spring.cloud.vault:
fail-fast: false
spring.cloud.vault.generic.enabled is deprecated in spring-cloud 3.0.0, but the 403 error is still there. To disable the warning (by telling spring to use the exact context), this is what I used:
spring:
config:
import: vault://
application:
name: my-application
cloud:
vault:
host: localhost
scheme: http
authentication: TOKEN
token: my-crazy-long-token-string
kv:
default-context: my-application
Other configs were set to default (such as port = 8200, backend = secret, etc.)
Related
I'm testing an API Gateway setup on Google Cloud to access specific endpoints on a service deployed on Cloud Run. I'm following the steps shown here. We need to authenticate using an API Key, so the API Key specific configuration that went into the API Gateway config was picked from this documentation.
The API Gateway config is as shown below:
# api_gateway_config.yaml
swagger: '2.0'
info:
title: myappapi
description: API with Cloudrun Backend
version: 1.0.0
schemes:
- https
produces:
- application/json
paths:
/:
get:
summary: Greet a User from service
operationId: hello
x-google-backend:
address: https://myappapi-asldfjoiewjfv-uc.a.run.app/
security:
- api_key: []
responses:
'200':
description: A successful response
schema:
type: string
/reports/results/{id}:
get:
summary: Get Report Results for specified report id
operationId: GetReportResults
x-google-backend:
address: https://myappapi-asldfjoiewjfv-uc.a.run.app/v1/reports/results/{id}
parameters:
- in: path
name: id
required: true
type: integer
security:
- api_key: []
responses:
'200':
description: A successful response
schema:
type: string
securityDefinitions:
# This section configures basic authentication with an API Key.
api_key:
type: "apiKey"
name: "key"
in: "query"
For a sample call to the /reports/results endpoint as http://myappapi/reports/results/1,
the expectation is for calls to get converted to https://myappapi-asldfjoiewjfv-uc.a.run.app/v1/reports/results/1?key=MyAPIKeyHere. But instead they turn out as https://myappapi-asldfjoiewjfv-uc.a.run.app/v1/reports/results?key=MyAPIKeyHere&id=1
Is there a way to get the API calls go as https://myappapi-asldfjoiewjfv-uc.a.run.app/v1/reports/results/1?key=MyAPIKeyHere ?
Thanks in Advance!
As mentioned in this documentation
Set path_translation as part of setting x-google-backend:
x-google-backend:
address: https://GCP_REGION-PROJECT_ID.cloudfunctions.net/hello
path_translation: [ APPEND_PATH_TO_ADDRESS | CONSTANT_ADDRESS ]
The default value of path_translation depends on where you set x-google->backend in your OpenAPI spec:
When x-google-backend is used at the top level of the OpenAPI specification, path_translation defaults to APPEND_PATH_TO_ADDRESS.
When x-google-backend is used at the operation level of the OpenAPI specification, path_translation defaults to CONSTANT_ADDRESS.
For more details on path translation, please see the Understanding path translation section. You can also check this stackoverflow thread.
I'm currently replacing an api gateway using Netflix Zuul with spring cloud gateway. The setup uses discovery client (Eureka) for most of the routes, but we also have a solr instance running which requires manually defined routes (as solr doesn't support eureka)
Using a static route to solr running on localhost works fine using the following config:
routes:
- id: solr
predicates:
- Path=/solr/**
uri: http://localhost:10983
filters:
- RewriteLocationResponseHeader=AS_IN_REQUEST, Location,
However, I would like to use a load-balanced uri for this route as we have multiple solr instances. Looking at the documentation I've found that the way to implement this is to define a Bean returning a ServiceInstanceListSupplier. I've imlemented the following function:
#Bean
ServiceInstanceListSupplier serviceInstanceListSupplier() {
List<String> servers = Arrays.asList(microserviceGatewayConfig.getServers().split(","));
return new SolrServiceInstanceListSupplier("solrhosts", servers);
}
However, this seems to override the ServiceInstances defined from Eureka, meaning only the manual services are used...
Do anyone know if it is possble to combine manually defined serviceinstances with those generated from eureka?
The approach with creating a Bean returning a ServiceInstanceListSupplier doesn't seem to work in any way... However, I've found a way to achieve the same in application.yml, by adding the following config:
spring:
cloud:
discovery:
client:
simple:
instances:
solr-cluster:
- instanceId: cluster1
serviceId: solr-cluster
host: soa03i-t.usrv.ubergenkom.no
port: 10983
- instanceId: cluster2
serviceId: solr-cluster
host: soa04i-t.usrv.ubergenkom.no
port: 10983
This can be combined with autogenerated routes from service discovery (e.g. Eureka)
DedupeResponseHeader is not working for me in Spring Cloud Greenwich.SR3, I have added CORS configuration in application.yml, and downstream application is also sending Access-Control-Allow-Origin in response header, which in ending up with:
The 'Access-Control-Allow-Origin' header contains multiple values 'http://localhost:4200, http://localhost:4200', but only one is allowed.
I have used DedupeResponseHeader but that is not working for me still seeing same error in browser console. Following is the config for CORS and DedupeResponseHeader:
spring:
cloud:
gateway:
default-filters:
- DedupeResponseHeader=Access-Control-Allow-Origin, RETAIN_UNIQUE
globalcors:
add-to-simple-url-handler-mapping: true
corsConfigurations:
'[/**]':
allowedOrigins: "http://localhost:4200"
allowedMethods: "*"
allowedHeaders: "*"
Tried in filters also, but also didn't work
spring:
cloud:
gateway:
routes:
- id: dedupe_response_header_route
uri: http://localhost:4200
predicates:
- Method=OPTIONS
- Method=GET
filters:
- DedupeResponseHeader=Access-Control-Allow-Origin
Couldn't figure out the reason why its not working, double checked the spring cloud version. I appreciate, if someone could help to understand why DedupeResponseHeader not working.
You can use the latest version of the spring cloud i.e. 2020.0.2 --- it is working perfectly there.
I have spring config server and vault as backend. i created a token in vault with an acl policy . when i use the token in spring.cloud.config.token it doesnt respect the acl
My sping config client has this boot strap properties
spring:
application:
name: app1
cloud:
config:
uri: https://config-server-ur:port
token: token-associated-to-acl-policy
i created an acl policy by name "app1" which allows only the "app1" to be read by the token in vault.
path "secret/app1" {
capabilities = ["read", "list"]
}
./vault token create -display-name="app1" -policy="app1"
i used the token generated in my client and it doesnt work.
when i changed the acl policy to below, it works
path "secret/*" {
capabilities = ["read", "list"]
}
However, when i access the vault directly with X-Vault-token it works perfectly as expected
I found the solution, Set spring.cloud.config.server.vault.defaultKey to empty, like this in config-server bootstrap.yml
spring.profiles.active=git, vault
spring.cloud.config.server.git.uri=properties-git-repo-url
spring.cloud.config.server.git.username=user
spring.cloud.config.server.git.password=password
spring.cloud.config.server.git.searchPaths=/{application}/xyz
spring.cloud.config.server.git.force-pull=true
spring.cloud.config.server.git.timeout=10
spring.cloud.config.server.git.order=2
spring.cloud.config.server.vault.host=vault-hostname
spring.cloud.config.server.vault.port=8200
spring.cloud.config.server.vault.scheme=https
spring.cloud.config.server.vault.backend=secret
spring.cloud.config.server.vault.defaultKey=
spring.cloud.config.server.vault.profileSeparator=/
spring.cloud.config.server.vault.skipSslValidation=true
spring.cloud.config.server.vault.order=1
spring.cloud.config.server.vault.kvVersion=1
by default spring.cloud.config.server.vault.defaultKey= is set to "application".
I setup my services to use the spring cloud eureka based config server.
version info: spring cloud 1.0.1.RELEASE
When I set it up as a fixed endpoint, I can see that it gets the right configuration file and that I can access actuator endpoints like health, info etc. so a .../manage/info returns the correct information.
However when I set it up to use discovery, the same actuator endpoints timeout on trying got access them.
In each case the configuration file is retrieved and downloaded (included log file).
Is there an issue with how I setup config server and bookmark service (the service which uses the config server)?
My configuration server setting is as follows:
server:
port: 8888
contextPath: /configurationservice
eureka:
client:
registerWithEureka: true
fetchRegistry: true
serviceUrl:
defaultZone: http://localhost:8761/eureka/
instance:
leaseRenewalIntervalInSeconds: 10
statusPageUrlPath: /configurationservice/info
homePageUrlPath: /configurationservice/
healthCheckUrlPath: /configurationservice/health
preferIpAddress: true
spring:
cloud:
config:
server:
native:
searchLocations: file:/Users/larrymitchell/libertas/configserver/configfiles
The service bootstrap.yml settings are:
spring:
profiles:
default: development
active: development
application:
name: bookmarkservice
cloud:
config:
enabled: true # note this needs to be turned on if you wnat the config server to work
# uri: http://localhost:8888/configurationservice
label: 1.0.0
discovery:
enabled: true
serviceId: configurationservice
The application.yml settings are:
# general spring settings
spring:
application:
name: bookmarkservice
profiles:
default: development
active: development
# name of the service
service:
name: bookmarkservice
# embedded web server settings
# some of these are specific to tomcat
server:
port: 9001
# the context path is the part after http:/localhost:8080
contextPath: /bookmarkservice
tomcat:
basedir: target/tomcat
uri-encoding: UTF-8
management:
context-path: /manage
security:
enabled: false
eureka:
client:
registerWithEureka: true
fetchRegistry: true
serviceUrl:
defaultZone: http://localhost:8761/eureka/
instance:
statusPageUrlPath: /bookmarkservice/manage/info
homePageUrlPath: /bookmarkservice/manage
healthCheckUrlPath: /bookmarkservice/manage/health
preferIpAddress: true
The startup log for bookmark service is as follows:
2015-06-24 17:52:49.806 DEBUG 11234 --- [ main] o.s.web.client.RestTemplate : Created GET request for "http://10.132.1.56:8888/configurationservice/bookmarkservice/development/1.0.0"
2015-06-24 17:52:49.890 DEBUG 11234 --- [ main] o.s.web.client.RestTemplate : Setting request Accept header to [application/json, application/*+json]
2015-06-24 17:52:50.439 DEBUG 11234 --- [ main] o.s.web.client.RestTemplate : GET request for "http://10.132.1.56:8888/configurationservice/bookmarkservice/development/1.0.0" resulted in 200 (OK)
2015-06-24 17:52:50.441 DEBUG 11234 --- [ main] o.s.web.client.RestTemplate : Reading [class org.springframework.cloud.config.environment.Environment] as "application/json;charset=UTF-8" using [org.springframework.http.converter.json.MappingJackson2HttpMessageConverter#2b07e607]
2015-06-24 17:52:50.466 INFO 11234 --- [ main] b.c.PropertySourceBootstrapConfiguration : Located property source: CompositePropertySource [name='configService', propertySources=[MapPropertySource [name='file:/Users/larrymitchell/libertas/configserver/configfiles/1.0.0/bookmarkservice-development.yml']]]
2015-06-24 17:52:50.503 INFO 11234 --- [ main] ationConfigEmbeddedWebApplicationContext : Refreshing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext#5fa23965: startup date [Wed Jun 24 17:52:50 EDT 2015]; parent: org.springframework.context.annotation.AnnotationConfigApplicationContext#5cced717
2015-06-24 17:52:51.723 WARN 11234 --- [ main] .i.s.PathMatchingResourcePatternResolver : Skipping [/var/folders/kq/ykvl3t4n3l71p7s9ymywb4ym0000gn/T/spring-boot-libs/06f98804e83cf4a94380b46591b976b1d17c36b8-eureka-client-1.1.147.jar] because it does not denote a directory
2015-06-24 17:52:53.662 INFO 11234 --- [ main] o.s.b.f.config.PropertiesFactoryBean : Loading properties file from URL [jar:file:/Users/larrymitchell/libertas/vipaas/applicationservices/bookmarkservice/target/bookmarkservice.jar!/lib/spring-integration-core-4.1.2.RELEASE.jar!/META-INF/spring.integration.default.properties]
Ok, after talking it over with another coworker I figured out what the actual issue is.
Part of the confusion is that I am using the spring cloud (https://github.com/VanRoy/spring-cloud-dashboard) which is a great front end by the way. So when the service starts we see it for to discovery and retrieve the correct configuration file and load it. After I go to the spring cloud console and see a setting of UP which means that it is discovered and registered though discovery. There is a second status indicator which is when the spring cloud dashboard takes the registered endpoint and get health. In my issue the endpoint was showing up as UNKNOWN.
If I then use the endpoint that shows up in the console and try the info actuator endpoint then the request times out. This was the nature of my problem
Ok, so what was the issue?
Basically since I defined the in the application.yml and since when the service registers in bootstrap it does not know the port yet and then it picks a default of 8080 (my assumption since that is what it does). The server port is set in application.yml to 9001 but discovery sees a registration of 8080 so the spring cloud console cannot access the localhost:8080/bookmarkservice/manage/health since there is no service at that endpoint (which is at 9001 actually). Other services also cannot find the service.
By moving the server.port to bootstrap.yml then the correct endpoint of rate service is registered and the service properly accessible.