I would like to route traffic to pods based on headers - with a fallback.
The desired result would be a k8s cluster where multiple versions of the same service could be deployed and routed to using header values.
svcA
svcB
svcC
each of these services (the main branch of git repo) would be deployed either to default namespace or labelled 'main'. any feature branch of each service can also be deployed, either into its own namespace or labelled with the branch name.
Ideally by setting a header X-svcA to a value matching a branch name, we would route any traffic to the in matching namespace or label. If there is no such name space or label, route the traffic to the default (main) pod.
if HEADERX && svcX:label
route->svcX:label
else
route->svcX
The first question - is this (or something like) even possible with istio or linkerd
You can do that using Istio VirtualService
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
...
spec:
hosts:
- reviews
http:
- match:
- headers:
end-user:
exact: jason
route:
- destination:
host: reviews
subset: v2
- route:
- destination:
host: reviews
subset: v1
Read more here.
Yes you can rout the request based on a header with Istion & Linkerd
For istio there is nice article : https://dwdraju.medium.com/simplified-header-based-routing-with-istio-for-http-grpc-traffic-ff9be55f83ca
in istio's virtual service you can update the header like :
http:
- match:
- headers:
x-svc-env:
regex: v2
For linkerd :
Kind = "service-router"
Name = "service"
Routes = [
{
Match {
HTTP {
PathPrefix = "/api/service/com.example.com.PingService"
}
}
Destination {
Service = "pinging"
},
},
{
Match {
HTTP {
PathPrefix = "/api/service/com.example.com.PingService"
Header = [
{
Name = "x-version"
Exact = "2"
},
]
}
}
Destination {
Service = "pinging"
ServiceSubset = "v2"
},
}
Related
I am trying to route to a flower server from an emissary/ambassador proxy with a root path different from /. Under the hood this is just an envoy front proxy.
There is an example of running celery flower behind nginx, but I am unable to replicate the same thing with ambassador.
My ambassador Mapping object is
apiVersion: getambassador.io/v2
generation: 16
host: strand.dev.REDACTED.info
kind: Mapping
metadata_labels:
ambassador_crd: strand-flower.front-proxy-internal
name: strand-flower
namespace: front-proxy-internal
prefix: /flower/
regex_rewrite:
pattern: ^/flower/(.*)$
substitution: /\1
service: http://strand-flower.default.svc.cluster.local:80
Which ends in an envoy configuration of
{
"match": {
"case_sensitive": true,
"headers": [
{
"exact_match": "strand.dev.REDACTED.info",
"name": ":authority"
}
],
"prefix": "/flower/",
"runtime_fraction": {
"default_value": {
"denominator": "HUNDRED",
"numerator": 100
},
"runtime_key": "routing.traffic_shift.cluster_http___strand_flower_default_svc-0"
}
},
"route": {
"cluster": "cluster_http___strand_flower_default_svc-0",
"priority": null,
"regex_rewrite": {
"pattern": {
"google_re2": {
"max_program_size": 200
},
"regex": "^/flower/(.*)$"
},
"substitution": "/\\1" <<< Is this supposed to be escaped?
},
"timeout": "3.000s"
}
}
This results in the static assets not being served
However, it looks like the path is not actually being rewritten because I need to go to
$HOSTNAME/flower/flower/ to get to the root, or to $HOSTNAME/flower/flower/task to get to the task part of the dashboard.
The flower server is started with --url_prefix=flower per the documentation here.
How do I get flower to work behind Ambassador?
nginx example
server {
location /flower/static {
alias /the/path/to/flower/static;
}
location /flower {
rewrite ^/flower/(.*)$ /$1 break;
proxy_pass http://localhost:5555;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
}
}
Answering my own question in-case anyone comes across this.
The problem is that you don't want the rewrite rule listed in the documentation.
You want an ambassador mapping that looks like this
---
apiVersion: getambassador.io/v2
kind: Mapping
metadata:
name: strand-flower
spec:
prefix: /flower/
rewrite: /flower/
service: http://strand-flower.default.svc.cluster.local:80
host: "strand.dev.REDACTED.info"
The docs recommend that you rewrite rewrite /flower/thing/ -> /thing/, but instead you just want to pass the path through and make sure to set --url_prefix=flower when starting your flower server.
I´m having little luck trying to send a PUT request with JSON containing an array of objects to my GRPC Server using REST. Using GRPC however it accepts an array just like expected. This is what I have defined in my proto file:
message UpdateRequest {
repeated Data data = 1;
int32 Id = 2;
}
message UpdateResponse {
}
message Data {
int32 id = 1;
string name = 2;
}
rpc Update(UpdateRequest) returns (UpdateResponse) {
option (google.api.http) = {
put: "/v1/data/{Id}"
body: "*"
};
}
This deploys successfully to GCP Endpoints but according to the GCP enpointsportal the request body is supposed to only contain a single object like:
{
"data": {
}
}
instead of an array of objects like expected:
{
"data": [
{},
{}
]
}
I´ve tried with replacing the "*" in the body with "data"
rpc Update(UpdateRequest) returns (UpdateResponse) {
option (google.api.http) = {
put: "/v1/data/{Id}"
body: "data"
};
}
This also compiles, but fails when trying to deploy to GCP endpoints with the following message:
kind: ERROR
message: "http: body field path 'data' must be a non-repeated message."
Any suggestions as to how I should go about solving this would be greatly appreciated.
Update:
Heres the contents of my .yaml file.
type: google.api.Service
config_version: 3
name: xxx.xxx-xxx.dev
title: xxxx
apis:
- name: x.x
- name: x.y
backend:
rules:
- selector: "*"
address: grpcs://xxx-xxx-app-xxxx-lz.a.run.app
This is a known issue, according to GCP support.
Here is the google issuetracker link: https://issuetracker.google.com/issues/178486575
There seems that this is a bug in GCP endpoints portal. I´m now successfully sending update requests with arrays containing object through CURL and my frontend application, although this does not work through endpoints.
I have Keycloak (10.0.3) server configured inside a Kubernetes Cluster.
The keycloak server has to handle authentification for external user (using an external url) and also handle oauth2 token for Spring microservices communications.
Then web application spring services uses oidc providers :
spring:
security:
oauth2:
client:
provider:
oidc:
issuer-uri: http://keycloak-cluster-http.keycloak-cluster.svc.cluster.local/auth/realms/myrealm
authorization-uri: http://keycloak-cluster-http.keycloak-cluster.svc.cluster.local/auth/realms/myrealm/protocol/openid-connect/auth
jwk-set-uri: http://keycloak-cluster-http.keycloak-cluster.svc.cluster.local/auth/realms/myrealm/protocol/openid-connect/certs
token-uri: http://keycloak-cluster-http.keycloak-cluster.svc.cluster.local/auth/realms/myrealm/protocol/openid-connect/token
user-name-attribute: preferred_username
The external URL of keycloak is https://keycloak.localhost, managed by ingress redirection handled by Traefik v2
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: keycloak-https
namespace: keycloak-cluster
annotations:
traefik.frontend.passHostHeader: "true"
spec:
entryPoints:
- websecure
routes:
- match: Host(`keycloak.localhost`)
kind: Rule
services:
- name: keycloak-cluster-http
port: 80
tls:
options:
name: mytlsoption
namespace: traefik
store:
name: default
I can access Keycloak using https://keycloak.localhost, no problem, it works.
The problem is that when I try to access my web application, it will always redirect to 'http://keycloak-cluster-http.keycloak-cluster.svc.cluster.local/auth/realms/myrealm', which is not resolved outside k8s.
If I change issuer-uri to http://keycloak.localhost then it doesn't work as keycloak.locahost is not resolved inside k8s.
I tried to set the KEYCLOAK_FRONTEND_URL to https://keycloak.localhost/auth, but no change.
Please, does someone has the same kind of settings and managed to make it working ?
Best regards
Managed to fix it using coredns and adding a rewrite rule... :
rewrite name keycloak.localhost keycloak-cluster-http.keycloak-cluster.svc.cluster.local
apiVersion: v1
data:
Corefile: |
.:53 {
errors
health
ready
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
fallthrough in-addr.arpa ip6.arpa
ttl 30
}
rewrite name keycloak.localhost keycloak-cluster-http.keycloak-cluster.svc.cluster.local
prometheus :9153
forward . /etc/resolv.conf
cache 30
loop
reload
loadbalance
}
kind: ConfigMap
metadata:
name: coredns
namespace: kube-system
The authorization_uri needs to be understood by the browser since that URI is processed in the front channel. The rest of the URIs are processed in the back channel.
Because of that, the authorization_uri should use the front channel way of addressing the authorization server:
authorization_uri: https://keycloak.localhost/auth/realms/myrealm/protocol/openid-connect/auth
EDIT Based on Joe Grandja's input below, it appears that it's also necessary to not specify the issuer-uri property. The issuer-uri property is a shortcut for specifying the other URIs, and since you are specifying those, you don't need it anyway.
Here A POC that helped me with the issue.
Similar configuration ,keycloak and spring gateway are in kubernetes
The external user uses keycloak external host with https protocol
https://external-https/auth/realms/myrealm/protocol/openid-connect/auth?...
The ingress break the https and moves it to http + change the host to internal-http
gateway uses internal-http to connect to keycloakon port 8080
In order for the issuer to be the same protocol as the external the configuration uses https in user-info-uri and authorization-uri but the rest are http
make sure that the keycloak pod is open for https connection (8443)
authorization-uri: https://internal-http:8443/auth/realms/myrealm/protocol/openid-connect/auth
user-info-uri: https://internal-http:8443/auth/realms/myrealm/protocol/openid-connect/userinfo
issuer-uri: http://internal-http:8080/auth/realms/myrealm
To fix the host part of the issuer
In the gateway code I updated the following based on https://github.com/spring-projects/spring-security/issues/8882#user-content-oauth2-client
#SneakyThrows
private WebClient webClient() {
SslContext sslContext = SslContextBuilder
.forClient()
.trustManager(InsecureTrustManagerFactory.INSTANCE)
.build();
HttpClient httpClient = HttpClient.create()
.secure(t -> t.sslContext(sslContext))
.wiretap(true)
;
ReactorClientHttpConnector conn = new ReactorClientHttpConnector(httpClient);
return WebClient.builder()
.defaultHeader("HOST", "external-https")
.clientConnector(conn)
.build();
}
#Bean
WebClientReactiveAuthorizationCodeTokenResponseClient webClientReactiveAuthorizationCodeTokenResponseClient() {
final WebClientReactiveAuthorizationCodeTokenResponseClient webClientReactiveAuthorizationCodeTokenResponseClient = new WebClientReactiveAuthorizationCodeTokenResponseClient();
final WebClient webClient = webClient();
webClientReactiveAuthorizationCodeTokenResponseClient.setWebClient(webClient);
return webClientReactiveAuthorizationCodeTokenResponseClient;
}
#Bean
WebClientReactiveClientCredentialsTokenResponseClient webClientReactiveClientCredentialsTokenResponseClient() {
final WebClientReactiveClientCredentialsTokenResponseClient webClientReactiveClientCredentialsTokenResponseClient = new WebClientReactiveClientCredentialsTokenResponseClient();
final WebClient webClient = webClient();
webClientReactiveClientCredentialsTokenResponseClient.setWebClient(webClient);
return webClientReactiveClientCredentialsTokenResponseClient;
}
#Bean
WebClientReactiveRefreshTokenTokenResponseClient webClientReactiveRefreshTokenTokenResponseClient() {
final WebClientReactiveRefreshTokenTokenResponseClient webClientReactiveRefreshTokenTokenResponseClient = new WebClientReactiveRefreshTokenTokenResponseClient();
final WebClient webClient = webClient();
webClientReactiveRefreshTokenTokenResponseClient.setWebClient(webClient);
return webClientReactiveRefreshTokenTokenResponseClient;
}
#Bean
WebClientReactivePasswordTokenResponseClient webClientReactivePasswordTokenResponseClient() {
final var client = new WebClientReactivePasswordTokenResponseClient();
final WebClient webClient = webClient();
client.setWebClient(webClient);
return client;
}
#Bean
DefaultReactiveOAuth2UserService reactiveOAuth2UserService() {
final DefaultReactiveOAuth2UserService userService = new DefaultReactiveOAuth2UserService();
final WebClient webClient = webClient();
userService.setWebClient(webClient);
return userService;
}
Disabled the certificate validation - the connection is only between keycloak and gateway , both are in the kubernetes and otherwise would have used http connection, if not for this issue
The host part tells the keyclock what is the host to use for the issuer
Another issue encountered is that the location return when redirecting to authentication contains the internal url and not the external which the outside world doesn't know of
For that ,update the location that returns from the gateway
#Bean
public SecurityWebFilterChain springSecurityFilterChain(ServerHttpSecurity http)
...
oauth2Login(oAuth2LoginSpec -> oAuth2LoginSpec
...
.addFilterAfter(new LoginLocationFilter("external-https"), SecurityWebFiltersOrder.LAST)
...
public class LoginLocationFilter implements WebFilter {
private final String externalUrl;
public LoginLocationFilter(String externalUrl) {
this.externalUrl = externalUrl;
}
#Override
public Mono<Void> filter(ServerWebExchange exchange, WebFilterChain chain) {
//before commit ,otherwise the headers will be read only
exchange.getResponse().beforeCommit(() -> {
fixLocation(exchange);
return Mono.empty();
});
return chain.filter(exchange);
}
...
I've got a Envoy Filter in which I add a header to every HTTP request. The header's value comes from API.
Let's assume two configurations of the filter. In the configuration below I added a hardcoded version of my header. It was checked in the logs of my target application and it works.
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: lua-filter
spec:
configPatches:
- applyTo: HTTP_FILTER
match:
context: ANY
listener:
portNumber: 7123
filterChain:
filter:
name: "envoy.http_connection_manager"
subFilter:
name: "envoy.router"
patch:
operation: INSERT_BEFORE
value:
name: envoy.lua
typed_config:
"#type": "type.googleapis.com/envoy.config.filter.http.lua.v2.Lua"
inlineCode: |
function envoy_on_request(request_handle)
request_handle:headers():add("authorization", "it works!")
end
This time I want to have the header's value coming from my API. Unfortunately, this setup doesn't work and I have no idea why. I have checked the Lua script on my local machine, and the script itself works but as soon as I provide the script to the filter, no header is added.
typed_config:
"#type": "type.googleapis.com/envoy.config.filter.http.lua.v2.Lua"
inlineCode: |
function envoy_on_request(request_handle)
local http = require('socket.http')
local json = require('json')
local ltn12 = require "ltn12"
local reqbody="my request body"
local respbody = {}
local body, code, headers, status = http.request {
method = "POST",
url = "http://my-address",
source = ltn12.source.string(reqbody),
headers =
{
["Accept"] = "*/*",
["Accept-Encoding"] = "gzip, deflate",
["Accept-Language"] = "en-us",
["Content-Type"] = "application/x-www-form-urlencoded",
["content-length"] = string.len(reqbody)
},
sink = ltn12.sink.table(respbody)
}
respbody = table.concat(respbody)
parsed = json.decode(respbody)
token = parsed["token-value"]
request_handle:headers():add("authorization",token)
end
How to achieve change servciceId based on request params ??
Below is our Zuul Config
zuul:
host:
connect-timeout-millis: 200000
connection-request-timeout-millis: 200000
socket-timeout-millis: 200000
ignored-services: "*"
routes:
route-1:
path: /path1/**
serviceId: ServiceA
route-2:
path: /path2/**
serviceId: ServiceB
Over here we are selecting serviceId based on path1/path2.
if http://localhost:8050/path1/endpointPath?requestParam=ParamValue1 this should call serviceA
if http://localhost:8050/path1/endpointPath?requestParam=ParamValue2 this should call serviceB
was able to achieve this by using route filter and config changes
Config :
routes:
route-1:
path: /**
serviceId: ServiceA
stripPrefix: true
RouteFilter:
Optional<String> parameter = Optional.ofNullable(ctx.getRequest().getParameter("requestparam"));
if (parameter.isPresent()) {
if (parameter.get().equalsIgnoreCase("ValueA")) {
ctx.set("serviceId", "ServiceA");
} else {
ctx.set("serviceId", "ServiceB");
}
}
Is this fine or do we have any simpler way to achieve ?
Here is there any we can limit not to define serviceId in properties file ?