k8s ingress setup - invalid type for io.k8s.api.networking.v1.ServiceBackendPort.number: got "string", expected - kubernetes

I'm trying to setup an ingress in kubernetes for my frontend, the code is as below.
datahub-frontend:
enabled: true
image:
repository: linkedin/datahub-frontend-react
tag: "v0.8.31"
ingress:
enabled: true
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: instance
alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:ap-southeast-2:601628467906:certificate/xxxxxxxxxxxxx
alb.ingress.kubernetes.io/inbound-cidrs: 0.0.0.0/0
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
hosts:
- host: xxxxx.com
redirectPaths:
- path: /*
name: ssl-redirect
port: use-annotation
paths:
- /*
This is giving me the following error.
Error: UPGRADE FAILED: error validating "": error validating data: ValidationError(Ingress.spec.rules[0].http.paths[0].backend.service.port.number): invalid type for io.k8s.api.networking.v1.ServiceBackendPort.number: got "string", expected "integer"

seems like the ingress definition you posted is incorrect. It is missing the required spec and rules according to kubernetes doc.
This contains an example ingress for ssl-redirect
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
namespace: default
name: ingress
annotations:
alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-west-2:xxxx:certificate/xxxxxx
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
alb.ingress.kubernetes.io/ssl-redirect: '443'
spec:
ingressClassName: alb
rules:
- http:
paths:
- path: /users/*
pathType: ImplementationSpecific
backend:
service:
name: user-service
port:
number: 80
- path: /*
pathType: ImplementationSpecific
backend:
service:
name: default-service
port:
number: 80
I see that you are using actions annotation, you need to specify Ingress.spec.rules[0].http.paths[0].backend.service.port.name with use-annotation. Make sure Ingress.spec.rules[0].http.paths[0].backend.service.port.number is not used.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
annotations:
...
...
alb.ingress.kubernetes.io/actions.my-rule: '{"type":"fixed-response","fixedResponseConfig":{"contentType":"text/plain","statusCode":"200","messageBody":"hello world"}}'
spec:
rules:
- http:
paths:
- path: /*
pathType: ImplementationSpecific
backend:
service:
name: my-rule
port:
name: use-annotation

Related

Example Nginx plus ingress for sticky sessions during canary deployment

I’m deploying 2 services to kubernetes pods which simply echo a version number; echo-v1 & echo-v2
Where echo-v2 is considered the canary deployment, I can demonstrate sticky sessions as canary weight is reconfigured from 0 to 100 using canary & canary-weight annotations.
2 ingresses are used:
The first routes to echo-v1 with a session cookie annotation.
The second routes to echo-v2 with canary true,canary weight and session cookie annotations.
The second ingress I can apply without impacting those sessions started on the first ingress and new sessions follow the canary weighting as expected.
However I’ve since learned that those annotations are for nginx community and won’t work with nginx plus.
How can I achieve the same using ingress(es) with nginx plus?
This is the ingress configuration that works for me using Nginx community vs Nginx plus.
Nginx community:
(coffee-v1 service)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/affinity: "cookie"
name: ingress-coffee
spec:
rules:
- http:
paths:
- path: /coffee
pathType: Exact
backend:
service:
name: coffee-v1
port:
number: 80
(coffee-v2 'canary' service)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/affinity: "cookie"
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-weight: "100"
name: ingress-coffee-canary
spec:
rules:
- http:
paths:
- path: /coffee
pathType: Exact
backend:
service:
name: coffee-v2
port:
number: 80
Nginx plus:
(coffee-v1 & coffee-v2 as type 'virtualserver' not 'ingress')
apiVersion: k8s.nginx.org/v1
kind: VirtualServer
metadata:
name: cafe
spec:
host: cloudbees-training.group.net
tls:
secret: cloudbees-trn.aks.group.net-tls
upstreams:
- name: coffee-v1
service: coffee-v1-svc
port: 80
sessionCookie:
enable: true
name: srv_id_v1
path: /coffee
expires: 2h
- name: coffee-v2
service: coffee-v2-svc
port: 80
sessionCookie:
enable: true
name: srv_id_v2
path: /coffee
expires: 2h
routes:
- path: /coffee
matches:
- conditions:
- cookie: srv_id_v1
value: ~*
action:
pass: coffee-v1
- conditions:
- cookie: srv_id_v2
value: ~*
action:
pass: coffee-v2
# 3 options to handle new session below:
#
# 1) All new sessions to v1:
# action:
# pass: coffee-v1
#
# 2) All new sessions to v2:
# action:
# pass: coffee-v2
#
# 3) Split new sessions by weight
# Note: 0,100 / 100,0 weightings causes sessions
# to drop for the 0 weighted service:
# splits:
# - weight: 50
# action:
# pass: coffee-v1
# - weight: 50
# action:
# pass: coffee-v2

Websocket connection fails for microfrontend (using Kubernetes ingress-nginx and port forwarding)

I am building a microfrontend-microservices app where the microfrontend is written in React and the backend utilizes a Kubernetes cluster, in particular an ingress-nginx service. There are many services (auth, company, companies, event bus, permissions ingress and query services) with a few corresponding micro (client) apps that are built using Webpack Module Federation and webpack dev server. I am using skaffold as a dev tool.
When I run skaffold dev --port-forwarding, I am able to load the app in the browser using the host domain of mavata.dev but with errors (which eventually make certain pages which rely on module federation fail to load entirely). I get the following errors in chrome dev tools:
WebSocket connection to 'wss://mavata.dev:9000/ws' failed
WebSocket connection to 'wss://mavata.dev:8081/ws' failed
WebSocket connection to 'wss://mavata.dev:8083/ws' failed
How do I fix the websocket connection?? Below are relevant code snippets..
(Client) Company App Deployment: client-company-depl.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: client-company-depl
spec:
replicas: 1
selector:
matchLabels:
app: client-company
template:
metadata:
labels:
app: client-company
spec:
containers:
- name: client-company
image: bridgetmelvin/client-company
---
apiVersion: v1
kind: Service
metadata:
name: client-company-srv
spec:
selector:
app: client-company
ports:
- name: client-company
protocol: TCP
port: 8083
targetPort: 8083
(a similar deployment for the (client) container app but with a port and targetPort of 9000)
(another similar deployment for the (client) marketing app but with a port and targetPort of 8081)
Ingress Service Config: ingress-srv.yaml:
# RUN: kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.3.1/deploy/static/provider/cloud/deploy.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-srv
annotations:
certmanager.k8s.io/cluster-issuer: core-prod
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: 'true'
nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/secure-backends: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/websocket-services: core-service
nginx.org/websocket-services: core-service
spec:
rules:
- host: mavata.dev # need to update 'hosts' file on local machine (in VS Code) C:\Windows\System32\drivers\etc
http:
paths:
- path: /company/create
pathType: Prefix
backend:
service:
name: company-clusterip-srv
port:
number: 4000
- path: /company
pathType: Prefix
backend:
service:
name: query-srv
port:
number: 4002
- path: /company/?(.*)
pathType: Prefix
backend:
service:
name: company-srv
port:
number: 4000
- path: /companies
pathType: Prefix
backend:
service:
name: companies-srv
port:
number: 4002
- path: /api/users/?(.*)
pathType: Prefix
backend:
service:
name: auth-server-srv
port:
number: 4004
- path: /permissions/?(.*)
pathType: Prefix
backend:
service:
name: permissions-srv
port:
number: 4001
- path: /?(.*)
pathType: Prefix
backend:
service:
name: client-container-srv
port:
number: 9000
(Client) Container webpack.dev.js:
const { merge } = require('webpack-merge');
const ModuleFederationPlugin = require('webpack/lib/container/ModuleFederationPlugin');
const commonConfig = require('./webpack.common');
const packageJson = require('../package.json');
const globals = require('../src/data-variables/global');
const port = globals.port;
const devConfig = {
mode: 'development',
output: {
publicPath: `http://localhost:${port}/` // don't forget the slash at the end
},
devServer: {
host: '0.0.0.0',
port: port,
allowedHosts: ['mavata.dev'],
historyApiFallback: {
index: 'index.html',
},
},
plugins: [
new ModuleFederationPlugin({
name: 'container',
filename: 'remoteEntry.js',
remotes: {
marketingMfe: 'marketingMod#http://localhost:8081/remoteEntry.js',
authMfe: 'authMod#http://localhost:8082/remoteEntry.js',
companyMfe: 'companyMod#http://localhost:8083/remoteEntry.js',
dataMfe: 'dataMod#http://localhost:8084/remoteEntry.js'
},
exposes: {
'./Functions': './src/functions/Functions',
'./Variables': './src/data-variables/Variables'
},
shared: {...packageJson.dependencies, ...packageJson.peerDependencies},
}),
],
};
module.exports = merge(commonConfig, devConfig);
(Client) Company webpack.dev.js:
const { merge } = require('webpack-merge');
const HtmlWebpackPlugin = require('html-webpack-plugin');
const ModuleFederationPlugin = require('webpack/lib/container/ModuleFederationPlugin');
const commonConfig = require('./webpack.common');
const packageJson = require('../package.json');
const path = require('path');
const globals = require('../src/variables/global')
const port = globals.port
const devConfig = {
mode: 'development',
output: {
publicPath: `http://localhost:${port}/`, // don't forget the slash at the end
},
devServer: {
port: port,
historyApiFallback: {
index: 'index.html',
},
},
plugins: [
new ModuleFederationPlugin({
name: 'companyMod',
filename: 'remoteEntry.js',
exposes: {
'./CompanyApp': './src/bootstrap',
},
remotes: {
containerMfe: 'container#http://localhost:9000/remoteEntry.js',
dataMfe: 'dataMod#http://localhost:8084/remoteEntry.js'
},
shared: {...packageJson.dependencies, ...packageJson.peerDependencies},
}),
new HtmlWebpackPlugin({
template: './public/index.html',
}),
],
};
module.exports = merge(commonConfig, devConfig);

error validating data: [ValidationError(Ingress.spec.rules[0].http.paths[0].backend): eks v1.22

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
alb.ingress.kubernetes.io/certificate-arn: "arn:aws:acm:us-west-1:0987654322:certificate/ee21bffb-ba4a-45a6-95a6-00551c1cfa32"
alb.ingress.kubernetes.io/healthcheck-interval-seconds: "15"
alb.ingress.kubernetes.io/healthcheck-port: traffic-port
alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
alb.ingress.kubernetes.io/healthcheck-timeout-seconds: "5"
alb.ingress.kubernetes.io/healthy-threshold-count: "2"
alb.ingress.kubernetes.io/listen-ports: "[{"HTTPS":443}, {"HTTP":80}]"
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-TLS-1-1-2017-01
alb.ingress.kubernetes.io/ssl-redirect: "443"
alb.ingress.kubernetes.io/success-codes: "200"
alb.ingress.kubernetes.io/target-group-attributes: "stickiness.enabled=true,stickiness.lb_cookie.duration_seconds=60"
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/unhealthy-threshold-count: "2"
kubernetes.io/ingress.class: alb
name: alb-alb
namespace: deploy
spec:
rules:
-
http:
paths:
-
backend:
serviceName: service-svc
servicePort: 80
path: /*
Try:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
...
name: alb-alb
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: service-svc
port: # <-- Correction to your spec starts here
number: 80

Kubernetes ingres.yml unknown field "service.port.number" in io.k8s.api.networking.v1

I'm running into issues with my ingress.yml. Getting this error:
error: error validating "ingress.yml": error validating data: [ValidationError(Ingress.spec.rules[0].http.paths[0].backend): unknown field "service.name" in io.k8s.api.networking.v1.IngressBackend, ValidationError(Ingress.spec.rules[0].http.paths[0].backend): unknown field "service.port.number" in io.k8s.api.networking.v1.IngressBackend, ValidationError(Ingress.spec.rules[0].http.paths[1].backend): unknown field "service.name" in io.k8s.api.networking.v1.IngressBackend, ValidationError(Ingress.spec.rules[0].http.paths[1].backend): unknown field "service.port.number" in io.k8s.api.networking.v1.IngressBackend];
Based on this documentation serviceName changed to service.name and servicePort changed to service.port and so on. Here is the ingress file:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: bitwarden
namespace: bitwarden
labels:
app: bitwarden
annotations:
# the name of the nginx-ingress-controller class
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "Request-Id: $req_id";
nginx.ingress.kubernetes.io/connection-proxy-header: "keep-alive"
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/limit-connections: "25"
nginx.ingress.kubernetes.io/limit-rps: "15"
nginx.ingress.kubernetes.io/proxy-body-size: 1024m
nginx.ingress.kubernetes.io/proxy-connect-timeout: "10"
nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
tls:
- hosts:
- kryskanbit.ddns.net
secretName: bitwarden-tls
rules:
- host: kryskanbit.ddns.net
http:
paths:
- path: /
pathType: Prefix
backend:
service.name: bitwarden
service.port.number: 80
- path: /notifications/hub
pathType: Exact
backend:
service.name: bitwarden
service.port.number: 3012
Kubernetes has a special treatment for YAML format, so basically, you just need to extend the properties individually the service.name and service.port.number:
...
paths:
- path: /
pathType: Prefix
backend:
service:
name: bitwarden
port:
number: 80
- path: /notifications/hub
pathType: Exact
backend:
service:
name: bitwarden
port:
number: 3012
It should be as mentioned below:
backend:
serviceName: bitwarden
servicePort: 80

Problem Sub Path Ingress Controller for Backend Service

I have problem setting path ingress controller for backend service. For example i want setup :
frontend app with angular (Path :/)
backend service with NodeJs (Path :/webservice).
NodeJS : Index.js
const express = require('express')
const app = express()
const port = 4000
app.get('/', (req, res) => res.send('Welcome to myApp!'))
app.use('/data/office', require('./roffice'));
app.listen(port, () => console.log(`Example app listening on port ${port}!`))
Another Route:roffice.js
var express = require('express')
var router = express.Router()
router.get('/getOffice', async function (req, res) {
res.send('Get Data Office')
});
module.exports = router
Deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: ws-stack
spec:
selector:
matchLabels:
run: ws-stack
replicas: 2
template:
metadata:
labels:
run: ws-stack
spec:
containers:
- name: ws-stack
image: wsstack/node/img
imagePullPolicy: IfNotPresent
ports:
- containerPort: 4000
Service.yaml
apiVersion: v1
kind: Service
metadata:
name: service-wsstack
labels:
run: service-wsstack
spec:
type: NodePort
ports:
- port: 80
protocol: TCP
nodePort: 30009
targetPort: 4000
selector:
run: ws-stack
ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: stack-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: hello-world.info
- http:
paths:
- path: /
backend:
serviceName: service-ngstack --> frondend
servicePort: 80
- path: /webservice
backend:
serviceName: service-wsstack --> backend
servicePort: 80
i setup deployment, service and ingress successfully. but when i called with curl
curl http://<minikubeip>/webservice --> Welcome to myApp! => Correct
curl http://<minikubeip>/webservice/data/office/getOffice --> Welcome to myApp! => Not correct
if i called another route, the result is the same 'Welcome to myApp'. But if i used Nodeport
curl http://<minikubeip>:30009/data/office/getOffice => 'Get Data Office', working properly.
What is the problem? any solution? Thank you
TL;DR
nginx.ingress.kubernetes.io/rewrite-target: /$2
path: /webservice($|/)(.*)
Explanation
The problem is from that line in your ingress:
nginx.ingress.kubernetes.io/rewrite-target: /
You're telling nginx to rewrite your url to / whatever it matched.
/webservice => /
/webservice/data/office/getOffice => /
To do what you're trying to do use regex, here is a simple example:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: stack-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: hello-world.info
- http:
paths:
- path: /
backend:
serviceName: service-ngstack --> frondend
servicePort: 80
- path: /webservice($|/)(.*)
backend:
serviceName: service-wsstack --> backend
servicePort: 80
This way you're asking nginx to rewrite your url with the second matching group.
Finally it gives you:
/webservice => /
/webservice/data/office/getOffice => /data/office/getOffice