The question is simple, yet I can not find a single example online... basically how do you write this in Terraform with kubernetes_ingress_v1. I basically have an app where I am using subdomains for each of the components due to overlapping paths.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-wildcard-host
spec:
rules:
- host: "foo.bar.com"
http:
paths:
- pathType: Prefix
path: "/bar"
backend:
service:
name: service1
port:
number: 80
- host: "*.foo.com"
http:
paths:
- pathType: Prefix
path: "/foo"
backend:
service:
name: service2
port:
number: 80
With kubernetes_ingress_v1, to have multiple host on same ingress I had to use multiple rule.
resource "kubernetes_ingress_v1" "monitoring" {
metadata {
name = "monitoring-alb"
namespace = "monitoring"
annotations = {
"kubernetes.io/ingress.class" = "alb"
#More annotation##
}
}
spec {
rule {
host = "monitoring.mydomain.io"
http {
path {
path_type = "ImplementationSpecific"
backend {
service {
name = "grafana"
port {
number = 80
}
}
}
}
}
}
rule {
host = "logs.mydomain.io"
http {
path {
path_type = "ImplementationSpecific"
backend {
service {
name = "loki-distributed-gateway"
port {
number = 80
}
}
}
}
}
}
}
}
Related
I am building a microfrontend-microservices app where the microfrontend is written in React and the backend utilizes a Kubernetes cluster, in particular an ingress-nginx service. There are many services (auth, company, companies, event bus, permissions ingress and query services) with a few corresponding micro (client) apps that are built using Webpack Module Federation and webpack dev server. I am using skaffold as a dev tool.
When I run skaffold dev --port-forwarding, I am able to load the app in the browser using the host domain of mavata.dev but with errors (which eventually make certain pages which rely on module federation fail to load entirely). I get the following errors in chrome dev tools:
WebSocket connection to 'wss://mavata.dev:9000/ws' failed
WebSocket connection to 'wss://mavata.dev:8081/ws' failed
WebSocket connection to 'wss://mavata.dev:8083/ws' failed
How do I fix the websocket connection?? Below are relevant code snippets..
(Client) Company App Deployment: client-company-depl.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: client-company-depl
spec:
replicas: 1
selector:
matchLabels:
app: client-company
template:
metadata:
labels:
app: client-company
spec:
containers:
- name: client-company
image: bridgetmelvin/client-company
---
apiVersion: v1
kind: Service
metadata:
name: client-company-srv
spec:
selector:
app: client-company
ports:
- name: client-company
protocol: TCP
port: 8083
targetPort: 8083
(a similar deployment for the (client) container app but with a port and targetPort of 9000)
(another similar deployment for the (client) marketing app but with a port and targetPort of 8081)
Ingress Service Config: ingress-srv.yaml:
# RUN: kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.3.1/deploy/static/provider/cloud/deploy.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-srv
annotations:
certmanager.k8s.io/cluster-issuer: core-prod
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: 'true'
nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/secure-backends: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/websocket-services: core-service
nginx.org/websocket-services: core-service
spec:
rules:
- host: mavata.dev # need to update 'hosts' file on local machine (in VS Code) C:\Windows\System32\drivers\etc
http:
paths:
- path: /company/create
pathType: Prefix
backend:
service:
name: company-clusterip-srv
port:
number: 4000
- path: /company
pathType: Prefix
backend:
service:
name: query-srv
port:
number: 4002
- path: /company/?(.*)
pathType: Prefix
backend:
service:
name: company-srv
port:
number: 4000
- path: /companies
pathType: Prefix
backend:
service:
name: companies-srv
port:
number: 4002
- path: /api/users/?(.*)
pathType: Prefix
backend:
service:
name: auth-server-srv
port:
number: 4004
- path: /permissions/?(.*)
pathType: Prefix
backend:
service:
name: permissions-srv
port:
number: 4001
- path: /?(.*)
pathType: Prefix
backend:
service:
name: client-container-srv
port:
number: 9000
(Client) Container webpack.dev.js:
const { merge } = require('webpack-merge');
const ModuleFederationPlugin = require('webpack/lib/container/ModuleFederationPlugin');
const commonConfig = require('./webpack.common');
const packageJson = require('../package.json');
const globals = require('../src/data-variables/global');
const port = globals.port;
const devConfig = {
mode: 'development',
output: {
publicPath: `http://localhost:${port}/` // don't forget the slash at the end
},
devServer: {
host: '0.0.0.0',
port: port,
allowedHosts: ['mavata.dev'],
historyApiFallback: {
index: 'index.html',
},
},
plugins: [
new ModuleFederationPlugin({
name: 'container',
filename: 'remoteEntry.js',
remotes: {
marketingMfe: 'marketingMod#http://localhost:8081/remoteEntry.js',
authMfe: 'authMod#http://localhost:8082/remoteEntry.js',
companyMfe: 'companyMod#http://localhost:8083/remoteEntry.js',
dataMfe: 'dataMod#http://localhost:8084/remoteEntry.js'
},
exposes: {
'./Functions': './src/functions/Functions',
'./Variables': './src/data-variables/Variables'
},
shared: {...packageJson.dependencies, ...packageJson.peerDependencies},
}),
],
};
module.exports = merge(commonConfig, devConfig);
(Client) Company webpack.dev.js:
const { merge } = require('webpack-merge');
const HtmlWebpackPlugin = require('html-webpack-plugin');
const ModuleFederationPlugin = require('webpack/lib/container/ModuleFederationPlugin');
const commonConfig = require('./webpack.common');
const packageJson = require('../package.json');
const path = require('path');
const globals = require('../src/variables/global')
const port = globals.port
const devConfig = {
mode: 'development',
output: {
publicPath: `http://localhost:${port}/`, // don't forget the slash at the end
},
devServer: {
port: port,
historyApiFallback: {
index: 'index.html',
},
},
plugins: [
new ModuleFederationPlugin({
name: 'companyMod',
filename: 'remoteEntry.js',
exposes: {
'./CompanyApp': './src/bootstrap',
},
remotes: {
containerMfe: 'container#http://localhost:9000/remoteEntry.js',
dataMfe: 'dataMod#http://localhost:8084/remoteEntry.js'
},
shared: {...packageJson.dependencies, ...packageJson.peerDependencies},
}),
new HtmlWebpackPlugin({
template: './public/index.html',
}),
],
};
module.exports = merge(commonConfig, devConfig);
My problem: i create ingress rule via kubernetes-client:
try (InputStream is = IOUtils.toInputStream(crd, StandardCharsets.UTF_8)) {
client.load(is).inNamespace(namespaceName).createOrReplace();
}
Where is - yaml file with ingress rule like:
spec:
ingressClassName: nginx
rules:
- host: {{host}}
http:
paths:
- backend:
service:
name: {{service_name}}
port:
number: 80
path: /
pathType: ImplementationSpecific
But in my task i need ensure that the rule is created, i need wait or check status in loop.
What is the best way to do it?
Thanx!
I think you might be able to do it with waitUntilCondition:
try (KubernetesClient client = new KubernetesClientBuilder().build()) {
Ingress ingress = client.network().v1().ingresses()
.load(IngressRuleWaitUntilCondition.class.getResourceAsStream("/ingress-rule.yml"))
.get();
client.resource(ingress)
.inNamespace("default")
.createOrReplace();
client.network().v1()
.ingresses()
.inNamespace("default")
.resource(ingress)
.waitUntilCondition(i -> !i.getSpec().getRules().isEmpty(), 2, TimeUnit.MINUTES);
}
I would like to block /public/configs in my k8s ingress.
My current settings doesnt work.
- host: example.com
http:
paths:
- path: /*
pathType: ImplementationSpecific
backend:
service:
name: service-myapp
port:
number: 80
- path: /public/configs
pathType: ImplementationSpecific
backend:
service:
name: service-myapp
port:
number: 88 // fake port
Is there any better (easy) way?
1- Create a dummy service and send it to that:
- path: /public/configs
pathType: ImplementationSpecific
backend:
service:
name: dummy-service
port:
number: 80
2- use server-snippets as bellow to return 403 or any error you want:
a) for k8s nginx ingress:
annotations:
nginx.ingress.kubernetes.io/server-snippet: |
location ~* "^/public/configs" {
deny all;
return 403;
}
b) for nginx ingress:
annotations:
nginx.org/server-snippet: |
location ~* "^/public/configs" {
deny all;
return 403;
}
I try to create an ingress resource over terraform. I receive the following error message
Error: Failed to create Ingress 'jenkins/jenkins-ingress' because: the
server could not find the requested resource (post
ingresses.extensions) │ │ with kubernetes_ingress.jenkins-ingress, │
on main.tf line 160, in resource "kubernetes_ingress"
"jenkins-ingress": │ 160: resource "kubernetes_ingress"
"jenkins-ingress" {
My terraform resource looks like this:
resource "kubernetes_ingress" "jenkins-ingress" {
metadata {
name = "${var.name}-ingress"
namespace = var.namespace
annotations = {
"ingress.kubernetes.io/rewrite-target" = "/"
"kubernetes.io/ingress.class" = "nginx"
}
}
spec {
rule {
host = "domain.com"
http {
path {
path = "/"
backend {
service_name = var.name
service_port = 8080
}
}
}
}
}
}
If I create the ingress over a yaml it works:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: jenkins-ingress
annotations:
ingress.kubernetes.io/rewrite-target: /
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: domain.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: jenkins
port:
number: 8080
What strikes me is the difference between rule (see kubernetes_ingress) and rules in the terraform script and in the yaml. Ideas?
I was getting the same error.
Try using kubernetes_ingress_v1 instead of kubernetes_ingress which uses networking.k8s.io/v1 instead of networking.k8s.io/v1beta1.
I want to connect a React Native application using Socket.io to a server that is inside a Kubernetes Cluster hosted on Google Cloud Platform (GKE).
There seems to be an issue with the Nginx Ingress Controller declaration but I cannot find it.
I have tried adding nginx.org/websocket-services; rewriting my backend code so that it uses a separate NodeJS server (a simple HTTP server) on port 3004, then exposing it via the Ingress Controller under a different path than the one on port 3003; and multiple other suggestions from other SO questions and Github issues.
Information that might be useful:
Cluster master version: 1.15.11-gke.15
I use a Load Balancer managed with Helm (stable/nginx-ingress) with RBAC enabled
All deployments and services are within the namespace gitlab-managed-apps
The error I receive when trying to connect to socket.io is: Error: websocket error
For the front-end part, the code is as follows:
App.js
const socket = io('https://example.com/app-sockets/socketns', {
reconnect: true,
secure: true,
transports: ['websocket', 'polling']
});
I expect the above to connect me to a socket.io namespace called socketdns.
The backend code is:
app.js
const express = require('express');
const app = express();
const server = require('http').createServer(app);
const io = require('socket.io')(server);
const redis = require('socket.io-redis');
io.set('transports', ['websocket', 'polling']);
io.adapter(redis({
host: process.env.NODE_ENV === 'development' ? 'localhost' : 'redis-cluster-ip-service.gitlab-managed-apps.svc.cluster.local',
port: 6379
}));
io.of('/').adapter.on('error', function(err) { console.log('Redis Adapter error! ', err); });
const nsp = io.of('/socketns');
nsp.on('connection', function(socket) {
console.log('connected!');
});
server.listen(3003, () => {
console.log('App listening to 3003');
});
The ingress service is:
ingress-service.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$1
nginx.ingress.kubernetes.io/proxy-body-size: "100m"
certmanager.k8s.io/cluster-issuer: letsencrypt-prod
nginx.ingress.kubernetes.io/proxy-connect-timeout: "7200"
nginx.ingress.kubernetes.io/proxy-read-timeout: "7200"
nginx.ingress.kubernetes.io/proxy-send-timeout: "7200"
nginx.org/websocket-services: "app-sockets-cluster-ip-service"
name: ingress-service
namespace: gitlab-managed-apps
spec:
tls:
- hosts:
- example.com
secretName: letsencrypt-prod
rules:
- host: example.com
http:
paths:
- backend:
serviceName: app-cms-cluster-ip-service
servicePort: 3000
path: /?(.*)
- backend:
serviceName: app-users-cluster-ip-service
servicePort: 3001
path: /app-users/?(.*)
- backend:
serviceName: app-sockets-cluster-ip-service
servicePort: 3003
path: /app-sockets/?(.*)
- backend:
serviceName: app-sockets-cluster-ip-service
servicePort: 3003
path: /app-sockets/socketns/?(.*)
The solution is to remove the nginx.ingress.kubernetes.io/rewrite-target: /$1 annotation.
Here is a working configuration: (please note that apiVersion has changed since the question has been asked)
Ingress configuration
ingress-service.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/proxy-body-size: "64m"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
name: ingress-service
namespace: default
spec:
tls:
- hosts:
- example.com
secretName: letsencrypt-prod
rules:
- host: example.com
http:
paths:
- backend:
service:
name: app-sockets-cluster-ip-service
port:
number: 3003
path: /app-sockets/?(.*)
pathType: Prefix
On the service (Express.js):
app.js
const redisAdapter = require('socket.io-redis');
const io = require('socket.io')(server, {
path: `${ global.NODE_ENV === 'development' ? '' : '/app-sockets' }/sockets/`,
cors: {
origin: '*',
methods: ['GET', 'POST'],
},
});
io.adapter(redisAdapter({
host: global.REDIS_HOST,
port: 6379,
}));
io.of('/').adapter.on('error', err => console.log('Redis Adapter error! ', err));
io.on('connection', () => {
//...
});
The global.NODE_ENV === 'development' ? '' : '/app-sockets' bit is related to an issue in development. If you change it here, you must also change it in the snippet below.
In development the service is under http://localhost:3003 (sockets endpoint is http://localhost:3003/sockets).
In production the service is under https://example.com/app-sockets (sockets endpoint is https://example.com/app-sockets/sockets).
On frontend
connectToWebsocketsService.js
/**
* Connect to a websockets service
* #param tokens {Object}
* #param successCallback {Function}
* #param failureCallback {Function}
*/
export const connectToWebsocketsService = (tokens, successCallback, failureCallback) => {
//SOCKETS_URL = NODE_ENV === 'development' ? 'http://localhost:3003' : 'https://example.com/app-sockets'
const socket = io(`${ SOCKETS_URL.replace('/app-sockets', '') }`, {
path: `${ NODE_ENV === 'development' ? '' : '/app-sockets' }/sockets/`,
reconnect: true,
secure: true,
transports: ['polling', 'websocket'], //required
query: {
// optional
},
auth: {
...generateAuthorizationHeaders(tokens), //optional
},
});
socket.on('connect', successCallback(socket));
socket.on('reconnect', successCallback(socket));
socket.on('connect_error', failureCallback);
};
Note: I wasn't able to do it on the project mentioned in the question, but I have on another project which is hosted on EKS, not GKE. Feel free to confirm if this works for you on GKE as well.
Just change annotations to
nginx.ingress.kubernetes.io/websocket-services: "app-sockets-cluster-ip-service"
instead of
nginx.org/websocket-services: "app-sockets-cluster-ip-service"
Mostly it will resolve your issue.