I have an Ingress setup and initially I used a placeholder name, "thesis.info". Now I would like change this hostname but whenever I to change it I just end up getting 404 errors.
Change the spec.tls.rules.host value in the yaml to the new hostname
Change CN value which openssl uses for the crt and key that are generated for TLS
Edit the value /etc/hosts value on my local machine
Is there a step I am missing that could be causing a problem. I am baffled by why it works with one value but not the other.
Below is the ingress yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: frontend-ingress
namespace: thesis
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/add-base-url: "true"
nginx.ingress.kubernetes.io/service-upstream: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
tls:
- hosts:
- thesis
secretName: ingress-tls
rules:
- host: pod1out.ie
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: frontend
port:
number: 3000
---
Most likely, you can find a hint on what is going on in the nginx logs. If you have access, you can access the logs using something like this:
kubectl -n <ingress-namespace> get pods
# should be one or more nginx pods
kubectl -n <ingress-namespace> logs <nginx-pod>
Not sure if this is the only issue, but according to the documentation, the host in 'tls' has to match explicitly the host in the rules:
spec:
tls:
- hosts:
- pod1out.ie
secretName: ingress-tls
rules:
- host: pod1out.ie
Before struggling with tls, I would recommend making the http route itself work (eg. by creating another ingress resource), and if this works with the host you want, go for tls.
Related
So my goal is to be able to access my CockroachDB from domain like db.test.com with cert.
I want to use cert-manager letsencrypt to issue keys. And it should work with CF (in non proxy mode as I think they do not support tcp for this)
At first to test everything I used normal kubectl port-forward which worked, but now I needed to expose it always.
I have tried using Ingress (using ingress-nginx)
I know that Ingress is mostly HTTP/HTTPS but I saw it can be used for the thing I need and IN CF I cannot point to port that I needed.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: tcp-example-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/tcp-services: "cluster-cockroachdb-public"
nginx.ingress.kubernetes.io/tcp-service-port: "26257"
nginx.ingress.kubernetes.io/backend-protocol: "TCP"
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- db.test.com
secretName: db-access-ssl-cert-production
rules:
- host: db.test.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: cluster-cockroachdb-public
port:
number: 26257
Attempting to connect does not work, and in logs I can see 400 status code with strange characters like \x20...
No matter what I tried I could not get it to work..
I did manage to get web-ui portion working that was easy enough.
Other resource that might be helpful is the values.yaml that I used
conf:
cache: "2Gi"
max-sql-memory: "2Gi"
# My WEB-UI that works
ingress:
enabled: true
labels: {}
annotations:
kubernetes.io/ingress.class: nginx
cert-manager.io/cluster-issuer: letsencrypt-production
paths: [/]
hosts:
- db-ui.test.com
tls:
- hosts: [db-ui.test.com]
secretName: ssl-cert-production
Everything else is default
I solved my issue by following the tutorial below:
https://mailazy.com/blog/exposing-tcp-udp-services-ingress/
also mentioned here
https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/
Ingress does not support TCP or UDP services so we use ingress-nginx config for it we patch ingress-nginx values of chart and add custom one (Copy default values.yaml from github helm chart for ingress-nginx)
I just edited this portion:
# -- TCP service key-value pairs
## Ref: https://github.com/kubernetes/ingress-nginx/blob/main/docs/user-guide/exposing-tcp-udp-services.md
##
tcp:
"26257": "default/cluster-cockroachdb-public:26257"
After that we run helm upgrade command to replace values of ingress-nginx and after that it should work for anyone else as well.
If you are using cloudflare make sure to disable proxy!
So I have a bunch of services running in a cluster, all exposed via HTTP only ingress object, example:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
name: some-ingress
spec:
ingressClassName: nginx
rules:
- http:
paths:
- backend:
service:
name: some-svc
port:
number: 80
path: /some-svc(/|$)(.*)
pathType: Prefix
They are accessed by http://<CLUSTER_EXTERNAL_IP>/some-svc, and it works ofc.
Now I want to create an additional ingress object for every service which will force SSL connections and allow the use of a domain instead of an IP address.
The problem is that the newer SSL ingresses always return 404 while testing the connection.
The manifests are as follows:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: "some-ingress-ssl"
annotations:
ingress.kubernetes.io/ssl-redirect: "true"
ingress.kubernetes.io/app-root: "/some-svc"
spec:
tls:
- hosts:
- foobar.com
secretName: foobar-tls
rules:
- host: foobar.com
http:
paths:
- path: /some-svc(/|$)(.*)
pathType: Prefix
backend:
service:
name: some-svc
port:
number: 80
tests (foobar.com point to CLUSTER_EXTERNAL_IP):
> curl -I http://<CLUSTER_EXTERNAL_IP>/some-svc
HTTP/1.1 200 OK
> curl -I https://foobar.com/some-svc
HTTP/2 404
Is it possible to have both ingresses simultaneously? (one enforcing SSL, the other not)
If so what am I doing wrong here?
Figured out I was missing this annotation:
nginx.ingress.kubernetes.io/rewrite-target: /$2
in SSL ingress...
works like a charm now, maybe someone will find this usefull
In my Kubernetes Cluster i have some challenges with the Ingress. As example i installed NodeRed und the Nginx-ingress via Helm. NodeRed is available via
FQDN: http://my.server.name:31827
IP: http://10.x.x.x:31827
Now i created an Ingress:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: nr-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- secretName: tls-secret1
hosts:
- my.server.name
rules:
- host: my.server.name
http:
paths:
- path: /nr
backend:
serviceName: my-nodered-node-red
servicePort: 1880
When i do a Get http://my.server.name/nr i see only parts working, see the screenshot:
It looks to me, that i missed the reverse proxy settings. Normally i would place those things in a reverse proxy setting in the nginx like this. But this is not possible because i am using the Nginx-ingress.
location / {
proxy_pass http://localhost:1880/;
}
But i do not know how to do that in Kubernetes ? What do i miss ? kubernetes version is 1.14.1.
I haven't used it so I'm not sure if it helps, but you might want to try adding an annotation for Proxy redirect.
With the annotations nginx.ingress.kubernetes.io/proxy-redirect-from and nginx.ingress.kubernetes.io/proxy-redirect-to it is possible to set the text that should be changed in the Location and Refresh header fields of a proxied server response
Setting "off" or "default" in the annotation nginx.ingress.kubernetes.io/proxy-redirect-from disables nginx.ingress.kubernetes.io/proxy-redirect-to, otherwise, both annotations must be used in unison. Note that each annotation must be a string without spaces.
By default the value of each annotation is "off".
Also you can use ConfigMap for customizing your Nginx configuration.
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app: ingress-nginx
data:
http-snippet: |
location = / {
proxy_pass http://localhost:1880/;
}
Or server-snippets annotation for your Ingress:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: nr-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/server-snippet: |
location = / {
proxy_pass http://localhost:1880/;
}
spec:
tls:
- secretName: tls-secret1
hosts:
- my.server.name
rules:
- host: my.server.name
http:
paths:
- path: /nr
backend:
serviceName: my-nodered-node-red
servicePort: 1880
I hope this helps.
Maybe too late for the answer, but I had the same problem and solve it:
1-changed httpRoot: '/nr' in node red settings.xml configuration file (in kubernetes, probably defined in a PV) (#vasili-angapov mentions)
2- set ingress:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nodered-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- http:
paths:
- path: /nr
pathType: Prefix
backend:
service:
name: nodered
port:
number: 1880
It looks like your static content is still using root path prefix, you can verify that using browser developer console. Common applications should have a configuration to understand that they are running on non-root path prefix. You should find that configuration option in your application and configure it properly. Nginx ingress has nothing to do with this error.
Could there be any reason why a webapp which perfectly loads up fine gives a *HTTP 400 Bad request - The plain HTTP request was sent to HTTPS* port after the webapp's ingress has been edited manually or edited through an automated job which updates the ingress modifying the Whitelisted IPs
Apparently, this issue gets fixed when we redeploy the webapp after purging the webapp deployment...
Any pointers to this would be great as this happens on our PROD env and not reproducible on any lower envs.
Points to note:-
- Nginx Ingress controller setup is the same across lower envs and Prod env.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
ingress.kubernetes.io/force-ssl-redirect: "true"
ingress.kubernetes.io/ingress.allow-http: "false"
ingress.kubernetes.io/proxy-body-size: 20M
ingress.kubernetes.io/secure-backends: "true"
ingress.kubernetes.io/whitelist-source-range: xxx.yy.zz.pp/32, yyy.ss.dd.kkl/32
ingress.kubernetes.io/whitelist-source-range-status: unlocked
creationTimestamp: 2018-11-29T15:34:05Z
generation: 5
labels:
app: xyz-abc-pqr
name: xxxx-webapp-yyyy-zzzz
namespace: nspsace-name
resourceVersion: "158237270"
selfLink: /apis/extensions/v1beta1/namespaces/nspsace-name/ingresses/xxxx-webapp-yyyy-zzzz
uid: 348f892e-f3ec-11e8-aa6f-0a0340041348
spec:
rules:
- host: ssssssss.wwwwweewwerwerw.co.uk
http:
paths:
- backend:
serviceName: xxxx-webapp-yyyy-zzzz
servicePort: 443
path: /
- host: xxxx-webapp-yyyy-zzzz.bbbbv.lasdfkla.ajksdfh.ioohhsaf.pp
http:
paths:
- backend:
serviceName: xxxx-webapp-yyyy-zzzz
servicePort: 443
path: /
tls:
- hosts:
- ssssssss.wwwwweewwerwerw.co.uk
- xxxx-webapp-yyyy-zzzz.bbbbv.lasdfkla.ajksdfh.ioohhsaf.pp
secretName: xxxx-webapp-yyyy-zzzz-server-tls
status:
loadBalancer:
ingress:
- {}
There may be something wrong with the ingress controller and how it updates its configuration. I'm assuming you are using a nginx ingress controller so you can inspect the configs before an after:
$ kubectl cp <nginx-ingress-controller-pod>:nginx.conf nginx.conf.before
$ kubectl edit ingress <your-ingress>
$ kubectl cp <nginx-ingress-controller-pod>:nginx.conf nginx.conf.after
$ diff nginx.conf.before nginx.conf.after
You can see the this may happen with nginx because of something like this: Dealing with nginx 400 "The plain HTTP request was sent to HTTPS port" error.
I've configured my Kubernetes to use one wildcard SSL certificate to all my apps using cert-manager and letsencrypt, now the problem is that I can't configure subdomain redirects cause Ingress is kinda "stiff". Here's how I'm trying to achieve this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-wildcard-ingress
namespace: mynamespace
annotations:
kubernetes.io/ingress.class: nginx
certmanager.k8s.io/cluster-issuer: letsencrypt-prod
certmanager.k8s.io/acme-challenge-type: dns01
certmanager.k8s.io/acme-dns01-provider: azuredns
ingress.kubernetes.io/force-ssl-redirect: "true"
ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: "domain.com"
http:
paths:
- path: /
backend:
serviceName: some-service
servicePort: 3000
- host: somesub.domain.com
http:
paths:
- path: /
backend:
serviceName: some-other-service
servicePort: 80
- host: othersub.domain.com
http:
paths:
- path: /
backend:
serviceName: one-more-service
servicePort: 8080
- host: "*.domain.com"
http:
paths:
- path: /
backend:
serviceName: default-service-to-all-other-non-mapped-subdomains
servicePort: 8000
tls:
- secretName: domain-com-tls
hosts:
- "*.domain.com.br"
The problem is that Ingress ignores the declared subdomain redirects just because they're not listed in the "tls:hosts" section. And if I do put them there, it tries to issue the SSL certificate using the wildcard and the other subdomains as well in the same cert, which causes the issuer to refuse the order, saying the obvious: "subdomain.domain.com and *.domain.com are redundant"
Is there any other way that I can declare those redirects and force them to use my SSL wildcard certificate?
Well, for anyone who's having this kind of trouble, I've managed to solve it (not the best solution, but it's a start). For this, I'll be using cert-manager and letsencrypt.
First, I've created a ClusterIssuer to issue for my certs with letsencrypt:
apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod-dns
spec:
acme:
dns01:
providers:
- azuredns:
clientID: MY_AZURE_CLIENT_ID
clientSecretSecretRef:
key: client-secret
name: azure-secret
hostedZoneName: mydomain.com
resourceGroupName: MY_AZURE_RESOURCE_GROUP_NAME
subscriptionID: MY_AZURE_SUBSCRIPTION_ID
tenantID: MY_AZURE_TENANT_ID
name: azuredns
email: somemail#mydomain.com
privateKeySecretRef:
key: ""
name: letsencrypt-prod-dns
server: https://acme-v02.api.letsencrypt.org/directory
Then I've created a fallback ingress to all my subdomains (this one will be the cert generator):
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
certmanager.k8s.io/acme-challenge-type: dns01
certmanager.k8s.io/acme-dns01-provider: azuredns
certmanager.k8s.io/cluster-issuer: letsencrypt-prod-dns
ingress.kubernetes.io/force-ssl-redirect: "true"
ingress.kubernetes.io/ssl-redirect: "true"
kubernetes.io/ingress.class: nginx
name: wildcard-ingress
namespace: some-namespace
spec:
rules:
- host: '*.mydomain.com'
http:
paths:
- backend:
serviceName: some-default-service
servicePort: 80
path: /
tls:
- hosts:
- '*.mydomain.com'
- mydomain.com
secretName: wildcard-mydomain-com-tls
Notice that I've declared at the TLS section the wildcard AND the absolute paths, so the cert will be valid for the URLs without subdomains too.
At this point, any requests to your domain, will be redirected to "some-default-service" with SSL(cert-manager will issue for a new cert as soon as you create the fallback ingress. This can take a while once cert-manager dns01 issuer is not mature yet), great!!!
But, what if you need to redirect some specific subdomain to another service? No problem (since they're running on the same namespace), all you have to do is to create a new ingress to your subdomain, pointing it to your existing wildcard-mydomain-com-tls cert secret:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
ingress.kubernetes.io/force-ssl-redirect: "false"
ingress.kubernetes.io/ssl-redirect: "true"
kubernetes.io/ingress.class: nginx
name: somesubdomain-ingress
namespace: some-namespace
spec:
rules:
- host: somesubdomain.mydomain.com
http:
paths:
- backend:
serviceName: some-other-service
servicePort: 8080
path: /
tls:
- hosts:
- somesubdomain.mydomain.com
secretName: wildcard-mydomain-com-tls
Easy peasy lemon squeezy!!! Now your somesubdomain.mydomain.com overrides your fallback rule and sends the user to another app. The only thing you should notice here is that the secret is valid only for "some-namespace" namespace, if you need to use this cert in another namespace, you could:
Copy the secret from namespace "some-namespace" to "other-namespace". If you do this, remember that cert-manager will NOT renew this cert automatically for "other-namespace", so, you'd have to copy the secret again, every time your cert expires.
Recreate the fallback ingress to every namespace you have, so you'd have a new cert for each of them. This approach is more ingress verbose, but, it's fully automatic.
I guess that's it. Hope someone out there can benefit from this info.
Cheers
So the best course of action here is probably to just not use ingress-shim to manage your Certificate resource.
Instead, you can manually create a Certificate resource and then reference the secret it produces in all of your ingresses.
We are exploring options to workaround this limitation in ingresses at the moment, however there has so far not been any progress!