I have a cluster in GKE and it is working, everything seems to be working. If I forward the ports I am able to see that the containers are working.
I am not able to setup a domain I own from namecheap.
These are the steps I followed
In Namecheap I setup a custom dns for the domain
ns-cloud-c1.googledomains.com.
ns-cloud-c2.googledomains.com.
ns-cloud-c3.googledomains.com.
ns-cloud-c3.googledomains.com.
I used the letter c because the cluster is in a c zone (I am not sure if this is right)
Because I am trying to setup as secure website I installed nginx ingress controller
kubectl create clusterrolebinding cluster-admin-binding \
--clusterrole cluster-admin \
--user $(gcloud config get-value account)
and
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.46.0/deploy/static/provider/cloud/deploy.yaml
I applied the issuer.yml
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
namespace: cert-manager
spec:
acme:
# The ACME server URL
server: https://acme-v02.api.letsencrypt.org/directory
# Email address used for ACME registration
email: example#email.com
# Name of a secret used to store the ACME account private key
privateKeySecretRef:
name: letsencrypt-prod
# Enable the HTTP-01 challenge provider
solvers:
- http01:
ingress:
class: nginx
I applied ingress
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
namespace: staging
name: ingress
annotations:
cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
tls:
- hosts:
- www.stagingmyappsrl.com
- api.stagingmyappsrl.com
secretName: stagingmyappsrl-tls
rules:
- host: wwwstaging.myappsrl.com
http:
paths:
- backend:
serviceName: myappcatalogo-svc
servicePort: 80
- host: apistaging.stagingmyappsrl.com
http:
paths:
- backend:
serviceName: myappnodeapi-svc
servicePort: 80
It seems that everything is created and working if I check in GKE website, but when I try to access I get DNS_PROBE_FINISHED_NXDOMAIN
I am not sure if I am missing an step or if I am setting up something wrong
GKE should have created a cloud load balancer for your ingress service. Depending on your config, the LB can be internal or external. You can get your LB information by looking at the services:
kubectl get svc -n ingress-nginx
Create a CNAME record in your DNS (namecheap) with the LB address and that should do it. Alternatively, if you have an IP address of the LB, create an A record in your DNS.
Cert-manager will create an ingress resource to resolve HTTPS01 challenges. Make sure your ingresses are reachable over the Internet for the HTTPS01 challenges to work. Alternatively, you could explore other solvers.
Related
I am trying to use 'Kubernetes Ingress with Traefik, CertManager, LetsEncrypt and HAProxy' for certificates management.
What I want to do is use certificates in my application which deployed on kubernetes.
My application contains following services:
my-app1 NodePort 10.43.204.206 16686:31149/TCP
my-app2 NodePort 10.43.204.208 2746:30972/TCP
So for my-app1 without certificates I was accessing it as http://{IP}:31149/app1. And with certificates I am now accessing it as https://my-dns.com/app1. For this I am using this link. I created following ingress resource:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: prod-ingress
namespace: my-ns
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
kubernetes.io/ingress.class: traefik
traefik.ingress.kubernetes.io/router.tls: "true"
spec:
tls:
- hosts:
- "my-dns.com"
secretName: prod-cert
rules:
- host: "my-dns.com"
http:
paths:
- path: /app1
pathType: Prefix
backend:
service:
name: my-app1
port:
number: 16686
But for my-app2 without certificates I was accessing it as https://{IP}:30972/app2. So as I am already using https for my-app2 but I want to also use certificates for this service.
Any idea how to do this?
So the issue was, I was deploying my application with self signed certificates. So because of this I am getting issue while accessing my dashboard.
So now I just disabled self signed certificates in my application. And now I am able
to access dashboard with domain name https://my-dns.com.
I have deployed airflow in kubernetes as is descrived in this link: https://github.com/apache/airflow/tree/master/chart
To access the airflow UI I can do:
kubectl port-forward svc/airflow2-webserver 8080:8080 --namespace default
But I would want to expose it in a url. I found this guide:
https://godatadriven.com/blog/deploying-apache-airflow-on-azure-kubernetes-service/
In the bottom part: FQDN with Ingress controller, he installs an nginx-ingress-controller and a cert manager.
Then I create a Cluster-issuer:
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
server: https://acme-staging-v02.api.letsencrypt.org/directory
email: my-email#gmail.com
privateKeySecretRef:
name: letsencrypt-staging
solvers:
- http01:
ingress:
class: nginx
podTemplate:
spec:
nodeSelector:
"kubernetes.io/os": linux
I install the ingress-routes.yml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: airflow-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
certmanager.k8s.io/cluster-issuer: letsencrypt-staging
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- mydomain.westeurope.cloudapp.azure.com
secretName: tls-secret-test2
rules:
- host: mydomain.westeurope.cloudapp.azure.com
http:
paths:
- path: /
backend:
serviceName: airflow2-webserver
servicePort: 8080
But when I try to get my certificate:
kubectl describe certificate
No resources found in default namespace.
I have deployed everything but I don´t know what is missing but when I go to my domain is still not trusted
It looks like your DNS A record is not pointing to your Ingress LoadBalancer public IP. Thus your cert-manager issuer is not able to validate your domain ownership from outside using http challange.
To verify you can use tool called MXToolBox. In your configuration (https://mxtoolbox.com/SuperTool.aspx?action=a%3amydomain.westeurope.cloudapp.azure.com&run=toolpage) it's pointing to private IP.
Solution
To resolve this issue, you should Add an A record to your DNS zone
az network dns record-set a add-record \
--resource-group myResourceGroup \
--zone-name MY_CUSTOM_DOMAIN \
--record-set-name '*' \
--ipv4-address MY_EXTERNAL_IP
So I am in the process of migrating my bare metal cluster onto GKE and ran into an issue with the ingress. On my bare metal cluster, I used the ingress controller from nginxinc which worked fine. Below is a sample of an Ingress file of a particular deployment:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
namespace: mynamespace
name: app-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: myhost
http:
paths:
- path: /dev/appname(/|$)(.*)
backend:
serviceName: app
servicePort: 80
Basically, when a user types in myhost/dev/appname, myhost is resolved to a HAProxy server. The proxy then routes the request to the NodePort that the ingress service is running on.
I tried to do the same thing on my GKE cluster with the only exception being that the Ingress controller on the GKE cluster is exposed using a LoadBalancer as per the documentation
However I get a 502 error if I try to access the URL.
TL;DR: Looking for the best way to access various applications (deployments) deployed on a GKE cluster using URL's like: myhost/dev/firstapp and myhost/dev/secondapp etc.
You can use Kong Ingress as your ingress controller on GKE for your path based ingresses. You can install Kong Ingress from GCP Marketplace. It is easy to integrate and also supports various plugins for authenticating, monitoring etc.
You'll get detailed information and installation instructions from https://github.com/Kong/google-marketplace-kong-app#basic-usage
I would follow this guide on setting up Nginx on GKE. The ingress looks like below:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-resource
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- http:
paths:
- path: /hello
backend:
serviceName: hello-app
servicePort: 8080
You should be able to access your app externally http://external-ip-of-ingress-controller/hello
Now to debug 502 issue verify that the health-checks from the Loadbalancer to your app is passing or not.
Digging into www, didn't found an answer:
I want to know how I can use cert-manager with haproxy-ingress and lets encrypt.
Any documentation / guidelines?
Deploy Certmanager with:
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.4.0/cert-manager.yaml
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.4.0/cert-manager.crds.yaml
Deploy a ClusterIssuer (cluster issuers are namespace agnostic)
cat > prod-issuer.yaml <<EOF
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: email#example.com
privateKeySecretRef:
name: letsencrypt-prod
solvers:
- http01:
ingress:
class: haproxy
EOF
Apply the cluster issuer with kubectl apply -f prod-issuer.yaml
Create an Ingress Resource (namespace gnostic, this example is using Nginx backend service)
cat > nginx-ingress.yaml <<EOF
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-router
namespace: production
annotations:
kubernetes.io/ingress.class: "haproxy"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
tls:
- hosts:
- "example.com"
secretName: nginx-tls-secret
rules:
- host: example.com
http:
paths:
- pathType: ImplementationSpecific
path: "/"
backend:
service:
name: nginx
port:
number: 80
- host: www.example.com
http:
paths:
- pathType: ImplementationSpecific
path: "/"
backend:
service:
name: nginx
port:
number: 80
EOF
Apply the ingress recourse with kubectl apply -f nginx-ingress.yaml
The important piece of information here, is that the Haproxy controller does NOT need the annotation acme.cert-manager.io/http01-edit-in-place: "true"
that nginx-ingress controller does. It works as expected without any extra annotations.
When you apply the Ingress Resourse to the cluster, the certificate will be issued in 1-2 minutes tops.
Use kubectl describe certificate nginx-tls-secret -n production to check the status of the certificate, and look at the event to get the certificate.
For more debugging info incase something went wrong, refer here https://cert-manager.io/docs/faq/acme/
you can try installing cert manager provided by jetstack, can be found here and then you need to follow the steps mentioned in this stackoverflow post and this will get things sorted for you.
An internal acme-challenge will be done by cert manager and once you patch the secret name, mentioned in the certificate to the TLS of ingress then certificate status will get ready state, Note that the secret will get created automatically, you need not create it
I'm trying to get a GKE ingress to require basic auth like this example from github.
The ingress works fine. It routes to the service. But the authentication isn't working. Allows all traffic right through. Has GKE not rolled this feature out yet? Something obviously wrong in my specs?
Here's the ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: super-ingress
annotations:
ingress.kubernetes.io/auth-type: basic
ingress.kubernetes.io/auth-secret: basic-auth
ingress.kubernetes.io/auth-realm: "Authentication Required"
spec:
rules:
- host: zzz.host.com
http:
paths:
- backend:
serviceName: super-service
servicePort: 9000
path: /*
And the basic-auth secret:
$ kubectl get secret/basic-auth -o yaml
apiVersion: v1
data:
auth: XXXXXXXXXXXXXXXXXXX
kind: Secret
metadata:
creationTimestamp: 2016-10-03T21:21:52Z
name: basic-auth
namespace: default
resourceVersion: "XXXXX"
selfLink: /api/v1/namespaces/default/secrets/basic-auth
uid: XXXXXXXXXXX
type: Opaque
Any insights are greatly appreciated!
The example you linked to is for nginx ingress controller. GKE uses GLBC, which doesn't support auth.
You can deploy an nginx ingress controller in your gke cluster. Note that you need to annotate your ingress to avoid the GLBC claiming the ingress. Then you can expose the nginx controller directly, or create a glbc ingress to redirect traffic to the nginx ingress (see this snippet written by bprashanh).