Certificate always in 'False' state using LetsEncrypt with cluster issuer in k8s - kubernetes

Question

I am unable to issue a working certificate for my ingress host in k8s. I use a ClusterIssuer to issue certificates and the same ClusterIssuer has issued certificates in the past for my ingress hosts under my domain name *xyz.com. But all of a sudden neither i can issue new Certificate with state 'True' for my host names nor a proper certificate secret (kubernetes.io/tls) gets created (but instead an Opaque secret gets created).
**strong text**
**kubectl describe certificate ingress-cert -n abc**
Name: ingress-cert
Namespace: abc
Labels: <none>
Annotations: <none>
API Version: cert-manager.io/v1beta1
Kind: Certificate
Metadata:
Creation Timestamp: 2021-09-08T07:48:32Z
Generation: 1
Owner References:
API Version: extensions/v1beta1
Block Owner Deletion: true
Controller: true
Kind: Ingress
Name: test-ingress
UID: c03ffec0-df4f-4dbb-8efe-4f3550b9dcc1
Resource Version: 146643826
Self Link: /apis/cert-manager.io/v1beta1/namespaces/abc/certificates/ingress-cert
UID: 90905ab7-22d2-458c-b956-7100c4c77a8d
Spec:
Dns Names:
abc.xyz.com
Issuer Ref:
Group: cert-manager.io
Kind: ClusterIssuer
Name: letsencrypt
Secret Name: ingress-cert
Status:
Conditions:
Last Transition Time: 2021-09-08T07:48:33Z
Message: Issuing certificate as Secret does not exist
Reason: DoesNotExist
Status: False
Type: Ready
Last Transition Time: 2021-09-08T07:48:33Z
Message: Issuing certificate as Secret does not exist
Reason: DoesNotExist
Status: True
Type: Issuing
Next Private Key Secret Name: ingress-cert-gdq7g
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Issuing 11m cert-manager Issuing certificate as Secret does not exist
Normal Generated 11m cert-manager Stored new private key in temporary Secret resource "ingress-cert-gdq7g"
Normal Requested 11m cert-manager Created new CertificateRequest resource "ingress-cert-dp6sp"
I checked the certificate request and it contains no events. Also i can see no challenges. I have added the logs below. Any help would be appreciated
kubectl describe certificaterequest ingress-cert-dp6sp -n abc
Namespace: abc
Labels: <none>
Annotations: cert-manager.io/certificate-name: ingress-cert
cert-manager.io/certificate-revision: 1
cert-manager.io/private-key-secret-name: ingress-cert-gdq7g
API Version: cert-manager.io/v1beta1
Kind: CertificateRequest
Metadata:
Creation Timestamp: 2021-09-08T07:48:33Z
Generate Name: ingress-cert-
Generation: 1
Owner References:
API Version: cert-manager.io/v1alpha2
Block Owner Deletion: true
Controller: true
Kind: Certificate
Name: ingress-cert
UID: 90905ab7-22d2-458c-b956-7100c4c77a8d
Resource Version: 146643832
Self Link: /apis/cert-manager.io/v1beta1/namespaces/abc/certificaterequests/ingress-cert-dp6sp
UID: fef72617-fc1d-4384-9f4b-a7e4502582d8
Spec:
Issuer Ref:
Group: cert-manager.io
Kind: ClusterIssuer
Name: letsencrypt
Request: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ2Z6Q0NBV2NDQVFBd0FEQ0NBU0l3RFFZSktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUxMNgphTGhZNjhuNnhmMUprYlF5ek9OV1J4dGtLOXJrbjh5WUtMd2l4ZEFMVUl0TERra0t6Uksyb3loZzRMMThSQmQvCkNJaGJ5RXBYNnlRditKclRTOC84T1A0MWdwTUxBLzROdVhXWWtyeWhtZFdNaFlqa21OOFpiTUk1SlZZcVV2cVkKRWQ1b2cydmVmSjU1QlJPRExsd0o3YjBZa3hXckUwMGJxQ1ExWER6ZzFhM08yQ2JWd1NQT29WV2x6Uy9CdzRYVgpMeVdMS3E4QU52b2dZMUxXRU8xcG9YelRObm9LK2U2YVZueDJvQ1ZLdGxPaG1iYXRHYXNSaTJKL1FKK0dOWHovCnFzNXVBSlhzYVErUzlxOHIvbmVMOXNPYnN2OWd1QmxCK09yQVg2eHhkNHZUdUIwVENFU00zWis2c2MwMFNYRXAKNk01RlY3dkFFeDQyTWpuejVoa0NBd0VBQWFBNk1EZ0dDU3FHU0liM0RRRUpEakVyTUNrd0p3WURWUjBSQkNBdwpIb0ljY25kemMyZHdMbU5zYjNWa1oyRjBaUzV0YVdOeWIyWnBiaTVrWlRBTkJna3Foa2lHOXcwQkFRc0ZBQU9DCkFRRUFTQ0cwTXVHMjZRbVFlTlBFdmphNHZqUUZOVFVINWVuMkxDcXloY2ZuWmxocWpMbnJqZURuL2JTV1hwdVIKTnhXTnkxS0EwSzhtMG0rekNPbWluZlJRS1k2eHkvZU1WYkw4dTgrTGxscDEvRHl3UGxvREE2TkpVOTFPaDM3TgpDQ0E4NWphLy9FYVVvK0p5aHBzaTZuS1d4UXRpYXdmYXhuNUN4SENPWGF5Qzg0Q0IzdGZ2WWp6YUF3Ykx4akxYCmxvd09LUHNxSE51ZktFM0NtcjZmWGgramd5VWhxamYwOUJHeGxCWEFsSVNBNkN5dzZ2UmpWamFBOW82TmhaTXUKbmdheWZON00zUzBBYnAzVFFCZW8xYzc3QlFGaGZlSUE5Sk51SWtFd3EvNXppYVY1RDErNUxSSnR5ZkVpdnJLTwpmVjQ5WkpCL1BGOTdiejhJNnYvVW9CSkc2Zz09Ci0tLS0tRU5EIENFUlRJRklDQVRFIFJFUVVFU1QtLS0tLQo=
Status:
Conditions:
Last Transition Time: 2021-09-08T07:48:33Z
Message: Waiting on certificate issuance from order abc/ingress-cert-dp6sp-3843501305: ""
Reason: Pending
Status: False
Type: Ready
Events: <none>
Here is the ingress.yaml
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: 20m
kubernetes.io/ingress.class: "nginx"
cert-manager.io/cluster-issuer: "letsencrypt"
spec:
rules:
- host: abc.xyz.com
http:
paths:
- path: /static
backend:
serviceName: app-service
servicePort: 80
- path: /
backend:
serviceName: app-service
servicePort: 8000
tls:
- hosts:
- abc.xyz.com
secretName: ingress-cert
Here is the clusterissuer:
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: example#user.de
privateKeySecretRef:
name: letsencrypt-key
solvers:
- http01:
ingress:
class: nginx

Answer 1

Works only with Nginx Ingress Controller
I was using ClusterIssuer but I changed it to Issuer and it works.
-- Install cert-manager (Installed version 1.6.1) and be sure that the three pods are running
-- Create an Issuer by appling this yml be sure that the issuer is running.
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: letsencrypt-nginx
namespace: default
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: example#example.com
privateKeySecretRef:
name: letsencrypt-nginx-private-key
solvers:
- http01:
ingress:
class: nginx
-- Add this to your ingress annotations
cert-manager.io/issuer: letsencrypt-nginx
-- Add the secretName to your ingress spec.tls.hosts
spec:
tls:
- hosts:
- yourdomain.com
secretName: letsencrypt-nginx
Notice that the Nginx Ingress Controller is able to generate the Certificate CRD automatically via a special annotation: cert-manager.io/issuer. This saves work and time, because you don't have to create and maintain a separate manifest for certificates as well (only the Issuer manifest is required). For other ingresses you may need to provide the Certificate CRD as well.

Answer 2

Ideally your ingress pointing to the secret which is storing the secret or SSL/TLS key cert.
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: 20m
kubernetes.io/ingress.class: "nginx"
cert-manager.io/cluster-issuer: "letsencrypt"
spec:
rules:
- host: abc.xyz.com
http:
paths:
- path: /static
backend:
serviceName: app-service
servicePort: 80
- path: /
backend:
serviceName: app-service
servicePort: 8000
tls:
- hosts:
- abc.xyz.com
secretName: letsencrypt-key
Your cluster issue storing the key
privateKeySecretRef:
name: letsencrypt-key
You have to use this secret and attach this to ingress.
If secret already storing cert with a domain
test.example.com and you are trying to get a new cert with hello.example.com
in this case using cluster issuer will overwrite the secret and might loss old cert stored inside secret.
You can create the multiple clusterissuer,
One storing and connect to single ingress, first.example.com
Second cluster issuer with different key name
privateKeySecretRef:
name: letsencrypt-key
and new key or secret will get attached to the ingress.