Let's Encrypt with Traefik - JWS has an invalid anti-replay nonce - docker-compose

Question

I have this composer file, it was working properly, but 2 days ago I added a new domain and it fails to get the Let's encrypt certficate,
version: '3.4'
services:
traefik:
image: "traefik:v2.4"
container_name: "traefik"
command:
- "--log.level=DEBUG"
- "--api.insecure=true"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.web.address=:80"
- "--entrypoints.web.http.redirections.entryPoint.to=websecure"
- "--entrypoints.web.http.redirections.entryPoint.scheme=https"
- "--entrypoints.web.http.redirections.entrypoint.permanent=true"
- "--entrypoints.websecure.address=:443"
- "--certificatesresolvers.myresolver.acme.dnschallenge=true"
- "--certificatesresolvers.myresolver.acme.dnschallenge.provider=cloudns"
- "--certificatesresolvers.myresolver.acme.email=myemail#gmail.com"
- "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"
labels:
- "com.centurylinklabs.watchtower.enable=false"
ports:
- "80:80"
- "443:443"
environment:
- "CLOUDNS_AUTH_ID=xxxx"
- "CLOUDNS_AUTH_PASSWORD=xxxx"
volumes:
- type: volume
source: traefik
target: /letsencrypt
- "/var/run/docker.sock:/var/run/docker.sock:ro"
networks:
- "organic"
php-apache:
image: "php7.4"
restart: "unless-stopped"
container_name: "php-apache"
labels:
- "traefik.enable=true"
- "traefik.http.routers.exampledomain.rule=Host(`example.com`)"
- "traefik.http.routers.exampledomain.entrypoints=websecure"
- "traefik.http.routers.exampledomain.tls.certresolver=myresolver"
- "com.centurylinklabs.watchtower.enable=false"
networks:
- "organic"
networks:
organic:
driver: bridge
Here are the errors I'm getting
traefik | time="2021-04-17T03:31:02Z" level=debug msg="legolog: [INFO] retry due to: acme: error: 400 :: POST :: :: urn:ietf:params:acme:error:badNonce :: JWS has an invalid anti-replay nonce:
traefik | time="2021-04-17T03:31:02Z" level=error msg="Unable to obtain ACME certificate for domains \"example.com\": unable to generate a certificate for thedomains [example.com]: error: one or more domains had a problem:\n[example.com] [example.com] acme: error presenting token: time limit exceeded: last error: %!w(<nil>)\n" providerName=myresolver.acme rule="Host(`example.com`)" routerName=exampledomain#docker
The tx is generated successfully in ClouDNS is normal, I have spent two days trying to figure out what might be the problem with no luck.

Answer 1

I had the same problem recently and after many configuration tries I deleted the traefik container (not recreate) as well as the letsencrypt volume. Then I redeployed with docker-compose up -d. On startup of traefik new certificates there generated immediately and without any complains. Hope this helps solves your issue as well.
traefik:
image: "traefik:v2.4"
container_name: "traefik"
restart: always
command:
#- "--log.level=DEBUG"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--providers.docker.network=traefik_proxy"
- "--providers.docker.endpoint=tcp://docker-socket-proxy:2375"
- "--ping=true"
- "--ping.entryPoint=ping"
- "--certificatesresolvers.letsencrypt.acme.dnschallenge=true"
- "--certificatesresolvers.letsencrypt.acme.dnschallenge.provider=cloudflare"
- "--certificatesresolvers.letsencrypt.acme.email=${CERTIFICATE_EMAIL}"
- "--certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json"
- "--certificatesResolvers.letsencrypt.acme.dnsChallenge.resolvers=1.1.1.1:53,8.8.8.8:53"
- "--entryPoints.ping.address=:8081"
- "--entrypoints.http.address=:80"
- "--entrypoints.https.address=:443"
- "--entrypoints.https.http.tls.certresolver=letsencrypt"
- "--entrypoints.https.http.tls.domains[0].main=${DOMAINNAME}"
- "--entrypoints.https.http.tls.domains[0].sans=*.${DOMAINNAME}"
environment:
- CF_DNS_API_TOKEN=${CLOUDFLARE_DNS_API_TOKEN}
healthcheck:
test: ["CMD", "wget", "-c", "http://localhost:8081/ping"]
retries: 3
interval: "15s"
timeout: "3s"
start_period: "15s"
networks:
- traefik_proxy
- socket-proxy
security_opt:
- no-new-privileges:true
ports:
- target: 80
published: 80
protocol: tcp
mode: host
- target: 443
published: 443
protocol: tcp
mode: host
volumes:
- "letsencrypt:/letsencrypt"
#- "/var/run/docker.sock:/var/run/docker.sock:ro"
labels:
- "com.centurylinklabs.watchtower.enable=true"
depends_on:
- docker-socket-proxy