Unable to start Patroni-Consul setup - postgresql

Question

We are trying to setup a 3-node PG with Patroni-Consul-Postgres, however Patroni fails to spinup the db with below error. I let it run for sometime but with no change in status. Which one is the service ID or is there some mismatch in my config?
[ec2-user#ip-172-31-68-200 ~]$ /home/ec2-user/.local/bin/patroni /etc/patroni.yml
2023-02-02 14:25:52,677 INFO: No PostgreSQL configuration items changed, nothing to reload.
2023-02-02 14:25:52,683 INFO: Deregister service pglab/pgdbhost1
2023-02-02 14:25:52,688 INFO: Lock owner: None; I am pgdbhost1
2023-02-02 14:25:52,688 INFO: Deregister service pglab/pgdbhost1
2023-02-02 14:25:52,693 INFO: waiting for leader to bootstrap
2023-02-02 14:25:52,695 INFO: Lock owner: None; I am pgdbhost1
2023-02-02 14:25:52,697 INFO: Deregister service pglab/pgdbhost1
2023-02-02 14:25:52,698 INFO: waiting for leader to bootstrap
2023-02-02 14:26:02,696 INFO: Lock owner: None; I am pgdbhost1
2023-02-02 14:26:02,698 INFO: Deregister service pglab/pgdbhost1
2023-02-02 14:26:02,700 INFO: waiting for leader to bootstrap
^C2023-02-02 14:26:06,200 INFO: Deregister service pglab/pgdbhost1
[ec2-user#ip-172-31-68-200 ~]$ journalctl -xen | less
-- Logs begin at Thu 2023-02-02 10:09:52 UTC, end at Thu 2023-02-02 14:26:06 UTC. --
Feb 02 14:24:17 ip-172-31-68-200.ec2.internal sshd[13313]: pam_unix(sshd:session): session opened for user ec2-user by (uid=0)
Feb 02 14:24:44 ip-172-31-68-200.ec2.internal dhclient[2925]: XMT: Solicit on eth0, interval 123490ms.
Feb 02 14:25:27 ip-172-31-68-200.ec2.internal sudo[13415]: ec2-user : TTY=pts/2 ; PWD=/home/ec2-user ; USER=root ; COMMAND=/usr/local/bin/patroni /etc/patroni.yml
Feb 02 14:25:27 ip-172-31-68-200.ec2.internal sudo[13415]: pam_unix(sudo:session): session opened for user root by ec2-user(uid=0)
Feb 02 14:25:27 ip-172-31-68-200.ec2.internal sudo[13415]: pam_unix(sudo:session): session closed for user root
Feb 02 14:25:52 ip-172-31-68-200.ec2.internal bash[7029]: 2023-02-02T14:25:52.683Z [ERROR] agent.http: Request error: method=PUT url=/v1/agent/service/deregister/pglab/pgdbhost1 from=127.0.0.1:42250 error="Unknown service ID \"pglab/pgdbhost1\". Ensure that the service ID is passed, not the service name."
Feb 02 14:25:52 ip-172-31-68-200.ec2.internal bash[7029]: 2023-02-02T14:25:52.689Z [ERROR] agent.http: Request error: method=PUT url=/v1/agent/service/deregister/pglab/pgdbhost1 from=127.0.0.1:42250 error="Unknown service ID \"pglab/pgdbhost1\". Ensure that the service ID is passed, not the service name."
Feb 02 14:25:52 ip-172-31-68-200.ec2.internal bash[7029]: 2023-02-02T14:25:52.698Z [ERROR] agent.http: Request error: method=PUT url=/v1/agent/service/deregister/pglab/pgdbhost1 from=127.0.0.1:42250 error="Unknown service ID \"pglab/pgdbhost1\". Ensure that the service ID is passed, not the service name."
Feb 02 14:26:02 ip-172-31-68-200.ec2.internal bash[7029]: 2023-02-02T14:26:02.699Z [ERROR] agent.http: Request error: method=PUT url=/v1/agent/service/deregister/pglab/pgdbhost1 from=127.0.0.1:42250 error="Unknown service ID \"pglab/pgdbhost1\". Ensure that the service ID is passed, not the service name."
Please find the Patroni & Consul server/agent configs below.
Patroni.yml
name: "pgdbhost1"
scope: pglab
namespace: /HAPG/
consul:
url: http://127.0.0.1:8500
register_service: true
postgresql:
connect_address: "172.31.68.200:5432"
bin_dir: /usr/pgsql-12/bin
data_dir: /var/lib/pgsql/12/
listen: "*:5432"
authentication:
replication:
username: replicator
password: replicator
superuser:
username: postgres
password: postgres
rewind:
username: rewind_user
password: rewind_user
parameters:
unix_socket_directories: '/var/run/postgresql'
synchronous_commit: "on"
synchronous_standby_names: "*"
restapi:
connect_address: "172.31.68.200:8008"
listen: "172.31.68.200:8008"
bootstrap:
dcs:
loop_wait: 10
retry_timeout: 5
maximum_lag_on_failover: 1048576
postgresql:
parameters:
use_pg_rewind: true
use_slots: true
users:
app_user:
password: "eW5guPae"
pg_hba:
- local all all md5
- host all all 127.0.0.1/32 md5
- host all all ::1/128 md5
- host all all ::1/128 md5
- host all all 0.0.0.0/0 md5
- host replication replicator 127.0.0.1/32 md5
- host replication replicator 172.31.68.200/0 md5
- host replication replicator 172.31.74.220/0 md5
- host replication replicator 172.31.75.194/0 md5
- host all all 0.0.0.0/0 md5
initdb:
- encoding: UTF8
Consul Config:
{
"node_name": "PGDB1",
"data_dir": "/consul/data",
"datacenter": "AWS",
"retry_join":[
"172.31.70.83"
],
"encrypt": "mUWL5XahLJ3GdLW13W/PUS6vnqXNt10ckAk/lCo9/S8=",
"recursors": ["127.0.0.11"],
"client_addr": "172.31.68.200",
"bind_addr": "172.31.68.200",
"advertise_addr": "172.31.68.200",
"addresses": {
"dns": "127.0.0.1",
"http": "127.0.0.1"
}
}
===========================================
{
"bootstrap_expect": 1,
"client_addr": "0.0.0.0",
"datacenter": "AWS",
"data_dir": "/var/consul",
"domain": "consul-pgdb",
"enable_script_checks": true,
"dns_config": {
"enable_truncate": true,
"only_passing": true
},
"enable_syslog": true,
"encrypt": "mUWL5XahLJ3GdLW13W/PUS6vnqXNt10ckAk/lCo9/S8=",
"leave_on_terminate": true,
"log_level": "INFO",
"rejoin_after_leave": true,
"server": true,
"start_join": [
"172.31.70.83"
],
"ui": true,
"ports": {
"dns": 9600
}
}