I am new to PostgreSql replication.
I tried to set up streaming replication, and at the end I created a database on the master, which I could afterwards see on the slave.
However, when I created a table on the master, it is not replicated to the slave.
Checking the table pg_stat_replication on the master, it looks OK as far as I can understand:
select usename,application_name,client_addr,backend_start,state,sync_state from pg_stat_replication ;
usename | application_name | client_addr | backend_start | state | sync_state
------------+------------------+-------------+-------------------------------+-----------+------------
replicator | walreceiver | 10.97.7.150 | 2020-06-28 20:48:15.463922+03 | streaming | async
select client_addr, state, sent_lsn, write_lsn,replitest,flush_lsn, replay_lsn from pg_stat_replication;
client_addr | state | sent_lsn | write_lsn | flush_lsn | replay_lsn
-------------+-----------+------------+------------+------------+------------
10.97.7.150 | streaming | 0/2701AFB8 | 0/2701AFB8 | 0/2701AFB8 | 0/2701AFB8
On the slave side I see this:
SELECT pg_last_xact_replay_timestamp();
pg_last_xact_replay_timestamp
-------------------------------
2020-06-28 20:52:22.915897+03
select pg_is_in_recovery();
pg_is_in_recovery
-------------------
t
Still when I create a table, I cannot find her on the slave side.
What should I check further?
Related
I'm trying to use pgpool to postgres HA.
node_id | hostname | port | status | pg_status | lb_weight | role | pg_role | select_cnt | load_bala
nce_node | replication_delay | replication_state | replication_sync_state | last_status_change
---------+----------+------+--------+-----------+-----------+---------+---------+------------+----------
---------+-------------------+-------------------+------------------------+---------------------
0 | master | 5432 | up | up | 0.500000 | primary | primary | 1 | false
| 0 | | | 2022-05-30 10:33:21
1 | slave | 5432 | up | up | 0.500000 | standby | primary | 0 | true
| 419431440 | | | 2022-05-30 10:33:21
In this process, other process is working well, but I got empty value replictation_state and replication_sync_state.
And I got high value in replication_delay.
Why those values are empty and high value?
Is there should change values in postgres.conf or pgpool.conf for replication?
In this case, I used 'pg_basebackup -h host -U Repuser -p port -D dir -X stream' for slave
this is pcp_node_info's result
master 5432 2 0.500000 up up primary primary 0 none none 2022-05-30 10:42:40
slave 5432 2 0.500000 up up standby primary 419431848 none none 2022-05-30 10:42:40
Sorry to my English Level, Thank you for your help
My version
postgres 14.2
pgpool 4.3.1
You need to provide application_name in both configurations files - myrecovery.conf (primary_conninfo variable) and pgpool.conf for each node.
Also you should check recovery_1st_stage and follow_primary.sh files as there you also find block with application_name. Script are used by pgpool to recover replica (with pcp_recover_node) or promote new master.
After all you can check current value with "select * from pg_stat_replication;" (on master) or "select * from pg_stat_wal_receiver;" (on replica)
More information: https://www.pgpool.net/docs/pgpool-II-4.3.1/en/html/example-cluster.html
This is a web application which uses Postgres to store data pushed from various modules of the web application.
From Postgres side, how can I know what data entered/modified into Postgres? I mean if there is any Postgres data logging?
Postgres has 4-5 schemas & each schema has 2-3 tables. Each table has 10-20 records.
EDIT (8 Sept 2021):
log_destination=csvlog - I tried different destinations to see if I'm getting the required logs. But all log destination are capturing the same logs.
2: reload - Did restarted server every time I made
changes to postgresql.conf.
3: The logs are being generated at
/var/lib/postgresql/data/pg_log directory with format
postgresql-%Y-%m-%d_%H%M%S.log. Where latest logs are generated, but as mentioned in comment, I'm not getting all the query logs that are being executed on Postgres by the web application
4: I installed Postgres 9.6 as a Docker container.
5: I'm making changes in
/var/lib/postgresql/data/postgresql.conf which is reflected in
database query SELECT name, setting FROM pg_settings WHERE name LIKE '%log%'; after restart.
EDIT (7 Sept 2021):
name |setting |
---------------------------+------------------------------+
log_autovacuum_min_duration|-1 |
log_checkpoints |off |
log_connections |off |
log_destination |csvlog |
log_directory |pg_log |
log_disconnections |off |
log_duration |off |
log_error_verbosity |default |
log_executor_stats |off |
log_file_mode |0600 |
log_filename |postgresql-%Y-%m-%d_%H%M%S.log|
log_hostname |off |
log_line_prefix | |
log_lock_waits |off |
log_min_duration_statement |-1 |
log_min_error_statement |error |
log_min_messages |warning |
log_parser_stats |off |
log_planner_stats |off |
log_replication_commands |off |
log_rotation_age |1440 |
log_rotation_size |10240 |
log_statement |all |
log_statement_stats |off |
log_temp_files |-1 |
log_timezone |Etc/UTC |
log_truncate_on_rotation |off |
logging_collector |on |
syslog_facility |local0 |
syslog_ident |postgres |
syslog_sequence_numbers |on |
syslog_split_messages |on |
wal_log_hints |off |
EDIT
Check a the logging configuration in postgresql.conf. You want to set up this What to log in particular log_statement. For your use the mod setting would be appropriate. Then you can look at the Postgres log to see what was changed and when. You might also want to set log_connections and log_disconnections to see what user the connection is running as.
After performing a fail over, I had the previous Primary down, and the old standby became the Primary, as expected.
$ repmgr -f /etc/repmgr.conf cluster show --compact
ID | Name | Role | Status | Upstream | Location | Prio. | TLI
----+-----------------+---------+-----------+----------+----------+-------+-----
1 | server1 | primary | - failed | | default | 100 | ?
2 | server2 | primary | * running | | default | 100 | 2
3 | PG-Node-Witness | witness | * running | server2 | default | 0 | 1
I would like to make the old Primary join the cluster as a standby.
I gather the rejoin command should do that.
However, when I try to rejoin it, to be the new standby, I get this (I run this on the old Primary which is down ):
repmgr -f /etc/repmgr.conf -d 'host=10.9.7.97 user=repmgr dbname=repmgr' node rejoin
--where 10.9.7.97 is the ip of node I am running from.
I get this error:
$ repmgr -f /etc/repmgr.conf -d 'host=10.97.7.97 user=repmgr dbname=repmgr' node rejoin --verbose -
NOTICE: using provided configuration file "/etc/repmgr.conf"
ERROR: connection to database failed
DETAIL:
could not connect to server: Connection refused
Is the server running on host "10.97.7.97" and accepting
TCP/IP connections on port 5432?
Of course postgres is down on 10.9.7.97 - the old primary.
If I start it however, it starts as another primary:
$ repmgr -f /etc/repmgr.conf cluster show --compact
ID | Name | Role | Status | Upstream | Location | Prio. | TLI
----+-----------------+---------+-----------+----------+----------+-------+-----
1 | server1 | primary | ! running | | default | 100 | 1
2 | server2 | primary | * running | | default | 100 | 2
3 | PG-Node-Witness | witness | * running | server2 | default | 0 | 1
so what is the way to make the old primary the new standby...?
Thanks
Apparently the
-d 'host=
in the rejoin command, should specify the current Primary (previous standby).
Data replication has stopped from one of my three nodes. The replication slot on the errant node has disappeared. Does anyone have insight as to what happened or how to fix it?
DETAILS:
Nodes SS1, SS2, and SS3 have publications to which SSK subscribes. Replication from SS2 is now failing. Using PostgreSQL 10.1.
SSK psql log:
2019-02-07 10:21:13.953 CST [26274] LOG: logical replication apply worker for subscription "SS2" has started
2019-02-07 10:21:14.309 CST [26274] ERROR: could not start WAL streaming: ERROR: replication slot "SS2" does not exist
2019-02-07 10:21:14.311 CST [1641] LOG: worker process: logical replication worker for subscription 17237 (PID 26274) exited with exit code 1
SS2 replication slots table:
slot_name | plugin | slot_type | datoid | database | temporary | active | active_pid | xmin | catalog_xmin | restart_lsn | confirmed_flush_lsn
-----------+--------+-----------+--------+----------+-----------+--------+------------+------+--------------+-------------+---------------------
(0 rows)
For comparison, SS1 replication slots table:
slot_name | plugin | slot_type | datoid | database | temporary | active | active_pid | xmin | catalog_xmin | restart_lsn | confirmed_flush_lsn
-----------+----------+-----------+--------+----------+-----------+--------+------------+------+--------------+-------------+---------------------
SS1 | pgoutput | logical | 33280 | DBAdd | f | t | 2113 | | 56655301 | 3/114FB460 | 3/114FB498
(1 row)
Replication slot don't just disappear.
Somebody or something must have deleted it.
Perhaps the PostgreSQL database log of the primary server has valuable information.
Did you promote a standby recently? Since replication slots are not replicated, that would make them disappear.
CLOUD SQL VERSION & DB ENGINE: Currently our CLOUD MYSQL Version is 5.6.21 n DB ENGINE is INNODB
1. Create User In Mysql
Create User 'USERNAME' # 'HOSTNAME' Identified By 'PASSWORD';
But This User Is Not Permanently Stored In mysql.user Table. This User Getting removed In The Table If Any Issue Comes In Script Side Or Server Restarts...and also sometimes, created user password gets empty.
2.Likewise Grant Execute Permission For Procedure Also Not Working Properly.
Grant Execute On Procedure Schemaname . Spname To 'USERNAME'#'%';
This Execute Permission Works For Some Time,But The Privileges Immediately Disappears For The Granted User.
Other Solutons We Tried Are:
1.Flush Tables-After Creating User
2.Flush Privilges- After Giving Any Grant Access/Revoke Access
But These 2 Solutions Are Also Not Working In Google Cloud Sql, Still Issue Remains Same.
But This Issue We Dont Have In Local Mysql Version, It Is Reproducible Only On Google Cloud Sql.
We are Struck With This Issue In Our Front End App.
Anyone knows how To resolve This Issue In Google Cloud Sql...
I'm not able to reproduce the fact that a creating a user doesn't survive a Cloud SQL instances.
Here is how I tested (I replaces some sensitive information with (edited)).
First I connect to an existing instance and create a user called xxx and checked that it shows up in the mysql.user table.
$ mysql -uroot -proot -h (edited)
mysql> SELECT host,user,password FROM mysql.user;
+-----------+------+-------------------------------------------+
| host | user | password |
+-----------+------+-------------------------------------------+
| localhost | root | |
| 127.0.0.1 | root | |
| ::1 | root | |
| localhost | | |
| % | root | *81F5E21E35407D884A6CD4A731AEBFB6AF209E1B |
+-----------+------+-------------------------------------------+
5 rows in set (0.07 sec)
mysql> CREATE USER xxx#'%' IDENTIFIED BY 'xxx';
Query OK, 0 rows affected (0.61 sec)
mysql> SELECT host,user,password FROM mysql.user;
+-----------+------+-------------------------------------------+
| host | user | password |
+-----------+------+-------------------------------------------+
| localhost | root | |
| 127.0.0.1 | root | |
| ::1 | root | |
| localhost | | |
| % | root | *81F5E21E35407D884A6CD4A731AEBFB6AF209E1B |
| % | xxx | *3D56A309CD04FA2EEF181462E59011F075C89548 |
+-----------+------+-------------------------------------------+
6 rows in set (0.06 sec)
mysql> Bye
Then I restart the Cloud SQL instances.
$ gcloud sql instances restart (edited) --project (edited)
Restarting Cloud SQL instance...done.
Restarted [https://www.googleapis.com/sql/v1beta3/projects/(edited)/instances/(edited)].
$
Then I connected again and check the mysql.user tables.
$ mysql -uroot -proot -h (edited)
mysql> SELECT host,user,password FROM mysql.user;
+-----------+------+-------------------------------------------+
| host | user | password |
+-----------+------+-------------------------------------------+
| localhost | root | |
| 127.0.0.1 | root | |
| ::1 | root | |
| localhost | | |
| % | root | *81F5E21E35407D884A6CD4A731AEBFB6AF209E1B |
| % | xxx | *3D56A309CD04FA2EEF181462E59011F075C89548 |
+-----------+------+-------------------------------------------+
6 rows in set (0.07 sec)
mysql> Bye
$