This may be a very basic question, but I'm struggling with it. I'm attempting to set up logical decoding between 2 separate servers.
On the Master, I've gotten it setup so that the changes that I make to a table (INSERT, UPDATE, DELETE) are sent to my logical replication slot, and I can see the changes using the pg_logical_slot_get/peek_changes functions - all on my Master server.
On the Slave, I'm attempting to run the pg_revclogical command (using command prompt), however, I can't seem to get it to receive the changes made on the Master. I've realized that no where have I told the 2 to communicate to each other. I tried to define the host as the Master to tell the replication process that I have to pull the changes from the Master to the Slave.
I've made all of the required changes to the postgresql.conf (wal_level=logical; max_replication_slots=3; max_wal_senders=3) and pg_hba.conf (whatever it told me to do to fix errors) files on both Master and Slave. The pg_recvlogical command and the resulting error message are below. Could someone please help me get my Master and Slave communicating?
pg_recvlogical --start --slot=wendy_test --plugin=test_decoding --dbname=testdb --file=C:\Logical_Decoding_Test.log --username=dbaadmin --host=127.##.##.### --port=5432
When I execute this command on the Slave, I don't receive any error messages, but it doesn't return to a command prompt either? While it's doing its thing, I can check the database and see that the wendy_test slot is active, so it appears to be doing something. However, when I INSERT a row into the table on my Master, nothing happens on my Slave (even after 5 minutes).
I can add all of the code that I have used to create the replication slots, populate the queue, etc. I was trying to limit the amount of reading and figured those to be more basic. Also, my end game is to transition this to have our AWS RDS system as our Slave, so any recommendations for that is appreciated as well. THank you for your time.
Related
I am not too familiar with Bucardo and Postgres , so I was hoping to get some feedback / how to from this question.
We have 3 computers running in parallel in various parts of the building. When one updates something the other shows the data etc thanks to bucardo syncs running between the computers.
However here is the requirement. At anytime one computer can be brought offline and reimaged. Now when this computer comes back on line, the operator should be able to hit replicate and get the data from the master computer.
What is the best way to accomplish this ?
My thought was to run a pg_dump on the master computer and run pg_restore on the reimaged computer.
or do you think setting bucardo onetimecopy=2 is the best course of action.
I have a problem. Currently I have a 1 TB Postgres 9.6 database which is backed up with Barman with streaming.
What I need:
A replication from the production/master to the slave server:
On which I can write, I don’t care if the written data on the replica
is not sent to the master server
Which can be configured almost in real time or with little delay
On which I can use dump without locking the master database
As said above I am using Barman for backing up. However I am not able to find out how I can build a replica from Barman which is sync by the master. It was set up by a someone else and i'm not sure its the right solution for what I need.
My questions:
Is Barman the good tool for what I want ?
If no. Which tools would you suggest to me ?
If yes. Do you know how to build replica from Barman which is
sync by the master ? Could you please explain to me how to do
it?
Thanks
in master-slave mode, you can't write on slave
if you want to write on replica to you should probably use something like this
also you can make sure all of your writes on master also written on replica via synchronous-wal-streaming feature
via this feature, before wiritng on master , first master makes sure write was written successfully on replica
except for writing on slave part , barman looks a fit tool for you
writing on slave is a uncommon thing in postgresql
I am very new to postgres and being new I got stuck at a point and need some help, please pardon if you find it silly.
I am doing a pgpool HA and at postgres level i have streaming replication between 3 nodes of postgresql-9.5 - 1 master and 2 slaves
I was trying to configure auto failover but when i switched back to my original master, and restarted the postgres service, I am getting the following error:
slave 1-highest timeline 1 of the primary is behind recovery timeline 11
slave 2-highest timeline 1 of the primary is behind recovery timeline 10
slave 3-highest timeline 1 of the primary is behind recovery timeline 3
I tried deleting pg_xlog files in slaves and copying all the files from master pg_xlog into the slaves and then did a rsync.
i also did a pg_rewind but it says:
target server needs to use either data checksums or wal_log_hints = on
(I have wal_log_hints = on set in postgresql.conf already)
I've tried doing a pg_basebackup but since the data base server in slaves are still starting up its not able to connect to the server
Is there any way to bring the master and the slave at a same timeline?
In my case, it happened because ( experimentally ), I updated the standby database tables and again when I simulate the master-standby streaming replication I got the same errors.
So once again I cleaned the whole standby database directory and migrate the master database using cmd like
"pg_basebackup -P -R -X stream -c fast -h 10.10.40.105 -U postgres -D standby/"
I think something is wrong in your pgpool configuration. What tool you have been using for manement of replication and master-slave control? Is it post master or repmgr?
I was trying to configure pgpool with 3 data nodes using a tutorial from http://jensd.be/591/linux/setup-a-redundant-postgresql-database-with-repmgr-and-pgpool and have done it correctly.
Also you can lean auto failover here.
(These question is obviously duplicate of this one, so I'll repeat the answer also.)
I'm not sure what you exactly mean by "when i switched back to my original master", but it looks that you are doing the wrongest possible thing in PostgreSQL streaming replication - introducing the second master.
The most important thing you should know about PostgreSQL replication is that once the failover is performed, you cannot simply "switch back to original master" - there's now a new master in cluster, and existence of two masters will make damage.
After a slave is promoted to master, the only way for you to re-join the old master is to:
Destroy it (delete the data directory);
Join it as a slave.
If you want it to be master again you'll continue with the following:
Let it run for awhile as a slave so that it can sync the data;
Kill temporary master and failover to old master;
Rejoin temporary master again as a slave.
You cannot simply switch master servers! Master can be created ONLY by failover (promoting a slave)
You should also know that whenever you are performing failover (whenever the master is changed), all slaves (except for the one that is promoted) need to be reconfigured to target the new master.
I suggest you reading this tutorial - it'll help.
I set up slony to replicate 3 tables from one opensuse pc (master node) to another opensuse pc (slave node). It works well at first. After running for few days, it suddenly come out the error message of -
ERROR: Slony-I: Table euprofiles is replicated and cannot be modified on a subscriber node - role=0
euprofiles is one of the tables being replicated by slony.
I know that this message may occur if you are trying to write to the table of slave node. But here I am writing to master node only.
Does anyone see similar problem?
Never had this happen. Are you certain you're connecting to the db you think you're connecting to? Slony may be a bit difficult to setup and such, but it doesn't just randomly decide a master is now a slave.
If you psql into the two databasesand do \d euprofiles on each what do they say? The source table should have something like this at the end:
Triggers:
_slony_www_logtrigger_228 AFTER INSERT OR DELETE OR UPDATE ON users FOR EACH ROW EXECUTE PROCEDURE _slony_www.logtrigger('_slony_www', '228', 'kvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv')
and the destination table should have something like this:
Triggers:
_slony_www_denyaccess_228 BEFORE INSERT OR DELETE OR UPDATE ON users FOR EACH ROW EXECUTE PROCEDURE _slony_www.denyaccess('_slony_www')
If they both look like this last trigger there's some problem. But I'm betting you're just connecting to the wrong server. Let's hope it's that simple.
My company's website uses a PostgreSQL database. In our data center we have a master DB and a few read-only slave DB's, and we use Londiste for continuous replication between them.
I would like to setup another read-only slave DB for reporting purposes, and I'd like this slave to be in a remote location (outside the data center). This slave doesn't need to be 100% up-to-date. If it's up to 24 hours old, that's fine. Also, I'd like to minimize the load I'm putting on the master DB. Since our master DB is busy during the day and idle at night, I figure a good idea (if possible) is to get the reporting slave caught up once each night.
I'm thinking about using log shipping for this, as described on
http://www.postgresql.org/docs/8.4/static/continuous-archiving.html
My plan is:
Setup WAL archiving on the master DB
Produce a full DB snapshot and copy it to the remote location
Restore the DB and get it caught up
Go into steady state where:
DAYTIME -- the DB falls behind but people can query it
NIGHT -- I copy over the day's worth of WAL files and get the DB caught up
Note: the key here is that I only need to copy a full DB snapshot one time. Thereafter I should only have to copy a day's worth of WAL files in order to get the remote slave caught up again.
Since I haven't done log-shipping before I'd like some feedback / advice.
Will this work? Does PostgreSQL support this kind of repeated recovery?
Do you have other suggestions for how to set up a remote semi-fresh read-only slave?
thanks!
--S
Your plan should work.
As Charles says, warm standby is another possible solution. It's supported since 8.2 and has relatively low performance impact on the primary server.
Warm Standby is documented in the Manual: PostgreSQL 8.4 Warm Standby
The short procedure for configuring a
standby server is as follows. For full
details of each step, refer to
previous sections as noted.
Set up primary and standby systems as near identically as possible,
including two identical copies of
PostgreSQL at the same release level.
Set up continuous archiving from the primary to a WAL archive located
in a directory on the standby server.
Ensure that archive_mode,
archive_command and archive_timeout
are set appropriately on the primary
(see Section 24.3.1).
Make a base backup of the primary server (see Section 24.3.2), and load
this data onto the standby.
Begin recovery on the standby server from the local WAL archive,
using a recovery.conf that specifies a
restore_command that waits as
described previously (see Section
24.3.3).
To achieve only nightly syncs, your archive_command should exit with a non-zero exit status during daytime.
Additional Informations:
Postgres Wiki about Warm Standby
Blog Post Warm Standby Setup
9.0's built-in WAL streaming replication is designed to accomplish something that should meet your goals -- a warm or hot standby that can accept read-only queries. Have you considered using it, or are you stuck on 8.4 for now?
(Also, the upcoming 9.1 release is expected to include an updated/rewritten version of pg_basebackup, a tool for creating the initial backup point for a fresh slave.)
Update: PostgreSQL 9.1 will include the ability to pause and resume streaming replication with a simple function call on the slave.