I find official pglogical documentation very confusing. There are too many things that are not clear. Perhaps someone who configured pglogical before could explain how to setup a basic logical replication.
There are two PostgreSQL 9.5 instances - 10.128.0.8 (archlinux1) and 10.128.0.9 (archlinux2). Extension is already installed, CREATE EXTENSION succeeded. There is a table on each instance:
create table test (k text primary key, v text);
I would like to replicate it from archlinux1 to archlinux2.
According to the documentation I should create a provider node:
SELECT pglogical.create_node(
node_name := 'provider1',
dsn := 'host=providerhost port=5432 dbname=db'
);
Should it be executed on master? Should providerhost be 127.0.0.1 or 10.128.0.8? Currenlty replication is allowed only from localhost (accordingly to docs) - should it be changed? My best guess - it should be executed on master like this:
SELECT pglogical.create_node(
node_name := 'provider1',
dsn := 'host=127.0.0.1 port=5432 dbname=eax'
);
create_node
-------------
2976894835
(1 row)
Next:
SELECT pglogical.replication_set_add_all_tables('default', ARRAY['public']);
Should it be executed on master, replica, or both? My best guess - only on master:
SELECT pglogical.replication_set_add_all_tables('default', ARRAY['public']);
replication_set_add_all_tables
--------------------------------
t
(1 row)
Next:
SELECT pglogical.create_node(
node_name := 'subscriber1',
dsn := 'host=thishost port=5432 dbname=db'
);
Apparently it should be executed on replica:
SELECT pglogical.create_node(
node_name := 'subscriber1',
dsn := 'host=127.0.0.1 port=5432 dbname=eax'
);
create_node
-------------
330520249
(1 row)
Next step:
SELECT pglogical.create_subscription(
subscription_name := 'subscription1',
provider_dsn := 'host=providerhost port=5432 dbname=db'
);
Best guess - execute it on replica like this:
SELECT pglogical.create_subscription(
subscription_name := 'subscription1',
provider_dsn := 'host=10.128.0.8 port=5432 dbname=eax'
);
ERROR: could not connect to the postgresql server: could not connect to server: Connection refused
Is the server running on host "10.128.0.8" and accepting
TCP/IP connections on port 5432?
DETAIL: dsn was: host=10.128.0.8 port=5432 dbname=eax
Oops. OK, modifying pg_hba.conf and postgresql.conf properly on master:
# pg_hba.conf
host all all 10.128.0.0/16 md5
# postgresql.conf
listen_addresses = 'localhost,10.128.0.8
Still no luck:
# SELECT pglogical.create_subscription(
subscription_name := 'subscription1',
provider_dsn := 'host=10.128.0.8 port=5432 dbname=eax user=eax password=qwerty'
);
ERROR: could not connect to the postgresql server in replication mode: FATAL: no pg_hba.conf entry for replication connection from host "10.128.0.9", user "eax", SSL off
DETAIL: dsn was: host=10.128.0.8 port=5432 dbname=eax user=eax password=qwerty
Adding to pg_hba.conf on master:
host replication eax 10.128.0.0/16 md5
On replica (success!):
SELECT pglogical.create_subscription(
subscription_name := 'subscription1',
provider_dsn := 'host=10.128.0.8 port=5432 dbname=eax user=eax password=qwerty'
);
create_subscription
---------------------
1763399739
(1 row)
Now on master:
eax=# insert into test values ('aaa', 'bbb');
INSERT 0 1
eax=# select * from test;
k | v
-----+-----
aaa | bbb
(1 row)
On replica:
eax=# select * from test;
k | v
---+---
(0 rows)
Naturally it didn't work. Nothing helpful in logs. Any advice?
UPD: I also created a corresponding issue in pglogical issue tracker.
Perhaps this is a better guide: http://bonesmoses.org/2016/10/14/pg-phriday-perfectly-logical/ - at least it tells you which nodes to execute each command on.
Depesz published an excellent blog post about pglogical configuration https://www.depesz.com/2016/11/08/major-version-upgrading-with-minimal-downtime/
When creating nodes you need to specify an external IP (not localhost)
On the provider
SELECT pglogical.create_node(
node_name := 'provider1',
dsn := 'host=subscriberhost port=5432 dbname=db'
);
On the subscriber
SELECT pglogical.create_node(
node_name := 'subscriber1',
dsn := 'host=providerhost port=5432 dbname=eax'
);
Related
How do I specify the search_path while connecting to my postgres db using PQconnectdb? I want to set the search_path to my_schema. I'm currently using the following connection command:
PQconnectdb("host=localhost user=my_user password=my_password port=5432 dbname=my_db")
You could add the following to your connection info:
options='-csearch_path=my_schema'
So, it would become:
PQconnectdb("host=localhost user=my_user password=my_password port=5432 dbname=my_db options='-csearch_path=my_schema'")
Reference:
https://www.postgresql.org/message-id/D960CB61B694CF459DCFB4B0128514C20886AD46#exadv11.host.magwien.gv.at
I'm using dblink_connect inside a postgres function, like this:
select dblink_connect('gw', 'dbname=databasename port=5432 host=RemoteIP user=username password=pass');
When the remote server is down, this dblink_connect take a long time to return
"ERROR: could not establish connection".
I need to set a timeout to this dblink_connect to return error as quick as possible.
I tried something like this:
begin;
SET session statement_timeout to 5000;
select dblink_connect('gw', 'dbname=databasename port=5432 host=RemoteIP user=username password=pass');
commit;
But not worked.
Any idea that I can set timeout ?
You need to specify the connect_timeout, in the connection string:
dblink_connect('gw', 'connect_timeout=2 dbname=databasename port=5432 host=RemoteIP user=username password=pass')
While it is blocked waiting for the connection, it is in a state where it can't be interrupted, which is why statement_timeout doesn't work.
I'm actually facing an issue. I've installed pgbouncer on a production server, on which i've a Odoo instance and postgresql as well.
Perhaps :
In my logs, i'm having this :
2018-09-10 16:39:16.389 10123 WARNING C-0x1eb5478:
(nodb)/(nouser)#unix(18272):6432 pooler error: no such database: postgres
2018-09-10 16:39:16.389 10123 LOG C-0x1eb5478: (nodb)/(nouser)#unix(18272):6432 login failed: db=postgres user=oerppreprod
Here is the actual conf of pgbouncer :
pgbouncer_archive = host=127.0.0.1 port=5432 dbname=archive
admin_users = postgres
ignore_startup_parameters = extra_float_digits
With aswell, the default config (i've only added/edited this).
Why is he trying to connect on the postgres database ?
When i go back on the previous conf (without PGBouncer, just swapping from port 6432 to 5432), everything is working ....
Any idea ?
Thanks in advance !
I had the same issue, and in my situation. Maybe it will be usefull to somebody:
I have solved this by a few steps:
At the beginning of every request - your Framework or PDO (or else) running the initial query to check if database you asking is exists in the postgres data to process you request.
I have replaced the part of line "user=project_user password=mytestpassword" from the database section of pgbouncer.ini file. As I tested, if you replace this part - then the pgbouncer will use your userlist.txt file (or your selected auth), in my case, it was the userlist.txt.
Added the line "postgres = host=127.0.0.1 port=5432 dbname=postgres"
[databases]
postgres = host=127.0.0.1 port=5432 dbname=postgres
my_database = host=127.0.0.1 port=5432 dbname=my_database
My userlist.txt file looks like this (I am using auth_type = md5, so my password was in md5):
"my_user" "md5passwordandsoelse"
I have added my admin users to my pgbouncer.ini file:
admin_users = postgres, my_user
After all manipulations I advise you to check from which user u are running queries, by usin this simple query:
select current_user;
At the end, with this query you must to receive you selected username (in my case it was - my_user)
p.s. also I must to mention, that I was using 127.0.0.1 - because my pgbouncer is installed on the same server with postgres.
Error:
postgres=# insert into company values(4,'tom',21,'pune' ,21 );
^CCancel request sent
WARNING: canceling wait for synchronous replication due to user request
DETAIL: The transaction has already committed locally, but might not have been replicated to the standby.
INSERT 0 1
Even after the error it is executing the query on the master as well as replicating transactions the slave.
On Master:
postgres=# SELECT pg_current_xlog_location();
pg_current_xlog_location
--------------------------
0/1900D0C0
(1 row)
On Slave:
postgres=# SELECT pg_last_xlog_receive_location();
pg_last_xlog_receive_location
-------------------------------
0/1900D0C0
(1 row)
synchronous_standby_name set on master from config file(set by me) is different from the application_name I see it on record of pg_stats_replication table. Many of the solutions have suggested to change the application name. However, I am not sure from where it is taking application name as walreceiver on the master.
On Master:
postgres=# select application_name, sync_state from pg_stat_replication;
application_name | sync_state
------------------+------------
walreceiver | async
(1 row)
postgres=# show synchronous_standby_names;
synchronous_standby_names
---------------------------
slave1
(1 row)
postgres=# show synchronous_commit;
synchronous_commit
--------------------
on
(1 row)
One of the solution I found is to create tablespace dir under path '/var/lib/pgsql/9.2/data/' which I currently dont have. I am not sure if that solution will work for 9.5 Postgresql.
Any help on this is appreciated. Thank you.
Changing synchronous_standby_names to 'walreceiver' resolved the error. Followed link To know more about synchronous_standby_names
I have such code:
package main
import (
"database/sql"
"fmt"
"log"
_ "github.com/lib/pq"
)
func main() {
db, err := sql.Open("postgres", "user=postgres dbname=vagrant sslmode=disable")
if err != nil {
log.Fatal(err)
}
rows, err := db.Query("SELECT 3+5")
if err != nil {
log.Fatal(err)
}
fmt.Println(rows)
}
And its result is:
[vagrant#localhost go-postgres]$ go run test.go
2015/12/19 11:03:53 pq: Ident authentication failed for user "postgres"
exit status 1
But I can access postgres:
[vagrant#localhost go-postgres]$ psql -U postgres vagrant
psql (9.4.4)
Type "help" for help.
vagrant=#
And I don't have any troubles with using Rails app with postgres.
Anyone has an idea?
EDIT:
Here is my pg_hba.conf:
local all all trust
host all all 127.0.0.1/32 ident
host all all ::1/128 ident
EDIT2:
I found this in my postgres logs:
< 2015-12-19 12:13:05.094 UTC >LOG: could not connect to Ident server at address "::1", port 113: Connection refused
< 2015-12-19 12:13:05.094 UTC >FATAL: Ident authentication failed for user "postgres"
< 2015-12-19 12:13:05.094 UTC >DETAIL: Connection matched pg_hba.conf line 84: "host all all ::1/128 ident"
I think it will really help ;)
Ok,
correct connection string is:
"user=postgres host=/tmp dbname=vagrant sslmode=disable"
I assumed pg library uses same defaults as psql or ruby driver. Lesson learned.
Run in console:
psql "user=postgres dbname=vagrant sslmode=disable"
If you cannot connect so it needs to change sslmode else I will think more.
For me, setting host to /run/postgresql worked.
Credit: https://github.com/lib/pq/issues/645#issuecomment-320799610
The host url depends on the socket Postgres is using. The socket location varies based on how and where postgres was built. The socket location can be found using:
Inside psql, run \conninfo. It shows the socket location along with other details
unix_socket_directory value in postgresql.conf
In my Ubuntu distro, /var/run is a soft link to /run, that's why /run/postgresql works even though unix_socket_directory and \conninfo mention /var/run/postgresql.