Executed create index concurrently statmt & it was disconnected session due to timeout error.but I do see in pg_stat_activity,it is in running state - postgresql

I have executed create an index on a big table from pgAdmin, and in a while, I lost connection to the server, so the execution window closed in pgAdnin. Then I reconnected to the server, and when checked pg_stat_activity, I do see that the create index statement is running (active) state, I just wondering to know whether this index being creating or stuck somewhere?
client disconnected with error,
cancelling statement due to statement timeout
when I reconnected to the server, in pg_stat_activity.
31937 "edsadmin" "09:54:44.280176" "CREATE INDEX CONCURRENTLY idx_src_record_date
ON pcd_t.l_esd_detail_report USING btree
(src_record_date COLLATE pg_catalog."default")
TABLESPACE pg_default;"
I'm really confused here wheather it is createing or not.

Late answer, but relevant none-the-less.
Based on my anecdotal experience right now, the index creation persists beyond the lifetime of the client.
I did the following:
database=> CREATE INDEX CONCURRENTLY IX_myIndex on table(column, column2);
It paused for a few minutes, then:
SSL SYSCALL error: EOF detected
The connection to the server was lost. Attempting reset: Succeeded.
psql (14.2 (Debian 14.2-1.pgdg110+1), server 10.18)
Immediately after reconnecting, I ran:
database=> \d table
Table "public.table"
Column | Type | Collation | Nullable |
Default
-----------------+-----------------------------+-----------+----------+---------
... blah blah blah ...
Indexes:
"IX_myIndex" btree (column, column2) INVALID
I waited a few more minutes, then checked again:
database=> \d table
Table "public.table"
Column | Type | Collation | Nullable |
Default
-----------------+-----------------------------+-----------+----------+---------
... blah blah blah ...
Indexes:
"IX_myIndex" btree (column, column2)
So, your index is likely fine.

Related

postgres database logical replication: ERROR: duplicate key value violates unique constraint "users_unique_key"

i am trying to set up master/slave postgres replication.
i have 2 servers, master with populated data ,i did complete backup of the database i want to replicate and restored it in slave, then i configured the replication with (copy_data=false) . when insert a new row in any table, it works fine, but when i do update an existing row, i get the error:
022-03-16 22:29:08.708 UTC [625449] ERROR: duplicate key value violates unique constraint "users_unique_key"
2022-03-16 22:29:08.708 UTC [625449] DETAIL: Key (userid, classid)=(4556, 4507) already exists.
2022-03-16 22:29:08.711 UTC [625285] LOG: background worker "logical replication worker" (PID 625449) exited with exit code 1
The table has primary key and set to "not null". My question is , how do i setup logical replication, with UPDATE statement possible ?

Why is id as SERIAL discontinuous values after failover in RDS Aurora PostgreSQL?

I'm testing failover using RDS Aurora PostgreSQL.
First, create RDS Aurora PostgreSQL and access the writer cluster to create users table.
$ CREATE TABLE users (
id SERIAL PRIMARY KEY NOT NULL,
name varchar(10) NOT NULL,
createAt TIMESTAMP DEFAULT Now() );
And I added one row and checked the table.
$ INSERT INTO users(name) VALUES ('test');
$ SELECT * FROM users;
+----+--------+----------------------------+
| id | name | createdAt |
+----+--------+----------------------------+
| 1 | test | 2022-02-02 23:09:57.047981 |
+----+--------+----------------------------+
After failover of RDS Aurora Cluster, I added another row and checked the table.
$ INSERT INTO users(name) VALUES ('temp');
$ SELECT * FROM users;
+-----+--------+----------------------------+
| id | name | createdAt |
+-----+--------+----------------------------+
| 1 | test | 2022-02-01 11:09:57.047981 |
| 32 | temp | 2022-02-01 11:25:57.047981 |
+-----+--------+----------------------------+
After failover, the id value that should be 2 became 32.
Why is this happening?
Is there any way to solve this problem?
That is to be expected. Index modifications are not WAL logged whenever nextval is called, because that could become a performance bottleneck. Rather, a WAL record is written every 32 calls. That means that the sequence can skip some values after a crash or failover to the standby.
You may want to read my ruminations about gaps in sequences.

Why is liquibase deleting databasechangelog rows and trying to create a renamed database table?

I am using postgres 10.5 and liquibase 3.6.2 on a Mac.
I nuke & re-create my database, run liquibase update, and it works.
But a second liquibase update fails with an exception that the pkey already exists.
After the first liquibase update, the databasechangelog table contains 97 entries. After the second, it contains 10, and the time and deployment ids for those are different than they were after the first update!
Table foo was created in an early change.
Later it was changed to be named bar, but the pkey is still foo.pkey.
Liquibase-update should not be trying to re-create foo, but it does, and fails because foo.pkey already exists.
A) In general, how can I get liquibase to output more info about what it's doing? I tried both of the commands:
liquibase --logLevel=debug --logFile=`pwd`/foo.log update
liquibase --logLevel debug --logFile `pwd`/foo.log update
Both seem to work the same, and foo.log isn't created and there's no more output in the terminal.
B) How can I stop liquibase from trying to re-make this and nuking my databasechangelog?
I tried to make a small example that fails, but this seems to work... Others here are using it with postgres 9.5.10 with no problem...
All I see in the terminal is:
Starting Liquibase at Wed, 14 Nov 2018 13:06:44 PST (version 3.6.2 built at 2018-07-03 11:28:09)
Unexpected error running Liquibase: ERROR: relation "cant_change_pkey" already exists [Failed SQL: CREATE TABLE nuss.cant_change (message_id UUID NOT NULL, origin VARCHAR(4), type VARCHAR(12) NOT NULL, CONSTRAINT CANT_CHANGE_PKEY PRIMARY KEY (message_id), UNIQUE (message_id))]
liquibase.exception.MigrationFailedException: Migration failed for change set db/changelog/changelog-new1.xml::first-one::rstrauss:
Reason: liquibase.exception.DatabaseException: ERROR: relation "cant_change_pkey" already exists [Failed SQL: CREATE TABLE nuss.cant_change (message_id UUID NOT NULL, origin VARCHAR(4), type VARCHAR(12) NOT NULL, CONSTRAINT CANT_CHANGE_PKEY PRIMARY KEY (message_id), UNIQUE (message_id))]
at liquibase.changelog.ChangeSet.execute(ChangeSet.java:637)
at liquibase.changelog.visitor.UpdateVisitor.visit(UpdateVisitor.java:53)
at liquibase.changelog.ChangeLogIterator.run(ChangeLogIterator.java:78)
at liquibase.Liquibase.update(Liquibase.java:202)
at liquibase.Liquibase.update(Liquibase.java:179)
at liquibase.integration.commandline.Main.doMigration(Main.java:1205)
at liquibase.integration.commandline.Main.run(Main.java:191)
at liquibase.integration.commandline.Main.main(Main.java:129)
Caused by: liquibase.exception.DatabaseException: ERROR: relation "cant_change_pkey" already exists [Failed SQL: CREATE TABLE nuss.cant_change (message_id UUID NOT NULL, origin VARCHAR(4), type VARCHAR(12) NOT NULL, CONSTRAINT CANT_CHANGE_PKEY PRIMARY KEY (message_id), UNIQUE (message_id))]
at liquibase.executor.jvm.JdbcExecutor$ExecuteStatementCallback.doInStatement(JdbcExecutor.java:356)
at liquibase.executor.jvm.JdbcExecutor.execute(JdbcExecutor.java:57)
at liquibase.executor.jvm.JdbcExecutor.execute(JdbcExecutor.java:125)
at liquibase.database.AbstractJdbcDatabase.execute(AbstractJdbcDatabase.java:1229)
at liquibase.database.AbstractJdbcDatabase.executeStatements(AbstractJdbcDatabase.java:1211)
at liquibase.changelog.ChangeSet.execute(ChangeSet.java:600)
... 7 common frames omitted
Caused by: org.postgresql.util.PSQLException: ERROR: relation "cant_change_pkey" already exists
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2476)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2189)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:300)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:428)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:354)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:301)
at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:287)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:264)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:260)
at liquibase.executor.jvm.JdbcExecutor$ExecuteStatementCallback.doInStatement(JdbcExecutor.java:352)
... 12 common frames omitted
For more information, please use the --logLevel flag

Cannot execute SQL query on streaming replication setup master server Postgresql 9.5

Error:
postgres=# insert into company values(4,'tom',21,'pune' ,21 );
^CCancel request sent
WARNING: canceling wait for synchronous replication due to user request
DETAIL: The transaction has already committed locally, but might not have been replicated to the standby.
INSERT 0 1
Even after the error it is executing the query on the master as well as replicating transactions the slave.
On Master:
postgres=# SELECT pg_current_xlog_location();
pg_current_xlog_location
--------------------------
0/1900D0C0
(1 row)
On Slave:
postgres=# SELECT pg_last_xlog_receive_location();
pg_last_xlog_receive_location
-------------------------------
0/1900D0C0
(1 row)
synchronous_standby_name set on master from config file(set by me) is different from the application_name I see it on record of pg_stats_replication table. Many of the solutions have suggested to change the application name. However, I am not sure from where it is taking application name as walreceiver on the master.
On Master:
postgres=# select application_name, sync_state from pg_stat_replication;
application_name | sync_state
------------------+------------
walreceiver | async
(1 row)
postgres=# show synchronous_standby_names;
synchronous_standby_names
---------------------------
slave1
(1 row)
postgres=# show synchronous_commit;
synchronous_commit
--------------------
on
(1 row)
One of the solution I found is to create tablespace dir under path '/var/lib/pgsql/9.2/data/' which I currently dont have. I am not sure if that solution will work for 9.5 Postgresql.
Any help on this is appreciated. Thank you.
Changing synchronous_standby_names to 'walreceiver' resolved the error. Followed link To know more about synchronous_standby_names

Postgres - How to debug/trace 'Idle in transaction' connection

I am using Postgres for one of my applications and sometimes (not very frequently) one of the connection goes into <IDLE> in transaction state and it keeps acquired lock that causes other connections to wait on these locks ultimately causing my application to hang.
Following is the output from pg_stat_activity table for that process:
select * from pg_stat_activity
24081 | db | 798 | 16384 | db | | 10.112.61.218 | | 59034 | 2013-09-12 23:46:05.132267+00 | 2013-09-12 23:47:31.763084+00 | 2013-09-12 23:47:31.763534+00 | f | <IDLE> in transaction
This indicates that PID=798 is in <IDLE> in transaction state. The client process on web server is found as following using the client_port (59034) from above output.
sudo netstat -apl | grep 59034
tcp 0 0 ip-10-112-61-218.:59034 db-server:postgresql ESTABLISHED 23843/pgbouncer
I know that something is wrong in my application code (I killed one of the running application cron and it freed the locks) that is causing the connection to hang, but I am not able to trace it.
This is not very frequent and I can't find any definite reproduction steps either as this only occurs on the production server.
I would like to get inputs on how to trace such idle connection, e.g. getting last executed query or some kind of trace-back to identify which part of code is causing this issue.
If you upgrade to 9.2 or higher, the pg_stat_activity view will show you what the most recent query executed was for idle in transaction connections.
select * from pg_stat_activity \x\g\x
...
waiting | f
state | idle in transaction
query | select count(*) from pg_class ;
You can also (even in 9.1) look in pg_locks to see what locks are being held by the idle in transaction process. If it only has locks on very commonly used objects, this might not narrow things down much, but if it was a peculiar lock that could tell you exactly where in your code to look.
If you are stuck with 9.1, you can perhaps use the debugger to get all but the first 22 characters of the query (the first 22 are overwritten by the <IDLE> in transaction\0 message). For example:
(gdb) printf "%s\n", ((MyBEEntry->st_activity)+22)