I am enabling postgresSQL statement log by SET log_statement = 'all'; and received output like below for an Java application that is using Hibernate. Default log format is used and the Postgres version is 13.1
I can guess the number in square bracket is threadID? I see that there is multiple BEGIN and multiple COMMIT scattered without explicit ID or order
Is there a way I can see the scope of a transaction from this log? (Actual statement and values are removed)
2021-11-08 05:45:52.827 UTC [107] LOG: execute S_4: BEGIN
2021-11-08 05:45:52.841 UTC [107] LOG: execute <unnamed>:
RETURNING *
2021-11-08 05:45:52.841 UTC [107] DETAIL: parameters:
2021-11-08 05:45:52.927 UTC [107] LOG: execute <unnamed>:
RETURNING *
2021-11-08 05:45:52.927 UTC [107] DETAIL: parameters:
2021-11-08 05:45:52.975 UTC [107] LOG: execute <unnamed>:
RETURNING *
2021-11-08 05:45:52.975 UTC [107] DETAIL: parameters:
2021-11-08 05:45:54.209 UTC [107] LOG: execute <unnamed>:
2021-11-08 05:45:54.209 UTC [107] DETAIL: parameters:
2021-11-08 05:45:54.251 UTC [107] LOG: execute <unnamed>:
2021-11-08 05:45:54.251 UTC [107] DETAIL: parameters:
2021-11-08 05:45:54.297 UTC [107] LOG: execute <unnamed>:
2021-11-08 05:45:54.297 UTC [107] DETAIL: parameters:
2021-11-08 05:45:54.565 UTC [107] LOG: execute <unnamed>:
2021-11-08 05:45:54.565 UTC [107] DETAIL: parameters:
2021-11-08 05:45:55.164 UTC [194] LOG: execute <unnamed>: SET extra_float_digits = 3
2021-11-08 05:45:55.209 UTC [194] LOG: execute <unnamed>: SET application_name = 'PostgreSQL JDBC Driver'
2021-11-08 05:45:55.253 UTC [194] LOG: execute <unnamed>: BEGIN
2021-11-08 05:45:55.257 UTC [194] LOG: execute <unnamed>:
2021-11-08 05:45:55.257 UTC [194] DETAIL:
2021-11-08 05:45:55.308 UTC [194] LOG: execute <unnamed>:
2021-11-08 05:45:55.308 UTC [194] DETAIL: parameters:
2021-11-08 05:45:55.354 UTC [194] LOG: execute <unnamed>:
RETURNING *
2021-11-08 05:45:55.354 UTC [194] DETAIL: parameters:
2021-11-08 05:45:55.412 UTC [195] LOG: execute <unnamed>: SET extra_float_digits = 3
2021-11-08 05:45:55.451 UTC [195] LOG: execute <unnamed>: SET application_name = 'PostgreSQL JDBC Driver'
2021-11-08 05:45:55.577 UTC [194] LOG: execute <unnamed>:
2021-11-08 05:45:55.577 UTC [194] DETAIL:
2021-11-08 05:45:55.617 UTC [194] LOG: execute S_1: COMMIT
2021-11-08 05:45:56.393 UTC [196] LOG: execute <unnamed>: SET extra_float_digits = 3
2021-11-08 05:45:56.435 UTC [196] LOG: execute <unnamed>: SET application_name = 'PostgreSQL JDBC Driver'
2021-11-08 05:45:56.486 UTC [196] LOG: execute <unnamed>:
2021-11-08 05:45:56.486 UTC [196] DETAIL: parameters:
2021-11-08 05:45:56.529 UTC [196] LOG: execute <unnamed>:
2021-11-08 05:45:56.529 UTC [196] DETAIL: parameters:
2021-11-08 05:45:56.571 UTC [196] LOG: execute <unnamed>: BEGIN
2021-11-08 05:45:56.572 UTC [196] LOG: execute <unnamed>:
RETURNING *
2021-11-08 05:45:56.572 UTC [196] DETAIL: parameters:
2021-11-08 05:45:56.622 UTC [196] LOG: execute S_1: COMMIT
2021-11-08 05:45:56.650 UTC [197] LOG: execute <unnamed>: SET extra_float_digits = 3
2021-11-08 05:45:56.677 UTC [107] LOG: execute <unnamed>:
2021-11-08 05:45:56.677 UTC [107] DETAIL: parameters:
2021-11-08 05:45:56.690 UTC [197] LOG: execute <unnamed>: SET application_name = 'PostgreSQL JDBC Driver'
2021-11-08 05:45:56.724 UTC [107] LOG: execute <unnamed>:
2021-11-08 05:45:56.724 UTC [107] DETAIL: parameters:
2021-11-08 05:45:58.603 UTC [196] LOG: execute <unnamed>: BEGIN READ ONLY
2021-11-08 05:45:58.604 UTC [196] LOG: execute <unnamed>:
2021-11-08 05:45:58.604 UTC [196] DETAIL: parameters:
2021-11-08 05:45:58.653 UTC [196] LOG: execute <unnamed>:
2021-11-08 05:45:58.653 UTC [196] DETAIL: parameters:
2021-11-08 05:45:58.710 UTC [196] LOG: execute <unnamed>:
2021-11-08 05:45:58.710 UTC [196] DETAIL: parameters:
2021-11-08 05:45:58.751 UTC [196] LOG: execute S_1: COMMIT
2021-11-08 05:45:58.792 UTC [196] LOG: execute <unnamed>: BEGIN READ ONLY
2021-11-08 05:45:58.792 UTC [196] LOG: execute <unnamed>:
2021-11-08 05:45:58.792 UTC [196] DETAIL: parameters:
2021-11-08 05:45:58.836 UTC [196] LOG: execute S_1: COMMIT
2021-11-08 05:45:58.880 UTC [196] LOG: execute <unnamed>: BEGIN READ ONLY
2021-11-08 05:45:58.881 UTC [196] LOG: execute <unnamed>:
2021-11-08 05:45:58.881 UTC [196] DETAIL: parameters:
2021-11-08 05:45:58.924 UTC [196] LOG: execute S_1: COMMIT
2021-11-08 05:45:58.965 UTC [196] LOG: execute <unnamed>:
2021-11-08 05:45:58.965 UTC [196] DETAIL: parameters:
2021-11-08 05:45:59.008 UTC [196] LOG: execute <unnamed>:
2021-11-08 05:45:59.008 UTC [196] DETAIL: parameters:
2021-11-08 05:45:59.192 UTC [107] LOG: execute S_1: COMMIT
According to Postgres document in log_line_prefix, You can set how Postgres print log for example log data, process id, username, etc
You can use %x for log transaction-id
Related
I have elk(elasticsearch, logstash , kibana) for monitor logs. postgresql logs with filebeat send to logstash and show logs in kibana.
I enabled log_duration in postgresql.conf and logged correctly.
for example, the postgresql log is below :
2023-01-11 06:17:09.754 EST [19751] user#books LOG: duration: 0.014 ms execute <unnamed>: SET SESSION CHARACTERISTICS AS TRANSACTION READ ONLY
2023-01-11 06:17:09.755 EST [19751] user#books LOG: duration: 0.016 ms bind S_1: BEGIN
2023-01-11 06:17:09.755 EST [19751] user#books LOG: duration: 0.006 ms execute S_1: BEGIN
2023-01-11 06:17:09.756 EST [19751] user#books LOG: duration: 0.488 ms parse <unnamed>: select * from books
but in kibana, The duration value of the field is not separate and it is displayed as one with the message field, and there is no possibility of aggregation and other opetaions.
how to extract and split duration value from message ???
All other database operation are running at full speed.
This on several hosts and Ubuntu 22.04 LXC containers on that we have been using otherwise very successfully for a couple of years.
Turning fsync off doesnt make any difference.
diskio and processor utilisation is minimal so that isnt it.
I tried logging with debug level 3 but could not find anything.
Listing the various process logs in sql and linux just shows processes quietly waiting for something but I cannot find out what exactly.
The template1 database is virtually empty.
2022-10-31 14:10:02.743 UTC [1086558] exodus#exodus LOG: duration: 17249.532 ms statement: CREATE DATABASE xo_dict WITH ENCODING='UTF8'
2022-10-31 14:10:11.569 UTC [1090734] exodus#exodus LOG: duration: 8010.033 ms statement: DROP DATABASE exodus2b
2022-10-31 14:10:13.359 UTC [1086558] exodus#exodus LOG: duration: 9596.890 ms statement: DROP DATABASE xo_dict
2022-10-31 14:10:15.076 UTC [1090734] exodus#exodus LOG: duration: 3491.147 ms statement: DROP DATABASE exodus3b
2022-10-31 14:10:32.291 UTC [1093962] exodus#exodus LOG: duration: 15510.507 ms statement: CREATE DATABASE exodus2b WITH ENCODING='UTF8'
2022-10-31 14:10:52.174 UTC [1093962] exodus#exodus LOG: duration: 19864.597 ms statement: CREATE DATABASE exodus3b WITH ENCODING='UTF8' TEMPLATE exodus2b
2022-10-31 14:10:52.932 UTC [1093962] exodus#exodus LOG: duration: 740.990 ms statement: DROP DATABASE exodus2b
2022-10-31 14:10:55.849 UTC [1093962] exodus#exodus LOG: duration: 2129.943 ms statement: DROP DATABASE exodus3b
2022-10-31 14:11:13.755 UTC [1102944] exodus#exodus LOG: duration: 17885.511 ms statement: CREATE DATABASE exodus2b WITH ENCODING='UTF8'
2022-10-31 14:11:43.537 UTC [1102944] exodus#exodus LOG: duration: 29769.648 ms statement: CREATE DATABASE exodus3b WITH ENCODING='UTF8'
2022-10-31 14:21:33.410 UTC [1247048] exodus#exodus LOG: duration: 15115.960 ms statement: CREATE DATABASE xo_dict WITH ENCODING='UTF8'
I may be encountering the same issue described in Horizon: does not exit if database connection fails #898 (https://github.com/stellar/go/issues/898) but with a different set up scenario.
I am in the process of migrating https://github.com/satoshipay/docker-stellar-horizon Docker Compose definitions to Kubernetes. I have been able to migrate most of the set up but hitting a problem with Horizon where the DB is not getting created during startup. I believe I have stellar core with the dependency on Postgres working as designed and the DB created as part of startup but the set up is different for Horizon.
The current issue I am hitting is the following...
Horizon Server Pod Logs
todkapmcbookpro:kubernetes todd$ kubectl get pods
NAME READY STATUS RESTARTS AGE
postgres-horizon-564d479db4-2xvqd 1/1 Running 0 20m
postgres-sc-9f5f7fb4-prlpr 1/1 Running 0 22m
stellar-core-7ff77b4db8-tx4mt 1/1 Running 0 18m
stellar-horizon-6cff98554b-d7djn 0/1 CrashLoopBackOff 8 18m
todkapmcbookpro:kubernetes todd$ kubectl logs stellar-horizon-6cff98554b-d7djn
Initializing Horizon database...
2019/05/02 12:58:09 connect failed: pq: database "stellar-horizon" does not exist
Horizon database initialization failed (possibly because it has been done before)
2019/05/02 12:58:09 pq: database "stellar-horizon" does not exist
todkapmcbookpro:kubernetes todd$
Horizon Postgres DB pod logs
todkapmcbookpro:kubernetes todd$ kubectl logs postgres-horizon-564d479db4-2xvqd
2019-05-02 12:40:06.424 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2019-05-02 12:40:06.424 UTC [1] LOG: listening on IPv6 address "::", port 5432
2019-05-02 12:40:06.437 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2019-05-02 12:40:06.444 UTC [23] LOG: database system was interrupted; last known up at 2019-05-02 12:38:19 UTC
2019-05-02 12:40:06.453 UTC [23] LOG: database system was not properly shut down; automatic recovery in progress
2019-05-02 12:40:06.454 UTC [23] LOG: redo starts at 0/1636FB8
2019-05-02 12:40:06.454 UTC [23] LOG: invalid record length at 0/1636FF0: wanted 24, got 0
2019-05-02 12:40:06.454 UTC [23] LOG: redo done at 0/1636FB8
2019-05-02 12:40:06.459 UTC [1] LOG: database system is ready to accept connections
2019-05-02 12:42:35.675 UTC [30] FATAL: database "stellar-horizon" does not exist
2019-05-02 12:42:35.690 UTC [31] FATAL: database "stellar-horizon" does not exist
2019-05-02 12:42:37.123 UTC [32] FATAL: database "stellar-horizon" does not exist
2019-05-02 12:42:37.136 UTC [33] FATAL: database "stellar-horizon" does not exist
2019-05-02 12:42:50.131 UTC [34] FATAL: database "stellar-horizon" does not exist
2019-05-02 12:42:50.153 UTC [35] FATAL: database "stellar-horizon" does not exist
2019-05-02 12:43:16.094 UTC [36] FATAL: database "stellar-horizon" does not exist
2019-05-02 12:43:16.115 UTC [37] FATAL: database "stellar-horizon" does not exist
2019-05-02 12:43:57.097 UTC [38] FATAL: database "stellar-horizon" does not exist
2019-05-02 12:43:57.111 UTC [39] FATAL: database "stellar-horizon" does not exist
2019-05-02 12:45:21.050 UTC [40] FATAL: database "stellar-horizon" does not exist
2019-05-02 12:45:21.069 UTC [41] FATAL: database "stellar-horizon" does not exist
2019-05-02 12:48:05.122 UTC [42] FATAL: database "stellar-horizon" does not exist
2019-05-02 12:48:05.145 UTC [43] FATAL: database "stellar-horizon" does not exist
2019-05-02 12:53:07.077 UTC [44] FATAL: database "stellar-horizon" does not exist
2019-05-02 12:53:07.099 UTC [45] FATAL: database "stellar-horizon" does not exist
2019-05-02 12:58:09.084 UTC [46] FATAL: database "stellar-horizon" does not exist
2019-05-02 12:58:09.098 UTC [47] FATAL: database "stellar-horizon" does not exist
2019-05-02 13:03:18.055 UTC [48] FATAL: database "stellar-horizon" does not exist
2019-05-02 13:03:18.071 UTC [49] FATAL: database "stellar-horizon" does not exist
2019-05-02 13:08:28.057 UTC [50] FATAL: database "stellar-horizon" does not exist
2019-05-02 13:08:28.078 UTC [51] FATAL: database "stellar-horizon" does not exist
2019-05-02 13:13:42.071 UTC [52] FATAL: database "stellar-horizon" does not exist
2019-05-02 13:13:42.097 UTC [53] FATAL: database "stellar-horizon" does not exist
2019-05-02 13:18:55.128 UTC [54] FATAL: database "stellar-horizon" does not exist
2019-05-02 13:18:55.152 UTC [55] FATAL: database "stellar-horizon" does not exist
It would be ideal if the setup for Horizon and Core were the same (especially as it relates to the DB configuration env properties). I think I have the settings correct but may be missing something subtle.
I have a branch of this WIP where the failure occurs. I have included a quick set up script as well as a minikube set up in this branch.
https://github.com/todkap/stellar-testnet/tree/k8-deploy/kubernetes
We were able to resolve and published an article demonstrating the end to end flow.
https://itnext.io/how-to-deploy-a-stellar-validator-on-kubernetes-with-helm-a111e5dfe437
Postgresql wont start in ubuntu 16.04,
Please give support.
These are the error log read:
2018-10-12 13:42:34 UTC [3371-2] LOG: received fast shutdown request
2018-10-12 13:42:34 UTC [3371-3] LOG: aborting any active transactions
2018-10-12 13:42:34 UTC [3376-2] LOG: autovacuum launcher shutting down
2018-10-12 13:42:34 UTC [3373-1] LOG: shutting down
2018-10-12 13:42:34 UTC [3373-2] LOG: database system is shut down
2018-10-12 13:42:52 UTC [3855-1] LOG: database system was shut down at
2018-10-12 13:42:34 UTC
2018-10-12 13:42:52 UTC [3855-2] LOG: MultiXact member wraparound
protections are now enabled
2018-10-12 13:42:52 UTC [3853-1] LOG: database system is ready to accept
connections
2018-10-12 13:42:52 UTC [3859-1] LOG: autovacuum launcher started
2018-10-12 13:42:52 UTC [3861-1] [unknown]#[unknown] LOG: incomplete
startup
packet
I'm running a postgres server locally on my computer and it seems that even the simple queries like the one below is giving me an EOF detected error.
For instance, this query
ALTER TABLE maintab ADD COLUMN testing numeric;
UPDATE maintab SET testing = numeric1 * numeric2;
And similar activities will throw an EOF error. I'm also running PostGIS with QGIS and my spatial queries, no matter how simple, will throw this error.
I've look around at forums and documentation but nothing can seem to help solve this problem. Is there anything I can do to stop this?
EDIT
I ran a check on my error logs after doing some Googling. Found these logs, not sure what to make of them
2015-09-04 11:18:31 EDT [1138-4] LOG: terminating any other active server processes
2015-09-04 11:18:31 EDT [1208-3] WARNING: terminating connection because of crash of another server process
2015-09-04 11:18:31 EDT [1208-4] DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2015-09-04 11:18:31 EDT [1208-5] HINT: In a moment you should be able to reconnect to the database and repeat your command.
2015-09-04 11:18:31 EDT [1138-5] LOG: all server processes terminated; reinitializing
2015-09-04 11:18:31 EDT [3861-1] LOG: database system was interrupted; last known up at 2015-09-04 15:08:49 EDT
2015-09-04 11:18:32 EDT [3861-2] LOG: database system was not properly shut down; automatic recovery in progress
2015-09-04 11:18:32 EDT [3861-3] LOG: record with zero length at 1D/123A250
2015-09-04 11:18:32 EDT [3861-4] LOG: redo is not required
2015-09-04 11:18:32 EDT [3861-5] LOG: MultiXact member wraparound protections are now enabled
2015-09-04 11:18:32 EDT [1138-6] LOG: database system is ready to accept connections
2015-09-04 11:18:32 EDT [3865-1] LOG: autovacuum launcher started
2015-09-04 16:07:22 EDT [1122-1] LOG: database system was interrupted; last known up at 2015-09-04 16:06:25 EDT
2015-09-04 16:07:22 EDT [1179-1] [unknown]#[unknown] LOG: incomplete startup packet
2015-09-04 16:07:23 EDT [1122-2] LOG: database system was not properly shut down; automatic recovery in progress
2015-09-04 16:07:23 EDT [1122-3] LOG: record with zero length at 1D/123A320
2015-09-04 16:07:23 EDT [1122-4] LOG: redo is not required
2015-09-04 16:07:23 EDT [1122-5] LOG: MultiXact member wraparound protections are now enabled
2015-09-04 16:07:23 EDT [1114-1] LOG: database system is ready to accept connections
2015-09-04 16:07:23 EDT [1183-1] LOG: autovacuum launcher started
2015-09-04 12:15:05 EDT [1183-2] LOG: stats collector's time 2015-09-04 16:07:23.363257-04 is later than backend local time 2015-09-04 12:15:05.07308-04
2015-09-04 12:17:34 EDT [1114-2] LOG: server process (PID 3824) was terminated by signal 11: Segmentation fault
2015-09-04 12:17:34 EDT [1114-4] LOG: terminating any other active server processes
2015-09-04 12:17:34 EDT [1183-3] WARNING: terminating connection because of crash of another server process
2015-09-04 12:17:34 EDT [1183-4] DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2015-09-04 12:17:34 EDT [1183-5] HINT: In a moment you should be able to reconnect to the database and repeat your command.
2015-09-04 12:17:34 EDT [1114-5] LOG: all server processes terminated; reinitializing
2015-09-04 12:17:34 EDT [3828-1] LOG: database system was interrupted; last known up at 2015-09-04 16:07:23 EDT
2015-09-04 12:17:35 EDT [3828-2] LOG: database system was not properly shut down; automatic recovery in progress
2015-09-04 12:17:35 EDT [3828-3] LOG: redo starts at 1D/123A388
2015-09-04 12:17:35 EDT [3828-4] LOG: unexpected pageaddr 1C/F9258000 in log segment 000000010000001D00000001, offset 2457600
2015-09-04 12:17:35 EDT [3828-5] LOG: redo done at 1D/1255C18
2015-09-04 12:17:36 EDT [3828-6] LOG: MultiXact member wraparound protections are now enabled
2015-09-04 12:17:36 EDT [3833-1] LOG: autovacuum launcher started
2015-09-04 12:17:36 EDT [1114-6] LOG: database system is ready to accept connections