I have a simple process that is reading logical replication messages from postgres. This process runs every second and generates a lot of messages in the postgres logs like:
2021-02-15 20:35:11.032 UTC [35] STATEMENT: SELECT * FROM pg_logical_slot_get_changes('lazy_cloud', NULL, NULL);
2021-02-15 20:35:11.032 UTC [35] LOG: logical decoding found consistent point at 0/167C618
2021-02-15 20:35:11.032 UTC [35] DETAIL: There are no running transactions.
I've configured logging with the following settings:
log_min_messages=ERROR
log_statement=none
log_replication_commands=0
But, the logical replication logs are still produced.
Is there a setting to disable these messages? I can use sed or something like that, but would prefer a built in solution.
There is no way to disable that message short of setting
log_min_messages = fatal
in postgresql.conf, but that is not a smart idea, because then you'd miss out on all error messages in the log file and essentially disable logging.
Related
Sometimes when the zabbix monitor executes the sql "select value from system.asynchronous_metrics where metric='ReplicasMaxAbsoluteDelay';", it will return a very large value like "52y 9m 18d" (currenttime - 1970).
According to the getAbsoluteDelay() function, we knew when queue_update_finish_time is null, this function will return current_time.
But we want to analyze when and why queue_update_finish_time is null, what can we check from clickhouse system table or log ? We've checked the error log and zookeeper log, but it seems there is no infomation about this problem.
Thanks.
Usually it means table is out-of-sync and gone in read only mode.
Read-only mode can be caused by:
some zookeeper synchronization issue
out of disk storage
stale metadata (replica had failed to apply some alter)
...
To get more information try look into system.replicas table:
SELECT is_readonly, zookeeper_exception
FROM system.replicas
WHERE table = 'appsflyer_events_hist'
In most cases it helps to run:
SYSTEM RESTART REPLICA db.table
https://clickhouse.com/docs/en/sql-reference/statements/system/#restart-replica
Trying to achieve sync streaming to barman server and i need to add an entry to postgresql.conf for this parameter, which already has an entry and tried a few variations but does not work. Any ideas? Also tried '&&' but in vain
synchronous_standby_names='ANY 1 (*)',barman-wal-archive
2022-06-10 16:50:54.272 BST [11241-43] # app= LOG: syntax error in
file "/var/lib/pgsql/13/data/postgresql.conf" line 22, near token ","
2022-06-10 16:50:54.272 BST [11241-44] # app= LOG: configuration file
"/var/lib/pgsql/13/data/postgresql.conf" contains errors; no changes
were applied
The syntax you are using is not valid, and you won't be able to specify that Barman should be kept synchronous and any one of the others. The best you can do is
synchronous_standby_names = 'FIRST 2 ("barman-wal-archive", standby1, standby2, standby3)'
(You have to double quote all names that are not standard SQL identifiers, for example if they contain -.)
Then PostgreSQL will always keep Barman synchronized, as well as the first available standby server. But that won't have transactions fail if Barman is not available, which seems to be what you want.
Keep just
synchronous_standby_names='ANY 1 (*)'
and set
synchronous_commit = on
or
synchronous_commit = remote_write
Im running a workflow in powercenter that is constatnly getting an SQL1224N error.
This process execute a query against one table (POLIZA) with 800k rows, it retrieves the first 10k rows and then it start to execute to another table with 75M rows, at ths moment in DB2 an idle thread error appear but the PWC process still running retrieving the 75M rows, when it is completed (after 20 minutes) the errros comes up related with the first table:
[IBM][CLI Driver] SQL1224N A database agent could not be started to service a request, or was terminated as a result of a database system shutdown or a force command. SQLSTATE=55032
sqlstate = 40003
[IBM][CLI Driver] SQL1224N A database agent could not be started to service a request, or was terminated as a result of a database system shutdown or a force command. SQLSTATE=55032
sqlstate = 40003
Database driver error...
Function Name : Fetch
SQL Stmt : SELECT POLIZA.BSPOL_BSCODCIA, POLIZA.BSPOL_BSRAMOCO
FROM POLIZA
WHERE
EXA01.POLIZA.BSPOL_IDEMPR='0015' for read only with ur
Native error code = -1224
DB2 Fatal Error].
I have a similar process runing against the same 2 tables and it is woking fine where the only difference I can see is that the DB2 user is different.
Any idea how can i fix this?
Regards
The common causes for -1224 are:
Your instance or database has crashed, or
Something/somebody is forcing off your application (FORCE APPLICATION or equivalent)
As for the crash, I think you would know by know. This typically requires a database or instance restart. At any rate, can you please have a look into your DIAGPATH to check for any FODC* directories whose timestamp would match the timestamp of the -1224 errors?
As for the FORCE case, you should find some evidence of the -1224 in db2diag.log. Try searching for the decimal -1224, but also for its hex representation (0xFFFFFB38).
My postgres crashes for long query. It's on Debian 7 64bit, and postgresql-9.3.2. I uses all default configuration. Could anyone suggest what problem it could be? thanks.
--part1:
SELECT r1.f2 as b, r1.e as l
FROM r r8,r r7,r r6,r r5,r r4,r r3,r r2,r r1
WHERE
r1.f2=r2.f1 AND
r1.f2=r3.f1 AND
r1.f2=r4.f1 AND
r1.f1=r5.f2 AND
r1.f1=r8.f1 AND
r2.f1=r3.f1 AND
r2.f1=r4.f1 AND
r2.f2=r6.f2 AND
r2.f2=r7.f1 AND
r3.f1=r4.f1 AND
r3.f2=r7.f2 AND
r3.f2=r8.f2 AND
r4.f2=r5.f1 AND
r4.f2=r6.f1 AND
r5.f1=r6.f1 AND
r5.f2=r8.f1 AND
r6.f2=r7.f1 AND
r7.f2=r8.f2 AND
r1.d=1 AND
r2.d=1 AND
r3.d=2 AND
r4.d=2 AND
r5.d=2 AND
r6.d=2 AND
r7.d=2 AND
r8.d=2
-- part2
group by r1.f2,r1.e
having
calc_empty_a() AND
calc_empty_b();
In the query, calc_empty_a() are just empty boolean functions (return true), so they should have no problem.
If I run the query in client, the server crashes. There is nothing useful information in the log (please refer to the error info at end of the post).
If I run the part 1 query, the query works well.
If I first run the part 1 query, then I run the whole query, it works well.
If I change the query, reduce the r numbers, for example, there are only r1 to r6 FROM tables, delete the predicates with r8, r7, but keep the part 2's GROUP BY and HAVING clause. The query still works well.
If the query have one empty function in HVING clause, the query also works well, but will crash if there are two functions.
The following query works well
SELECT r1.f2 as b, r1.f1 as a , r1.e as e
FROM r r8,r r7,r r6,r r5,r r4,r r3,r r2,r r1
WHERE
r1.f2=r2.f1 AND
r1.f2=r3.f1 AND
r1.f2=r4.f1 AND
r1.f1=r5.f2 AND
r1.f1=r8.f1 AND
r2.f1=r3.f1 AND
r2.f1=r4.f1 AND
r2.f2=r6.f2 AND
r2.f2=r7.f1 AND
r3.f1=r4.f1 AND
r3.f2=r7.f2 AND
r3.f2=r8.f2 AND
r4.f2=r5.f1 AND
r4.f2=r6.f1 AND
r5.f1=r6.f1 AND
r5.f2=r8.f1 AND
r6.f2=r7.f1 AND
r7.f2=r8.f2
group by r1.f2,r1.f1, r1.e
having
calc_empty_a() AND
calc_empty_a();
I have set the os to use strict overcommit mode:
sysctl -w vm.overcommit_memory=2
Error info:
at client
The connection to the server was lost. Attempting reset: Succeeded.
at server
LOG: terminating any other active server processes
WARNING: terminating connection because of crash of another server process
DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT: In a moment you should be able to reconnect to the database and repeat your command.
WARNING: terminating connection because of crash of another server process
DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT: In a moment you should be able to reconnect to the database and repeat your command.
LOG: all server processes terminated; reinitializing
LOG: database system was interrupted; last known up at 2014-11-07 16:47:03 GMT
LOG: database system was not properly shut down; automatic recovery in progress
LOG: redo starts at 0/2126C98
LOG: record with zero length at 0/21A9D98
LOG: redo done at 0/21A9D68
LOG: last completed transaction was at log time 2014-11-07 16:47:26.844406+00
LOG: autovacuum launcher started
LOG: database system is ready to accept connections
I want to log each query execution time which is run in a day.
For example like this,
2012-10-01 13:23:38 STATEMENT: SELECT * FROM pg_stat_database runtime:265 ms.
Please give me some guideline.
If you set
log_min_duration_statement = 0
log_statement = all
in your postgresql.conf, then you will see all statements being logged into the Postgres logfile.
If you enable
log_duration
that will also print the time taken for each statement. This is off by default.
Using the log_statement parameter you can control which type of statement you want to log (DDL, DML, ...)
This will produce an output like this in the logfile:
2012-10-01 13:00:43 CEST postgres LOG: statement: select count(*) from pg_class;
2012-10-01 13:00:43 CEST postgres LOG: duration: 47.000 ms
More details in the manual:
http://www.postgresql.org/docs/8.4/static/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-WHEN
http://www.postgresql.org/docs/8.4/static/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-WHAT
If you want a daily list, you probably want to configure the logfile to rotate on a daily basis. Again this is described in the manual.
I believe OP was actually asking for execution duration, not the timestamp.
To include the duration in the log output, open pgsql/<version>/data/postgresql.conf, find the line that reads
#log_duration = off
and change it to
log_duration = on
If you can't find the given parameter, just add it in a new line in the file.
After saving the changes, restart the postgresql service, or just invoke
pg_ctl reload -D <path to the directory of postgresql.conf>
e.g.
pg_ctl reload -D /var/lib/pgsql/9.2/data/
to reload the configuration.
I think a better option is to enable pg_stat_statements by enabling the PG stats extension. This will help you to find the query execution time for each query nicely recorded in a view