I need to log all activity for some specific user on database. I have set up the logging with ALTER ROLE username SET log_statement TO 'all'; and the logging works fine, all queries from user are logged. The problem is that for this user queries to Postgres internal schemas (pg_catalog) from clients like psql and pgAdmin are also logged. I have a bunch of lines with SELECT pg_catalog.quote_ident(n.nspname) || '.' || pg_catalog.quote_ident(c.relname).... in the log that are of no use to me. Even worse this queries are more then one line in the log so it's not easy to filter them out.
Is it possible to somehow restrict the logging only to one specific database or schema and not to include queries to other schemas like pg_catalog?
I don't know if the standard logging utility in postgres has that option (my guess is no). But maybe it's worth a look to the pgaudit external library for postgres.
The module pgadmin is designed to generate audit logs, but it uses the standard postgres logging tool. You can tweak several parameters to customize the logs, and it has a specific parameter that I think is perfect for your use case. From the documentation:
pgaudit.log_catalog
Specifies that session logging should be enabled in the case where all
relations in a statement are in pg_catalog. Disabling this setting
will reduce noise in the log from tools like psql and PgAdmin that
query the catalog heavily.
The default is on.
I hope it helps!
Change your logging format from text to csv (log_destination=csvlog) — you can then import the data to the database and then filter out the queries you are not interested in:
Using CSV-Format Log Output
Related
When checking log files, there is a lot of internal/system queries (runned internally by pgadmin tool for example) and not directly executed by user.
I there any way to distinguish which queries run by user directly, and which is internal queries?
(btw, config params I use for logging: log_statement=all and log_min_duration_statement=0)
I am not sure what you mean by "internal/system queries". Unless you use auto_explain with auto_explain.log_nested_statements = on, PostgreSQL will only log the queries that get sent by the client.
It may well be that some of these queries are internal queries of your database driver or application server, but PostgreSQL doesn't know that. For that, you'd have to enable logging at a component that has that information.
Queries of multiple schemas of DB are logged in the pg_log directory in an application server, and the performance of specific schema needs to be profiled.
The same queries that are running, the difference is that search_path is set to include different schemas in each case.
How to log queries using a specific in pg_log? Or is there any way to "tail" specific the log for queries using a specific schema on the command line?
There is no way to do that.
What you could do is use different users to run the queries with different search_path settings, and then only log queries by a certain user:
ALTER ROLE myuser SET log_min_duration_statement = 0;
I want to log the actual sql statements executed against a POSTGRES instance. I am aware that I can enable logging of the sql statements. Unfortunately, this doesn't log the actual sql, but rather a parsed version, with certain parameters stripped out and listed separately.
Is there a tool for reliably reconstituting this output into executable sql statements?
Or is there a way of intercepting the sql that is send to the postgres instance, such that that sql can be logged?
We want to be able to replay these sql statements against another database.
Thanks for your help!
Actually, PostgreSQL does log exactly the SQL that got executed. It doesn't strip parameters out. Rather, it doesn't interpolate them in, it logs what the application sent, with bind parameters separate. If your app sends insert into x(a,b) values ($1, $2) with bind params 42 and 18, that's what gets logged.
There's no logging option to interpolate bind parameters into the query string.
Your last line is the key part. You don't want logging at all. You're trying to do statement based replication via the logs. This won't work well, if at all, due to volatile functions, the search_path, per-user settings, sequence allocation order/gap issues, and more. If you want replication don't try to do it by log parsing.
If you want to attempt statement-based replication look into PgPool-II. It has a limited ability to do so, with caveats aplenty.
Via setting log_statement to all on postgresql.conf. See the documentation for runtime-config-logging
With the sql
select * from pg_stat_activity
I can see all users connected to my database, I need something like that to show which schema is using each user connected
You can't connect to a schema, so it isn't clear what you are looking for. A schema is just a logical namespace for groups within the system. Now:
Determining which schemas a user has access to requires connecting to the relevant db and checking. You can't do this globally since schemas are not global objects.
It should be possible to show the search_path since this only attaches to global objects (databases and roles), but I could not figure out how to do this by glancing through the system catalog docs. That's probably where you'd have to start if that's what you wanted to look for.
My application uses PostgreSQL for storing data. I need to gather information about all user actions (any INSERTs, UPDATEs or DELETEs) executed on the database. This made me wonder whether PostgreSQL provides any default implementations/tables for this? As per my prior searches, I haven't found anything usable yet - however proper confirmation to my suspicions would be nice. If PostgreSQL truly doesn't provide any default implementations for this, then I will design my own history table.
PostgreSQL supports several methods for logging server messages, including stderr, csvlog and syslog. On Windows, eventlog is also supported. Set this parameter to a list of desired log destinations separated by commas. The default is to log to stderr only. This parameter can only be set in the postgresql.conf file or on the server command line.
SEE HERE