I have a table in my database (postgres), that keeps in it the user accounting trace. I need to create a trigger that will perform a clean up at a given amount of time (let's say 30 minutes.) Is it possible to do that from postgres, or do I need to use something else? (By something else I mean C or Java or any other programming language that can sleep for a given time).
Without PgAgent, things like this can be done using a cron script.
In both cases the periodic thing will be running from a different context
(database owner, superuser, postgres), and in a separate transaction, obviously.
Related
I work with a software that is used by a lot of different clients in several countries, with different needs, rules and constraints on their data.
When I make a change to the database's structure, I have a tool to test it on every client's database, obviously with read-only rights. This means that the best way to test a query like UPDATE table SET x = y WHERE condition
is to call the "read-only part" SELECT x FROM table WHERE condition.
It works but it's not ideal, as sometimes it is writing data that causes problems (mostly deadlocks or timeouts), meaning I can't see the problem until a client suffers from it.
I'm wondering if there is a way to grant write permissions in Postgres, but only when inside a transaction, and force a rollback on every transaction. This way, changes could be tested accurately on real data and still prevent any dev from editing it.
Any ideas?
Edit: the volumes are too large to consider cloning data for every dev who needs to run a query
This sounds similar to creating an audit table to record information about transactions. I would consider using a trigger to write a copy of the data to a "rollback" table/row and then copy the "rollback" table/row back on completion of the update.
I have a stored procedure on Postgres, which processes large data and takes a good time to complete.
In my application, there is a chance that 2 processes or schedulers can run this procedure at same time. I want to know if there is a built in mechanism in db to allow only instance of this procedure to run at db level.
I searched the internet, but didn't find anything concrete.
There is nothing built in to define a procedure (or function) so that concurrent execution is prevented.
But you can use advisory locks to achieve something like that.
At the very beginning of the procedure, you can add something like:
perform pg_advisory_lock(987654321);
which will then wait to get the lock. If a second session invokes the procedure it will have to wait.
Make sure you release the lock at the end of the procedure using pg_advisory_unlock() as they are not released when the transaction is committed.
If you use advisory locks elsewhere, make sure you use a key that can't be used in other places.
I am using PostgreSQL 10 from RDS (AWS).
So note that I don't have full permissions to do whatever I want.
In PostgreSQL I have some functions written in PL/pgSQL.
From my experience in these function I cannot start/commit/rollback transactions. In a DO block I cannot do that either.
Is that correct? So what is the logic behind this... seems PostgreSQL expects each function to be called in the context of an existing transaction. Right?
But what if I want every statement in my function to be executed in a separate (short) transaction i.e. to have a behavior something like AUTOCOMMIT = ON?
I found some extension which maybe can do that but I am not sure.
I don't know if it's relevant.
https://www.postgresql.org/docs/10/ecpg-sql-set-autocommit.html
Isn't there a standard way of doing this in Postgres without the need to download and install additional packages/extensions?
Again: I want every statement in my function to be executed in a separate (short) transaction i.e. to have a behavior something like AUTOCOMMIT = ON.
So I want something like this:
https://learn.microsoft.com/en-us/sql/t-sql/statements/set-implicit-transactions-transact-sql?view=sql-server-2017
All statements in a function run in the same transaction, and no plugin can change that.
You can use procedures from v11 on, but you still have to explicitly manage transactions then.
I suspect that the best thing would be to run your functions on the database client, where you have autocommit automatically, rather than as a function in the database.
i need to monitor my postgres server. i need to get an alarm if there is no change in certain tables after a given time. i've been trying to get xymon and nagios to do this and i have not been able to. please help
You probably want to look at pg_stat_user_tables and note whether the statistics for row insertion/deletion/updates have changed for the table. That's the easiest way to check for this sort of activity in monitoring software.
You might also get ideas in this area from looking at the source code to the best of the PostgreSQL monitoring plug-in, the Nagios one: check_postgres
First, create a trigger on the table that activates on any modification statement (INSERT/UPDATE/DELETE). This trigger should update a "last-changed" timestamp somewhere (e.g. a field in some other control table).
Then, you'll need a separate process that is started regularly by some external means (e.g. cron on Unix). This process is run e.g. every 10 minutes, or every hour -- whatever granularity you need. It simply checks the last-changed timestamp to determine whether there has been any activity in the period since the last check.
It's not a free solution, but LogicMonitor's postgres monitoring can do this trivially.
If you have a means to get an alert when a file does not change in some time, then I have a less elegant, but probably simpler solution: try to find out the filename where Postgres stores the table in question (someone should dig into system tables in Postgres - maybe ask this in a separate question) and then have your monitoring tool set up to watch the modify time of that file.
Does anyone knows how can I set up an insert trigger so when a perform an insert from my application, the data gets inserted and postgres returns, even before the trigger finishes executing?
There is no built-in support for this; you will have to hack something up. Options include:
Write the trigger in C, Perl, or Python and have it launch a separate process to do the things you want. This can get tricky and possibly slightly dangerous to your database system, and it only works if the things you want to do are outside of the database.
Write a lightweight trigger function that only records an entry into a log or task table, and have a separate job or daemon that looks into that table on its own schedule and executes things from there. That's more or less how Slony works.
The question is : why do you need it? Triggers should be fast. If you need to do something complicated, write trigger that send notification to some daemon that does the complex part - for example using LISTEN/NOTIFY feature of PostgreSQL.