I have a table in DB2 database that has a few columns, one of which is L_TIMESTAMP. The need is to delete records where difference between L_TIMESTAMP and CURRENT TIMESTAMP is greater than 5 minutes. This check needs to happen every hour. Please let me know if there is an approach to accomplish this at the DB end rather than scheduling a cron job at the appserver end.
The administrative task scheduler in DB2 would be a good way to accomplish this. You need to wrap the DELETE statement in a stored procedure, then submit it to the scheduler. The syntax for defining the schedule is based on cron but it is all handled inside DB2.
http://pic.dhe.ibm.com/infocenter/db2luw/v9r7/index.jsp?topic=%2Fcom.ibm.db2.luw.admin.gui.doc%2Fdoc%2Fc0054380.html
Related
In my project i wanna auto delete the data from the database after a fixed interval of time say (30 days); Is there any automatic process to delete the data from the database without the user intervention..
i use nest js and postgres.
i try to use createdAt but never worked.
Postgresql does not have a TTL function for data. You will be better off writing a cron job. Or better yet you can use nestjs-scheduler to write the cron job. Refer to Documentation of nestjs-scheduler
I can do currently active count of all INSERT queries executed on the PostgreSQL server like this:
SELECT count(*) FROM pg_stat_activity where query like 'INSERT%'
But is there a way to count all INSERT queries executed on the server in a given period of time? E.g. in the past minute?
I have a bunch of tables into which I send a lot of inserts and I would like to somehow aggregate how many rows I am inserting per minute. I could code a solution for this, but it'd be so much easier if this was possible to somehow extract directly from the server.
Any type of stats like this, in a certain period of time, would be very helpful, an average time it takes for the query to process, or knowing the bandwidth that goes through per minute, etc.
Note: I am using PostgreSQL 12
If not already done, install pg_stat_statements extension and take some snapshots of the view pg_stat_statements: the diff will give the number of queries executed between 2 snapshots.
Note: It doesn’t save each individual query, rather it parameterizes them and then saves the aggregated result.
See https://www.citusdata.com/blog/2019/02/08/the-most-useful-postgres-extension-pg-stat-statements/
I believe that you can use the audit trigger.
This audit will create a table that register INSERT, UPDATE and DELETE actions. So you can adapt. So every time that your database runs one of those commands, the audit table register the action, the table and the time of the action. So, it will be easy to do a COUNT() on desired table with a WHERE from a minute ago.
I couldn't come across anything solid, so I have created a table where I log a number of insert transactions using a script that runs as a cron job. It was simple enough to implement and I do not get estimations, but the real values instead. I actually count all new rows inserted to tables in a given interval.
Scenario
I have a table which contains tasks that need to be completed by a specific datetime. If the task is not completed by this datetime (+- variable interval) then it will run a script to 'escalate' this task.
This variable interval can be as small as 2 seconds or as large as 2 years
Thoughts so far
Running a cron job every second either via pg_cron or similar will technically allow me to do a check on the database every second, however there is a lot of wasted processing here and a lot of database overhead and i'd rather not do this if possible.
Triggers can be fired on row insert/update/delete. so worst case scenario is we have an external script watching for these triggers being fired.
Question
Is there a way to schedule a query to run at a specific time, ideally within postgresql itself rather than via a bash/cron script. ie:
at 2017-09-30 09:32:00 - select * from table where datetime <= now
Edit
As it came up in the comments PGAgent is a possibility and the scenario for such would be:
The task is created by the user in the application, and the due date is set (eg 2017-09-28 13:00:00) the user has an interval before/after this due date where the task is escalated (eg One Hour Before) so at 12:00:00 on 2017-09-28 i want PGAgent/other option to run my sql script that does the escalation.
The script to escalate is already written and works, the date and time for this PGAgent script to be run is already calculated by another script.
I want to delete rows on one my tables that are more than 7 days old. What is the best way to do this? to make a cron job that runs every night or does PostgreSQL have built in features for doing something like this?
delete from the_table
where the_timestamp < now() - interval '7 days'
PostgreSQL does not currently have a built-in cron-like functionality, so I'd use the system's cron to run a suitable delete statement. If you already have a btree index on the timestamp column, you might as well run the delete much more frequently than nightly, taking out small chunks at a time. If you don't have the index, then running it nightly during off-peak times would be better.
If these don't have the performance you need, you could try partitioning. But I'd do that as a last resort, not a first resort.
The easiest way (for me) to schedule a DB job is to create a cron job, that executes a SQL script using psql.
Here you can read about psql. Use -f or -c to pass the SQL commands to psql.
Also it might be easier to write a PL/pgSQL function, that does your job and call it from psql with SELECT my_function();
i need to monitor my postgres server. i need to get an alarm if there is no change in certain tables after a given time. i've been trying to get xymon and nagios to do this and i have not been able to. please help
You probably want to look at pg_stat_user_tables and note whether the statistics for row insertion/deletion/updates have changed for the table. That's the easiest way to check for this sort of activity in monitoring software.
You might also get ideas in this area from looking at the source code to the best of the PostgreSQL monitoring plug-in, the Nagios one: check_postgres
First, create a trigger on the table that activates on any modification statement (INSERT/UPDATE/DELETE). This trigger should update a "last-changed" timestamp somewhere (e.g. a field in some other control table).
Then, you'll need a separate process that is started regularly by some external means (e.g. cron on Unix). This process is run e.g. every 10 minutes, or every hour -- whatever granularity you need. It simply checks the last-changed timestamp to determine whether there has been any activity in the period since the last check.
It's not a free solution, but LogicMonitor's postgres monitoring can do this trivially.
If you have a means to get an alert when a file does not change in some time, then I have a less elegant, but probably simpler solution: try to find out the filename where Postgres stores the table in question (someone should dig into system tables in Postgres - maybe ask this in a separate question) and then have your monitoring tool set up to watch the modify time of that file.