I'm working on a project which requires me to write a query to create a materialized view in PostgreSQL. My requirement is that How I can get PostgreSQL materialized view refresh history time for specific materialized view.
PostgreSQL does not store the time when an SQL statement like REFRESH MATERIALIZED VIEW is run.
Any attempt to rely on the file modification time of the underlying data file is in vain, as jobs like autovacuum may modify the file.
The only way to retain such information is to store the times when you run the statement in a table yourself.
An alternative could be to log all DDL statements (log_statement = 'ddl') and retrieve the information from the log file.
Related
How can I (using PG-Admin) access or create a log (ideally timestamped) that displays the changes that have happened to a table?
First of the way enable pg_stat_statements on PostgreSQL.
create extension pg_stat_statements;
After then you can view all SQL queries executed on your DB. At this you can view when executed SQL, when finished, duration of execute times and etc.
For more details: PostgreSQL Documentation - pg_stat_statements
If you need history of updated or deleted records on the tables, you can do it for all tables manually writing triggers or own functions, using JSON(JSONB) data types.
Main question: I have several views depending on a PostgreSQL/PostGIS table and a final materialized view created by querying the other views. I need a fast and updatable final result (i.e. MV) to use in a QGIS project.
My aim is to update the starting table by overwriting it with new (lots of) values and hopefully have update views and materialized view. I use QGIS DB Manager to overwrite existing table but I get an error because of mv depending on it. If I delete mv, overwrite table and then recreate mv everything is ok but I'd like to avoid manual operations as much as possible.
Is there a better way to reach my goal?
Another question: If I set a trigger to refresh a mv when I update/insert/delete values in a table, would it work even in case of overwriting entire table with a new one?
Refreshing a materialized view runs the complete defining query, so that is a long running and heavy operation for a complicated query.
It is possible to launch REFRESH MATERIALIZED VIEW from a trigger (it had better be a FOR EACH STATEMENT trigger then), but that would make every data modification so slow that I don't think that is practically feasible.
One thing that might work is to implement something like a materialized view that refreshes immediately “by hand”:
create a regular table for the “materialized view” and fill it with data by running the query
on each of the underlying tables, define a row level trigger that modifies the materialized view in accordance with the changes that triggered it
This should work for views where the definition is simple enough, for complicated queries it will not be possible.
I want to write a query that will update all materialized views in the database. The materialized views have dependencies on each other so they have to be materialized in a certain order.
Is there a way to come up with a query that will do this in a hands off way? Meaning I don't have to maintain some data structure that records the dependencies. Just every-time I add a new materialized view it will be updated in the right order. This could be an internal sql routine or an external app that calls the query periodically.
I was thinking of using row creation time of pg_matview but postgresql doesn't record this anywhere.
We have a view that aggregates from a backing table. The idea is to reduce cpu load by using a pre-aggregated table, and to periodically refresh it with the following:
create new_backing_table (fill it)
begin
drop backingtable
rename new_backingtable to backingtable
commit
while in production. The latency caused by the refresh interval is acceptable. Incremental updates are possible but not desirable.
Anyone has a comment on this scheme ?
Check out materialized views. This may suit your use case. It can be used to store query results at creation then refreshed at a later time.
A materialized view is defined as a table which is actually physically stored on disk, but is really just a view of other database tables. In PostgreSQL, like many database systems, when data is retrieved from a traditional view it is really executing the underlying query or queries that build that view.
https://www.postgresql.org/docs/9.3/static/sql-creatematerializedview.html
I'm unfortunately getting stuck on database stuff with PostGreSQL..
I created some materialized views in two schemas "schema1" and "schema2". These materialized views work well.
Then I made a backup of my database with PgAdmin.
And finally, I restored that database in an other server.
But this does not work as expected : all the materialized views have not been created (whereas both tables and classic views of "schema1" and "schema2" did)...
Hence my question : is it possible to restore materialized views. And if yes, can PgAdmin do the trick ?
Thanks you !
It makes sense that PG won't include the calculated materialized view in the backup since they can be generated from the data in the tables.
This is explained in more detail on the answer to this question from the Database Admin SO
To get the materialized view populated again after the backup you could try refreshing it.
REFRESH MATERIALIZED VIEW mymatview;
If you need to keep the data as it was at the time each backup, then perhaps you could insert select it into a table before the backup operation.