I am looking for a mechanism to reload tables from a production environment.
Using materialized views ultimately bring little benefit compare to a basic truncate / insert as select on existing table.
what added value give materialized views?
Related
CREATE VIEW materialized_view WITH SCHEMABINDING AS
SELECT ...
FROM ext.external_table
Fails with
The option 'SCHEMABINDING' is not supported with external tables.
If I understand correctly SCHEMABINDING is necessary to make a materialized view.
How can I correct this query?
You cannot create an indexed view based on tables that are in a different database.
I think your options are:
a) create the indexed view in the other database and create a regular view in this database to query that indexed view
b) create a copy of the table in this database and a mechanism to update this table whenever the data is changed in the table which is in the other database; this could be done with triggers, replication, a stored procedure called on a schedule, etc.
I have been using Postgres for a while now, but I have not implemented any triggers yet. I wanted to check if this will do what I intend it to do.
On a daily basis, I am adding new rows to a table (COPY) and also updating existing rows if there is a primary key conflict (ON CONFLICT DO UPDATE SET). I then have a materialized view using that table and a few other joins, and this view is used for a lot of reporting.
I want the materialized view to update when the original table has been updated, without me needing to schedule it or run it manually. (Right now I have it scheduled with a Python psycopg2 execute command).
CREATE OR REPLACE FUNCTION refresh_mat_view()
RETURNS TRIGGER LANGUAGE plpgsql
AS $$
BEGIN
REFRESH MATERIALIZED VIEW schema_name.materialized_view_name;
RETURN NULL;
END $$;
CREATE TRIGGER refresh_view
AFTER INSERT OR UPDATE OR DELETE OR TRUNCATE
ON sutherland.dimension_peoplesoft FOR EACH STATEMENT
EXECUTE PROCEDURE refresh_mat_view();
Would that refresh the view for every single row which is updated too? I am just imagining it trigger a refresh for each individual row which might be 100k+. It would be better to happen AFTER all inserts have been done (I have Python looping through each row in a pandas DataFrame to UPSERT into the database).
When you use a pure Materialized View, every time you refresh it, it will rebuild the whole thing. So if your data changes a lot and you need the data available fast it's not an optimized choice.
You should use Eager Materialized Views or Lazy Materialized Views that basically are "A well use of triggers". Obviously it's harder to do than a pure materialized view, but the results are better (depending on the case of use).
You should check this article Materialized View Strategies Using PostgreSQL
I have a materialized view based on a table that on which a column type changed. and I'd like to change the column on the materialized view.
I'm aware that a view is based on it's definition but I couldn't find how to updated the select in the definition.
Is the only way to do it to drop the view and re-create it with a new definition?
It's not what you were hoping to get, but currently the only way to change the query on which a materialized view is based is to drop and recreate it. Still the case in Postgres 10.
ALTER MATERIALIZED VIEW can only change auxiliary properties. You can also change column names, but not data types.
If concurrent access is required and the MV takes a long time to recreate, you might create a new MV under a different name, populate it and use it instead of the old one to keep downtime to a minimum - if that's an option.
Related:
Replace a materialized view in Postgres
In Oracle, it is possible to refresh just part of the data. But in PostgreSQL, materialized views are supported since 9.3 (the current version now), which is not so long. So I wonder: is it possible to refresh just part of the data in the materialized view in PostgreSQL 9.3? If yes, how to do it?
PostgreSQL doesn't support progressive / partial updates of materialized views yet.
9.4 adds REFRESH MATERIALIZED VIEW CONCURRENTLY but it still has to be regenerated entirely.
Hopefully we'll see support in 9.5 if someone's enthusiastic enough. It's only possible to do this without user-defined triggers/rules for simple materialized views though, and special support would be needed to even handle things like incremental update of a count(...) ... GROUP BY ....
The Oracle answer you refer to isn't actually incremental refresh, though. It's refresh by-partitions. For PostgreSQL to support that natively, it'd first have to support real declarative partitioning - which it doesn't, though we're discussing whether it can be done for 9.5.
I just came across a similar problem. Learning from Craig's answer that it is not possible, I used a workaround. I deconstructed the materialized view and joined and/or unioned the individual parts in a VIEW:
Create a MATERIALIZED VIEW for each row or column group in question (material_col1, material_col2, etc. or with more complex disjunct where conditions), using e.g. a common id column.
Use a regular VIEW (fake_materialized_view) joining the MATERIALIZED VIEWs tables on the id column
in the case of disjunct rows one has to union all them
REFRESH MATERIALIZED VIEW as needed
Use your query on fake_materialized_view instead
The VIEW would look somewhat like this:
CREATE VIEW fake_materialized_view AS
SELECT m1.id, m1.col1, m2.col2
FROM material_col1 as m1 LEFT JOIN
material_col2 as m2
ON m1.id = m2.id
-- in case of additional row partitioning, e.g.
-- UNION ALL SELECT m3.id, m3.col1, m3.col2
-- FROM material_col3 m3
(Upd1: Thx to Barry for his comment utilizing row partitioning, which I added to the answer.)
Is there any way I can edit the MVIEW query without dropping it in TOAD.I am not sure if we can do it?If not how Can I do it in a way that does not affect the table contents?
Thanks in advance
You can't alter a materialized view query, you have to drop and recreate it.
See: http://docs.oracle.com/cd/B28359_01/server.111/b28286/statements_2001.htm
Use the ALTER MATERIALIZED VIEW statement to modify an existing
materialized view in one or more of the following ways:
To change its storage characteristics
To change its refresh method, mode, or time
To alter its structure so that it is a different type of materialized
view
To enable or disable query rewrite
As pointed out by Jeffre Kemp, you cannot change the query of an existing Materialized View.
However, if you expect that changes will be necessary in the future, you can choose one of these approaches:
use an intermediate VIEW that does all the heavy lifting; your MATERIALIZED VIEW then does a simple SELECT * FROM myView. This allows you to change the query - however, it does not allow you to add / remove columns to/from the MATERIALIZED VIEW
use a pre-built table for your materialized view; this logically separates the TABLE that holds the content from the query that is used to update the content. Now, you can drop the MATERIALIZED VIEW, add/remove columns from your TABLE as needed, and re-create the MATERIALIZED VIEW afterwards
use a synonym for all code that queries the MATERIALIZED VIEW; if necessary, create a backup table using CTAS, point the synonym to this backup table, drop and re-create your MATERIALIZED VIEW, point the synonym back to the MATERIALIZED VIEW, and drop your backup table (that's how we do it)