CREATE VIEW materialized_view WITH SCHEMABINDING AS
SELECT ...
FROM ext.external_table
Fails with
The option 'SCHEMABINDING' is not supported with external tables.
If I understand correctly SCHEMABINDING is necessary to make a materialized view.
How can I correct this query?
You cannot create an indexed view based on tables that are in a different database.
I think your options are:
a) create the indexed view in the other database and create a regular view in this database to query that indexed view
b) create a copy of the table in this database and a mechanism to update this table whenever the data is changed in the table which is in the other database; this could be done with triggers, replication, a stored procedure called on a schedule, etc.
Related
We have a database in SQL server and, in PostgreSQL, we define the foreign data table on its tables.
Since the index cannot be defined on the fields within fdt, if we can define the view on these fdt's, can columns of that view be indexed?
If we define an index, how do these indices help to improve searches while the main data is, for example, in SQL server.Should we use materialized views or simple views?
If we use the materialized view, is the SQL server data transmitted to PostgreSQL and then the PostgreSQL create indices on the transferred data?
You can define an index on a materialized view because it is backed by a physical PostgreSQL table.
You cannot define an index on a view, because the view is replaced with its definition during query optimization, and you cannot define an index on a foreign table.
If you query a materialized view that is defined on foreign tables, the foreign tables themselves are not queried at all.
We're in the process of running a handful of hourly scripts on our Redshift cluster which build summary tables for data consumers. After assembling a staging table, the script then runs a transaction which deletes the existing table and replaces it with the staging table, as such:
BEGIN;
DROP TABLE IF EXISTS public.data_facts;
ALTER TABLE public.data_facts_stage RENAME TO data_facts;
COMMIT;
The problem with this operation is that long-running analysis queries will place an AccessShareLock on public.data_facts, preventing it from being dropped and thrashing our ETL cycle. I'm thinking a better solution would be one which renames the existing table, as such:
ALTER TABLE public.data_facts RENAME TO data_facts_old;
ALTER TABLE public.data_facts_stage RENAME TO data_facts;
DROP TABLE public.data_facts_old;
However, this approach presupposes that 1) public.data_facts exists, and 2) public.data_facts_old does not exist.
Do you know if there's a way to conduct this operation safely in SQL, without relying on application logic? (eg. something like ALTER TABLE IF EXISTS).
I haven't tried it but looking at the documentation of CREATE VIEW it seems that this can be done with late-binding views.
The main idea would be a view public.data_facts that users interact with. Behind the scenes, you can load new data and then swap the view to “point” to the new table.
Bootstrap
-- load data into public.data_facts_v0
CREATE VIEW public.data_facts AS
SELECT * from public.data_facts_v0 WITH NO SCHEMA BINDING;
Update
-- load data into public.data_facts_v1
CREATE OR REPLACE VIEW public.data_facts AS
SELECT * from public.data_facts_v1 WITH NO SCHEMA BINDING;
DROP TABLE public.data_facts_v0;
The WITH NO SCHEMA BINDING means the view will be late-binding. “A late-binding view doesn't check the underlying database objects, such as tables and other views, until the view is queried.” This means the update can even introduce a table with renamed columns or a completely new structure.
Notes:
It might be a good idea to wrap the swap operations into a transaction to make sure we don't drop the previous table if the VIEW swap failed.
You can add a new load time timestamp encode runlength default getdate() column to your target table, and make your ETL do this:
INSERT INTO public.data_facts
SELECT * FROM public.data_facts_staging;
DELETE FROM public.data_facts
WHERE load_time<(select max(load_time) from public.data_facts);
DROP TABLE public.data_facts_staging;
note: public.data_facts_staging should have exactly the same structure as public.data_facts except that the last column of public.data_facts is load_time, so that on insert it will be populated with the current timestamp.
The only implication is that it would require extra disk space for a moment between you insert new rows and delete the old rows, and load_time has to be always the last column. Also you have to vaccum table every time you do this.
Another good thing about this is that if your ETL fails and staging table is empty or there is no staging table you won't lose your data. In the pure SQL scenario of swapping tables with DDL you're not protected from dropping the target table when staging table is missing. In the suggested scenario if no new rows are inserted the delete statement deletes nothing (there are no rows less than max load time), so worst case is just having the old version of data.
p.s. there is a command that instead of insert ... select ... just changes the pointer from staging to target table (alter table ... append from ...) but it requires the same type of lock as alter table I guess, so I don't suggest this
I use PostgreSQL for time series data. There is an Event table and partitioned tables like Event_2016, Event_2017 which inherit from Event with a CONSTRAINT CHECK for the date range. So when querying the Event table, PostgreSQL uses only the child tables that are needed.
To roll up events I use an EventByDay materialized view. Refreshing this requires reading from all Event_* tables.
Can I use Materialized Views the same way as tables above to limit the amount of data in each Materialized View? (EventByDay_2016 inherits from EventByDay).
No, a MVIEW can not participate in table inheritance.
But you can create a (regular) child table and then use insert into .. select ... using the query from the MVIEW. If you want to store the MVIEW's query in the database then create a view that you use for populating the child table.
Something like this:
Initial setup:
create view v_event_by_day_2016
as
-- this is the complete query from your MVIEW
select ...;
create table event_by_day_2016
as
select *
from v_view_one;
alter table event_by_day_2016 inherit event_by_day;
alter table event_by_day_2016 add constraint check (...);
Refresh the table:
truncate table event_by_day_2016;
insert into event_by_day_2016
select *
from v_event_by_day_2016;
Alternatively you can use delete to so that the data in the child table can be refreshed in a transactional manner.
I am adding distkey/sortkey to all my tables in redshift and would like to automate this. I am doing the following:
ALTER TABLE table RENAME TO tmp_table;
CREATE TABLE table
distkey(id)
sortkey(id)
AS
select * from tmp_table;
DROP TABLE tmp_table;
This works great, except the views don't get migrated. When you ALTER TABLE, the existing views would point to the tmp_table. Ideally I want to restore the views to the way before, possibly in the same query transaction or as part of a script.
In your migration procedure:
For each view execute:
select definition from pg_views where viewname = 'my_view';
And store all the results.
Execute for all views:
drop view 'my_view'; // or rename only to have a rollback option
Alter your tables
Execute the create view commands obtained in step 2
Just drop and recreate the views. That could be part of the script.
Views might not form part of a transaction, so some testing might be required.
Is there any way I can edit the MVIEW query without dropping it in TOAD.I am not sure if we can do it?If not how Can I do it in a way that does not affect the table contents?
Thanks in advance
You can't alter a materialized view query, you have to drop and recreate it.
See: http://docs.oracle.com/cd/B28359_01/server.111/b28286/statements_2001.htm
Use the ALTER MATERIALIZED VIEW statement to modify an existing
materialized view in one or more of the following ways:
To change its storage characteristics
To change its refresh method, mode, or time
To alter its structure so that it is a different type of materialized
view
To enable or disable query rewrite
As pointed out by Jeffre Kemp, you cannot change the query of an existing Materialized View.
However, if you expect that changes will be necessary in the future, you can choose one of these approaches:
use an intermediate VIEW that does all the heavy lifting; your MATERIALIZED VIEW then does a simple SELECT * FROM myView. This allows you to change the query - however, it does not allow you to add / remove columns to/from the MATERIALIZED VIEW
use a pre-built table for your materialized view; this logically separates the TABLE that holds the content from the query that is used to update the content. Now, you can drop the MATERIALIZED VIEW, add/remove columns from your TABLE as needed, and re-create the MATERIALIZED VIEW afterwards
use a synonym for all code that queries the MATERIALIZED VIEW; if necessary, create a backup table using CTAS, point the synonym to this backup table, drop and re-create your MATERIALIZED VIEW, point the synonym back to the MATERIALIZED VIEW, and drop your backup table (that's how we do it)