Postgres table sync between different schemas - postgresql

I have table A in schema A and table B in schema B with same structure in the same database. Whenever a DML change happens in table A, I need the same in table B. For now, I am using triggers to do the same. Is there any better alternative than using triggers for this scenario?
As the tables belong to different microservices, I need one of the tables with data unmodified even if the other table is dropped.

Related

How do I copy data from one table to another, perform schema change and keep them in sync until cut off in Postgres?

I have workloads that have heavy schema changes and other ETL operations that are locking.
Before doing schema changes on my primary table, I would like to first copy the existing contents from the primary table on to a temporary table, then perform the schema change, then sync all new changes and once the "time is right" (cutoff?), do the cut over and have the temporary table become the primary table.
I know that I can use Triggers in postgres to sync data between two tables, and also use COPY to copy data from one table to another.
But I am not sure how can I can copy existing data first, then issue trigger to ensure no data is lost. Then also do the cut off so that the new table is primary.
What I am thinking is -
I issue a COPY table from primary table (TableA) to temp table TableB.
I then perform the schema change in TableB
I then setup Trigger from TableA to TableB for INSERT/UPDATE/DELETE
... Now I am not sure how can I cut off so TableB becomes TableA. I can use RENAME perhaps?
It feels like I can run into some lost changes between Step 1 and Step 2?
Basically I am trying to ensure no data between the three high level operations. Is there a better way to do this?

maintaining two table structures the same on postgresql

We have two tables A and B in PostgreSQL 9.4. We want to ensure that A's columns are always a subset of the columns of B (that is, preventing an ALTER on A from adding/dropping/modifying columns that would make it differ from B). How can this be achieved?
My guess is with a kind of trigger on ALTER (though triggers happen only on CRUD) or a special constraint on A ? (though a Foreign Key on every column seems like an overkill).
(the use case: B is like a shadow of A, it will periodically receive a dump of A's records, so we want to be sure that if the structure of A changes we don't forget to change B accordingly)
As you already know triggers are in CRUD. I think maybe you can create a trigger where you compare both table on each CRUD and raise an alarm. But that only will alert after the change was made and someone play with tableA.
You can create multiple FK for each tableB columns to tableA columns, but that only will inform if you try to delete or rename a columns, If you change field type or add new columns you wont get the alarm.
#Abelisto is right in his comment, a good way to implement this in PostgreSQL is using Inheritance.
Another way is having a master a table and an "updatable VIEW" (or a view with RULEs...)

Tableau - Blend Multiple Tableau Server Data Sources

I have three Tableau Server data sources in a workbook that represent three tables in a database. Table A is joined to Table B and table B is joined to Table C. There is no foreign key relationship between Table A and Table C, but the implicit relationship exists through Table B (junction table). When using Table A as the primary data source on a report, I receive an error message when attempting to add data from Table C.
In order to use fields from Table C, a relationship needs to be created with Table A. Select Data > Edit Relationships to open the Relationships dialog box.
Is there any way to join multiple Tableau Server data sources to accomplish the described scenario?
Tableau has the concept of Primary and Secondary data sources, and every secondary needs to have a key into the primary to be considered. You can't link a secondary to a secondary data source.
My suggestion here would be creating another data source already joined/blended. This can be done by connecting directly to the database and joining the tables using Tableau wizard (see image below):
Or, if you feel comfortable, you can use the New Custom SQL button and use your own query (see image below):

SQL Merge Query - Executing Additional Query

I have written a working T-SQL MERGE statement. The premise is that Database A contains records about customer's support calls. If they are returning a product for repair, Database B is to be populated with certain data elements from Database A (e.g. customer name, address, product ID, serial number, etc.) So I will run an SQL Server job that executes an SSIS package every half hour or so, in which the MERGE will do one of the following:
If the support call in Database A requires a product return and it
is not in Database B, INSERT it into Database B..
If the support call in Database A requires a product return and it
is in Database B - but data has changed - UPDATE it in Database B.
If there is a product return in Database B but it is no longer
indicated as a product return in Database A (yes, this can happen - a customer can change their mind at a later time/date and not want to pay for a replacement product), DELETE it from Database
B.
My problem is that Database B has an additional table with a 1-to-many FK relationship with the table being populated in the MERGE. I do not know how, or even if, I can go about using a MERGE statement to first delete the records in the table with FK constraint before deleting the records as I am currently doing in my MERGE statement.
Obviously, one way would be to get rid of the DELETE in the MERGE and hack out writing IDs to delete in a temp table, then deleting from the FK table, then the PK table. But if I can somehow delete from both tables in WHEN NOT MATCHED BY SOURCE that would be cleaner code. Can this be done?
You can only UPDATE, DELETE, or INSERT into/from one table per query.
However, if you added an ON DELETE CASCADE to the FK relationship, the sub-table would be cleaned up as you delete from the primary table, and it would be handled in a single operation.

trigger to update object in different tablespace

I have a table X in tablespace T1 and a table Y in tabelspace T2.(Oracle DB)
I have to create a trigger in tablespace T1 that will,
on the event of updating a column C in table X,
update column D in table Y (tablespace T2).
Because they are in different tablespaces, my first question is can this be done at all?
And if yes then how it can be done? What privileges are required to do such a thing?
It has not so much to do with the tablespace. You do need privileges to insert into the table (and that particular column) though, and if the table Y is in another schema than the trigger, you need to use the qualified table name: . (In Oracle, the schemaname is the name of the user that owns the object)
CREATE TRIGGER aur_x
AFTER UPDATE OF c ON x
FOR EACH ROW
UPDATE schema_containing_y.Y SET D = ...
;
EDIT:
It just occurred to me that you might not be familiar with the distinction between schema and tablespace, so here's a short explanation. A tablespace is a logical storage container: it dedfines datafiles, growth characteristics, logging types etc. Tablespaces can then be used as to store data associated with schema objects (tables, indexes, views definitions, but also packages and stored procedure definitions etc).
A schema is a collection of objects (like tables, views, pacakages etc.) These objects are owned by a user, and as far as i am aware, in oracle the schema has an name identical to the user that owns the objects. THe objects rely on the storage provided by one or more tablespaces, but tablespaces themselves are not schema objects.
Typically, a schema is used to group functionally related objects (for example, you'd typically create one schema for one application). Tablespaces can also be created especially to store all objects of one application, but you can also create different tablespaces for tables with different characteristics.
Normally, application developers shouldn't worry too much about tablespaces. Your DBA will typically set them up in a way that is convenient for things like backup plan.