i am trying to create table in talend using below code i see no error but in database this table is not getting created
do $$ declare begin execute 'DROP TABLE IF EXISTS tname'; execute 'CREATE TABLE IF NOT EXISTS tname (ACTIVITY VARCHAR(32))'; end $$ ;
Please help me i am new in Talend
Should be something like this
DO
$$
DECLARE BEGIN
EXECUTE 'DROP TABLE IF EXISTS tname';
EXECUTE 'CREATE TABLE IF NOT EXISTS tname (ACTIVITY VARCHAR(32))';
END;
$$
You will have to just use this component tDBRow
And a very important thing is to to use tDBCommit after the tDBRow if not the table would not be created in your Postgres Database
or just tick the commit (in advanced settings if you are using tDBConnection component)
I have setup postgres_fdw between two databases (sourcedb, targetdb) so that I can create Foreign Data Tables in targetdb from a schema in sourcedb.
All the above is configured and working as expected.
The next step was to re-Import Foreign Schema each time I have changes in the views in sourcedb.
In order to achieve this I created two functions in sourcedb:
fn_create_views
fn_recreate_foreign_data_tables
In the first function (fn_create_views) I am creating the views dynamically in a loop.
After the loop ends I am calling the second function that drops all foreign data tables and Import foreign schema through a dblink connecting on the targerdb.
CREATE FUNCTION fn_create_views ()
RETURNS BOOLEAN
LANGUAGE plpgsql
as $$
BEGIN
FOR .. IN
EXECUTE '..'
LOOP
EXECUTE format('CREATE OR REPLACE VIEW .. AS
SELECT * FROM ...', params);
END LOOP;
PERFORM fn_recreate_foreign_data_tables('source_foreign_server','target_foreign_server');
return true;
END $$;
CREATE FUNCTION fn_recreate_foreign_data_tables(_source_foreign_server varchar, _targer_foreign_server varchar)
returns void
language plphsql
as $$
DECLARE
_sql_exec text;
BEGIN
_sql_exec := (SELECT format('SELECT public.dblink_exec(%L,
''DO
$dblink$
DECLARE
l_rec record;
BEGIN
FOR l_rec IN (SELECT foreign_table_schema, foreign_table_name
FROM information_schema.foreign_tables
WHERE foreign_server_name = ''%L'')
LOOP
EXECUTE format(''''drop foreign table %I.%I'''', l_rec.foreign_table_schema, l_rec.foreign_table_name);
END LOOP;
IMPORT FOREIGN SCHEMA ..
FROM SERVER foreign_server INTO ..;
END $dblink$;'')', _source_foreign_server, _target_foreign_server));
EXECUTE _sql_exec;
end $$;
The issue I am experiencing with the above is that during the 'IMPORT FOREIGN SCHEMA' the 'CREATE VIEW' is not committed as a result although all the foreign tables are dropped its not Importing anything into the targetdb schema.
After reading several posts here in SO, some recommend to run the 'CREATE VIEW' command through dblink on the same DB.
Apparently this works perfectly since I guess dblink would open a separate transaction each time.
My question now is, is there another simpler way to do the above without calling the above functions separately ?
Thank you!
You need to do COMMIT the local transaction in which you create the views before you can use them with foreign tables.
I see two options:
Create the views in a dblink call to the local database. Then the transaction will be committed when dblink_exec is done.
Run a COMMIT between the calls to fn_create_views and fn_recreate_foreign_data_tables.
I'm trying to set up a trigger in postgresql 9.6 that will use dblink to insert a row into another database when a row is inserted into its own table. Since there can be a large volume of these inserts, I don't want to connect and disconnect to the database for every insertion, so I would prefer to have a persistent connection that gets used by every insert. But I think I would then have to test if the connection is available since a number of things could cause the connection to drop.
My pseudo-code is as follows:
-- 1. test if named dblink connection already exists
-- 2. if it does not, create a named dblink connection
-- 3. insert data via dblink
What I have so far is like this:
CREATE OR REPLACE FUNCTION db_link_trigger()
RETURNS trigger AS
$BODY$
BEGIN
-- 1. test if named dblink connection already exists
IF (SELECT COALESCE('dblinktest' = ANY (dblink_get_connections()), false)) = false THEN
-- 2. if it does not, create a named dblink connection
RAISE NOTICE 'dblink connection not established. Connecting now';
PERFORM dblink_connect('dblinktest', 'hostaddr=192.168.1.30 port=5433 dbname=otherdb user=myuser password=mypassword');
ELSE
RAISE NOTICE 'dblink connection already established';
END IF;
-- 3. insert data via dblink
PERFORM dblink_exec('dblinktest', 'insert into mytable(data) values(''' || NEW.data || ''');');
RETURN NEW;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
My main concerns are how to handle errors and how to handle near-simultaneous invocation of the trigger. Consider if two INSERTS come in at the same time, and there is no previous dblink named 'dblinktest'. When the first sees that the connection does not exist, it proceeds to set it up. Then the second could see that the link is also not established and try to connect itself, but it will fail since the first will connect before it, and it will raise an error as:
ERROR: duplicate connection name
How can I handle an error like that? Does postgresql have something like this python-inspired pseudo-code?
try:
if dblink connection is not established:
establish dblink connection
except 'ERROR: duplicate connection name':
pass # do nothing
finally:
insert row into other db via dblink connection
You can get the error code from this link https://www.postgresql.org/docs/10/errcodes-appendix.html i this case is:
42710 -> duplicate_object
and manage this error with PL/pgSQL BEGIN block with an EXCEPTION clause
for example:
CREATE OR REPLACE FUNCTION db_link_function()
RETURNS void AS
$BODY$
declare
rec record;
BEGIN
BEGIN
PERFORM dblink_connect('dblinktest', 'hostaddr=127.0.0.1 port=5435 dbname=dell user=postgres password=password');
--the EXCEPTION
EXCEPTION
WHEN duplicate_object THEN --code error 42710
RAISE NOTICE 'this connections exists';
END;
--select data via dblink
SELECT * FROM dblink('dblinktest','SELECT * FROM categories where category=1') AS t(cid int, cname text) into rec;
raise notice 'value of rec: %,%', rec.cid, rec.cname;
END;
$BODY$
LANGUAGE plpgsql;
I suggest using FDW instead of dblink https://www.postgresql.org/docs/10/postgres-fdw.html
I have created a trigger that works like this:
After deleting data from table flux_tresorerie_historique it insert this row in the table flux_tresorerie_historique that is located in another database archive
I use dblink to insert data in the remote database, the problem is that the creation of the query is too hard especially that the table contain more than 20 columns, and I want to create similar functions for 10 other tables.
Is there another rapid way to ensure this task?
Here an example that works fine:
CREATE OR REPLACE FUNCTION flux_tresorerie_historique_backup_row()
RETURNS trigger AS
$BODY$
DECLARE date_rapprochement_flux TEXT;
DECLARE code_commission TEXT;
DECLARE reference_flux TEXT;
BEGIN
IF OLD.date_rapprochement_flux is null
THEN
date_rapprochement_flux = 'NULL';
ELSE
date_rapprochement_flux = ''''||to_char(OLD.date_rapprochement_flux, 'YYYY-MM-DD')||'''';
END IF;
IF OLD.code_commission is null
THEN
code_commission = 'NULL';
ELSE
code_commission = ''''||replace(OLD.code_commission,'''','''''')||'''';
END IF;
IF OLD.reference_flux is null
THEN
reference_flux = 'NULL';
ELSE
reference_flux = ''''||replace(OLD.reference_flux,'''','''''')||'''';
END IF;
perform dblink_connect('dbname=gtr_bd_archive user=postgres password=postgres');
perform dblink_exec('insert into flux_tresorerie_historique values('||OLD.id_flux_historique||','''||OLD.date_operation_flux||''','''||OLD.date_valeur_flux||''','||date_rapprochement_flux||','''||replace(OLD.libelle_flux,'''','''''')||''','||OLD.montant_flux||','||OLD.contre_valeur_dzd||','''||replace(OLD.rib_compte_bancaire,'''','''''')||''','||OLD.frais_flux||','''||replace(OLD.sens_flux,'''','''''')||''','''||replace(OLD.statut_flux,'''','''''')||''','''||replace(OLD.code_devise,'''','''''')||''','''||replace(OLD.code_mode_paiement,'''','''''')||''','''||replace(OLD.code_agence,'''','''''')||''','''||replace(OLD.code_compte,'''','''''')||''','''||replace(OLD.code_banque,'''','''''')||''','''||OLD.date_maj_flux||''','''||replace(OLD.statut_frais,'''','''''')||''','||reference_flux||','||code_commission||','||OLD.id_flux||');');
perform dblink_disconnect();
RETURN NULL;
END;
This is a limited application of replication. Requirements vary a lot, so there are a number of different established solutions, addressing different situations. Consider the overview in the manual.
Your hand-knit, trigger-based solution is one viable option for relatively few deletions. Opening and closing a separate connection for every row incurs quite an overhead. There are other various options.
While working with dblink I suggest some modifications. Most importantly:
Use format() to escape strings more elegantly.
Pass the whole row instead of passing and escaping every single column.
Don't place the password in every single trigger function.
Use a FOREIGN SERVER plus USER MAPPING. Detailed instructions here:
Persistent inserts in a UDF even if the function aborts
Basically, run once on the source server:
CREATE SERVER myserver FOREIGN DATA WRAPPER dblink_fdw
OPTIONS (hostaddr '127.0.0.1', dbname 'gtr_bd_archive');
CREATE USER MAPPING FOR role_source SERVER myserver
OPTIONS (user 'postgres', password 'secret');
Preferably, don't log in as superuser at the target server. Use a dedicated role with limited privileges to avoid privilege escalation.
And use a password file on the target server to allow password-less access. This way you don't even have to store the password in the USER MAPPING. Instructions in the last chapter of this related answer:
Run batch file with psql command without password
Then:
CREATE OR REPLACE FUNCTION pg_temp.flux_tresorerie_historique_backup_row()
RETURNS trigger AS
$func$
BEGIN
PERFORM dblink_connect('myserver'); -- name of foreign server from above
PERFORM dblink_exec( format(
$$
INSERT INTO flux_tresorerie_historique -- provide target column list!
SELECT (r).id_flux_historique
, (r).date_operation_flux
, (r).date_valeur_flux
, (r).date_rapprochement_flux::date -- 'YYYY-MM-DD' is default ISO format anyway
, (r).libelle_flux
, (r).montant_flux
, (r).contre_valeur_dzd
, (r).rib_compte_bancaire
, (r).frais_flux
, (r).sens_flux
, (r).statut_flux
, (r).code_devise
, (r).code_mode_paiement
, (r).code_agence
, (r).code_compte
, (r).code_banque
, (r).date_maj_flux
, (r).statut_frais
, (r).reference_flux
, (r).code_commission
, (r).id_flux
FROM (SELECT %L::flux_tresorerie_historique) t(r)
$$, OLD::text)); -- cast whole row type
PERFORM dblink_disconnect();
RETURN NULL; -- only for AFTER trigger
END
$func$ LANGUAGE plpgsql;
You should spell out the list of columns for the target table if the row types don't match.
If you are serious about this:
insert this row in the table flux_tresorerie_historique
I.e., you insert the whole row and the target row type is identical (no extracting a date from a timestamp etc.), you can simplify much further passing the whole row.
CREATE OR REPLACE FUNCTION flux_tresorerie_historique_backup_row()
RETURNS trigger AS
$func$
BEGIN
PERFORM dblink_connect('myserver'); -- name of foreign server
PERFORM dblink_exec( format(
$$
INSERT INTO flux_tresorerie_historique
SELECT (%L::flux_tresorerie_historique).*
$$
, OLD::text));
PERFORM dblink_disconnect();
RETURN NULL; -- only for AFTER trigger
END
$func$ LANGUAGE plpgsql;
Related:
How do I do large non-blocking updates in PostgreSQL?
You can use quote_nullable for this! Also, concat_ws comes very handy:
CREATE OR REPLACE FUNCTION flux_tresorerie_historique_backup_row()
RETURNS trigger AS
$BODY$
BEGIN
perform dblink_connect('dbname=gtr_bd_archive user=postgres password=postgres');
perform dblink_exec('insert into flux_tresorerie_historique values('||
concat_ws(', ', quote_nullable(OLD.id_flux_historique),
quote_nullable(OLD.date_operation_flux),
quote_nullable(OLD.date_valeur_flux),
quote_nullable(to_char(OLD.date_rapprochement_flux, 'YYYY-MM-DD')),
quote_nullable(OLD.libelle_flux),
quote_nullable(OLD.montant_flux),
quote_nullable(OLD.contre_valeur_dzd),
quote_nullable(OLD.rib_compte_bancaire),
quote_nullable(OLD.frais_flux),
quote_nullable(OLD.sens_flux),
quote_nullable(OLD.statut_flux),
quote_nullable(OLD.code_devise),
quote_nullable(OLD.code_mode_paiement),
quote_nullable(OLD.code_agence),
quote_nullable(OLD.code_compte),
quote_nullable(OLD.code_banque),
quote_nullable(OLD.date_maj_flux),
quote_nullable(OLD.statut_frais),
quote_nullable(OLD.reference_flux),
quote_nullable(OLD.code_commission),
quote_nullable(OLD.id_flux)
)||');');
perform dblink_disconnect();
RETURN NULL;
END;
Note that it is OK to place non-sting values between single quotes, since a quoted literal is for PostgreSQL just as good a literal value as one without the quotes, so it is convenient to place all of the columns processed by quote_nullable. Also note that quote_nullable will already output dates in YYYY-MM-DD format (e.g. select quote_nullable(now()::date) would result in '2016-05-04'), so you may want to simplify OLD.date_rapprochement_flux even further by removing the to_char.
I am in the middle of converting an existing SQL Server 2005 DB into a PostgreSQL 9.0 DB.
Everything works fine until now. I want to translate a SQL trigger into PostgreSQL but I have a problem with the trigger function.
I don't know how to implement the temp table inserted in the PostgreSQL syntax. In SQL Server the inserted table exists but not in PostgreSQL. Any ideas?
My code (PostgreSQL):
CREATE OR REPLACE FUNCTION func_co_insert()
RETURNS trigger AS
$BODY$begin
declare
aa bigint;
begin
select aa = co_id from inserted;
update com03 set co_creationdate = CURRENT_TIMESTAMP,
co_creationby = USER where co_id = aa;
end;
end;
Here the code of the trigger body of the SQL Server 2005 code
begin
declare #aa bigint;
select #aa = se_id from inserted;
update server set se_creationdate = CURRENT_TIMESTAMP , se_creationby = USER where se_id = #aa;
end;
thanks
Chris
The default in PostgreSQL is a row level trigger (as opposed to SQL Server where it's a statement level trigger), so there is no need for an "inserted" table to select from.
The new and old values can be accessed using the keyword new and old (old does not exist for an insert trigger).
In your case the statement would simply be:
update com03
set co_creationdate = CURRENT_TIMESTAMP,
co_creationby = CURRENT_USER
where co_id = new.co_id;
No need to "select from inserted".
This assumes the trigger is not firing for the table com03. If your trigger fires for com03 (which you didn't tell us), then it' even easier:
new.co_creationdate := current_timestamp;
new.co_creationby := current_user;
For details please refer to the manual: http://www.postgresql.org/docs/current/static/plpgsql-trigger.html
That page also contains an example which does exactly what you are trying to achieve