Context: I'm new on a big project, with a huge number of PostgreSQL tables (46) and foreign keys, and no documentation. We work with Java, Spring, Hibernate and TestNG.
Goal: I'm looking for a script that's able to empty the database (to call it before every unit-test case). Unfortunately, I can't spend too much time to find foreign keys in order to empty tables in the correct order.
Question: is there a way to empty (I don't want to drop it) a table without checking constraints on foreign keys ?
As #a_horse_with_no_name commented, using truncate ... cascade for each table will work. The link also provides a function that automatically loops through all the tables in the database.
If you have large tables, with a lot of foreign keys, it will probably take a lot of time to travel through all the foreign key indexes. In that case it would probably be better to dump the schema, drop the database, recreate the database and reload the schema in a batch. There is an example in the link, but it doesn't tell you how to run them. This is a little more refined:
#!/bin/sh
# Run this as the postgres user because you need create database permissions
# I'm assuming you're using a Unix variant,
# the postgresql database is at localhost,
# and that psql trusts the Unix account.
# Otherwise add -h hostname to the command lines
# and set $PGPASSWORD variable before the first command.
pg_dump –s yourdb > /tmp/clean.sql
psql –c "drop database yourdb;"
psql –c "create database yourdb with owner you;"
psql yourdb < /tmp/clean.sql
rm –f /tmp/clean.sql
Related
Im currently using the tool pg_dump to backup my database.
I have a lot of database. I don't know their full name but I know that all DATABASES have a well defined prefix.
I would like to automate the process of backup all my DATABASES, however, i have not found a way to specify pg_dump to dump multiple DATABASE that have the same prefix.
I said database and not table nor schema, because I tried the commands in the pgsql doc which gives the options -n for schemas and -t for tables.
But that's not what I want to do, I want to save all my databases with a defined prefix.
Any help in this matter would be greatly appreciated
I am new to Postgresql and so far I have not found a way to drop a table from specific database. To give some context:
We are doing a synchronization from Oracle to PostgreSQL of 5 tables. In postgres I have a database SoloCopy and the schema is the default public. In the Postgresql instance we have also 2 more databases SoloSynch and postgres (the default one).
What I want to do is to select SoloCopy database and:
DROP TABLE public.table1;
When I do the above DROP statement table1 is deleted only from the database that was selected when opening SQL Query. But I want to specify the database before that and to be irrelevant from where the SQL Query was open. How can I do that?
I found an answer on my own. Setup can be found here:
Psql in Task Scheduler does not run a file script
Basically I needed to use PSQL and with the connection string there I connect to a specific DB, which I can drop the tables from. The details for PGPASSWORD and the .bat file I ended up creating are in the link above. Had to start everything from CMD and switch to psql from there.
I dumped the database with:
pg_dump dsmartgpp -f /data1/master/backup/db.sql
and I copied the database with
nohup psql -h gpnm1 -d dsmartgpp_1 -U gpadmin < /data1/master/backup/db.sql
the log information showed some errors about the function and datatype of postgis, such as
ALTER FUNCTION
ERROR: function "gidx_out" already exists with same argument types.
ALTER FUNCTION
ERROR: type "gidx" already exists
some tables between the two database have the same number or records, but about 1MB difference in size.
That is as expected. Restoring a pg_dump creates a logical copy of the database. In particular, the copy won't be bloated (don't worry – a bit of bloat is healthy).
If you want to avoid these errors, create the new database (dsmartgpp_1) with template0 as template (empty database). You seem to have PostGIS installed in template1, which gets copied into every new database. Or you installed PostGIS in your new database, before importing the dump.
Either way, create the empty database (dsmartgpp_1) and let the dump install the PostGIS functions.
Oh, one more thing, you can use "-f /data1/master/backup/db.sql" instead of the "<" redirect.
If you want to be super careful, also add "-v ON_ERROR_STOP=1". This will stop at the first error.
I have two databases running on the same PostgreSQL 9.3 server (running on Windows).
The two databases have the very same structure: same tables, same keys, same constraints, etc; the only difference is that data gets periodically moved from one to the other.
Think of it like a database with info on clients and orders, where every week data related to clients older than, say, 2 years is moved from one to the other.
Is there a quick way to create tables which combine data from the new and the old database? Something like:
select * from NewDatabase.MyTable
UNION
select * from OldDatabase.MyTable
to be run on about 25 of the 170 tables of the databases.
The 2 key problems I see are:
1) PostgreSQL doesn't really allow cross database queries. Foreign data wrappers or db links (as mentioned here: Possible to perform cross-database queries with postgres?) are very cumbersome to use because (if I understand correctly) I must pre-declare the entire structure of the tables I wish to access.
2) I'd need to find a way to automate this so that I can code it automatically for each of the 25 tables I need, and it runs every week.
I have no preference whatsoever on how this is done (Pentaho-Kettle, command script, sql script, etc.) as long as it can be automated efficiently.
Thanks!
I believe I may have found a way. Of course suggestions and comments are more than welcome. My idea is similar to what was suggested here: I want to restore the database with a different schema (not in the most voted answer, though).
Since I am only interested in tables in the public schema, I did the following:
Backup of the public schema of database1
"C:\Program Files\PostgreSQL\9.3\bin\pg_dump.exe" --host MYSQLHOST --port 5432 --username "JohnDoe" --format custom --section pre-data --section data --section post-data --verbose -n "public" --file "\\mypath\db1.backup" "db1"
Renamed the public schema of db2 and created a new, empty, public schema:
ALTER SCHEMA public RENAME TO public_db2 ;
CREATE SCHEMA public ;
Restore the backup of db1’s public schema into the newcly created (and empty) public schema in db2:
"C:\Program Files\PostgreSQL\9.3\bin\pg_restore.exe" --host MYSQLHOST --port 5432 --username "JohnDoe" --dbname "db2" --section pre-data --section data --section post-data --schema public --verbose "\\mypath\db1.backup"
Renamed the newly imported public schema in db2:
ALTER SCHEMA public RENAME TO public_db1 ;
My syntax above creates a binary (and automatically compressed) backup file. If you have a backup file in text format, you cannot restore just one schema. The only part I haven't managed to automate (yet) is that the shell command prompts me for a password.
I use Windows but I understand the syntax should be the same on other systems, too.
I have a production Postgresql database which consist of 70 tables. Some of them are very big and some are small. And I have my local Postgresql database on my local machine. I want to make some of my local database tables's content be the same as production ones. If I just backup some tables with pgAdmin on production database and then try to restore on my local machine I got constrain errors. Because for example table A has foreign key to table B and so on.
How could I copy some tables from production database and restore normally on my local machine which has already scheme and tables and without constrain errors?
P.s. I couldn't just dump all production database because some of tables are VERY BIG.
Dump complete production database, but without data in case of large tables:
$ pg_dump -t <VERY_BIG_TABLE_NAME> -s
If you want data also, avoid the -s option. Since you will have to repeat this 70 times, quicker solution is dividing tables into schemas:
$ pg_dump -n <SCHEMA_NAME_WITH_VERY_BIG_TABLES> -s
$ pg_dump -n <SCHEMA_NAME_WITH_SMALL_TABLES>
I'm not sure if I understood, but if you got constraint check errors you can disable the foreign key constraints, restore the tables and enable them again.
Re-create table structure in your local database but add DEFERRABLE INITIALLY DEFERRED options to problematic foreign key CONSTRAINTs
Use pg_dump to dump your selected table data from production DB to a file and write at the very beginning: BEGIN;. At the end of the file append:
UPDATE TABLE problem_no_1 SET fkey_column = NULL; for every FK column that causes problems and of course COMMIT; at the end
Execute this file on your local DB