I am trying to backup my database over the network with mysqldump and rsync.
I wanted to ask if there is any way to know if the database has been modified since the last time I did my old dump, before doing a new dump or update the old one.
Thank you.
You could access each database that might need to be dumped and ask for the last modified time. It's available through the information_schema database:
SELECT UPDATE_TIME
FROM information_schema.tables
WHERE TABLE_SCHEMA = 'dbname'
AND TABLE_NAME = 'tabname'
Also see here.
Related
I have two 'similar' databases and I'm trying to replicate role membership between them.
Replicating the full database is not an option and they are quite big (about nine hours to restore).
I've tried with pg_dumpall --globals-only but it only creates the users and roles, there is nothing about role membership.
On the other hand, pg_dump works with single databases, and I can't find any information about dumping the roles membership.
I probably end writing some script that queries for membership information in one database and then 'grants' this information in the other one.
But I would like to know if there is an easier way to do this.
You're right, pg_dumpall --globals-only includes role membership.
The problem is that I was migrating and merging two different versions of Postgres. I needed some intermediate clusters to upgrade some extensions, but I forgot restoring roles to them.
After upgrading the extensions, I dumped the data and restored it in the final database without the role information :-(
The whole process takes about 14 hours, so, instead of starting the process again, I wrote a small script to copy the roles:
psql --host SOURCE_HOST --tuples-only --file get_role_grants.sql | psql -v ON_ERROR_STOP=1
With the contents of the file 'get_role_grants' being:
select concat('grant ', c.rolname, ' to ', a.rolname, ';') as grantcommand
from pg_roles a
inner join pg_auth_members b on a.oid=b.member
inner join pg_roles c on b.roleid=c.oid
order by a.rolname, c.rolname;
I have two postgres databases on the same server, a live one and another one which I use as a test database.
Periodically I need to copy my live database (both structure of the tables and their data) into the test one, but everything I've found online only copies table by table. Since I often create new tables in my live db I can't do this, otherwise I'd have to update the job every time.
Anybody knows how can I pull the whole live postgres db into the test postgres one?
Thanks!
You want to use pg_dump and pg_restore.
Example usage would be:
$ pg_dump -Fc <database_name> > dumpfile
$ pg_restore <another_database_name> < dumpfile
If you want to do this for all database you might want to consider pg_dumpall
A bit more can be found in https://www.postgresql.org/docs/current/backup-dump.html
I have a funny question about PostgreSQL database: What happens if the postgres database is dropped?
dropdb postgres worked.
createdb postgres worked too.
psql worked.
But I thought the users would be lost. Yet the old users are still there.
So where are the users stored for the database and which implications does dropping the postgres database have?
PostgreSQL metadata are stored in catalog tables, which are in the pg_catalog schema. These are accessible like regular views and tables.
There are shared system catalog tables which are shared between all databases. These tables are not affected when databases are dropped.
pg_authid, the table where the users are stored, is one of those shared catalogs. This is because in PostgreSQL, users don't belong to a database, but to the whole database cluster.
You can list all shared catalog tables like this:
SELECT relname FROM pg_class
WHERE relisshared AND relkind = 'r';
In the documentation you can find more information about the system catalogs.
When connecting to a Postgres server, you always need to specify which database you want to connect to.
When you set up a new server, you need something to connect to before you can run your first CREATE DATABASE statement.
That's all the postgres database is: a dummy database to use as a connection target for admin commands. There's no data in there, and you're free to drop it and use a different one instead (though whoever inherits your system will probably not thank you for it...).
As gil.fernandes said in his answer, server-wide objects like users are accessible from every database, but aren't stored inside any database in particular.
I am having an issue with pg_restore --clean not clearing the database.
Or do I misunderstand what the --clean does, I am expecting it to truncate the database tables and reinitialize the indexes/primary keys.
I am using 9.5 on rds
This is the full command we use
pg_restore --verbose --clean --no-acl --no-owner -h localhost -U superuser -d mydatabase backup.dump
Basically what is happening is this.
I do a nightly backup of my production db, and restore it to an analytics db for the analyst to churn and run their reports.
I found out recently that the rails application used to view the reports was complaining that the primary keys were missing from the restored analytics database.
So I started investigating the production db, the analytics db etc. Which was when I realized that multiple rows with the same primary key existed in the analytics database.
I ran a few short experiments and realized that every time the pg_restore script is run it inserts duplicate data into the tables, this leads me to think that the --clean is not dropping and restoring the data. Because if I were to drop the schema beforehand, I don't get duplicate data.
To remove all tables from a database (but keep the database itself), you have two options.
Option 1: Drop the entire schema
You will need to re-create the schema and its permissions. This is usually good enough for development machines only.
DROP SCHEMA public CASCADE;
CREATE SCHEMA public;
GRANT ALL ON SCHEMA public TO postgres;
GRANT ALL ON SCHEMA public TO public;
Applications usually use the "public" schema. You may encounter other schema names when working with a (legacy) application's database.
Note that for Rails applications, dropping and recreating the database itself is usually fine in development. You can use bin/rake db:drop db:create for that.
Option 2: Drop each table individually
Prefer this for production or staging servers. Permissions may be managed by your operations team, and you do not want to be the one who messed up permissions on a shared database cluster.
The following SQL code will find all table names and execute a DROP TABLE statement for each.
DO $$ DECLARE
r RECORD;
BEGIN
FOR r IN (SELECT tablename FROM pg_tables WHERE schemaname = current_schema()) LOOP
EXECUTE 'DROP TABLE IF EXISTS ' || quote_ident(r.tablename) || ' CASCADE'; -- DROP TABLE IF EXISTS instead DROP TABLE - thanks for the clarification Yaroslav Schekin
END LOOP;
END $$;
Original:
https://makandracards.com/makandra/62111-how-to-drop-all-tables-in-postgresql
I'm trying to rename a database in my Redshift cluster.
You cannot rename the database when you're connected to it so I've created a temporary database, reconnected with SQL Workbench to the temporary db and issued:
ALTER DATABASE olddb RENAME to newdb;
I get an error stating ERROR: database "olddb" is being accessed by other users [SQL State=55006]
I've checked who is connected and there appear to be some connections from user rdsdb to the database. I assume this is a service account that AWS Redshift use to perform maintenance tasks etc.
How can I rename the database when this superuser is connected?
Many thanks.
You cannot alter the name of (or delete!) the database that is created during the initial cluster creation. I don't believe this is mentioned in the docs but I've confirmed it with them.
We can change the database name which is already created.
Detailed steps on how to do
Connect to the old database and create a new database if you do not have another one already.
create database databasebasename
In this example, I will call the databasename as 'newdb'.
Connect to newdb using connecting string as, jdbc:redshift://.us-east-1.redshift.amazonaws.com:8192/newdb, with the same password and username of your superuser (or the other eligible users as mentioned above).
Now you can alter the database name. Substitute 'database_name_new' with the desired databasename.
alter database old-db-name rename to database_name_new;
If there are any active sessions, you'll have to kill them. To find the pid of active sessions:
select * from STV_SESSIONS where user_name='rdsdb';
Then to kill a session:
SELECT
pg_terminate_backend(<pid>)
FROM
pg_stat_activity
WHERE
-- don't kill my own connection!
procpid <> pg_backend_pid()
-- don't kill the connections to other databases
AND datname = '<old-db-name>';
Once complete, you can connect back to that new database using the new name in the connection string as
jdbc:redshift://<cluser-id>.us-east-1.redshift.amazonaws.com:8192/database_name_new
You can delete the temporary 'newdb'.
drop database databasebasename
That's possible now -- I just renamed the database that was created during the initial cluster creation.
We had a similar situation.
Step 1: Connect to the database which is not the one you are trying to rename. Check the same by executing SELECT CURRENT_DATABASE();.
Step 2: Execute the query below -
SELECT
ss.*, 'select pg_terminate_backend('||process||');'
FROM
stv_sessions ss
ORDER BY
db_name;
The output of the query will have a column at the end with the select statements. Execute those to kill the sessions.
Step 3(Optional): If you are not the owner of the database try to modify the ownership of the database -
ALTER DATABASE <database to be renamed>
OWNER TO <user which is going to do the rename>;
Step 4: Rename the database