I want to transfer 1 or more Redshift Database from a Redshift cluster to another cluster in the same region.
The existing target cluster has the same schema.
Redshift Cluster Source
|
|_ DB_1
|_ DB_2
|_ DB_3
Redshift Cluster Target
|
|_ DB_A
|_ DB_B
|_ DB_C
After transfer...
Redshift Cluster Source
|
|_ DB_1
Redshift Cluster Target
|
|_ DB_A
|_ DB_B
|_ DB_C
|_ DB_2
|_ DB_3
So far I've only read of UNLOAD & COPY command, but that only works for tables, and I have a lot of tables for each database. Is there a more efficient way to transfer database? Or an existing tool?
Related
I am trying to follow the Debezium tutorial for Postgres and set up a replication user. I am creating the replication user as follows:
CREATE ROLE replication_role WITH REPLICATION LOGIN;
CREATE USER debezium WITH PASSWORD 'my-secret-pw';
GRANT replication_role TO debezium;
CREATE ROLE replication_group WITH LOGIN;
GRANT replication_group TO postgres;
GRANT replication_group TO debezium;
ALTER TABLE person OWNER TO replication_group;
and it sets up the replication user as follows:
List of roles
Role name | Attributes | Member of
-------------------+------------------------------------------------------------+--------------------------------------
debezium | | {replication_role,replication_group}
postgres | Superuser, Create role, Create DB, Replication, Bypass RLS | {replication_group}
replication_group | | {}
replication_role | Replication | {}
When I try to start the Postgres source connector, I get the error as follows:
io.debezium.DebeziumException: Creation of replication slot failed
...
Caused by: org.postgresql.util.PSQLException: FATAL: must be superuser or replication role to start walsender
How do I fix the permissions issue so that I can start replicating from Postgres?
The replication attribute is not inherited. That attribute must be set directly on the role which will use it. You say you are following the tutorial, but as far as I can tell you are not. You seem to be confusing the permissions/attributes needed to create publications, with the ones needed to stream the publications.
I am trying to spin a postgres container from docker-compose, copying *.sql files to /docker-entrypoint-initdb.d in order to create the database and the schema.
The logs from the docker-compose run:
/usr/local/bin/docker-entrypoint.sh: sourcing /docker-entrypoint-initdb.d/02-schema.sql
db_1 | CREATE DATABASE
db_1 | GRANT
db_1 | SET
db_1 | CREATE TABLE
db_1 | CREATE INDEX
db_1 | CREATE TABLE
db_1 | CREATE TABLE
The database is created, but non of the tables is created. Below is my schema.sql file
CREATE DATABASE sample_db OWNER user1;
GRANT ALL PRIVILEGES ON DATABASE sample_db TO user1;
CREATE TABLE PUBLIC.PROFILES(
USER_ID VARCHAR(50) PRIMARY KEY,
NAME VARCHAR(50) NOT NULL
);
I used to do the same with my sql container, adding : USE sample_db, but I couldn't find the equivalent postgres command.
Have you use the POSTGRES_DB environment variable in the docker file? If it isn't set it will use the POSTGRES_USER as database name
Check the environment variable in the documentation https://hub.docker.com/_/postgres
Where can I get Postgresql unique installation ID?
like...
SELECT dbid from v$database; (in oracle)
I need it to identify my different database installation across different sites.
Getting MAC id of server is a way but if there is a way to get it from postgresql it would be preferred.
if you have replication slot available, you can use IDENTIFY_SYSTEM:
MacBook-Air:~ vao$ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
Timing is on.
Pager usage is off.
systemid | timeline | xlogpos | dbname
---------------------+----------+------------+----------
6401169876745218982 | 1 | 1/7A470D80 | postgres
(1 row)
I am new on Amazon RDS.
I'm trying to create the tablespaces of my application, but I can't find where I should store it.
Is it possible to create tablespaces on PostgreSQL RDS?
I didn't find RDS documentation about this, but apparently Postgres tablespaces work fine on RDS. You don't have direct access to the underlying EBS volume that RDS is using, but if you try a CREATE TABLESPACE command with some arbitrary directory:
CREATE TABLESPACE testspace LOCATION '/foo/bar/baz';
You can see that the tablespace gets created like so:
mydb=> \db
List of tablespaces
Name | Owner | Location
------------+------------+-------------------------------------------
pg_default | rdsadmin |
pg_global | rdsadmin |
testspace | db_user | /rdsdbdata/db/base/tablespace/foo/bar/baz
(3 rows)
And you should be able to create tables in that tablespace just fine:
mydb=> CREATE TABLE test (a int) TABLESPACE testspace ;
CREATE TABLE
At the time of writing in 2019 RDS does support PostgreSQL tablespaces, although not for IO splitting or isolation (as all tablespaces will be created on the same volume). AWS documentation for this feature can be found here.
Things like log-shipping/standby servers heavily depend on tablespace location. I wouldn't risk it and blindly use a location I don't have access to and not fully aware of(as Josh suggested)
I guess we just all have to admit it that much as we like AWS services, there are things where they lag behind, especially with regards to SaaS solutions to external\independent products(RDS, Docker orchestration on ECS, etc.)
UPDATE:
If this post is correct, RDS replication for Postgres does depend on streaming replication, which makes any manipulations with tablespace paths very dangerous unless you can find it in their official docs or via enterprise support that such strategy will always work as you expect.
I know this is a very basic question, but i could not find a answer around google.
But what is the main difference between those two tablespaces?
From Documentation:
Two tablespaces are automatically created by initdb. The pg_global tablespace is used for shared system catalogs. The pg_default tablespace is the default tablespace of the template1 and template0 databases (and, therefore, will be the default tablespace for other databases as well, unless overridden by a TABLESPACE clause in CREATE DATABASE).