So I'm trying to create a FDW in a running v12 Postgres RDS instance.
The extension already exists, but for some reason I cannot create one. This seems crazy simple having come from an on premises set up.
What am I missing here ?
sourceDatabase=> create extension postgres_fdw ;
ERROR: extension "postgres_fdw" already exists
sourceDatabase=> create foreign data wrapper testwrap ;
ERROR: permission denied to create foreign-data wrapper "testwrap"
HINT: Must be superuser to create a foreign-data wrapper.
sourceDatabase=> \du
List of roles
Role name | Attributes | Member of
---------------------------+------------------------------------------------+--------------------------------------------------------------
rds_superuser | Cannot login | {pg_monitor,pg_signal_backend,rds_replication,rds_password}
rdsadmin | Superuser, Create role, Create DB, Replication+| {}
| Password valid until infinity |
rdsrepladmin | No inheritance, Cannot login, Replication | {}
sourceUser | Create role, Create DB +| {rds_superuser}
| Password valid until infinity |
If you have the postgres_fdw extension installed, you already have a foreign data wrapper called postgres_fdw, so there is no need to create another one.
Use \dew in psql to verify that there is already a foreign data wrapper.
What you need to do now is create a foreign server that contains the connect string to the remote PostgreSQL database, a user mapping that contains the authentication information for that database and foreign tables which point to tables in the remote database.
What Adrian Klaver said is correct. (What is different is whether you want to create a wrapper or a server).
Foreign data wrapper:
This is Postgres's native feature to access the data across the multiple databases including databases like Postgres, Oracle, Redis and etc. The funny fact is you can read your Twitter messages via the foreign-data wrapper. Here is the list of active wrappers.
But you have to write the handlers and etc to access the data.
https://www.postgresql.org/docs/10/sql-createforeigndatawrapper.html
Postgres_FDW:
postgres_fdw is an extension developed under the postgres's contrib library and it's only specific to Postgres to make the Postgres integration much easier. All the necessary handlers and the functions are bundled with this extension.
Creating the foreign data wrappers needs the real superuser permission and it won't work rds_superuser.
The functionality provided by this module overlaps substantially with
the functionality of the older dblink module. But postgres_fdw
provides more transparent and standards-compliant syntax for accessing
remote tables, and can give better performance in many cases.
So please refer to the postgres_fdw documentation instead of the Foreign data wrapper document.
OR better refer to the official AWS RDS doc which is already shared by VynlJunkie(in the comments section)
Related
I have created a file_fdw extension and a corresponding server as superuser.
CREATE EXTENSION file_fdw;
CREATE SERVER myserver FOREIGN DATA WRAPPER file_fdw;
ALTER SERVER myserver OWNER TO nonsuperuser;
I want a non-superuser nonsuperuser to use this server to create a foreign table
CREATE FOREIGN TABLE test (
a text NULL,
b text NULL
)
SERVER myserver
OPTIONS (filename '/home/me/mycsvfile.csv', format 'csv', header 'true', delimiter ';');
Executing this, leads to `only superuser can change options of a file_fdw foreign table
What can I do to enable nonsuperuser to create foreign tables? If possible I would not mind declaring the options as super user.
Only highly privileged users are allowed to access files on the database server, that's why you need high permissions to create a file_fdw foreign table.
From the error message it becomes clear that you are using an old version of PostgreSQL; on more recent versions, the error message would look like:
only superuser or a member of the pg_read_server_files role may specify the filename option of a file_fdw foreign table
So, as an alternative to dealing out superuser privileges, you may add the user to the pg_read_server_files role.
Upgrade PostgreSQL!
I am connecting from a containerized asp.net core 3.1 application running code-first EF core to an Amazon Aurora instance with PostgreSQL compatibility and wish to perform database credential rotation. I have set up a role representing the database owner, and a role representing the current valid login credentials that we will expire and replace with new credentials.
I have followed the suggestion from this blog post:
http://davidhollenberger.com/2017/03/16/postgres-credential-rotation/, which is essentially:
create role db_owner nologin;
create role foo_a with encrypted password ...;
grant db_owner to foo_a;
alter role foo_a set role db_owner;
I understand that whenever foo_a logs in to postgres, their default role is set to db_owner. If I log into the database using psql this seems to work consistently.
However, with EF Core, when connecting using the foo_a database credentials migrating the database to a new schema the object owner of new objects is listed as foo_a.
Example:
List of relations
Schema | Name | Type | Owner
--------+------+-------+----------------------------------------
public | test | table | foo_a
I expected the owner to be db_owner, since foo_a should always log in as db_owner.
Is there something I can do, either from EF Core, or when setting up the postgresql database that will allow us to set default ownership for all objects created by the user to the role group representing our database owner? I do not wish to make these temporary accounts some kind of 'superuser' for the instance, since we have multiple tenants in our database instance, instead I wish to have something similar to a 'dbo' role that has ownership of the database and the temporary users will always connect as the 'dbo' role.
Pooled connections are reset using the DISCARD ALL statement, which in turn resets the session and current user identifiers to be the originally authenticated user name. In other words:
Pooled connection reset (https://www.npgsql.org/doc/performance.html#pooled-connection-reset)
Runs DISCARD ALL (https://www.postgresql.org/docs/current/sql-discard.html)
Which in turn runs SET SESSION AUTHORIZATION DEFAULT (https://www.postgresql.org/docs/8.1/sql-set-session-authorization.html)
During a db migration, this sets the owner of any objects created to the user instead of the dbo role that he automatically inherits by ALTER ROLE... SET ROLE...
Options to resolve this issue will depend on your needs. Here are a few options we considered to resolve this question:
Make the db owner a member of the db user. Supplement this with a script that runs as part of our database/user provisioning and rotation strategy that sets all object ownership to the owner instead of the user. This requires zero code changes for the .net core app, but does seem messy from the perspective of object ownership.
In .Net use an IDBCommandInterceptor to set the appropriate role when we reuse a connection from the pool. This is a more invasive solution affecting your .net core project, but if your requirements involve credential rotation for one or only a few projects it may be practical.
Append the option No reset on close=true to the npgsql connection string. However, be aware that this risks leaking session state if you are using connection pooling. Reference: https://www.npgsql.org/doc/connection-string-parameters.html#performance
Other options, such as running a proxy are also worthy of consideration.
I have a funny question about PostgreSQL database: What happens if the postgres database is dropped?
dropdb postgres worked.
createdb postgres worked too.
psql worked.
But I thought the users would be lost. Yet the old users are still there.
So where are the users stored for the database and which implications does dropping the postgres database have?
PostgreSQL metadata are stored in catalog tables, which are in the pg_catalog schema. These are accessible like regular views and tables.
There are shared system catalog tables which are shared between all databases. These tables are not affected when databases are dropped.
pg_authid, the table where the users are stored, is one of those shared catalogs. This is because in PostgreSQL, users don't belong to a database, but to the whole database cluster.
You can list all shared catalog tables like this:
SELECT relname FROM pg_class
WHERE relisshared AND relkind = 'r';
In the documentation you can find more information about the system catalogs.
When connecting to a Postgres server, you always need to specify which database you want to connect to.
When you set up a new server, you need something to connect to before you can run your first CREATE DATABASE statement.
That's all the postgres database is: a dummy database to use as a connection target for admin commands. There's no data in there, and you're free to drop it and use a different one instead (though whoever inherits your system will probably not thank you for it...).
As gil.fernandes said in his answer, server-wide objects like users are accessible from every database, but aren't stored inside any database in particular.
I am new on Amazon RDS.
I'm trying to create the tablespaces of my application, but I can't find where I should store it.
Is it possible to create tablespaces on PostgreSQL RDS?
I didn't find RDS documentation about this, but apparently Postgres tablespaces work fine on RDS. You don't have direct access to the underlying EBS volume that RDS is using, but if you try a CREATE TABLESPACE command with some arbitrary directory:
CREATE TABLESPACE testspace LOCATION '/foo/bar/baz';
You can see that the tablespace gets created like so:
mydb=> \db
List of tablespaces
Name | Owner | Location
------------+------------+-------------------------------------------
pg_default | rdsadmin |
pg_global | rdsadmin |
testspace | db_user | /rdsdbdata/db/base/tablespace/foo/bar/baz
(3 rows)
And you should be able to create tables in that tablespace just fine:
mydb=> CREATE TABLE test (a int) TABLESPACE testspace ;
CREATE TABLE
At the time of writing in 2019 RDS does support PostgreSQL tablespaces, although not for IO splitting or isolation (as all tablespaces will be created on the same volume). AWS documentation for this feature can be found here.
Things like log-shipping/standby servers heavily depend on tablespace location. I wouldn't risk it and blindly use a location I don't have access to and not fully aware of(as Josh suggested)
I guess we just all have to admit it that much as we like AWS services, there are things where they lag behind, especially with regards to SaaS solutions to external\independent products(RDS, Docker orchestration on ECS, etc.)
UPDATE:
If this post is correct, RDS replication for Postgres does depend on streaming replication, which makes any manipulations with tablespace paths very dangerous unless you can find it in their official docs or via enterprise support that such strategy will always work as you expect.
What are the default databases in PostgreSQL at the time of installation?
template1: the "default" database, which is copied when you perform "create database foo"
template0: the "minimal default" database, which serves essentially the same purpose, but typically used to create databases when restoring dumps which might already have the extra objects that are in template1, or to create a database that uses a different character encoding to the server default (template0 should only contain ASCII characters in strings)
postgres: an "administrative" database, which clients can assume exists to connect to merely to list which databases are available etc. Also, for example, pgAdmin will install the pg_agent schema in this database.
Apparently there is a database "postgres" that is created by default on each postgresql server installation.
It appears that it does not really have a well-defined purpose. According to the docs:
Creating a database cluster consists of creating the directories in
which the database data will live, generating the shared catalog
tables (tables that belong to the whole cluster rather than to any
particular database), and creating the template1 and postgres
databases. When you later create a new database, everything in the
template1 database is copied. (Therefore, anything installed in
template1 is automatically copied into each database created later.)
The postgres database is a default database meant for use by users,
utilities and third party applications.
(Source: http://www.postgresql.org/docs/8.2/static/app-initdb.html )