I have two Postgres 9.3.5 instances in RDS, both in one security group that allows all inbound traffic from within the security group and all outbound traffic. I'm trying to set up one database to be able to select from a few tables from the other via postgres_fdw.
I've created the server -
create server master
foreign data wrapper postgres_fdw
OPTIONS (dbname 'main',
host 'myinstance.xxxxx.amazonaws.com');
as well as the requisite user mapping and foreign table -
create foreign table condition_fdw (
cond_id integer,
cond_name text
) server master options(table_name 'condition', schema_name 'data');
However, a simple select count(*) from condition_fdw gives me
ERROR: could not connect to server "master"
DETAIL: could not connect to server: Connection timed out
Is the server running on host "myinstance.xxxxxx.amazonaws.com" (xx.xx.xx.xx) and accepting
TCP/IP connections on port 5432?
I can connect to both databases via psql from an EC2 instance. I know until recently RDS didn't support postgres_fdw, but I'm running the newer versions that do.
In the create server statement, I have tried replacing "myinstance.xxxxxx.amazonaws.com" with the IP address it resolves to, no luck.
Any ideas?
Further Testing
I installed postgres on an ec2 instance with the same security group, foreign tables to the master server behave as expected.
postgres_fdw between databases on the same RDS instance works.
This all leads me to think it must some issue with outgoing connections from postgres_fdw on my Postgres RDS instance.
To get postgres_fdw to work between two instances in AWS RDS (with VPC) I had to:
Enable custom_dns_resolution https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.html#Appendix.PostgreSQL.CommonDBATasks.CustomDNS
Add the public IP of the querying database in the inbound rules of the Security group OR add the internal IP of the RDS instance as server host. https://forums.aws.amazon.com/thread.jspa?threadID=235154
The first one is documented pretty well but I could not get it to work until stumbling over the second one.
It would appear that Amazon does not allow outgoing connections from RDS instances, so until that changes using postgres_fdw across RDS instances is not possible. I'll have to run an ec2 instance as my postgres server in order to use a foreign table to a database on another server.
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.RDSSecurityGroups.html
I experienced a similar problem and found that postgres_fdw would only work in the EU-WEST-1 RDS region when the master and remote instances were in the same availability zone and on 9.3.5, if you crossed AZs it wouldn't connect. My security group was 5432 TCP inbound only and all outbound.
If you are in the same availability zone, there is no reason why it wouldn't work.
Related
My plan is to migrate from SQL Server to Postgresql, hosted on AWS. My Postgresql DB is in a VPC and the security group attached to it has the following inbound rules:
The first one is for my EC2 instance (where my server-side application is running) to connect to the DB, and the second one is the home IP address so that I can connect to it via pgAdmin.
This is what I get (I don't know what to fill in at 'Maintenance database', so I left it at postgres.
What am I missing? I need help with this.
I have noticed that for some combinations of VPN, local router, etc. the IP address the security group filled in for 'my IP' was not the one that RDS recognized as trying to connect. I found that curl --silent ifconfig.me returns the IP address that works the best for the security group.
I'm trying to create a subscription.
It works well when I create it on a PostgreSQL instance or cluster (to PostgreSQL).
But when I try to do logical replication from PostgreSQL to AWS Aurora PostgreSQL I see the following error:
ERROR: could not connect to the publisher: could not connect to server: No route to host
Is the server running on host "my-db.dfsfdsfsdfd.us-east-1.rds.amazonaws.com" (10.2.5.7) and accepting
TCP/IP connections on port 5432?
Some notes:
I updated the parameter group to enable replication: rds.logical_replication (from 0 to 1)
The RDS policy has enough permissions.
Both DBs are in the same VPC, same subnets and share the security group
I can connect to both DBs, and as I said, subscription works if instead of using Aurora I use PostgreSQL
Any idea why this could be happening?
Thanks!
I am brand new to RDS and have only used postgres through Rails and/or Heroku for the most part, so not that deep into database management. All I am trying to do is verify I can connect to the RDS instance I just created on AWS, but it is hanging, psql reporting this after hanging for about 30 seconds or a few minutes I guess:
$ psql postgresql://myuser:mypass#myawshost.rds.amazonaws.com:5432/my-db-name
psql: error: could not connect to server: Operation timed out
Is the server running on host "myawshost.rds.amazonaws.com" (<the ip address>) and accepting
TCP/IP connections on port 5432?
How do I connect to my AWS RDS instance from localhost?
I am sure I have the correct username and password.
I am sure the host is correct.
I am not sure if I should be including the port.
I am not sure if I am supposed to put a DB name, as under the "Configuration" tab in the RDS admin console for my database, it says DB instance ID: my-db-name, Engine version: 13.3, DB name: -, I am not sure if that - is my actual postgres db name or my-db-name is....
I tried this with the db name and it still hangs:
$ psql postgresql://myuser:mypass#myawshost.rds.amazonaws.com:5432/-
I edited the security group which is linked under the VPC security groups section of the Connectivity & security tab of https://console.aws.amazon.com/rds/home, so it allows all incoming connections. I also tried limiting to just my IP address as incoming connections.
Any help would be appreciated, thanks. Not sure why it would just be hanging. I have used a local version of postgres just fine, but connecting to postgres RDS is not working.
Underneath the Connectivity and Security > Security section I just noticed it says Public accessibility: No. Must I enable something else? There is also one VPC, one Subnet group, and several Subnets, which I don't know too much about, must I do something there?
Finally, "status" says Available with a green light, so things seem fine there.
I found a useful link in AWS knowledge center, I hope this will help you.
Btw if your RDS is deployed in a public subnet, yes you need to enable if you want to access RDS over the internet.
My DB instance is in a public subnet, and I can't connect to it over the internet from my local computer
This issue can occur when the Publicly Accessible property of the DB instance is set to No. To check whether a DB instance is publicly accessible, you can use the Amazon RDS Console or the AWS CLI.
To change the Publicly Accessible property of the Amazon RDS instance to Yes:
Verify that your VPC has an internet gateway attached to it. Make sure that the inbound rules for the security group allow connections.
Open the Amazon RDS console.
Choose Databases from the navigation pane, and then select the DB instance.
Choose Modify.
Under Connectivity, extend the Additional configuration section, and then choose Publicly accessible.
Choose Continue.
Choose Modify DB Instance.
Note: You don't need to choose Apply Immediately. For more information about how Apply Immediately can affect downtime, see Using the Apply Immediately parameter.
https://aws.amazon.com/premiumsupport/knowledge-center/rds-connectivity-instance-subnet-vpc/
I am trying to connect to my RDS database from my computer with a python script using psycopg2.
python code:
import psycopg2
from db_credentials import *
import logging
def get_psql_conn():
conn = psycopg2.connect(dbname=DB_NAME, user=DB_USER, password=DB_PASS, host=DB_HOST)
logging.info("connected to DB!")
return conn
I get the following error:
psycopg2.OperationalError: could not connect to server: Operation timed out
Is the server running on host ********* and accepting
TCP/IP connections on port 5432?
My security groups assigned to the RDS database:
SG 1:
SG 2:
Now i tried to make a security group which allows my computer IP to access the DB.
SG 3:
I can connect to the DB from my ec2 instances, running the same python script as above. This seemingly has to do with the 2nd security group, as when i remove it, i can no longer connect from my ec2 instances either. It then throws the same error i get when trying to connect from my computer.
I have little understanding of RDS or security groups, i just followed internet tutorials, but seemingly couldnt make much sense out of it.
Any help is greatly appreciated! Thanks
When accessing an Amazon RDS database from the Internet, the database needs to be configured for Publicly Accessible = Yes.
This will assign a Public IP address to the database instance. The DNS Name of the instance will also resolve to the public IP address.
For good security on publicly-accessible databases, ensure that the Security Group only permits access from your personal IP address.
I am trying to use the PostgreSQL foreign data wrapper (postgres_fdw) extension from an Azure DB for PostgreSQL instance. The foreign data wrapper should be supported as mentioned in Azure DB for PostgreSQL: https://learn.microsoft.com/en-us/azure/postgresql/concepts-extensions
I manage to create a server and a mapping for the user, but when I try to import a table or a schema I have the following error:
ERROR: could not connect to server "<server with public hostname on AWS>"
DETAIL: could not translate host name "<server with public hostname on AWS>" to address: Unknown host
When replacing the hostname with a public IP I get the following:
ERROR: could not connect to server "<SERVER NAME>"
DETAIL: could not connect to server: Network is down (0x00002742/10050)
Is the server running on host "<public IP of the server" and accepting
TCP/IP connections on port 5432?
In the firewall options of the Azure DB for PostgreSQL instance, I already have the AllowAllIps rule starting 0.0.0.0 and ending 255.255.255.255. is there any thing else to configure to get access to servers outside of Azure, DNS or firewall configuration?
Not entirely sure about this, but we were trying to do something like this the other day, and stumbled upon this (somewhat disheartening) quote:
"Currently, outbound connections from Azure Database for PostgreSQL are
not supported, except for connections to other Azure Database for
PostgreSQL servers."
Source: https://learn.microsoft.com/en-us/azure/postgresql/concepts-extensions#dblink-and-postgres_fdw