I'm trying to create a subscription.
It works well when I create it on a PostgreSQL instance or cluster (to PostgreSQL).
But when I try to do logical replication from PostgreSQL to AWS Aurora PostgreSQL I see the following error:
ERROR: could not connect to the publisher: could not connect to server: No route to host
Is the server running on host "my-db.dfsfdsfsdfd.us-east-1.rds.amazonaws.com" (10.2.5.7) and accepting
TCP/IP connections on port 5432?
Some notes:
I updated the parameter group to enable replication: rds.logical_replication (from 0 to 1)
The RDS policy has enough permissions.
Both DBs are in the same VPC, same subnets and share the security group
I can connect to both DBs, and as I said, subscription works if instead of using Aurora I use PostgreSQL
Any idea why this could be happening?
Thanks!
Related
I'm exploring AWS RDS for the first time.
I have two MySQL instances. The first is the traditional MySQL RDS and the second is an Aurora MySQL instance.
Both RDSs have the same region, VPC, and security groups.
I can successfully connect to the traditional MySQL RDS from MySQL workbench on my localhost (Mac OS Monterey FWIW)
I cannot connect to the Aurora Instance. When I attempt to do this I get message "Unable to connect to localhost"
Clearly the hostname I'm using is not localhost (see attached). I've tried connecting from other SQL clients without any success. For example, from a windows box using ODBC I get "unable to connect to " without any other explanation.
I am trying to connect to my RDS database from my computer with a python script using psycopg2.
python code:
import psycopg2
from db_credentials import *
import logging
def get_psql_conn():
conn = psycopg2.connect(dbname=DB_NAME, user=DB_USER, password=DB_PASS, host=DB_HOST)
logging.info("connected to DB!")
return conn
I get the following error:
psycopg2.OperationalError: could not connect to server: Operation timed out
Is the server running on host ********* and accepting
TCP/IP connections on port 5432?
My security groups assigned to the RDS database:
SG 1:
SG 2:
Now i tried to make a security group which allows my computer IP to access the DB.
SG 3:
I can connect to the DB from my ec2 instances, running the same python script as above. This seemingly has to do with the 2nd security group, as when i remove it, i can no longer connect from my ec2 instances either. It then throws the same error i get when trying to connect from my computer.
I have little understanding of RDS or security groups, i just followed internet tutorials, but seemingly couldnt make much sense out of it.
Any help is greatly appreciated! Thanks
When accessing an Amazon RDS database from the Internet, the database needs to be configured for Publicly Accessible = Yes.
This will assign a Public IP address to the database instance. The DNS Name of the instance will also resolve to the public IP address.
For good security on publicly-accessible databases, ensure that the Security Group only permits access from your personal IP address.
I am planning to have a timescale database running in an EC2 or Lightsail instance. I would like to be able to connect to and run queries on this timescale database from a Lambda function to insert data into and read data from the DB.
I know timescaleDB is a Postgres plugin and there are plenty of articles online documenting the process of connecting to a Postgres DB running inside of AWS RDS from a lambda, but I can't seem to find any describing how I would connect to one running in an EC2 or Lightsail instance.
Question: How do I connect to a timescaleDB running in an EC2 or Lightsail instance from a lambda function?
I'd say the answer is the same as for how to connect to RDS, as documented here:
https://docs.aws.amazon.com/lambda/latest/dg/vpc-rds.html
This answer also gives a good example how to connect to a PostgreSQL RDS, but instead of using the rds_config, you'll need to specify the hostname/ip in such a way that they point to your EC2 instance.
The differences being that you will need to specify the hostname/ip and other connection details to point to the EC2 instance. For example, if your EC2 instances
import sys, logging, psycopg2
host = "ec2-3-14-229-184.us-east-2.compute.amazonaws.com"
name = "demo_user"
password = "p#assword"
db_name = "demo"
try:
conn = psycopg2.connect(host=host,
database=db_name,
user=name,
password=password)
cur = conn.cursor()
except:
logger.error("ERROR: Unexpected error: Could not connect to Postgresql instance.")
sys.exit()
I am trying to use the PostgreSQL foreign data wrapper (postgres_fdw) extension from an Azure DB for PostgreSQL instance. The foreign data wrapper should be supported as mentioned in Azure DB for PostgreSQL: https://learn.microsoft.com/en-us/azure/postgresql/concepts-extensions
I manage to create a server and a mapping for the user, but when I try to import a table or a schema I have the following error:
ERROR: could not connect to server "<server with public hostname on AWS>"
DETAIL: could not translate host name "<server with public hostname on AWS>" to address: Unknown host
When replacing the hostname with a public IP I get the following:
ERROR: could not connect to server "<SERVER NAME>"
DETAIL: could not connect to server: Network is down (0x00002742/10050)
Is the server running on host "<public IP of the server" and accepting
TCP/IP connections on port 5432?
In the firewall options of the Azure DB for PostgreSQL instance, I already have the AllowAllIps rule starting 0.0.0.0 and ending 255.255.255.255. is there any thing else to configure to get access to servers outside of Azure, DNS or firewall configuration?
Not entirely sure about this, but we were trying to do something like this the other day, and stumbled upon this (somewhat disheartening) quote:
"Currently, outbound connections from Azure Database for PostgreSQL are
not supported, except for connections to other Azure Database for
PostgreSQL servers."
Source: https://learn.microsoft.com/en-us/azure/postgresql/concepts-extensions#dblink-and-postgres_fdw
I have two Postgres 9.3.5 instances in RDS, both in one security group that allows all inbound traffic from within the security group and all outbound traffic. I'm trying to set up one database to be able to select from a few tables from the other via postgres_fdw.
I've created the server -
create server master
foreign data wrapper postgres_fdw
OPTIONS (dbname 'main',
host 'myinstance.xxxxx.amazonaws.com');
as well as the requisite user mapping and foreign table -
create foreign table condition_fdw (
cond_id integer,
cond_name text
) server master options(table_name 'condition', schema_name 'data');
However, a simple select count(*) from condition_fdw gives me
ERROR: could not connect to server "master"
DETAIL: could not connect to server: Connection timed out
Is the server running on host "myinstance.xxxxxx.amazonaws.com" (xx.xx.xx.xx) and accepting
TCP/IP connections on port 5432?
I can connect to both databases via psql from an EC2 instance. I know until recently RDS didn't support postgres_fdw, but I'm running the newer versions that do.
In the create server statement, I have tried replacing "myinstance.xxxxxx.amazonaws.com" with the IP address it resolves to, no luck.
Any ideas?
Further Testing
I installed postgres on an ec2 instance with the same security group, foreign tables to the master server behave as expected.
postgres_fdw between databases on the same RDS instance works.
This all leads me to think it must some issue with outgoing connections from postgres_fdw on my Postgres RDS instance.
To get postgres_fdw to work between two instances in AWS RDS (with VPC) I had to:
Enable custom_dns_resolution https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.html#Appendix.PostgreSQL.CommonDBATasks.CustomDNS
Add the public IP of the querying database in the inbound rules of the Security group OR add the internal IP of the RDS instance as server host. https://forums.aws.amazon.com/thread.jspa?threadID=235154
The first one is documented pretty well but I could not get it to work until stumbling over the second one.
It would appear that Amazon does not allow outgoing connections from RDS instances, so until that changes using postgres_fdw across RDS instances is not possible. I'll have to run an ec2 instance as my postgres server in order to use a foreign table to a database on another server.
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.RDSSecurityGroups.html
I experienced a similar problem and found that postgres_fdw would only work in the EU-WEST-1 RDS region when the master and remote instances were in the same availability zone and on 9.3.5, if you crossed AZs it wouldn't connect. My security group was 5432 TCP inbound only and all outbound.
If you are in the same availability zone, there is no reason why it wouldn't work.