Connect to AWS RDS database via psycopg2 - postgresql

I am trying to connect to my RDS database from my computer with a python script using psycopg2.
python code:
import psycopg2
from db_credentials import *
import logging
def get_psql_conn():
conn = psycopg2.connect(dbname=DB_NAME, user=DB_USER, password=DB_PASS, host=DB_HOST)
logging.info("connected to DB!")
return conn
I get the following error:
psycopg2.OperationalError: could not connect to server: Operation timed out
Is the server running on host ********* and accepting
TCP/IP connections on port 5432?
My security groups assigned to the RDS database:
SG 1:
SG 2:
Now i tried to make a security group which allows my computer IP to access the DB.
SG 3:
I can connect to the DB from my ec2 instances, running the same python script as above. This seemingly has to do with the 2nd security group, as when i remove it, i can no longer connect from my ec2 instances either. It then throws the same error i get when trying to connect from my computer.
I have little understanding of RDS or security groups, i just followed internet tutorials, but seemingly couldnt make much sense out of it.
Any help is greatly appreciated! Thanks

When accessing an Amazon RDS database from the Internet, the database needs to be configured for Publicly Accessible = Yes.
This will assign a Public IP address to the database instance. The DNS Name of the instance will also resolve to the public IP address.
For good security on publicly-accessible databases, ensure that the Security Group only permits access from your personal IP address.

Related

Can't connect to my AWS Postgresql instance via local pgAdmin

My plan is to migrate from SQL Server to Postgresql, hosted on AWS. My Postgresql DB is in a VPC and the security group attached to it has the following inbound rules:
The first one is for my EC2 instance (where my server-side application is running) to connect to the DB, and the second one is the home IP address so that I can connect to it via pgAdmin.
This is what I get (I don't know what to fill in at 'Maintenance database', so I left it at postgres.
What am I missing? I need help with this.
I have noticed that for some combinations of VPN, local router, etc. the IP address the security group filled in for 'my IP' was not the one that RDS recognized as trying to connect. I found that curl --silent ifconfig.me returns the IP address that works the best for the security group.

Error trying to do logical replication between PostgreSQL and AWS Aurora PostgreSQL could not connect to the publisher: could not connect to server

I'm trying to create a subscription.
It works well when I create it on a PostgreSQL instance or cluster (to PostgreSQL).
But when I try to do logical replication from PostgreSQL to AWS Aurora PostgreSQL I see the following error:
ERROR: could not connect to the publisher: could not connect to server: No route to host
Is the server running on host "my-db.dfsfdsfsdfd.us-east-1.rds.amazonaws.com" (10.2.5.7) and accepting
TCP/IP connections on port 5432?
Some notes:
I updated the parameter group to enable replication: rds.logical_replication (from 0 to 1)
The RDS policy has enough permissions.
Both DBs are in the same VPC, same subnets and share the security group
I can connect to both DBs, and as I said, subscription works if instead of using Aurora I use PostgreSQL
Any idea why this could be happening?
Thanks!

How to connect to RDS Postgres instance with psql

I am brand new to RDS and have only used postgres through Rails and/or Heroku for the most part, so not that deep into database management. All I am trying to do is verify I can connect to the RDS instance I just created on AWS, but it is hanging, psql reporting this after hanging for about 30 seconds or a few minutes I guess:
$ psql postgresql://myuser:mypass#myawshost.rds.amazonaws.com:5432/my-db-name
psql: error: could not connect to server: Operation timed out
Is the server running on host "myawshost.rds.amazonaws.com" (<the ip address>) and accepting
TCP/IP connections on port 5432?
How do I connect to my AWS RDS instance from localhost?
I am sure I have the correct username and password.
I am sure the host is correct.
I am not sure if I should be including the port.
I am not sure if I am supposed to put a DB name, as under the "Configuration" tab in the RDS admin console for my database, it says DB instance ID: my-db-name, Engine version: 13.3, DB name: -, I am not sure if that - is my actual postgres db name or my-db-name is....
I tried this with the db name and it still hangs:
$ psql postgresql://myuser:mypass#myawshost.rds.amazonaws.com:5432/-
I edited the security group which is linked under the VPC security groups section of the Connectivity & security tab of https://console.aws.amazon.com/rds/home, so it allows all incoming connections. I also tried limiting to just my IP address as incoming connections.
Any help would be appreciated, thanks. Not sure why it would just be hanging. I have used a local version of postgres just fine, but connecting to postgres RDS is not working.
Underneath the Connectivity and Security > Security section I just noticed it says Public accessibility: No. Must I enable something else? There is also one VPC, one Subnet group, and several Subnets, which I don't know too much about, must I do something there?
Finally, "status" says Available with a green light, so things seem fine there.
I found a useful link in AWS knowledge center, I hope this will help you.
Btw if your RDS is deployed in a public subnet, yes you need to enable if you want to access RDS over the internet.
My DB instance is in a public subnet, and I can't connect to it over the internet from my local computer
This issue can occur when the Publicly Accessible property of the DB instance is set to No. To check whether a DB instance is publicly accessible, you can use the Amazon RDS Console or the AWS CLI.
To change the Publicly Accessible property of the Amazon RDS instance to Yes:
Verify that your VPC has an internet gateway attached to it. Make sure that the inbound rules for the security group allow connections.
Open the Amazon RDS console.
Choose Databases from the navigation pane, and then select the DB instance.
Choose Modify.
Under Connectivity, extend the Additional configuration section, and then choose Publicly accessible.
Choose Continue.
Choose Modify DB Instance.
Note: You don't need to choose Apply Immediately. For more information about how Apply Immediately can affect downtime, see Using the Apply Immediately parameter.
https://aws.amazon.com/premiumsupport/knowledge-center/rds-connectivity-instance-subnet-vpc/

What could cause a connection to a Google Cloud SQL instance to fail even with the ip registered in the list of allowed addresses?

I am unable to connect to a Google Cloud SQL instance. I get the error: [MySQL] [ODBC 5.1 Driver] Can't connect to MySQL server on 'XX.XXX.XX.XXX' (10060)
The IP from which I try to connect is a fixed ip and is registered in the list of allowed IPs in the Cloud SQL instance.
The machine that needs to connect is a Windows server with Windows Server 2016 operating system and can connect to other instances of Google Cloud SQL. We only have problems connecting to this instance. Another detail of the error is that the ping performed on my Windows instance returns total loss in relation to the Cloud SQL instance IP. Finally, I can access the databases using my local internet or another server.
I will be very grateful if someone can give me a tip to identify the problem

Azure PostgreSQL foreign data wrapper postgres_fdw

I am trying to use the PostgreSQL foreign data wrapper (postgres_fdw) extension from an Azure DB for PostgreSQL instance. The foreign data wrapper should be supported as mentioned in Azure DB for PostgreSQL: https://learn.microsoft.com/en-us/azure/postgresql/concepts-extensions
I manage to create a server and a mapping for the user, but when I try to import a table or a schema I have the following error:
ERROR: could not connect to server "<server with public hostname on AWS>"
DETAIL: could not translate host name "<server with public hostname on AWS>" to address: Unknown host
When replacing the hostname with a public IP I get the following:
ERROR: could not connect to server "<SERVER NAME>"
DETAIL: could not connect to server: Network is down (0x00002742/10050)
Is the server running on host "<public IP of the server" and accepting
TCP/IP connections on port 5432?
In the firewall options of the Azure DB for PostgreSQL instance, I already have the AllowAllIps rule starting 0.0.0.0 and ending 255.255.255.255. is there any thing else to configure to get access to servers outside of Azure, DNS or firewall configuration?
Not entirely sure about this, but we were trying to do something like this the other day, and stumbled upon this (somewhat disheartening) quote:
"Currently, outbound connections from Azure Database for PostgreSQL are
not supported, except for connections to other Azure Database for
PostgreSQL servers."
Source: https://learn.microsoft.com/en-us/azure/postgresql/concepts-extensions#dblink-and-postgres_fdw