We have an RDS AWS Postgres database cluster (v13.7) from which we want to be able to invoke a Lambda function. We have carefully followed the directions laid out in this AWS document and paid special attention to our RDS security group allowing TCP traffic to port 443 and to any IPv4 addresses (0.0.0.0/0).
Using psql to connect to the database, we have been running the following command to try and invoke a lambda call:
SELECT * from aws_lambda.invoke(aws_commons.create_lambda_function_arn('arn:aws:lambda:ca-central-1:123456789012:function:lambda-postgres-test', 'ca-central-1'), '{"body": "Hello from Postgres!"}'::json, 'Event');
However the result is the following error after about 30s:
ERROR: invoke API failed
DETAIL: AWS Lambda client returned 'curlCode: 28, Timeout was reached'.
CONTEXT: SQL function "invoke" statement 1
It feels like this is some simple inbound or outbound rules issue but so far nothing we have tried has changed the outcome. Does anyone know about another change that may be required to permit RDS Postgres to invoke a lambda?
Also: our RDS Postgres cluster is publicly available and so we've followed the security-setup directions from the AWS docuement labelled: "To configure connectivity to AWS Lambda for a public DB instance"
Related
My plan is to migrate from SQL Server to Postgresql, hosted on AWS. My Postgresql DB is in a VPC and the security group attached to it has the following inbound rules:
The first one is for my EC2 instance (where my server-side application is running) to connect to the DB, and the second one is the home IP address so that I can connect to it via pgAdmin.
This is what I get (I don't know what to fill in at 'Maintenance database', so I left it at postgres.
What am I missing? I need help with this.
I have noticed that for some combinations of VPN, local router, etc. the IP address the security group filled in for 'my IP' was not the one that RDS recognized as trying to connect. I found that curl --silent ifconfig.me returns the IP address that works the best for the security group.
Background
I have a NestJS project with Prisma ORM, and I am continually receiving the error:
PrismaClientInitializationError: Can't reach database server at `localhost`:`5432`
This is happening during the Cloud Build Deploy step.
Since this is a containerized application (attempting to) run in a Cloud Run instance, I'm supposed to use a socket connection. Here's the documentation from Prisma on connecting to a Postgres DB through a socket connection: https://www.prisma.io/docs/concepts/database-connectors/postgresql#connecting-via-sockets
Connecting via sockets
To connect to your PostgreSQL database via sockets, you must add a host field as a query parameter to the connection URL (instead of setting it as the host part of the URI). The value of this parameter then must point to the directory that contains the socket, e.g.: postgresql://USER:PASSWORD#localhost/database?host=/var/run/postgresql/
Note that localhost is required, the value itself is ignored and can be anything.
I've done this to the letter, as described in the Cloud SQL documentation, with the exception that I percent-encoded my path to the directory containing the socket. I've included and excluded the trailing slash.
So my host var looks like this, mapped from the percent-encoded values:
/cloudsql/<MY CLOUD SQL CONNECTION NAME>/<DB>
I've read over the Cloud Run documentation, and in my mind, I should expect a different error if the instance itself can't connect to the Cloud SQL instance. I've followed the "Make sure you have the appropriate permissions and connection" from the documentation a few times now.
Is there anything obvious that I'm missing? Am I wrong about an error related to Cloud Run instance just not connecting with Cloud SQL instance?
Things I've tried & things I know
I CAN connect directly to the Cloud SQL instance locally through psql
I CAN run a local server with the Cloud SQL instance public IP and establish a client connection & interact with the database
I CAN successfully create an image and run a container from that image locally
My big concern
It doesn't make sense to me in which order things should connect to the Cloud SQL instance. To me, the Cloud Run - Cloud SQL connection MUST be established before the application run inside the Cloud Run instance can establish its connection through the socket to the Cloud SQL instance. -- Am I thinking through that correctly?
I am brand new to RDS and have only used postgres through Rails and/or Heroku for the most part, so not that deep into database management. All I am trying to do is verify I can connect to the RDS instance I just created on AWS, but it is hanging, psql reporting this after hanging for about 30 seconds or a few minutes I guess:
$ psql postgresql://myuser:mypass#myawshost.rds.amazonaws.com:5432/my-db-name
psql: error: could not connect to server: Operation timed out
Is the server running on host "myawshost.rds.amazonaws.com" (<the ip address>) and accepting
TCP/IP connections on port 5432?
How do I connect to my AWS RDS instance from localhost?
I am sure I have the correct username and password.
I am sure the host is correct.
I am not sure if I should be including the port.
I am not sure if I am supposed to put a DB name, as under the "Configuration" tab in the RDS admin console for my database, it says DB instance ID: my-db-name, Engine version: 13.3, DB name: -, I am not sure if that - is my actual postgres db name or my-db-name is....
I tried this with the db name and it still hangs:
$ psql postgresql://myuser:mypass#myawshost.rds.amazonaws.com:5432/-
I edited the security group which is linked under the VPC security groups section of the Connectivity & security tab of https://console.aws.amazon.com/rds/home, so it allows all incoming connections. I also tried limiting to just my IP address as incoming connections.
Any help would be appreciated, thanks. Not sure why it would just be hanging. I have used a local version of postgres just fine, but connecting to postgres RDS is not working.
Underneath the Connectivity and Security > Security section I just noticed it says Public accessibility: No. Must I enable something else? There is also one VPC, one Subnet group, and several Subnets, which I don't know too much about, must I do something there?
Finally, "status" says Available with a green light, so things seem fine there.
I found a useful link in AWS knowledge center, I hope this will help you.
Btw if your RDS is deployed in a public subnet, yes you need to enable if you want to access RDS over the internet.
My DB instance is in a public subnet, and I can't connect to it over the internet from my local computer
This issue can occur when the Publicly Accessible property of the DB instance is set to No. To check whether a DB instance is publicly accessible, you can use the Amazon RDS Console or the AWS CLI.
To change the Publicly Accessible property of the Amazon RDS instance to Yes:
Verify that your VPC has an internet gateway attached to it. Make sure that the inbound rules for the security group allow connections.
Open the Amazon RDS console.
Choose Databases from the navigation pane, and then select the DB instance.
Choose Modify.
Under Connectivity, extend the Additional configuration section, and then choose Publicly accessible.
Choose Continue.
Choose Modify DB Instance.
Note: You don't need to choose Apply Immediately. For more information about how Apply Immediately can affect downtime, see Using the Apply Immediately parameter.
https://aws.amazon.com/premiumsupport/knowledge-center/rds-connectivity-instance-subnet-vpc/
I am trying to connect to my RDS database from my computer with a python script using psycopg2.
python code:
import psycopg2
from db_credentials import *
import logging
def get_psql_conn():
conn = psycopg2.connect(dbname=DB_NAME, user=DB_USER, password=DB_PASS, host=DB_HOST)
logging.info("connected to DB!")
return conn
I get the following error:
psycopg2.OperationalError: could not connect to server: Operation timed out
Is the server running on host ********* and accepting
TCP/IP connections on port 5432?
My security groups assigned to the RDS database:
SG 1:
SG 2:
Now i tried to make a security group which allows my computer IP to access the DB.
SG 3:
I can connect to the DB from my ec2 instances, running the same python script as above. This seemingly has to do with the 2nd security group, as when i remove it, i can no longer connect from my ec2 instances either. It then throws the same error i get when trying to connect from my computer.
I have little understanding of RDS or security groups, i just followed internet tutorials, but seemingly couldnt make much sense out of it.
Any help is greatly appreciated! Thanks
When accessing an Amazon RDS database from the Internet, the database needs to be configured for Publicly Accessible = Yes.
This will assign a Public IP address to the database instance. The DNS Name of the instance will also resolve to the public IP address.
For good security on publicly-accessible databases, ensure that the Security Group only permits access from your personal IP address.
I am unable to connect to a Google Cloud SQL instance. I get the error: [MySQL] [ODBC 5.1 Driver] Can't connect to MySQL server on 'XX.XXX.XX.XXX' (10060)
The IP from which I try to connect is a fixed ip and is registered in the list of allowed IPs in the Cloud SQL instance.
The machine that needs to connect is a Windows server with Windows Server 2016 operating system and can connect to other instances of Google Cloud SQL. We only have problems connecting to this instance. Another detail of the error is that the ping performed on my Windows instance returns total loss in relation to the Cloud SQL instance IP. Finally, I can access the databases using my local internet or another server.
I will be very grateful if someone can give me a tip to identify the problem