In aws, I have an amazon linux instance running with docker installed and my app running as a container. It's running in tomcat. However I need to connect it to my database.
I have made this work with a postgres container earlier doing this:
docker run --link <dbcontainername>:db -P -d tomcat-image
But to have the database more reliable it is wanted to use amazon RDS instead.
I have created a VPC with two subnets which both the instance and the RDS uses
And they are also both in the same Security Group.
I am able to access the tomcat fine through the public ip, but it throws errors because it isn't connected to the db.
Networking is not my strong suit, so there might be something there I am missing, but I find it hard to find any text describing this process without mentioning Elastic bean stalk.(It is my impression that it should be possible to do everything EBS does, manually)
There's a similar question asked here about 8 months ago, but he didn't get any responses so I'm trying again.
Related
I am brand new to RDS and have only used postgres through Rails and/or Heroku for the most part, so not that deep into database management. All I am trying to do is verify I can connect to the RDS instance I just created on AWS, but it is hanging, psql reporting this after hanging for about 30 seconds or a few minutes I guess:
$ psql postgresql://myuser:mypass#myawshost.rds.amazonaws.com:5432/my-db-name
psql: error: could not connect to server: Operation timed out
Is the server running on host "myawshost.rds.amazonaws.com" (<the ip address>) and accepting
TCP/IP connections on port 5432?
How do I connect to my AWS RDS instance from localhost?
I am sure I have the correct username and password.
I am sure the host is correct.
I am not sure if I should be including the port.
I am not sure if I am supposed to put a DB name, as under the "Configuration" tab in the RDS admin console for my database, it says DB instance ID: my-db-name, Engine version: 13.3, DB name: -, I am not sure if that - is my actual postgres db name or my-db-name is....
I tried this with the db name and it still hangs:
$ psql postgresql://myuser:mypass#myawshost.rds.amazonaws.com:5432/-
I edited the security group which is linked under the VPC security groups section of the Connectivity & security tab of https://console.aws.amazon.com/rds/home, so it allows all incoming connections. I also tried limiting to just my IP address as incoming connections.
Any help would be appreciated, thanks. Not sure why it would just be hanging. I have used a local version of postgres just fine, but connecting to postgres RDS is not working.
Underneath the Connectivity and Security > Security section I just noticed it says Public accessibility: No. Must I enable something else? There is also one VPC, one Subnet group, and several Subnets, which I don't know too much about, must I do something there?
Finally, "status" says Available with a green light, so things seem fine there.
I found a useful link in AWS knowledge center, I hope this will help you.
Btw if your RDS is deployed in a public subnet, yes you need to enable if you want to access RDS over the internet.
My DB instance is in a public subnet, and I can't connect to it over the internet from my local computer
This issue can occur when the Publicly Accessible property of the DB instance is set to No. To check whether a DB instance is publicly accessible, you can use the Amazon RDS Console or the AWS CLI.
To change the Publicly Accessible property of the Amazon RDS instance to Yes:
Verify that your VPC has an internet gateway attached to it. Make sure that the inbound rules for the security group allow connections.
Open the Amazon RDS console.
Choose Databases from the navigation pane, and then select the DB instance.
Choose Modify.
Under Connectivity, extend the Additional configuration section, and then choose Publicly accessible.
Choose Continue.
Choose Modify DB Instance.
Note: You don't need to choose Apply Immediately. For more information about how Apply Immediately can affect downtime, see Using the Apply Immediately parameter.
https://aws.amazon.com/premiumsupport/knowledge-center/rds-connectivity-instance-subnet-vpc/
My team and I are currently experiencing an issue where we can't connect to Cloud SQL's Postgres instance(s) from anything other than the psql cli tool. We get a too many connections for database "postgres" error (in PGAdmin, DBeaver, and our node typeorm/pg backend). It initially happened on our (only) Postgres database instance. After restarting, stopping and starting again, increasing machine CPU/memory proved to do nothing, I deleted the database instance entirely and created a new one from scratch.
However, after a few hours the problem came back. I know that we're not actually having too many connections as I am able to query pg_stat_activity from psql command line and see the following:
Only one of those (postgres username) connections is ours.
My coworker also can't connect at all - not even from psql cli.
If it matters, we are using PostgreSQL 13, europe-west2 (London), single zone availability, db-g1-small instance with 1.7GB memory, 10GB HDD, and we have public IP enabled and the correct IP addresses whitelisted.
I'd really appreciate if anyone has any insights into what's causing this.
EDIT: I further increased the instance size (to no longer be a shared core), and I managed to successfully connect my backend to it. However my psql cli no longer works - it appears that only the first client to connect is allowed to connect after a restart (even if it disconnects, other clients can't connect...).
From the error message, it is clear that the database "postgres" has a custom connection limit (set, for example, by ALTER DATABASE postgres CONNECTION LIMIT 1). And apparently, it is quite small. Why is everyone try to connect to that database anyway? Usually 'postgres' database is reserved for maintenance operations, and you should create other databases for daily use.
You can see the setting with:
select datconnlimit from pg_database where datname='postgres';
I don't know if the low setting is something you did, or maybe Google does it on its own for their cloud offering.
#jjanes had the right idea/mention.
I created another database within the Cloud SQL instance that wasn't named postgres and then it was fine.
It wasn't anything to do with maximum connection settings (as this was within Google Cloud SQL) or not closing connections (as TypeORM/pg does this already).
Most of the documentation Ive seen only covers using mLab as a service for mongodb on heroku,
I would like to use mongoDB locally on the heroku server is this possible?
Heroku dynos use an ephemeral filesystem. This means that you cannot run a database on a Heroku dyno. All of the data will be lost whenever your process restarts. Your process restarts at least once every day, whenever you deploy new code, whenever you change config variables, or whenever you update add-ons.
You can read more about Heroku's ephemeral filesystem here: https://devcenter.heroku.com/articles/dynos#ephemeral-filesystem
You can read more about dyno restarts here: https://devcenter.heroku.com/articles/dynos#restarting
So you will have to use a Heroku Add-on or a third-party service to add persistence to your application.
This doesn't mean you have to use the mLab Add-on. If you have your own MongoDB database, you can still connect to that in your application. You can add the connection string as a config variable and connect to it in the same way you would connect to an mLab Add-on.
I have 2 Amazon EC2 instances. I am using one for development. I started a new one that I want to get working properly so I can use this cleaner one to make an AMI. I am using Django with a Postgres backend in an RDS instance. The RDS instance is running Postgresql 9.4.4. The development EC2 instance (which is the one that works) is running Postgresql 9.3.9. The new instance is running Postgresql 9.3.10.
On the development instance I have no trouble connecting to and using the RDS instance with the command line:
psql --host django.xxxxxxxxx.us-east-1.rds.amazonaws.com --port 5432 --username django_login --dbname django_db
But if I use the same command on the new instance, I get
psql: could not translate host name "django.xxxxxxxxx.us-east-1.rds.amazonaws.com" to address: Name or service not known
Both EC2 instances are in the same security group (launch-wizard-1). The RDS has a security group with all TCP, all UDP and all IMCP set with launch-wizard-1 as the source.
The development instance is in us-east-1d. The new instance is in us-east-1c. The RDS instance is in us-east-1d. I suspect that might be the problem but as I understand the RDS documentation it should be fine. If however, that is the problem, do you know how to change the Availability Zone of an EC2 instance?
I have tried this with the RDS instance set to private and then to public. It did not make a difference.
Any ideas will be appreciated.
The problem was as I had suspected. The instance was in a different subnet. I made an ami from the instance in us-east-1c, made a new instance in the us-east-1d subnet from the ami. Now I can connect from the new instance.
BTW - It is not immediately obvious when you create an instance that you should set the subnet to match your other instances. Look for that option on the instance configuration page.
My understanding is that Heroku Postgres runs on top of AWS. Is it possible to configure which datacenter your database is running in? I'm also wondering if the database files are stored on an encrypted filesystem.
Yes, Heroku runs on AWS. But you are not able to specify which datacenter to run your database. For encryption look at http://www.postgresql.org/docs/current/static/pgcrypto.html.
Heroku runs out of Amazon US-East - once you've add a postgres db to your app heroku config will give you the database connection URL which you would be able to tracert on to see where it is