When providing the connection.url for a source connector, I pass an IP address as part of the JDBC connection string. However, this IP is not static all the time, as the source system will be migrated soon.
Two questions arise:
does the connector support DNS names, so we can move away from IPs in the connection string?
does the JDBC connector resolve the IP, if we put it in /etc/hosts mapped to another hostname and change the host in the connection string accordingly? This suggestion came up during some discussions, but I have quite some doubts about that solution.
Java is passing the DNS resolution to the OS, so the framework doesn't control that.
The /etc/hosts file is also static, however, so you might want to look at other service-discovery options such as HashiCorp Consul.
Related
I have a new Linux database server I am working on with a certain hostname and ipaddress. I want to change it's hostname and ipaddress with my old Linux database server's at the time of going live with the new server. I understand this requires only update on the new server's hosts file.
My team says it would need a DNS update. I don't see why it would require a DNS update. Because a DNS is a mapping between hostname and ipaddress and it hasn't changed.
Could anyone clarify?
Thank You, Madhuri Dara
normally, yes indeed, a DNS is a mapping between IP addresses and hostname.
However, I would also recommend to delete the old one an re-create them to perform a re-discovery of the routes.
The best option for you would be here to have a DNS manager into your domain, this way, you could modify them here, instead of into your DNS file in each machine.
In java we had an option to use multiple hosts configuration with failover mechanism:
jdbc:postgresql://node1:port1,node2:port2,node3:port3/accounting?targetServerType=primary
do we have such support in Go? How the connection string should look like? i've seen that lib/pq is not maintained
i didn't find any information about jackc/pgx if they support multi host, or how the connection string should look like. Please if someone can provide an example.
I am looking for GCP networking best practice, where I can allow connection of auto-scaled instances to Postgresql server installed on separate instance.
So far I tried whitelisting load-balancer IP within firewall and postgresql config file, but failed.
Any help or pointer is highly appreciated.
The load-balancer doesn't process information by itself, it just redirects Frontend addresse(s) and manage the requests with Instance Groups.
That instance group should manage the HTTP requests and connect with the database instance.
The load-balancer is used to dynamically distribute (or even create additional instances) to handle the requests over the same Frontend address.
--
So first you should make it work with a regular instance, configure it and save the instance template. Then you can proceed with creating an instance group that can be managed by a load-balancer.
EDIT - Extended the answer from my comment
"I don't think your problem is related to Google cloud platform now. If you have a known IP address for the PostgreSQL server (connect using an internal network IP address so it doesn't change), then make sure your auto-balanced instances are in the same internal network, use db's internal IP and connect to it."
I am currently developing a service application that pulls data from Mongo and returns it to consumers. There is a layer of authentication involved and I am using Heroku to host the service. Mongo was being hosted on MongoLabs, but there were some significant performance concerns and so we have moved to hosting Mongo on one of our cloud servers. We want to be able to secure access to Mongo using a firewall, white-listing the ip address of the service app on Heroku.
There are a couple of issues with this.
Issues
Well, at least these are main ones...
Heroku, while providing some nice features like easily managing cluster settings, s/w upgrades, etc., draws ip addresses from a pool. While the dns value of an application's url may not change, the underlying ip address can and will change.
to be better secured, mongo-server01 is placed behind a firewall that requires rules to be added using static ip addresses to allow access.
Since Heroku can't provide static ip addresses, we need to consider options for how Heroku can access mongo-server01 while still protecting the data it hosts.
Static IP addresses for outbound requests
There seem to be a couple of options, specifically for Heroku. Fixie and QuotaGuard Static both seem to serve that function, but these seem to be geared toward HTTP and HTTPS communication only (perhaps not even HTTPS).
Mongo doesn't use HTTP, it uses its own network protocol over port 27017, by default
https://groups.google.com/forum/embed/#!topic/mongodb-user/eX_RIv2cZVw
Does this mean these proxies won't work for calls to Mongo? In theory, there doesn't seem to be any reason that a proxy is only for HTTP or HTTPS requests. That being said, there doesn't seem to be any way to get in to these Heroku plugins and configure the proxy to use a different port or to handle Mongo's particular protocol.
If we could get into the proxy, perhaps we could put an additional set of ssh keys in place so the ssl tunnel chain could continue on to mongo-server01. But there doesn't see to be any way to ssh to these proxies or access configuration through the plugin dashboards.
The question (finally!)
How can one connect from Heroku to a firewalled host to get data from MongoDb? Are there proxies that can be used to achieve this?
The simple approach. Won't work because Heroku applications don't use static ip addresses.
Using a proxy. The Heroku proxy plugins don't know how to proxy mongodb protocol. Can't install ssh keys on proxy for ssh tunneling.
What can be done to get a connection without opening up the Mongo server to the world?
I spoke with the folks at QuotaGuard and they do have something that does the trick.
we offer a SOCKS proxy which should do the trick as it proxies at the TCP layer
https://devcenter.heroku.com/articles/quotaguardstatic#socks-proxy-setup
I did need to make a simple change to bin/qgsocksify
#SOCKS_DIR="$(dirname $(dirname $(readlink -f ${BASH_SOURCE[0]})))/vendor/dante
SOCKS_DIR="${HOME}/vendor/dante"
After that, the proxy worked like a charm.
We are currently encountering the following error when trying to connect to a Cloud SQL instance: Lost connection to MySQL server at 'reading initial communication packet', system error: 0.
This is a familiar error, and as detailed here usually means the IP address needs to be whitelisted. However, we believe we have done so.
Is there a way to see connection attempts and their IP addresses that have been made (and refused) to the Cloud SQL instance?
Currently we don't expose that information but it is something we would like fix. :-)
According to #Razvan, as of September 2014, this information isn't exposed.
We ended up using CIDR blocks to search the space and find the actual IP address. This is unsatisfying, obviously, but it's a way to pin down the problem.
If other people want to sanity check that the problem is their IP is being refused, you can add 0.0.0.0/0 in order to accept all ranges and try to connect. If it works, you know what is the problem.
Be absolutely sure to remove this as an accepted range, after you are done, however!
Figured I might help someone who stumbles here.
Had exactly the same issue essentially trying to connect to a GCP SQL instance from a hosting provider.
Whitelist the IP address that is shown in my cpanel and it will not connect. (It used to, but the provider made some changes with their infrastructure lately and it stopped working)
put 0.0.0.0/0 in my Cloud Platform whitelist and it connects no problem.
So now I know that my cpanel IP is not the IP trying to connect to GCP.
After some hair pulling (figured that the bare metal server had a different IP than my cpanel IP, it did, but this also didn't work.)
finally tried the IP address for the name servers that point to my domain and bam. All is good.
If you are facing this issue, try your name server (usually something like NS1.hostingprovider.com etc..). I put both the NS1 and NS2 ip's in the whitelist and we are working fine.