I have a number of identical local postgreSql databases (identical in structure - not data) on several laptops that have intermittant access to internet. Records are being added to each DB daily. So Branch A,B,C each with a local Postgresql database. I would like all records from A,B,C in each table in a cloud Database.Also A,B,C data is separate - there is no overlap - A doesnt change B, or C etc. There are no duplicated unique keys.
NEED: I would like to collect all this data on a cloud based database by adding daily incremental data to a single cloud databse - so I can query the whole consolidated data using SQL and pull reports as needed.
Please can anyone point me in the right direction?
Thanks
It sounds like you want logical replication from each laptop to the cloud server. The problem there might be that contact must be made by the replica to each of the masters, so when your laptops are online, they would need to have predictable IP addresses so that they can be reached.
Maybe the best way around this is with a reverse SSH tunnel. On the central replica, you would tell it to subscribe to a publication hosted on some non-standard port on localhost. With a different port reserved for each laptop. So, for example, 9997, 9998, and 9999.
Then when each laptop has connectivity, it could run something like:
ssh rajb#1centralserver.example.com -R9999:localhost:5432 -f -N -T
This establishes an ssh connection to the central server (requiring a password, or private key, or however you have ssh set up) and sends instructions to the central server that whenever someone connects to port 9999 on the central server it should really send that connection back over ssh tunnel and hook it up to port 5432 (the default postgres server port) of the laptop.
For initially setting things up and debugging, you might want to omit the -f -N -T. That way, in addition to setting up the tunnel, you also get an interactive ssh session you can use for monitoring things.
Once the central service notices the connection is available, it will start downloading changes since the last time it could connect. When there is no connection, you will get a lot of nuisance messages to the log file as it checks each server every ~5 seconds to see if it is available.
From each laptop's perspective, the connection is coming from within, so the replication connection will use whatever authentication is set up or 127.0.0.1 or ::1, not the authentication set up for the actual remote IP.
Related
I have a Postgres database that uses physical replication. I want to get list of replicas.
Tried to select * from pg_stat_replication,I have two rows for replicas, but client_hostname field is empty. Documentation says that "it indicates that the client is connected via a Unix socket on the server machine". So how can I get replica connection strings/hostname/IP or any other way I can connect to replica to send query?
You cannot get that by querying the primary server. All the primary server sees is the client IP address from which the standby server connects, and that could be on a different network or even (as in your case) using a different connection method from a client connection to the standby server.
In my case, it turned out to be the lack of permissions for user. I added him a corresponding role and everything fixed.
I have two servers and both are working fine.
How to sync all my data from one server to another server/backup-storage/remote-storage.
I want to know if one of my server is down due to heavy load then how to use instantly second server and what is the role of DNS in this, because if we use another server then we have to change DNS also for particular website so how to overcome this.
You can check cloudflare load balancer.
Architecturally you have two problems to solve:
load balancing (how clients are routed to one of the servers) - this involves sometimes DNS settings but because cloudflare hosts your DNS as well, you are cool
Synchronization: files and database sync between hosting accounts. Now here there is no standard way to go especially because your are hosted using cpanel
DATABASE:
You can't use master-master or master-slabe database replication mechanisms like Galera Cluster has.
You're best bet is to have a cron that will export the database from one server to the other. (using mysqldump - basically exporting and then importing)
on live:
mysqldump -u userName -p yourLiveDatabaseName > live_database_export.sql
on the hot backup (your other account):
mysql -u username -p yourOtherServerDatabaseName < live_database_export.sql
FILES:
If you have SSH access use rsync.
Otherwise you may need to invent something.
For instance you can check the Cpanel API in regards to account transfers -> that will solve the database as well https://api.docs.cpanel.net/openapi/whm/operation/create_remote_user_transfer_session/
As a remark - you are not in the best position to do HA having two cPanel shared accounts. What I usually do is to use virtual machines that are sync at the hypervisor level.
I am using jmeter to test an application which uses PostgreSQL. I can connect to the database by using ssh tunnel provided by the database applications.
Can someone please tell me how do I do this using jmeter. I do not see any ssh tunnel option in jmeter database connection confi element.
You could use port forwarding,as explained in this answer:
https://stackoverflow.com/a/1968446/460802
I don't think you should be load testing the database directly, your load test should simulate real-life application under test usage. So instead of testing the database you should focus on the application itself and treat it like a black box so my general recommendation is reconsidering the approach.
If you have performed normal load testing already and identified that the database is the bottleneck and would like to load test the database separately - performance testing it over the SSH tunnel is not the best idea itself as the SSH tunnel traffic might be the next bottleneck due to the nature of TCP protocol and immense CPU footprint required for encryption/decryption of the data sent over SSH. So I would recommend talking to network administrators and asking them to temporarily open the Postgres network port to the machine(s) you're running JMeter from or provide you access to the machine(s) where you can install JMeter which will be having access to the database directly (preferably in the same subnet / physical location, otherwise you might be suffering from high latencies)
If for any reason the above instructions are not applicable for you - you can use SSH Local Forwarding in order to map remote Postgres port to your local port, the relevant command would be:
ssh -L 2345:localhost:5432 username#your_postgresql_server
Once done you should be able to connect to Postgres instance as it is installed locally on port 2345 like:
postgres://localhost:2345/your_database
I have two servers on Linode that can connect to each other through the local Linode regional network. The problem is that any other Linode in the region can also connect using that IP. One server hosts the python application and the other hosts the MongoDB.
Would it be a good idea to connect to the database using an SSH Tunnel? What happens if the tunnel fails? Are SSH tunnels known to fail at all?
Or am I approaching the problem the wrong way? Another alternative I can think of is setting up iptables to only accept connections from a particular source IP.
I'm thinking in a more hypothetical situation, perhaps a DB password is all I need. I've been taking an computer security course and it makes everything seem more vulnerable than it really is.
i m creating inapp purchage subscription module,
in this app i want to access remote database but problem is that how i connect
my objective-c code with the mysql on the server,
i am not found any sufficient refrence please help me if any refrence or solution is there.
In mySQL, all access from your application to the database server is already remote access. Local access is simply one case of remote access.
If you're working locally, you may be using "localhost" or "127.0.0.1" as the hostname for your data base, and 3306 as the port number. You're using the data base name you set up on your local server, perhaps "arunsdata" or some such thing
You need to find out the hostname and port number of the remote data base server. (The port number is probably 3306.) Then you need to modify your application code to specify that hostname and port number.
Before you do that you will need to have a username and password, and create your data base ("arunsdata" or whatever) on the remote data base and create your tables and other schema items. The administrator of the remote database server can probably help you with this.
Good luck! I remember how confusing this was the first time I faced it. It's simpler than it seems.
You should probably create a web service to access the remote mySQL server database. You can then send a request to the service using NSMutableURLRequest. If you need to return data back, return json since its more light weight than XML.