my Google cloud SQL facing large incoming connections. I need to reduce the SQL load. Is there anyway? when I search for load balancing it saying about compute engine load balancing. Please suggest me a way to reduce the load in my Google cloud SQl. I am using compute engine. Thank you for your time.
Related
We’re uploading sensor data to an InfluxDB bucket hosted via InfluxDB Cloud GCP. We’d like a way to “calibrate” this data by simply offsetting it for given time ranges (and also correcting incoming data as it arrives). Is there any out-of-the-box solution for this in InfluxDB that we could use for free with our account without additional compute resources? Or do we have to host something like Telegraf or Kapacitor? We really want the bare bones and minimal setup.
We are also using Grafana to query the data so any solution native to Grafana would work as well.
I am setting up a read only GraphQL instance using Java. GraphQL as I understand it needs to be told when to re-query its data sources. We are using GCP, and Cloud SQL for our primary data source. Our monolithic system is what is responsible for updating the data.
Is there a way to trigger a web request or pub/sub message from cloud sql without sys_eval(sys_eval('curl https://example.com'));?
or is there a way to turn on sys_eval in cloud sql?
After some brainstorming around sys_eval alternatives such as binary logs and so on, I think the course of action I'd recommend is to move the MySQL client to the GCE instance, and establish the connection to the Cloud SQL instance through a Private IP.
Such connection will be guaranteed a much lower latency, and an a lot higher network security, since, the service does not use Public IPs and it would be protected from the "outside" Internet; all compared to your current architecture.
You can find connection examples using VPC networks in the documentation provided.
In my gcp project App engine located in Central US and my Postgres Sql located in East US region.
Any one suggest best way to resolve the connection and latency issue for the above problem?
If I correctly understand there is high latency between your app engine and PostgreSql instance. You can avoid high latency if you enable Geo-replication for your instances.
You can also migrate your instances closer to each other but overall latency across the world would increase significantly.
So my suggestion would be to go for geo-replication.
Hello I actually have a REST api running on Cloud Functions for Firebase using http request, but now I need to sync the data on real time requesting to the functions. I read something about web sockets.
Is there a way to run a web socket on Cloud Functions for Firebase?
This is not going to be a good fit for Cloud Functions. Websockets rely on long-lived connections to the same server over time; Cloud Functions are ephemeral compute instances that are spun down when there's no traffic. There's no way to force or guarantee that a Cloud Function will keep running or hold a connection open indefinitely.
I would encourage you to investigate using the Firebase Realtime Database as a conduit here instead of trying to add realtime to Cloud Functions.
Theoretically you could use two different layers: one to manage the websocket connections and another layer to handle the data processing.
The websocket layer will not be Cloud Functions, but a Docker container running Push Pin in Cloud Run and that’ll route HTTP calls to your Cloud Functions to do the actual data processing.
This is possible using an Apigee Java callout, where the Java (if needed) calls a Cloud Function. See https://cloud.google.com/apigee/docs/api-platform/develop/how-create-java-callout
TL:DR Does Heroku support pgpool-II for master-slave read only load balancing? Other solutions?
We are planning to go live with our application on heroku within the next couple of months. The biggest potential scaling hurdle at the moment for us is getting past the database connection limits. We've already employed pgbouncer. (https://github.com/gregburek/heroku-buildpack-pgbouncer) Now we are looking at master-slave read only query load balancing. We would prefer to use something like pgpool instead of rolling our own in-app balancer. (playframework)
Vertical scaling of the database doesn't make much sense for us because the size of data will remain relatively small. Plus the price or higher heroku db tiers increases exponentially. (we would prefer linear scale)
Have you had success with read only query load balancing on heroku? What was your approach?