Says that I have 3 different Postgres database hosting on a different cloud providers.
So I wonder if I could use 1 Debezium container to connect to these 3 databases to send datas to Kafka ?
If yes, I wonder if there are documents or articles that I can use to reference.
Or do I need to have 3 different Debeizum container for each databases?
Related
I need to deploy apache-kafka-connect in kubernetes. I have following questions:
Do I need to create MS SQL Server db tables in MySQL database? Or Kafka connect will create?
Any reference how to implement Kafka Connect without Avro Schema registry?
How to give configurations of key and value convertors?
I am getting below error while using AWS data migration service.
Source - One single postgres rds instance having multiple databases in it (80 GB)
Target - One single postgres rds instance where each src database will be a schema in same database
Number of tables - Total number of tables including all databases is around 200
Replication instance - t2.medium
I created four tasks for replicating four databases from source into four different schemas in target. But the fifth task for fifth database is failing with below error -
ERROR: all replication slots are in use;, Error while executing the
query
How can I increase the replication slots so that dms can have 10 tasks running together?
Found the answer, There is an option to configure maximum number of replication slots in aws postgres rds instance which should solve my issue.
Is it possible to have multiple databases per one Heroku postgreSQL plan(instance)?
Unlike Amazon RDS, Heroku doesn't allow creating multiple databases – your DB role simply doesn't have permissions to CREATE DATABASE ..;.
However, you can create several "apps", each one with its own Postgres and then use multiple Postgres DBs in single app (see for example https://devcenter.heroku.com/articles/heroku-postgresql#sharing-heroku-postgres-between-applications – this is a way to change "attached" database, but you can just add config vars pointing to multiple database credentials with heroku config and then use those credentials inside your app).
Alternatively, you can create Amazon RDS Postgres (one or as many as you wish) in the same Availability Zone as your Heroku app, and use this Postgres instance (or several) in your Heroku app.
Actually this is not completly true.
You are right, you do not have the permisson to create a database,
but instead it is possible to just add more Heroku Postgres databases as Resources.
This way you will have multiple plans(instances)
Currently I have 1 postgres instance which is starting to receive too much load and want create a cluster of 2 postgres nodes.
From reading the documentation for postgres and pgpool, it seems like I can only write to a master and read from a slave or run parallel queries.
What I'm looking for is a simple replication of a database but with master/slave based on which table is being updated. Is this possible? Am i missing it somewhere in the documentation?
e.g.
update users will be executed on server1 and replicated to server2
update big_table will be executed on server2 and replicated back to server1
What you are looking for is called MASTER/MASTER replication. This is supported natively (without PgPool) since 9.5. Note, that it's an "eventually consistent" architecture, so your application should be aware of possible temporary differences between the two servers.
See PG documentation for more details and setup instructions.
It's possible to connect meteor manually to 2 or more databases in order to have a normal mongo that saves to the disk and a memory one like redis?
I'm asking because mongo has already full support from meteor and using it would be a lot easier than redis or other database
Right now, a Meteor server can only connect to one (and exactly one) Mongo database.
Redis support is on the roadmap, as is SQL support. Once Meteor supports multiple databases, you will have more options for how to set up your databases as well as dividing up your data between them. The only way to do what you are saying right now is to have your Meteor client connect to two different Meteor servers, and have one of them clear/dump the database regularly.
Source: discussions at Meteor's offices.