I'm trying to migrate existing data and replicate ongoing changes
the source database is PostgreSQL it's managed by aws.
the target is kafka.
I'm facing the below issue.
Last Error No tables were found at task initialization. Either the selected table(s) or schemas(s) no longer exist or no match was found for the table selection pattern(s). If you would like to start a Task that does not initially capture any tables, set Task Setting FailOnNoTablesCaptured to false and restart task. Stop Reason FATAL_ERROR Error Level FATAL
Related
We are using AWS DMS to do a live migration between AWS RDS to AWS RDS.This task is in Full load+CDC mode. Full load is completed successfully but CDC is now failing with below error:
Last Error Load utility network error. Task error notification received from subtask 0, thread 0 [reptask/replicationtask.c:2883] [1020458] Error executing source loop; Stream component failed at subtask 0, component st_0_ADWXVXURDV4UXYIGPH5US2PQW6XSQVFD5K4NFAY; Stream component 'st_0_ADWXVXURDV4UXYIGPH5US2PQW6XSQVFD5K4NFAY' terminated [reptask/replicationtask.c:2891] [1020458] Stop Reason RECOVERABLE_ERROR Error Level RECOVERABLE
In the cloudwatch I can only see below error:
WAL reader terminated with broken connection / recoverable error. [1020458].
I am not sure what might be happening here and my only guess is to fix this I may need to run CDC again with custom checkpoint. Can anyone help me on this?
I tried debugging this issue with further logging levels and also tested the connectivities. I looked into cloudwatch metrics but nothing seems suspicious. Also do note that, CDC did start successfully but has now entered into failed state.
I have a schedule job that appeared an error today:
Error on Databricks
org.postgresql.util.PSQLException: ERROR: canceling statement due to conflict with recovery
There's a way to solve this on Databricks? How? Thanks!!
Tried to find something on the configs, but didn't have success.
If you are querying a secondary installation (replica) and there are conflicting changes that need to be replayed from the primary there are two options:
Delay replaying the changes (replication will fall further behind)
Cancel the running query that conflicts with the changes
PostgreSQL will do 1 until it hits a configured timeout and then do 2 so your replication doesn't fall too far behind.
https://www.postgresql.org/docs/current/hot-standby.html#HOT-STANDBY-CONFLICT
I've been running queries on an Aurora PostgreSQL cluster having 1 writer instance and 1 reader instance.
My application is in .NET6 and it's using the Npgsql library to connect to the cluster. I've been testing with multiple versions of the library: 6.0.7 and 7.0.0.
Here is the scenario:
I executed queries using the cluster's reader endpoint (load balancer).
In the metrics all connections appeared on the reader instance.
Added another reader instance to the cluster.
Once the new reader instance was up and running, some of the new queries may have been transferred to the new reader, but sometimes all queries were still executing on the first reader.
I removed the first reader instance from the AWS Management Console. At this point I expected that all new queries are executed on the newly added reader since the first one was getting deleted.
But usually most of the queries were still executed on the first one.
While the original reader was being deleted, queries executed fine for a while, but at some point the reader was deleted and queries failed with error:
Npgsql.NpgsqlException (0x80004005): Exception while reading from stream ---> System.IO.EndOfStreamException: Attempted to read past the end of the stream.
Shouldn't new queries (using a newly created NpgSqlConection object) execute on the second reader as soon as the first one is marked to be deleted?
Can someone provide some details on how this should be done? Can this be avoided without using a retry mechanism?
I want to migrate my PostgresDB hosted in Citus cloud service to AWS RDS Aurora Postgres.
I am using AWS DMS service. Have created task but getting following errors:
Last failure message Last Error Stream Component Fatal error. Task
error notification received from subtask 0, thread 0
[reptask/replicationtask.c:2860] [1020101] Error executing source
loop; Stream component failed at subtask 0, component
st_0_QOIS7XIGJDKNPY6RXMGYRLJQHY2P7IQBWIBA5NQ; Stream component
'st_0_QOIS7XIGJDKNPY6RXMGYRLJQHY2P7IQBWIBA5NQ' terminated
[reptask/replicationtask.c:2868] [1020101] Stop Reason FATAL_ERROR
Error Level FATAL
Frankly speaking not able to understand what is wrong here, so any help is appreciated.
cloudwatch logs:
I changed type to Full load it worked so it is not working for ongoing replication Citus Cloud service don't support it.
I had a similar error to this using Aurora PostgreSQL v14.5 and AWS DMS. I was using a DMS Full load + CDC job (using pglogical behind the scenes) to migrate from one table to another (on the same system).
Issue was resolved by rolling back my PostgreSQL version from 14.5 to 13.7.
I am trying to make a PoC for MongoDB to DocDB migration with DMS.
I've set up a MongoDB instance with some dummy data and an empty DocDB. Source and Target endpoints are also set in DMS and both of them are connecting successfully to my databases.
When I create a migration task in DMS everything seems to be working fine. All existing data is successfully replicated from the MongoDB instance to DocDB and the migration task state is at "Load complete, replication ongoing".
At this point I tried creating new entries in existing collections as well as creating new empty collections in MongoDB but nothing happens in DocDB. If I understand correctly the replication should be real time and anything I create should be replicated instantly?
Also there are no indication of errors or warnings whatsoever... I don't suppose its a connectivity issue to the databases since the initial data is being replicated.
Also the users I am using for the migration have admin privileges in both databases.
Does anyone have any suggestions?
#PPetkov - could you check the following?
1. Check if right privileges were assigned to the user in the MongoDB endpoint according to https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.MongoDB.html.
2. Check if replicate set to capture changes was appropriately configured in the MongoDB instance.
3. Once done, try to search for "]E:" or "]W:" in the CloudWatch logs to understand if there are any noticeable failures/warnings.