I want to start existing and ongoing replication from source MongoDB to destination RDS Postgres.
All the security network is done and successful.
RDS VPC and DMS VPC are connected using VPC peering, target endpoint test connection status is successful
DMS VPC IP is whitelisted in another AWS account MongoDB server, source endpoint test connection status is successful
In the Endpoint schemas section both database schemas are visible.
Created a DMS migration task to start the migration of one document of MongoDB to Postgres Database.
Premigration assessments were successful.
Empty scheme with the MongoDB schema name got created in PostgresDB.
The table awsdms_apply_execution is also empty in Postgres DB.
But the task shows running with error and error, and cloud watch logs also show no error.
Im trying to create a connection for my SQL instance in GCP following their guide:
https://cloud.google.com/sql/docs/mysql/connect-admin-proxy
I set up the proxy running but I cant connect to my server.
I use MySQL workbench and the connection just timeout.
I went trough the trouble shoot guide and could not find the issue.
No errors in the cloud logs.
I try to connect using the owner google account of the project (I have all the permissions).
Cloud SQL Admin API is enabled.
I entered the password in the menu.
I saw another google guide telling to white list your IP.
I did this and its the same error.
It seems like there is a firewall or something is blocking from GCP to connect to the server but Im not sure what.
The solution for me was:
Use Cloud SQL authorized network as JM Gelilio suggested and to use pgAdmin 4 for Postgres connections.
I need to connect AWS Athena (which is binded to AWS s3 bucket) to a Postgres SQL database.
I have tried to connect with Tableau and PowerBI successfully (following the instructions in documentation) and the result was successful.
I think that I need to use the JDBC connector already installed on my machine and try to create a server in Postgres, but I cannot see any option in pgadmin in order to connect AWS athena to the server.
Any ideas?
Thank you in advance!
We are trying to export data from an Amazon Aurora PostgreSQL database to an S3 buckets. The code being used is like this:
SELECT * FROM analytics.my_test INTO OUTFILE S3
's3-us-east-2://myurl/sampledata'
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n';
MANIFEST ON
OVERWRITE ON;
All permissions have been set up but we get the error
SQL Error [42601]: ERROR: syntax error at or near "INTO" Position: 55
Does this only work with a MySQL database?
It is fairly new feature on Aurora Postgres, but it is possible to export the query result into a file on s3: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/postgresql-s3-export.html#postgresql-s3-export-file
The syntax is not the same as for MySQL though. For Postgres it is:
SELECT * from aws_s3.query_export_to_s3('select * from sample_table',
aws_commons.create_s3_uri('sample-bucket', 'sample-filepath', 'us-west-2')
);
I believe saving SQL select output data in S3 ONLY works for Amazon Aurora MySQL DB. I don't see any reference in the official documentation that mentions the same for Amazon Aurora PostgresSQL.
Here are snippets from official documentation that I referred to
Integrating Amazon Aurora MySQL with Other AWS Services
Amazon Aurora MySQL integrates with other AWS services so that you can
extend your Aurora MySQL DB cluster to use additional capabilities in
the AWS Cloud. Your Aurora MySQL DB cluster can use AWS services to do
the following:
Synchronously or asynchronously invoke an AWS Lambda function using
the native functions lambda_sync or lambda_async. For more
information, see Invoking a Lambda Function with an Aurora MySQL
Native Function.
Load data from text or XML files stored in an Amazon Simple Storage
Service (Amazon S3) bucket into your DB cluster using the LOAD DATA
FROM S3 or LOAD XML FROM S3 command. For more information, see Loading
Data into an Amazon Aurora MySQL DB Cluster from Text Files in an
Amazon S3 Bucket.
Save data to text files stored in an Amazon S3 bucket from your DB
cluster using the SELECT INTO OUTFILE S3 command. For more
information, see Saving Data from an Amazon Aurora MySQL DB Cluster
into Text Files in an Amazon S3 Bucket.
Automatically add or remove Aurora Replicas with Application Auto
Scaling. For more information, see Using Amazon Aurora Auto Scaling
with Aurora Replicas.
Integrating Amazon Aurora PostgreSQL with Other AWS Services
Amazon Aurora integrates with other AWS services so that you can
extend your Aurora PostgreSQL DB cluster to use additional
capabilities in the AWS Cloud. Your Aurora PostgreSQL DB cluster can
use AWS services to do the following:
Quickly collect, view, and assess performance for your Aurora
PostgreSQL DB instances with Amazon RDS Performance Insights.
Performance Insights expands on existing Amazon RDS monitoring
features to illustrate your database's performance and help you
analyze any issues that affect it. With the Performance Insights
dashboard, you can visualize the database load and filter the load by
waits, SQL statements, hosts, or users.
For more information about Performance Insights, see Using Amazon RDS
Performance Insights.
Automatically add or remove Aurora Replicas with Aurora Auto Scaling.
For more information, see Using Amazon Aurora Auto Scaling with Aurora
Replicas.
Configure your Aurora PostgreSQL DB cluster to publish log data to
Amazon CloudWatch Logs. CloudWatch Logs provide highly durable storage
for your log records. With CloudWatch Logs, you can perform real-time
analysis of the log data, and use CloudWatch to create alarms and view
metrics. For more information, see Publishing Aurora PostgreSQL Logs
to Amazon CloudWatch Logs.
Ther is no mention of saving data to S3 for PostgresSQL
I just downloaded microstrategy 10.6.0 and want to connect to my redshift cluster. I've added data source and all information needed in the picture:
but it keeps getting error :
The jdbc used is com.microstrategy.jdbc.redshift.RedshiftDriver
Any solution to connect to redshift data source? Thanks