According to Presto MySQL connector docs I see that DROP table is supported operation for this connector.
Yet I receive DROP TABLE is disabled in this catalog when I try to do DROP table operation.
Is this some setting in the catalog .properties file?
I found that there is a property in .properties file of the catalog.
allow-drop-table=true
It does the job.
Related
Hi I created a Database in Azure Synapse Studio and I can see the database and table in there, Now I have created a Notebook where I have added the required libraries but I am unable to read the table by below code. Can anyone fix what wrong am i doing here ?
My database name is Utilities_66_Demo . It gives me error as
AnalysisException: Path does not exist:
abfss://users#stcdmsynapsedev01.dfs.core.windows.net/Utilities_66_Demo.parquet
From where should I take the path? I tried to follow the MS article. Where Do I read path? if I click on edit Database, i get this
%%pyspark
df = spark.read.load('abfss://users#stcdmsynapsedev01.dfs.core.windows.net/Utilities_66_Demo.parquet', format='parquet')
display(df.limit(10))
Trying to access the created Lake Database Table:
Selected Azure Synapse Analytics:
I select my workspace and in dropdown there is no table shown:
I select Edit and put my Db name and Table name and it says Invalid
details.
Now I select Azure Dedicated Synapse Pool from Linked Service,
I get no option to select in SQL Pool or Table, and without SQL Pool I am unable to create a Linked service just by inserting Table name:
You can directly go to your ADLS and right click the parquet file and select properties. There, you will be able to find the ABFSS path which is in the format :
abfss://<container_name>#<storage_account_name>.dfs.core.windows.net/<path
I have been trying to add a new column to a target Postgres DB using the transformation rule. I have set up the DMS with CDC.
DMS is able to sync all changes from the table but not performing ADD/Modify column operation.
Any comments on the above? What I am missing here?
I tried to build an Offer-Ready Docker container on Azure Cloud. Although I created a new (blank) table in PostgreSQL, I got this strange error message.
javax.servlet.ServletException: org.eclipse.jetty.servlet.ServletHolder$1: org.flywaydb.core.api.FlywayException: Found non-empty schema(s) "public" without schema history table! Use baseline() or set baselineOnMigrate to true to initialize the schema history table.
I double-checked the database, there is no table in schema "public". I didn't have that problem on AWS. Has anybody an idea what is different on Azure?
I had the same experience once.
The PostgreSQL database on Azure seemed empty (\dt returned no results),
But Flyway claimed the database was not empty (and therefore would not apply the migration scripts, for fear of interfering with whatever was already there).
Here is what I did was:
Create a new schema within the database e.g. myschema
Delete the default schema called public
Add the parameter currentSchema=myschema to the JDBC URL
And then it worked. I never got to find out what the root cause of this problem was.
EDIT: This link might provide more information on what objects are in the "public" schema by default on Azure PostgreSQL: https://community.atlassian.com/t5/Jira-questions/Re-quot-database-that-is-not-empty-quot-when-trying-to-use-azure/qaq-p/1308795/comment-id/410329#M410329
I'm trying to setup a simple replication from MySQL to Postgres. Identical schemas. After following the steps in the Demo Tutorial with a slight change (using MySQL and Postgres drivers) I am still unable to get the replication working.
A few changes were needed based on complaints after running bin/sym
SET GLOBAL show_compatibility_56 = ON needed to be set in the MySQL DB
For Postgres I needed to use protocolVersion=3 instead of 2 which was set in the example.
The weird thing is that SymmetricDS is able to create the sym_* tables, but complains about not being able to read them. I have verified that the tables do not exist before bin/sym is run, but do exist after. Here is an excerpt from the log
// Successful creation of table
[store-001] - PostgreSqlSymmetricDialect - DDL applied: CREATE TABLE "sym_notification"(
"notification_id" VARCHAR(128) NOT NULL,
...
PRIMARY KEY ("notification_id")
)
...
// Unable to read from created table
[store-001] - PostgreSqlDdlReader - Failed to read table: sym_notification
[store-001] - PostgreSqlDdlReader - Failed to read table: sym_notification
[store-001] - AbstractDatabaseWriter - Did not find the sym_notification table in the target database
[store-001] - PostgreSqlDdlReader - Failed to read table: sym_monitor
[store-001] - PostgreSqlDdlReader - Failed to read table: sym_monitor
[store-001] - AbstractDatabaseWriter - Did not find the sym_monitor table in the target database
The same error apply for all the sym_* tables.
The databases are running in Docker, but since SymmetricDS is not complaining about being unable to connect, and is able to create the tables, I assume it is not related to Docker.
The database in the Postgres DB is created by the same user as specified in engines/store-001.properties. Could this still have something to do with roles and access privileges?
If you upgrade to the latest JDBC driver from Postgres it will work.
Replace the existing Postgres driver from the lib directory from the latest from here: https://jdbc.postgresql.org/download.html
Try to connect to the postgres database with the same username/password used by symmetric-ds from some DB navigator, for example Jetbrain's Datagrip and then try inserting, updating, selecting something from sym_* tables. Assign access rights to the user if necessary.
When using Postgres 9.6.1 (current latest release) the following error is logged on the server when running bin/sym
ERROR: column am.amcanorder does not exist at character 427
The problem was resolved by using Postgres 9.5.5 instead thanks to Nick Barnes pointing this out in a comment.
I'm trying to use flyway on a DB2 / zOS instance and I get an error when trying to migrate or baseline.
java.lang.UnsupportedOperationException: Create Schema - is not supported in db2 on zOS
at org.flywaydb.core.internal.dbsupport.db2zos.DB2zosSchema.doCreate(DB2zosSchema.java:62)
This limitations is documented on Flyway website but how to workaround it?
Actually the DB is in place and schema is yet created with the same name gave into config as first and only schema. ("flyway.schemas=MYSCHEMA")
Migration should only execute a bunch of insert.
Baseline should only create flyway metadata table (flyway.table)
What am I missing?