I have once execute SQL task in my KTR and its has multiple Insert statement. I want to understand how transaction control work here. Do I need COMMIT after every Insert or only at end.
You do not need to commit while using 'execute SQL' step in Spoon. The commit happens automatically.
Related
I'm trying to use boto3 redshift-data client to execute transactional SQL for external table (Redshift spectrum) with following statement,
ALTER TABLE schema.table ADD IF NOT EXISTS
PARTITION(key=value)
LOCATION 's3://bucket/prefix';
After submit using execute_statement, I received error "ALTER EXTERNAL TABLE cannot run inside a transaction block".
I tried use VACUUM and COMMIT commands before the statement, but it will just mention that VACUUM or COMMIT cannot run inside a transaction block.
How may I successfully execute such statement?
This has to do with the settings of your bench. You have an open transaction at the start of every statement you run. Just add “END;” before the statement that needs to run outside of a transaction and things should work. Just make sure you launch both commands at the same time from your bench.
Like this:
END; VACUUM;
It seems not quite easy to run transactional SQL through boto3. However, I found a workaround using the redshift_connector library.
import redshift_connector
connection = redshift_connector.connect(
host=host, port=port, database=database, user=user, password=password
)
connection.autocommit = True
connection.cursor.execute(transactional_sql)
connection.autocommit = False
Reference - https://docs.aws.amazon.com/redshift/latest/mgmt/python-connect-examples.html#python-connect-enable-autocommit
If some script fails during migration , flyway won't add record to schema_version in DB2 db for failed statement.
Do you have any idea how to avoid this situation?
I did a migration, 4th script failed, i expect this script will have status ABORTED/FAILED
One explanation for flyway behavior difference that you observe is the way Oracle handles DDL (implicit commit before/after each DDL) as compared with how Db2 handles DDL (implements DDL under transaction control by default). So with Db2 it's possible to arrange for each migration to be atomic and to rollback upon failure - meaning that there is nothing to repair, and therefore no repair action required as the flyway Oracle implementation may need.
I am trying to export one table from Aquastudio into CSV file. The table has approximately 4.4 million rows. When I am trying to use the export window function in the aqua studio, I am facing the following error:
Error: ERROR: current transaction is aborted, commands ignored until end of transaction block
I am not understanding what the problem is. I read few articles regarding this error and found that this is happening due to some error in the last postgreSQL command. I did not use any SQL commands for this export and I dont know how to debug this. I am also unable to view the log files.
Use rollback to cancel the previous query. After that, you will be able to execute your current query.
You probably shouldn't be exporting millions of rows through a JDBC/ODBC connection, especially for Redshift.
For Redshift, please use the UNLOAD command documented here. You'll have to UNLOAD the file to S3 and download it from there.
For Postgres, use COPY TO as documented here.
I am using execute command tag from my liquibase changesets and this inturn is configured to run the sqls in oracle instant client sql plus.
when i run a liquibase update on my changelogxml everything works fine and the liquibase update is sucessfull.I can see the changes to the table also.
But when i try to fail the update process by giving a syntax error in my sql file refered in the changeset.Liquibase still returns liquibase update sucessfull.I expected it to throw sql errors.The sql when run seperately in toad throws syntax error.What should i do to get the error displayed out.?
Datical has created a custom Liquibase change tag that executes SQL using the sqlplus command line client. It was surprisingly much more complicated that you might think.
Some of the issues we had to deal with:
we had to do things to ensure that the sql files always had certain statements in place, and never had certain other statements. This might include things like setting the schema, ensuring that the only spool commands were ones we knew about, that the script had an 'EXIT' command, and ensuring that whenever there was a SQL error that the exit code was returned.
The sqlplus executable does not return an exit code (i.e. a non-zero exit code form the native process) in all cases, and instead will write errors to an error table in the database. The table where sqlplus writes errors is called sperrorlog, and this may be what you will need to look into.
I can't really go into all the details, but just know that what you are attempting to do is neither simple nor straightforward.
The updateSQL liquibase command does not seem to throw an error when running updateSQL command line as it generates the relevant SQL statements for the changeset along with an entry to be created in DATABASECHANGELOG table.
My requirement is that I can only generate the SQL and hand it over to my DBA. But will liquibase throw error even if one of the changeSets in the xml fails to get created prior to generating the SQL for the changeset ?
Please help
UpdateSQL will not fail if a changeSet is already ran. The purpose of Liquibase is to track which changeSets have been applied and only "execute" changeSets that have not been ran while skipping ones that have.
If you run in regular update mode, Liquibase will directly execute each changeSet in turn. If you run in updateSql mode, Liquibase will not actually execute the SQL but instead output what it would have ran.
Liquibase will not throw any errors in updateSQL. However, if the state of the database you are going to execute the SQL file against is different than the database you run updateSQL against, the resulting SQL may not be valid. There is no re-checking of whether a changeSet has executed in the SQL output, it is just a simple "run these commands" script.