I'm trying to use boto3 redshift-data client to execute transactional SQL for external table (Redshift spectrum) with following statement,
ALTER TABLE schema.table ADD IF NOT EXISTS
PARTITION(key=value)
LOCATION 's3://bucket/prefix';
After submit using execute_statement, I received error "ALTER EXTERNAL TABLE cannot run inside a transaction block".
I tried use VACUUM and COMMIT commands before the statement, but it will just mention that VACUUM or COMMIT cannot run inside a transaction block.
How may I successfully execute such statement?
This has to do with the settings of your bench. You have an open transaction at the start of every statement you run. Just add “END;” before the statement that needs to run outside of a transaction and things should work. Just make sure you launch both commands at the same time from your bench.
Like this:
END; VACUUM;
It seems not quite easy to run transactional SQL through boto3. However, I found a workaround using the redshift_connector library.
import redshift_connector
connection = redshift_connector.connect(
host=host, port=port, database=database, user=user, password=password
)
connection.autocommit = True
connection.cursor.execute(transactional_sql)
connection.autocommit = False
Reference - https://docs.aws.amazon.com/redshift/latest/mgmt/python-connect-examples.html#python-connect-enable-autocommit
Related
I am trying to run the following query in pgAdmin:
CREATE DATABASE abc;
CREATE DATABASE xyz;
And I get the following error:
ERROR: current transaction is aborted, commands ignored until end of transaction block
SQL state: 25P02
I'm relatively new to postgres.
With SQL Server it's possible to create multiple databases in a single query with the "GO" statement in between if necessary.
I've tried to google this error, and most answers are to simply run each line separately.
That would work, but I'm curious why this doesn't work.
It may also be a setting in pgAdmin.
The "autocommit" is currently on. I've tried it off, and same result.
I'm using postgres 14.5 (in aws)
We have a connection in Exasol (v7.0.18) to PostgreSQL (v14) created like this
create or replace connection POSTGRES_DB to
'jdbc:postgresql://hostname:5432/my_db?useCursorFetch=true&defaultFetchSize=2147483648'
user 'abc'
identified by <>;
I am running an export statement using this connection like this:
EXPORT MY_SCHEMA.TEST_TABLE
INTO JDBC AT POSTGRES_DB
TABLE pg_schema.test_table
truncate;
This works without any error.
The issue is that it runs only one insert statement in the PostgresSQL at a time. I am expecting multiple inserts running at a time in PostgresSQL.
This documentation page says Importing from Exasol databases is always parallelized. For Exasol, loading tables directly is significantly faster than using the STATEMENT option.
How can I make the export statement do parallel insert into PostgreSQL?
I have a PostgreSQL database running on an Azure machine. When I try to create a table on a database, I get an error "cannot execute CREATE TABLE in a read-only transaction". The SQL query is being executed by a python script using a sqlalchemy engine. But I tried a similar query in PGAdmin installed on my machine and I get the same error. And I noticed that I do not have this issue if I connect to the database from a colleague's machine.
After further research, I found that if I execute SELECT pg_is_in_recovery(); in my PGAdmin, it returns true. And false on my colleague's machine.
Let me know if there is any way to correct this
SELECT pg_is_in_recovery() - returned true = Database has only Read Acces
can you check your permission?
you can check postgresql.conf file and atribute default_transaction_read_only
or try this:
begin;
set transaction read write;
alter database exercises set default_transaction_read_only = off;
commit;
The issue was that our posgtresql machine is a HA machine, and that I was connecting to an IP address rather than the domain.
I want to make a change to the postgres.conf file at runtime. However, when I execute the sql with "alter system" via hibernate I get an error
Transaction is marked for rollback only or has timed out
I think this has something to do with alter system commands not allowed to execute inside a transaction block as per the documentation
Only superusers can use ALTER SYSTEM. Also, since this command acts directly on the file system and cannot be rolled back, it is not allowed inside a transaction block or function.
Im trying to understand if its possible to execute this type of command with hibernate and what I need to do to be able to do that?
I am using execute command tag from my liquibase changesets and this inturn is configured to run the sqls in oracle instant client sql plus.
when i run a liquibase update on my changelogxml everything works fine and the liquibase update is sucessfull.I can see the changes to the table also.
But when i try to fail the update process by giving a syntax error in my sql file refered in the changeset.Liquibase still returns liquibase update sucessfull.I expected it to throw sql errors.The sql when run seperately in toad throws syntax error.What should i do to get the error displayed out.?
Datical has created a custom Liquibase change tag that executes SQL using the sqlplus command line client. It was surprisingly much more complicated that you might think.
Some of the issues we had to deal with:
we had to do things to ensure that the sql files always had certain statements in place, and never had certain other statements. This might include things like setting the schema, ensuring that the only spool commands were ones we knew about, that the script had an 'EXIT' command, and ensuring that whenever there was a SQL error that the exit code was returned.
The sqlplus executable does not return an exit code (i.e. a non-zero exit code form the native process) in all cases, and instead will write errors to an error table in the database. The table where sqlplus writes errors is called sperrorlog, and this may be what you will need to look into.
I can't really go into all the details, but just know that what you are attempting to do is neither simple nor straightforward.