REVINFO table is missing the sequence "revinfo_seq" - postgresql

I am migrating to SpringBoot 3.0.1 and updated "hibernate-envers" version to "6.1.6.Final". My DB is PostgreSQL 13.6.
Hibernate is configured to create the DB schema:
spring.jpa.hibernate.ddl-auto:create
After starting the application I get the following error:
pim 2022-12-27 12:00:13,715 WARN C#c7b942ec-33b4-4749-b113-22cbb2946a8d [http-nio-9637-exec-1] SqlExceptionHelper/133 - SQL Error: 0, SQLState: 42P01
pim 2022-12-27 12:00:13,715 ERROR C#c7b942ec-33b4-4749-b113-22cbb2946a8d [http-nio-9637-exec-1] SqlExceptionHelper/138 - ERROR: relation "revinfo_seq" does not exist
Position: 16
The revinfo table look like this:
create table revinfo
(
revision bigint not null
primary key,
client_id varchar(255),
correlation_id varchar(255),
origin varchar(255),
request_id varchar(255),
revision_timestamp bigint not null,
timestamp_utc timestamp with time zone,
user_name varchar(255)
);
The sequence "revinfo_seq" does not exist, but in the old DB structure with envers
5.6.8.Final
and SpringBoot 2.6.6 it didn't exist either without any problems.
What am i Missing?
I tried to toggle the paramter
org.hibernate.envers.use_revision_entity_with_native_id
but it did not help.

You can solve it with this property:
spring.jpa.properties.hibernate.id.db_structure_naming_strategy: legacy
Tested with Spring Boot 3.0.1
Reason:
Hibernate 6 changed the sequence naming strategy, so it was searching for a sequence ending with "_seq".
You can a really detailed explanation here: https://thorben-janssen.com/sequence-naming-strategies-in-hibernate-6/

Related

How do I stop Liquibase updateSQL from generating Start and successful messages that end up in the SQL?

This command:
liquibase --logLevel=off --changeLogFile=./database_change_log.xml --url='offline:postgresql?outputLiquibaseSql=true' updateSql > database_up.sql
generates this:
Starting Liquibase at Thu, 14 Mar 2019 11:29:12 CDT (version 3.6.3 built at 2019-01-29 11:34:48)
-- *********************************************************************
-- Update Database Script
-- *********************************************************************
-- Change Log: ./database_change_log.xml
-- Ran at: 3/14/19, 11:29 AM
-- Against: null#offline:postgresql?outputLiquibaseSql=true
-- Liquibase version: 3.6.3
-- *********************************************************************
CREATE TABLE databasechangelog (ID VARCHAR(255) NOT NULL, AUTHOR VARCHAR(255) NOT NULL, FILENAME VARCHAR(255) NOT NULL, DATEEXECUTED TIMESTAMP WITHOUT TIME ZONE NOT NULL, ORDEREXECUTED INTEGER NOT NULL, EXECTYPE VARCHAR(10) NOT NULL, MD5SUM VARCHAR(35), DESCRIPTION VARCHAR(255), COMMENTS VARCHAR(255), TAG VARCHAR(255), LIQUIBASE VARCHAR(20), CONTEXTS VARCHAR(255), LABELS VARCHAR(255), DEPLOYMENT_ID VARCHAR(10));
-- Changeset ./database_change_log.xml::1::Jim Barrows
CREATE TABLE topic (id UUID NOT NULL, name VARCHAR(100) NOT NULL, description TEXT, CONSTRAINT TOPIC_PKEY PRIMARY KEY (id));
INSERT INTO databasechangelog (ID, AUTHOR, FILENAME, DATEEXECUTED, ORDEREXECUTED, MD5SUM, DESCRIPTION, COMMENTS, EXECTYPE, CONTEXTS, LABELS, LIQUIBASE, DEPLOYMENT_ID) VALUES ('1', 'Jim Barrows', './database_change_log.xml', NOW(), 1, '8:7b01de4dcdab4ed25f79ce192ba538f6', 'createTable tableName=topic', '', 'EXECUTED', NULL, NULL, '3.6.3', '2580954069');
Liquibase command 'updateSql' was executed successfully.
The problem is that I don't want the "Starting liquibase" or success lines in the SQL.
Note: off is from the docs: --logLevel= Execution log level (debug, info, warning, severe, off).
I've tried to use "severe" & "error" as well
This is version 3.6.3, and version 3.5.3 does not do this. How do I fix this?
Someone has already opened a bug ticket for this: https://liquibase.jira.com/browse/CORE-3342.
This seemingly got introduced from version 3.6 on.
EDIT:
So maybe downgrade or even better propose a fix in form of a pull request. :-)
Wondering about why updateSQL output would be written with a logging framework
I just discovered that you could pass in migrationSqlOutputFile as a parameter instead of redirecting the standard output to a file. That way the logging should not be mingled with output from updateSQL.
(Have not tried this though.)
Use --outputFile option in the liquibase command to direct the sql statements to a file. I use liquibase version 3.7.0 and this option works.
Since 3.4.0 release of Liquibase, you can use outputFile flag. Details are here:
https://www.liquibase.org/2015/07/without-a-connection.html

Why is liquibase deleting databasechangelog rows and trying to create a renamed database table?

I am using postgres 10.5 and liquibase 3.6.2 on a Mac.
I nuke & re-create my database, run liquibase update, and it works.
But a second liquibase update fails with an exception that the pkey already exists.
After the first liquibase update, the databasechangelog table contains 97 entries. After the second, it contains 10, and the time and deployment ids for those are different than they were after the first update!
Table foo was created in an early change.
Later it was changed to be named bar, but the pkey is still foo.pkey.
Liquibase-update should not be trying to re-create foo, but it does, and fails because foo.pkey already exists.
A) In general, how can I get liquibase to output more info about what it's doing? I tried both of the commands:
liquibase --logLevel=debug --logFile=`pwd`/foo.log update
liquibase --logLevel debug --logFile `pwd`/foo.log update
Both seem to work the same, and foo.log isn't created and there's no more output in the terminal.
B) How can I stop liquibase from trying to re-make this and nuking my databasechangelog?
I tried to make a small example that fails, but this seems to work... Others here are using it with postgres 9.5.10 with no problem...
All I see in the terminal is:
Starting Liquibase at Wed, 14 Nov 2018 13:06:44 PST (version 3.6.2 built at 2018-07-03 11:28:09)
Unexpected error running Liquibase: ERROR: relation "cant_change_pkey" already exists [Failed SQL: CREATE TABLE nuss.cant_change (message_id UUID NOT NULL, origin VARCHAR(4), type VARCHAR(12) NOT NULL, CONSTRAINT CANT_CHANGE_PKEY PRIMARY KEY (message_id), UNIQUE (message_id))]
liquibase.exception.MigrationFailedException: Migration failed for change set db/changelog/changelog-new1.xml::first-one::rstrauss:
Reason: liquibase.exception.DatabaseException: ERROR: relation "cant_change_pkey" already exists [Failed SQL: CREATE TABLE nuss.cant_change (message_id UUID NOT NULL, origin VARCHAR(4), type VARCHAR(12) NOT NULL, CONSTRAINT CANT_CHANGE_PKEY PRIMARY KEY (message_id), UNIQUE (message_id))]
at liquibase.changelog.ChangeSet.execute(ChangeSet.java:637)
at liquibase.changelog.visitor.UpdateVisitor.visit(UpdateVisitor.java:53)
at liquibase.changelog.ChangeLogIterator.run(ChangeLogIterator.java:78)
at liquibase.Liquibase.update(Liquibase.java:202)
at liquibase.Liquibase.update(Liquibase.java:179)
at liquibase.integration.commandline.Main.doMigration(Main.java:1205)
at liquibase.integration.commandline.Main.run(Main.java:191)
at liquibase.integration.commandline.Main.main(Main.java:129)
Caused by: liquibase.exception.DatabaseException: ERROR: relation "cant_change_pkey" already exists [Failed SQL: CREATE TABLE nuss.cant_change (message_id UUID NOT NULL, origin VARCHAR(4), type VARCHAR(12) NOT NULL, CONSTRAINT CANT_CHANGE_PKEY PRIMARY KEY (message_id), UNIQUE (message_id))]
at liquibase.executor.jvm.JdbcExecutor$ExecuteStatementCallback.doInStatement(JdbcExecutor.java:356)
at liquibase.executor.jvm.JdbcExecutor.execute(JdbcExecutor.java:57)
at liquibase.executor.jvm.JdbcExecutor.execute(JdbcExecutor.java:125)
at liquibase.database.AbstractJdbcDatabase.execute(AbstractJdbcDatabase.java:1229)
at liquibase.database.AbstractJdbcDatabase.executeStatements(AbstractJdbcDatabase.java:1211)
at liquibase.changelog.ChangeSet.execute(ChangeSet.java:600)
... 7 common frames omitted
Caused by: org.postgresql.util.PSQLException: ERROR: relation "cant_change_pkey" already exists
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2476)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2189)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:300)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:428)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:354)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:301)
at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:287)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:264)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:260)
at liquibase.executor.jvm.JdbcExecutor$ExecuteStatementCallback.doInStatement(JdbcExecutor.java:352)
... 12 common frames omitted
For more information, please use the --logLevel flag

Confluent Kafka Sink Connector is not loading data to Postgres table

I am trying to load data to Postgres table(s) through Kafka Sink connector but I am getting the following error:
Caused by: org.apache.kafka.connect.errors.ConnectException: Cannot ALTER to add missing field SinkRecordField{schema=Schema{STRING}, name='A_ABBREV', isPrimaryKey=false}, as it is not optional and does not have a default value
The table in Postgres DB already has the field A_ABBREV, but now sure why I am getting missing field error.
Did anyone face the similar kind of issue?
Below is my Sink Connector Configuration:
connector.class=io.confluent.connect.jdbc.JdbcSinkConnector
table.name.format=AGENCY
connection.password=passcode
topics=AGENCIES
tasks.max=1
batch.size=10000
fields.whitelist=A_ID, A_NAME, A_ABBREV
connection.user=pmmdevuser
name=partner5-jdbcSinkConnector
connection.url=jdbc:postgresql://aws-db.sdfdgfdrwwisc.us-east- 1.rds.amazonaws.com:3306/pmmdevdb?currentSchema=ams
insert.mode=upsert
pk.mode=record_value
pk.fields=A_ID
auto.create=false
I am using Liquibase scripts to create tables and below is the create query from the postgres DB which got created through Liquibase scripts:
"CREATE TABLE gds.agency
(
a_id integer NOT NULL,
a_name character varying(100) COLLATE pg_catalog."default" NOT NULL,
a_abbrev character varying(8) COLLATE pg_catalog."default" NOT NULL,
source character varying(255) COLLATE pg_catalog."default" NOT NULL DEFAULT 'AMS'::character varying,
CONSTRAINT pk_agency PRIMARY KEY (a_id),
CONSTRAINT a_abbrev_uk1 UNIQUE (a_abbrev)
)"
From my experience, this means that that the field definition for the sink does not match the field definition for the source table/database. Make sure the field definitions match. Inspect the individual record the sink connector is trying to write to your target db. You should be able to see this insert statement in debug mode in the stack trace. take that query and run it manually to get a clearer idea of the error from the database.

Neos 4.0.x setup fails

While installing neos it throws this exception:
An exception occurred while executing 'CREATE TABLE flow3_resource_resourcepointer (hash VARCHAR(255) NOT NULL, PRIMARY KEY(hash)) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci ENGINE = InnoDB':
SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key length is 767 bytes
The exception details have been logged to the Flow system log.
Exception in line 184 of /Users/julius/Documents/dev/neos/Packages/Libraries/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php: An exception occurred while executing 'CREATE TABLE flow3_resource_resourcepointer (hash VARCHAR(255) NOT NULL, PRIMARY KEY(hash)) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci ENGINE = InnoDB':
SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key length is 767 bytes - See also: 201805011913565a6e91.txt
Exception Code 1355480641
Exception Type Neos\Flow\Core\Booting\Exception\SubProcessException
Log Reference 20180501191355663ebf
Thrown in File Packages/Framework/Neos.Flow/Classes/Core/Booting/Scripts.php
Line 677
With Neos 4.0 the default charset and collation was changed to utf8mb4 which requires longer keys and therefore requires at least MySQL 5.7.x or MariaDB 10.2.x. I assume you try to install Neos 4.0 on an older database version.

Spring Boot 2 - H2 Database - #SpringBootTest - Failing on org.h2.jdbc.JdbcSQLException: Table already exists

Unable to test Spring Boot & H2 with a script for creation of table using schema.sql.
So, what’s happening is that I have the following properties set:
spring.datasource.driver-class-name=org.h2.Driver
spring.datasource.initialization-mode=always
spring.datasource.username=sa
spring.datasource.password=
spring.datasource.platform=h2
spring.datasource.url=jdbc:h2:mem:city;MODE=PostgreSQL;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE
spring.jpa.database-platform=org.hibernate.dialect.H2Dialect
spring.jpa.generate-ddl=false
spring.jpa.hibernate.ddl-auto=update
spring.jpa.show-sql=true
and, I expect the tables to be created using the schema.sql. The application works fine when I run gradle bootRun. However, when I run tests using gradle test, my tests for Repository passes, but the one for my Service fails stating that it’s trying to create the table when the table already exists:
Exception raised:
Caused by: org.h2.jdbc.JdbcSQLException: Table "CITY" already exists;
SQL statement:
CREATE TABLE city ( id BIGINT NOT NULL, country VARCHAR(255) NOT NULL, map VARCHAR(255) NOT NULL, name VARCHAR(255) NOT NULL, state VARCHAR(2555) NOT NULL, PRIMARY KEY (id) ) [42101-196]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
at org.h2.message.DbException.get(DbException.java:179)
at org.h2.message.DbException.get(DbException.java:155)
at org.h2.command.ddl.CreateTable.update(CreateTable.java:117)
at org.h2.command.CommandContainer.update(CommandContainer.java:101)
at org.h2.command.Command.executeUpdate(Command.java:260)
at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:192)
at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:164)
at com.zaxxer.hikari.pool.ProxyStatement.execute(ProxyStatement.java:95)
at com.zaxxer.hikari.pool.HikariProxyStatement.execute(HikariProxyStatement.java)
at org.springframework.jdbc.datasource.init.ScriptUtils.executeSqlScript(ScriptUtils.java:471)
... 105 more
The code is setup and ready to recreate the scenario. README has all the information ->
https://github.com/tekpartner/learn-spring-boot-data-jpa-h2
If the tests are run individually, they pass. I think the problem is due to schema.sql being executed twice against the same database. It fails the second time as the tables already exist.
As a workaround, you could set spring.datasource.continue-on-error=true in application.properties.
Another option is to add the #AutoConfigureTestDatabase annotation where appropriate so that a unique embedded database is used for each test.
There are 2 other possible solutions you could try:
Add a drop table if exists [tablename] in your schema.sql before you create the table.
Change the statement from CREATE TABLE to CREATE TABLE IF NOT EXISTS