error inserting/Updating row in pentaho? - postgresql

I am trying to insert data from CSV file to Table Output in pentaho .
I got following error
2015/04/28 01:07:14 - Table output.0 - Connected to database [vps] (commit=5000)
2015/04/28 01:08:05 - Table output.0 - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : Because of an error, this step can't continue:
2015/04/28 01:08:05 - Table output.0 - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : org.pentaho.di.core.exception.KettleException:
2015/04/28 01:08:05 - Table output.0 - Error inserting row into table [fk_test] with values: [NFT-140721054GN00079XXXXXXX], [postpaid], [NON-FA], [L23-APRICOT/PINK-115(40)], [null], [OD40709024274], [82662472], [9-Jul-14], [10-Jul-14], [null], [null], [21-Jul-14], [return_completed], [1], [1379], [0], [0], [0], [0], [0], [39.5], [0], [0], [0], [0], [0], [0.3], [0], [0], [NATIONAL], [0], [0], [0], [0], [FKST-00151], [10-Jul-14], [1379], [sandal], [0], [0], [0]
2015/04/28 01:08:05 - Table output.0 -
2015/04/28 01:08:05 - Table output.0 - Error inserting/updating row
2015/04/28 01:08:05 - Table output.0 - ERROR: value too long for type character varying(21)
2015/04/28 01:08:05 - Table output.0 -
2015/04/28 01:08:05 - Table output.0 -
2015/04/28 01:08:05 - Table output.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.writeToTable(TableOutput.java:377)
2015/04/28 01:08:05 - Table output.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.processRow(TableOutput.java:118)
2015/04/28 01:08:05 - Table output.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2015/04/28 01:08:05 - Table output.0 - at java.lang.Thread.run(Thread.java:745)
2015/04/28 01:08:05 - Table output.0 - Caused by: org.pentaho.di.core.exception.KettleDatabaseException:
2015/04/28 01:08:05 - Table output.0 - Error inserting/updating row
2015/04/28 01:08:05 - Table output.0 - ERROR: value too long for type character varying(21)
2015/04/28 01:08:05 - Table output.0 -
2015/04/28 01:08:05 - Table output.0 - at org.pentaho.di.core.database.Database.insertRow(Database.java:1268)
2015/04/28 01:08:05 - Table output.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.writeToTable(TableOutput.java:255)
2015/04/28 01:08:05 - Table output.0 - ... 3 more
2015/04/28 01:08:05 - Table output.0 - Caused by: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(21)
2015/04/28 01:08:05 - Table output.0 - at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2198)
2015/04/28 01:08:05 - Table output.0 - at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1927)
2015/04/28 01:08:05 - Table output.0 - at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
2015/04/28 01:08:05 - Table output.0 - at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:561)
2015/04/28 01:08:05 - Table output.0 - at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:419)
2015/04/28 01:08:05 - Table output.0 - at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:365)
2015/04/28 01:08:05 - Table output.0 - at org.pentaho.di.core.database.Database.insertRow(Database.java:1235)
2015/04/28 01:08:05 - Table output.0 - ... 4 more
2015/04/28 01:08:05 - Table output.0 - Finished processing (I=0, O=186, R=187, W=186, U=0, E=1)
2015/04/28 01:08:05 - Table output.0 - Error inserting/updating row
2015/04/28 01:08:05 - Table output.0 - ERROR: value too long for type character varying(21)
how i can solve this error .
I checked table defination in postgresql there data type is character varying without size

The error is
ERROR: value too long for type character varying(21)
and there are two values that are over 21 characters.
NFT-140721054GN00079XXXXXXX
L23-APRICOT/PINK-115(40)
You need to either shorten the values or change the character size for the columns in the table fk_test

Related

Implicit conversion of UUID to character varying when using TEIID with postgres

I have a VDB containing a view with a UUID column with this definition
<vdb name="_GENERATED_users_info" version="6">
<import-vdb name="_INTERNAL_internal" version="1"/>
<model name="GEN_view" type="VIRTUAL">
<metadata type="DDL">
<![CDATA[ CREATE VIEW "_view_workflow_data" ("uuid" VARCHAR, "email" VARCHAR) OPTIONS (UPDATABLE 'true') AS SELECT "public"."security_users_info"."uuid" AS "uuid", "public"."security_users_info"."email" AS "email" FROM "public"."security_users_info" ]]></metadata>
</model>
</vdb>
Column security_users_info.uuid is of type UUID.
When I try to execute a query
SELECT "uuid", "email" FROM "_view_workflow_data" WHERE ("uuid" IN (?''718e52dd-058a-4306-9213-c999181163c7') AND 1 = 1)
I get an error org.postgresql.util.PSQLException: ERROR: operator does not exist: uuid = character varying. The error doesn't come from the query I sent but from a query generated by TEIID executed on the physical table
TEIID11004 Error executing statement(s): [Prepared Values: ['718e52dd-058a-4306-9213-c999181163c7'] SQL: SELECT cast(g_0."uuid" AS varchar), g_0."email" FROM "public"."security_users_info" AS g_0 WHERE g_0."uuid" = ?]
I tried to convert the column in the view definition to varchar like this CONVERT("public"."security_users_info"."uuid", VARCHAR) AS "uuid" but I get the same error.
The failing query works fine directly in postgresql as it implicitly converts the uuid text into a uuid type.
I'm aware of the section about UID types in http://teiid.github.io/teiid-documents/master/content/reference/as_jdbc-translators.html but as the implicit conversion seems to work when I execute the query directly in postgresql, I would expect that section not to apply in this situation.
I'm using TEIID 16 with postgresql 9.5 and postgres JDBC driver 42.2.
Am I doing something in a wrong way or is this not supposed to work?
Whole stacktrace:
ERROR# - org.jooq.exception.DataAccessException: SQL [select "uuid", "email" from "_view_workflow_data" where ("uuid" in (?) and 1 = 1)]; TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 _SOURCE_7772c4bdad42f2468b3ecdc71d89634bd9b80ff2ca87900862ab740105049694: 0 TEIID11008:TEIID11004 Error executing statement(s): [Prepared Values: ['718e52dd-058a-4306-9213-c999181163c7'] SQL: SELECT cast(g_0."uuid" AS varchar), g_0."email" FROM "public"."security_users_info" AS g_0 WHERE g_0."uuid" = ?] (c.m.w.c.d.DataObjectVerticle#?)org.jooq.exception.DataAccessException: SQL [select "uuid", "email" from "_view_workflow_data" where ("uuid" in (?) and 1 = 1)]; TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 _SOURCE_7772c4bdad42f2468b3ecdc71d89634bd9b80ff2ca87900862ab740105049694: 0 TEIID11008:TEIID11004 Error executing statement(s): [Prepared Values: ['718e52dd-058a-4306-9213-c999181163c7'] SQL: SELECT cast(g_0."uuid" AS varchar), g_0."email" FROM "public"."security_users_info" AS g_0 WHERE g_0."uuid" = ?]
at org.jooq_3.13.0-SNAPSHOT.DEFAULT.debug(Unknown Source)
at org.jooq.impl.Tools.translate(Tools.java:2717)
at org.jooq.impl.DefaultExecuteContext.sqlException(DefaultExecuteContext.java:755)
at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:382)
at com.mathesio.workflow.core.dataObject.DataObjectService.lambda$select$16(DataObjectService.java:365)
at com.mathesio.teiid.Connections.ConnectionManager.lambda$getConnectionJOOQ$0(ConnectionManager.java:187)
at org.jooq.impl.DefaultDSLContext$3.run(DefaultDSLContext.java:609)
at org.jooq.impl.DefaultDSLContext$3.run(DefaultDSLContext.java:606)
at org.jooq.impl.DefaultDSLContext.lambda$transactionResult0$0(DefaultDSLContext.java:537)
at org.jooq.impl.Tools$12$1.block(Tools.java:4847)
at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3128)
at org.jooq.impl.Tools$12.get(Tools.java:4844)
at org.jooq.impl.DefaultDSLContext.transactionResult0(DefaultDSLContext.java:589)
at org.jooq.impl.DefaultDSLContext.transactionResult(DefaultDSLContext.java:506)
at org.jooq.impl.DefaultDSLContext.transaction(DefaultDSLContext.java:606)
at com.mathesio.teiid.Connections.ConnectionManager.lambda$getConnectionJOOQ$1(ConnectionManager.java:186)
at com.mathesio.workflow.core.database.transaction.TransactionJPA.lambda$new$42(TransactionJPA.java:1813)
at com.mathesio.workflow.core.database.transaction.TransactionJPA.lambda$new$19(TransactionJPA.java:388)
at com.mathesio.workflow.core.database.transaction.TransactionJPA.runCore(TransactionJPA.java:993)
at com.mathesio.workflow.core.database.transaction.TransactionJPA.lambda$run$25(TransactionJPA.java:1306)
at com.mathesio.workflow.core.database.transaction.TransactionJPA$1.call(TransactionJPA.java:755)
at com.mathesio.workflow.core.database.transaction.TransactionJPA$1.call(TransactionJPA.java:738)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at com.mathesio.workflow.core.database.transaction.PriorityFuture.run(PriorityFuture.java:58)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.teiid.jdbc.TeiidSQLException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 _SOURCE_7772c4bdad42f2468b3ecdc71d89634bd9b80ff2ca87900862ab740105049694: 0 TEIID11008:TEIID11004 Error executing statement(s): [Prepared Values: ['718e52dd-058a-4306-9213-c999181163c7'] SQL: SELECT cast(g_0."uuid" AS varchar), g_0."email" FROM "public"."security_users_info" AS g_0 WHERE g_0."uuid" = ?]
at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:131)
at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:67)
at org.teiid.jdbc.StatementImpl.postReceiveResults(StatementImpl.java:783)
at org.teiid.jdbc.StatementImpl.access$100(StatementImpl.java:62)
at org.teiid.jdbc.StatementImpl$2.onCompletion(StatementImpl.java:581)
at org.teiid.client.util.ResultsFuture.done(ResultsFuture.java:131)
at org.teiid.client.util.ResultsFuture.access$200(ResultsFuture.java:36)
at org.teiid.client.util.ResultsFuture$1.receiveResults(ResultsFuture.java:75)
at org.teiid.net.socket.SocketServerInstanceImpl.receivedMessage(SocketServerInstanceImpl.java:281)
at org.teiid.net.socket.SocketServerInstanceImpl.read(SocketServerInstanceImpl.java:333)
at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:423)
at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:590)
at org.teiid.jdbc.PreparedStatementImpl.execute(PreparedStatementImpl.java:240)
at org.jooq.tools.jdbc.DefaultPreparedStatement.execute(DefaultPreparedStatement.java:209)
at org.jooq.impl.Tools.executeStatementAndGetFirstResultSet(Tools.java:3935)
at org.jooq.impl.AbstractResultQuery.execute(AbstractResultQuery.java:294)
at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:368)
... 23 common frames omitted
Caused by: org.teiid.core.TeiidProcessingException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 _SOURCE_7772c4bdad42f2468b3ecdc71d89634bd9b80ff2ca87900862ab740105049694: 0 TEIID11008:TEIID11004 Error executing statement(s): [Prepared Values: ['718e52dd-058a-4306-9213-c999181163c7'] SQL: SELECT cast(g_0."uuid" AS varchar), g_0."email" FROM "public"."security_users_info" AS g_0 WHERE g_0."uuid" = ?]
at org.jboss.teiid#16.0.0//org.teiid.dqp.internal.process.DataTierTupleSource.exceptionOccurred(DataTierTupleSource.java:397)
at org.jboss.teiid#16.0.0//org.teiid.dqp.internal.process.DataTierTupleSource.nextTuple(DataTierTupleSource.java:157)
at org.jboss.teiid#16.0.0//org.teiid.query.processor.relational.AccessNode.nextBatchDirect(AccessNode.java:401)
at org.jboss.teiid#16.0.0//org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
at org.jboss.teiid#16.0.0//org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:139)
at org.jboss.teiid#16.0.0//org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:147)
at org.jboss.teiid#16.0.0//org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:110)
at org.jboss.teiid#16.0.0//org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:160)
at org.jboss.teiid#16.0.0//org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:142)
at org.jboss.teiid#16.0.0//org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:491)
at org.jboss.teiid#16.0.0//org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:361)
at org.jboss.teiid#16.0.0//org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:43)
at org.jboss.teiid#16.0.0//org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:284)
at org.jboss.teiid#16.0.0//org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:283)
at org.jboss.teiid#16.0.0//org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:124)
at org.jboss.teiid#16.0.0//org.teiid.dqp.internal.process.ThreadReuseExecutor$2.run(ThreadReuseExecutor.java:212)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.teiid.translator.TranslatorException: 0 Remote org.teiid.translator.jdbc.JDBCExecutionException: 0 TEIID11008:TEIID11004 Error executing statement(s): [Prepared Values: ['718e52dd-058a-4306-9213-c999181163c7'] SQL: SELECT cast(g_0."uuid" AS varchar), g_0."email" FROM "public"."security_users_info" AS g_0 WHERE g_0."uuid" = ?]
at org.jboss.teiid.translator.jdbc#16.0.0//org.teiid.translator.jdbc.JDBCQueryExecution.execute(JDBCQueryExecution.java:127)
at org.jboss.teiid#16.0.0//org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:405)
at jdk.internal.reflect.GeneratedMethodAccessor106.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.jboss.teiid#16.0.0//org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:228)
at org.jboss.teiid#16.0.0//com.sun.proxy.$Proxy45.execute(Unknown Source)
at org.jboss.teiid#16.0.0//org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:302)
at org.jboss.teiid#16.0.0//org.teiid.dqp.internal.process.DataTierTupleSource.nextTuple(DataTierTupleSource.java:138)
... 17 common frames omitted
Caused by: java.sql.SQLException: Remote org.postgresql.util.PSQLException: ERROR: operator does not exist: uuid = character varying
Hint: No operator matches the given name and argument type(s). You might need to add explicit type casts.
Position: 109
at org.postgresql#42.2.2//org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2433)
at org.postgresql#42.2.2//org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2178)
at org.postgresql#42.2.2//org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306)
at org.postgresql#42.2.2//org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
at org.postgresql#42.2.2//org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
at org.postgresql#42.2.2//org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:155)
at org.postgresql#42.2.2//org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:118)
at org.jboss.ironjacamar.jdbcadapters#1.4.20.Final//org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:504)
at org.jboss.teiid.translator.jdbc#16.0.0//org.teiid.translator.jdbc.JDBCQueryExecution.execute(JDBCQueryExecution.java:119)
... 25 common frames omitted

How to insert rows if not exist in Liquibase in Yaml format

I have a Table A:
A
--------------
id | name |
How to insert rows in this table if row with such NAME already exists.
I need to do it in Liquibase in Yaml format
Liquibase changesets can be executed on a given precondition. In your case you could run a sqlCheck:
- changeSet:
id: 1
author: me
preConditions:
- onFail: MARK_RAN
- sqlCheck:
expectedResult: 0
sql: SELECT COUNT(*) FROM person WHERE name = 'John'
changes:
- insert:
tableName: person
columns:
- column:
name: id
value: 2
- column:
name: name
value: John

Does cascade happen in 1 transaction?

I save the Product which cascade persist the productMaterial. However, when the productMaterial throws DataIntegrityViolationException the product is rollbacked, which seems like cascade is done in 1 transaction, but i don't find any docs saying that it does. Can someone clarify it for me?
NOTE: I DO NOT use #Transactional
Material material = new Material();
material.setId(1);
Product newProduct = new Product();
ProductMaterial productMaterial = new ProductMaterial();
newProduct.setName("bàn chải");
newProduct.setPrice(1000);
newProduct.setCreatedAt(new Date());
newProduct.setProductMaterials(Collections.singletonList(productMaterial));
productMaterial.setProduct(newProduct);
productMaterial.setMaterial(material);
productRepository.save(newProduct);
Here is the hibernate execution:
Hibernate:
/* insert com.vietnam.hanghandmade.entities.Product
*/ insert
into
product
(created_at, name, price, id)
values
(?, ?, ?, ?)
2020-11-10 14:55:38.281 TRACE 65729 --- [nio-8080-exec-2] o.h.type.descriptor.sql.BasicBinder : binding parameter [1] as [TIMESTAMP] - [Tue Nov 10 14:55:38 JST 2020]
2020-11-10 14:55:38.281 TRACE 65729 --- [nio-8080-exec-2] o.h.type.descriptor.sql.BasicBinder : binding parameter [2] as [VARCHAR] - [bàn chải]
2020-11-10 14:55:38.281 TRACE 65729 --- [nio-8080-exec-2] o.h.type.descriptor.sql.BasicBinder : binding parameter [3] as [INTEGER] - [1000]
2020-11-10 14:55:38.281 TRACE 65729 --- [nio-8080-exec-2] o.h.type.descriptor.sql.BasicBinder : binding parameter [4] as [OTHER] - [e5729490-a0f8-48e7-9600-eeeba8b8f279]
Hibernate:
/* insert com.vietnam.hanghandmade.entities.ProductMaterial
*/ insert
into
product_material
(material_id, product_id)
values
(?, ?)
2020-11-10 14:55:38.324 TRACE 65729 --- [nio-8080-exec-2] o.h.type.descriptor.sql.BasicBinder : binding parameter [1] as [INTEGER] - [1]
2020-11-10 14:55:38.324 TRACE 65729 --- [nio-8080-exec-2] o.h.type.descriptor.sql.BasicBinder : binding parameter [2] as [OTHER] - [e5729490-a0f8-48e7-9600-eeeba8b8f279]
2020-11-10 14:55:38.328 WARN 65729 --- [nio-8080-exec-2] o.h.engine.jdbc.spi.SqlExceptionHelper : SQL Error: 0, SQLState: 23503
2020-11-10 14:55:38.328 ERROR 65729 --- [nio-8080-exec-2] o.h.engine.jdbc.spi.SqlExceptionHelper : ERROR: insert or update on table "product_material" violates foreign key constraint "product_material_material_id_fkey"
Detail: Key (material_id)=(1) is not present in table "material".
NOTE: This answer missed the point of the question, which is about “cascading persist” – it talks about “cascading delete” for foreign keys.
The cascading delete or update is part of the action of the system trigger that implements foreign key constraints, and as such it runs in the same transaction as the triggering statement.
I cannot find a place in the fine manual that spells this out, but it is obvious if you think about it: if the cascading delete were run in a separate transaction, it would be possible that the delete succeeds and the cascading delete fails, which would render the database inconsistent and is consequently not an option.

Hibernate Sequence constraint violation with pooled, pooled-lo optimizers on top of Postgresql

Recently I had to partition one of the big PostgreSql table in my project and due to that, I had to change the strategy of generated values for it from IDENTITY to SEQUENCE. After that, I got a problem with sequence generation that was used by Hibernate by default. I use Hibernate 5.2.17-Final, and by default I got: use_new_id_generator_mappings=true. There is a peace of my entity object:
#Setter
#Getter
#ToString(of = { "id", "msisdnA", "msisdnB", "voiceChannel", "idMelody" })
/*#SequenceGenerator(name="id_generator", sequenceName="report_calls_idreportcall_seq")*/
#SQLInsert(sql = "INSERT INTO public.report_calls (dateadded, dateend, datestart, idmelody, idregion, idmelodytype, msisdn_a, msisdn_b, idvoicechannel, idreportcall) " +
"VALUES (?,?,?,?,?,?,?,?,?,?)", check= ResultCheckStyle.NONE)
#Entity
#Table(schema = "public", name = "report_calls")
public class ReportCall {
#Id
#GenericGenerator(name="sequence_generator", strategy="enhanced-sequence",
parameters = {#org.hibernate.annotations.Parameter(name = "sequence_name", value = "report_calls_idreportcall_seq"),
#org.hibernate.annotations.Parameter(name = "optimizer", value="pooled-lo"),
#org.hibernate.annotations.Parameter(name = "initial_value", value = "1"),
#org.hibernate.annotations.Parameter(name = "increment_size", value = "10")})
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "sequence_generator")
#Column(name = "idreportcall", nullable = false)
private long id;
#Column(name = "dateadded")
private LocalDateTime dateAdded = LocalDateTime.now();
#Column(name = "datestart")
private LocalDateTime dateStart;
...
}
My Hibernate configuration is:
#Bean
public EntityManagerFactory entityManagerFactoryBilling() {
log.debug("###Configuring EntityManager for billing");
LocalContainerEntityManagerFactoryBean emf = new LocalContainerEntityManagerFactoryBean();
emf.setPersistenceProvider(new HibernatePersistenceProvider());
emf.setPersistenceUnitName("billingUnit");
emf.setJpaDialect(new HibernateJpaDialect());
emf.setJpaVendorAdapter(new HibernateJpaVendorAdapter());
emf.setDataSource(dataSourceBilling());
emf.setJpaProperties(jpaProperties());
emf.setPackagesToScan("ru.intech.rbt.common.entity.billing");
emf.afterPropertiesSet();
return emf.getObject();
}
private Properties jpaProperties() {
Properties props = new Properties();
props.put(DIALECT, env.getProperty("hibernate.dialect"));
props.put(DRIVER, env.getProperty("hibernate.driver"));
props.put(USE_NEW_ID_GENERATOR_MAPPINGS, env.getProperty("billing.hibernate.use_new_id_generator_mappings", Boolean.class));
props.put(SHOW_SQL, env.getProperty("billing.hibernate.show_sql", Boolean.class));
props.put(FORMAT_SQL, env.getProperty("billing.hibernate.format_sql", Boolean.class));
props.put(USE_SQL_COMMENTS, env.getProperty("billing.hibernate.use_sql_comments", Boolean.class));
props.put(CACHE_REGION_FACTORY, env.getProperty("billing.hibernate.cache.region.factory_class"));
props.put(DEFAULT_CACHE_CONCURRENCY_STRATEGY, env.getProperty("billing.hibernate.cache.default_cache_concurrency_strategy"));
props.put(USE_SECOND_LEVEL_CACHE, env.getProperty("billing.hibernate.cache.use_second_level_cache", Boolean.class));
props.put(USE_QUERY_CACHE, env.getProperty("billing.hibernate.cache.use_query_cache", Boolean.class));
props.put(GENERATE_STATISTICS, env.getProperty("billing.hibernate.generate_statistics", Boolean.class));
log.info("###billingJpaProperties:" + props);
return props;
}
where properties file:
billing.hibernate.use_new_id_generator_mappings=true
billing.hibernate.generate_statistics=false
billing.hibernate.show_sql=false
billing.hibernate.format_sql=true
billing.hibernate.use_sql_comments=true
billing.hibernate.cache.use_query_cache=false
billing.hibernate.cache.use_second_level_cache=true
billing.hibernate.cache.default_cache_concurrency_strategy=READ_WRITE
billing.hibernate.cache.region.factory_class=org.hibernate.cache.ehcache.SingletonEhCacheRegionFactory
After changing of sequence generation strategy, I ran my application and test it with 11 calls of the save() operation of the mentioned entity via EntityManager that I got from "billingUnit" persistence unit. The first 10 calls were stored correctly, no problems were observed, but 11-th call failed. There are Hibernate debug traces:
1 save call()
2018-05-14 13:28:13,144 [992937304751>222[IVR] IVR] DEBUG o.h.e.t.internal.TransactionImpl - begin
2018-05-14 13:28:13,156 [992937304751>222[IVR] IVR] DEBUG org.hibernate.SQL -
select
nextval ('report_calls_idreportcall_seq')
2018-05-14 13:28:13,164 [992937304751>222[IVR] IVR] DEBUG o.h.id.enhanced.SequenceStructure - Sequence value obtained: 219049790
2018-05-14 13:28:13,164 [992937304751>222[IVR] IVR] DEBUG o.h.r.j.i.ResourceRegistryStandardImpl - HHH000387: ResultSet's statement was not registered
2018-05-14 13:28:13,164 [992937304751>222[IVR] IVR] DEBUG o.h.e.i.AbstractSaveEventListener - Generated identifier: 219049790, using strategy: org.hibernate.id.enhanced.SequenceStyleGenerator
2018-05-14 13:28:13,182 [992937304751>222[IVR] IVR] DEBUG o.h.e.t.internal.TransactionImpl - committing
2018-05-14 13:28:13,182 [992937304751>222[IVR] IVR] DEBUG o.h.e.i.AbstractFlushingEventListener - Processing flush-time cascades
2018-05-14 13:28:13,183 [992937304751>222[IVR] IVR] DEBUG o.h.e.i.AbstractFlushingEventListener - Dirty checking collections
2018-05-14 13:28:13,185 [992937304751>222[IVR] IVR] DEBUG o.h.e.i.AbstractFlushingEventListener - Flushed: 1 insertions, 0 updates, 0 deletions to 1 objects
2018-05-14 13:28:13,185 [992937304751>222[IVR] IVR] DEBUG o.h.e.i.AbstractFlushingEventListener - Flushed: 0 (re)creations, 0 updates, 0 removals to 0 collections
2018-05-14 13:28:13,187 [992937304751>222[IVR] IVR] DEBUG o.h.internal.util.EntityPrinter - Listing entities:
2018-05-14 13:28:13,187 [992937304751>222[IVR] IVR] DEBUG o.h.internal.util.EntityPrinter - ru.intech.rbt.common.entity.billing.ReportCall{dateStart=2018-05-14T13:28:10.591, msisdnA=992937304751, melodyType=NO_TYPE, idRegion=1, msisdnB=222, idMelody=0, id=219049790, dateEnd=2018-05-14T13:28:13.142, voiceChannel=SIP_IVR, dateAdded=2018-05-14T13:28:13.142}
2018-05-14 13:28:13,195 [992937304751>222[IVR] IVR] DEBUG org.hibernate.SQL -
INSERT
INTO
public.report_calls
(dateadded, dateend, datestart, idmelody, idregion, idmelodytype, msisdn_a, msisdn_b, idvoicechannel, idreportcall)
VALUES
(?,?,?,?,?,?,?,?,?,?)
2 save call()
2018-05-14 13:29:09,136 [992937304751>222[IVR] IVR] DEBUG o.h.e.t.internal.TransactionImpl - begin
2018-05-14 13:29:09,137 [992937304751>222[IVR] IVR] DEBUG o.h.e.i.AbstractSaveEventListener - Generated identifier: 219049791, using strategy: org.hibernate.id.enhanced.SequenceStyleGenerator
2018-05-14 13:29:09,138 [992937304751>222[IVR] IVR] DEBUG o.h.e.t.internal.TransactionImpl - committing
2018-05-14 13:29:09,138 [992937304751>222[IVR] IVR] DEBUG o.h.e.i.AbstractFlushingEventListener - Processing flush-time cascades
2018-05-14 13:29:09,138 [992937304751>222[IVR] IVR] DEBUG o.h.e.i.AbstractFlushingEventListener - Dirty checking collections
2018-05-14 13:29:09,138 [992937304751>222[IVR] IVR] DEBUG o.h.e.i.AbstractFlushingEventListener - Flushed: 1 insertions, 0 updates, 0 deletions to 1 objects
2018-05-14 13:29:09,138 [992937304751>222[IVR] IVR] DEBUG o.h.e.i.AbstractFlushingEventListener - Flushed: 0 (re)creations, 0 updates, 0 removals to 0 collections
2018-05-14 13:29:09,138 [992937304751>222[IVR] IVR] DEBUG o.h.internal.util.EntityPrinter - Listing entities:
2018-05-14 13:29:09,138 [992937304751>222[IVR] IVR] DEBUG o.h.internal.util.EntityPrinter - ru.intech.rbt.common.entity.billing.ReportCall{dateStart=2018-05-14T13:29:06.538, msisdnA=992937304751, melodyType=NO_TYPE, idRegion=1, msisdnB=222, idMelody=0, id=219049791, dateEnd=2018-05-14T13:29:09.135, voiceChannel=SIP_IVR, dateAdded=2018-05-14T13:29:09.135}
2018-05-14 13:29:09,139 [992937304751>222[IVR] IVR] DEBUG org.hibernate.SQL -
INSERT
INTO
public.report_calls
(dateadded, dateend, datestart, idmelody, idregion, idmelodytype, msisdn_a, msisdn_b, idvoicechannel, idreportcall)
VALUES
(?,?,?,?,?,?,?,?,?,?)
... and so on till 11-th call where the problem occurs:
2018-05-14 13:35:47,871 [992937304751>222[IVR] IVR] DEBUG o.h.e.t.internal.TransactionImpl - begin
2018-05-14 13:35:47,873 [992937304751>222[IVR] IVR] DEBUG org.hibernate.SQL -
select
nextval ('report_calls_idreportcall_seq')
2018-05-14 13:35:47,875 [992937304751>222[IVR] IVR] DEBUG o.h.id.enhanced.SequenceStructure - Sequence value obtained: 219049791
2018-05-14 13:35:47,875 [992937304751>222[IVR] IVR] DEBUG o.h.r.j.i.ResourceRegistryStandardImpl - HHH000387: ResultSet's statement was not registered
2018-05-14 13:35:47,875 [992937304751>222[IVR] IVR] DEBUG o.h.e.i.AbstractSaveEventListener - Generated identifier: 219049791, using strategy: org.hibernate.id.enhanced.SequenceStyleGenerator
2018-05-14 13:35:47,876 [992937304751>222[IVR] IVR] DEBUG o.h.e.t.internal.TransactionImpl - committing
2018-05-14 13:35:47,876 [992937304751>222[IVR] IVR] DEBUG o.h.e.i.AbstractFlushingEventListener - Processing flush-time cascades
2018-05-14 13:35:47,876 [992937304751>222[IVR] IVR] DEBUG o.h.e.i.AbstractFlushingEventListener - Dirty checking collections
2018-05-14 13:35:47,876 [992937304751>222[IVR] IVR] DEBUG o.h.e.i.AbstractFlushingEventListener - Flushed: 1 insertions, 0 updates, 0 deletions to 1 objects
2018-05-14 13:35:47,876 [992937304751>222[IVR] IVR] DEBUG o.h.e.i.AbstractFlushingEventListener - Flushed: 0 (re)creations, 0 updates, 0 removals to 0 collections
2018-05-14 13:35:47,876 [992937304751>222[IVR] IVR] DEBUG o.h.internal.util.EntityPrinter - Listing entities:
2018-05-14 13:35:47,877 [992937304751>222[IVR] IVR] DEBUG o.h.internal.util.EntityPrinter - ru.intech.rbt.common.entity.billing.ReportCall{dateStart=2018-05-14T13:35:43.831, msisdnA=992937304751, melodyType=NO_TYPE, idRegion=1, msisdnB=222, idMelody=0, id=219049791, dateEnd=2018-05-14T13:35:47.870, voiceChannel=SIP_IVR, dateAdded=2018-05-14T13:35:47.870}
2018-05-14 13:35:47,877 [992937304751>222[IVR] IVR] DEBUG org.hibernate.SQL -
INSERT
INTO
public.report_calls
(dateadded, dateend, datestart, idmelody, idregion, idmelodytype, msisdn_a, msisdn_b, idvoicechannel, idreportcall)
VALUES
(?,?,?,?,?,?,?,?,?,?)
2018-05-14 13:35:47,891 [992937304751>222[IVR] IVR] DEBUG o.h.e.jdbc.spi.SqlExceptionHelper - could not execute statement [n/a]
org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "report_calls_pkey"
Detail: Key (idreportcall)=(219049791) already exists.
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2455)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2155)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:288)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:430)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:356)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:168)
at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:135)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:175)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3032)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3547)
at org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:89)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:600)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:474)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:337)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1437)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:494)
at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3245)
at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2451)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:473)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:156)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$100(JdbcResourceLocalTransactionCoordinatorImpl.java:38)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:231)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:68)
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:517)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:761)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:730)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:504)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:292)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
at ru.intech.rbt.common.dao.service.ReportService$$EnhancerBySpringCGLIB$$b718a67e.save(<generated>)
at ru.intech.rbt.common.dao.reserve.proxy.ReportServiceProxy.save(ReportServiceProxy.java:88)
at ru.intech.rbt.common.context.SubscriberContext.createReport(SubscriberContext.java:568)
at ru.intech.ivr.box.threads.CallProcessor.processCDR(CallProcessor.java:466)
at ru.intech.ivr.box.threads.IvrProcessor.run(IvrProcessor.java:184)
at java.lang.Thread.run(Thread.java:745)
2018-05-14 13:35:47,892 [992937304751>222[IVR] IVR] WARN o.h.e.jdbc.spi.SqlExceptionHelper - SQL Error: 0, SQLState: 23505
2018-05-14 13:35:47,892 [992937304751>222[IVR] IVR] ERROR o.h.e.jdbc.spi.SqlExceptionHelper - ERROR: duplicate key value violates unique constraint "report_calls_pkey"
Detail: Key (idreportcall)=(219049791) already exists.
2018-05-14 13:35:47,898 [992937304751>222[IVR] IVR] DEBUG o.h.r.t.b.j.i.JdbcResourceLocalTransactionCoordinatorImpl - JDBC transaction marked for rollback-only (exception provided for stack trace)
java.lang.Exception: exception just for purpose of providing stack trace
Can anybody explain me such strange behavior of Hibernate framework, please? I don't understand why does it generate the same id that Postgres method nextval() using my report_calls_idreportcall_seq? By the way, the sequence initially (before test) looked like that:
current value: 219049790
increment: 1
minimum: 1
maximum: 9223372036854775807
cache: 1
cycled: NO
If I set increment_size = 1 as a parameter of #GenericGenerator everything becomes fine, but I don't want to do that and since it will reduce performance due to the fact that hibernate will hit the database each time when it needs to store new report_call and in production environment I have about 300 threads that in concurrent mode saves these items all the time. Is there any chance to avoid setting increment_size=1 or optimizer="hi/lo" (as I see it is also hit the database to get new id by nextval() each time) ?
It seems that I understood the root cause of my problem. I forced Hibernate to use increment_size=10 and at the same time configured my Postgres sequence to increment new primary key id with value 1 and Postgres was discouraged when Hibernate tried to save an item with id value when 10 values bigger than id had already been stored.
In such a way there was nothing special in that Hibernate took exactly the value that db sequence provide it with since it expected that db also configured to increment pk value to the same gap=>10. When I changed "increment" of my report_calls_idreportcall_seq everything become fine and Hibernate doesn't generate exceptions anymore and at the same time, it hit the db with nextval() request once per 10 save() calls.

Weird issue with hasMany in Grails using PostgreSQL. "ERROR: column <column_name> does not exist"

I'm having a hard time trying to get "hasMany" to work in Grails 2.0.1 using PostgreSQL 9.1. I got two tables:
CREATE TABLE "_QUESTIONS"
(
"QUESTION_ID" bigint NOT NULL,
"TEXT" text,
CONSTRAINT "PK" PRIMARY KEY ("QUESTION_ID" )
)
WITH (
OIDS=FALSE
);
ALTER TABLE "_QUESTIONS"
OWNER TO postgres;
CREATE TABLE "_ANSWERS"
(
"ANSWER_ID" bigint NOT NULL,
"TEXT" text,
"QUESTION_ID" bigint,
CONSTRAINT "PK1" PRIMARY KEY ("ANSWER_ID" ),
CONSTRAINT "FK" FOREIGN KEY ("QUESTION_ID")
REFERENCES "_QUESTIONS" ("QUESTION_ID") MATCH SIMPLE
ON UPDATE NO ACTION ON DELETE NO ACTION
)
WITH (
OIDS=FALSE
);
ALTER TABLE "_ANSWERS"
OWNER TO postgres;
and two domain classes:
class Question {
String text
String toString(){
text
}
static constraints = {
}
static hasMany = [answers: Answer]
static mapping = {
table '`_QUESTIONS`'
version false
id generator: 'identity'
id column: '`QUESTION_ID`'
text column: '`TEXT`'
}
}
class Answer {
String text
Question question
String toString(){
text
}
static constraints = {
}
static belongsTo = [question : Question]
static mapping = {
table '`_ANSWERS`'
version false
id generator: 'identity'
id column: '`ANSWER_ID`'
text column: '`TEXT`'
question column: '`QUESTION_ID`'
}
}
I've generated Views and Controllers for both of them and when I try to browse a particular Question I get the following error:
URI:/hasManyTest/question/show/1
Class:org.postgresql.util.PSQLException
Message:ERROR: column answers0_.question_id does not exist Position: 8
with stack trace:
Line | Method
->> 8 | runWorker in \grails-app\views\question\show.gsp
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Caused by SQLGrammarException: could not initialize a collection: [hasmanytest.Question.answers#1]
->> 26 | doCall in C__Users_root_IdeaProjects_hasManyTest_grails_app_views_question_show_gsp$_run_closure2
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
| 55 | run in C__Users_root_IdeaProjects_hasManyTest_grails_app_views_question_show_gsp
| 1110 | runWorker in java.util.concurrent.ThreadPoolExecutor
| 603 | run in java.util.concurrent.ThreadPoolExecutor$Worker
^ 722 | run . . . in java.lang.Thread
Caused by PSQLException: ERROR: column answers0_.question_id does not exist
Position: 8
->> 2103 | receiveErrorResponse in org.postgresql.core.v3.QueryExecutorImpl
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
| 1836 | processResults in ''
| 257 | execute . in ''
| 512 | execute in org.postgresql.jdbc2.AbstractJdbc2Statement
| 388 | executeWithFlags in ''
| 273 | executeQuery in ''
| 96 | executeQuery in org.apache.commons.dbcp.DelegatingPreparedStatement
| 26 | doCall in C__Users_root_IdeaProjects_hasManyTest_grails_app_views_question_show_gsp$_run_closure2
| 55 | run . . . in C__Users_root_IdeaProjects_hasManyTest_grails_app_views_question_show_gsp
| 1110 | runWorker in java.util.concurrent.ThreadPoolExecutor
| 603 | run . . . in java.util.concurrent.ThreadPoolExecutor$Worker
^ 722 | run in java.lang.Thread
I've done a lot of gymnastics for the past couple of days and nothing seems to help, when I remove association everything works fine though. Am I missing something obvious?
The issue appears to be a mismatch between assumptions made by Grails/GORM mapping and the SQL used to create the tables.
If you omit the quotes around table and column names in the above SQL, tables and columns will be case-insensitive. From http://wiki.postgresql.org/wiki/Things_to_find_out_about_when_moving_from_MySQL_to_PostgreSQL:
Database, table, field and columns names in PostgreSQL are case-independent, unless you created them with double-quotes around their name, in which case they are case-sensitive. In MySQL, table names can be case-sensitive or not, depending on which operating system you are using.
Taking Grails out of the equation, if you create the table using the following:
CREATE TABLE "_QUESTIONS"
(
"QUESTION_ID" bigint NOT NULL,
"TEXT" text,
CONSTRAINT "PK" PRIMARY KEY ("QUESTION_ID" )
)
WITH (
OIDS=FALSE
);
Then:
test=> select * from _questions;
ERROR: relation "_questions" does not exist
LINE 1: select * from _questions
^
test=> select * from _QUESTIONS;
ERROR: relation "_questions" does not exist
LINE 1: select * from _QUESTIONS;
^
test=> select * from "_QUESTIONS";
QUESTION_ID | TEXT
-------------+------
(0 rows)
However, if you create the table without the quotes around table and column names:
CREATE TABLE _QUESTIONS
(
QUESTION_ID bigint NOT NULL,
TEXT text,
CONSTRAINT PK PRIMARY KEY (QUESTION_ID)
)
WITH (
OIDS=FALSE
);
Then:
test=> select * from _questions;
question_id | text
-------------+------
(0 rows)
test=> select * from _QUESTIONS;
question_id | text
-------------+------
(0 rows)
test=> select * from "_QUESTIONS";
ERROR: relation "_QUESTIONS" does not exist
LINE 1: select * from "_QUESTIONS";
^
I've managed to figure this out. Turned out, somewhy ORM tries to access association column through its lowercased name, changing the name solved the problem.