How to avoid id collisions in Spring Data JPA/Hibernate-generated database? - postgresql

We use a dockerized postgres database and have hibernate auto-generate the tables (using spring.jpa.hibernate.ddl-auto: create) for our integration tests. Using something like H2 is not an option because we do some database-specific operations in a few places, e.g. native SQL queries.
Is there any way to avoid id collisions when all entities use auto-incremented ids? Either by offsetting the start id or, better yet, having all tables use a shared sequence?
Schema is created when the docker container is launched, tables are created by Spring Data JPA/Hibernate
Example
Examples use kotlin syntax and assumes the "allopen"-plugin is applied for entities.
Sometimes we've had bugs where the wrong foreign key was used, e.g. something like this:
#Entity
class EntityOne(
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id", nullable = false, columnDefinition = "SERIAL")
var id: Long,
)
#Entity
class EntityTwo(
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id", nullable = false, columnDefinition = "SERIAL")
var id: Long,
)
#Entity
class JoinEntity(
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id", nullable = false, columnDefinition = "SERIAL")
var id: Long,
#ManyToOne
#JoinColumn(name = "entity_one_id")
var entityOne: EntityOne,
#ManyToOne
#JoinColumn(name = "entity_two_id")
var entityTwo: EntityTwo,
)
#Repository
interface JoinEntityRepository : JpaRepository<JoinEntity, Long> {
//
// Bug here! Should be "WHERE entityOne.id = :entityOneId"
//
#Query("SELECT entityTwo FROM JoinEntity WHERE entityTwo.id = :entityOneId")
fun findEntityTwoByEntityOneId(entityOneId: Long): Collection<EntityTwo>
}
These bugs can in some circumstances be very hard to find because when the table is created, there may very well be an Entity2 with the same id as some Entity1, and so the query succeeds but the test fails somewhere down the line because while it is returning one or more Entity2, it's not the expected ones.
Even worse, depending on the scope of the test it may pass even if the wrong entity is fetched, or fail only when tests are run in a specific order (due to ids getting "out of sync"). So ideally it should fail to even find an entity when the wrong id is passed. But because the database structure is created from scratch and the ids are auto-incremented they always start at 1.

I found a solution to this.
In my resources/application.yml (in the test folder, you most likely do not want to do this in your main folder) I add spring.datasource.initialization-mode: always and a file data.sql.
The contents of data.sql are as follows:
DROP SEQUENCE IF EXISTS test_shared_sequence;
CREATE SEQUENCE test_shared_sequence;
ALTER TABLE entity_one ALTER COLUMN id SET DEFAULT nextval('test_shared_sequence');
ALTER TABLE entity_two ALTER COLUMN id SET DEFAULT nextval('test_shared_sequence');
After Spring has auto-generated the tables (using spring.jpa.hibernate.ddl-auto: create) it will run whatever is in this script, and the script will change all tables to auto-generate ids based on the same sequence, meaning that no two entities will ever have the same id regardless of which table they're stored in, and as such any query that looks in the wrong table for an id will fail consistently.

Related

Is there a limitation to JPA generated id?

I'm inserting data into Postgres database, here is my Entity:
public class FileData {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#Column(name = "id", nullable = false)
private Long id;
}
with intellJ idea auto generated repository interface:
fileDataRepository.save(fileData);
I was able to insert data into my database fine, until the Id number goes to the current value of 152560, then every time I try to insert a line of data, I get the following error:
PSQLException: ERROR: duplicate key value violates unique constraint "file_data_pkey"
Detail: Key (id)=(53) already exists.
I confirmed that the fileData I constructed have null for it's id value, and everytime I call the save(or saveAndFlush), the duplicate key value increases, seems to me that JPA have somehow decided to reset the counter on this table.
So my question is, it's there a limit to the number generated by JPA? And is there a way to configure it?
The only limitation is the database datatype.
But, you shouldn't use #GeneratedValue(strategy = GenerationType.AUTO) because it's not clear how the id will be generated. Hibernate will try to figure that out by itself.
As you are using Postgresql you should go with a Sequence like this:
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "filedata_seq")
#SequenceGenerator(name = "filedata_seq")
Read more about https://vladmihalcea.com/jpa-entity-identifier-sequence/

Hibernate postgres auto increment after manual insert

I have a basic spring application, with a simple entity. I have a flyway script, to create the postgres table, and add some starting data.
create table user (
id serial primary key,
username varchar (50) unique not null,
password varchar (150) not null
);
insert into user (id, username, password) values (1, 'name', 'somehashed');
insert into etc...
I've set up my entity as follows:
#Entity
#Table(name = "user")
public class User {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id", updatable = false, columnDefinition = "serial")
private Long id;
...
other fields, constructor, getters setters etc...
My problem is, that on start-up, the basic entities are persisted by flyway, but upon trying to save a new entity, hibernate tries to give it the ID 1, although it is already given to another one.
I tried it also with SEQUENCE strategy, the problem didn't get solved.
Ok, problem was that I specified explicitly the ID I wanted to give while the insert script, and I didn't let postgres do the magic...

spring data JPA association in Map

I'm trying to use Map in Spring Data JPA to handle the relationship to store records of equipment quantity.
I followed this guide to create the entity.
Meeting{
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "meeting_id", updatable = false)
#JsonIgnore
private int id;
#ElementCollection
#MapKeyColumn(name = "equipment_type")
#MapKeyEnumerated(EnumType.STRING)
private Map<EquipmentType, Integer> equipment = new HashMap<>();
}
EquipmentType is an Enum.
This is the table for the property:
CREATE TABLE IF NOT EXISTS meeting_equipment (
meeting_id INTEGER NOT NULL REFERENCES meeting (meeting_id),
equipment_type VARCHAR(10) NOT NULL,
quantity INTEGER NOT NULL DEFAULT 0
);
Once I try to create a meeting entity, I get error ERROR:column "meeting_meeting_id" of relation "meeting_equipment" does not exist
May I know what's the problem here?
Your table meeting_equipment does not match what JPA is expecting.
It has a column meeting_id but your JPA implementation expects meeting_meeting_id
Either rename the column to the expected meeting_meeting_id or configure your mapping to use the current column name. I think this might do the trick:
#JoinTable(joinColumns={#JoinColumn(name="meeting_id")}
Of course, you probably can create your own naming strategy if you have many cases like this and want to keep your column names as they are.

query for nextval sequence generation taking long time

I have a process in application where we insert 1000+ records in application. There is a attribute in the table as mentioned below.
#Index(name = "_IDX_UNIQUE_UUID", columnNames = {"UUID"})
#Column(name = "UUID", nullable = false)
private Long uuid;
During populating the entity class the value is set as following
entity.setUuid(service.getNextUuid());
which calls this method in service class:
public Long getNextUuid() {
Query query = entityManager.createNativeQuery("select nextval('seq_xxxx_uuid')");
//This takes lot of time to execute
Obj result = query.getSingleResult();
return ((BigInteger) result).longValue();
}
To get next value from sequence, it takes around 200-700 ms. This accumulates to several minutes if I try to insert 1000+ records. Moreover, the query is quite simple and executes in less than 1ms if I execute it in a database client.
I am using postgresql 9.4-1206-jdbc4, hibernate 4.2.0.Final and Spring 4.2.5.
It's slow, because you're going through all the layers of security and consistency checks of a JPA full query, plus you're creating a new NativeQuery object every time and not doing it though a #NamedNativeQuery, thus taking even more time to evaluate.
The solution is to annotate the column with:
#Id
#Index(name = "_IDX_UNIQUE_UUID", columnNames = {"UUID"})
#SequenceGenerator(name = "seq_xxxx_uuid_gen", sequenceName = "seq_xxxx_uuid", allocationSize = 1)
#GeneratedValue(generator = "seq_xxxx_uuid_gen")
#Column(name = "UUID", nullable = false)
private Long id;
You do want that allocationSize to be the same value as the increment_by value of your sequence, or you'll have other problems after the first persist.
If you need to use that Id for multiple objects (say it's part of a composite key, but only part of it is serial-generated), just grab it from the first one after persisting it.
Edit:
Alternatively you can forego the SequenceGenerator and use this:
#GeneratedValue(strategy = GenerationType.IDENTITY)
It may be faster, if the generated query is properly optimized and uses java.sql.Statement.getGeneratedKeys() upon insert.

Storing ManyToMany with extra column using merge method

I have an entity class that is simply a ManyToMany with extra column, as follows:
#Entity
#Table(name = "view_templates_device_types")
public class ViewTemplateDeviceTypeEntity implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#ManyToOne
#JoinColumn(name = "view_template_id")
private ViewTemplateEntity viewTemplate;
#Id
#ManyToOne
#JoinColumn(name = "device_type_id")
private DeviceTypeEntity deviceType;
#Column(name = "priority", nullable = false)
private int priority;
public ViewTemplateDeviceTypeEntity() {
}
...
}
I noticed that when I create a new object of this type, set viewTemplate and deviceType to values that have corresponding data in the db and I call entityManager.persist(entity) the data is stored. But when I call entityManager.merge(entity) instead of persist I get an exception:
SQL Error: 1048, SQLState: 23000
Column 'view_template_id' cannot be null
I thought that calling merge should result with data inserted into database in case it is not stored yet. It is quite important to me to use merge here (because of cascades). What can I do to make it work?
As per the JPA spec, section 2.4
"A composite primary key must correspond to either a single persistent field or property or to a set of such fields or properties as described below. A primary key class must be defined to represent a composite primary key. Composite primary keys typically arise when mapping from legacy databases when the database key is comprised of several columns. The EmbeddedId or IdClass annotation is used to denote a composite primary key. See Sections 11.1.17 and 11.1.22.".
So you either need #IdClass or #EmbeddedId. Anything else is non-portable and prone to error. I am very surprised of any JPA provider that does not throw out warnings for this.