query for nextval sequence generation taking long time - postgresql

I have a process in application where we insert 1000+ records in application. There is a attribute in the table as mentioned below.
#Index(name = "_IDX_UNIQUE_UUID", columnNames = {"UUID"})
#Column(name = "UUID", nullable = false)
private Long uuid;
During populating the entity class the value is set as following
entity.setUuid(service.getNextUuid());
which calls this method in service class:
public Long getNextUuid() {
Query query = entityManager.createNativeQuery("select nextval('seq_xxxx_uuid')");
//This takes lot of time to execute
Obj result = query.getSingleResult();
return ((BigInteger) result).longValue();
}
To get next value from sequence, it takes around 200-700 ms. This accumulates to several minutes if I try to insert 1000+ records. Moreover, the query is quite simple and executes in less than 1ms if I execute it in a database client.
I am using postgresql 9.4-1206-jdbc4, hibernate 4.2.0.Final and Spring 4.2.5.

It's slow, because you're going through all the layers of security and consistency checks of a JPA full query, plus you're creating a new NativeQuery object every time and not doing it though a #NamedNativeQuery, thus taking even more time to evaluate.
The solution is to annotate the column with:
#Id
#Index(name = "_IDX_UNIQUE_UUID", columnNames = {"UUID"})
#SequenceGenerator(name = "seq_xxxx_uuid_gen", sequenceName = "seq_xxxx_uuid", allocationSize = 1)
#GeneratedValue(generator = "seq_xxxx_uuid_gen")
#Column(name = "UUID", nullable = false)
private Long id;
You do want that allocationSize to be the same value as the increment_by value of your sequence, or you'll have other problems after the first persist.
If you need to use that Id for multiple objects (say it's part of a composite key, but only part of it is serial-generated), just grab it from the first one after persisting it.
Edit:
Alternatively you can forego the SequenceGenerator and use this:
#GeneratedValue(strategy = GenerationType.IDENTITY)
It may be faster, if the generated query is properly optimized and uses java.sql.Statement.getGeneratedKeys() upon insert.

Related

Is there a limitation to JPA generated id?

I'm inserting data into Postgres database, here is my Entity:
public class FileData {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#Column(name = "id", nullable = false)
private Long id;
}
with intellJ idea auto generated repository interface:
fileDataRepository.save(fileData);
I was able to insert data into my database fine, until the Id number goes to the current value of 152560, then every time I try to insert a line of data, I get the following error:
PSQLException: ERROR: duplicate key value violates unique constraint "file_data_pkey"
Detail: Key (id)=(53) already exists.
I confirmed that the fileData I constructed have null for it's id value, and everytime I call the save(or saveAndFlush), the duplicate key value increases, seems to me that JPA have somehow decided to reset the counter on this table.
So my question is, it's there a limit to the number generated by JPA? And is there a way to configure it?
The only limitation is the database datatype.
But, you shouldn't use #GeneratedValue(strategy = GenerationType.AUTO) because it's not clear how the id will be generated. Hibernate will try to figure that out by itself.
As you are using Postgresql you should go with a Sequence like this:
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "filedata_seq")
#SequenceGenerator(name = "filedata_seq")
Read more about https://vladmihalcea.com/jpa-entity-identifier-sequence/

Hibernate, #SequenceGenerator and Postgresql cache property

I use Spring Boot 2 and Spring data jpa (Hibernate 5.2.1) to work with the database.
I use Postgresql.
In the database I created a sequence
create sequence USERS_SEQ cache 5
and created entity
#Entity
#Table(name = "users")
public class User {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "USERS_SEQ")
#SequenceGenerator(name = "USERS_SEQ", sequenceName = "USERS_SEQ", allocatedSize = 1)
private Long id;
}
I assumed that the numbers from the sequence would be given to me sequentially: 1,2, 3 and so on, even if I restart the application. But after restarting, hibernate assigns values to the records with an offset of 5, that is, the first start:1, the second restart:6, the third restart:11. As far as I understand, this is because of the cache property.
At the same time, if I run select nextval('USERS_SEQ') in db after the second restart, I will get not 7, but 2, that is, the next number after 1
How can I avoid this? I have two instances of the application and I'm afraid that the ids will overlap

How to avoid id collisions in Spring Data JPA/Hibernate-generated database?

We use a dockerized postgres database and have hibernate auto-generate the tables (using spring.jpa.hibernate.ddl-auto: create) for our integration tests. Using something like H2 is not an option because we do some database-specific operations in a few places, e.g. native SQL queries.
Is there any way to avoid id collisions when all entities use auto-incremented ids? Either by offsetting the start id or, better yet, having all tables use a shared sequence?
Schema is created when the docker container is launched, tables are created by Spring Data JPA/Hibernate
Example
Examples use kotlin syntax and assumes the "allopen"-plugin is applied for entities.
Sometimes we've had bugs where the wrong foreign key was used, e.g. something like this:
#Entity
class EntityOne(
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id", nullable = false, columnDefinition = "SERIAL")
var id: Long,
)
#Entity
class EntityTwo(
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id", nullable = false, columnDefinition = "SERIAL")
var id: Long,
)
#Entity
class JoinEntity(
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id", nullable = false, columnDefinition = "SERIAL")
var id: Long,
#ManyToOne
#JoinColumn(name = "entity_one_id")
var entityOne: EntityOne,
#ManyToOne
#JoinColumn(name = "entity_two_id")
var entityTwo: EntityTwo,
)
#Repository
interface JoinEntityRepository : JpaRepository<JoinEntity, Long> {
//
// Bug here! Should be "WHERE entityOne.id = :entityOneId"
//
#Query("SELECT entityTwo FROM JoinEntity WHERE entityTwo.id = :entityOneId")
fun findEntityTwoByEntityOneId(entityOneId: Long): Collection<EntityTwo>
}
These bugs can in some circumstances be very hard to find because when the table is created, there may very well be an Entity2 with the same id as some Entity1, and so the query succeeds but the test fails somewhere down the line because while it is returning one or more Entity2, it's not the expected ones.
Even worse, depending on the scope of the test it may pass even if the wrong entity is fetched, or fail only when tests are run in a specific order (due to ids getting "out of sync"). So ideally it should fail to even find an entity when the wrong id is passed. But because the database structure is created from scratch and the ids are auto-incremented they always start at 1.
I found a solution to this.
In my resources/application.yml (in the test folder, you most likely do not want to do this in your main folder) I add spring.datasource.initialization-mode: always and a file data.sql.
The contents of data.sql are as follows:
DROP SEQUENCE IF EXISTS test_shared_sequence;
CREATE SEQUENCE test_shared_sequence;
ALTER TABLE entity_one ALTER COLUMN id SET DEFAULT nextval('test_shared_sequence');
ALTER TABLE entity_two ALTER COLUMN id SET DEFAULT nextval('test_shared_sequence');
After Spring has auto-generated the tables (using spring.jpa.hibernate.ddl-auto: create) it will run whatever is in this script, and the script will change all tables to auto-generate ids based on the same sequence, meaning that no two entities will ever have the same id regardless of which table they're stored in, and as such any query that looks in the wrong table for an id will fail consistently.

spring data JPA association in Map

I'm trying to use Map in Spring Data JPA to handle the relationship to store records of equipment quantity.
I followed this guide to create the entity.
Meeting{
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "meeting_id", updatable = false)
#JsonIgnore
private int id;
#ElementCollection
#MapKeyColumn(name = "equipment_type")
#MapKeyEnumerated(EnumType.STRING)
private Map<EquipmentType, Integer> equipment = new HashMap<>();
}
EquipmentType is an Enum.
This is the table for the property:
CREATE TABLE IF NOT EXISTS meeting_equipment (
meeting_id INTEGER NOT NULL REFERENCES meeting (meeting_id),
equipment_type VARCHAR(10) NOT NULL,
quantity INTEGER NOT NULL DEFAULT 0
);
Once I try to create a meeting entity, I get error ERROR:column "meeting_meeting_id" of relation "meeting_equipment" does not exist
May I know what's the problem here?
Your table meeting_equipment does not match what JPA is expecting.
It has a column meeting_id but your JPA implementation expects meeting_meeting_id
Either rename the column to the expected meeting_meeting_id or configure your mapping to use the current column name. I think this might do the trick:
#JoinTable(joinColumns={#JoinColumn(name="meeting_id")}
Of course, you probably can create your own naming strategy if you have many cases like this and want to keep your column names as they are.

Spring Data JPA self join query giving unexpected result

I have a mysql database table 'Faculties' like below,
My entity class (FacultyEntity) is as below
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "Faculty_Id")
private Integer facultyId;
#Column(name = "Faculty_Name", length = 100, nullable = false, unique = true)
private String facultyName;
#ManyToOne(cascade = { CascadeType.REFRESH })
#JoinColumn(name = "HOD_ID", nullable = true)
private FacultyEntity hodId;
I want to display the result in my application sorted by HOD's and and then faculties. This is what I've tried
#Query("from FacultyEntity f ORDER BY f.hodId.facultyName, f.facultyName")
But I'm getting total 4 rows sorted like this. I think it is because of the null value in the HOD_ID column, But I'm not sure. I'm newbie here.
The expected result will be
Please help me to fix this issue.
When you do f.hodId in the query, it implicitly translates to inner join, which eliminates the two rows with null in HOD_Id column.
Try this instead
#Query("from FacultyEntity f left join f.hodId hod ORDER BY hod.facultyName, f.facultyName")