JPA Create Parent/child relationship with a lot of children - postgresql

I am trying to store entity Track with children entities TrackPoints with JPA method create. However, to store Track with its children TrackPoints last really long - about 30 seconds. I tried GenerationType.Identity and GenerationType.SEQUENCE. If I also have Hibernate Spatial (Postgis) column, it lasts even longer - about 60 seconds to store parent and all children. JPA sends insert sequentially one followed by another. How can I optimize this? Can anybody tell me what is the main problem?
Technologies:
Wildfly 8.1, JPA 2.1 (hibernate), Hibernate Spatial, EJB, JTA
PostgreSQL 9.3 + PostGis - default setup (just install from Ubuntu package)
Track.java
#Entity
#Table(name = "TRACKS")
public class Track implements Serializable {
#Id
#Column(name = "track_id", nullable = false, unique = true)
#GeneratedValue(strategy = GenerationType.SEQUENCE)
private Long id;
#NotNull
#NotEmpty
#Size(min = 1, max = 100)
#Column(nullable = false, length = 100)
private String name;
#Size(max = 200)
#Column(nullable = false, length = 200)
private String description;
#OneToOne(optional = false)
#JoinColumn(name = "userId", nullable = false)
private User userOwner;
#NotNull
#NotEmpty
#Column(nullable = false, length = 55)
private String type;
#NotNull
private Boolean isShared;
#OneToMany(mappedBy = "track")
private List<TrackPoint> trackPoints;
}
TrackPoint.java
#Entity
#Table(name = "TRACK_POINTS")
public class TrackPoint implements Serializable {
private static final long serialVersionUID = 8089601593251025235L;
#Id
#Column(name = "trackpoint_id", nullable = false, unique = true)
#GeneratedValue(generator = "track_point_sequence", strategy = GenerationType.SEQUENCE)
#SequenceGenerator(name = "track_point_sequence", sequenceName = "track_point_sequence", allocationSize = 1000)
private Long id;
#NotNull
private int trackSegment;
#NotNull
private double elevation;
#NotNull
#Temporal(TemporalType.TIMESTAMP)
private Date timeStamp;
#NotNull
#ManyToOne(optional = false, fetch = FetchType.EAGER)
#JoinColumn(name = "track_id")
private Track track;
/*Hibernate Spatial - Postgis field.
#NotNull
#Column(nullable = false)
#Type(type = "org.hibernate.spatial.GeometryType")
private Geometry location;*/
}
TrackService.java
#Stateless
#TransactionAttribute(TransactionAttributeType.REQUIRED)
public class TracksService implements ITracksService {
#Inject
private IDaoService dao;
#Override
public Long createTrack(GpxType gpx, String userId, String name, String desc) {
// Map GPX to Track, TrackPoint object.
track = dao.create(track);
int batch_size = 50;
int i = 0;
for(TrackPoint point: track.getTrackPoints()) {
dao.create(point);
if(i++ % batch_size == 0) {
dao.flush();
dao.clear();
}
}
return track.getId();
}
DaoService.java
#Stateless
#TransactionAttribute(TransactionAttributeType.REQUIRED)
public class DaoService implements IDaoService {
#PersistenceContext()
private EntityManager em;
#Override
public <T extends Serializable> T create(T t) {
em.persist(t);
return t;
}
}
persistence.xml
<persistence xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
version="2.1" xmlns="http://xmlns.jcp.org/xml/ns/persistence"
xsi:schemaLocation=
"http://xmlns.jcp.org/xml/ns/persistence
http://xmlns.jcp.org/xml/ns/persistence/persistence_2_1.xsd">
<persistence-unit name="postgisTourbookPU" transaction-type="JTA">
<description>PostgresSQL database with PostGIS extension</description>
<provider>org.hibernate.jpa.HibernatePersistenceProvider</provider>
<jta-data-source>${tourbook.datasource.postgresql.jndi-name}</jta-data-source>
<exclude-unlisted-classes>false</exclude-unlisted-classes>
<shared-cache-mode>NONE</shared-cache-mode>
<properties>
<!-- JPA properties -->
<property name="javax.persistence.schema-generation.database.action"
value="drop-and-create"/>
<!-- <property name="javax.persistence.schema-generation-target"
value="database"/>-->
<!-- Creation Schema Properties -->
<property name="javax.persistence.schema-generation.create-source"
value="metadata"/>
<!-- <!– DDL Script location, when script is used –>
<property name="javax.persistence.schema-generation.create-script-source"
value="META-INF/create-script.sql"/>-->
<!-- Drop Schema Properties -->
<property name="javax.persistence.schema-generation.drop-source"
value="metadata"/>
<!-- <property name="javax.persistence.schema-generation.drop-script-source"
value="META-INF/drop-script.sql"/>-->
<property name="javax.persistence.sql-load-script-source"
value="META-INF/load-script.sql"/>
<!-- JPA driver information -->
<property name="javax.persistence.jdbc.driver"
value="org.postgresql.Driver"/>
<!-- Hibernate properties -->
<property name="hibernate.connection.characterEncoding"
value="UTF-8"/>
<property name="hibernate.dialect"
value="org.hibernate.spatial.dialect.postgis.PostgisDialect"/>
<property name="hibernate.default_schema"
value="public"/>
<property name="hibernate.show_sql"
value="true"/>
<property name="hibernate.jdbc.batch_size" value="50"/>
<property name="hibernate.jdbc.fetch_size"
value="50"/>
<property name="hibernate.order_inserts"
value="true"/>
<property name="hibernate.order_updates"
value="true"/>
<property name="hibernate.cache.use_query_cache"
value="false"/>
<!-- Hibernate caching -->
</properties>
</persistence-unit>
</persistence>
Edited
So I have tried, batch insert in Hibernate, but I still get 30 seconds for saving 2000 points.

you're inserting a parent with all the children. In that case the Hibernate JPA indeed can be slow, but there are a few tips to improve the performance
- check the hibernate batch guide http://docs.jboss.org/hibernate/core/4.0/devguide/en-US/html/ch04.html
- I've used the hibernate.jdbc.batch_size parameter (set to e.g. 50)
Good luck
Gabriel

Related

Eclipselink with jpa errorfor persistenceunit load

hi I'm trying to use JPA with eclipse link but it gives me following error:
I'm using maven, and context.xml file is in web module and persistaence.xml is in access/db module.
Failed to connect to MyModalTestApp: Exception [EclipseLink-30005] (Eclipse
Persistence Services - 2.6.4.v20160829-44060b6):
org.eclipse.persistence.exceptions.PersistenceUnitLoadingException
Exception Description: An exception was thrown while searching for
persistence archives with ClassLoader: ParallelWebappClassLoader
context: modaltestapp-api
delegate: false
----------> Parent Classloader:
java.net.URLClassLoader#3fee733d
Internal Exception: javax.persistence.PersistenceException: Exception
[EclipseLink-28018] (Eclipse Persistence Services - 2.6.4.v20160829-
44060b6): org.eclipse.persistence.exceptions.EntityManagerSetupException
Exception Description: Predeployment of PersistenceUnit [MyModalTestApp]
failed.
Internal Exception: Exception [EclipseLink-7161] (Eclipse Persistence
Services - 2.6.4.v20160829-44060b6):
org.eclipse.persistence.exceptions.ValidationException
Exception Description: Entity class [class
com.modaltestapp.entity.RegistrationEntity] has no primary key specified.
It should define either an #Id, #EmbeddedId or an #IdClass. If you have
defined PK using any of these annotations then make sure that you do not
have mixed access-type (both fields and properties annotated) in your
entity class hierarchy.
persistence.xml
<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.1" xmlns="http://xmlns.jcp.org/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/persistence http://xmlns.jcp.org/xml/ns/persistence/persistence_2_1.xsd">
<persistence-unit name="MyModalTestApp" transaction-type="RESOURCE_LOCAL">
<non-jta-data-source>java:comp/env/jdbc/MODALTESTAPP_DATASOURCE</non-jta-data-source>
<class>com.modaltestapp.entity.RegistrationEntity</class>
<exclude-unlisted-classes>true</exclude-unlisted-classes>
<properties>
<property name="eclipselink.ddl-generation.output-mode" value="database" />
<property name="eclipselink.ddl-generation" value="none"/>
<property name="eclipselink.weaving" value="static" />
<property name="eclipselink.target-database" value="Auto"/>
<property name="eclipselink.target-server" value="None"/>
<property name="eclipselink.jdbc.batch-writing" value="JDBC"/>
<property name="eclipselink.logging.level" value="FINEST"/>
<property name="eclipselink.logging.timestamp" value="true"/>
<property name="eclipselink.logging.session" value="false"/>
<property name="eclipselink.logging.thread" value="true"/>
<property name="eclipselink.logging.exceptions" value="true"/>
<!-- 0 is a valid ID
<property name="eclipselink.id-validation" value="NULL"/> -->
<!-- We need the following, else EclipseLink cannot properly translate column names in Native SQL queries. -->
<property name="eclipselink.jdbc.uppercase-columns" value="true"/>
</properties>
</persistence-unit>
</persistence>
pojo is:
package com.modaltestapp.entity;
import java.io.Serializable;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.Table;
#Entity
#Table(name = "REGISTRATION")
public class RegistrationEntity implements Serializable{
private static final long serialVersionUID = 1L;
#Column(name = "FIRSTNAME")
private String firstName;
#Column(name = "LASTNAME")
private String lastName;
#Column(name = "LASTNAME")
private String email;
#Column(name = "MESSAGE")
private String message;
#Column(name = "EDUCATION")
private String education;
#Column(name = "HOBBIES")
private String hobbies;
#Column(name = "GENDER")
private String gender;
public String getFirstName() {
return firstName;
}
public void setFirstName(String firstName) {
this.firstName = firstName;
}
public String getLastName() {
return lastName;
}
public void setLastName(String lastName) {
this.lastName = lastName;
}
public String getEmail() {
return email;
}
public void setEmail(String email) {
this.email = email;
}
public String getMessage() {
return message;
}
public void setMessage(String message) {
this.message = message;
}
public String getEducation() {
return education;
}
public void setEducation(String education) {
this.education = education;
}
public String getHobbies() {
return hobbies;
}
public void setHobbies(String hobbies) {
this.hobbies = hobbies;
}
public String getGender() {
return gender;
}
public void setGender(String gender) {
this.gender = gender;
}
}
context.xml is:
<Context>
<!-- Default set of monitored resources -->
<WatchedResource>WEB-INF/web.xml</WatchedResource>
<!-- Uncomment this to disable session persistence across Tomcat restarts -->
<!--
<Manager pathname="" />
-->
<!-- Uncomment this to enable Comet connection tacking (provides events
on session expiration as well as webapp lifecycle) -->
<!--
<Valve className="org.apache.catalina.valves.CometConnectionManagerValve" />
-->
<Resource name="jdbc/MODALTESTAPP_DATASOURCE" auth="Container" type="javax.sql.DataSource"
maxTotal="8" maxIdle="2"
username="root" password="unnati#123" driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/MODALTESTAPP" validationQuery="select 1"/>
</Context>
and my implementation code is :
try {
EntityManager em = Persistence.createEntityManagerFactory("MyModalTestApp").createEntityManager();
System.out.println("in database portion1");
/* Persist entity */
em.getTransaction().begin();
System.out.println("in database portion2");
em.persist(REG);
System.out.println("in database portion3");
em.getTransaction().commit();
System.out.println("yes transcaction done");
} catch (Exception e) {
System.out.println("Failed to connect to MyModalTestApp: " + e);
// e.printStackTrace();
}

How to persist several rows in a #OneToMany relationship with a composite key

I have found this OLD link which resolve my database schema:
http://randomthoughtsonjavaprogramming.blogspot.com.es/2014/09/jpa-manytoone-with-composite-primary-key.html
I can retrieve data and everything works great, but I don't know how to insert data as I posted to the author.
In my case for every guild inserted I need to insert several guildranks at the same time, so one use case will be:
BEGIN TRANSACTION
GUILD
Guildname1
GUILDRANK
Guildname1, 1
Guildname1, 2
Guildname1, 3
Guildname1, 4
END TRANSACTION
Any suggestions?
Thanks
UPDATE
#Entity
#NamedQueries({
#NamedQuery(name = "Cuestionario.findAll", query = "SELECT c FROM Cuestionario c")
})
public class Cuestionario implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#Column(name = "ID", nullable = false)
#GeneratedValue(strategy = GenerationType.IDENTITY)
private long id;
#OneToMany(mappedBy = "cuestionario", cascade = { CascadeType.PERSIST })
private Set<Hoja> hojas;
}
#Entity
#NamedQuery(name = "Hoja.findAll", query = "SELECT h FROM Hoja h")
public class Hoja implements Serializable {
private static final long serialVersionUID = 1L;
#EmbeddedId
private HojaPK id;
// bi-directional many-to-one association to Cuestionario
#ManyToOne(cascade = { CascadeType.PERSIST, CascadeType.REMOVE })
#JoinColumn(name = "ID", insertable = false, updatable = false)
private Cuestionario cuestionario;
public Hoja() {
}
}
#Embeddable
public class HojaPK implements Serializable {
// default serial version id, required for serializable classes.
private static final long serialVersionUID = 1L;
//Correspond with "id" in Cuestionario
#Column(name = "ID")
private long id;
private int dia;
public HojaPK() {
}
}
persistence.xml
<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.1"
xmlns="http://xmlns.jcp.org/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/persistence http://xmlns.jcp.org/xml/ns/persistence/persistence_2_1.xsd">
<persistence-unit name="pu"
transaction-type="RESOURCE_LOCAL">
<properties>
<property name="javax.persistence.jdbc.user" value="***" />
<property name="javax.persistence.jdbc.password" value="***" />
<property name="javax.persistence.jdbc.url"
value="jdbc:sqlserver://localhost:1433;databaseName=mydatabase" />
<property name="javax.persistence.jdbc.driver"
value="com.microsoft.sqlserver.jdbc.SQLServerDriver" />
<property name="hibernate.dialect" value="org.hibernate.dialect.SQLServerDialect" />
<property name="hibernate.show_sql" value="true" />
<!--
<property name="hibernate.hbm2ddl.auto" value="update" />
-->
<property name="hibernate.ejb.naming_strategy" value="org.hibernate.cfg.ImprovedNamingStrategy" />
<property name="hibernate.connection.charSet" value="UTF-8" />
</properties>
</persistence-unit>
</persistence>
TestCase.java
EntityManagerFactory emf = Persistence.createEntityManagerFactory("pu");
EntityManager em = emf.createEntityManager();
em.getTransaction().begin();
em.persist(cuestionario);
em.flush();
for (int i = 0; i <= 32; i++) {
HojaPK hojaPk = new HojaPK(cuestionario.getId(), i);
Hoja hoja = new Hoja();
hoja.setId(hojaPk);
em.persist(hoja);
}
em.getTransaction().commit();
em.close();
emf.close();

#DocumentId on a Non-#Id Field Seems to Make Indexing Invalid for Existing Data in Database

I have a class named Book that contains 2 fields, id and isbn.
And I want to perform normal Hibernate queries using the id field as the primary key, while perform Hibernate Search queries using the isbn field as the primary key.
So I put the #Id annotation on the id field and the #DocumentId annotation on the isbn field, as follows.
#Entity
#Indexed
public class Book {
#Id
private Long id;
#DocumentId(name = "_documentId")
#Field(index = Index.YES, analyze = Analyze.YES, store = Store.YES)
private Long isbn;
...
}
Then I found that, after the Hibernate Search indexing process starts, although all new updates to the Book entities are correctly indexed, the existing Book entities in the database are not indexed.
And after I move the #DocumentId annotation from the isbn field onto the the id field as follows, the existing Book entities in the database will be indexed again when the Hibernate Search indexing process starts.
#Entity
#Indexed
public class Book {
#Id
#DocumentId(name = "_documentId")
private Long id;
#Field(index = Index.YES, analyze = Analyze.YES, store = Store.YES)
private Long isbn;
...
}
So my question is
Can the Hibernate Search indexing process work same as before, even when the #DocumentId and #Id annotations are on different fields?.
I use the Luke to check the Hibernate Search index.
Libraries
hibernate-search-orm: 5.5.4.Final
hibernate-core: 5.0.3.Final
jdk: 1.7.0_79
Code
Entity Class
Book.java
#Entity
#Indexed
public class Book {
#Id
private Long id;
#DocumentId(name = "_documentId")
#Field(index = Index.YES, analyze = Analyze.YES, store = Store.YES)
private Long isbn;
#Field(index = Index.YES, analyze = Analyze.YES, store = Store.YES)
private int version;
#Field(index = Index.YES, analyze = Analyze.YES, store = Store.YES)
private String content;
public Book() {
}
public Book(Long isbn, int version, String content) {
this.isbn = isbn;
this.version = version;
this.content = content;
}
#Override
public String toString() {
return "Book [id=" + id + ", isbn=" + isbn + ", version=" + version + ", content=" + content + "]";
}
// Getters and setters...
}
Hibernate Mapping File
Book.hbm.xml
<hibernate-mapping package="com.raychen518.study.hibernate">
<class name="Book" table="BOOKS">
<id name="id" column="ID">
<generator class="increment" />
</id>
<property name="isbn" column="ISBN" />
<property name="version" column="VERSION" />
<property name="content" column="CONTENT" />
</class>
</hibernate-mapping>
Hibernate Configuration File
hibernate.cfg.xml
<hibernate-configuration>
<session-factory>
<property name="connection.driver_class">org.postgresql.Driver</property>
<property name="connection.url">jdbc:postgresql://localhost:5432/test</property>
<property name="connection.username">postgres</property>
<property name="connection.password">admin</property>
<property name="connection.pool_size">1</property>
<property name="dialect">org.hibernate.dialect.PostgreSQLDialect</property>
<property name="current_session_context_class">thread</property>
<property name="cache.provider_class">org.hibernate.cache.internal.NoCacheProvider</property>
<property name="show_sql">true</property>
<property name="hbm2ddl.auto">validate</property>
<property name="hibernate.search.lucene_version">LUCENE_CURRENT</property>
<property name="hibernate.search.default.directory_provider">filesystem</property>
<property name="hibernate.search.default.indexBase">hibernate.search.test/lucene/indexes</property>
<mapping resource="Book.hbm.xml" />
</session-factory>
</hibernate-configuration>
Application Launcher Class
BookManager.java
After executing the main(String[]) method in the class, the Book entity which the isbn field is 789 will be created in the index while the existing Book entities (in database) which the isbn fields are 123 and 456 respectively are not indexed.
public class BookManager {
public static void main(String[] args) throws InterruptedException {
BookManager manager = new BookManager();
manager.startIndexing();
manager.saveSome();
}
private void startIndexing() throws InterruptedException {
FullTextSession fullTextSession = Search.getFullTextSession(HibernateUtil.getSessionFactory().openSession());
fullTextSession.createIndexer(Book.class).startAndWait();
}
public void saveSome() {
Session session = HibernateUtil.getSessionFactory().openSession();
session.beginTransaction();
// session.save(new Book(123L, 1, "abc"));
// session.save(new Book(456L, 1, "def"));
session.save(new Book(789L, 1, "ghi"));
session.getTransaction().commit();
session.close();
}
}
HibernateUtil.java
public class HibernateUtil {
private static final SessionFactory sessionFactory = buildSessionFactory();
private static SessionFactory buildSessionFactory() {
try {
StandardServiceRegistry registry = new StandardServiceRegistryBuilder().configure().build();
return new MetadataSources(registry).buildMetadata().buildSessionFactory();
} catch (Throwable ex) {
System.err.println("Initial SessionFactory creation failed." + ex);
throw new ExceptionInInitializerError(ex);
}
}
public static SessionFactory getSessionFactory() {
return sessionFactory;
}
}

No property found for type error in Sprint JPA2

I have error in Spring JPA when I call findAll method:
org.springframework.data.mapping.PropertyReferenceException: No property package found for type models.Apk
#Entity
#Table(name="Apks")
public class Apk implements Serializable {
#EmbeddedId private ApkPk id;
#Column private String fileName;
#Column private String description;
#Column(updatable=false)
#Temporal(TemporalType.TIMESTAMP)
private Date creationTime;
#Column
#Temporal(TemporalType.TIMESTAMP)
private Date lastUpdateTime;
#ManyToOne
#JoinColumn(name="appId", referencedColumnName="appId", insertable=false, updatable=false)
#NotFound(action=NotFoundAction.IGNORE)
#JsonIgnore
private App app;
//Getters and setters
}
package mypackage.repositories;
public interface ApkRepository extends PagingAndSortingRepository<Apk, ApkPk> {
public List<Apk> findByFileName(String value);
}
applicationContext.xml
<bean id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="persistenceUnitName" value="myjpa" />
</bean>
I have many repository classes but only this one occurs error. I found that Spring Data checks the naming convention according to the entity class, so my repository name should be ApkRepository. But it is not the case.
What can I fix this problem?
Any help would be appreciated.
EDIT:
When I do unit test, it works fine. I happens error when it calls through service class.
repository.findAll(new PageRequest(1, 30));

Persistence Provider for EntityManager not found

I'm trying to persist elements using JPA and EclipseLink. So I've created a class to persist
#Entity
public class Couple implements Serializable{
private static final long serialVersionUID = 1L;
#Column(name = "OBJECTID")
private String objectID;
#Column(name = "EPCNUMBER")
private String epcNumber;
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
And so on.
I've created a class to "use" that :
public class Main {
private static final String PERSISTENCE_UNIT_NAME = "MyPU";
private static EntityManagerFactory factory;
public static void main(String[] args) {
factory = Persistence.createEntityManagerFactory(PERSISTENCE_UNIT_NAME);
EntityManager em = factory.createEntityManager();
TypedQuery<Couple> q = em.createQuery("SELECT c FROM Couple c", Couple.class);
List<Couple> couples = (List<Couple>) q.getResultList();
And then, i've the following persistence.xml :
<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.0" xmlns="http://java.sun.com/xml/ns/persistence"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
<persistence-unit name="MyPU">
<provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>
<jta-data-source>localJTA</jta-data-source>
<class>fr.mypackage.com.Couple</class>
<properties>
<property name="eclipselink.ddl-generation" value="create-tables"/>
<property name="eclipselink.ddl-generation.output-mode" value="database" />
<property name="eclipselink.logging.level" value="INFO"/>
</properties>
</persistence-unit>
</persistence>
But, even when I change the properties, I've got the same error :
Exception in thread "main" javax.persistence.PersistenceException: No Persistence provider for EntityManager named MyPU
(when calling factory = Persistence.createEntityManagerFactory(PERSISTENCE_UNIT_NAME);).
Did I do something wrong to ling xml and persistence unit ? I've weel added to classpath the following :
javax.persistence.jar
javax.ejb.jar
eclipselink.jar
javax.persistence_1.0.0.jar
javax.persistence_2.0.4.v201112161009.jar
derby.jar
Could you help me ? Thanks !
On my application, I was having this problem until I compiled my Netbeans project. Then it worked.