Spring JPA Entities: related problems of Ignite error, LAZY fetching, and too-m, minimizing database use and working with Ignite - spring-data-jpa

I'm writing some entity relationships using Spring Data and Java. I have this pair of classes (edited):
Subject:
#Entity
#Table(name = "SUBJECT")
// Lombok, etc., attributes removed
public class Subject {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE)
#Column(name = "ID", updatable = false, nullable = false)
#JsonProperty("id")
private Long id;
#OneToMany(targetEntity = SubjectResource.class, mappedBy = "subject", fetch = FetchType.LAZY, cascade = CascadeType.ALL)
private List<SubjectResource> resources;
}
SubjectResource:
#Entity
#Table(name = "SUBJECT_RESOURCE")
// Lombok, etc., attributes removed
public class SubjectResource {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE)
#Column(name = "ID", updatable = false, nullable = false)
private Long id;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "SUBJECT_ID")
private Subject subject;
}
I'm trying to solve these issues:
Question 1: Can I manipulate #OneToMany or #ManyToOne to NOT have the child class recurse its parent?
Fetch of resources returns subject data:
/subject/101:
{"id":101,"resources":[{"id":1001,"subject":101},{"id":1002,"subject":101},{"id":1003,"subject":101},{"id":1004,"subject":101}]}
/subjectResource/1001:
{id:1001,subject:{"id":101,"resources":[{"id":1001,"subject":101},{"id":1002,"subject":101},{"id":1003,"subject":101},{"id":1004,"subject":101}]}}
That is, /subjectResource/1001 returns its ID and the entire /subject/101 query.
How can I have just the subjectResource data, without its parent?
Question 2: Through #OneToMany or #ManyToOne can I get Hibernate to fetch on a "1" (O(1)) basis?
When /subjects does its thing, it works with Hibernate on a "n+1" (O(n)) basis: 1 fetch of subjects, n fetches of resources, one for each subject ID.
I could force a single fetch through a fancy repository #Query annotation ("select s from subject s left join fetch s.resources"). But that means putting the subject : subject_resource definitions in two places, etc.
Can JPA implementation / Hibernate be forced to do a join, and thus make only one database call, through annotation within an entity class?
Question 3: How do I get my Spring Data / Spring Repository to cooperate with Ignite, and have the cache return the data it already had on the first call?
I'm usng FetchType.LAZY, as all good pupils do. I'm also storing things in Apache Ignite. For /subject/101 the initial call fetches everything OK, returning it in JSON. But the second call gets from the Ignite cache, which complains about being out of transaction.
How do get my LAZY fetches to cooperate with Ignite?
Thanks,
Jerome.

Related

JPA-Eclipselink caching nested tables (entities)

I am using Java JPA-Eclipselink persistence (v2.6.9) in combination with Apache Tomcat Web Server. I am fetching from database large amount of data from nested tables on each web service invocation. Data in the database is constantly being expanded with new data.
We predict that number of fetches are going to increase, so we ran stress test and find out that expected traffic is going to cause database clogging. In order to avoid clogging, I would like to implement temporary caching in a way that eclipse link should fetch data and hold it for a minute in a cache and fetch new data after a minute. If I understood right I am looking for L2 caching in Eclipselink.
I have tried several options, but eclipse link is always fetching data from database on each call of web service method.
Could you please help me make eclipselink use caching in intervals of 60 seconds?
Please find below my attempts.
Simplified enities
#Entity
#Cache(
type=CacheType.SOFT, // Cache everything until the JVM decides memory is low.
size=64000, // Use 64,000 as the initial cache size.
expiry=36000000 // 10 minutes
)
#Table(name="SITUATION_DATA")
#NamedQuery(name="SituationData.findAll", query="SELECT s FROM SituationData s")
public class DatexSituationData implements Serializable {
private static final long serialVersionUID = 1L;
//bi-directional many-to-one association to SituationRecord
#OneToMany(mappedBy="datexSituationData", cascade = CascadeType.PERSIST, fetch =
FetchType.EAGER)
#JoinFetch(value=JoinFetchType.OUTER)
private List<SituationRecord> situationRecords;
}
#Entity
#Table(name="SituationRecord")
#NamedQuery(name="SituationRecord.findAll", query="SELECT s FROM SituationRecord s")
public class SituationRecord implements Serializable {
private static final long serialVersionUID = 1L;
#OneToMany(mappedBy="situationRecord", cascade = CascadeType.PERSIST, fetch = FetchType.EAGER)
#JoinFetch(value=JoinFetchType.OUTER)
private List<SituationRecordComment> situationRecordComment;
#OneToMany(mappedBy="situationRecord", cascade = CascadeType.PERSIST, fetch = FetchType.EAGER)
#JoinFetch(value=JoinFetchType.OUTER)
private List<SituationRecordTypeElement> situationRecordTypeElements;
//bi-directional many-to-one association to SituationLocation
#ManyToOne(cascade = CascadeType.PERSIST, fetch = FetchType.EAGER)
#JoinFetch(value=JoinFetchType.OUTER)
#JoinColumn(name="ID_LOKACIJE")
private SituationLocation situationLocation;
//bi-directional many-to-one association to DatexSituationData
#ManyToOne()
#JoinColumns({
#JoinColumn(name="SITUATION_ID", referencedColumnName="ID", nullable=false),
#JoinColumn(name="SITUATION_VERSION", referencedColumnName="VERSION", nullable=false)
})
private DatexSituationData datexSituationData;
}
#Entity
#Table(name="SITUATION_LOCATIONS")
#NamedQuery(name="SituationLocation.findAll", query="SELECT s FROM SituationLocation s")
public class SituationLocation implements Serializable {
private static final long serialVersionUID = 1L;
#Id
GeneratedValue(strategy=GenerationType.SEQUENCE, generator="situation_location_seq")
#SequenceGenerator(name="situation_location_seq", sequenceName="SEQ_SITUATION_LOCATION",
allocationSize=1)
#Column(name="ID_LOKACIJE", unique=true, nullable=false)
private long idLokacije;
//bi-directional many-to-one association to SituationRecord
#OneToMany(mappedBy="situationLocation", cascade = CascadeType.PERSIST)
private List<SituationRecord> situationRecords;
}
Fetching from database
String sQuery =
"SELECT ds FROM SituationData ds where CONCAT(ds.id.version, ds.id.id) in (select
CONCAT(max(FUNC('TO_NUMBER', ds1.id.version)), ds1.id.id) from SituationData ds1 group by
ds1.id.id)";
EntityManager em = Emf.getInstance().getFactory().createEntityManager();
Query q = em.createQuery(sQuery, DatexSituationData.class);
q.setHint("eclipselink.join-fetch", "ds.situationRecords");
q.setHint("eclipselink.join-fetch", "ds.situationRecords.situationLocation");
q.setHint("eclipselink.join-fetch", "ds.situationRecords.situationRecordTypeElements");
q.setHint("eclipselink.join-fetch",
"ds.situationRecords.situationRecordGeneralPublicCommentMappings.
situationRecordGeneralPublicComments");
q.setHint("eclipselink.sql.hint", "/*+ leading(t1 t0) */");
q.setHint("eclipselink.query-results-cache", "true");
q.setHint("eclipselink.query-results-cache.expiry", "120000");
lResult = q.getResultList();
Persistence.xml
I have tried following properties
<!--property name="eclipselink.query-results-cache" value="true"/>
<property name="eclipselink.query-results-cache.expiry" value="120000"/>
<property name="eclipselink.cache.shared.default" value="true"/-->
<property name="eclipselink.cache.shared.default" value="false"/>
<property name="eclipselink.cache.shared.SituationData" value="true"/>
<property name="eclipselink.cache.type.SituationData" value="SOFT"/>
<property name="eclipselink.cache.size.SituationData" value="9400000"/>
The settings you have put on your entity are for entity caching, but complex queries still need to use the database to figure out which entities match read all/list queries. You should still see improvements though as executing this query multiple times will use that entity cache - while the data is still 'fetched' from the DB, you are avoiding the overhead of building/refreshing instances. A better test might be to check the overheads involved in:
EntityManager em1 = Emf.getInstance().getFactory().createEntityManager();
EntityManager em2 = Emf.getInstance().getFactory().createEntityManager();
DatexSituationData data1 = em1.find(DatexSituationData.class, 1L);
DatexSituationData data2 = em2.find(DatexSituationData.class, 1L);
The first find will go to the database and create instances to populate the shared and local caches, while the second should just use the shared cache to create a copy for its local EntityManager cache. This is a test that shows if your shared cache and its settings are working without other complexities.
Some issues with your settings and what you've shown though:
Entity class (and name) is "DatexSituationData" while you have named queries using "SituationData"
"eclipselink.cache.shared.SituationData" again is for SituationData
"eclipselink.cache.shared.default" is false. This turns off the cache for all entities, which will cause some confusion in your app as SituationData/DatexSituationData might be referenced in the shared cache, but its referenced SituationRecords won't forcing DB hits to load.
You've also set "query-results-cache" options as a query hint passed to the query. This is described in the docs and is what you may want, as it will allow the query results, based on the parameters used, to be cached. Limitations from the documentation:
Only named queries can have their results cached, dynamic queries
cannot use the query results cache. As well, if you modify a named
query before execution, such as setting hints or properties, then it
cannot use the cached results.
So to test query caching, you should use only a named query with no changes to it:
EntityManager em1 = Emf.getInstance().getFactory().createEntityManager();
Query namedQuery = em.createNamedQuery("DatexSituationData.findAll", DatexSituationData.class);
namedQuery.getResultList();
em.clear();
namedQuery = em.createNamedQuery("DatexSituationData.findAll", DatexSituationData.class);
namedQuery.getResultList();
If you need to add query hints, such as for fetch joins and the query result caching itself, add them in the query definition:
#NamedQuery(name="DatexSituationData.complexFind",
query="SELECT ds fetch join ds.situationRecords FROM DatexSituationData ds where CONCAT(ds.id.version, ds.id.id) in (select
CONCAT(max(FUNC('TO_NUMBER', ds1.id.version)), ds1.id.id) from SituationData ds1 group by
ds1.id.id)",
hints={#QueryHint(name="eclipselink.join-fetch", value="ds.situationRecords.situationLocation"),
#QueryHint(name="eclipselink.join-fetch", value="ds.situationRecords.situationRecordTypeElements"),
#QueryHint(name="eclipselink.join-fetch", value="ds.situationRecords.situationRecordGeneralPublicCommentMappings.situationRecordGeneralPublicComments"),
#QueryHint(name="eclipselink.sql.hint", value="/*+ leading(t1 t0) */"),
#QueryHint(name="eclipselink.query-results-cache", value="true"),
#QueryHint(name="eclipselink.query-results-cache.expiry", value="120000")})//2 minutes
or add named queries in the runtime:
Query namedQueryToSave = em.createQuery(yourQueryString, DatexSituationData.class);
//add all hints
em.getEntityManagerFactory().addNamedQuery("DatexSituationData.complexFind", namedQueryToSave);
em.close();
Either way, then you can execute the query. Using the previous test in a single em using "DatexSituationData.complexFind" should then show a hit to the database on the first run, but not the second - as long as they are less than 2 minutes apart.

Why does JPA call sql update on delete?

Let´s assume these two entities:
#Entity
public class MyEntity {
#Id private String id;
#OneToMany(mappedBy = "myEntity", cascade = ALL) private Set<MyEntityPredecessor> predecessors;
}
#Entity
public class MyEntityPredecessor{
#Id private String id;
#ManyToOne(name = "entityID", nullable = false) private MyEntity myEntity;
#ManyToOne(name = "entityPre", nullable = false) private MyEntity predecessor;
}
When I try to call a delete with Spring Boot Data (JPA) with a MyEntity Instance, it will work some times (I see the select and then the delete statements in correct order), but sometimes it will try to run an update on the second entity trying to set the "entityPre" Field to null (even thoug it is set to nullable=falsE), causing the DB to send an error (null not allowed!! from DB constraint).
Strangely, this will happen at "random" calls to the delete...
I just call "myEntityRepository.getOne(id)", and then myEntityRepository.delete() with the result... There is no data difference in the DB between calls, the data structure has no null values when calling the delete method, so that should not be the reason.
Why is JPA sometimes trying to call updates on the Predecessor Table, and sometimes directly deleting the values? Am I missing something?
Add a similar ManyToOne annotated set to MyEntity which refers to the other non-nullable property, like:
#OneToMany(mappedBy = "predecessor", cascade = ALL) private Set<MyEntityPredecessor> other;
some explanation:
The issue doesn't happen randomly, but happen when you try to delete an entity which is linked to one (or more) MyEntityPredecessor via the predecessor property (which is mapped to the entityPre field)
Only the other field (entityID) is mapped back to the MyEntity object, so the deletion-cascade only happens via by that field.

JPA: Usage of #GeneratedValue on non-id column

I am attempting to persist an entity with an attribute that I want to be populated from a DB sequence. I'm using Oracle, have created the sequence, verified the sequence works via sql, and yet my attribute isn't getting populated. Here's what I have:
#GeneratedValue(generator = "RFQ_LINE_IDS_SEQUENCE", strategy=GenerationType.SEQUENCE)
#SequenceGenerator(name="RFQ_LINE_IDS_SEQUENCE", sequenceName="RFQ_LINE_IDS_SEQUENCE", allocationSize=1000000000)
#Column(name = "external_line_item_id")
private String externalLineItemId;
All the examples I'm seen online show this annotation being used with #Id, but I have another attribute that I'm using for my id.
I've also tried the following to no avail:
#GeneratedValue(generator = "RFQ_LINE_IDS_SEQUENCE", strategy=GenerationType.SEQUENCE)
#GenericGenerator(name = "RFQ_LINE_IDS_SEQUENCE", strategy = "sequence",
parameters = {#Parameter(name = "sequence", value = "RFQ_LINE_IDS_SEQUENCE")})
#Column(name = "external_line_item_id")
private String externalLineItemId;
JPA only mandates support for #GeneratedValue on #Id fields. Some JPA implementations (such as DataNucleus JPA) support it but not all do.
I have created a proposal for JPA to support #GeneratedValue for non-id attributes. Please vote here for this to be included in 2.2
https://java.net/jira/browse/JPA_SPEC-113

Adding entity doesn't refresh parent's collection

the question and problem is pretty simple, though annoying and I am looking for a global solution, because it's application-wide problem for us.
The code below is really not interesting but I post it for clarification!
We use PostgreSQL database with JPA 2.0 and we generated all the facades and entities, of course we did some editing but not much really.
The problem is that every entity contains a Collection of its children, which however (for us only?) is NOT updated after creation a children element.
The objects are written to database, you can select them easily, but what we really would like to see is the refreshed collection of children in parent object.
Why is this happening? If we (manually) refresh the entity of parent em.refresh(parent) it does the trick but it would mean for us a lot of work in Facades I guess. But maybe there is no other way?
Thanks for support!
/* EDIT */
I guess it has to be some annotation problem or cache or something, but I've already tried
#OneToMany(mappedBy = "idquestion", orphanRemoval=true, fetch= FetchType.EAGER)
and
#Cacheable(false)
didn't work properly.
/* EDIT */
Some sample code for understanding.
Database level:
CREATE TABLE Question (
idQuestion SERIAL,
questionContent VARCHAR,
CONSTRAINT Question_idQuestion_PK PRIMARY KEY (idQuestion)
);
CREATE TABLE Answer (
idAnswer SERIAL,
answerContent VARCHAR,
idQuestion INTEGER,
CONSTRAINT Answer_idAnswer_PK PRIMARY KEY (idAnswer),
CONSTRAINT Answer_idQuestion_FK FOREIGN KEY (idQuestion) REFERENCES Question(idQuestion)
);
Than we have generated some Entities in Netbeans 7.1, all of them look similar to:
#Entity
#Table(name = "question", catalog = "jobfairdb", schema = "public")
#XmlRootElement
#NamedQueries({ BLAH BLAH BLAH...})
public class Question implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#Basic(optional = false)
#NotNull
#GeneratedValue(strategy= GenerationType.IDENTITY)
#Column(name = "idquestion", nullable = false)
private Integer idquestion;
#Size(max = 2147483647)
#Column(name = "questioncontent", length = 2147483647)
private String questioncontent;
#OneToMany(mappedBy = "idquestion", orphanRemoval=true)
private Collection<Answer> answerCollection;
Getters... setters...
We use (again) generated facades for them, all implementing AbstractFacade like:
public abstract class CCAbstractFacade<T> {
private Class<T> entityClass;
public CCAbstractFacade(Class<T> entityClass) {
this.entityClass = entityClass;
}
protected abstract EntityManager getEntityManager();
public void create(T entity) {
getEntityManager().persist(entity);
}
The father entity is updated automatically if you use container managed transactions and you fetch the collection after the transaction is complete. Otherwise, you have to update yourself the collection.
This article explains in detail this behaviour: JPA implementation patterns: Bidirectional associations
EDIT:
The simplest way to use Container Managed Transactions is to have transaction-type="JTA" in persistence.xml and use Container-Managed Entity Managers.
You seem to be setting the ManyToOne side, but not adding to the OneToMany, you have to do both.
In JPA, and in Java in general you must update both sides of a bi-directional relationship, otherwise the state of your objects will not be in sync. Not doing so, would be wrong in any Java code, not just JPA.
There is no magic in JPA that will do this for you. EclipseLink does have a magic option for this that you could set through a customizer (mapping.setRelationshipPartnerAttributeName()), but it is not recommended, fixing your code to be correct is the best solution.
See,
http://en.wikibooks.org/wiki/Java_Persistence/Relationships#Object_corruption.2C_one_side_of_the_relationship_is_not_updated_after_updating_the_other_side

Merging an object with FetchType.EAGER relation leads to "FailedObject"

I have an entity VM with a relationship to another entity BP. The relationship is eagerly fetched. First I load a VM. After loading the VM is detached, serialized and changed at the client side. Now I want to update the changed entity so I use the EntityManager.merge() method from JPA. Now I run into the following error from OpenJPA:
"Encountered new object in persistent field "Vm.bp" during attach. However, this field does not allow cascade attach. Set the cascade attribute for this field to CascadeType.MERGE or CascadeType.ALL (JPA annotations) or "merge" or "all" (JPA orm.xml). You cannot attach a reference to a new object without cascading."
Why do I have to add a Cascade.MERGE to a relationship to another entity that will never change? And why does JPA think that BP is a new object ("...cannot attach reference to a new object...")?
When using ManyToOne relationships do I always have to add Cascade.MERGE in order to update the entity or is this because of the EAGER fetch type?
Here's my entity:
#Entity
#Table(name = "VM")
public class Vm extends BaseEntity implements Serializable {
public static final long serialVersionUID = -8495541781540291902L;
#Id
#SequenceGenerator(name = "SeqVm", sequenceName = "SEQ_VM")
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "SeqVm")
#Column(name = "ID")
private Long id;
// lots of other fields and relations
#ManyToOne(fetch = FetchType.EAGER)
#JoinColumn(name = "BP_ID")
private Bp bp;
// ...
}
I found the reason why this error message comes up: The #Version annotated database field of the related Bp entity was initialized with "0". Apparently OpenJPA (1.2.3) is not able to cope with entity versions of zero.
Setting the version to 1 solved my issue.