JPQL Query working in testing, not in production - postgresql

I have two Entities related by a ManyToMany and I want to select them via a named Query. This works in my test (with a H2 DB set up) and throws exceptions at runtime (with postgresql set up). Other than the H2 and PG I am hard pressed to find differences between test and production.
The Entities and the Query look like so (abbreviated):
#Entity(name = "Enrichment")
#Table(name = "mh_Enrichment")
NamedQueries({
#NamedQuery(name = "findByLink",
query = "SELECT e FROM Enrichment e INNER JOIN e.links l WHERE l.link in (:links)") })
public class EnrichmentImpl {
#Id
#Column(name = "enrichmentId")
#GeneratedValue(strategy = GenerationType.AUTO)
private long id;
#ManyToMany
#JoinTable(name = "mh_EnrichmentLinks", joinColumns = { #JoinColumn(name = "EnrichmentId",
referencedColumnName = "enrichmentId") }, inverseJoinColumns = { #JoinColumn(name = "Link",
referencedColumnName = "link") })
private List<Link> links;
}
#Entity(name = "Link")
#Table(name = "mh_enrichment_link")
public class LinksImpl {
#Id
#Column(name = "link", length = 1024)
private String link;
}
Upon running the query with a String value in production I get:
Internal Exception: org.postgresql.util.PSQLException: ERROR: operator does not exist: character varying = bigint
Hinweis: No operator matches the given name and argument type(s). You might need to add explicit type casts.
Position: 215
Error Code: 0
Call: SELECT t1.enrichmentId FROM mh_enrichment_link t0, mh_EnrichmentLinks t2, mh_Enrichment t1 WHERE ((t0.link IN (?)) AND ((t2.EnrichmentId = t1.enrichmentId) AND (t0.link = t2.Link)))
Any ideas what's wrong? It is the query, isn't it?
The query is supposed to retrieve a list of Enrichments that are related to the given link.
Update #1
As requested: the tables in the DB look as follows:
For entity Link
CREATE TABLE mh_enrichment_link
(
link character varying(1024) NOT NULL,
CONSTRAINT mh_enrichment_link_pkey PRIMARY KEY (link)
)
For entity Enrichment
CREATE TABLE mh_enrichment
(
enrichmentid bigint NOT NULL,
CONSTRAINT mh_enrichment_pkey PRIMARY KEY (enrichmentid)
)
For the relation (See answer, this was where it went wrong)
CREATE TABLE mh_enrichmentlinks
(
link character varying(1024) NOT NULL,
CONSTRAINT mh_enrichment_link_pkey PRIMARY KEY (link)
)

The issue was fixed by dropping all related tables and having JPA regenerate them. Table definitions didn't match Entity definitions.
Thats also the quite obviously the reason why the test worked and the production didn't. In testing the tables are generated on runtime, in production they existed already (with an outdated definition).
Side note: The query is correct and does what it should.

Related

How to avoid id collisions in Spring Data JPA/Hibernate-generated database?

We use a dockerized postgres database and have hibernate auto-generate the tables (using spring.jpa.hibernate.ddl-auto: create) for our integration tests. Using something like H2 is not an option because we do some database-specific operations in a few places, e.g. native SQL queries.
Is there any way to avoid id collisions when all entities use auto-incremented ids? Either by offsetting the start id or, better yet, having all tables use a shared sequence?
Schema is created when the docker container is launched, tables are created by Spring Data JPA/Hibernate
Example
Examples use kotlin syntax and assumes the "allopen"-plugin is applied for entities.
Sometimes we've had bugs where the wrong foreign key was used, e.g. something like this:
#Entity
class EntityOne(
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id", nullable = false, columnDefinition = "SERIAL")
var id: Long,
)
#Entity
class EntityTwo(
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id", nullable = false, columnDefinition = "SERIAL")
var id: Long,
)
#Entity
class JoinEntity(
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id", nullable = false, columnDefinition = "SERIAL")
var id: Long,
#ManyToOne
#JoinColumn(name = "entity_one_id")
var entityOne: EntityOne,
#ManyToOne
#JoinColumn(name = "entity_two_id")
var entityTwo: EntityTwo,
)
#Repository
interface JoinEntityRepository : JpaRepository<JoinEntity, Long> {
//
// Bug here! Should be "WHERE entityOne.id = :entityOneId"
//
#Query("SELECT entityTwo FROM JoinEntity WHERE entityTwo.id = :entityOneId")
fun findEntityTwoByEntityOneId(entityOneId: Long): Collection<EntityTwo>
}
These bugs can in some circumstances be very hard to find because when the table is created, there may very well be an Entity2 with the same id as some Entity1, and so the query succeeds but the test fails somewhere down the line because while it is returning one or more Entity2, it's not the expected ones.
Even worse, depending on the scope of the test it may pass even if the wrong entity is fetched, or fail only when tests are run in a specific order (due to ids getting "out of sync"). So ideally it should fail to even find an entity when the wrong id is passed. But because the database structure is created from scratch and the ids are auto-incremented they always start at 1.
I found a solution to this.
In my resources/application.yml (in the test folder, you most likely do not want to do this in your main folder) I add spring.datasource.initialization-mode: always and a file data.sql.
The contents of data.sql are as follows:
DROP SEQUENCE IF EXISTS test_shared_sequence;
CREATE SEQUENCE test_shared_sequence;
ALTER TABLE entity_one ALTER COLUMN id SET DEFAULT nextval('test_shared_sequence');
ALTER TABLE entity_two ALTER COLUMN id SET DEFAULT nextval('test_shared_sequence');
After Spring has auto-generated the tables (using spring.jpa.hibernate.ddl-auto: create) it will run whatever is in this script, and the script will change all tables to auto-generate ids based on the same sequence, meaning that no two entities will ever have the same id regardless of which table they're stored in, and as such any query that looks in the wrong table for an id will fail consistently.

JPA generate partial unique key

I have a table with 3 columns
UUID - A UUID that is the primary key of the table
ID - A human readable ID of the resource (for a new resource, the ID should be automatically generated by a sequence)
Version - A version number
I am using JPA.
The table can contain multiple records with the same "human readable" ID and different versions.
I would like to be able to insert a new record without specifying the ID: the database should generate the ID automatically.
At the same time, when I need to insert a new version of the same resource, I would like to be able to insert a new row specifying the ID.
I have created a table where the UUID is the primary key, ID is defined as "integer generated by default as identity" and version is just an integer.
Using SQL query I can do what I want, but I do not know how to do it using JPA.
If I define the column as:
#Column(name="ID", insertable = false, updatable = false, nullable = false)
I can insert new records but the ID is always generated as new even if the resource already has one because the insert does not include the column.
If I define the column as:
#Column(name="ID", insertable = true, updatable = false, nullable = false)
The insert include the column and I am able to insert new rows specifying the ID but I cannot insert a row without the ID because the SQL generated is passing a null value for that column.
UPDATE
I have modified the configuration adding the annotation #Generated:
#Generated(value = GenerationTime.INSERT)
#Column(name="ID", updatable = false, nullable = false)
private Integer id;
With this, I am having the same problem: if I pass a value for id, the database is still generating a new one.
You can try to use #DynamicInsert annotation.
Assuming that you have the following table:
create table TST_MY_DATA
(
dt_id uuid,
dt_auto_id integer generated by default as identity,
dt_version integer,
primary key(dt_id)
);
Appropriate entity will look like this:
#Entity
#Table(name = "TST_MY_DATA")
#DynamicInsert
public class TestData
{
#Id
#GeneratedValue
#Column(name = "dt_id")
private UUID id;
// Unfortunately you cannot use #Generated annotation here,
// otherwise this column will be always absent in hibernate generated insert query
// #Generated(value = GenerationTime.INSERT)
#Column(name = "dt_auto_id")
private Long humanReadableId;
#Version
#Column(name = "dt_version")
private Long version;
// getters/setters
}
and then you can persist entities:
TestData test1 = new TestData();
session.persist(test1);
TestData test2 = new TestData();
test2.setHumanReadableId(27L);
session.persist(test2);
session.flush();
// here test1.getHumanReadableId() is null
/*
* You can use session.refresh(entity) only after session.flush() otherwise you will have:
* org.hibernate.UnresolvableObjectException: No row with the given identifier exists:
* [this instance does not yet exist as a row in the database#ff09c202-cd17-4d4a-baea-057e475fabb9]
**/
session.refresh(test1);
// here you can use the test1.getHumanReadableId() value fetched from DB

spring data JPA association in Map

I'm trying to use Map in Spring Data JPA to handle the relationship to store records of equipment quantity.
I followed this guide to create the entity.
Meeting{
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "meeting_id", updatable = false)
#JsonIgnore
private int id;
#ElementCollection
#MapKeyColumn(name = "equipment_type")
#MapKeyEnumerated(EnumType.STRING)
private Map<EquipmentType, Integer> equipment = new HashMap<>();
}
EquipmentType is an Enum.
This is the table for the property:
CREATE TABLE IF NOT EXISTS meeting_equipment (
meeting_id INTEGER NOT NULL REFERENCES meeting (meeting_id),
equipment_type VARCHAR(10) NOT NULL,
quantity INTEGER NOT NULL DEFAULT 0
);
Once I try to create a meeting entity, I get error ERROR:column "meeting_meeting_id" of relation "meeting_equipment" does not exist
May I know what's the problem here?
Your table meeting_equipment does not match what JPA is expecting.
It has a column meeting_id but your JPA implementation expects meeting_meeting_id
Either rename the column to the expected meeting_meeting_id or configure your mapping to use the current column name. I think this might do the trick:
#JoinTable(joinColumns={#JoinColumn(name="meeting_id")}
Of course, you probably can create your own naming strategy if you have many cases like this and want to keep your column names as they are.

Using CriteriaBuilder to select entities referenced by refTable

I'm very new to JPA (based on eclipselink) so be patient with me and my question. I found JPA is an elegant way to handle simple DB statements. But, now I'd like to do some more complicated task.
So let me explain what I want: I've got a table holding person data (lets say the persons name). In an other table I'd like to keep address lists. Because a person way part of several adresslists and one adress list may keep several persons I need an ref-table jo join these collections.
So my DB-tables look like this:
SQL (not exact DDL syntax but it helps to understand!) --without constraints etc
--the person keeps data of persons
CREATE TABLE a_person
(
oid serial PrimaryKey NOT NULL,
vorname character varying(50),
nachname character varying(50) NOT NULL
--
)
--the adresslist is quite simple: just a name of the list is storred
CREATE TABLE a_adressliste (
(
oid serial PrimaryKey NOT NULL,
label character varying(25) NOT NULL
--<more collumns here>
)
--the table refering the list the the containing persons
CREATE TABLE a_listen_person
(
fk_adressliste bigint PrimaryKey NOT NULL, --ref to a_person.oid
fk_person bigint PrimaryKey NOT NULL, --ref to a_adressliste.oid
)
Having this structure and some data in the DB it's easy to select the adress lists using the following SQL statement:
select * from a_person p where p.oid in (
select
lp.fk_person
from
a_listen_person lp, a_adresssliste al
where
lp.fk_adressliste = al.oid AND al.label = {LISTE_LABEL}
)
Now, According tho the structure within the DB, I've got the corresponding JAVA POJO's to these tables (I've skipped the annotations to keep my code a little shorter)
JAVA
public class Person implements Serializable {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Basic(optional = false)
#Column(name = "oid")
private Integer oid;
private String vornmae;
private String nachname;
//getter; setter and the other missing stuff...
}
public class Adressliste implements Serializable {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Basic(optional = false)
#Column(name = "oid")
private Integer oid;
private String label;
}
public class ListenPerson implements Serializable {
#EmbeddedId
private ListenPersonPK listenPerson;//build from Person.oid and Adressliste.oid
private Person person;
private Adressliste adresssliste;
//getter; setter and the other missing stuff...
}
Now I've written a finder method in JAVA where I use a CriteriaBuilder to filter the entities by several attributes (according to the Person POJO). But I did not manage to select the Person according to a given list name.
By now my method looks like this:
public TypedQuery<Person> prepareQueryFiltered(Filter filter) {
TypedQuery<Person> retVal;
EntityManager em = this.getEntityManager();
CriteriaBuilder builder = em.getCriteriaBuilder();
CriteriaQuery<Person> query = builder.createQuery(Person.class);
Root<Person> cust = query.from(Person.class);
query.select(cust);
List<Predicate> predicateList = new ArrayList<Predicate>();
Predicate predicate;
if (null != filter.getVorname()) {
predicate = builder.like(
builder.lower(cust.<String>get("vorname")),
"%" + filter.getVorname().toLowerCase() + "%");
predicateList.add(predicate);
}
if (null != filter.getNachname()) {
predicate = builder.like(
builder.lower(cust.<String>get("nachname")),
"%" + filter.getNachname().toLowerCase() + "%");
predicateList.add(predicate);
}
//some more filtered attributes ...
query.where(predicateList.toArray(new Predicate[predicateList.size()]));
retVal = em.createQuery(query);
return retVal;
}
As you can imagine the filter attribute keeps all the data to filter my entities with. But how does the code look like if I'd like to get all the person entities belonging to a given list name?
I started to use 'Subquery's but I did not get the correct syntax. Could you please give me some hints?
For examples of sub-queries in the Criteria API see,
http://en.wikibooks.org/wiki/Java_Persistence/Criteria#Subquery
Also, I don't think you need a sub-query, a simple join should work.
Select p from Person p, ListenPerson l where l.person = p and l.adresssliste.label = :label

Java EE 6 JPA 2 ManyToOne Relation Creates Invalid Foreign Key

I am trying to create two entities where both entities have embeddedIds. One of the entities have 2 references to the other entity, where both of those references are related as ManyToOne.
Example codes are written below;
#Embeddable
public class ItemPK {
#Column(nullable = false, length = 100)
private String itemId;
#Column(name = "item_client_id", nullable = false)
private int clientId;
...
}
#Entity
#Table(name = "item")
public class Item {
#EmbeddedId
private ItemPK id;
#ManyToOne
#JoinColumn(name = "item_client_id")
private Client client;
#OneToMany(mappedBy="item", cascade = CascadeType.ALL, orphanRemoval = true)
private Set<RelatedItem> relatedItems;
#OneToMany(mappedBy="relatedItem", cascade = CascadeType.ALL, orphanRemoval = true)
private Set<RelatedItem> relatedItemsRHS;
...
}
#Embeddable
public class RelatedItemPK {
#Column(name = "itemId", length = 100, nullable = false)
private String itemId;
#Column(name = "item_client_id", nullable = false)
private int clientId;
#Column(name = "relatedItemId", length = 100, nullable = false)
private String relatedItemId;
#Column(name = "related_item_client_id", nullable = false)
private int relatedItemClientId;
...
}
#Entity
#Table(name = "related_item")
public class RelatedItem {
#EmbeddedId
private RelatedItemPK id;
#ManyToOne(cascade = CascadeType.ALL, optional = false)
#JoinColumns({
#JoinColumn(name="itemId", referencedColumnName="itemId", insertable=false, updatable=false),
#JoinColumn(name="item_client_id", referencedColumnName="item_client_id", insertable=false, updatable=false)
})
private Item item;
#ManyToOne(cascade = CascadeType.ALL, optional = false)
#JoinColumns({
#JoinColumn(name="related_item_client_id", referencedColumnName="item_client_id", insertable=false, updatable=false),
#JoinColumn(name="relatedItemId", referencedColumnName="itemId", insertable=false, updatable=false)
})
private Item relatedItem;
...
}
The problem is while creating foreign keys for RelatedItem entity, I got an SQLException. It is the second ManyToOne relation that fails. The foreign key generation sql is below,
ALTER TABLE related_item ADD CONSTRAINT FK_related_item_related_item_client_id FOREIGN KEY (related_item_client_id, relatedItemId) REFERENCES item (item_client_id, itemId)
Since item table is indexed first by itemId then by item_client_id, this statement causes MySQL to produce an error.
I would like to switch the places of columns so that the SQL should look like the following,
ALTER TABLE related_item ADD CONSTRAINT FK_related_item_relatedItemId FOREIGN KEY (relatedItemId, related_item_client_id) REFERENCES item (itemId,item_client_id)
I tried changing the order of "JoinColumn"s but the result didn't change. I also tried renaming the fields to check if persistence provider choses the order by column name but again the result didn't change.
So, is there a way to enforce the column ordering?
p.s. I use following stuff:
MySQL 5.1
EclipseLink 2.0.0
Java EE 6
JPA 2
GlassFish v3
Edit: EclipseLink produces following SQL, which fails to run;
CREATE TABLE related_item (SIMILARITY DOUBLE, widget_id INTEGER NOT NULL, relatedItemId VARCHAR(100) NOT NULL, itemId VARCHAR(100) NOT NULL, related_item_client_id INTEGER NOT NULL, item_client_id INTEGER NOT NULL, PRIMARY KEY (widget_id, relatedItemId, itemId, related_item_client_id, item_client_id));
CREATE TABLE item (IMAGEURL VARCHAR(2048), STATUS VARCHAR(64), URL VARCHAR(2048), PRICE DOUBLE, STOCK INTEGER, DESCRIPTION TEXT(64000), NAME VARCHAR(255), ITEMID VARCHAR(100) NOT NULL, item_client_id INTEGER NOT NULL, PRIMARY KEY (ITEMID, item_client_id));
ALTER TABLE related_item ADD CONSTRAINT FK_related_item_itemId FOREIGN KEY (itemId, item_client_id) REFERENCES item (itemId, item_client_id);
ALTER TABLE related_item ADD CONSTRAINT FK_related_item_related_item_client_id FOREIGN KEY (related_item_client_id, relatedItemId) REFERENCES item (item_client_id, itemId);
ALTER TABLE item ADD CONSTRAINT FK_item_item_client_id FOREIGN KEY (item_client_id) REFERENCES client (ID);
Please include the stack trace. However, I strongly recommend you skip the #JoinColumn tags unless you have a VERY good reason for specifying the foreign keys yourself. By specifying the mappedBy attribute in one of the directions, JPA can figure out what to do by itself.
Java EE 6 and JPA put a lot of effort into enabling Convention over Configuration, which means that most of the time, things will work out of the box. It's desirable for you, the programmer because you have less boiler plate code to worry about, and it's desirable for the JPA and Jave EE container implementors because it gives them freedom to chose the best performing solutions. By declaring the foreign key relationships yourself, you rob both you and JPA of this advantage.
Edit: In fact, I suspect that both specifying mappedBy and specifying the #JoinTable could be the root cause of your problem. But I need to see the stack trace to tell for sure.
The order of the columns should not matter. If it does, then you could change the order in your index to match, or change the order you list your primary key in, or just use your scripts to generate your DDL.