How to use MySQL's full text search from JPA - jpa

I want to use MySQL's full text search features using JPA, without having to use a native query.
I am using EclipseLink, which has a function to support native SQL commands: FUNC. However, the help examples only show this being use with simple MySQL functions. My best effort attempt to get it to work with MATCH & AGAINST is as follows:
#PersistenceContext(name="test")
EntityManager em;
Query query = em.createQuery("SELECT person FROM People person WHERE FUNC('MATCH', person.name) FUNC('AGAINST', :searchTerm)");
...
query.getResultList();
Which gives the following exception:
Caused by: NoViableAltException(32#[()* loopback of 822:9: (m= MULTIPLY right= arithmeticFactor | d= DIVIDE right= arithmeticFactor )*])
at org.eclipse.persistence.internal.libraries.antlr.runtime.DFA.noViableAlt(DFA.java:159)
at org.eclipse.persistence.internal.libraries.antlr.runtime.DFA.predict(DFA.java:116)
at org.eclipse.persistence.internal.jpa.parsing.jpql.antlr.JPQLParser.arithmeticTerm(JPQLParser.java:4557)
... 120 more
I am open to alternatives other that using the FUNC method.
I am using EJB 3 and EclipseLink 2.3.1.

An improved answer of #Markus Barthlen which works for Hibernate.
Create custom dialect
public class MySQLDialectCustom extends MySQL5Dialect {
public MySQLDialect() {
super();
registerFunction("match", new SQLFunctionTemplate(StandardBasicTypes.DOUBLE,
"match(?1) against (?2 in boolean mode)"));
}
}
and register it by setting hibernate.dialect property.
Use it
in JPQL:
Query query = entityManager
.createQuery("select an from Animal an " +
"where an.type = :animalTypeNo " +
"and match(an.name, :animalName) > 0", Animal.class)
.setParameter("animalType", "Mammal")
.setParameter("animalName", "Tiger");
List<Animal> result = query.getResultList();
return result;
or with Criteria API:
CriteriaBuilder criteriaBuilder = entityManager.getCriteriaBuilder();
CriteriaQuery<Animal> criteriaQuery = criteriaBuilder.createQuery(Animal.class);
Root<Animal> root = criteriaQuery.from(Animal.class);
List<Predicate> predicates = new ArrayList<>();
Expression<Double> match = criteriaBuilder.function("match", Double.class, root.get("name"),
criteriaBuilder.parameter(String.class, "animalName"));
predicates.add(criteriaBuilder.equal(root.get("animalType"), "Mammal"));
predicates.add(criteriaBuilder.greaterThan(match, 0.));
criteriaQuery.where(predicates.toArray(new Predicate[]{}));
TypedQuery<Animal> query = entityManager.createQuery(criteriaQuery);
List<Animal> result = query.setParameter("animalName", "Tiger").getResultList();
return result;
Some more details in this blog post: http://pavelmakhov.com/2016/09/jpa-custom-function

FUNC only works with normal printed functions,
i.e.
MATCH(arg1, arg2)
since MATCH arg1 AGAINST arg2 is not printed the way a function is normally printed, FUNC cannot be used to call it.
EclipseLink ExpressionOperators do support printing functions like this, so you could define your own ExpressionOperator, but ExpressionOperators are only supported through EclipseLink Expression queries currently, not through JPQL. You could log an enhancement to have operator support in JPQL.
You could also use a native SQL query.

Just to complete the answer: I had the same problem, but using the criteria builder. This is how you can get around the limitations in the standart implementation, if you are using EclipseLink:
Cast JPA expression to EclipseLink expression
Use the sql method
If you match against a compound index, create it using the function method
Example:
JpaCriteriaBuilder cb = (JpaCriteriaBuilder) cb;
List<String> args = new ArrayList();
args.add("Keyword");
Expression<Boolean> expr = cb.fromExpression (
cb.toExpression(
cb.function("", String.class,
table.get(Table_.text1), table.get(Table_.text2))
)
.sql("MATCH ? AGAINST (?)", args)
);
query.where(expr);
If you need to cast the expression to a predicate use the following:
query.where( cb.gt(expr, 0));

What about new SQL operator in EclipseLink 4.0? I think it can help you to do fulltext search from JPQL. But you have to upgrade to EclipseLink 4.0.
http://wiki.eclipse.org/EclipseLink/UserGuide/JPA/Basic_JPA_Development/Querying/Support_for_Native_Database_Functions#SQL
Edit:
Sorry for late update.
Verified correct use of EclispeLink 2.4.0 "SQL" operator with MySQL fulltext search is
SELECT person FROM People person WHERE SQL('MATCH(name) AGAINST( ? )', :searchTerm)"
where name is column on which Fulltext index is defined. :searchTerm is string you use for searching.
Works without problems.

To elaborate on the answer of James:
It seems like I had luck extending the mysql dialect using
registerFunction("match", new SQLFunctionTemplate(DoubleType.INSTANCE, "match(?1) against (?2 in boolean mode)"));
and invoking the function via the following jpql fragment
match(" + binaryDataColumn + ",'" + StringUtils.join(words, " ") + "') > 0
I had to guess the return type, but this should get you started.

FInally work
if you set your table colums wit index full search
#NamedNativeQuery(name = "searchclient",
query = "SELECT * FROM client WHERE MATCH(clientFullName, lastname, secondname, firstphone,"
+ " secondphone, workphone, otherphone, otherphone1,"
+ " otherphone2, detailsFromClient, email, company,"
+ " address, contractType, paymantCondition) AGAINST(?)",
List list = em.createNamedQuery("searchclient").setParameter(1, searchKey).getResultList();

The simplest variant is to use NativeQuery
Example of use it with mapping to JPA entity (FiasAddress):
public class FiasServiceBean implements FiasService {
#PersistenceContext(unitName = "fias")
EntityManager entityManager;
#Override
public Collection<FiasAddress> search(String name, int limit, int aolevel) {
Query query = entityManager.createNativeQuery(
"SELECT fa.* FROM fias.addressobject fa" +
" WHERE MATCH(FORMALNAME) AGAINST (:name IN NATURAL LANGUAGE MODE)" +
" AND AOLEVEL = :AOLEVEL" +
" LIMIT :limit",
FiasAddress.class
);
query.setParameter("name", name);
query.setParameter("limit", limit);
query.setParameter("AOLEVEL", aolevel);
Iterator iterator = query.getResultList().iterator();
ArrayList<FiasAddress> result = new ArrayList<>();
while (iterator.hasNext()) {
result.add((FiasAddress) iterator.next());
}
return result;
}
}

Related

How to avoid N+1 problem with native SQL query in springboot with Hibernate?

I'm querying my DB with POSTGIS built-ins to retrieve the closest Machines given a Location.
I have to use a native SQL because Hibernate does not support POSTGIS and CTEs:
#Repository
public interface MachineRepository extends JpaRepository<Machine, Long>{
#Query(value =
"with nearest_machines as\n" +
" (\n" +
" select distance_between_days(:id_day, machine_availability.id_day) as distance_in_time,\n" +
" ST_Distance(geom\\:\\:geography, ST_SetSrid(ST_MakePoint(:longitude, :latitude), 4326)\\:\\:geography) as distance_in_meters,\n" +
" min(id_day) over (partition by machine.id) as closest_timeslot_per_machine,\n" +
" machine_availability.id_day,\n" +
" machine.*\n" +
" from machine\n" +
" join machine_availability on machine.id = machine_availability.id_machine\n" +
" where machine_availability.available = true\n" +
" and machine_availability.id_day >= :today\n" +
" and ST_DWithin(geom\\:\\:geography, ST_SetSrid(ST_MakePoint(:longitude, :latitude), 4326)\\:\\:geography, 1000)\n" +
" )\n" +
"select nearest_machines.*\n" +
"from nearest_machines\n" +
"where id_day = closest_timeslot_per_machine\n" +
"order by distance_in_time, distance_in_meters\n" +
"limit 20;",
nativeQuery = true)
List<Machine> findMachinesAccordingToAvailabilities(#Param("longitude") BigDecimal longitude,
#Param("latitude") BigDecimal latitude,
#Param("id_day") String idDay,
#Param("today") String today);
}
Of course, Machine and MachineAvailability are #Entity's. And they are #OneToMany(fetch = FetchType.EAGER) related. I changed the default LAZY to EAGER cause i need the MachineAvailability in the final JSON.
The problem is that it triggers 2 more requests by resulting machine(ie the famous N+1 problem).
1.How can i solve that in only ONE request?
2.Is it possible to create my on JSON somehow and returning it directly in the MachineController?
Solving this in 1 request is tough as you will have to use Hibernate native APIs to map the table aliases for the availability collection. You would need to add a join for the availabilities in the main query and do something like this: session.createNativeQuery("...").addEntity("m", Machine.class).addFetch("av", "m", "availabilities")
Another alternative would be to use Blaze-Persistence Entity Views since Blaze-Persistence comes with support for CTEs and many more goodies that PostgreSQL provides, this might be an interesting solution for you.
I created the library to allow easy mapping between JPA models and custom interface or abstract class defined models, something like Spring Data Projections on steroids. The idea is that you define your target structure(domain model) the way you like and map attributes(getters) via JPQL expressions to the entity model.
I don't know your model, but a possible DTO model for your use case could look like the following with Blaze-Persistence Entity-Views:
#EntityView(Machine.class)
#With(NearestMachineCteProvider.class)
#EntityViewRoot(name = "nearest", entity = NearestMachine.class, condition = "machineId = VIEW(id)", joinType = JoinType.INNER)
public interface MachineDto {
#IdMapping
Integer getId();
String getName();
#Mapping("nearest.distanceInTime")
Integer getDistanceInTime();
#Mapping("nearest.distanceInMeters")
Double getDistanceInMeters();
Set<MachineAvailabilityDto> getAvailabilities();
#EntityView(MachineAvailability.class)
interface MachineAvailabilityDto {
#IdMapping
Integer getId();
String getName();
}
class NearestMachineCteProvider implements CTEProvider {
#Override
public void applyCtes(CTEBuilder<?> builder, Map<String, Object> optionalParameters) {
builder.with(NearestMachine.class)
.from(Machine.class, "m")
.bind("distanceInTime").select("CAST_INTEGER(FUNCTION('distance_between_days', :id_day, m.availabilities.idDay))")
.bind("distanceInMeters").select("CAST_DOUBLE(FUNCTION('ST_Distance', m.geom, FUNCTION('ST_SetSrid', FUNCTION('ST_MakePoint', :longitude, :latitude), 4326)))")
.bind("closestTimeslotId").select("min(m.availabilities.idDay) over (partition by m.id)")
.bind("machineId").select("m.id")
.bind("machineAvailabilityDay").select("m.availabilities.idDay")
.where("m.availabilities.available").eqLiteral(true)
.where("m.availabilities.idDay").geExpression(":today")
.where("FUNCTION('ST_DWithin', m.geom, FUNCTION('ST_SetSrid', FUNCTION('ST_MakePoint', :longitude, :latitude), 4326), 1000)").eqLiteral(true)
.end();
}
}
}
#CTE
#Entity
public class NearestMachine {
private Integer distanceInTime;
private Double distanceInMeters;
private Integer closestTimeslotId;
private Integer machineId;
private Integer machineAvailabilityDay;
}
Querying is a matter of applying the entity view to a query, the simplest being just a query by id.
MachineDto a = entityViewManager.find(entityManager, MachineDto.class, id);
The Spring Data integration allows you to use it almost like Spring Data Projections: https://persistence.blazebit.com/documentation/entity-view/manual/en_US/index.html#spring-data-features
Page<MachineDto> findAll(Pageable pageable);
You can then sort by using Sort.asc("distanceInTime") and Sort.asc("distanceInMeters")
The best part is, it will only fetch the state that is actually necessary!

Spring JPA repository casting error when using JPQL

I have a PagingAndSorting JPA repository declared. I am using the #Query annotation.
I am getting an exception when I call the get() method on an Optional object from the findById(id) method of the repository.
The weird thing is it only happens when I use JPQL.
The code works if my query is native:
#Override
public BatchDto findById(String id) {
Optional<Batch> findResult = this.batchRepository.findById(id);
if (!findResult.isPresent()) return null;
Batch entity = findResult.get(); **<-------- Cast Exception Here**
BatchDto dto = this.mapper.toDto(entity, BatchDto.class);
List<BatchTransaction> transactions = entity.getTransactions();
dto.setTransactionDtos(mapper.toListDto(transactions, TransactionDto.class));
return dto;
}
Inspecting the findResult object with a breakpoint - I can see:
Optional[net.domain.data.batch#4b8bb6f]
when I have nativeQuery = true in the #Query annotation.
#Query(value = Sql.FindBatchById, nativeQuery = true)
Here is the query being used:
SELECT DISTINCT(B.batchNumber), COUNT(B.batchNumber) as TransactionCount FROM BATCH B WHERE B.batchReferenceNumber = :id GROUP BY B.batchNumber
However if I change it to JPQL and remove the nativeQuery=true attribute - the findResult is
Optional[[Ljava.lang.Object;#76e04327].
and I get a ClassCastException:
java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to net.domain.data.batch
So bottom line - this works when specify nativeQuery=true and fails when I try to use JPQL.
I would prefer not to specify nativeQuery as we will eventually port this db to Oracle.
First of all the query shown below doesn't return a single Batch instance. Since there are distinct and count aggregate functions, the query will return a List of aggregates.
To be able to read that statistics you can add appropriate method into the batchRepository. Something like this:
#Query("SELECT DISTINCT(B.batchNumber) as dist, COUNT(B.batchNumber) as cnt FROM BATCH B GROUP BY B.batchNumber")
List<Map<Long, Long>> findStatistics();
and then iterate through the list.
UPD
If the id parameter exactly guarantee that will return a single record, you can change a return type to a Map
#Query("SELECT DISTINCT(B.batchNumber) as dist, COUNT(B.batchNumber) as cnt FROM BATCH B WHERE B.batchReferenceNumber = :id GROUP BY B.batchNumber")
Map<Long, Long> findStatisticsById(#Param("id") Long id);

Specifying a LIMIT clause in a JPA2 repository #Query statement

I have the following MySQL statement:
"SELECT * FROM $this->tableName WHERE news_publication_id = '" + newsPublicationId + "' AND list_order > '" + listOrder +"' ORDER BY list_order LIMIT 1"
I started on the data handling path by first doing an Hibernate DAO layer.
It had the following criteria based method:
#Override
public NewsHeading findNextWithListOrder(NewsPublication newsPublication, int listOrder) {
Criteria criteria = getSession().createCriteria(getPersistentClass());
criteria.add(Restrictions.eq("newsPublication", newsPublication));
criteria.add(Restrictions.gt("listOrder", listOrder)).addOrder(Order.asc("listOrder")).setMaxResults(1);
return (NewsHeading) criteria.uniqueResult();
}
And it works just fine.
Then I tried doing a JPA2 repository layer.
It now has the following JPQL based method:
#Query("SELECT nh FROM NewsHeading nh WHERE nh.newsPublication = :newsPublication AND nh.listOrder > :listOrder ORDER BY nh.listOrder ASC LIMIT 1")
public NewsHeading findByNextListOrder(#Param("newsPublication") NewsPublication newsPublication, #Param("listOrder") int listOrder);
But it doesn't work. Because the LIMIT clause is not valid.
So I wonder what to do.
Can I still use the criteria based method of above in this JPA2 repository layer ?
I would like to avoid trusting the client to pass the limit value and prefer to hard code that value (of 1) into my layer.
Also, that statement returns only one object and so there is no pagination needed.
Any tip would be most welcomed.
Kind Regards,
Stephane Eybert
You can probably this with a PageRequest. http://docs.spring.io/spring-data/jpa/docs/1.6.2.RELEASE/reference/html/repositories.html#repositories.core-concepts
Hibernate wants you to do setMaxResults() on the Query object, but JPA doesn't interface with that directly.
Other option would be to use a #NamedNativeQuery

Using Soundex and CriteriaBuilder API from EclipseLink

Currently in the process of creating a object search using CriteriaBuilder, Predicates, JPA 2.0 with EclipseLink as provider.
My challenge is accessing the soundex capabilities and applying it to a dynamically built criteria.
CriteriaBuilder cb = PersistenceUtil.getEntityManager().getCriteriaBuilder();
CriteriaQuery<Registration> q = cb.createQuery(Registration.class);
Root<Registration> dr = q.from(Registration.class);
List<Predicate> predicates = new ArrayList<Predicate>();
... Loop of my inputs to the query
predicates.add(cb.equal(dr.get(fieldName),fieldValue));
... and finally
q.select(dr).where(predicates.toArray(new Predicate[]{}));
TypedQuery<Registration> query = PersistenceUtil.getEntityManager().createQuery(q).setMaxResults(maxResults);
List<DeathRegistration> results = query.getResultList();
This obviously works fine for the simple criteria builder items, and I can use 'like', 'greaterThan', date comparison and so on.
I want to enable use of a Expression that uses the EclipseLink soundex() operator. Using EclipseLink provider opens up my ability to create an eclipselink Expression, but I canot figure out how to apply it to a predicate.
ReadAllQuery raq = new ReadAllQuery(Registration.class);
ExpressionBuilder eb = raq.getExpressionBuilder();
org.eclipse.persistence.expressions.Expression fnExp = ExpressionOperator.soundex().expressionFor(eb.get(fieldName));
org.eclipse.persistence.expressions.Expression fnVal = fnExp.equal(fieldValue);
Having a lot of trouble finding documentation that would allow me to create an Expression that can be used in CriteriaBuilder. Is it possible? Can an EclipseLink Expression be converted to a parametrized persistence Expression<>? ... and then be set as a predicate for the built criteria query?
After sleeping on the issue, and some more googling. Here is where I ended up:
Expression<String> exp = dr.get(fieldName);
Expression<String> expName = CriteriaBuilderHelper.functionSoundexName(cb, exp);
Expression<String> expValue = CriteriaBuilderHelper.functionSoundexValue(cb, fieldValue);
predicates.add(cb.equal(expName, expValue));
and my function:
private static final String SOUNDEX = "soundex";
/**
* #param cb the CriteriaBuilder to use
* #param value the value to soundex
* #return Expression<String>
*/
public static Expression<String> functionSoundexName(CriteriaBuilder cb, Expression<String> exp) {
return cb.function(
SOUNDEX,
String.class,
cb.lower(exp)
);
}
public static Expression<String> functionSoundexValue(CriteriaBuilder cb, String value) {
return cb.function(
SOUNDEX,
String.class,
cb.lower(cb.literal(value))
);
}

How to call Named Query

I wrote a named query in the entity class Voter
NamedQuery(name = "Voter.findvoter", query = "SELECT count(*) FROM Voter v WHERE v.voterID = :voterID" and where v.password= : password),
I want to call this named query and I also need to set voterID and password.
Can you help me. Thank you
I assume you've missed the # symbol on your NamedQuery annotation?
In the code, you'd call it like this:
List results = em.createNamedQuery("Voter.findvoter")
.setParameter("voterID", "blah")
.setParameter("password","blahblahblah")
.getResultList();
There are two obvious issues with your named query that would cause a problems:
It is an annotation so it should be #NamedQuery not just NamedQuery
Your query is currently:
query = "SELECT count(*) FROM Voter v WHERE v.voterID = :voterID" and where v.password= : password.
The problem is that you terminate your String after :voterID, instead of after :password and you have "where" twice and you have a space between ":" and "password". Your query should look like this:
query = "SELECT count(*) FROM Voter v WHERE v.voterID = :voterID and v.password= :password"
(I have just moved the " to the end and removed the second "where" and the space after the ":")
The common steps are (named query or otherwise)
Create a query - em has five create methods.
Set the query up with parameters if needed - the query interface has these methods.
Execute the query - the query interface has 3 execution related methods.
with the above three steps you can run any JPA query.
Actually brent is right your NameQuery should be something like this,
#NamedQuery(name = "Voter.findvoter", query = "SELECT count(*) FROM Voter v WHERE v.voterID = :voterID AND where v.password = :password")
#Entity
public class Voter implements Serializable{ ... }
and somewhere else you should try this one (which Dick has already said)
public class VoterFasade{
public List<Voter> findVoter(long id,String password){
List<Voter> results = em.createNamedQuery("Voter.findvoter")
.setParameter("voterID", id)
.setParameter("password",password)
.getResultList();
return result;
}
}
then you could use it like
#Inject
VoterFasade voterFasade;
///
long id=12;
voterFasade.findVoter(id);
should actually working.(its an uncompiled code).
you could also do it with Repository, check the link below, part Repository Listing23.Example repository
enter link description here