I have millions of objects populated in GemFire regions. I don't want default findAll() SDR query to be executed to retrieve the millions of objects in one shot. I am trying to figure out if there is a way to override the default findAll query and provide the LIMIT param to restrict the number of objects retrieved from GemFire Regions. Here is an example of what I want to do:
NoRepositoryBean
public interface CrudRepository<T, ID extends Serializable> extends Repository<T, ID> {
/**
* Returns all instances of the type.
*
* #return all entities
*/
Iterable<T> findAll();
}
public interface MyRepository extends CrudRepository<MyRepoObject, String> {
#Query("SELECT * FROM MyRegion LIMIT $1")
Iterable<CellTower> findAll(#Param("limit") String limit);
}
Currently, I am on Spring Data Gemfire 1.4.0.BUILD-SNAPSHOT and Spring Data REST 2.0.0.BUILD-SNAPSHOT version
This handy getting started guide on Accessing GemFire Data with REST (https://spring.io/guides/gs/accessing-gemfire-data-rest/) was written not long ago and may help with your particular use case.
The following worked for me. Try using an Integer instead of a String as your parameter to findAll
#Query("SELECT * FROM /Customer LIMIT $1")
List<Customer> findAll(#Param("limit") Integer max);
Related
How can I avoid unnecessary queries to the DB?
I have LoadEntity with two nested entity - CarrierEntity and DriverEntity. Java class:
#Entity
public class LoadEntity {
...
#ManyToOne
#JoinColumn(name="carrier_id", nullable=false)
private CarrierEntity carrierEntity;
#ManyToOne
#JoinColumn(name="driver_id", nullable=false)
private DriverEntity driverEntity;
}
But API send me carrierId and driverId. I make it:
DriverEntity driverEntity = driverService.getDriverEntityById(request.getDriverId());
loadEntity.setDriverEntity(driverEntity);
loadRepository.save(loadEntity);
How can I write only driverId with JPA?
With Spring Data JPA you can always fall back on plain SQL.
Of course, this will side step all the great/annoying logic JPA gives you.
This means you won't get any events and the entities in memory might be out of sync with the database.
For this reason you might also increase the version column, if you are using optimistic locking.
That said you could update a sing field like this:
interface LoadRepository extends CrudRepository<LoadEntity, Long> {
#Query(query="update load_entity set driver_id = :driverId where carrier_id=:carrier_id", nativeQuery=true)
#Modifying
void updateDriverId(Long carrierId, Long driverId);
}
If you just want to avoid the loading of the DriverEntity you may also use JpaRepository.getById
In Spring Data Jpa to get first 10 rows I can do this findTop10By...(). In my case the number or rows is not defined and comes as a parameter.
Is there something like findTopNBy...(int countOfRowsToGet)?
Here is another way without native query. I added Pageable as a parameter to the method in the interface.
findAllBySomeField(..., Pageable pageable)
I call it like this:
findAllBySomeField(..., PageRequest.of(0, limit)) // get first N rows
findAllBySomeField(..., Pageable.unpaged()) // get all rows
I don't know of a way to do exactly what you want, but if you are open to using #Query in your JPA repository class, then a prepared statement is one alternative:
#Query("SELECT * FROM Entity e ORDER BY e.id LIMIT :limit", nativeQuery=true)
Entity getEntitiesByLimit(#Param("limit") int limit);
Did it by using pagination, as described in the first answer. Just adding a more explicit example.
This example will give you the first 50 records ordered by id.
Repository:
#Repository
public interface MyRepository extends JpaRepository<MyEntity, String> {
Page<MyEntity> findAll(Pageable pageable);
}
Service:
#Service
public class MyDataService {
#Autowired
MyRepository myRepository;
private static final int LIMIT = 50;
public Optional<List<MyEntity>> getAllLimited() {
Page<MyEntity> page = myRepository.findAll(PageRequest.of(0, LIMIT, Sort.by(Sort.Order.asc("id"))));
return Optional.of(page.getContent());
}
}
Found the original idea here:
https://itqna.net/questions/16074/spring-data-jpa-does-not-recognize-sql-limit-command
(which will also link to another SO question btw)
I am working with Spring Data 2.0.6.RELEASE.
I am working about pagination for performance and presentation purposes.
Here about performance I am talking about that if we have a lot of records is better show them through pages
I have the following and works fine:
interface PersonaDataJpaCrudRepository extends PagingAndSortingRepository<Persona, String> {
}
The #Controller works fine with:
#GetMapping(produces=MediaType.TEXT_HTML_VALUE)
public String findAll(Pageable pageable, Model model){
Through Thymeleaf I am able to apply pagination. Therefore until here the goal has been accomplished.
Note: The Persona class is annotated with JPA (#Entity, Id, etc)
Now I am concerned about the following: even when pagination works in Spring Data about the amount the records, what about of the content of each record?.
I mean: let's assume that Persona class contains 20 fields (consider any entity you want for your app), thus for a view based in html where a report only uses 4 fields (id, firstname, lastname, date), thus we have 16 unnecessary fields for each entity in memory
I have tried the following:
interface PersonaDataJpaCrudRepository extends PagingAndSortingRepository<Persona, String> {
#Query("SELECT p.id, id.nombre, id.apellido, id.fecha FROM Persona p")
#Override
Page<Persona> findAll(Pageable pageable);
}
If I do a simple print in the #Controller it fails about the following:
java.lang.ClassCastException:
[Ljava.lang.Object; cannot be cast to com.manuel.jordan.domain.Persona
If I avoid that the view fails with:
Caused by:
org.springframework.expression.spel.SpelEvaluationException:
EL1008E:
Property or field 'id' cannot be found on object of type
'java.lang.Object[]' - maybe not public or not valid?
I have read many posts in SO such as:
java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to
I understand the answer and I am agree about the Object[] return type because I am working with specific set of fields.
Is mandatory work with the complete set of fields for each entity? Should I simply accept the cost of memory about the 16 fields in this case that never are used? It for each record retrieved?
Is there a solution to work around with a specific set of fields or Object[] with the current API of Spring Data?
Have a look at Spring data Projections. For example, interface-based projections may be used to expose certain attributes through specific getter methods.
Interface:
interface PersonaSubset {
long getId();
String getNombre();
String getApellido();
String getFecha();
}
Repository method:
Page<PersonaSubset> findAll(Pageable pageable);
If you only want to read a specific set of columns you don't need to fetch the whole entity. Create a class containing requested columns - for example:
public class PersonBasicData {
private String firstName;
private String lastName;
public PersonBasicData(String firstName, String lastName) {
this.firstName = fistName;
this.lastName = lastName;
}
// getters and setters if needed
}
Then you can specify query using #Query annotation on repository method using constructor expression like this:
#Query("SELECT NEW some.package.PersonBasicData(p.firstName, p.lastName) FROM Person AS p")
You could also use Criteria API to get it done programatically:
CriteriaBuilder cb = entityManager.getCriteriaBuilder();
CriteriaQuery<PersonBasicData> query = cb.createQuery(PersonBasicData.class);
Root<Person> person = query.from(Person.class);
query.multiselect(person.get("firstName"), person.get("lastName"));
List<PersonBasicData> results = entityManager.createQuery(query).getResultList();
Be aware that instance of PersonBasicData being created just for read purposes - you won't be able to make changes to it and persist those back in your database as the class is not marked as entity and thus your JPA provider will not work with it.
Is it possible to use Spring Data Rest and Spring Security to return current user related entities, using the findAll() method without specifying this user in the GET query parameter?
My only solution is to pass user as a parameter, but maybe it's another option to get him from SpringSecurityContext
public interface InvoiceRepository extends CrudRepository<Invoice, Long> {
#RestResource
#PreAuthorize("hasRole('ROLE_ADMIN') or user?.username == authentication.name")
List<Invoice> findAllByUser(#Param("user") User user);
You can use SpEL EvaluationContext extension that makes security properties and expressions available in SpEL expressions in the #Query annotations. This allows you to get only those business objects that relate to the current user:
interface SecureBusinessObjectRepository extends Repository<BusinessObject, Long> {
#Query("select o from BusinessObject o where o.owner.emailAddress like ?#{hasRole('ROLE_ADMIN') ? '%' : principal.emailAddress}")
List<BusinessObject> findBusinessObjectsForCurrentUser();
}
More details are here.
Say I have a #MappedSuperClass like this:
#MappedSuperclass
public abstract class Rating
{
#Id
private Long id;
#Column(name="USER_ID")
private Long userId;
private int rating;
...
With a concrete child entity like this
#Entity
#Table(name="ACTIVITY_RATING")
public class ActivityRating extends Rating
{
private Long activitySpecificData;
...
Then there is a Spring Data JPA repository like this:
#NoRepositoryBean
public interface RatingRepository<R extends Rating> extends JpaRepository<R, ID>
{
public List<R> findByUserId(Long userId);
...
and this:
public interface ActivityRatingRepository extends RatingRepository<ActivityRating>
{
}
This all works great and I can call findByUserId() on any of specific rating repositories that extend RatingRepository. I am now wanting to write some JPQL in the RatingRepository that all the child interfaces can inherit. I just don't know what (or if it's even possible) to put after the FROM in the query. For example:
#Query("SELECT NEW com.foo.RatingCountVo(e.rating, COUNT(e.rating)) FROM ??????? e GROUP BY e.rating")
public List<RatingCountVo> getRatingCounts();
I can add this method to each of the individual repositories that extend RatingRepository but everything would be exactly the same except for the specific entity name. If I want to change the query, I'd then have to go to all the child repositories and update them individually. I really want the query to live in the parent class and not be duplicated. Is there any way to accomplish this?
I'm currently using spring-data-jpa 1.7.2 and eclipselink 2.5.2. I'm not necessarily opposed to switching to newer versions if necessary.
Will it work if you will split query into 3 parts: start, entity and end of query? Than, if it'll work, in each interface you define constant like
String ENTITY = "ActivityRating";
And then you can use it like
#Query(RatingRepository.QUERY_START + ENTITY + RatingRepository.QUERY_END)
List<RatingCountVo> getRatingCounts();
BTW, there is no need to define public modifier in interface.
UPDATE: here is described another way:
#Query("SELECT NEW com.foo.RatingCountVo(e.rating, COUNT(e.rating)) FROM #{#entityName} e GROUP BY e.rating