Hibernate Search: Indexing only a few entites of a huge table without loading all entries - hibernate-search

Question

I have a huge table which I write into a Lucene Index using Hibernate Search Version 5.11.5. After a (existing) raw SQL import into this table I have to manually update the search index. This SQL import runs multiple times a day and reindexing should not block incoming search requests for more than a few seconds.
On each entity there exists a date field "modified" so I already annotated this #Indexed Entity with an EntityIndexingInterceptor like following:
public class CustomEntityIndexingInterceptor implements EntityIndexingInterceptor<HugeTableEntity> {
public static Date lastModified = //some logic;
#Override
public IndexingOverride onAdd(HugeTableEntity entity) {
return IndexingOverride.APPLY_DEFAULT;
}
#Override
public IndexingOverride onUpdate(HugeTableEntity entity) {
if (entity.getModified().after(lastModified)) {
return IndexingOverride.APPLY_DEFAULT;
}
return IndexingOverride.SKIP;
}
}
This code works, but reindexing takes a lot of time because all entites get loaded. Only < 0.001% match the condition of the modified date.
I saw that deep inside of Hibernate Search there exists a class IdentifierProducer, which loads all IDs in the method loadAllIdentifiers. I would love to add a SQL filter to the inner criteria - something like "where modified > :given_date".
Do you know if I can customize the IdentifierProducer without copying all the code? Do you know another smart way to solve my problem?
Regards

Answer 1

The MassIndexer cannot be used to index just part of your data... yet. IdentifierProducer is an internal class an you should not try to alter it.
What you can do is run a query to list the entities that were affected by your import, and ask Hibernate Search to reindex them, for example by batches of 20 elements. You can take some inspiration from this example from the documentation. In your case, of course, you will add filters to the Hibernate ORM query, to only go through entities that actually changed because of your import.
Don't forget to remove your indexing interceptor, which should no longer be useful.