Hibernate Search 6 Scrollable returning results in a strange way - hibernate-search

Since we upgraded from Hibernate Search 5.11 to Hibernate Search 6 we are having problems with Scrollable results
When we get a chunk of hits from the SearchScroll object each hit is stored in an Arrays.ArrayList
What we expected is that each chunk hits would be an ArrayList of say for example of type long
What we get is an ArrayList where where each hit is an Arrays.ArrayList with the Long value
Current code
SearchScroll scroll = searchSession
.search(scope)
.select(projectionArray)
.where(searchPredicate)
.sort(getSort(resultType))
.scroll(20);
Old code with Hibernate Search 5
FullTextQuery fullTextQuery = fullTextSession
.createFullTextQuery(query, resultType)
.setSort(getSort(resultType));
fullTextQuery.setProjection(fields);
ScrollableResults scrollableResults = fullTextQuery.scroll();
Any suggestions welcome
At worst we can loop through the results and convert the Arrays.ArrayList item to a long but cannot find a way to make that work either
The acual search results are correct just coming back in a different format that what we expect
Changing the code to
SearchScroll<Long> scroll = searchSession
.search(scope)
.select(projectionArray)
.where(searchPredicate)
.sort(getSort(resultType))
.scroll(20);
Makes no difference which seems to match the example in the docs
try ( SearchScroll<Book> scroll = searchSession.search(
Book.class )
.where( f -> f.matchAll() )
.scroll( 20 ) ) {
for ( SearchScrollResult<Book> chunk = scroll.next();
chunk.hasHits(); chunk = scroll.next() ) {
for ( Book hit : chunk.hits() ) {
// ... do something with the hits ...
}
totalHitCount = chunk.total().hitCount();
entityManager.flush();
entityManager.clear();
}
}
Not sure if the projection is what is causing the problem
Tested further if I remove the projection I get the results as an ArrayList of the object as expected so obviously I am doing something wrong with the use of projections in Hibernate Search 6
Without projection everything is good
With projection the results are Arrays.ArrayList

If I understand correctly, you are surprised that you get a List for each hit instead of just a Long.
First, I would recommend that you don't ignore raw type warnings.
I'll wager that your projectionArray is defined this way:
SearchProjection[] projectionArray = new SearchProjection[1];
That's wrong because you're using a "raw" type for SearchProjection, which basically disables all kinds of type-checking for all the code afterwards.
The correct way of defining that array is as follows:
SearchProjection<?>[] projectionArray = new SearchProjection<?>[1];
If you do that, then you'll get a compile-time error with the following code, telling you something like "cannot convert SearchScroll<List<?>> to SearchScroll<Long>":
SearchScroll<Long> scroll = searchSession
.search(scope)
.select(projectionArray)
.where(searchPredicate)
.sort(getSort(resultType))
.scroll(20);
Now, the reason you're getting a SearchScroll<List<?>> is you're passing an array of projections to .select(), so you're calling this method from SearchQuerySelectStep:
SearchQueryWhereStep<?, List<?>, LOS, ?> select(SearchProjection<?>... projections);
This method takes an array of projections as an argument, and (ultimately) returns a query whose hits are lists, with the results of requested projections in the same order as your array of projections.
You want to call that method instead:
<P> SearchQueryWhereStep<?, P, LOS, ?> select(SearchProjection<P> projection);
That method takes a single projection as an argument, and (ultimately) returns a query whose hits are directly the result of the requested projection.
To call that method, pass a single projection instead of an array of projections; then you will get the Long values you expect instead of Lists:
SearchProjection<Long> projection = ...;
SearchScroll<Long> scroll = searchSession
.search(scope)
.select(projection)
.where(searchPredicate)
.sort(getSort(resultType))
.scroll(20);

Related

Eloquent - load accessor along with model data

My model was eager loading a lot of things with accessors. I want to change it to specify accessors in each case. How do I include such accessors with the query, so that I get the basic model data plus the accessor data
Accessors would previously be eager loaded with:
protected $appends = [
'status',
]
But if I get rid of eager loading, and I want to include this acccessor:
public function getStatusAttribute() {
return self::STATUS_ACTIVE;
}
Then I can do this according to the documentation:
$prod = \App\Product::find(736)->status;
That works but I don't get the basic model data.
I can't do: return $prod = \App\Product::find(736)->with('status')->first()
It gives error: Call to undefined relationship [status] on model
So how do I add such accessors to be included with the model data?
Edit:
As Staudenmeir commented, i can do \App\Product::find(736)->append('status');
That solves it for single results. But how do I append data for many results?
Neither append or appends work:
This: \App\Product::whereIn([34, 55])->appends('status');
results in "Method appends does not exist.",
I saw that you can use "appends" on "->paginate()"
$products = \App\Product::whereIn([34, 55])
->paginate(12)
->appends('status');
But that appends it as a query string to the url. Very strange - I want to append it in the same way as for a single result in the json response.
You have not fetched the collection first, and then access the attribute of result.
$prod = \App\Product::find(736);
$status = $prod->status;

How to debug in TYPO3 if <f:debug> returns strings instead of object?

In a custom TYPO3 8.7.12 extbase extension I am unable to f:debug items in templates.
We are in the listAction controller and simply do:
$institutions = $this->institutionRepository->findAll();
$this->view->assignMultiple([
'institutions' => $institutions,
// ... pagination limit ...
]
);
And in the template:
<f:debug>
{institutions}
</f:debug>
This returns
sometimes the string 'Array' in the fluid debugger (can't reproduce now)
When the code is on 3 lines: #1273753083: Cannot cast object of type "TYPO3\CMS\Extbase\Persistence\Generic\QueryResult" to string.
Or also #1273753083: Cannot cast object of type "TYPO3\CMS\Extbase\Persistence\Generic\LazyObjectStorage" to string.
When the code is on 1 line: #1234386924: Cannot create empty instance of the class "TYPO3\CMS\Extbase\Persistence\ObjectStorage" because it does not implement the TYPO3\CMS\Extbase\DomainObject\DomainObjectInterface.
If I loop through {institutions} with f:for and then f:debug:
<f:for each="{institutions}" as="institution" iteration="i">
<f:debug>
{institution}
</f:debug>
</f:for>
I get the first property of the object, e.g. the name.
EDIT: this is due to a __toString() magic method in the model. If I remove it, instead I get the namespace and uid STUBR\Extension\Domain\Model\Institution:55 – this looks again as if the object isn't rendered.
Wait... php.net says The __toString() method allows a class to decide how it will react when it is treated like a string. So could something be treating (typecasting?) the object as a string?
Working with the properties is normal, the issue just occurs when trying to print the whole object.
Where should I look? Lazy loading? There are some lazy loading properties, but not that many. Or maybe something is missing from the class? Or is there a workaround.
PS:
Unable to print_r or var_dump the query result, I get a memory limit error.
I saw https://wiki.typo3.org/Exception/CMS/1234386924 but initStorageObjects() is already called in the constructor
To answer the question;
<f:debug>{institutions}</f:debug>
will be parsed as an object, but any whitespace inside will make it parse as a string so.
The following methods do the same job as <f:debug> and work similarly in my case:
\TYPO3\CMS\Core\Utility\DebugUtility::debug(
$var = $variable,
$header = 'Institutions',
$group = ''
);
\TYPO3\CMS\Extbase\Utility\DebuggerUtility::var_dump(
$variable,
$title = 'Institutions',
$maxDepth = 8,
$plainText = FALSE,
$ansiColors = TRUE,
$return = FALSE,
$blacklistedClassNames = NULL,
$blacklistedPropertyNames = NULL
);
execute in list or show action in controller.
It's less convenient than with f:debug (because you have to do the work in two different places, e.g. when you're in a loop in the template, you have to go to the controller and build that loop again), but it's a helpful workaround.
EDIT: I found it's sufficient to do
<f:debug>{var}</f:debug>
on one line

How to map over an array, use an if clause, and filter out the bad data?

I have an array of words, some may or may not have typos.
potentialWords = ["hello", "lkasjdf", "hunry"]
What I want to do is, return an array of all valid words, and also those words that were able to be autocorrected using a function I created correctWord. It returns an array of potential matches. so "hunry" might return ["hungry", "hurry"]. I will select the first index for the best guess.
Some words cannot be corrected however! e.g. "lkasjdf" will not find any corrections, but "hunry" will.
I was trying something like:
potentialWords.map {
if correctWord($0) != nil {
return correctWord($0)[0]
}
}
of course this will complain and say that I need a return outside the if clause. I can filter the list based on if the word can be corrected, and then map over the filtered list, re-checking which words need to be corrected, but this runs the correctWord function way too many times, and it is very sensitive.
I would like to be able to do one single pass through, and return an array of all valid words, and also corrected words.
P.S. I am calling correctWord twice in the map function for brevity, but of course I would assign correctWord($0) to a variable, and then if it isn't nil, take the first index and add it to the new list.
I think you're after flatMap. It's the same as map except it will also filter out any nil values.
potentialWords.flatMap { correctWord($0)?.first }

Groovy sql.rows returns org.postgresql.util.PSQLException: No hstore extension installed

I am using Groovy Sql in Grails with named parameters to get results from a Postgres DB. My statement is generated dynamically, i.e. concatenated to become the final statement, with the params being added to a map as I go along.
sqlWhere += " AND bar = :namedParam1"
paramsMap.namedParam1 = "blah"
For readability, I am using the groovy string syntax which allows me to write my sql statement over multiple lines, like this:
sql = """
SELECT *
FROM foo
WHERE 1=1
${sqlWhere}
"""
The expression is evaluated as a string containing the linebreaks as \n:
SELECT *\n ...
This is not a problem when I pass params like this
results = sql.rows(sqlString, paramsMap)
but it does become one if paramsMap is empty (which happens since AND bar = :namedParam1 is not always concatenated into the query). I then get an error
org.postgresql.util.PSQLException: No hstore extension installed
which does not really seem to relate to the true nature of the problem. I have for now fixed this with an if...else
if (sqlQuery.params.size() > 0) {
results = sql.rows(sqlString, paramsMap)
} else {
results = sql.rows(sqlString.replace('\n',' '))
}
But this seems a bit weird (especially since it does not work if I use the replace in the if-branch as well).
My question is: why do I really get this error message and is there a better way to prevent it from occuring?
It's certainly a bug in groovy.sql.SQL implementation. The method rows() can't deal with an empty map passed as params. As a workaround, you can test for it and pass an empty list instead.
def paramsMap = [:]
...
if (paramsMap.isEmpty())
paramsMap= []
Issue created at https://issues.apache.org/jira/browse/GROOVY-8082

How to bind parameters in replaced expression nodes in Entity Framework on the fly

I'm trying to replace a function call like (simplified) Utility.GetString(MyEntity.SomePropertyWithRelatedEntity)=="abc" with an expression visitor into something like p => p.SubRelatedEntities.FirstOrDefault(sre => sre.SomeFlag==true).SomePropertyWithRelatedEntity.
It means, the datamodel goes like:
MyEntity -> RelatedEntity -> SubRelatedEntity
I'm trying to return a string value from the SubRelatedEntity, based on some rules in the RelatedEntity, so I don't have to re-write / copy/paste the whole filtering rules in every usage; that's why I put inside a "call-signature", so my expression visitor can identify it and replace the fake-call to Utility.GetString to some complicated lambda expressions.
My expression visitor contains something like:
public override Expression Visit(Expression node)
{
if (node == null)
return null;
Expression result = null;
if (node.NodeType == ExpressionType.Call)
{
MethodCallExpression mce = node as MethodCallExpression;
if (mce.Method.DeclaringType == typeof(Utility) && mce.Method.Name == "GetString")
{
Expression<Func<RelatedEntity, string>> exp = re => re.SubRelatedEntities.FirstOrDefault(sre => sre.SomeFlag == true).SomeStringValue;
result = exp.Body;
}
else
result = base.Visit(node);
}
else
result = base.Visit(node);
return result;
}
Now, the problem is, the "sre" parameter is not bound when called the injected lambda expression. After much research, I see the lambda parameters should be replaced with another expression visitor, specifically searching for the new parameters and replacing them with the old ones. In my situation, however, I don't have an "old parameter" - I have the expression MyEntity.SomePropertyWithRelatedEntity (e.g. an property filled with the related entities) which I need to insert somehow in the generated lambda.
I hope my problem is understandable. Thank you for any insights!
After getting no answers for long time and trying hard to find a solution, I've solved it at the end :o)! It goes like this:
The newly injected lambda expression gets an ParameterExpression - well, this is a 'helper', used when directly calling the lambda, what I don't want (hence, 'parameter not bound' exception when ToEnumerable is called). So, the clue is to make a specialized ExpressionVisitor, which replaces this helper with the original expression, which is of course available in the Arguments[] for the method call, which I try to replace.
Works like a charm, like this you can reuse the same LINQ expressions, something like reusable sub-queries, instead of writing all the same LINQ stuff all time. Notice as well, that expression calling a method is not allowed in EF, in Linq2Sql it worked. Also, all the proposed web articles only replace the parameter instances, when constructing/merging more LINQ expressions together - here, I needed to replace a parameter with an faked-method-call argument, e.g. the method should not be called, it only stands for a code-marker, where I need to put my LINQ sub-query.
Hope this helps somebody, at the end it's pretty simple and logical, when one knows how the expression trees are constructed ;-).
Bye,
Andrej