Need advice on hierarchical data pagination - perl

Question

I have pretty simple database structure which, with the exception for some self-referencing and intermediate relations, reduces to products-to-category conception.
Final data structure i retrieve using ResultClass::HashRefInflator after some conversions looks like that:
my $data = $self->db->resultset('Category')->with_translation($lang)->with_categories->with_products->display_flattened;
[
[0] {
parent_name "Parent Category",
id 3,
name "First Child category",
parent_id 1,
position 1,
products [
[0] {
name "Product One",
},
...
],
}
...
]
Things were going well until, in attempt to reduce initial products index page size, i decided to implement infinite scroll feature there. So in general, it's a pagination issue. The thing i am having hard times with is the fact that i can apply paging only on Category or Product resultset, not on a whole hierarchy to retrieve piece of data i want for next screen.
For instance, if i want 20 items per screen (item can either be a Category or Product) and i apply ->page(1) on Schema::ResultSet::Category, it will contain 20 categories with all products in them instead of 1st category with 19 related products and so on.
The only option which comes to my mind at the moment is storing the whole data structure as a single-dimensional array in some kind of in-memory storage like Redis or memcached and slice it as intended but i know it's wrong.

Answer 1

I don't totally know what you want here, because it sounds like you want the
category to count as part of the pagination; seems like an odd choice to me. If
you can avoid counting the category, then a better option might be to instead of
getting the first page of categories, get the first page of Products and their
related categories. That way you have a bound set of data instead of the
possible explosion you listed above.
You'd need to make some changes (by at least adding methods) to your resultsets,
but that shouldn't be too hard. Here's what you could do instead:
my $data = $self->db->resultset('Product')
->with_translation($lang)
->with_categories
->with_products
->search(undef, { page => 1 })
->display_flattened;
So given this, you'll get a page of products (I think that's 25) and then their
related categories.
As a side note, it's a little odd that you do with_categories on a category
resultset. I might be misunderstanding, but at the very least that's a
confusing thing to do.

Answer 2

Did you try:
my $data = $self->db->resultset('Category')->with_translation($lang)->with_categories->with_products->search_rs(undef, { page => 1 })->display_flattened;