Pagination - Backend Implementation Pattern

[benle...@gmail.com]

Hi everyone,

 

Josh recently added a description of pagination to the style guide. There are a few ways in which it might be implemented throughout our services.

 

The simplest scenario is one in which a controller seeks to expose data provided by an auto-generated Spring Data repository. An example of this is the GET /geographicZones endpoint in the reference-data service. As code in the master branch shows, the getAllGeographicZones method has been updated such that:

 

1. Users may optionally pass in a Pageable. This object defines the pagination related page (as in page-number) and size (as in page-size) values. Note that page is zero based.

 

2. The code passes the Pageable to the repository’s findAll method. This method is automatically generated when the PagingAndSortingRepository interface is used.

 

3. The getAllGeographicZones returns an instance of ResponseEntity<Page<GeographicZone>>.

 

Essentially, in this case, Spring Data gives us everything for free. A more involved case is one in which the controller doesn’t rely on an auto-generated Spring Data repository. In this scenario, it’s up to the controller to implement pagination itself. The GET /requisitions/search endpoint in the master branch of the requisition service shows how this can be done. Specifically:

 

1. The relevant method (searchRequisitions in this case) is updated such that users may optionally pass in a Pageable.

 

2. The pageable is passed on to requisitionService.searchRequisitions, which is defined in RequisitionRepositoryImpl. This custom repository is responsible for returning a Page<Requisition>, and does so by:

 

2a) Retrieving the relevant data from the database via the criteria API. Note that setFirstResult() and setMaxResults() are used to appropriately limit the number of results.

 

2b) Retrieving the total count of all possible results

 

2c) Calling Pagination.getPage to combine the data retrieved in steps 2a and 2b into a Page returned to the client.

 

I’ve tried to comment this code thoroughly, but hope folks won’t hesitate to ask if they have any questions.

 

The final step to making an endpoint pageable is to update its RAML. The best example for doing so is currently the GET /geographicZones endpoint already described. Its RAML:

 

1. Defines a trait called “paginated” and associates it with the GET method of /geographicZones.

 

2. Creates a schema called geographicZonePage, defined in geographicZonePage.json, and sets it as a possible response body for /geographicZones.

In terms of the UI: none of these changes require that endpoints be accessed in a different way. The page and size query parameters they define are optional. If omitted, all results are returned to the client just as before. However, in all cases, the results are returned within an array called "content." (Please see the styleguide for details). The UI must therefore be updated to handle this new response format.

 

Again, please let me know if you have any questions. The 3.0 release is fast approaching, and I hope these patterns will prove quick and easy to adopt throughout our services.

 

Thank you,

Ben

[Weronika Ciecierska]

Hi Ben!
I am currently working on OLMIS-1778 which is about implementing our pagination pattern to the /requisitionsForConvert endpoint.
Your solution works perfectly for standard endpoints, but for this endpoint it seems to be not possible to use Criteria API to limit number of results.
After retrieving requisitions to convert they are additionally filtered (it was done in controller, I moved this logic to the service) to get only requisitions where at least one of supplying depots is user managed facility (basing on info retrieved from reference-data).
It creates situation where we may get wrong content & page meta-data. To make it work I implemented pagination logic in the RequisitionService instead (it is still in review so it may be not the final version).
I just wanted to let you know, that this pattern isn't working for all of our cases (maybe it is worth noting somewhere in readme?).

Regards,
Weronika

Weronika Ciecierska
Software Developer
wciec...@soldevelo.com

SolDevelo Sp. z o. o. [LLC]
Office: +48 58 782 45 40 / Fax: +48 58 782 45 41 Al. Zwycięstwa 96/98 81-451, Gdynia
http://www.soldevelo.com

Place of registration: Regional Court for the City of Gdansk KRS: 0000332728, TAX ID: PL5862240331, REGON: 220828585, Share capital: 60,000.00 PLN

[Darius Jazayeri]

I don't know if this is relevant here or not, but in OpenMRS we took the approach that where our API allowed it, we'd delegate paging to our API queries. But in some cases our Java API didn't support this, and it was low-value to add it (either it was too complex, or it was an infrequently-used method, or just one that would rarely return lots of results).

We implemented a shared utility method for those cases that takes a non-paged data + a request context (with start index and page size), and generates the page for you (i.e. it converts to a map that Jackson will properly convert when sending back to the client).

Executive summary: consider having a utility method to handle unpaged query results, where it's too low value to do proper paging in the DB/API.

-Darius

--
You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.
To post to this group, send email to openlm...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/ba9e285f-23d4-48a4-985a-a4c7e0bf274e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Darius JazayeriPrincipal Architect - Global Health

Email

djaz...@thoughtworks.com

Telephone

+1 617 383 9369

[benle...@gmail.com]

Hi Weronika!

Thank you very much for your note. I like the way you moved the filtering logic from the controller into the service, and think it’s good work. Following up on Darius’ note, though, I think that the static methods in the Pagination utility class can be used more than they have been. Specifically, it seems like the math toward the end of searchApprovedRequisitionsWithSortAndFilterAndPaging which is used to create and return a subList may not be necessary. Can the Pagination.getPage(List<T> originalList, Pageable pageable) method be used instead? (You'd pass it the responseList.) If not, the Pagination class should be revisited. My hope, though, is that it can simplify scenarios like this as-is.

Kind regards,

Ben

[Weronika Ciecierska]

Hi,
Thank you for your feedback!
You are right Ben, getPage method from Pagination utility class works perfectly in this case.
I am not sure why, but I missed it. Thank you for your help.

Regards,
Weronika

Weronika Ciecierska
Software Developer
wciec...@soldevelo.com

SolDevelo Sp. z o. o. [LLC]
Office: +48 58 782 45 40 / Fax: +48 58 782 45 41 Al. Zwycięstwa 96/98 81-451, Gdynia
http://www.soldevelo.com

Place of registration: Regional Court for the City of Gdansk KRS: 0000332728, TAX ID: PL5862240331, REGON: 220828585, Share capital: 60,000.00 PLN

[benle...@gmail.com]

Hi Weronika,

Thank you so much for letting me know - I’m glad that it works well for you.

Kind regards,

Ben