Paging the $everything Operation

61 views
Skip to first unread message

Michael Litcher

unread,
Nov 23, 2023, 11:59:25 AM11/23/23
to HAPI FHIR
Is there a reason "offset" paging is not allowed on the /Patient/{id}/$everything operation using the JPA starter? Am I able to override this?

i.e. I want to be able to do this:

/Patient/{id}/$everything?_count=100&_offset=100

Anton Kurylovich

unread,
Nov 24, 2023, 9:37:01 AM11/24/23
to HAPI FHIR
Hey there, just to clarify, what do you mean by 'not allowed' are you seeing an error or something? Or everything/nothing being returned regardless of the query params? Because I see the query param available in the swagger of the test server: https://hapi.fhir.org/baseR4/swagger-ui/?page=Patient#/Patient/post_Patient__id___everything so it should be allowed I think.

Does it work for other searches you are performing? I wonder if you need to set a paging provider? https://hapifhir.io/hapi-fhir/docs/server_plain/paging.html 

Do you have the same in your JPA starter?

Unless I am missing something special about the $everything operator of course, but if it returns a bundle, I imagine paging should work?

Michael Litcher

unread,
Nov 24, 2023, 10:08:28 AM11/24/23
to HAPI FHIR
I'm seeing a "not supported" response when you include the _offset parameter in the $everything operation. All other paging works as expected.

You can see example here: https://hapi.fhir.org/baseR4/Patient/123/$everything?_count=10&_offset=10


Some context, I'm trying to use "offset" paging in my implementation exclusively, because the database platform I'm trying to use does a really good job of caching, so I wanted to try avoiding the database writes for searches.


Anton Kurylovich

unread,
Nov 24, 2023, 10:36:32 AM11/24/23
to HAPI FHIR
Interesting, thanks for the additional info. It looks like this is issue where HAPI started to reject the param in the everything operation: https://github.com/hapifhir/hapi-fhir/pull/2579, so if we wanted to try to re-introduce it, that's what we would have to look out for. So I guess the answer of the original question of why it is unsupported can be traced to this comment: https://github.com/hapifhir/hapi-fhir/pull/2579#discussion_r622896945 

Michael Litcher

unread,
Nov 24, 2023, 12:39:20 PM11/24/23
to HAPI FHIR
Hmm, ok. I'm not trying to support "type/$everything", just "instance/$everything". Do you think there is any reasonable way to provide an override to this while still using the JPA server?

Anton Kurylovich

unread,
Nov 24, 2023, 1:08:45 PM11/24/23
to HAPI FHIR
I don't think so. Since the error is thrown pretty deep in the codebase, at the DAO level, I doubt we have many options in terms of overriding. For example none of the interceptor pointcuts can go that deep: https://hapifhir.io/hapi-fhir/docs/interceptors/server_pointcuts.html 

Only ideas I have are: 
1) Branch HAPI, and try to make a change to not throw an exception and point your starter project dependency to your version of HAPI (just for testing purpose, and can submit a pull request if it works) 
2) If you want to get real hacky you can create your own dao class in your project and make sure it has the exact same name + path as the existing one so that maven should use your DAO when it compiles the project. This is a super iffy and hacky approach, I'm not sure it would even work

Hope that made some sense

Michael Litcher

unread,
Nov 24, 2023, 8:10:58 PM11/24/23
to HAPI FHIR
That makes sense. Number 2 does work, but like you said, not ideal. However, for my use case, might be ok for a tactical approach while I look into number 1.

It was mentioned in the github PR comments, but the Patient is repeated on each page. I think I need to test offset paging a bit more in general. When using includes/revincludes, the synchronous search operates a little different. What I would have expected is: the search generates a list of resource ids, including whatever is brought in by the include, and then paginates that entire set. But, when includes/revincludes are involved, it loads all those at once, or up to the defined max include per page, and even though a "next" link might be shown, doesn't actually paginate the whole result set.

Michael Litcher

unread,
Nov 26, 2023, 9:12:10 PM11/26/23
to HAPI FHIR
I re-read the spec on the _count parameter, and I understand now that it doesn't include the resources brought in by _include/_revinclude. The separate max_includes_per_page setting makes sense now.

I'm still looking into what other issues are present (if any) with offset paging on the $everything operation.

Anton Kurylovich

unread,
Nov 27, 2023, 10:27:31 AM11/27/23
to HAPI FHIR
Good to know!
Reply all
Reply to author
Forward
0 new messages