Search only shows up to Page 50

[A N]

When you browse annotations, any page of search results after 50 just shows the 50th page.

This is an issue I raised previously (a long time ago) and was able to be replicated albeit in a ticket.

[Robert Knight]

Hello,

This is unfortunately a known issue. It arises from a constraint that the underlying search engine only allows the first 10,000 results to be fetched from a given start point. Each page shows 200 annotations, 200 * 50 = 10,000. There are some ways we can work around this, but it hasn't been a high priority in recent times.

Kind regards,

Robert.

(For developers reading this, see references to `OFFSET_MAX` in the Hypothesis server code)

On 4 Mar 2024, at 01:11, A N <correspo...@gmail.com> wrote:

When you browse annotations, any page of search results after 50 just shows the 50th page.

This is an issue I raised previously (a long time ago) and was able to be replicated albeit in a ticket.

--
You received this message because you are subscribed to the Google Groups "Hypothes.is Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hypothesis-for...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hypothesis-forum/a29aa259-5ab2-4bbb-8a4e-35c4a56b5836n%40googlegroups.com.

[AA]

Thank you. The phrase "given start point" makes me wonder if you could define the start point differently, but I assume you have already considered this.

 I wonder whether date-based search could be a possibility? I feel like searching your annotations by date should be a basic function. This would be even better because you could combine this filter with search queries.

Also I apologize if I accidentally sent this multiple times.

[Robert Knight]

Thank you. The phrase "given start point" makes me wonder if you could define the start point differently, but I assume you have already considered this.

 I wonder whether date-based search could be a possibility? I feel like searching your annotations by date should be a basic function. This would be even better because you could combine this filter with search queries.

It is not currently possible to specify date constraints in the search UI, but it would indeed make sense to have. The Hypothesis API does already have a `search_after` parameter which allows specifying the start point for results. As a result it doesn't have the limits of the web UI - you can iterate through all the annotations in a group even if there are millions.

In database querying parlance, the web UI at https://hypothes.is/search uses offset-based pagination (specify page index) whereas the API supports cursor-based pagination (specify start point). We could resolve the 50-page limit in Hypothesis in future by using a hybrid of the two (you can jump forwards only a certain number of pages at a time, but you can do that repeatedly until you get all the way to the end).

Regards,

Robert.

To view this discussion visit https://groups.google.com/d/msgid/hypothesis-forum/802e72f2-f9e8-4e1b-831d-7a56b9844fb2n%40googlegroups.com.