Pagination Question

[ShovelAppDev]

So I'm working on a better method of paginating results...

Right now I do the obvious thing,

"get stories with a count of x with offset of [(pageNumber-1)*x]"

so, 10@0, 10@10, 10@20 and so forth.

This works just fine so long as your underlying data-set is static...
which in the case of digg couldn't be farther from the truth! Pulling
up subsequent data-sets often results in repeated stories, which have
been pushed down the index by newly promoted stories.

So I began investigating the use of max_promoted_date instead. Simply
grab 10 stories, then pull the promoted_date value from the last one,
subtract 1 and use it as the max_promoted_date in the next pull.

This led to several questions that I just wanted to run by you really
quick before moving on.

First... it seems that the timestamp value used here (being treated
as an integer) is being rounded to the nearest ten. When I decrement
the last value by 1, often the resulting data will still include that
story, despite being newer than the max_promote_date. Through trial
and error I found that this will be the case until I reach the nearest
10 value. (So if I start with 2345634... i would have to reduce this
to 2345630 before that story wouldn't show up in the results
anymore). Is this the case?

Second... the max_date argument doesn't seem to work for stories, but
it would be nice if there were some standard argument for stories,
comments, etc so that standard paged-list elements can be created
without having to account for the many "date" arguments that might be
needed to make it work.

Thanks!

-Scott

[Jeff Hodsdon]

> First... it seems that the timestamp value used here (being treated
> as an integer) is being rounded to the nearest ten. When I decrement
> the last value by 1, often the resulting data will still include that
> story, despite being newer than the max_promote_date. Through trial
> and error I found that this will be the case until I reach the nearest
> 10 value. (So if I start with 2345634... i would have to reduce this
> to 2345630 before that story wouldn't show up in the results
> anymore). Is this the case?

Yes this is true, we round the dates for caching reasons. Therefore you will need to subtract 5 rather than 1.

> Second... the max_date argument doesn't seem to work for stories, but
> it would be nice if there were some standard argument for stories,
> comments, etc so that standard paged-list elements can be created
> without having to account for the many "date" arguments that might be
> needed to make it work.

We only do this to be more explicit about submit dates and promote dates. Are you able in the application to change min_date to min_submit_date for stories?

-jeff

[ShovelAppDev]

Thanks for the clarification on the timestamps. Works just like it
should.

As far as the submit and promote dates... There's nothing wrong with
the distinction, and the app can be coded to handle this whichever way
it needs to, but in terms of ease of use and code abstraction, it
would be helpful if there were a "max_date" "min_date" arguments as
well as a "date" value that were standard across all collections of
data (such as stories and comments and..... )

Alright, well I got through my shpeal and realized those are the only
two. All other collections aren't organized by date at all but rather
alphabetically.

As you were! And thanks ;)

Scott