any explicit rate limiting i should be mindful of?

[Jason Wang]

Hey guys

I am planning on grabbing some data from the api, and i don't want to get banned. From my searches on the forum, it appears others have run afoul of the acceptable usage with one req per 5 sec with a lot of customization. Can i run for example 1 req every 2 sec if i don't do any customization on custom weighting function etc?

Thanks :)

Jason

[Andres Morey]

Sure, that sounds fine. What does your application do?

[Jason Wang]

well just some idle data analysis for now (still in brainstorming phase of what to do). Also i noticed that older entries on hn (anything older than 25 days in fact) are gone, I am fairly annoyed since a lot of my bookmarks are for hn threads (they are sometimes more valuable than the page it links to), i'd also like to recover some of that if possible and throw it up  on a page. 

Do you have any idea why that is happening?

Thanks a lot for offering this api to the public, definitely beats working around the HN rate limiter :)

[Andres Morey]

I'm not sure what is going on on the HN-side. Can you send me an example?

In terms of the HNSearch API, if you want to reconstruct an old discussion you can do a search with a filter on the discussion sigid:

http://api.thriftdb.com/api.hnsearch.com/items/_search?filter[fields][discussion.sigid]=4949659-1f5f0 

[Jason Wang]

this one is unavailable: http://news.ycombinator.com/item?id=4910000 (from 25 days ago roughly)

this one is working: http://news.ycombinator.com/item?id=5000000

it's a rolling thing i think, because the first link was working for me 2 days ago...