How to Continue Scraping Specific Products in ParseHub After Reaching a Limit?

Linh

unread,

Dec 3, 2024, 3:44:40 AM12/3/24

to Web Scraping

Hi, everyone,

I’m currently using ParseHub on a free trial, which allows scraping up to 200 pages. I’ve already scraped data from the first book to the 90th book, but now I’d like to continue scraping from the 91st book onward without starting over.

Could someone guide me on how to do this?

Thank you very much!

Andrew11

unread,

Dec 3, 2024, 9:57:21 AM12/3/24

to Web Scraping

This isn’t probably something they encourage, but since this is my last post on here for a while I’ll describe the broad strokes. First of all you need to make sure there’s exactly the same number of books between runs, and that they always appear in the same order. Then you add in the “start value” of the project a variable “book_index” which for the first run is -1, and next run is 89, +90 each time. Then in the select books you add a counter variable which is +1 for each book, and only when the counter is greater than book_index and less than book_index + 90 does it click the books. But it’s a pain to go through and reassemble your data into one big excel file when you’re done, so impractical for really big jobs.

Sauban Abdullahi

unread,

Sep 5, 2025, 11:52:40 AMSep 5

to Web Scraping

Hi there!

I am a skilled Python developer offering custom automation, Telegram bots, web scraping, and Telegram scraping services tailored to your business needs. My Services Include: