Django dynamic scraper process flow

[Reinard Lazuardi Kuwandy]

Hi guys, i am a newbie with dds, i want to know process flow of dds.

if there are 100 pages, and all of it already scraped, is the scraper stop or will it start scrap from the beginning again ?

Thank you very much.

Best Regards,

Reinard

[Holger Drewes]

Hmm, no, it will start from the beginning, you can define a max_pages_read options though for testing:

http://django-dynamic-scraper.readthedocs.io/en/latest/getting_started.html#running-testing-your-scraper

An extra command line option for setting a start point definitely makes sense, I'll think about it though I can't promise that it will make it into the next release.

Best regards

Holger

--
Sie erhalten diese Nachricht, weil Sie in Google Groups E-Mails von der Gruppe "django-dynamic-scraper" abonniert haben.
Wenn Sie sich von dieser Gruppe abmelden und keine E-Mails mehr von dieser Gruppe erhalten möchten, senden Sie eine E-Mail an django-dynamic-scraper+unsub...@googlegroups.com.
Weitere Optionen finden Sie unter https://groups.google.com/d/optout.

[Reinard Lazuardi Kuwandy]

Hi Holger, thankyou for your explanation. 

Best regards,

Reinard