www.crunchbase.com how crunchbase think me as bot

18 views
Skip to first unread message

xiaoli...@gmail.com

unread,
May 18, 2016, 12:24:14 AM5/18/16
to CasperJS
www.crunchbase.com  how crunchbase think me as bot, when using casperjs to crawl it

Ken

unread,
May 25, 2017, 8:20:35 PM5/25/17
to CasperJS
Maybe they track user agent. Or some behavior, like no wait time between actions etc. This is an useful article from a friend on this topic. But I recommend not to automate for websites which don't want to serve automated / headless browsers.

Reply all
Reply to author
Forward
0 new messages