Regarding Data crawler on twitter

185 views
Skip to first unread message

fazil mohd

unread,
May 15, 2014, 9:28:09 AM5/15/14
to icwsm...@googlegroups.com
Hi everyone
 Myself, Mohd Fazil, Pursuing Ph.D. from Jamia Millia Islamia, New Delhi. If any body has any idea or useful material regarding how to build crawler for twitter, please help me.
 Thanks and Regard
Mohd Fazil

Ian Soboroff

unread,
May 15, 2014, 9:47:48 AM5/15/14
to icwsm...@googlegroups.com
That depends on what you want to crawl.  There are lots of APIs for pulling specific Twitter data.  The TREC microblog track tools (at twittertools.cc) have both a gardenhose collector as well as a crawler for pulling specific tweets in bulk.  If you want to collect graph information or profiles or other stuff from the API, you might have to do that yourself, following the info at dev.twitter.com.

Hope that helps.
Ian


--
You received this message because you are subscribed to the Google Groups "icwsm-data" group.
To unsubscribe from this group and stop receiving emails from it, send an email to icwsm-data+...@googlegroups.com.
To post to this group, send email to icwsm...@googlegroups.com.
Visit this group at http://groups.google.com/group/icwsm-data.
For more options, visit https://groups.google.com/d/optout.

fazil mohd

unread,
May 15, 2014, 10:29:02 AM5/15/14
to icwsm...@googlegroups.com
Thanks Ion 
 I want to crawl tweets then what to do?
Please suggest me.
Thanks 
Mohd Fazil

Ian Soboroff

unread,
May 15, 2014, 10:35:29 AM5/15/14
to icwsm...@googlegroups.com
If you just want to collect tweets from the gardenhose, you first need to register a free API key with Twitter.  This will give you credentials that you can use with the twittertools.cc gardenhose sampler.  Read the documentation at both those sites.


--

Derek Ruths

unread,
May 15, 2014, 5:30:47 PM5/15/14
to icwsm...@googlegroups.com
And if you don’t need as high a volume, then you could always start with the spritzer - also documented at dev.twitter.com.

D
Reply all
Reply to author
Forward
0 new messages