Twitter data

[Srinivas Karuturi]

Ladies and Gentlemen,

I am working on building a sentiment analysis dashboard and would like to check if anyone is aware of good scraping tools that can gather tweets with the following information based on user input of keywords and location

1. Original tweet, number of replies, number of likes, number of retweets, location, twitterHandle, link to the tweet
2. Retweet, number of replies, number of likes, number of retweets, location, twitterHandle, link to the tweet
3. Replies, number of replies, number of likes, number of retweets, location, twitterHandle, link to the tweet

Any information would be helpful

Regards,

Srinivas

[kracekumar ramaraju]

Hi Srinivas

வியா., 21 மே, 2020, முற்பகல் 11:28 அன்று, Srinivas Karuturi <srini.k...@gmail.com> எழுதியது:

I wrote a small Python script a month back. You can get the code from Gist[0]. I have tried code snippets only for few hundred tweets. So it may not be robust enough. 

It's possible to get what you are looking for using tweepy [1] and tweets are stored in SQLite using sqlite_utils [2]. The best part is `tweepy.Cursor` handles rate limiting.

I'm not aware of any out of box FOSS tools. If you find it, please share it.

[0]: https://gist.github.com/kracekumar/df9bf05119260bcf61b1a8fbdd140e7e 

[1]: http://docs.tweepy.org/en/latest/

[2]: https://sqlite-utils.readthedocs.io/en/stable/

Regards,

Srinivas

--
Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org
---
You received this message because you are subscribed to the Google Groups "datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to datameet+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/datameet/BD1EA256-5761-4511-A4AD-30642160095D%40gmail.com.

--

Thanks & Regards

kracekumar
http://kracekumar.com

[Rahul Gupta]

Hey Srini,

Best way to go about it is to register for a Twitter developer account and use the Twitter Data API.

Thanks & Regards-

Rahul Gupta

[Sidharth]

Hi,

Ive done a similar project, and used Tweepy library in Python to scrape and then analyze sentiment and then display a simple live graph.

Check out this my github link and see if any code is useful for you.

https://github.com/SidSub94/My-Codes

My credit goes to pythonprogramming.net for having tutorials.

Take care.

Sid

[Srinivas Karuturi]

Thank you guys for all the feedback..

Regards,

Srinivas

To view this discussion on the web visit https://groups.google.com/d/msgid/datameet/CAKxLuZfYv5Vv-i2Em2WXX9BVR_bVaHT-omE2tSzkR%2BDqbbaVqg%40mail.gmail.com.

[Craig Dsouza]

Hey all,

if i can piggyback on this question

how long does it typically take for the Twitter developer account to be approved?

I applied for one about 6 days ago , received one follow up mail and nothing after that

Best

Craig

To view this discussion on the web visit https://groups.google.com/d/msgid/datameet/AB99FEBD-C3F7-4154-95A6-29040A862F6D%40gmail.com.

--

Independent Researcher - Water Resources Management

Open Water Data Project

Web App: https://water-data-web-app.appspot.com/

Website: https://datameet-pune.github.io/open-water-data/

My Blog: https://craigdsouza.github.io/blog

[Sidharth]

Hi,

I could be due to Corona, the offices are in low capacity.

For me it was approved within hours.

Do let us know what happens.

Regards,

Sid

To view this discussion on the web visit https://groups.google.com/d/msgid/datameet/CAFCtJgyP6i6F4-3oUNXOX85-cEqVSa0gieTLS2_obnvKLBti9w%40mail.gmail.com.

[Craig Dsouza]

It got approved! didn't get an email update though,

realized only when I visited dev.twitter.com

To view this discussion on the web visit https://groups.google.com/d/msgid/datameet/CAGjKL%2B1%2BEbqPYcSbhinfQ4TfAxZ3wy5u_QrbDxBqBsN1up3%2BkQ%40mail.gmail.com.

[Srinivas Karuturi]

I am waiting for approval after I sent them the entire business case and what we are trying to do with the data.

To view this discussion on the web visit https://groups.google.com/d/msgid/datameet/CAFCtJgyJHiBq5rWXaULQJvmH5Ohc0YZZHK_xSs%2BKRuMZp0Ldig%40mail.gmail.com.

[kracekumar ramaraju]

Hi

I came across the tool, twint [0] which let's one to scrape twitter data without *developer API key*.

[0]: https://github.com/twintproject/twint

செவ்., 26 மே, 2020, பிற்பகல் 6:35 அன்று, Srinivas Karuturi <srini.k...@gmail.com> எழுதியது:

To view this discussion on the web visit https://groups.google.com/d/msgid/datameet/D8134E7A-C73D-4B1A-9799-AF2F3529A9AC%40gmail.com.