Twitter data collection problems with v1.1.0

[Abe_S]

Hi Laurence,

I have just downloaded the latest windows version (v1.1.0) and am having a problem with the Twitter data collector.

Filtered searches don't seem to be streaming any tweets at all for me. Sample searches and user tweet history both work fine (great introduction by the way, the user tweet history search). But when it comes to filtered searches, nothing at all. FireAnt behaves as if it is streaming (the start button becomes greyed out and the green progress bar moves), but nothing is collected and the counter doesn't change.

I am using it in the same way as previous versions. I have tried on 3 different PCs. I have tried both the 64bit and 32bit versions. 

I'm not sure if I am missing something, but I am doing everything as before with previous versions, which are still streaming filtered searches fine.

Thanks,

Abe.

[Abe_S]

Is anyone else having a problem streaming filtered searches in Windows v1.1.0? Or conversely, can anyone confirm if they are able to run Twitter keyword searches fine? Is it just me?

Thanks,

Abe.

[Laurence Anthony]

Hi Abe,

Really sorry to not get back to you on this. I've been swamped with so much other stuff. You are the only one reporting the problem. Let me have a look at this in the next day.

Are you requiring a feature in the new version? It's only a minor update, so I would expect that you can use the old version for everything you need.

Laurence.

###############################################################
Laurence ANTHONY, Ph.D.
Professor
Center for English Language Education in Science and Engineering (CELESE)
Faculty of Science and Engineering
Waseda University
3-4-1 Okubo, Shinjuku-ku, Tokyo 169-8555, Japan
E-mail: antho...@gmail.com
WWW: http://www.laurenceanthony.net/
###############################################################

--
You received this message because you are subscribed to the Google Groups "FireAnt-Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fireant+u...@googlegroups.com.
To post to this group, send email to fir...@googlegroups.com.
Visit this group at https://groups.google.com/group/fireant.
To view this discussion on the web visit https://groups.google.com/d/msgid/fireant/3c44f339-c1e6-4c7b-942b-aa58fb343117%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Abe_S]

Hi Laurence,

No problem. And no rush.

Yes, I've been (very happily) using the previous version and have been mixing use of the two - old for keyword searches, and latest version for collecting user accounts' historic tweets.

Thanks,

Abe.

[Laurence Anthony]

Hi Abe,

I checked FireAnt today after someone else reported that the radiobuttons in FireAnt 110 were not working correctly. I could not replicate the problem that you reported but I did confirm the radiobutton problem. I wonder if that might have been causing the issue that you talked about.

I've now released FireAnt 111, which fixes the radiobutton problem. Can you check if the problem you had is fixed, too?

Regards,

Laurence.

###############################################################
Laurence ANTHONY, Ph.D.
Professor
Center for English Language Education in Science and Engineering (CELESE)
Faculty of Science and Engineering
Waseda University
3-4-1 Okubo, Shinjuku-ku, Tokyo 169-8555, Japan
E-mail: antho...@gmail.com
WWW: http://www.laurenceanthony.net/
###############################################################

To view this discussion on the web visit https://groups.google.com/d/msgid/fireant/9e0b082d-745a-46d3-a56a-29aee9effacf%40googlegroups.com.

[Abe_S]

Hi Laurence,

Thanks. I downloaded v1.1.1. Sadly still facing the same issue with filtered searches not collecting any tweets. Sample searches and user tweet history both still working fine though. Very strange.

No need to worry though if it is just me. I'm happily running searches with an older version.

Abe. 

[Laurence Anthony]

Hi Abe,

I know what the problem is now. I've just fixed it and uploaded version 1.1.1 again to my website. Try it now. It should work.

The problem was a bug in translating user handle ids into  usernames. When no user handle ids were supplied, the translation tool was returning a value that cause the Twitter API to not work at all. When a user handle was supplied, everything worked fine.

Sorry for the trouble. Let me know if it works now.

Regards,

Laurence.

###############################################################
Laurence ANTHONY, Ph.D.
Professor
Center for English Language Education in Science and Engineering (CELESE)
Faculty of Science and Engineering
Waseda University
3-4-1 Okubo, Shinjuku-ku, Tokyo 169-8555, Japan
E-mail: antho...@gmail.com
WWW: http://www.laurenceanthony.net/
###############################################################

To view this discussion on the web visit https://groups.google.com/d/msgid/fireant/3a127a82-8631-4acc-8414-01219a1402a3%40googlegroups.com.

[Abe_S]

Hi Laurence,

Thanks! Yes, tried the update and filtered searches working fine now.

One more very small bug I've just noticed - the 'max tweets' selection box doesn't seem to have any effect for me. Whether it is ticked or not, the collection stops at whatever the number is set as. Easy to work around as a user by just setting a high number, but thought you'd want to know.

Thanks again,

Abe.

[Laurence Anthony]

Oh, thank you!

Is that bug in the older version, too?

Laurence.

###############################################################
Laurence ANTHONY, Ph.D.
Professor
Center for English Language Education in Science and Engineering (CELESE)
Faculty of Science and Engineering
Waseda University
3-4-1 Okubo, Shinjuku-ku, Tokyo 169-8555, Japan
E-mail: antho...@gmail.com
WWW: http://www.laurenceanthony.net/
###############################################################

To view this discussion on the web visit https://groups.google.com/d/msgid/fireant/1fcc8ccf-0158-436f-8505-1005a23c717d%40googlegroups.com.

[Abe_S]

The max tweets bug wasn't present in v1.02 which is what I was using until fully moving over to the updated v1.1.1 this morning. In v1.02 max tweets could be toggled off fine.

I don't know if the bug was there in v1.1.0 though as I didn't get any tweets downloaded so couldn't see if they stopped.

Sorry,

Abe.

[Laurence Anthony]

Hi Abe,

I've fixed that bug, too, now. I've left the version at 1.1.1. Try it out. It seems that the change in interface design moving from 1.0.2 to 1.1.0 changed a few of the default widget names, which is why the radioboxes and the max hits started operating differently.

It should be better now.

Laurence.

###############################################################
Laurence ANTHONY, Ph.D.
Professor
Center for English Language Education in Science and Engineering (CELESE)
Faculty of Science and Engineering
Waseda University
3-4-1 Okubo, Shinjuku-ku, Tokyo 169-8555, Japan
E-mail: antho...@gmail.com
WWW: http://www.laurenceanthony.net/
###############################################################

--
You received this message because you are subscribed to the Google Groups "FireAnt-Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fireant+u...@googlegroups.com.
To post to this group, send email to fir...@googlegroups.com.
Visit this group at https://groups.google.com/group/fireant.

To view this discussion on the web visit https://groups.google.com/d/msgid/fireant/30b2abd5-85d5-4f5f-9539-65de97a170ea%40googlegroups.com.

[Abe_S]

Hi Laurence,

Yes, working perfect now. Thanks.

have a nice weekend,

Abe.

[Laurence Anthony]

Great!

By the way, do you still have problems with the program timing out on long searches?

Laurence.

###############################################################
Laurence ANTHONY, Ph.D.
Professor
Center for English Language Education in Science and Engineering (CELESE)
Faculty of Science and Engineering
Waseda University
3-4-1 Okubo, Shinjuku-ku, Tokyo 169-8555, Japan
E-mail: antho...@gmail.com
WWW: http://www.laurenceanthony.net/
###############################################################

--
You received this message because you are subscribed to the Google Groups "FireAnt-Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fireant+u...@googlegroups.com.
To post to this group, send email to fir...@googlegroups.com.
Visit this group at https://groups.google.com/group/fireant.

To view this discussion on the web visit https://groups.google.com/d/msgid/fireant/08d8a58b-7038-429e-ac68-3ba3ad923c11%40googlegroups.com.

[Abe_S]

Afraid so. The longest an individual search has kept running for me has been about 5 days. But on average I find they stop about once a day. And interestingly not at the same time. So when I look at my computer first thing in the morning I sometimes find for example one or two of the 5 searches I currently have running have stopped collecting tweets at some point during the night but the others are still going.

Abe.   

[k4t...@gmail.com]

I wanted to ask if this problem with the collecting of tweets appearing to time out was still happening, or whether you managed to solve this?
I've been having the same problem (the collection just stopping at arbitrary points) using FireAnt 1.1.3.  I've just set it to run again with a massive number in the max tweets box (which I have been leaving unchecked) so if that *was* the problem, I thought I'd let you know that it was still occurring in the latest version, at least for me.
Kathryn

[Abe_S]

Hi Kathryn,

I don't know about others, but yes, I still face the issue of Tweet collection randomly stopping. I have generally always had a number of searches running over the last year and would say that on average I find a search might stop collecting every day or two. Sometimes a search stops after a few hours, sometimes a few days. And they won't necessarily go at the same time, which I think might discount internet connection at the user end being the issue. I have also tried from different locations with different internet connections to try and be sure that it wasn't connected to internet dropouts. Same result.

I've tried with and without setting a max number of tweets. I didn't notice any difference either way.

I'd be very interested to know if anyone has had searches running long term and if they had an uninterupted experience.

My presumption has been that it is an issue at the Twitter end of things, although I might be wrong.  

Best wishes,

Abe.

[k4t...@gmail.com]

Hi Abe

Thanks for letting me know it's not just me! 

My collection stopped after about 3-4 hours, even with the unchecked 'max tweets' box set to 99999999.  Previously (running on a netbook with Windows 10) it's timed out anywhere between 10 minutes and 7 hours, though on an older sturdier laptop (with Windows 7) it ran for about a day and a half. 

Restarting FireAnt (or even the laptop) seems to help, as simply stopping and starting results in progressively shorter collection periods, but even after a restart, the longest period is shorter than the previous longest period.  I can quite believe that my machines aren't really up to a large collection, but it sounds like you might be having similar difficulty with something more powerful.

I've edited the subject in the hope we might hear from others in a similar position.

Cheers

Kathryn

[Florent Moncomble]

Hello,

I am faced with the same issue on the most recent versions of FireAnt (1.1.3 on Mac, 1.1.4 on Windows) when I collect a random sample of tweets (no filters). The collection stops after a very short time, leaving me with a random number of tweets (so far, anything between 60 and a couple thousand). I need 50,000 unique tweets in total…

Any ideas much appreciated!

Regards,

Florent

[Laurence Anthony]

Hi,

Earlier today, I downloaded FireAnt 1.1.4 activated the Twitter collector and started downloading a sample with no limit. Two hours into the experiment, the current count is 70,000 tweets without any issue at all.

So, I'm not sure what the problem is. Living in Tokyo, Japan with a very fast fibre optic connection, I perhaps have one of the most stable Internet connections in the world. So, perhaps the problems you are having are with unstable connections which cause the feed to drop every so often.

I think the solution is for FireAnt to forcibly cut off the connection and restart it again every 30 min or 1 hr. I'm a bit worried that reconnecting too quickly might cause the feed to also be broken, but I'll experiment with this a bit more.

In the meantime, can anybody else report if they are experiencing the same problems?

Laurence.

###############################################################
Laurence ANTHONY, Ph.D.
Professor of Applied Linguistics

Faculty of Science and Engineering
Waseda University
3-4-1 Okubo, Shinjuku-ku, Tokyo 169-8555, Japan
E-mail: antho...@gmail.com
WWW: http://www.laurenceanthony.net/
###############################################################

--
You received this message because you are subscribed to the Google Groups "FireAnt-Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to fireant+unsubscribe@googlegroups.com.

To post to this group, send email to fir...@googlegroups.com.
Visit this group at https://groups.google.com/group/fireant.

To view this discussion on the web visit https://groups.google.com/d/msgid/fireant/555ee667-5640-4bbf-a292-5be3ff9acdd7%40googlegroups.com.