Question about Twitter import

[matti.p...@gmail.com]

Dear all,

Did the new ORA data import Twitter JSON v2 refer to academic Twitter import?  

When I import I get the following error message (screenshot attached).I am using the Mac ORA 64-bit.  

If there is no data import yet for the academic Twitter, no problems, I can then manually create the network.  Just was not sure what the v2 referred to.  The automated import functions of Ora are fantastic so hope that it is for the academic Twitter as well :)!

Many thanks!

Matti

[Kathleen Carley]

Yes that is correct

--
Computational Analysis of Social and Organizational Systems (CASOS)
The Institute for Software Research, Department of Computer Science
Carnegie Mellon University, Pittsburgh, PA, USA
http://casos.cs.cmu.edu/
---
You received this message because you are subscribed to the Google Groups "ORA Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ORA-google-gro...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ORA-google-group/c32a276d-3e24-4754-834a-871ccda40fbfn%40googlegroups.com.

[matti.p...@gmail.com]

Thanks!  

Would you know what is causing the following error when I am doing this on a Mac, then:

An unexpected error occurred while importing.  

Some or all of the file may not have been loaded.

A JSONObject text must begin with '{' at 1 [character 2 line 1]

I am downloading the tweets via R's "academictwitterR" package.

Much appreciated!

Matti

[Kathleen Carley]

please identify which version of ora you are using

We can then check this out.

To view this discussion on the web visit https://groups.google.com/d/msgid/ORA-google-group/885583b4-db8c-4c95-ae2e-4f8c46b9d973n%40googlegroups.com.

[matti.p...@gmail.com]

Hi,

I am using ORA 3.0.9.127 (Mac, 64-bit).  Build date June 24, 2021.

Much appreciated!

Matti

[Kathleen Carley]

we are looking in to it - it may be that R version is different than the Twitter direct API - but we are checking.

To view this discussion on the web visit https://groups.google.com/d/msgid/ORA-google-group/7dc2ec1e-0799-4a79-8b9b-4230574f1d70n%40googlegroups.com.

[Kathleen Carley]

can you send a data file - that will help us debug - just send to me at kathlee...@cs.cmu.edu

On Thu, Nov 18, 2021 at 1:31 PM matti.p...@gmail.com <matti.p...@gmail.com> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/ORA-google-group/7dc2ec1e-0799-4a79-8b9b-4230574f1d70n%40googlegroups.com.

[matti.p...@gmail.com]

a sample file has been sent to the email address.  Thanks!

Matti

[matti.p...@gmail.com]

Dear all,

Any update on this :)?  I am thinking whether I will create a hack around this or wait for the bug fix.

Much appreciated.

Matti

[Jeff Reminga]

Hi Matti,

I am going to look at it this morning - right now in fact ! - sorry for the delay.

Jeff

To view this discussion on the web visit https://groups.google.com/d/msgid/ORA-google-group/491e3f23-17ff-4033-b467-6748ddc79ce4n%40googlegroups.com.

[Jeff Reminga]

Hi Matti,

The TwitteR importer was designed for an CSV/TSV format, and not JSON, as you provided.

Are you able to export to CSV/TSV?

The format looks similar to the v2 Twitter API, but not similar enough so that you could use the Twitter v2 format to import it.

We could use the general-purpose JSON importer; this would require some work configuring the importer. I will try it now.

Thanks,

Jeff

[matti.p...@gmail.com]

Dear Jeff,

Thanks! This was downloaded from academic Twitter v2 through the AcademictwitterR R package (not TwitteR): https://cran.r-project.org/web/packages/academictwitteR/index.html

I was wondering if the ORA in-built Twitter v2 importer worked for this, hence the question.  That is why the file was in JSON as it came from Twitter API through AcademictwitterR's in-built functions.  

If this format does not work for the data import wizard, which format/package/system should I use to access the Twitter Academic API?

Many thanks!

Matti

[Jeff Reminga]

Hi Matti,

The importer does not currently support importing the twitterR JSON output. I believe there is some tool chain that lets one convert that twitterR JSON into a CSV file (but I am no user of R) and that is the reason that ORA's importer for twitterR is CSV-based.

We will keep you informed of plans to implement a twitterR JSON importer.

Thanks,

Jeff

Sorry about that,

Jeff

To view this discussion on the web visit https://groups.google.com/d/msgid/ORA-google-group/8792bce9-4651-45d1-b19c-5f896c4d32afn%40googlegroups.com.

[matti.p...@gmail.com]

Dear Jeff,

Sorry to repeat this, but just to clarify one more time whether TwitterR and academicTwitterR are the same packages?  I though they were two separate ones.

 In any case, does not matter, I understand this is not supported. But could you kindly let me know which method can be used to import Twitter v2 data in a way that is compatible with Ora data import for Twitter v2?  I am happy to use Python libraries (such as twarc) or other ways to do this as well if they work?

Rgds,

Matti

[Jeff Reminga]

Hi Matti,

Here is what I would advise: ORA currently imports JSON files formatted according to the Twitter v2 specification, and so I would try to get your data into that format.

I looked again (briefly) at the Academic TwitteR json output, and it seems to output the Twitter v2 json into separate "data" and "includes" files which when combined into a json object become valid v2 twitter json. That might be the easiest thing to try.

Thanks,

Jeff 

To view this discussion on the web visit https://groups.google.com/d/msgid/ORA-google-group/f3a06dc1-c3eb-4563-8150-40315fba32abn%40googlegroups.com.

[matti.p...@gmail.com]

Dear Jeff,

Many thanks, now I now where the problem is with the R package!  I will try the twarc python package if it works without modification as well.

Rgds,

Matti