JSON Parse Error

[OakStream]

Hello,
I am using a filter query to parse the twitter streaming API.   It doesn't appear that I'm getting valid JSON back as it starts with
metadata about the source/spout, etc... " source:spout:3.....  
So I get an error JSONArray text must start with '[' at 1 [character 2 line 1]

I'm using the starter project

For my code I'm just setting the tuple toString and errors out when instantiating JSON Array

  String urlContents = tuple.toString();
      System.out.println("TUPLE=="+urlContents);
try {
// parse JSON
JSONArray jsonArray = new JSONArray(urlContents);

// use
for (int i = 0; i < jsonArray.length(); i++) {
        JSONObject jsonObject = jsonArray.getJSONObject(i);

        System.out.println(jsonObject.getString("id"));
        System.out.println(jsonObject.getString("text"));
        System.out.println(jsonObject.getString("created_at"));




source: spout:3, stream: default, id: {}, [StatusJSONImpl{createdAt=Thu Jul 12 23:11:47 EDT 2012, id=223615680690397184, text='@XXX be money', source='web', isTruncated=false, inReplyToStatusId=-1, inReplyToUserId=279363091, isFavorited=false, inReplyToScreenName='user', geoLocation=null, place=null, retweetCount=0, wasRetweetedByMe=false, contributors=null, annotations=null, retweetedStatus=null, userMentionEntities=[UserMentionEntityJSONImpl{start=0, end=13, name='Tweet User', screenName='jxxx', id=999999}], urlEntities=[], hashtagEntities=[], user=UserJSONImpl{id=9999, name='Oak', screenName='xx_sender', location='null', description='null', isContributorsEnabled=false, profileImageUrl='http://a0.twimg.com/sticky/default_profile_images/default_profile_5_normal.png', profileImageUrlHttps='https://si0.twimg.com/sticky/default_profile_images/default_profile_5_normal.png', url='null', isProtected=false, followersCount=1, status=null, profileBackgroundColor='C0DEED', profileTextColor='333333', profileLinkColor='0084B4', profileSidebarFillColor='DDEEF6', profileSidebarBorderColor='C0DEED', profileUseBackgroundImage=true, showAllInlineMedia=false, friendsCount=2, createdAt=Tue Jul 10 19:52:52 EDT 2012, favouritesCount=0, utcOffset=-1, timeZone='null', profileBackgroundImageUrl='http://a0.twimg.com/images/themes/theme1/bg.png', profileBackgroundImageUrlHttps='https://si0.twimg.com/images/themes/theme1/bg.png', profileBackgroundTiled=false, lang='en', statusesCount=6, isGeoEnabled=false, isVerified=false, translator=false, listedCount=0, isFollowRequestSent=false}}]
2012-07-12 23:12:00 STDIO [ERROR] twitter4j.internal.org.json.JSONException: A JSONArray text must start with '[' at 1 [character 2 line 1]

[Ashley Brown]

Hi,

I am using a filter query to parse the twitter streaming API.   It doesn't appear that I'm getting valid JSON back as it starts with
metadata about the source/spout, etc... " source:spout:3.....  

That's because you're using toString on the Tuple. You need to use tuple.getString(0) to get the 0th element in the tuple as a string (if it is a string). Otherwise you can use getValue(0) to get it as an object and cast to whatever it should be.

A

--
Dr Ashley Brown
Chief Architect

e: ash...@spider.io
a: spider.io, 10 Warwick St, London, W1B 5LZ
w: http://spider.io/

[OakStream]

thanks for your response,

Let me re-phrase my question.  Why is source: spout:3, stream: default, id: {}, being returned as part of the response.  I don't see anywhere in the code where this is appended to the JSON response.

Additionally,   Shouldn't I expect a response from the twitter API such as

.
.
.

{

  "createdAt":"Thu Jul 12 23:11:47 EDT 2012",

  "id":223615680690397184,

  "text":"@XXX be money"

etc......

instead of id=223615680690397184, text='@XXX be money',   (the comma delimited format)

[Ashley Brown]

Why is source: spout:3, stream: default, id: {}, being returned as part of the response.  I don't see anywhere in the code where this is appended to the JSON response.

toString() is generating a human-readable representation of the Tuple object. It is that call which is prepending 'source: spout:3'.

To rephrase my original response: you are not using Tuple correctly. You should not be using toString.  You need to select one of the values from the Tuple object - I presume the first one - using a getString or getValue call. There are some examples in the tutorial here: https://github.com/nathanmarz/storm/wiki/Tutorial

Additionally,   Shouldn't I expect a response from the twitter API such as <snipped>

You're only seeing that as a response because you are using toString.

It looks as though the first element in the tuple (which you can get using getValue(0)) is already parsed into a response object, given the class names in the output.

Does the API documentation say it will return an unparsed JSON string, or a pre-parsed JSON object?

A

 

[OakStream]

Still having an issue with this. The object coming back still is not in the standard JSON format. I've used getValues, getValue, getString, etc... I have tried to identify where in the code the JSON is getting parsed. The API docs don't discuss whether the JSON is parsed or not. I would expect it to not be parsed or what is the point.

Anyone have any ideas?

Thanks again for your help,
Mike

[OakStream]

I think I found the issue.  There is a STATUS object which looks like it's parsing the JSON and just setting it to a STRING.