Is urlfetch working?

[Giacecco]

Hi all,
did any of you test AppengineJS' port of Google's
google.appengine.api.urlfetch ?

The simple code below queries search.twitter.com for an Atom feed of
all tweets from @username or referring to @username . The server
responds ok (statusCode == 200) but the Atom feed is empty. If you
test the same URL with a conventional RSS reader you can see that
there's plenty of content.

var urlfetch = require("google/appengine/api/urlfetch");

exports.FromOrAbout = function(username) {
var result;
result = urlfetch.fetch("http://search.twitter.com/search.atom?q=%40"
+ username + "+OR+from%3A" + username);
// return "http://search.twitter.com/search.atom?q=%40" + username +
"+OR+from%3A" + username;
if (result.statusCode != 200)
return "" + result.status_code;
return JSON.stringify(result);
}

An example output of this function is below. The empty "content" is at
the very beginning.

{"content":{},"finalUrl":"null","statusCode":200,"headers":
{"Date":"Sun, 25 Jul 2010 08:22:54 GMT","Server":"hi","Status":"200
OK","X-Served-From":"sjc1r165","X-Runtime":"0.04593","Content-
Type":"application/atom+xml; charset=utf-8","X-Timeline-Cache-
Hit":"Hit","X-Served-By":"sjc1r034.prod.twitter.com","Cache-
Control":"max-age=15, must-revalidate, max-age=1800","Expires":"Sun,
25 Jul 2010 08:52:54 GMT","Content-Length":"20397","Vary":"Accept-
Encoding","X-Varnish":"2395602926","Age":"0","Via":"1.1 varnish","X-
Cache-Svr":"sjc1r034.prod.twitter.com","X-
Cache":"MISS","Connection":"close"}}

Thanks for any hint!

G.

[Giacecco]

Btw, I get similar results with other feeds, e.g.

http://www.techmeme.com/firehose.xml

{"content":{},"finalUrl":"null","statusCode":200,"headers":

{"Date":"Sun, 25 Jul 2010 09:03:24 GMT","Server":"Apache/2.2.3 (Red
Hat)","Last-Modified":"Sun, 25 Jul 2010 05:50:51
GMT","ETag":"\"31554db-490f-d9b964c0\"","Accept-
Ranges":"bytes","Content-Length":"18703","Connection":"close","Content-
Type":"text/xml"}}

http://googleappengine.blogspot.com/atom.xml

{"content":{},"finalUrl":"http://feeds.feedburner.com/
GoogleAppEngineBlog","statusCode":200,"headers":{"Content-Type":"text/
xml; charset=UTF-8","ETag":"Bhjx7qNK0Y541vuv5PNb6EoWeqk","Last-
Modified":"Sun, 25 Jul 2010 07:07:35 GMT","Date":"Sun, 25 Jul 2010
09:06:12 GMT","Expires":"Sun, 25 Jul 2010 09:06:12 GMT","Cache-
Control":"private, max-age=0","X-Content-Type-Options":"nosniff","X-
XSS-Protection":"1; mode=block","Server":"GSE","Transfer-
Encoding":"chunked"}}

Could it be that AppengineJS uses the urlfetch.fetch parameters
differently than the Python SDK? The very simple call as you see it in
my example has been working well for months on a little App Engine
Python web site I have, but perhaps the AppengineJS method wants the
other parameters as well (e.g. method or payload...).

G.

[George Moschovitis]

'result.content' is a CommonJS ByteString (http://ringojs.org/api/master/binary#ByteString) to handle binary responses.

to extract the response content as a string, use:

result.content.decodeToString();

dunno why, JSON.stringify() reports content as empty...

hope this helps,

-g.

--
http://www.gmosx.com/blog
http://www.appenginejs.org

[Giacecco]

Wow that *was* tricky!

On Jul 25, 10:39 am, George Moschovitis <george.moschovi...@gmail.com>
wrote: