Why the urlfetch can not fetch the feed from delicious.com?

Ping Chen

unread,

Jul 27, 2009, 1:37:16 AM7/27/09

to Google App Engine

I can run the program in my local environment. But when i deploy to
the GAE, I will get error when i fetch the delicious json feed using
urlfetch.

The code is:

url = "http://feeds.delicious.com/v2/json/tags/%s" %
username
result = urlfetch.fetch(url,
method=urlfetch.GET,
headers={'Content-Type': 'application/json'})
if result.status_code == 200:
getLogger(__name__).debug("delicious content: %s" %
result.content)

The delicious return content will be following (I got from the console
log). I don't know why the yahoo delicious server will refuse the GAE
server.

Google, please advise?

#
07-26 10:18PM 31.290

delicious content: HTTP/1.1 999 Unable to process request at this time
-- error 999

#
E 07-26 10:18PM 31.512

Date: Mon, 27 Jul 2009 05:18:31 GMT

#
E 07-26 10:18PM 31.512

Expires: Thu, 01 Jan 1970 22:00:00 GMT

#
E 07-26 10:18PM 31.512

Cache-Control: no-cache

#
E 07-26 10:18PM 31.512

Cache-Control: no-store

#
E 07-26 10:18PM 31.513

Pragma: no-cache

#
E 07-26 10:18PM 31.513

Cache-Control: private

#
E 07-26 10:18PM 31.513

Connection: close

#
E 07-26 10:18PM 31.513

Transfer-Encoding: chunked

#
E 07-26 10:18PM 31.513

Content-Type: text/html;charset=UTF-8

#
E 07-26 10:18PM 31.514

#
E 07-26 10:18PM 31.514

108d

#
E 07-26 10:18PM 31.514

<HTML>
<HEAD>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" >


<TITLE>
Yahoo! - 999 Unable to process request at this time -- error 999
</TITLE>


<style>
/* nn4 hide */
/*/*/
body {font:small/1.2em arial,helvetica,clean,sans-serif;font:x-
small;text-align:center;}table {font-size:inherit;font:x-small;}
html>body {font:83%/1.2em arial,helvetica,clean,sans-serif;}input
{font-size:100%;vertical-align:middle;}p, form {margin:0;padding:0;}
p {padding-bottom:6px;margin-bottom:10px;}#doc {width:48.5em;margin:0
auto;border:1px solid #fff;text-align:center;}#ygma {text-
align:right;margin-bottom:53px}
#ygma img {float:left;}#ygma div {border-bottom:1px solid #ccc;padding-
bottom:8px;margin-left:152px;}#bd {clear:both;text-align:left;width:
75%;margin:0 auto 20px;}
h1 {font-size:135%;text-align:center;margin:0 0 15px;}legend
{display:none;}fieldset {border:0 solid #fff;padding:.8em 0 .8em
4.5em;}
form {position:relative;background:#eee;margin-bottom:15px;border:1px
solid #ccc;border-width:1px 0;}
#s1p {width:15em;margin-right:.1em;}
form span {position:absolute;left:70%;top:.8em;}form a {font:78%/1.2em
arial;display:block;padding-left:.8em;white-space:nowrap;background:
url(http://us.i1.yimg.com/us.yimg.com/i/s/bullet.gif) no-repeat left
center;}
form .sep {display:none;}.more {text-align:center;}#ft {padding-top:
10px;border-top:1px solid #999;}#ft p {text-align:center;font:78%
arial;}
/* end nn4 hide */
</style>

</HEAD>
<body><div id="doc">
<div id="ygma">


<a href="http://us.rd.yahoo.com/500/*http://www.yahoo.com"><img
src=http://us.i1.yimg.com/us.yimg.com/i/yahoo.gif width=147 height=31
border=0 alt="Yahoo!"></a>


<div>


<a href="http://us.rd.yahoo.com/500/*http://www.yahoo.com">Yahoo!</a>


-


<a href="http://us.rd.yahoo.com/500/*http://help.yahoo.com">Help</a>


</div></div>
<div id="bd"><h1>


Sorry, Unable to process request at this time -- error 999.


</h1>
<table>
<tr>
<td valign=top>


<img src=http://arc.help.yahoo.com/error.gif?
r=1248671911&property=delicious2&srv=feeds.delicious.com alt="Yahoo!"
width=42 height=41 border=0>


</td>
<td>
<p>


Unfortunately we are unable to process your request at this time. This
error
is usually temporary. Please try again later.


<p>


If you continue to experience this error, it may be caused by one of
the
following:


<ul>
<li>


You may want to scan your system for spyware and viruses, as they
may interfere with your ability to connect to Yahoo!. For detailed
information on spyware and virus protection, please visit the
<a href="http://security.yahoo.com/">Yahoo! Security Center</a>.


<li>


This problem may be due to unusual network activity coming from
your Internet Service Provider. We recommend that you report this
problem to them.


</ul>

<p>


While this error is usually temporary, if it continues and the above
solutions don't resolve your problem, please
<a href="http://arc.help.yahoo.com/
arc/?.intl=us&property=delicious2&srv=feeds.delicious.com">let us
know</a>.


<p>


Return to <a href=http://www.yahoo.com>Yahoo!</a>


</p>

</td>
</tr>
</table>
<p class="more">


Please try <strong><a href="http://us.rd.yahoo.com/500/*http://
help.yahoo.com">Yahoo! Help Central</a></strong> if you need more
assistance.


</p>
</div><div id="ft"><p>


Copyright © 2006 Yahoo! Inc.
All rights reserved.



<a href="http://us.rd.yahoo.com/500/*http://privacy.yahoo.com">Privacy
Policy</a>


 


<a href="http://us.rd.yahoo.com/500/*http://docs.yahoo.com/info/
terms/">Terms of Service</a>


</p></div>
</div>
</BODY>
</HTML>

Nick Johnson (Google)

unread,

Jul 27, 2009, 5:13:18 AM7/27/09

to google-a...@googlegroups.com

Hi cpedia,

'999' is not a valid HTTP response code. Delicious is choosing to return this for their own reasons - you'd have to ask them why. It's possible they've disabled access from Google IPs.

-Nick Johnson

Calvin Slayden

unread,

Jul 27, 2009, 2:25:16 PM7/27/09

to Google App Engine

You can use an App Engine-hosted CGI proxy to validate Nick's
hypothesis (e.g. http://proxypy.com/). http://www.delicious.com works;
http://feeds.delicious.com does not.

The best way around this would be to use the Delicious API rather than
scraping the feeds page: http://delicious.com/help/api. This appears
not to block GAE (at a minimum, I can promise that Yahoo's BOSS API
does not block GAE applications).

On Jul 27, 2:13 am, "Nick Johnson (Google)" <nick.john...@google.com>
wrote:

> > src=http://us.i1.yimg.com/us.yimg.com/i/yahoo.gifwidth=147 height=31

> > border=0 alt="Yahoo!"></a>
> > 
>
> > <div>
>
> > 
> > <a href="http://us.rd.yahoo.com/500/*http://www.yahoo.com">Yahoo!</a>
> > 
>
> > -
>
> > 
> > <a href="http://us.rd.yahoo.com/500/*http://help.yahoo.com">Help</a>
> > 
>
> > </div></div>
> > <div id="bd"><h1>
>
> > 
> > Sorry, Unable to process request at this time -- error 999.
> > 
>
> > </h1>
> > <table>
> > <tr>
> > <td valign=top>
>
> > 
> > <img src=http://arc.help.yahoo.com/error.gif?

> > r=1248671911&property=delicious2&srv=feeds.delicious.com<http://arc.help.yahoo.com/error.gif?%0Ar=1248671911&property=deliciou...>alt="Yahoo!"

> > arc/?.intl=us&property=delicious2&srv=feeds.delicious.com<http://arc.help.yahoo.com/%0Aarc/?.intl=us&property=delicious2&srv=fe...>">let

> > terms/ <http://us.rd.yahoo.com/500/*http://docs.yahoo.com/info/%0Aterms/>">Terms

Ping Chen

unread,

Jul 27, 2009, 10:08:09 PM7/27/09

to Google App Engine

Thank you all.

I have submitted the issue to yahoo support team. Hope they can
resolve the problem.

Because the code worked in previous release of my project:
http://code.google.com/p/cpedialog
I also doubt it's the problem of yahoo.

On Jul 28, 2:25 am, Calvin Slayden <calvin.slay...@gmail.com> wrote:
> You can use an App Engine-hosted CGI proxy to validate Nick's

> hypothesis (e.g.http://proxypy.com/).http://www.delicious.comworks;http://feeds.delicious.comdoes not.

Reply all

Reply to author

Forward