I am using remote_api called by Plone to send content to app engine
(to appspot, not the dev_appserver)
and all goes well. However every once and a while after a few hundred
records have been pushed (could be just a few records or several
hundred - no consistancy here) the remote_api calls starts throughing
HTTPError 302.
At this point if I wait a little while and (any where from 30 secs up
to a few minutes) and then retry
everything is fine again until, the error gets thrown again. I can't
predict how long I have to wait before it works
again.
I am having trouble working out how to go about solving this one.
Discussion subject changed to "remote_api intermittantly throwing HTTPError 302 after 100's of transactions (and found bug in apiproxy code" by Tim Hoffman
Further to my report about the remote server send HTTPError 302, after
a number of remote api calls, the stack trace I am getting suggests
that the
path in the apiproxy code has a problem as a second exception is
raised when the 302 is sent.
Traceback (most recent call last):
File "/opt/ktstudio/zope/instances/prod/swantafe.buildout/
google_appengine/google/appengine/api/datastore.py", line 1989, in
RunInTransactionCustomRetries
tx.handle, resp)
File "/opt/ktstudio/zope/instances/prod/swantafe.buildout/
google_appengine/google/appengine/api/apiproxy_stub_map.py", line 72,
in MakeSyncCall
apiproxy.MakeSyncCall(service, call, request, response)
File "/opt/ktstudio/zope/instances/prod/swantafe.buildout/
google_appengine/google/appengine/api/apiproxy_stub_map.py", line 266,
in MakeSyncCall
rpc.CheckSuccess()
File "/opt/ktstudio/zope/instances/prod/swantafe.buildout/
google_appengine/google/appengine/api/apiproxy_rpc.py", line 111, in
CheckSuccess
raise self.exception
NameError: global name 'txdata' is not defined
My first traceback
Traceback (innermost last):
Module ZPublisher.Publish, line 119, in publish
Module ZPublisher.mapply, line 88, in mapply
Module ZPublisher.Publish, line 42, in call_object
Module Shared.DC.Scripts.Bindings, line 313, in __call__
Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec
Module Products.PythonScripts.PythonScript, line 328, in _exec
Module None, line 8, in send_uos
- <PythonScript at /swan/portal_skins/custom/send_uos>
- Line 8
Module Products.PlonePSC.tools.psc_manager, line 547, in
push_to_appengine
Module google.appengine.ext.db, line 1064, in get_or_insert
Module google.appengine.api.datastore, line 1884, in
RunInTransaction
Module google.appengine.api.datastore, line 1982, in
RunInTransactionCustomRetries
Module google.appengine.ext.db, line 1059, in txn
Module google.appengine.ext.db, line 981, in get_by_key_name
Module google.appengine.ext.db, line 1180, in get
Module google.appengine.api.datastore, line 234, in Get
Module google.appengine.api.apiproxy_stub_map, line 72, in
MakeSyncCall
Module google.appengine.api.apiproxy_stub_map, line 266, in
MakeSyncCall
Module google.appengine.api.apiproxy_rpc, line 111, in CheckSuccess
HTTPError: HTTP Error 302: Found
T
On Nov 7, 8:53 am, Tim Hoffman <zutes...@gmail.com> wrote:
> I am using remote_api called by Plone to send content to app engine
> (to appspot, not the dev_appserver)
> and all goes well. However every once and a while after a few hundred
> records have been pushed (could be just a few records or several
> hundred - no consistancy here) the remote_api calls starts throughing
> HTTPError 302.
> At this point if I wait a little while and (any where from 30 secs up
> to a few minutes) and then retry
> everything is fine again until, the error gets thrown again. I can't
> predict how long I have to wait before it works
> again.
> I am having trouble working out how to go about solving this one.
On further investigation I am finding the the remote_api always gets a
HTTPError 302 after a number of transactions.
The exact number varies. Anywhere from 10's to a few hundred.
I wait for around 2min, and then resubmit it works find again.
I don't believe this is a time based problem, as I often have a
console with an authenticated remote_api session open for hours (if
not days)
and never fails then next time a perform a fetch/get etc...
I wonder if some sort of token from the authentication of the
remote_api connection is only valid for a varying number of
transactions ?
Further investigation shows that it is fact failing on 201st
transaction. (some times a lot less, but I can't get past 200
transactions).
Then if I wait for approximately 2 minutes I can continue.
I am using the remote api in a single threaded fashion. This problem
does not exhibit itself on the SDK.
Will keep plugging away.
T
On Nov 8, 9:55 am, Tim Hoffman <zutes...@gmail.com> wrote:
> On further investigation I am finding the the remote_api always gets a
> HTTPError 302 after a number of transactions.
> The exact number varies. Anywhere from 10's to a few hundred.
> I wait for around 2min, and then resubmit it works find again.
> I don't believe this is a time based problem, as I often have a
> console with an authenticated remote_api session open for hours (if
> not days)
> and never fails then next time a perform a fetch/get etc...
> I wonder if some sort of token from the authentication of the
> remote_api connection is only valid for a varying number of
> transactions ?
On Sun, Nov 8, 2009 at 8:06 AM, Tim Hoffman <zutes...@gmail.com> wrote:
> Further investigation shows that it is fact failing on 201st
> transaction. (some times a lot less, but I can't get past 200
> transactions).
> Then if I wait for approximately 2 minutes I can continue.
> I am using the remote api in a single threaded fashion. This problem
> does not exhibit itself on the SDK.
> Will keep plugging away.
> T
> On Nov 8, 9:55 am, Tim Hoffman <zutes...@gmail.com> wrote:
> > On further investigation I am finding the the remote_api always gets a
> > HTTPError 302 after a number of transactions.
> > The exact number varies. Anywhere from 10's to a few hundred.
> > I wait for around 2min, and then resubmit it works find again.
> > I don't believe this is a time based problem, as I often have a
> > console with an authenticated remote_api session open for hours (if
> > not days)
> > and never fails then next time a perform a fetch/get etc...
> > I wonder if some sort of token from the authentication of the
> > remote_api connection is only valid for a varying number of
> > transactions ?
> > T
-- Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
No not yet, that will definately be the next step. (I had to
concentrate on getting the data in first ;-) (Deadlines looming and
bulk loads from Plone won't happen very much)
What I did find, was after I get a 302 I couldn't successfully perform
another remote_api call on the connection for about 10minutes. Then
everything was fine again.
Will the URL be stuffed into the exception some where or do I have to
try and trap it in remote_api code somewhere?
Thanks
Tim
On Nov 9, 7:17 pm, "Nick Johnson (Google)" <nick.john...@google.com>
wrote:
> How have you determined that it's returning a 302? What URL is the 302
> redirect to?
> -Nick Johnson
> On Sun, Nov 8, 2009 at 8:06 AM, Tim Hoffman <zutes...@gmail.com> wrote:
> > Further investigation shows that it is fact failing on 201st
> > transaction. (some times a lot less, but I can't get past 200
> > transactions).
> > Then if I wait for approximately 2 minutes I can continue.
> > I am using the remote api in a single threaded fashion. This problem
> > does not exhibit itself on the SDK.
> > Will keep plugging away.
> > T
> > On Nov 8, 9:55 am, Tim Hoffman <zutes...@gmail.com> wrote:
> > > On further investigation I am finding the the remote_api always gets a
> > > HTTPError 302 after a number of transactions.
> > > The exact number varies. Anywhere from 10's to a few hundred.
> > > I wait for around 2min, and then resubmit it works find again.
> > > I don't believe this is a time based problem, as I often have a
> > > console with an authenticated remote_api session open for hours (if
> > > not days)
> > > and never fails then next time a perform a fetch/get etc...
> > > I wonder if some sort of token from the authentication of the
> > > remote_api connection is only valid for a varying number of
> > > transactions ?
> > > T
> --
> Nick Johnson, Developer Programs Engineer, App Engine
> Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
> 368047
On Mon, Nov 9, 2009 at 11:00 PM, Tim Hoffman <zutes...@gmail.com> wrote:
> HI Nick
> No not yet, that will definately be the next step. (I had to
> concentrate on getting the data in first ;-) (Deadlines looming and
> bulk loads from Plone won't happen very much)
> What I did find, was after I get a 302 I couldn't successfully perform
> another remote_api call on the connection for about 10minutes. Then
> everything was fine again.
> Will the URL be stuffed into the exception some where or do I have to
> try and trap it in remote_api code somewhere?
If you can get the exception returned by urllib, it's there - otherwise,
you'll need to add some logging to remote_api, or use something such as
ethercap to monitor the HTTP requests manually.
> On Nov 9, 7:17 pm, "Nick Johnson (Google)" <nick.john...@google.com>
> wrote:
> > Hi Tim,
> > How have you determined that it's returning a 302? What URL is the 302
> > redirect to?
> > -Nick Johnson
> > On Sun, Nov 8, 2009 at 8:06 AM, Tim Hoffman <zutes...@gmail.com> wrote:
> > > Further investigation shows that it is fact failing on 201st
> > > transaction. (some times a lot less, but I can't get past 200
> > > transactions).
> > > Then if I wait for approximately 2 minutes I can continue.
> > > I am using the remote api in a single threaded fashion. This problem
> > > does not exhibit itself on the SDK.
> > > Will keep plugging away.
> > > T
> > > On Nov 8, 9:55 am, Tim Hoffman <zutes...@gmail.com> wrote:
> > > > On further investigation I am finding the the remote_api always gets
> a
> > > > HTTPError 302 after a number of transactions.
> > > > The exact number varies. Anywhere from 10's to a few hundred.
> > > > I wait for around 2min, and then resubmit it works find again.
> > > > I don't believe this is a time based problem, as I often have a
> > > > console with an authenticated remote_api session open for hours (if
> > > > not days)
> > > > and never fails then next time a perform a fetch/get etc...
> > > > I wonder if some sort of token from the authentication of the
> > > > remote_api connection is only valid for a varying number of
> > > > transactions ?
> > > > T
> > --
> > Nick Johnson, Developer Programs Engineer, App Engine
> > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
> Number:
> > 368047
-- Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
|
