This looked like an interesting problem, and I already had most of the
pieces in place, so here's my first attempt, which is implemented in a
single class (also attached, along with some test files):
Next, create a class that implements the
com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface; the
"doTask()" method of this class is where you implement your task
logic.
Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer()
method to queue up your task:
(Note that it would be possible to pass arguments as attributes to
your Deferrable instance, rather than using varargs; I'm not sure it
makes much difference which you choose).
Just as for the Python implementation, if the arguments size exceeds
10KB, the arguments are stored in a datastore entity, which is then
deleted when the task is executed. Also, your doTask() method can
throw a PermanentTaskFailure exception to halt retries; any other
exceptions cause the task to be retried.
Let me know if you find this useful, have any questions or encounter
any problems.
On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google) <apija...@google.com> wrote:
> Hi David. This may be coming to Java eventually, but it hasn't been started
> yet. If you or anyone else is interested in contributing, let me know.
> - Jason
> On Wed, Oct 28, 2009 at 7:52 AM, David Chandler <turboman...@gmail.com>
> wrote:
On Sat, Oct 31, 2009 at 2:08 PM, Vince Bonfanti <vbonfa...@gmail.com> wrote:
> This looked like an interesting problem, and I already had most of the
> pieces in place, so here's my first attempt, which is implemented in a
> single class (also attached, along with some test files):
> Next, create a class that implements the
> com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface; the
> "doTask()" method of this class is where you implement your task
> logic.
> Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer()
> method to queue up your task:
> (Note that it would be possible to pass arguments as attributes to
> your Deferrable instance, rather than using varargs; I'm not sure it
> makes much difference which you choose).
> Just as for the Python implementation, if the arguments size exceeds
> 10KB, the arguments are stored in a datastore entity, which is then
> deleted when the task is executed. Also, your doTask() method can
> throw a PermanentTaskFailure exception to halt retries; any other
> exceptions cause the task to be retried.
> Let me know if you find this useful, have any questions or encounter
> any problems.
> Vince
> On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google) <apija...@google.com> wrote:
>> Hi David. This may be coming to Java eventually, but it hasn't been started
>> yet. If you or anyone else is interested in contributing, let me know.
>> - Jason
>> On Wed, Oct 28, 2009 at 7:52 AM, David Chandler <turboman...@gmail.com>
>> wrote:
- Perhaps make PermanentTaskFailure a runtime exception, which would
eliminate the need to declare it on methods that throw it.
- If you do the above, you could make the Deferrable interface extend
Runnable, as suggested in the meeting. I'm not sure that's appropriate,
since the implications of Runnable are different.
- I'm not sure there's any need to be able to pass arguments to the
Deferrable, since users will already have to declare a deferrable class and
instantiate it. I'm open to convincing, though.
- Have you tested tasks of size exactly MaxTaskSizeBytes? At least in
Python, the actual limit seems to include some overhead that varies from
task to task.
- If payload isn't an instance of TaskHolder, the task will silently return
- it should log an error.
Are you interested in having this included in the official SDK?
On Sat, Oct 31, 2009 at 7:08 PM, Vince Bonfanti <vbonfa...@gmail.com> wrote:
> This looked like an interesting problem, and I already had most of the
> pieces in place, so here's my first attempt, which is implemented in a
> single class (also attached, along with some test files):
> Next, create a class that implements the
> com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface; the
> "doTask()" method of this class is where you implement your task
> logic.
> Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer()
> method to queue up your task:
> (Note that it would be possible to pass arguments as attributes to
> your Deferrable instance, rather than using varargs; I'm not sure it
> makes much difference which you choose).
> Just as for the Python implementation, if the arguments size exceeds
> 10KB, the arguments are stored in a datastore entity, which is then
> deleted when the task is executed. Also, your doTask() method can
> throw a PermanentTaskFailure exception to halt retries; any other
> exceptions cause the task to be retried.
> Let me know if you find this useful, have any questions or encounter
> any problems.
> Vince
> On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google) <apija...@google.com>
> wrote:
> > Hi David. This may be coming to Java eventually, but it hasn't been
> started
> > yet. If you or anyone else is interested in contributing, let me know.
> > - Jason
> > On Wed, Oct 28, 2009 at 7:52 AM, David Chandler <turboman...@gmail.com>
> > wrote:
Thanks for your feedback. Yes, I'm very interested in seeing this
included in the official SDK. What are the steps from here?
See responses to specific questions inserted below.
Vince
> - Perhaps make PermanentTaskFailure a runtime exception, which would
> eliminate the need to declare it on methods that throw it.
That's fine. I generally prefer checked exceptions, but if the
AppEngine team prefers unchecked exceptions, I'm happy to make this
change.
> - If you do the above, you could make the Deferrable interface extend
> Runnable, as suggested in the meeting. I'm not sure that's appropriate,
> since the implications of Runnable are different.
I think Runnable is not appropriate. I also looked at Callable, but in
the end thought a new interface--Deferrable--was the best option.
> - I'm not sure there's any need to be able to pass arguments to the
> Deferrable, since users will already have to declare a deferrable class and
> instantiate it. I'm open to convincing, though.
I put the arguments there to mimic the Python API and then realized
later they're not really needed. Removing the arguments also
simplifies the code by eliminating the need for the TaskHolder
class--the Deferrable class could be serialized directly. I'm happy to
remove the arguments if that's what the AppEngine team decides is
best.
> - Have you tested tasks of size exactly MaxTaskSizeBytes? At least in
> Python, the actual limit seems to include some overhead that varies from
> task to task.
No, I haven't tested this yet.
> - If payload isn't an instance of TaskHolder, the task will silently return
> - it should log an error.
I think the only way payload won't be an instance of TaskHolder is if
the deserialize() method returns null, in which case an error has
already been logged. But I'll double-check this.
> Are you interested in having this included in the official SDK?
> -Nick
> On Sat, Oct 31, 2009 at 7:08 PM, Vince Bonfanti <vbonfa...@gmail.com> wrote:
>> This looked like an interesting problem, and I already had most of the
>> pieces in place, so here's my first attempt, which is implemented in a
>> single class (also attached, along with some test files):
>> Next, create a class that implements the
>> com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface; the
>> "doTask()" method of this class is where you implement your task
>> logic.
>> Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer()
>> method to queue up your task:
>> (Note that it would be possible to pass arguments as attributes to
>> your Deferrable instance, rather than using varargs; I'm not sure it
>> makes much difference which you choose).
>> Just as for the Python implementation, if the arguments size exceeds
>> 10KB, the arguments are stored in a datastore entity, which is then
>> deleted when the task is executed. Also, your doTask() method can
>> throw a PermanentTaskFailure exception to halt retries; any other
>> exceptions cause the task to be retried.
>> Let me know if you find this useful, have any questions or encounter
>> any problems.
>> Vince
>> On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google) <apija...@google.com>
>> wrote:
>> > Hi David. This may be coming to Java eventually, but it hasn't been
>> > started
>> > yet. If you or anyone else is interested in contributing, let me know.
>> > - Jason
>> > On Wed, Oct 28, 2009 at 7:52 AM, David Chandler <turboman...@gmail.com>
>> > wrote:
> See responses to specific questions inserted below.
> Vince
> > - Perhaps make PermanentTaskFailure a runtime exception, which would
> > eliminate the need to declare it on methods that throw it.
> That's fine. I generally prefer checked exceptions, but if the
> AppEngine team prefers unchecked exceptions, I'm happy to make this
> change.
I wouldn't say we prefer them in general - but personally, I think common
uses of PermanentTaskFailure fit into the same sort of category as out of
memory errors, divisions by zero, and so forth - they're not predictable at
design-time.
> > - If you do the above, you could make the Deferrable interface extend
> > Runnable, as suggested in the meeting. I'm not sure that's appropriate,
> > since the implications of Runnable are different.
> I think Runnable is not appropriate. I also looked at Callable, but in
> the end thought a new interface--Deferrable--was the best option.
Fair enough.
> > - I'm not sure there's any need to be able to pass arguments to the
> > Deferrable, since users will already have to declare a deferrable class
> and
> > instantiate it. I'm open to convincing, though.
> I put the arguments there to mimic the Python API and then realized
> later they're not really needed. Removing the arguments also
> simplifies the code by eliminating the need for the TaskHolder
> class--the Deferrable class could be serialized directly. I'm happy to
> remove the arguments if that's what the AppEngine team decides is
> best.
Since the Java version is inevitably different due to the need for a custom
class instead of a function object, I'd vote to drop the arguments.
> > - Have you tested tasks of size exactly MaxTaskSizeBytes? At least in
> > Python, the actual limit seems to include some overhead that varies from
> > task to task.
> No, I haven't tested this yet.
> > - If payload isn't an instance of TaskHolder, the task will silently
> return
> > - it should log an error.
> I think the only way payload won't be an instance of TaskHolder is if
> the deserialize() method returns null, in which case an error has
> already been logged. But I'll double-check this.
The other possibility is that something else serialized an object and sent
it to your task handler - or possibly, that a future version of your own
code did so. I think logging an error (but returning a 200 status code)
would be the best policy, so these sorts of situations are debuggable.
> > Are you interested in having this included in the official SDK?
> > -Nick
> > On Sat, Oct 31, 2009 at 7:08 PM, Vince Bonfanti <vbonfa...@gmail.com>
> wrote:
> >> This looked like an interesting problem, and I already had most of the
> >> pieces in place, so here's my first attempt, which is implemented in a
> >> single class (also attached, along with some test files):
> >> Next, create a class that implements the
> >> com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface; the
> >> "doTask()" method of this class is where you implement your task
> >> logic.
> >> Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer()
> >> method to queue up your task:
> >> (Note that it would be possible to pass arguments as attributes to
> >> your Deferrable instance, rather than using varargs; I'm not sure it
> >> makes much difference which you choose).
> >> Just as for the Python implementation, if the arguments size exceeds
> >> 10KB, the arguments are stored in a datastore entity, which is then
> >> deleted when the task is executed. Also, your doTask() method can
> >> throw a PermanentTaskFailure exception to halt retries; any other
> >> exceptions cause the task to be retried.
> >> Let me know if you find this useful, have any questions or encounter
> >> any problems.
> >> Vince
> >> On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google) <apija...@google.com>
> >> wrote:
> >> > Hi David. This may be coming to Java eventually, but it hasn't been
> >> > started
> >> > yet. If you or anyone else is interested in contributing, let me know.
> >> > - Jason
> >> > On Wed, Oct 28, 2009 at 7:52 AM, David Chandler <
> turboman...@gmail.com>
> >> > wrote:
I've started looking at using Vince's Defer helper class.
As a suggestion, could we add a TemporaryTaskFailure exception so it
is explicit what behaviour is expected. I realise that a task retry is
the default behaviour by propagating an exception, I just think it's
better documentation if it's explicit.
I'm not sure about adding a TemporaryTaskFailure (TransientTaskFailure?) exception, but regardless of whether we do or not, it seems that the Deferrable.doTask method should be declared to throw Exception. Otherwise, implementations will be restricted to throwing only RuntimeException (or subclasses thereof). Think of cases where you're using third-party libraries and you simply want exceptions to propagate up.
My current implementation is based on the Python implementation, which says this about task failures:
"'Failure' in the case of a deferred task is defined as your task throwing any uncaught exception. Normally, automatic retries are what you want - for example, if the task does datastore operations, and the datastore threw a Timeout exception. If you want to deliberately cause your task to fail so it will be retried, you can simply throw an exception."
That seems reasonable: any uncaught exception causes a task retry; normal termination or PermanentTaskFailure exception do not retry.
On Tue, Nov 10, 2009 at 1:35 AM, Roy <roy.smith....@googlemail.com> wrote:
> I've started looking at using Vince's Defer helper class.
> As a suggestion, could we add a TemporaryTaskFailure exception so it > is explicit what behaviour is expected. I realise that a task retry is > the default behaviour by propagating an exception, I just think it's > better documentation if it's explicit.
Here are the changes I've made since the original implementation:
- The Deferrable.doTask method is now declared to throw Exception
(instead of PermanentTaskFailure). This allows implementations to
throw any subclass of Exception in order to have the task retried.
(PermanentTaskFailure remains a subclass of Exception).
- The Deferrable.doTask method no longer takes any arguments.
- Removed the TaskHolder nested class since it's no longer needed.
- Allow an extra 240 bytes for task overhead; in testing the actual
overhead was 101 bytes, so this will leave some head-room. In
addition, the IllegalArgumentException that's thrown if the task size
is exceeded is caught (and logged), and the task it then written to
the datastore.
- An exception is logged if payload is not an instance of Deferrable.
- If the task is written to the datastore and there's a subsequent
failure to queue the task, the datastore entity is deleted.
- Try twice for datatstore puts and deletes in case of
DatastoreTimeoutException.
- Try twice when queuing a task in case of TransientFailureException.
- Added comments.
Let me know if you have additional feedback.
Vince
On Sun, Nov 8, 2009 at 3:34 PM, Nick Johnson (Google)
>> See responses to specific questions inserted below.
>> Vince
>> > - Perhaps make PermanentTaskFailure a runtime exception, which would
>> > eliminate the need to declare it on methods that throw it.
>> That's fine. I generally prefer checked exceptions, but if the
>> AppEngine team prefers unchecked exceptions, I'm happy to make this
>> change.
> I wouldn't say we prefer them in general - but personally, I think common
> uses of PermanentTaskFailure fit into the same sort of category as out of
> memory errors, divisions by zero, and so forth - they're not predictable at
> design-time.
>> > - If you do the above, you could make the Deferrable interface extend
>> > Runnable, as suggested in the meeting. I'm not sure that's appropriate,
>> > since the implications of Runnable are different.
>> I think Runnable is not appropriate. I also looked at Callable, but in
>> the end thought a new interface--Deferrable--was the best option.
> Fair enough.
>> > - I'm not sure there's any need to be able to pass arguments to the
>> > Deferrable, since users will already have to declare a deferrable class
>> > and
>> > instantiate it. I'm open to convincing, though.
>> I put the arguments there to mimic the Python API and then realized
>> later they're not really needed. Removing the arguments also
>> simplifies the code by eliminating the need for the TaskHolder
>> class--the Deferrable class could be serialized directly. I'm happy to
>> remove the arguments if that's what the AppEngine team decides is
>> best.
> Since the Java version is inevitably different due to the need for a custom
> class instead of a function object, I'd vote to drop the arguments.
>> > - Have you tested tasks of size exactly MaxTaskSizeBytes? At least in
>> > Python, the actual limit seems to include some overhead that varies from
>> > task to task.
>> No, I haven't tested this yet.
>> > - If payload isn't an instance of TaskHolder, the task will silently
>> > return
>> > - it should log an error.
>> I think the only way payload won't be an instance of TaskHolder is if
>> the deserialize() method returns null, in which case an error has
>> already been logged. But I'll double-check this.
> The other possibility is that something else serialized an object and sent
> it to your task handler - or possibly, that a future version of your own
> code did so. I think logging an error (but returning a 200 status code)
> would be the best policy, so these sorts of situations are debuggable.
> -Nick Johnson
>> > Are you interested in having this included in the official SDK?
>> > -Nick
>> > On Sat, Oct 31, 2009 at 7:08 PM, Vince Bonfanti <vbonfa...@gmail.com>
>> > wrote:
>> >> This looked like an interesting problem, and I already had most of the
>> >> pieces in place, so here's my first attempt, which is implemented in a
>> >> single class (also attached, along with some test files):
>> >> Next, create a class that implements the
>> >> com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface; the
>> >> "doTask()" method of this class is where you implement your task
>> >> logic.
>> >> Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer()
>> >> method to queue up your task:
>> >> (Note that it would be possible to pass arguments as attributes to
>> >> your Deferrable instance, rather than using varargs; I'm not sure it
>> >> makes much difference which you choose).
>> >> Just as for the Python implementation, if the arguments size exceeds
>> >> 10KB, the arguments are stored in a datastore entity, which is then
>> >> deleted when the task is executed. Also, your doTask() method can
>> >> throw a PermanentTaskFailure exception to halt retries; any other
>> >> exceptions cause the task to be retried.
>> >> Let me know if you find this useful, have any questions or encounter
>> >> any problems.
>> >> Vince
>> >> On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google) <apija...@google.com>
>> >> wrote:
>> >> > Hi David. This may be coming to Java eventually, but it hasn't been
>> >> > started
>> >> > yet. If you or anyone else is interested in contributing, let me
>> >> > know.
>> >> > - Jason
>> >> > On Wed, Oct 28, 2009 at 7:52 AM, David Chandler
>> >> > <turboman...@gmail.com>
>> >> > wrote:
> Here are the changes I've made since the original implementation:
> - The Deferrable.doTask method is now declared to throw Exception
> (instead of PermanentTaskFailure). This allows implementations to
> throw any subclass of Exception in order to have the task retried.
> (PermanentTaskFailure remains a subclass of Exception).
> - The Deferrable.doTask method no longer takes any arguments.
> - Removed the TaskHolder nested class since it's no longer needed.
> - Allow an extra 240 bytes for task overhead; in testing the actual
> overhead was 101 bytes, so this will leave some head-room. In
> addition, the IllegalArgumentException that's thrown if the task size
> is exceeded is caught (and logged), and the task it then written to
> the datastore.
> - An exception is logged if payload is not an instance of Deferrable.
> - If the task is written to the datastore and there's a subsequent
> failure to queue the task, the datastore entity is deleted.
> - Try twice for datatstore puts and deletes in case of
> DatastoreTimeoutException.
> - Try twice when queuing a task in case of TransientFailureException.
> - Added comments.
> Let me know if you have additional feedback.
> Vince
> On Sun, Nov 8, 2009 at 3:34 PM, Nick Johnson (Google)
> <nick.john...@google.com> wrote:
> > Hi Vince,
> > On Sun, Nov 8, 2009 at 7:58 PM, Vince Bonfanti <vbonfa...@gmail.com>
> wrote:
> >> Hi Nick,
> >> Thanks for your feedback. Yes, I'm very interested in seeing this
> >> included in the official SDK. What are the steps from here?
> >> See responses to specific questions inserted below.
> >> Vince
> >> > - Perhaps make PermanentTaskFailure a runtime exception, which would
> >> > eliminate the need to declare it on methods that throw it.
> >> That's fine. I generally prefer checked exceptions, but if the
> >> AppEngine team prefers unchecked exceptions, I'm happy to make this
> >> change.
> > I wouldn't say we prefer them in general - but personally, I think common
> > uses of PermanentTaskFailure fit into the same sort of category as out of
> > memory errors, divisions by zero, and so forth - they're not predictable
> at
> > design-time.
> >> > - If you do the above, you could make the Deferrable interface extend
> >> > Runnable, as suggested in the meeting. I'm not sure that's
> appropriate,
> >> > since the implications of Runnable are different.
> >> I think Runnable is not appropriate. I also looked at Callable, but in
> >> the end thought a new interface--Deferrable--was the best option.
> > Fair enough.
> >> > - I'm not sure there's any need to be able to pass arguments to the
> >> > Deferrable, since users will already have to declare a deferrable
> class
> >> > and
> >> > instantiate it. I'm open to convincing, though.
> >> I put the arguments there to mimic the Python API and then realized
> >> later they're not really needed. Removing the arguments also
> >> simplifies the code by eliminating the need for the TaskHolder
> >> class--the Deferrable class could be serialized directly. I'm happy to
> >> remove the arguments if that's what the AppEngine team decides is
> >> best.
> > Since the Java version is inevitably different due to the need for a
> custom
> > class instead of a function object, I'd vote to drop the arguments.
> >> > - Have you tested tasks of size exactly MaxTaskSizeBytes? At least in
> >> > Python, the actual limit seems to include some overhead that varies
> from
> >> > task to task.
> >> No, I haven't tested this yet.
> >> > - If payload isn't an instance of TaskHolder, the task will silently
> >> > return
> >> > - it should log an error.
> >> I think the only way payload won't be an instance of TaskHolder is if
> >> the deserialize() method returns null, in which case an error has
> >> already been logged. But I'll double-check this.
> > The other possibility is that something else serialized an object and
> sent
> > it to your task handler - or possibly, that a future version of your own
> > code did so. I think logging an error (but returning a 200 status code)
> > would be the best policy, so these sorts of situations are debuggable.
> > -Nick Johnson
> >> > Are you interested in having this included in the official SDK?
> >> > -Nick
> >> > On Sat, Oct 31, 2009 at 7:08 PM, Vince Bonfanti <vbonfa...@gmail.com>
> >> > wrote:
> >> >> This looked like an interesting problem, and I already had most of
> the
> >> >> pieces in place, so here's my first attempt, which is implemented in
> a
> >> >> single class (also attached, along with some test files):
> >> >> Next, create a class that implements the
> >> >> com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface; the
> >> >> "doTask()" method of this class is where you implement your task
> >> >> logic.
> >> >> Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer()
> >> >> method to queue up your task:
> >> >> (Note that it would be possible to pass arguments as attributes to
> >> >> your Deferrable instance, rather than using varargs; I'm not sure it
> >> >> makes much difference which you choose).
> >> >> Just as for the Python implementation, if the arguments size exceeds
> >> >> 10KB, the arguments are stored in a datastore entity, which is then
> >> >> deleted when the task is executed. Also, your doTask() method can
> >> >> throw a PermanentTaskFailure exception to halt retries; any other
> >> >> exceptions cause the task to be retried.
> >> >> Let me know if you find this useful, have any questions or encounter
> >> >> any problems.
> >> >> Vince
> >> >> On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google) <apija...@google.com
> >> >> wrote:
> >> >> > Hi David. This may be coming to Java eventually, but it hasn't been
> >> >> > started
> >> >> > yet. If you or anyone else is interested in contributing, let me
> >> >> > know.
> >> >> > - Jason
> >> >> > On Wed, Oct 28, 2009 at 7:52 AM, David Chandler
> >> >> > <turboman...@gmail.com>
> >> >> > wrote:
I've visited codereview.appspot.com, but am confused. I tried creating
a new issue, but:
- what do I specify for "SVN base"?
- it will only let me upload a patch file (not a java source file),
but I'm not sure how to create a patch for a brand-new file?
Can you offer some help? thanks.
Vince
On Tue, Nov 10, 2009 at 5:12 PM, Nick Johnson (Google)
<nick.john...@google.com> wrote:
> Hi Vince,
> Thanks for doing this! Could you upload your code to codereview.appspot.com
> and link to it from the issue? We can continue the discussion there.
> -Nick
> On Tue, Nov 10, 2009 at 9:37 PM, Vince Bonfanti <vbonfa...@gmail.com> wrote:
>> I've faxed over the CLA and created issue #2381 for this:
>> Here are the changes I've made since the original implementation:
>> - The Deferrable.doTask method is now declared to throw Exception
>> (instead of PermanentTaskFailure). This allows implementations to
>> throw any subclass of Exception in order to have the task retried.
>> (PermanentTaskFailure remains a subclass of Exception).
>> - The Deferrable.doTask method no longer takes any arguments.
>> - Removed the TaskHolder nested class since it's no longer needed.
>> - Allow an extra 240 bytes for task overhead; in testing the actual
>> overhead was 101 bytes, so this will leave some head-room. In
>> addition, the IllegalArgumentException that's thrown if the task size
>> is exceeded is caught (and logged), and the task it then written to
>> the datastore.
>> - An exception is logged if payload is not an instance of Deferrable.
>> - If the task is written to the datastore and there's a subsequent
>> failure to queue the task, the datastore entity is deleted.
>> - Try twice for datatstore puts and deletes in case of
>> DatastoreTimeoutException.
>> - Try twice when queuing a task in case of TransientFailureException.
>> - Added comments.
>> Let me know if you have additional feedback.
>> Vince
>> On Sun, Nov 8, 2009 at 3:34 PM, Nick Johnson (Google)
>> <nick.john...@google.com> wrote:
>> > Hi Vince,
>> > On Sun, Nov 8, 2009 at 7:58 PM, Vince Bonfanti <vbonfa...@gmail.com>
>> > wrote:
>> >> Hi Nick,
>> >> Thanks for your feedback. Yes, I'm very interested in seeing this
>> >> included in the official SDK. What are the steps from here?
>> >> See responses to specific questions inserted below.
>> >> Vince
>> >> > - Perhaps make PermanentTaskFailure a runtime exception, which would
>> >> > eliminate the need to declare it on methods that throw it.
>> >> That's fine. I generally prefer checked exceptions, but if the
>> >> AppEngine team prefers unchecked exceptions, I'm happy to make this
>> >> change.
>> > I wouldn't say we prefer them in general - but personally, I think
>> > common
>> > uses of PermanentTaskFailure fit into the same sort of category as out
>> > of
>> > memory errors, divisions by zero, and so forth - they're not predictable
>> > at
>> > design-time.
>> >> > - If you do the above, you could make the Deferrable interface extend
>> >> > Runnable, as suggested in the meeting. I'm not sure that's
>> >> > appropriate,
>> >> > since the implications of Runnable are different.
>> >> I think Runnable is not appropriate. I also looked at Callable, but in
>> >> the end thought a new interface--Deferrable--was the best option.
>> > Fair enough.
>> >> > - I'm not sure there's any need to be able to pass arguments to the
>> >> > Deferrable, since users will already have to declare a deferrable
>> >> > class
>> >> > and
>> >> > instantiate it. I'm open to convincing, though.
>> >> I put the arguments there to mimic the Python API and then realized
>> >> later they're not really needed. Removing the arguments also
>> >> simplifies the code by eliminating the need for the TaskHolder
>> >> class--the Deferrable class could be serialized directly. I'm happy to
>> >> remove the arguments if that's what the AppEngine team decides is
>> >> best.
>> > Since the Java version is inevitably different due to the need for a
>> > custom
>> > class instead of a function object, I'd vote to drop the arguments.
>> >> > - Have you tested tasks of size exactly MaxTaskSizeBytes? At least in
>> >> > Python, the actual limit seems to include some overhead that varies
>> >> > from
>> >> > task to task.
>> >> No, I haven't tested this yet.
>> >> > - If payload isn't an instance of TaskHolder, the task will silently
>> >> > return
>> >> > - it should log an error.
>> >> I think the only way payload won't be an instance of TaskHolder is if
>> >> the deserialize() method returns null, in which case an error has
>> >> already been logged. But I'll double-check this.
>> > The other possibility is that something else serialized an object and
>> > sent
>> > it to your task handler - or possibly, that a future version of your own
>> > code did so. I think logging an error (but returning a 200 status code)
>> > would be the best policy, so these sorts of situations are debuggable.
>> > -Nick Johnson
>> >> > Are you interested in having this included in the official SDK?
>> >> > -Nick
>> >> > On Sat, Oct 31, 2009 at 7:08 PM, Vince Bonfanti <vbonfa...@gmail.com>
>> >> > wrote:
>> >> >> This looked like an interesting problem, and I already had most of
>> >> >> the
>> >> >> pieces in place, so here's my first attempt, which is implemented in
>> >> >> a
>> >> >> single class (also attached, along with some test files):
>> >> >> Next, create a class that implements the
>> >> >> com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface;
>> >> >> the
>> >> >> "doTask()" method of this class is where you implement your task
>> >> >> logic.
>> >> >> Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer()
>> >> >> method to queue up your task:
>> >> >> (Note that it would be possible to pass arguments as attributes to
>> >> >> your Deferrable instance, rather than using varargs; I'm not sure it
>> >> >> makes much difference which you choose).
>> >> >> Just as for the Python implementation, if the arguments size exceeds
>> >> >> 10KB, the arguments are stored in a datastore entity, which is then
>> >> >> deleted when the task is executed. Also, your doTask() method can
>> >> >> throw a PermanentTaskFailure exception to halt retries; any other
>> >> >> exceptions cause the task to be retried.
>> >> >> Let me know if you find this useful, have any questions or encounter
>> >> >> any problems.
>> >> >> Vince
>> >> >> On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google)
>> >> >> <apija...@google.com>
>> >> >> wrote:
>> >> >> > Hi David. This may be coming to Java eventually, but it hasn't
>> >> >> > been
>> >> >> > started
>> >> >> > yet. If you or anyone else is interested in contributing, let me
>> >> >> > know.
>> >> >> > - Jason
>> >> >> > On Wed, Oct 28, 2009 at 7:52 AM, David Chandler
>> >> >> > <turboman...@gmail.com>
>> >> >> > wrote:
> On Tue, Nov 10, 2009 at 5:12 PM, Nick Johnson (Google)
> <nick.john...@google.com> wrote:
> > Hi Vince,
> > Thanks for doing this! Could you upload your code to
> codereview.appspot.com
> > and link to it from the issue? We can continue the discussion there.
> > -Nick
> > On Tue, Nov 10, 2009 at 9:37 PM, Vince Bonfanti <vbonfa...@gmail.com>
> wrote:
> >> I've faxed over the CLA and created issue #2381 for this:
> >> Here are the changes I've made since the original implementation:
> >> - The Deferrable.doTask method is now declared to throw Exception
> >> (instead of PermanentTaskFailure). This allows implementations to
> >> throw any subclass of Exception in order to have the task retried.
> >> (PermanentTaskFailure remains a subclass of Exception).
> >> - The Deferrable.doTask method no longer takes any arguments.
> >> - Removed the TaskHolder nested class since it's no longer needed.
> >> - Allow an extra 240 bytes for task overhead; in testing the actual
> >> overhead was 101 bytes, so this will leave some head-room. In
> >> addition, the IllegalArgumentException that's thrown if the task size
> >> is exceeded is caught (and logged), and the task it then written to
> >> the datastore.
> >> - An exception is logged if payload is not an instance of Deferrable.
> >> - If the task is written to the datastore and there's a subsequent
> >> failure to queue the task, the datastore entity is deleted.
> >> - Try twice for datatstore puts and deletes in case of
> >> DatastoreTimeoutException.
> >> - Try twice when queuing a task in case of TransientFailureException.
> >> - Added comments.
> >> Let me know if you have additional feedback.
> >> Vince
> >> On Sun, Nov 8, 2009 at 3:34 PM, Nick Johnson (Google)
> >> <nick.john...@google.com> wrote:
> >> > Hi Vince,
> >> > On Sun, Nov 8, 2009 at 7:58 PM, Vince Bonfanti <vbonfa...@gmail.com>
> >> > wrote:
> >> >> Hi Nick,
> >> >> Thanks for your feedback. Yes, I'm very interested in seeing this
> >> >> included in the official SDK. What are the steps from here?
> >> >> See responses to specific questions inserted below.
> >> >> Vince
> >> >> > - Perhaps make PermanentTaskFailure a runtime exception, which
> would
> >> >> > eliminate the need to declare it on methods that throw it.
> >> >> That's fine. I generally prefer checked exceptions, but if the
> >> >> AppEngine team prefers unchecked exceptions, I'm happy to make this
> >> >> change.
> >> > I wouldn't say we prefer them in general - but personally, I think
> >> > common
> >> > uses of PermanentTaskFailure fit into the same sort of category as out
> >> > of
> >> > memory errors, divisions by zero, and so forth - they're not
> predictable
> >> > at
> >> > design-time.
> >> >> > - If you do the above, you could make the Deferrable interface
> extend
> >> >> > Runnable, as suggested in the meeting. I'm not sure that's
> >> >> > appropriate,
> >> >> > since the implications of Runnable are different.
> >> >> I think Runnable is not appropriate. I also looked at Callable, but
> in
> >> >> the end thought a new interface--Deferrable--was the best option.
> >> > Fair enough.
> >> >> > - I'm not sure there's any need to be able to pass arguments to the
> >> >> > Deferrable, since users will already have to declare a deferrable
> >> >> > class
> >> >> > and
> >> >> > instantiate it. I'm open to convincing, though.
> >> >> I put the arguments there to mimic the Python API and then realized
> >> >> later they're not really needed. Removing the arguments also
> >> >> simplifies the code by eliminating the need for the TaskHolder
> >> >> class--the Deferrable class could be serialized directly. I'm happy
> to
> >> >> remove the arguments if that's what the AppEngine team decides is
> >> >> best.
> >> > Since the Java version is inevitably different due to the need for a
> >> > custom
> >> > class instead of a function object, I'd vote to drop the arguments.
> >> >> > - Have you tested tasks of size exactly MaxTaskSizeBytes? At least
> in
> >> >> > Python, the actual limit seems to include some overhead that varies
> >> >> > from
> >> >> > task to task.
> >> >> No, I haven't tested this yet.
> >> >> > - If payload isn't an instance of TaskHolder, the task will
> silently
> >> >> > return
> >> >> > - it should log an error.
> >> >> I think the only way payload won't be an instance of TaskHolder is if
> >> >> the deserialize() method returns null, in which case an error has
> >> >> already been logged. But I'll double-check this.
> >> > The other possibility is that something else serialized an object and
> >> > sent
> >> > it to your task handler - or possibly, that a future version of your
> own
> >> > code did so. I think logging an error (but returning a 200 status
> code)
> >> > would be the best policy, so these sorts of situations are debuggable.
> >> > -Nick Johnson
> >> >> > Are you interested in having this included in the official SDK?
> >> >> > -Nick
> >> >> > On Sat, Oct 31, 2009 at 7:08 PM, Vince Bonfanti <
> vbonfa...@gmail.com>
> >> >> > wrote:
> >> >> >> This looked like an interesting problem, and I already had most of
> >> >> >> the
> >> >> >> pieces in place, so here's my first attempt, which is implemented
> in
> >> >> >> a
> >> >> >> single class (also attached, along with some test files):
> >> >> >> Next, create a class that implements the
> >> >> >> com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface;
> >> >> >> the
> >> >> >> "doTask()" method of this class is where you implement your task
> >> >> >> logic.
> >> >> >> Then, invoke the
> com.newatlanta.appengine.taskqueue.Deferred.defer()
> >> >> >> method to queue up your task:
> >> >> >> (Note that it would be possible to pass arguments as attributes to
> >> >> >> your Deferrable instance, rather than using varargs; I'm not sure
> it
> >> >> >> makes much difference which you choose).
> >> >> >> Just as for the Python implementation, if the arguments size
> exceeds
> >> >> >> 10KB, the arguments are stored in a datastore entity, which is
> then
> >> >> >> deleted when the task is executed. Also, your doTask() method can
> >> >> >> throw a PermanentTaskFailure exception to halt retries; any other
> >> >> >> exceptions cause the task to be retried.
> >> >> >> Let me know if you find this useful, have any questions or
> encounter
> >> >> >> any problems.
> >> >> >> Vince
> >> >> >> On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google)
> >> >> >> <apija...@google.com>
> >> >> >> wrote:
> >> >> >> > Hi David. This may be coming to Java eventually, but it hasn't
> >> >> >> > been
> >> >> >> > started
> >> >> >> > yet. If you or anyone else is interested in contributing, let me
> >> >> >> > know.
> >> >> >> > - Jason
> >> >> >> > On Wed, Oct 28, 2009 at 7:52 AM, David Chandler
> >> >> >> > <turboman...@gmail.com>
Vince, this is great! I hadn't been watching my own issue, so didn't
see this until now. Thanks so much!
Only enhancement I would suggest is to enable multiple deferred
queues. For example. I would like to defer tasks in an email throttle
queue separately from a general background queue. Perhaps the
Deferrable interface could have a getQueueName() method, or
Deferred.defer could have an additional signature defer(Deferrable
task, String queueName).
Thanks again,
/dmc
On Nov 10, 4:37 pm, Vince Bonfanti <vbonfa...@gmail.com> wrote:
> Here are the changes I've made since the original implementation:
> - The Deferrable.doTask method is now declared to throw Exception
> (instead of PermanentTaskFailure). This allows implementations to
> throw any subclass of Exception in order to have the task retried.
> (PermanentTaskFailure remains a subclass of Exception).
> - The Deferrable.doTask method no longer takes any arguments.
> - Removed the TaskHolder nested class since it's no longer needed.
> - Allow an extra 240 bytes for task overhead; in testing the actual
> overhead was 101 bytes, so this will leave some head-room. In
> addition, the IllegalArgumentException that's thrown if the task size
> is exceeded is caught (and logged), and the task it then written to
> the datastore.
> - An exception is logged if payload is not an instance of Deferrable.
> - If the task is written to the datastore and there's a subsequent
> failure to queue the task, the datastore entity is deleted.
> - Try twice for datatstore puts and deletes in case of
> DatastoreTimeoutException.
> - Try twice when queuing a task in case of TransientFailureException.
> - Added comments.
> Let me know if you have additional feedback.
> Vince
> On Sun, Nov 8, 2009 at 3:34 PM, Nick Johnson (Google)
> >> See responses to specific questions inserted below.
> >> Vince
> >> > - Perhaps make PermanentTaskFailure a runtime exception, which would
> >> > eliminate the need to declare it on methods that throw it.
> >> That's fine. I generally prefer checked exceptions, but if the
> >> AppEngine team prefers unchecked exceptions, I'm happy to make this
> >> change.
> > I wouldn't say we prefer them in general - but personally, I think common
> > uses of PermanentTaskFailure fit into the same sort of category as out of
> > memory errors, divisions by zero, and so forth - they're not predictable at
> > design-time.
> >> > - If you do the above, you could make the Deferrable interface extend
> >> > Runnable, as suggested in the meeting. I'm not sure that's appropriate,
> >> > since the implications of Runnable are different.
> >> I think Runnable is not appropriate. I also looked at Callable, but in
> >> the end thought a new interface--Deferrable--was the best option.
> > Fair enough.
> >> > - I'm not sure there's any need to be able to pass arguments to the
> >> > Deferrable, since users will already have to declare a deferrable class
> >> > and
> >> > instantiate it. I'm open to convincing, though.
> >> I put the arguments there to mimic the Python API and then realized
> >> later they're not really needed. Removing the arguments also
> >> simplifies the code by eliminating the need for the TaskHolder
> >> class--the Deferrable class could be serialized directly. I'm happy to
> >> remove the arguments if that's what the AppEngine team decides is
> >> best.
> > Since the Java version is inevitably different due to the need for a custom
> > class instead of a function object, I'd vote to drop the arguments.
> >> > - Have you tested tasks of size exactly MaxTaskSizeBytes? At least in
> >> > Python, the actual limit seems to include some overhead that varies from
> >> > task to task.
> >> No, I haven't tested this yet.
> >> > - If payload isn't an instance of TaskHolder, the task will silently
> >> > return
> >> > - it should log an error.
> >> I think the only way payload won't be an instance of TaskHolder is if
> >> the deserialize() method returns null, in which case an error has
> >> already been logged. But I'll double-check this.
> > The other possibility is that something else serialized an object and sent
> > it to your task handler - or possibly, that a future version of your own
> > code did so. I think logging an error (but returning a 200 status code)
> > would be the best policy, so these sorts of situations are debuggable.
> > -Nick Johnson
> >> > Are you interested in having this included in the official SDK?
> >> > -Nick
> >> > On Sat, Oct 31, 2009 at 7:08 PM, Vince Bonfanti <vbonfa...@gmail.com>
> >> > wrote:
> >> >> This looked like an interesting problem, and I already had most of the
> >> >> pieces in place, so here's my first attempt, which is implemented in a
> >> >> single class (also attached, along with some test files):
> >> >> Next, create a class that implements the
> >> >> com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface; the
> >> >> "doTask()" method of this class is where you implement your task
> >> >> logic.
> >> >> Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer()
> >> >> method to queue up your task:
> >> >> (Note that it would be possible to pass arguments as attributes to
> >> >> your Deferrable instance, rather than using varargs; I'm not sure it
> >> >> makes much difference which you choose).
> >> >> Just as for the Python implementation, if the arguments size exceeds
> >> >> 10KB, the arguments are stored in a datastore entity, which is then
> >> >> deleted when the task is executed. Also, your doTask() method can
> >> >> throw a PermanentTaskFailure exception to halt retries; any other
> >> >> exceptions cause the task to be retried.
> >> >> Let me know if you find this useful, have any questions or encounter
> >> >> any problems.
> >> >> Vince
> >> >> On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google) <apija...@google.com>
> >> >> wrote:
> >> >> > Hi David. This may be coming to Java eventually, but it hasn't been
> >> >> > started
> >> >> > yet. If you or anyone else is interested in contributing, let me
> >> >> > know.
> >> >> > - Jason
> >> >> > On Wed, Oct 28, 2009 at 7:52 AM, David Chandler
> >> >> > <turboman...@gmail.com>
> >> >> > wrote:
Attached is a modified version of that class that lets you define any
path you want for the servlet and lets you specify which queue to use
like this: Deferred.defer(task, "queueName");
(I needed this for my own purposes)
Do with it as you wish :-)
The only other major change I would make is to stop masking all the
exceptions during the task processing. Or at least, if you're going
to log the exception and stop propagation, log the whole thing so we
get a stacktrace in the logs.
On Fri, Nov 20, 2009 at 1:03 PM, David Chandler <turboman...@gmail.com> wrote:
> Vince, this is great! I hadn't been watching my own issue, so didn't
> see this until now. Thanks so much!
> Only enhancement I would suggest is to enable multiple deferred
> queues. For example. I would like to defer tasks in an email throttle
> queue separately from a general background queue. Perhaps the
> Deferrable interface could have a getQueueName() method, or
> Deferred.defer could have an additional signature defer(Deferrable
> task, String queueName).
Thanks for the suggestions and the code. David Chandler sent me a patch to
support user-specified queue names (almost exactly the same as your
changes), and I've committed that patch to SVN. Regarding your other
changes:
- I've probably make the <url-pattern> init parameter optional and have it
default to "/_ah/<queue_name>" rather than throw an exception from the init
method if it's missing.
- While we're at it, the default queue name could be optionally specified
by an init parameter also.
- I'm not sure a stacktrace from the doPost method is really going to give
you much more information, but would be open to that change.
It looks like the Deferred class is going to be added to the official GAE/J
SDK, though I'm not sure when. Rather than make any more changes to the code
right now, I'd rather wait to see what shows up the SDK and then work
through normal Google channels to get any further modifications made. In the
mean time, I'm glad it's working for you.
On Tue, Nov 24, 2009 at 8:09 PM, Jeff Schnitzer <j...@infohazard.org> wrote:
> Attached is a modified version of that class that lets you define any
> path you want for the servlet and lets you specify which queue to use
> like this: Deferred.defer(task, "queueName");
> (I needed this for my own purposes)
> Do with it as you wish :-)
> The only other major change I would make is to stop masking all the
> exceptions during the task processing. Or at least, if you're going
> to log the exception and stop propagation, log the whole thing so we
> get a stacktrace in the logs.
> Jeff
> On Fri, Nov 20, 2009 at 1:03 PM, David Chandler <turboman...@gmail.com>
> wrote:
> > Vince, this is great! I hadn't been watching my own issue, so didn't
> > see this until now. Thanks so much!
> > Only enhancement I would suggest is to enable multiple deferred
> > queues. For example. I would like to defer tasks in an email throttle
> > queue separately from a general background queue. Perhaps the
> > Deferrable interface could have a getQueueName() method, or
> > Deferred.defer could have an additional signature defer(Deferrable
> > task, String queueName).
I should have said this before, but thanks for providing this code!
For reasons that are still not clear to me, my task was not
deserializing properly (very odd complaints about HashMap). Not
having a stack trace made it even more confusing than usual.
Ultimately I fixed the problem by making my class Externalizable and
serializing my object graph as JSON.
I would definitely want to see deserialization exceptions in the logs.
Even if the container is behaving perfectly rationally, developers
can easily hork up the process with Externalizable. What is to be
gained by hiding the stack trace?
On Wed, Nov 25, 2009 at 11:03 AM, Vince Bonfanti <vbonfa...@gmail.com> wrote:
> Hi Jeff,
> Thanks for the suggestions and the code. David Chandler sent me a patch to
> support user-specified queue names (almost exactly the same as your
> changes), and I've committed that patch to SVN. Regarding your other
> changes:
> - I've probably make the <url-pattern> init parameter optional and have it
> default to "/_ah/<queue_name>" rather than throw an exception from the init
> method if it's missing.
> - While we're at it, the default queue name could be optionally specified
> by an init parameter also.
> - I'm not sure a stacktrace from the doPost method is really going to give
> you much more information, but would be open to that change.
> It looks like the Deferred class is going to be added to the official GAE/J
> SDK, though I'm not sure when. Rather than make any more changes to the code
> right now, I'd rather wait to see what shows up the SDK and then work
> through normal Google channels to get any further modifications made. In the
> mean time, I'm glad it's working for you.
> Vince
> On Tue, Nov 24, 2009 at 8:09 PM, Jeff Schnitzer <j...@infohazard.org> wrote:
>> Attached is a modified version of that class that lets you define any
>> path you want for the servlet and lets you specify which queue to use
>> like this: Deferred.defer(task, "queueName");
>> (I needed this for my own purposes)
>> Do with it as you wish :-)
>> The only other major change I would make is to stop masking all the
>> exceptions during the task processing. Or at least, if you're going
>> to log the exception and stop propagation, log the whole thing so we
>> get a stacktrace in the logs.
>> Jeff
>> On Fri, Nov 20, 2009 at 1:03 PM, David Chandler <turboman...@gmail.com>
>> wrote:
>> > Vince, this is great! I hadn't been watching my own issue, so didn't
>> > see this until now. Thanks so much!
>> > Only enhancement I would suggest is to enable multiple deferred
>> > queues. For example. I would like to defer tasks in an email throttle
>> > queue separately from a general background queue. Perhaps the
>> > Deferrable interface could have a getQueueName() method, or
>> > Deferred.defer could have an additional signature defer(Deferrable
>> > task, String queueName).
>> > Thanks again,
>> > /dmc
>> --
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to google-appengine-java@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
I haven't had a chance to integrate your code yet, so feel free to continue
making modifications. In particular, I think it would be useful to provide
the same set of options the Python deferred API accepts, which include 'eta'
and 'countdown' parameters, as well as specifying the queue and URL.
On Wed, Nov 25, 2009 at 7:03 PM, Vince Bonfanti <vbonfa...@gmail.com> wrote:
> Hi Jeff,
> Thanks for the suggestions and the code. David Chandler sent me a patch to
> support user-specified queue names (almost exactly the same as your
> changes), and I've committed that patch to SVN. Regarding your other
> changes:
> - I've probably make the <url-pattern> init parameter optional and have
> it default to "/_ah/<queue_name>" rather than throw an exception from the
> init method if it's missing.
> - While we're at it, the default queue name could be optionally specified
> by an init parameter also.
> - I'm not sure a stacktrace from the doPost method is really going to
> give you much more information, but would be open to that change.
> It looks like the Deferred class is going to be added to the official GAE/J
> SDK, though I'm not sure when. Rather than make any more changes to the code
> right now, I'd rather wait to see what shows up the SDK and then work
> through normal Google channels to get any further modifications made. In the
> mean time, I'm glad it's working for you.
> Vince
> On Tue, Nov 24, 2009 at 8:09 PM, Jeff Schnitzer <j...@infohazard.org>wrote:
>> Attached is a modified version of that class that lets you define any
>> path you want for the servlet and lets you specify which queue to use
>> like this: Deferred.defer(task, "queueName");
>> (I needed this for my own purposes)
>> Do with it as you wish :-)
>> The only other major change I would make is to stop masking all the
>> exceptions during the task processing. Or at least, if you're going
>> to log the exception and stop propagation, log the whole thing so we
>> get a stacktrace in the logs.
>> Jeff
>> On Fri, Nov 20, 2009 at 1:03 PM, David Chandler <turboman...@gmail.com>
>> wrote:
>> > Vince, this is great! I hadn't been watching my own issue, so didn't
>> > see this until now. Thanks so much!
>> > Only enhancement I would suggest is to enable multiple deferred
>> > queues. For example. I would like to defer tasks in an email throttle
>> > queue separately from a general background queue. Perhaps the
>> > Deferrable interface could have a getQueueName() method, or
>> > Deferred.defer could have an additional signature defer(Deferrable
>> > task, String queueName).
>> > Thanks again,
>> > /dmc
>> --
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-java@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscribe@googlegroups.com<google-appengine-java%2B unsubscribe@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
-- Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
I'm seeing problems with deserialization, too, when deployed on
AppEngine. In dev, I can deserialize(serialize(task)) and it works
just fine, but not so in AppEngine. I get the same error whether the
task payload is the serialized Deferrable task itself or just the Key
with the task in the db, but I haven't yet figured out which is the
cause of the problem.
The error I'm getting on AppEngine is invalid type code: 00 in
Deferred.deserialize method. I've tried Base64 encoding after
serialization in case it's related to special binary escape sequences
that happen to be in the serialized object stream but still no
success.
I'm pretty sure AppEngine has written some custom code to do
serialization because I previously got an AppEngine acess control
exception when calling enableResolveObject(true) in a subclass of
ObjectInputStream, and I'm wondering if that's in play somehow...
/dmc
On Nov 26, 5:31 am, Jeff Schnitzer <j...@infohazard.org> wrote:
> I should have said this before, but thanks for providing this code!
> For reasons that are still not clear to me, my task was not
> deserializing properly (very odd complaints about HashMap). Not
> having a stack trace made it even more confusing than usual.
> Ultimately I fixed the problem by making my class Externalizable and
> serializing my object graph as JSON.
> I would definitely want to see deserialization exceptions in the logs.
> Even if the container is behaving perfectly rationally, developers
> can easily hork up the process with Externalizable. What is to be
> gained by hiding the stack trace?
> Thanks,
> JeffOn Wed, Nov 25, 2009 at 11:03 AM, Vince Bonfanti <vbonfa...@gmail.com> wrote:
> > Hi Jeff,
> > Thanks for the suggestions and the code. David Chandler sent me a patch to
> > support user-specified queue names (almost exactly the same as your
> > changes), and I've committed that patch to SVN. Regarding your other
> > changes:
> > - I've probably make the <url-pattern> init parameter optional and have it
> > default to "/_ah/<queue_name>" rather than throw an exception from the init
> > method if it's missing.
> > - While we're at it, the default queue name could be optionally specified
> > by an init parameter also.
> > - I'm not sure a stacktrace from the doPost method is really going to give
> > you much more information, but would be open to that change.
> > It looks like the Deferred class is going to be added to the official GAE/J
> > SDK, though I'm not sure when. Rather than make any more changes to the code
> > right now, I'd rather wait to see what shows up the SDK and then work
> > through normal Google channels to get any further modifications made. In the
> > mean time, I'm glad it's working for you.
> > Vince
> > On Tue, Nov 24, 2009 at 8:09 PM, Jeff Schnitzer <j...@infohazard.org> wrote:
> >> Attached is a modified version of that class that lets you define any
> >> path you want for the servlet and lets you specify which queue to use
> >> like this: Deferred.defer(task, "queueName");
> >> (I needed this for my own purposes)
> >> Do with it as you wish :-)
> >> The only other major change I would make is to stop masking all the
> >> exceptions during the task processing. Or at least, if you're going
> >> to log the exception and stop propagation, log the whole thing so we
> >> get a stacktrace in the logs.
> >> Jeff
> >> On Fri, Nov 20, 2009 at 1:03 PM, David Chandler <turboman...@gmail.com>
> >> wrote:
> >> > Vince, this is great! I hadn't been watching my own issue, so didn't
> >> > see this until now. Thanks so much!
> >> > Only enhancement I would suggest is to enable multiple deferred
> >> > queues. For example. I would like to defer tasks in an email throttle
> >> > queue separately from a general background queue. Perhaps the
> >> > Deferrable interface could have a getQueueName() method, or
> >> > Deferred.defer could have an additional signature defer(Deferrable
> >> > task, String queueName).
> >> > Thanks again,
> >> > /dmc
> >> --
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine for Java" group.
> > To post to this group, send email to google-appengine-java@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine-java+unsubscribe@googlegroups.com.
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine-java?hl=en.
I've solved the serialization problems by Base64 encoding the
serialized task before calling payload() and decoding it in the
deserialize(HttpServletRequest) method. I'm guessing something in the
task queue chain (either task queue payload storage or the servlet
call when task is run) has problems transmitting the binary data, as
the exception I was getting at one point was a
java.io.StreamCorruptedException.
I'll send another patch to Vince. My solution works, but I'm not
completely satisfied that I've diagnosed the problem correctly. I used
Apache commons-codec for Base64. Is it safe to use com.sun.misc
instead? I see it in appengine.repackaged chain, as well, but don't
see it as part of an official Google API.
Somewhat related, I've wired up the Deferred servlet through the
GuiceFilter so I can pass a Guice injector to Deferrable tasks. Since
the task objects are not created through Guice, but rather by
deserializing, I modified doTask() to accept an additional Injector
argument. We probably don't want that in the official version of
Deferrable, though.
Is there a better way for my Deferrable classes to get access to a
Guice "context"? I'm trying to avoid Guice.createInjector(), as it
results in a duplicate PersistenceManagerFactory being created since
other servlets are also being served through Guice.
Thank you,
/dmc
On Nov 27, 5:42 pm, David Chandler <turboman...@gmail.com> wrote:
> I'm seeing problems with deserialization, too, when deployed on
> AppEngine. In dev, I can deserialize(serialize(task)) and it works
> just fine, but not so in AppEngine. I get the same error whether the
> task payload is the serialized Deferrable task itself or just the Key
> with the task in the db, but I haven't yet figured out which is the
> cause of the problem.
> The error I'm getting on AppEngine is invalid type code: 00 in
> Deferred.deserialize method. I've tried Base64 encoding after
> serialization in case it's related to special binary escape sequences
> that happen to be in the serialized object stream but still no
> success.
> I'm pretty sure AppEngine has written some custom code to do
> serialization because I previously got an AppEngine acess control
> exception when calling enableResolveObject(true) in a subclass of
> ObjectInputStream, and I'm wondering if that's in play somehow...
> /dmc
> On Nov 26, 5:31 am, Jeff Schnitzer <j...@infohazard.org> wrote:
> > I should have said this before, but thanks for providing this code!
> > For reasons that are still not clear to me, my task was not
> > deserializing properly (very odd complaints about HashMap). Not
> > having a stack trace made it even more confusing than usual.
> > Ultimately I fixed the problem by making my class Externalizable and
> > serializing my object graph as JSON.
> > I would definitely want to see deserialization exceptions in the logs.
> > Even if the container is behaving perfectly rationally, developers
> > can easily hork up the process with Externalizable. What is to be
> > gained by hiding the stack trace?
> > Thanks,
> > JeffOn Wed, Nov 25, 2009 at 11:03 AM, Vince Bonfanti <vbonfa...@gmail.com> wrote:
> > > Hi Jeff,
> > > Thanks for the suggestions and the code. David Chandler sent me a patch to
> > > support user-specified queue names (almost exactly the same as your
> > > changes), and I've committed that patch to SVN. Regarding your other
> > > changes:
> > > - I've probably make the <url-pattern> init parameter optional and have it
> > > default to "/_ah/<queue_name>" rather than throw an exception from the init
> > > method if it's missing.
> > > - While we're at it, the default queue name could be optionally specified
> > > by an init parameter also.
> > > - I'm not sure a stacktrace from the doPost method is really going to give
> > > you much more information, but would be open to that change.
> > > It looks like the Deferred class is going to be added to the official GAE/J
> > > SDK, though I'm not sure when. Rather than make any more changes to the code
> > > right now, I'd rather wait to see what shows up the SDK and then work
> > > through normal Google channels to get any further modifications made. In the
> > > mean time, I'm glad it's working for you.
> > > Vince
> > > On Tue, Nov 24, 2009 at 8:09 PM, Jeff Schnitzer <j...@infohazard.org> wrote:
> > >> Attached is a modified version of that class that lets you define any
> > >> path you want for the servlet and lets you specify which queue to use
> > >> like this: Deferred.defer(task, "queueName");
> > >> (I needed this for my own purposes)
> > >> Do with it as you wish :-)
> > >> The only other major change I would make is to stop masking all the
> > >> exceptions during the task processing. Or at least, if you're going
> > >> to log the exception and stop propagation, log the whole thing so we
> > >> get a stacktrace in the logs.
> > >> Jeff
> > >> On Fri, Nov 20, 2009 at 1:03 PM, David Chandler <turboman...@gmail.com>
> > >> wrote:
> > >> > Vince, this is great! I hadn't been watching my own issue, so didn't
> > >> > see this until now. Thanks so much!
> > >> > Only enhancement I would suggest is to enable multiple deferred
> > >> > queues. For example. I would like to defer tasks in an email throttle
> > >> > queue separately from a general background queue. Perhaps the
> > >> > Deferrable interface could have a getQueueName() method, or
> > >> > Deferred.defer could have an additional signature defer(Deferrable
> > >> > task, String queueName).
> > >> > Thanks again,
> > >> > /dmc
> > >> --
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "Google App Engine for Java" group.
> > > To post to this group, send email to google-appengine-java@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > > google-appengine-java+unsubscribe@googlegroups.com.
> > > For more options, visit this group at
> > >http://groups.google.com/group/google-appengine-java?hl=en.
- Added additional overloads to the defer() method to allow optional
TaskOptions to be specified. This lets you set the following task options:
url, countdownMIllis, etaMillis, taskName.
- Added support for optional "queueName" and "taskUrl" init parameters.
- The defer() method and its overloads no longer throw IOException;
instead, an unchecked QueueFailureException is thrown if there's a failure
to serialize the task object (Deferrable instance).
- The defer() method and its overloads now return a TaskHandle.
- A stacktrace is now logged upon failure to deserialize the task.
- Updated javadoc comments.
For convenience, here's the direct link to the source:
> I haven't had a chance to integrate your code yet, so feel free to continue
> making modifications. In particular, I think it would be useful to provide
> the same set of options the Python deferred API accepts, which include 'eta'
> and 'countdown' parameters, as well as specifying the queue and URL.
> -Nick
> On Wed, Nov 25, 2009 at 7:03 PM, Vince Bonfanti <vbonfa...@gmail.com>wrote:
>> Hi Jeff,
>> Thanks for the suggestions and the code. David Chandler sent me a patch to
>> support user-specified queue names (almost exactly the same as your
>> changes), and I've committed that patch to SVN. Regarding your other
>> changes:
>> - I've probably make the <url-pattern> init parameter optional and have
>> it default to "/_ah/<queue_name>" rather than throw an exception from the
>> init method if it's missing.
>> - While we're at it, the default queue name could be optionally
>> specified by an init parameter also.
>> - I'm not sure a stacktrace from the doPost method is really going to
>> give you much more information, but would be open to that change.
>> It looks like the Deferred class is going to be added to the official
>> GAE/J SDK, though I'm not sure when. Rather than make any more changes to
>> the code right now, I'd rather wait to see what shows up the SDK and then
>> work through normal Google channels to get any further modifications made.
>> In the mean time, I'm glad it's working for you.
>> Vince
>> On Tue, Nov 24, 2009 at 8:09 PM, Jeff Schnitzer <j...@infohazard.org>wrote:
>>> Attached is a modified version of that class that lets you define any
>>> path you want for the servlet and lets you specify which queue to use
>>> like this: Deferred.defer(task, "queueName");
>>> (I needed this for my own purposes)
>>> Do with it as you wish :-)
>>> The only other major change I would make is to stop masking all the
>>> exceptions during the task processing. Or at least, if you're going
>>> to log the exception and stop propagation, log the whole thing so we
>>> get a stacktrace in the logs.
The behavior you're reporting is exactly the opposite of what I'm seeing.
For me, deserialization always fails on the dev server and always works in
production. What I'm seeing on the dev server is that the bytes retrieved
from the task payload (req.getInputStream) are not the same bytes that are
sent; the result is the StreamCorruptedException. Again, though, I only see
this on the dev server and never in production. Can you send me an example
that fails in production? I'd like to understand what's going on before
applying the Base64 encoder workaround.
I'm not at all familiar with Guice, but I think I understand what you're
trying to do. One thought I've had is that we could pass the
HttpServletRequest object to the doTask() method. This would allow you to
add attributes (objects) to the HttpServletRequest within your Guice filter
and then retrieve them within your doTask() method.
Vince
On Fri, Nov 27, 2009 at 7:53 PM, David Chandler <turboman...@gmail.com>wrote:
> I've solved the serialization problems by Base64 encoding the
> serialized task before calling payload() and decoding it in the
> deserialize(HttpServletRequest) method. I'm guessing something in the
> task queue chain (either task queue payload storage or the servlet
> call when task is run) has problems transmitting the binary data, as
> the exception I was getting at one point was a
> java.io.StreamCorruptedException.
> I'll send another patch to Vince. My solution works, but I'm not
> completely satisfied that I've diagnosed the problem correctly. I used
> Apache commons-codec for Base64. Is it safe to use com.sun.misc
> instead? I see it in appengine.repackaged chain, as well, but don't
> see it as part of an official Google API.
> Somewhat related, I've wired up the Deferred servlet through the
> GuiceFilter so I can pass a Guice injector to Deferrable tasks. Since
> the task objects are not created through Guice, but rather by
> deserializing, I modified doTask() to accept an additional Injector
> argument. We probably don't want that in the official version of
> Deferrable, though.
> Is there a better way for my Deferrable classes to get access to a
> Guice "context"? I'm trying to avoid Guice.createInjector(), as it
> results in a duplicate PersistenceManagerFactory being created since
> other servlets are also being served through Guice.
> Thank you,
> /dmc
> On Nov 27, 5:42 pm, David Chandler <turboman...@gmail.com> wrote:
> > Jeff,
> > I'm seeing problems with deserialization, too, when deployed on
> > AppEngine. In dev, I can deserialize(serialize(task)) and it works
> > just fine, but not so in AppEngine. I get the same error whether the
> > task payload is the serialized Deferrable task itself or just the Key
> > with the task in the db, but I haven't yet figured out which is the
> > cause of the problem.
> > The error I'm getting on AppEngine is invalid type code: 00 in
> > Deferred.deserialize method. I've tried Base64 encoding after
> > serialization in case it's related to special binary escape sequences
> > that happen to be in the serialized object stream but still no
> > success.
> > I'm pretty sure AppEngine has written some custom code to do
> > serialization because I previously got an AppEngine acess control
> > exception when calling enableResolveObject(true) in a subclass of
> > ObjectInputStream, and I'm wondering if that's in play somehow...
Regarding a way to access the Guice injector, I've figured out I can
simply create and access the injector via a static factory, so no
changes are needed to Deferred or Deferrable. This works for AppEngine
since there is only one WAR per JVM. If there were more than web app,
I'm not sure whether Guice ServletModules would work, but that's not
an issue for now.
> The behavior you're reporting is exactly the opposite of what I'm seeing.
> For me, deserialization always fails on the dev server and always works in
> production. What I'm seeing on the dev server is that the bytes retrieved
> from the task payload (req.getInputStream) are not the same bytes that are
> sent; the result is the StreamCorruptedException. Again, though, I only see
> this on the dev server and never in production. Can you send me an example
> that fails in production? I'd like to understand what's going on before
> applying the Base64 encoder workaround.
> I'm not at all familiar with Guice, but I think I understand what you're
> trying to do. One thought I've had is that we could pass the
> HttpServletRequest object to the doTask() method. This would allow you to
> add attributes (objects) to the HttpServletRequest within your Guice filter
> and then retrieve them within your doTask() method.
> Vince
> On Fri, Nov 27, 2009 at 7:53 PM, David Chandler <turboman...@gmail.com>wrote:
> > I've solved the serialization problems by Base64 encoding the
> > serialized task before calling payload() and decoding it in the
> > deserialize(HttpServletRequest) method. I'm guessing something in the
> > task queue chain (either task queue payload storage or the servlet
> > call when task is run) has problems transmitting the binary data, as
> > the exception I was getting at one point was a
> > java.io.StreamCorruptedException.
> > I'll send another patch to Vince. My solution works, but I'm not
> > completely satisfied that I've diagnosed the problem correctly. I used
> > Apache commons-codec for Base64. Is it safe to use com.sun.misc
> > instead? I see it in appengine.repackaged chain, as well, but don't
> > see it as part of an official Google API.
> > Somewhat related, I've wired up the Deferred servlet through the
> > GuiceFilter so I can pass a Guice injector to Deferrable tasks. Since
> > the task objects are not created through Guice, but rather by
> > deserializing, I modified doTask() to accept an additional Injector
> > argument. We probably don't want that in the official version of
> > Deferrable, though.
> > Is there a better way for my Deferrable classes to get access to a
> > Guice "context"? I'm trying to avoid Guice.createInjector(), as it
> > results in a duplicate PersistenceManagerFactory being created since
> > other servlets are also being served through Guice.
> > Thank you,
> > /dmc
> > On Nov 27, 5:42 pm, David Chandler <turboman...@gmail.com> wrote:
> > > Jeff,
> > > I'm seeing problems with deserialization, too, when deployed on
> > > AppEngine. In dev, I can deserialize(serialize(task)) and it works
> > > just fine, but not so in AppEngine. I get the same error whether the
> > > task payload is the serialized Deferrable task itself or just the Key
> > > with the task in the db, but I haven't yet figured out which is the
> > > cause of the problem.
> > > The error I'm getting on AppEngine is invalid type code: 00 in
> > > Deferred.deserialize method. I've tried Base64 encoding after
> > > serialization in case it's related to special binary escape sequences
> > > that happen to be in the serialized object stream but still no
> > > success.
> > > I'm pretty sure AppEngine has written some custom code to do
> > > serialization because I previously got an AppEngine acess control
> > > exception when calling enableResolveObject(true) in a subclass of
> > > ObjectInputStream, and I'm wondering if that's in play somehow...
I can't seem to find any additional postings on this topic and I'm not sure what the conclusion for being able to use the Deferred capability with the latest Java SDK release. I'm just wondering if Vince's solution works on both server and development environments.
On Dec 16, 10:42 am, David Chandler <turboman...@gmail.com> wrote:
> Regarding a way to access the Guice injector, I've figured out I can > simply create and access the injector via a static factory, so no > changes are needed to Deferred or Deferrable. This works for AppEngine > since there is only one WAR per JVM. If there were more than web app, > I'm not sure whether Guice ServletModules would work, but that's not > an issue for now.
> On Dec 2, 2:47 pm, Vince Bonfanti <vbonfa...@gmail.com> wrote:
> > The behavior you're reporting is exactly the opposite of what I'm seeing. > > For me, deserialization always fails on the dev server and always works in > > production. What I'm seeing on the dev server is that the bytes retrieved > > from the task payload (req.getInputStream) are not the same bytes that are > > sent; the result is the StreamCorruptedException. Again, though, I only see > > this on the dev server and never in production. Can you send me an example > > that fails in production? I'd like to understand what's going on before > > applying the Base64 encoder workaround.
> > I'm not at all familiar with Guice, but I think I understand what you're > > trying to do. One thought I've had is that we could pass the > > HttpServletRequest object to the doTask() method. This would allow you to > > add attributes (objects) to the HttpServletRequest within your Guice filter > > and then retrieve them within your doTask() method.
> > Vince
> > On Fri, Nov 27, 2009 at 7:53 PM, David Chandler <turboman...@gmail.com>wrote:
> > > I've solved the serialization problems by Base64 encoding the > > > serialized task before calling payload() and decoding it in the > > > deserialize(HttpServletRequest) method. I'm guessing something in the > > > task queue chain (either task queue payload storage or the servlet > > > call when task is run) has problems transmitting the binary data, as > > > the exception I was getting at one point was a > > > java.io.StreamCorruptedException.
> > > I'll send another patch to Vince. My solution works, but I'm not > > > completely satisfied that I've diagnosed the problem correctly. I used > > > Apache commons-codec for Base64. Is it safe to use com.sun.misc > > > instead? I see it in appengine.repackaged chain, as well, but don't > > > see it as part of an official Google API.
> > > Somewhat related, I've wired up the Deferred servlet through the > > > GuiceFilter so I can pass a Guice injector to Deferrable tasks. Since > > > the task objects are not created through Guice, but rather by > > > deserializing, I modified doTask() to accept an additional Injector > > > argument. We probably don't want that in the official version of > > > Deferrable, though.
> > > Is there a better way for my Deferrable classes to get access to a > > > Guice "context"? I'm trying to avoid Guice.createInjector(), as it > > > results in a duplicate PersistenceManagerFactory being created since > > > other servlets are also being served through Guice.
> > > Thank you, > > > /dmc
> > > On Nov 27, 5:42 pm, David Chandler <turboman...@gmail.com> wrote: > > > > Jeff,
> > > > I'm seeing problems with deserialization, too, when deployed on > > > > AppEngine. In dev, I can deserialize(serialize(task)) and it works > > > > just fine, but not so in AppEngine. I get the same error whether the > > > > task payload is the serialized Deferrable task itself or just the Key > > > > with the task in the db, but I haven't yet figured out which is the > > > > cause of the problem.
> > > > The error I'm getting on AppEngine is invalid type code: 00 in > > > > Deferred.deserialize method. I've tried Base64 encoding after > > > > serialization in case it's related to special binary escape sequences > > > > that happen to be in the serialized object stream but still no > > > > success.
> > > > I'm pretty sure AppEngine has written some custom code to do > > > > serialization because I previously got an AppEngine acess control > > > > exception when calling enableResolveObject(true) in a subclass of > > > > ObjectInputStream, and I'm wondering if that's in play somehow...
On Thu, Dec 31, 2009 at 5:19 PM, John Howe <jhowe...@gmail.com> wrote: > I can't seem to find any additional postings on this topic and I'm not > sure what the conclusion for being able to use the Deferred capability > with the latest Java SDK release. I'm just wondering if Vince's > solution works on both server and development environments.
> On Dec 16, 10:42 am, David Chandler <turboman...@gmail.com> wrote: > > Thanks, Vince.
> > Regarding a way to access the Guice injector, I've figured out I can > > simply create and access the injector via a static factory, so no > > changes are needed to Deferred or Deferrable. This works for AppEngine > > since there is only one WAR per JVM. If there were more than web app, > > I'm not sure whether Guice ServletModules would work, but that's not > > an issue for now.
> > On Dec 2, 2:47 pm, Vince Bonfanti <vbonfa...@gmail.com> wrote:
> > > The behavior you're reporting is exactly the opposite of what I'm > seeing. > > > For me, deserialization always fails on the dev server and always works > in > > > production. What I'm seeing on the dev server is that the bytes > retrieved > > > from the task payload (req.getInputStream) are not the same bytes that > are > > > sent; the result is the StreamCorruptedException. Again, though, I only > see > > > this on the dev server and never in production. Can you send me an > example > > > that fails in production? I'd like to understand what's going on before > > > applying the Base64 encoder workaround.
> > > I'm not at all familiar with Guice, but I think I understand what > you're > > > trying to do. One thought I've had is that we could pass the > > > HttpServletRequest object to the doTask() method. This would allow you > to > > > add attributes (objects) to the HttpServletRequest within your Guice > filter > > > and then retrieve them within your doTask() method.
> > > Vince
> > > On Fri, Nov 27, 2009 at 7:53 PM, David Chandler <turboman...@gmail.com > >wrote:
> > > > I've solved the serialization problems by Base64 encoding the > > > > serialized task before calling payload() and decoding it in the > > > > deserialize(HttpServletRequest) method. I'm guessing something in the > > > > task queue chain (either task queue payload storage or the servlet > > > > call when task is run) has problems transmitting the binary data, as > > > > the exception I was getting at one point was a > > > > java.io.StreamCorruptedException.
> > > > I'll send another patch to Vince. My solution works, but I'm not > > > > completely satisfied that I've diagnosed the problem correctly. I > used > > > > Apache commons-codec for Base64. Is it safe to use com.sun.misc > > > > instead? I see it in appengine.repackaged chain, as well, but don't > > > > see it as part of an official Google API.
> > > > Somewhat related, I've wired up the Deferred servlet through the > > > > GuiceFilter so I can pass a Guice injector to Deferrable tasks. Since > > > > the task objects are not created through Guice, but rather by > > > > deserializing, I modified doTask() to accept an additional Injector > > > > argument. We probably don't want that in the official version of > > > > Deferrable, though.
> > > > Is there a better way for my Deferrable classes to get access to a > > > > Guice "context"? I'm trying to avoid Guice.createInjector(), as it > > > > results in a duplicate PersistenceManagerFactory being created since > > > > other servlets are also being served through Guice.
> > > > Thank you, > > > > /dmc
> > > > On Nov 27, 5:42 pm, David Chandler <turboman...@gmail.com> wrote: > > > > > Jeff,
> > > > > I'm seeing problems with deserialization, too, when deployed on > > > > > AppEngine. In dev, I can deserialize(serialize(task)) and it works > > > > > just fine, but not so in AppEngine. I get the same error whether > the > > > > > task payload is the serialized Deferrable task itself or just the > Key > > > > > with the task in the db, but I haven't yet figured out which is the > > > > > cause of the problem.
> > > > > The error I'm getting on AppEngine is invalid type code: 00 in > > > > > Deferred.deserialize method. I've tried Base64 encoding after > > > > > serialization in case it's related to special binary escape > sequences > > > > > that happen to be in the serialized object stream but still no > > > > > success.
> > > > > I'm pretty sure AppEngine has written some custom code to do > > > > > serialization because I previously got an AppEngine acess control > > > > > exception when calling enableResolveObject(true) in a subclass of > > > > > ObjectInputStream, and I'm wondering if that's in play somehow...
> > > > > /dmc
> --
> You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > google-appengine-java@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscribe@googlegroups.com<google-appengine-java%2B unsubscribe@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en.
