datastore.*Multi deadline exceeded

[Erik Troan]

It seems like this must be documented somewhere, but I can't find it anywhere.

I'm getting "Canceled: Deadline exceeded (timeout)" once in a while from bulk operations. For DeleteMulti() is seems safe to just retry it, and for GetMulti() it's clearly safe. But what about for PutMulti()? If I have incomplete keys in the PutMulti() and I redo it will I create duplicate entries when I didn't mean to? The "Canceled" part implies that none of the operation has succeeded but I'd like to know that's true before retrying it. I would have expected a MultiError, but in this case it doesn't seem to be what's returned.

While I'm at it, the python api lets you set a custom deadline for operations; is that an option in the go api? Finally, is there a better way to check for the deadline error than

   if e.Error() != "Canceled: Deadline exceeded (timeout)"

It seems like that text is from the rpc layer and isn't exposed as a go error instance?

Thanks.

Erik

[Matthew Zimmerman]

On Thu, Jan 23, 2014 at 11:11 PM, Erik Troan <e...@troan.org> wrote:
> If I have incomplete keys in the PutMulti() and I redo it will I create
> duplicate entries when I didn't mean to? The "Canceled" part implies that
> none of the operation has succeeded but I'd like to know that's true before
> retrying it. I would have expected a MultiError, but in this case it doesn't
> seem to be what's returned.

Good question, I'm not sure, but I'd like to know the answer too.
Have you checked through the keys that have been returned? I think
it's safe to assume that no entries have been created if no keys are
returned, but I could be wrong.

> While I'm at it, the python api lets you set a custom deadline for
> operations; is that an option in the go api?

Use the "timeout" context.
appengine.Timeout(appengine.Context, time.Duration) appengine.Context

>Finally, is there a better way
> to check for the deadline error than
>
> if e.Error() != "Canceled: Deadline exceeded (timeout)"
>
> It seems like that text is from the rpc layer and isn't exposed as a go
> error instance?

if appengine.IsTimeoutError(err) {
// yes it timed out
}

[Glenn Lewis]

Additionally, on this page: https://developers.google.com/appengine/docs/go/datastore/

it says:

• If a write operation reports a timeout error, it cannot be determined (without attempting to read the data) whether the operation succeeded or failed.

Also, I believe that the maximum timeout is 60*time.Second which is enforced by App Engine.

-- Glenn

}

--
You received this message because you are subscribed to the Google Groups "google-appengine-go" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengin...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Erik Troan]

Thanks for all the pointers.

I should probably change the subject for this, but I'll just ask anyway. I regularly find extended periods of time where trying to update my app with appcfg returns 503 or 500 errors (for both the main app and backends). This probably seems to have gotten worse towards the end of this week but is really making me question the suitability of appengine + go for real workloads. Am I unusual in seeing these problems?

Erik

[David Symonds]

On 24 January 2014 19:15, Erik Troan <e...@troan.org> wrote:

> I should probably change the subject for this, but I'll just ask anyway. I
> regularly find extended periods of time where trying to update my app with
> appcfg returns 503 or 500 errors (for both the main app and backends). This
> probably seems to have gotten worse towards the end of this week but is
> really making me question the suitability of appengine + go for real
> workloads. Am I unusual in seeing these problems?

App Engine in general (not just Go) has had some deployment problems
this week. I don't believe it has affected serving though.

[Jeff Huter]

Is this still the case if the PutMulti is done within a transaction?  In other words, will the exceeded deadline trigger a rollback?

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine-go+unsub...@googlegroups.com.

[Glenn Lewis]

Yes, that is my understanding.

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengin...@googlegroups.com.

[Brandon Thomson]

This probably seems to have gotten worse towards the end of this week but is really making me question the suitability of appengine + go for real workloads. Am I unusual in seeing these problems?

I've been using appengine commercially for many years now. Sometimes there are short periods where some deployments error out, but it doesn't mean the service is down. If you retry immediately, there's a good chance it will succeed.

About two years ago I added a couple lines to my deploy script that automatically retry when there's a deployment error, and I haven't noticed any of these hiccups since then. Hopefully they will add automatic retry to "appcfg.py update" eventually.

[Brandon Thomson]

On Mon, Jan 27, 2014 at 02:18:26PM -0500, Erik Troan wrote:
>How often have you been having deployment issues? Since the end part of
>last week we've had a lot of trouble getting code onto appengine. Have you
>been deploying, had no need to deploy...? We are seriously considering
>moving off of appengine now because of the issues we're seeing. We haven't
>been using it all that long, and the problems in the last few days have
>been really discouraging. I'd love to know if it's a "just us" thing these
>past few days.

It definitely wasn't just you. I saw several errors in my deployment logs this
weekend, so I think the problem was affecting everyone. Usually they get those
fixed pretty quick. Today it seemed to be back to normal.

Since Friday I've done 12 deployments and they've all gotten through
eventually.

Were you still getting the errors today? Does it happen every time, or only a
certain percentage of times?

Cheers,
Brandon