How to graceful shutdown on go111 runtime?

[Masahiro Wakame]

Hi, I'm trying go111 runtime. It is very fine!

I have a question.

I want to move `google.golang.org/appengine/log` to `cloud.google.com/go/logging`.

so, I want to flush the logs when instance will be spindown.

commonly, I think server written in Go that uses graceful shutdown feature.

https://github.com/golang/go/issues/4674

but AppEngine instance can't receive SIGTERM on spindown (isn't it?).

I want to know how to implement log flushing in go111 runtime.

my code is here.

https://github.com/vvakame/til/pull/1

thx!

[Steren Giannini]

Hi,

Thanks for giving go111 a try. We are publishing the log post today

Are you really sure you need to handle log flushing? I would expect the module to handle that for you.

After receiving requests, App Engine keep your instance alive for some minutes before scaling it down if needed. I would think this leaves plenty of time for the logging package to send logs.

--
You received this message because you are subscribed to the Google Groups "google-appengine-go" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengin...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine-go/c9fd9820-664d-4b83-b589-a1770be0f0a8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Shinta HATATANI]

Hi,

I'm also trying go111 runtime and struggling about linking app logs and requests.

> After receiving requests, App Engine keep your instance alive for some minutes before scaling it down if needed.

I saw this behavior for the first time. It seems to be undocumented.

Are the instances which received request recently, excluded from the target of eviction?

I also want to use the structured logging to stdout which is the same as Kubernetes Engine and Flex Environement like this below issue.

https://issuetracker.google.com/issues/115477968

Is it on roadmap?

2018年10月16日火曜日 23時41分15秒 UTC+9 Steren Giannini:

Hi,

Thanks for giving go111 a try. We are publishing the log post today

Are you really sure you need to handle log flushing? I would expect the module to handle that for you.

After receiving requests, App Engine keep your instance alive for some minutes before scaling it down if needed. I would think this leaves plenty of time for the logging package to send logs.

On Tue, Oct 16, 2018 at 12:56 AM Masahiro Wakame <vva...@gmail.com> wrote:

Hi, I'm trying go111 runtime. It is very fine!

I have a question.

I want to move `google.golang.org/appengine/log` to `cloud.google.com/go/logging`.

so, I want to flush the logs when instance will be spindown.

commonly, I think server written in Go that uses graceful shutdown feature.

https://github.com/golang/go/issues/4674

but AppEngine instance can't receive SIGTERM on spindown (isn't it?).

I want to know how to implement log flushing in go111 runtime.

my code is here.

https://github.com/vvakame/til/pull/1

thx!

-- 
You received this message because you are subscribed to the Google Groups "google-appengine-go" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine-go+unsub...@googlegroups.com.

[Steren Giannini]

Hi,

For linking app logs and request logs, we capture how to do so in this doc page: https://cloud.google.com/appengine/docs/standard/go111/writing-application-logs#related-app-logs

We are working to provide samples and probably a library to help you do so

The App Engine scheduling behavior has not changed with Go 1.11 and it the same as every other App Engine Standard runtimes.

For structured logging, I encourage you to use a logging library, is this what you are doing? If so, I recommend opening a bug on the library side, as the library should be able to output structured logs if it prints log entries in the appropriate format.

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengin...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine-go/c9fd9820-664d-4b83-b589-a1770be0f0a8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "google-appengine-go" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengin...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine-go/36c7614a-81b9-42a6-a17d-b519e1436db6%40googlegroups.com.

[Steven Buss]

Thank you for trying the new runtime! Please don't hesitate to email this list with questions; they will help everyone trying it :)

> I want to flush the logs when instance will be spindown.

Like Steren, I don't think you need to manually flush the logs. Your app will stay in a running state for several minutes after serving a request, and log flushing should happen automatically in the background within that time frame. Have you observed lost logs?

> I'm also trying go111 runtime and struggling about linking app logs and requests.

I made a gist that shows how you can do request log grouping: https://gist.github.com/sbuss/d47aea3751a1fbb477f8937d992bc1e0 -- It's a little more complex than I'd like right now, but I'll be working with the stackdriver logging team to see if we can simplify it, or at least make a common helper function.

> I also want to use the structured logging to stdout which is the same as Kubernetes Engine and Flex Environement like this below issue.

> https://issuetracker.google.com/issues/115477968

> Is it on roadmap?

I believe we are planning to add support for this, but I don't have a time estimate yet. 

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine-go/CANHKq5GrNu6%2BWqNZqJ3AX27QBab1LRjOxMmdCKBjGG8zSRLQTw%40mail.gmail.com.

[Shinta HATATANI]

Hi,

1. log flushing outside of request

I had linked the app logs and requests successfully but I was not sure to achieve the both low latency and preventing loss of logs.

I didn't know the automatic scaling of App Engine Standard keeps the recently used instances for minutes before I read this thread.

(Cloud Functions freezes instance outside of function execution so flushing seems to be more important.)

I understood it is safe to use Stackdriver Logging client library without forcing to flush logs within request execution.

2. structured logging

I was trying to emit the structure log to stdout but it seems not to be supported yet.

I understood the structured logging is planned(but no ETA currently).

2018年10月17日水曜日 0時53分24秒 UTC+9 Steren Giannini:

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine-go+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine-go/c9fd9820-664d-4b83-b589-a1770be0f0a8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "google-appengine-go" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine-go+unsub...@googlegroups.com.

[Masahiro Wakame]

Thank you for your all advices!

I was satisfied with the reply shortly.

My code will be works fine!

> It's a little more complex than I'd like right now, but I'll be working with the stackdriver logging team to see if we can simplify it, or at least make a common helper function.

Sounds great! cloud.google.com/go/logging is too difficult to setup for me.

it got help from HATATANI that made it possible (before post this thread).

so, In the long term, I want to make my app works fine in non-AppEngine environment (for example, GKE...).

because I want to make it work properly in the general way in golang.

We can write logging and tracing (and more) feature by same code phrase as GKE. It's really wonderful and exciting for me!

I'm happy if we can write a shutdown code like GKE ;)

thanks!

2018年10月17日水曜日 1時06分55秒 UTC+9 Steven Buss:

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine-go+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine-go/c9fd9820-664d-4b83-b589-a1770be0f0a8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "google-appengine-go" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine-go+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine-go/36c7614a-81b9-42a6-a17d-b519e1436db6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "google-appengine-go" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine-go+unsub...@googlegroups.com.