Stackdriver Structured Logging from App Engine Flex

[Janghwan Lee]

Google App Engine flexible environment automatically pipes stdout and stderr to Stackdriver (Google Cloud Logging). But this only supports plain text log message without any metadata (not even logging levels).

in this document, it says app logs can be text or structured, But I couldn't find a way to log structured message from Java application.

https://cloud.google.com/appengine/articles/logging#appengine-vms

I found Logback appender for google cloud logging.

https://cloud.google.com/logging/docs/setup/java

But this does not seem to support structured logging yet. And also, it makes GRPC calls for every log entry under the hood. So, I wonder how scalable it is (especially compare to current app engine structure which has separate Fluentd agent handles logs).

Is there any out of box or simple solution to send structured log from App Engine to Stackdriver?

I asked the same question in Stackoverflow, but they don't seem to be interested in app engine much, so I ask it again here.

[Jordan (Cloud Platform Support)]

If you are having issues with your stdout and stderr messages not appearing in Stackdriver Logs Viewer, you can directly log to Stackdriver Logging via the Google Cloud Java Client for Logging. 

- Note: Google Groups is reserved for general product discussions and is not for technical support. If you require further technical support for logging to Stackdriver, it is recommended to post your detailed questions to Stack Exchange using the supported Cloud tags.  

[Janghwan Lee]

I am not having issue with stdout messages. I am just talking about its limited capability.
Why don’t you support structured logging when there is already logback(which is de facto standard) appender and fluentd agents running on app engine nodes which is very performent.
It would be much simpler and more performant solution then directly implementing logging API.
I found a github issue and related PR created many months ago which implemented what i mentioned, but it’s not merged yet somehow.
https://github.com/GoogleCloudPlatform/google-cloud-java/issues/2669

P.S I’m not sure what “general discussions” is if this is not.

On Mon, Jun 25, 2018 at 1:45 PM 'Jordan (Cloud Platform Support)' via Google App Engine <google-a...@googlegroups.com> wrote:

If you are having issues with your stdout and stderr messages not appearing in Stackdriver Logs Viewer, you can directly log to Stackdriver Logging via the Google Cloud Java Client for Logging. 

- Note: Google Groups is reserved for general product discussions and is not for technical support. If you require further technical support for logging to Stackdriver, it is recommended to post your detailed questions to Stack Exchange using the supported Cloud tags.  

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/sybth9EVEEk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/9e08da63-a43b-42fb-b3c0-51684db52a76%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jordan (Cloud Platform Support)]

For clarity, the fluentd logging agent running as a side-car container in the Flexible environment tails log files in /var/log/app_engine/app and sends them to Stackdriver via the same Stackdriver Logging API. Using the API directly in your application's container would be faster than if your application were to write the logs out to the disk, then wait for the fluentd agent to read it and make the same API call. 

Of course it would be much cleaner to keep everything local to stdout and stderr, and this is indeed a valid feature request. Since Google Groups is reserved for general discussions (and not for reporting issues, feature requests, or code assistance), it is better to continue communicating via the already existing feature request on the Google Github issue tracker.