Taskcluster Structured Logging Landing Soon

[Brian Stack]

Hi all,

If you never look at the papertrail logs/signalfx dashboards/sentry for the
taskcluster services themselves, you can happily archive this email and
move on with your day.

For the rest of us, some pretty big changes are coming:

1. We have added new log drains to our heroku services so that they log to
Stackdriver in GCP. Any members of the taskcluster team have already been
added to this project. If you would like access, just ask me and I'll add
you as well! Links to resources below.
Logs will continue to exist in papertrail for now as well.

2. We will deploy services with structured logging today. This will remove
Signalfx and Sentry support but Stackdriver will replace those for our use
cases. The log format is mozlog with some extra bits tacked on for
stackdriver's purposes. Anything that is built to work with mozlog should
work for our new logs.

This should increase the robustness of our error reporting and make things
simpler in general. I'm sure there will be a period of getting used to the
new way of things working but also feel free to message me anytime with
feedback.

Links:
Google Project that contains heroku logs
<https://console.cloud.google.com/home/dashboard?organizationId=442341870013&project=heroku-logging>
Stackdriver logging for heroku instances
<https://console.cloud.google.com/logs/viewer?project=heroku-logging&folder&organizationId=442341870013&minLogLevel=0&expandAll=false&customFacets&limitCustomFacetWidth=true&interval=JUMP_TO_TIME&resource=gce_instance%2Finstance_id%2Fheroku&scrollTimestamp=2019-02-21T20%3A00%3A41.453117000Z&advancedFilter=resource.type%3D%22gce_instance%22%0Aresource.labels.instance_id%3D%22heroku%22%0A>
Error reporting for heroku apps
<https://console.cloud.google.com/errors?time=P1D&order=COUNT_DESC&resolution=OPEN&resolution=ACKNOWLEDGED&project=heroku-logging&organizationId=442341870013>
Original RFC
<https://github.com/taskcluster/taskcluster-rfcs/blob/master/rfcs/0035-structured-logging.md>

Thanks for all of the hard work various people have put into this
transition!