PSR for distributed tracing

[Adam Allport]

Hi All,

Observability for web applications is becoming increasingly common, and as such decreasingly interoperable.

Whilst tools like Sentry & Elastic exist, the lack of a standard restricts the ability for frameworks and libraries to add useful spans to the possibly slow processes that they are building.

This lack of interoperability makes integrating or migrating between tools a lengthy process, as you are then required to re-implement all of your tracing efforts.

As such, I suggesting PSR & interface's package to bring the OpenTelemetry spec to PHP in a manner that may push frameworks and libraries to finally standardise.

I look forward to hearing your feedback on this suggestion

Many Thanks

Adam Allport

[Larry Garfield]

I'm not too familiar with this space myself, so a few key questions for such a PSR:

1) How would this be different from/not serviced by PSR-3?

2) Do you have a rough, vague sketch of what such a spec might look like? Or a sample of an existing implementation/API docs you can link to for those of us who don't know the space well?

3) Who are the players in that space that should be involved in such a working group? Off hand it sounds like we'd want someone from Sentry involved, for instance, both for their expertise and to help ensure it actually gets adopted. Have you reached out to them to see if they're interested? (Making a spec from on high and then handing it to 3rd party implementers to adopt doesn't have the greatest track record.)

--Larry Garfield

[Adam Allport]

Hi Larry,

Thanks for your response.

1) Differences to PSR-3

Whilst PSR-3 (Logging) does improve observability within an application, it is just that, logging.

Tracing (sometimes referred to as application performance monitoring/APM) takes it a step further to answer the questions When is my application doing x, and why

2) Rough Sketch of what the spec might look like
Yeah! I think we should consult the methodologies set out by https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/api.md

I believe bringing the interfaces into a PSR could provide the needed encouragement for adoption & standardisation

I will note, opentracing has been adopted as a standard by various large players for other languages.
The lack of a standard for PHP may be holding it back

This would allow developers to integrate various parts of their application to OT systems like Zipkin and Jaeger. In a standardised way.

An implementation could look similar to how Sentry have implemented the tracing section of the library: https://github.com/getsentry/sentry-php/tree/master/src/Tracing

There is an OpenTracing PHP SDK available, however, I do not feel it's appropriate for framework/library developers to add this as a requirement, since it would increase the inherent size of the application.

The PSR would naturally form an interop layer for libraries to contribute to an applications traces, should a TraceProvider be available/provided

The motivation behind this is due to how there are currently (for Laravel projects as an example) 3 distinct ways to handle insights for what an application is doing, each of which have their own, very similar, methods for adding traces to a project

- Tools like Sentry

- Tools like Clockwork

- Tools like Laravel Debugbar 

3) Members who could form a working group

You've rightly identified Sentry, I have not contacted them directly, however, their other implementations do confirm to the OpenTracing spec, which would suggest they would be interested.

There will almost certainly be PHP Developers from the CNCF who may be interested (although not contacted)

Please let me know if this raises any further questions!

A

[Larry Garfield]

On Tue, Mar 15, 2022, at 10:04 AM, Adam Allport wrote:
> Hi Larry,
> Thanks for your response.
>
> 1) Differences to PSR-3
> Whilst PSR-3 (Logging) does improve observability within an
> application, it is just that, logging.
> Tracing (sometimes referred to as application performance
> monitoring/APM) takes it a step further to answer the questions When is

> my application doing *x*, and why

>
> 2) Rough Sketch of what the spec might look like
> Yeah! I think we should consult the methodologies set out by
> https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/api.md
> I believe bringing the interfaces into a PSR could provide the needed
> encouragement for adoption & standardisation
>
> I will note, opentracing has been adopted as a standard by various
> large players for other languages.
> The lack of a standard for PHP may be holding it back
>
> This would allow developers to integrate various parts of their
> application to OT systems like Zipkin and Jaeger. In a standardised way.

> *An* implementation could look similar to how Sentry have implemented

> the tracing section of the library:
> https://github.com/getsentry/sentry-php/tree/master/src/Tracing
> There is an OpenTracing PHP SDK available, however, I do not feel it's
> appropriate for framework/library developers to add this as a
> requirement, since it would increase the inherent size of the
> application.
> The PSR would naturally form an interop layer for libraries to
> contribute to an applications traces, should a TraceProvider be
> available/provided
>
> The motivation behind this is due to how there are currently (for
> Laravel projects as an example) 3 distinct ways to handle insights for
> what an application is doing, each of which have their own, very
> similar, methods for adding traces to a project
> - Tools like Sentry

> <https://docs.sentry.io/platforms/php/performance/instrumentation/>
> - Tools like Clockwork <https://underground.works/clockwork/>

> - Tools like Laravel Debugbar

> <https://github.com/barryvdh/laravel-debugbar>

>
> 3) Members who could form a working group
> You've rightly identified Sentry, I have not contacted them directly,
> however, their other implementations do confirm to the OpenTracing
> spec, which would suggest they would be interested.
> There will almost certainly be PHP Developers from the CNCF who may be
> interested (although not contacted)
>
> Please let me know if this raises any further questions!
> A

That all sounds reasonable. Your next step would probably be to speak to various vendors and see if you can get them on board. If 3-4 vendors showed up and said "hey, we want to standardize a PHP version of a known cross-language spec that we all use", I expect FIG would respond with a loud "yes please!" :-) You'll also need a CC member as a sponsor; I don't have the bandwidth but you can probably find one who does.

(Also, please stick to bottom-posting if someone bottom-posts a thread. It makes it easier to track. Thanks.)

--Larry Garfield

[Adam Allport]

Hi Larry

Sounds good. There's been a bit of discussion in the Discord as to possible vendors to talk to. so I'll see if I can get in contact with a few people!

I will follow up if/when the time comes to seek a sponsor.

Nb: Not entirely sure how to handle bottom-posting when it comes to google groups webUI, as I don't seem to be receiving replies in gmail.
Happy to discuss more on Discord if that works for you :)

Thanks

A

[Brett McBride]

Hello,

The maintainers of opentelemetry-php [1] are supportive of the idea of a PSR for tracing. The project is still in an alpha state so there is plenty of scope for change, but we are also bound by the opentelemetry spec [2].

It's worth noting that opentelemetry is a CNCF project and is itself an attempt to unify many different tracing/APM implementations, and is a successor to OpenTracing.

Most of the big players in APM have contributed to the spec, and several already support the otel protocol directly (elastic and newrelic, and possibly others), or indirectly via exporters in the opentelemetry-collector [3] and opentelemetry-collector-contrib [4] projects.

So, the tracing community is already well on the way to having a standard, and we feel that having a supporting PSR would help to drive adoption of tracing.

Thanks,

Brett

1. https://github.com/open-telemetry/opentelemetry-php

2. https://github.com/open-telemetry/opentelemetry-specification

3. https://opentelemetry.io/docs/collector/

4. https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter

[Ben Edmunds]

I'm very interested in being on a working group for this.  We've strongly considered developing similar in house at Wayfair because there a real need for better OpenTel support in PHP.

[Alessandro Chitolina]

Hello everyone!

It's good to see such an interest in this proposal from the community!

It's a pleasure to inform you that an initial draft of the PSR meta document has been published in this PR: https://github.com/php-fig/fig-standards/pull/1273

Anyone interested is welcomed to comment on the PR, here or on Discord.

If there are no objections or substantial change requests, I'll call for the entrance vote very soon.

Thanks,

Alessandro

[Alessandro Lai]

Hello everyone,

I'm pretty happy to see such commitment. FYI, I'm one of the maintainers of sentry-php (though not officially affiliated to Sentry) so I would love something like this to lean on!

I don't have much bandwidth right now, but I'll forward this discussion to Sentry Community Discord and see if I can fish someone out to collaborate on this!

[Adam Allport]

Hi Alessandro,

Thanks for passing it on!

I will note in case you hadn't seen, we do have Alex Bouma (@stayallive) on board who also does maintenance on the sentry libs.
That being said, I would more than happily have an additional member from the Sentry community/team/etc

[Levi Morrison]

Hello,

I am not particularly knowledgeable about how PSRs are developed. I am chiming in to say that it seems a bit premature to write a PSR for this. The OpenTelemetry SDK for PHP is pre-alpha and OTEL itself is not even considered GA. Are you sure the time is right? I am very interested in OTEL and the impact it can have in the PHP ecosystem, but want to make sure the effort is put in the right place at the right time.

Full disclosure: I have previously contributed to the OTEL for PHP project and work for Datadog.

[Levi Morrison]

On Thursday, March 24, 2022 at 9:56:42 AM UTC-6 Levi Morrison wrote:

The OpenTelemetry SDK for PHP is pre-alpha and OTEL itself is not even considered GA.

As of a month ago, the tracing spec did go to v1.0, so I guess the relevant parts are now GA. Sorry for that slip; I wasn't aware of the 1.0 announcement. Still, I wonder if it's a bit premature as the PHP SDK specifically is still pre-alpha. 

[Larry Garfield]

Related question: If there is an official PHP-SDK for the spec, what does a PSR add to it? (He says, knowing next to nothing about OpenTel and tracing systems.) That would be something to include in the metadoc, probably.

--Larry Garfield

[Ken Guest]

I think the strategy here is to come to a Standard before those frameworks/solutions solidify to a state where it might be more awkward to come to a common ground.

I'm open to correction of course ;-)

--
You received this message because you are subscribed to the Google Groups "PHP Framework Interoperability Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to php-fig+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/39629ac6-ef99-46e0-a6f1-e0653d17246fn%40googlegroups.com.

--

http://about.me/kenguest/

[Adam Allport]

Hi All,

I agree with Ken there! (But great question)

Additionally (and with no intent to diminish the work of the OTEL team so far), getting some of the larger players (Sentry, Elastic, DataDog, etc) to adopt the OTEL lib for creating tracing signals may be difficult due to the large scale changes.

By creating this set of interfaces we will allow various frameworks to implement a standardised set of signals to all of the other libs (should they wish) can subscribe/contribute to!

I note my use of the term "loosely modelled" as opposed to "strictly follows"

As per Larry's suggestion, I'll write this up in more detail as a "Why bother" section shortly!

Thanks

Adam

[Larry Garfield]

On Fri, Mar 25, 2022, at 12:08 PM, Adam Allport wrote:
> Hi All,
>
> I agree with Ken there! (But great question)
>
> Additionally (and with no intent to diminish the work of the OTEL team
> so far), getting some of the larger players (Sentry

> <https://github.com/getsentry/sentry-php/tree/master/src/Tracing>,
> Elastic <https://github.com/elastic/apm-agent-php>, DataDog
> <https://github.com/DataDog/dd-trace-php/tree/master/src/api>, etc) to

> adopt the OTEL lib for creating tracing signals may be difficult due to
> the large scale changes.
> By creating this set of interfaces we will allow various frameworks to
> implement a standardised set of signals to all of the other libs
> (should they wish) can subscribe/contribute to!
> I note my use of the term "loosely modelled" as opposed to "strictly
> follows"
>
> As per Larry's suggestion, I'll write this up in more detail as a "Why
> bother" section shortly!
>
> Thanks
> Adam

So the idea would be an interface on top of the official library, so there's some PSR<-->official library bridge, as well as however many other implementations people feel like creating?

If so, having people from the official library involved sounds like a good idea.

--Larry Garfield

[Adam Allport]

Hi All

Clarity (why bother) added here: https://github.com/php-fig/fig-standards/pull/1273/files#diff-ea4e78ab6feb773a5774ff8d44701faa2b5b1efa62532fecf508c42efa8ef37fR10-R18 

Thanks!

[Enrico Zimuel]

Hi Adam,

as representative of Elastic and PHP FIG I'm very interested in supporting this PSR. I can promote it for the PHP projects in Elastic, including the APM team.

Regards,

Enrico Zimuel

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/028f1716-1b46-42a1-9497-e2153be6c758n%40googlegroups.com.

[Adam Allport]

Larry

Accidentally replied to Larry's message privately, but there's value in it being here:

> So the idea would be an interface on top of the official library, so there's some PSR<-->official library bridge

For the most part... yes.

My understanding is whilst OTEL does split the actual interfaces (API) and implementation (SDK), to allow libraries to send signals without requiring the full SDK, it does not seem to be encouraged to write your own implementation.

This PSR would serve as a subset of the API, providing a minimal base for interop.
 
My thoughts are given most providers are following the trace->span->subspan layout, we could get in integrated to client libs for sentry without too much work.
Rather than the possible full rewrite that going full OTEL would require (since that would essentially mean throwing away current implementations, in exchange for writing an exporter).
From a frameworks perspective, it could be seen as *too much work* to implement the full OTEL API or SDK, so, the PSR would allow frameworks like Laravel to *just* resolve an instance of the Tracer, and give it some info

If that is what you would see as PSR<-->OTEL Bridge, then yes, it is indeed :)

> Ah, so the PSR intent is be basically "OTEL Jr" to make eventual transition to a common API easier?

> In that case I'd recommend ensuring that it's open for future PSRs to extend as more parts of OTEL standardize.

Yeah, OTEL Jr sounds like a good way of describing it!

Long term, I'd expect the PSR to be used for "lightweight" tracing purposes, and then if frameworks need to add even more info, they can (without many/any BC breaks) migrate to the full OTEL lib should they need.

Enrico

That's fantastic to hear!
I'll add your name to the list of WG members if you're okay with that?

The chat is yet to be created, I believe that will happen after the acceptance Vote

Thanks for your feedback everyone!

[Enrico Zimuel]

Hi Adam,

yes, please add my name. Happy to help on this PSR.

Regards,

Enrico

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/6c5fea64-4fff-43be-9619-4e3534520336n%40googlegroups.com.

[Levi Morrison]

Larry

Accidentally replied to Larry's message privately, but there's value in it being here:

> So the idea would be an interface on top of the official library, so there's some PSR<-->official library bridge

For the most part... yes.

My understanding is whilst OTEL does split the actual interfaces (API) and implementation (SDK), to allow libraries to send signals without requiring the full SDK, it does not seem to be encouraged to write your own implementation.

This PSR would serve as a subset of the API, providing a minimal base for interop.
 
My thoughts are given most providers are following the trace->span->subspan layout, we could get in integrated to client libs for sentry without too much work.
Rather than the possible full rewrite that going full OTEL would require (since that would essentially mean throwing away current implementations, in exchange for writing an exporter).
From a frameworks perspective, it could be seen as *too much work* to implement the full OTEL API or SDK, so, the PSR would allow frameworks like Laravel to *just* resolve an instance of the Tracer, and give it some info

If that is what you would see as PSR<-->OTEL Bridge, then yes, it is indeed :)

This sounds like deficiencies of the PHP OTEL API and SDK. Maybe the working group can act as stakeholders or case studies for the OTEL authors? I've contributed in the past to OTEL and could do so again so I'm not volunteering others to work on something :)

> Ah, so the PSR intent is be basically "OTEL Jr" to make eventual transition to a common API easier?

> In that case I'd recommend ensuring that it's open for future PSRs to extend as more parts of OTEL standardize.

Yeah, OTEL Jr sounds like a good way of describing it!

Long term, I'd expect the PSR to be used for "lightweight" tracing purposes, and then if frameworks need to add even more info, they can (without many/any BC breaks) migrate to the full OTEL lib should they need.

It would be helpful to have some examples of what this lightweight tracing is and is not.

[Luc Vieillescazes]

Hello everyone,

I work for Blackfire.io/Platform.sh on the Profiler and APM PHP extension.

We are interested by this working group and would be happy to help.

If you want to talk to me, I've just joined the Discord with the username iamluc.

Kind regards