Dynamic offers discussion, continued

3 views
Skip to first unread message

Hunter Freyer

unread,
Apr 30, 2021, 2:19:30 PM4/30/21
to Gabe Schine, Bryan Henry, Gary Bressler, Abdulla Kamar, Suraj Malhotra, Ben Wright, Yegor Pomortsev, drive...@fuchsia.dev, component-framework-dev
Great discussion on dynamic offers (go/ctp-038) yesterday! 

I'll list what I see as the outstanding issues, along with a proposed resolution. If anyone objects to the resolution given, reply here with a "-1" and ideally with your reasoning, so we can expand on the topic asynchronously. If you object, but think it's best discussed in a meeting, lemme know. Also let me know if I'm missing anything outstanding from yesterday's review, or related to the design in general.

In increasing order of consequence:

API Bikeshedding
There's plenty to argue about in terms of the particulars of the API, but I'm gonna table those discussions until everything else is settled.

Local Optimum
Perhaps predictably, the review yesterday got into some pretty deep questions with respect to Driver Framework's design, and how CF should handle dynamism in general. That's the next section. But putting aside those questions, and assuming minimal rethinking of the drivers-as-components design, I posit that there's a compromise that everybody can live with: the design, as written, but with "wildcard" protocols added to the main proposal.

With that, CFv2 becomes expressive enough to accommodate many of the proposals brought up last night: we could just have one collection with all the drivers, routing arbitrary protocols between them however driver_manager wants, or we could divide logical groups of drivers into separate collections and put tighter restrictions on what routes can be created between collections, etc. CFv2 itself wouldn't take a stance on when it's okay to allow dynamic offers for arbitrary protocols, we simply provide the tools and let the drivers and security teams figure it out.

I'm not worried about rampant abuse, because when static routing is an option, static routing is substantially easier for a developer than dynamic routing. The secure way is the easy way. When static routing isn't an option, and when even knowing the set of protocols that might get routed isn't an option, then allowing this flexibility is the only obvious way to prevent people from resorting to evil side-channel nonsense. It's better to support and keep an eye on the behavior than to drive users away.

As for the non-obvious ways...

CFv2 and Dynamism
bryanhenry@ sez:

... the static things we've largely dealt with so far are the easy case... I want to be cautious of answers of the shape of "this is hard and dynamic, so we just kick the can down the road". Of course we can't wait for the perfect answer that solves everything either, but IMO if the value CF brings to the table ends when we get to more realistic software then that makes me question if the architecture is right.

CFv2 is designed for a mostly static, mostly stateless world. The fact that there's misalignment when taking on Driver Framework (a fairly dynamic, mostly stateless world) isn't that surprising. It would be lovely if CFv2 was capable enough that there wouldn't even need to be a Driver Framework; drivers would just be regular old components. The drivers-as-components project doesn't get us there. It's more like "Driver Framework running on top of CFv2." But that's still a big improvement over the current state of affairs where we have two totally different stacks that talk to each other over devfs. (Not for nothing: the very fact that Driver Framework even can run on top of CFv2 shows we've done some things right.)

One day CF will need to be able to handle this sort of thing better natively. I think drivers-as-components represents a meaningful step towards that, and I don't think we're going to figure out the right model without the experience we'll gain by taking that step and onboarding more products.

So in sum, I propose we kick the can down the road :)

Thanks,

Hunter

Gabe Schine

unread,
Apr 30, 2021, 2:53:35 PM4/30/21
to Hunter Freyer, Bryan Henry, Gary Bressler, Abdulla Kamar, Suraj Malhotra, Ben Wright, Yegor Pomortsev, drive...@fuchsia.dev, component-framework-dev
Great discussion on dynamic offers (go/ctp-038) yesterday! 

I'll list what I see as the outstanding issues, along with a proposed resolution. If anyone objects to the resolution given, reply here with a "-1" and ideally with your reasoning, so we can expand on the topic asynchronously. If you object, but think it's best discussed in a meeting, lemme know. Also let me know if I'm missing anything outstanding from yesterday's review, or related to the design in general.

In increasing order of consequence:

API Bikeshedding
There's plenty to argue about in terms of the particulars of the API, but I'm gonna table those discussions until everything else is settled.

Local Optimum
Perhaps predictably, the review yesterday got into some pretty deep questions with respect to Driver Framework's design, and how CF should handle dynamism in general. That's the next section. But putting aside those questions, and assuming minimal rethinking of the drivers-as-components design, I posit that there's a compromise that everybody can live with: the design, as written, but with "wildcard" protocols added to the main proposal.

With that, CFv2 becomes expressive enough to accommodate many of the proposals brought up last night: we could just have one collection with all the drivers, routing arbitrary protocols between them however driver_manager wants, or we could divide logical groups of drivers into separate collections and put tighter restrictions on what routes can be created between collections, etc. CFv2 itself wouldn't take a stance on when it's okay to allow dynamic offers for arbitrary protocols, we simply provide the tools and let the drivers and security teams figure it out.

Abdulla made a point that stuck with me. In the notes it's captured as: "We are still describing (today, and this won't change) what leaves the collection and what goes into the collection". The way I interpret this is: if we treat a collection as a black box kind of like we treat a single component as a black box, do we care what internal routing is done? For a single component, the answer is "no": that single component can have arbitrarily complex internals.

To continue the devil's advocate position: how much do we care to expose information about and limit what internal routing occurs within a collection?

Anyway your proposal of allowing wildcards is a nice compromise position, and allows for the full spectrum of "allow anything, I don't care" to "only these 2 protocols, please". I think Yegor suggested something similar, too: put a "marker" in the manifest that identifies the collections with "open routing" so that we are at least aware of them.
 

I'm not worried about rampant abuse, because when static routing is an option, static routing is substantially easier for a developer than dynamic routing. The secure way is the easy way. When static routing isn't an option, and when even knowing the set of protocols that might get routed isn't an option, then allowing this flexibility is the only obvious way to prevent people from resorting to evil side-channel nonsense. It's better to support and keep an eye on the behavior than to drive users away.

+1 to this. From personal experience with v1 where even simple routing often requires code to set up, static routing is so very very attractive.
 
As for the non-obvious ways...

CFv2 and Dynamism
bryanhenry@ sez:

... the static things we've largely dealt with so far are the easy case... I want to be cautious of answers of the shape of "this is hard and dynamic, so we just kick the can down the road". Of course we can't wait for the perfect answer that solves everything either, but IMO if the value CF brings to the table ends when we get to more realistic software then that makes me question if the architecture is right.

CFv2 is designed for a mostly static, mostly stateless world. The fact that there's misalignment when taking on Driver Framework (a fairly dynamic, mostly stateless world) isn't that surprising. It would be lovely if CFv2 was capable enough that there wouldn't even need to be a Driver Framework; drivers would just be regular old components. The drivers-as-components project doesn't get us there. It's more like "Driver Framework running on top of CFv2." But that's still a big improvement over the current state of affairs where we have two totally different stacks that talk to each other over devfs. (Not for nothing: the very fact that Driver Framework even can run on top of CFv2 shows we've done some things right.)

One day CF will need to be able to handle this sort of thing better natively. I think drivers-as-components represents a meaningful step towards that, and I don't think we're going to figure out the right model without the experience we'll gain by taking that step and onboarding more products.

I'd like to reframe the first sentence, if I may: CFv2 has, to date, had to solve topological challenges that fit pretty well into a static/stateless world. It is inevitable to have to accommodate the more dynamic use-cases, and it must do so.
 

So in sum, I propose we kick the can down the road :)

Thanks,

Hunter


--
Gabe Schine
Software Engineer / Manager

Hunter Freyer

unread,
Apr 30, 2021, 3:11:58 PM4/30/21
to Gabe Schine, Bryan Henry, Gary Bressler, Abdulla Kamar, Suraj Malhotra, Ben Wright, Yegor Pomortsev, drive...@fuchsia.dev, component-framework-dev
On Fri, Apr 30, 2021 at 2:53 PM Gabe Schine <tha...@google.com> wrote:
Great discussion on dynamic offers (go/ctp-038) yesterday! 

I'll list what I see as the outstanding issues, along with a proposed resolution. If anyone objects to the resolution given, reply here with a "-1" and ideally with your reasoning, so we can expand on the topic asynchronously. If you object, but think it's best discussed in a meeting, lemme know. Also let me know if I'm missing anything outstanding from yesterday's review, or related to the design in general.

In increasing order of consequence:

API Bikeshedding
There's plenty to argue about in terms of the particulars of the API, but I'm gonna table those discussions until everything else is settled.

Local Optimum
Perhaps predictably, the review yesterday got into some pretty deep questions with respect to Driver Framework's design, and how CF should handle dynamism in general. That's the next section. But putting aside those questions, and assuming minimal rethinking of the drivers-as-components design, I posit that there's a compromise that everybody can live with: the design, as written, but with "wildcard" protocols added to the main proposal.

With that, CFv2 becomes expressive enough to accommodate many of the proposals brought up last night: we could just have one collection with all the drivers, routing arbitrary protocols between them however driver_manager wants, or we could divide logical groups of drivers into separate collections and put tighter restrictions on what routes can be created between collections, etc. CFv2 itself wouldn't take a stance on when it's okay to allow dynamic offers for arbitrary protocols, we simply provide the tools and let the drivers and security teams figure it out.

Abdulla made a point that stuck with me. In the notes it's captured as: "We are still describing (today, and this won't change) what leaves the collection and what goes into the collection". The way I interpret this is: if we treat a collection as a black box kind of like we treat a single component as a black box, do we care what internal routing is done? For a single component, the answer is "no": that single component can have arbitrarily complex internals.

To continue the devil's advocate position: how much do we care to expose information about and limit what internal routing occurs within a collection?

It's a really interesting question. To me it seems wrong to say a collection is encapsulated. We already have a unit of encapsulation in CF: the component itself. You shouldn't need to understand what goes on within a component in order to use it.

I think the drivers-as-components design could make better use of this. For instance, there's a pattern of "platform dependent vs platform independent" drivers. One (platform dependent) driver implements the ETHERNET_IMPL protocol, and another (platform independent) driver adapts that to the ETHERNET protocol. In drivers-as-components, these are two drivers, and therefore two components. In this case, you could imagine the two of them working together to be considered "one component", in which case the platform-dependent part would be hidden from view. If those could be merged together, maybe drivers that speak third party protocols could also be merged together into a single component? Perhaps one day we could turn off our wildcard dynamic offer declarations?

I'm not sure whether there's anything there, but if there is, I think drivers-as-components still puts us on the path towards it.

Gary Bressler

unread,
Apr 30, 2021, 4:09:15 PM4/30/21
to Hunter Freyer, Gabe Schine, Bryan Henry, Abdulla Kamar, Suraj Malhotra, Ben Wright, Yegor Pomortsev, drive...@fuchsia.dev, component-framework-dev
I agree with defining components as the "unit of encapsulation". There is not necessarily any intrinsic grouping between components in a collection. In fact, the components within may be quite diverse.

Personally I'm ok with wildcard syntax, as long as we have a way to feature flag wildcards to the driver collection only. I think we'll need experience with other usages before we can understand if it's acceptable as a general feature. Calling it an "escape hatch" may not be off the mark, but it's worth pointing out that collections themselves are kind of an escape hatch. When you define a collection, you set no bounds on the components that may be instantiated in it. The interior of a collection effectively is completely opaque to any build-time tooling, and we rely on the parent components to use the collection "as intended".

Suraj Malhotra

unread,
Apr 30, 2021, 4:21:26 PM4/30/21
to Hunter Freyer, Gabe Schine, Bryan Henry, Gary Bressler, Abdulla Kamar, Ben Wright, Yegor Pomortsev, drive...@fuchsia.dev, component-framework-dev
On Fri, Apr 30, 2021 at 12:11 PM Hunter Freyer <hjfr...@google.com> wrote:


On Fri, Apr 30, 2021 at 2:53 PM Gabe Schine <tha...@google.com> wrote:
Great discussion on dynamic offers (go/ctp-038) yesterday! 

I'll list what I see as the outstanding issues, along with a proposed resolution. If anyone objects to the resolution given, reply here with a "-1" and ideally with your reasoning, so we can expand on the topic asynchronously. If you object, but think it's best discussed in a meeting, lemme know. Also let me know if I'm missing anything outstanding from yesterday's review, or related to the design in general.

In increasing order of consequence:

API Bikeshedding
There's plenty to argue about in terms of the particulars of the API, but I'm gonna table those discussions until everything else is settled.

Local Optimum
Perhaps predictably, the review yesterday got into some pretty deep questions with respect to Driver Framework's design, and how CF should handle dynamism in general. That's the next section. But putting aside those questions, and assuming minimal rethinking of the drivers-as-components design, I posit that there's a compromise that everybody can live with: the design, as written, but with "wildcard" protocols added to the main proposal.

With that, CFv2 becomes expressive enough to accommodate many of the proposals brought up last night: we could just have one collection with all the drivers, routing arbitrary protocols between them however driver_manager wants, or we could divide logical groups of drivers into separate collections and put tighter restrictions on what routes can be created between collections, etc. CFv2 itself wouldn't take a stance on when it's okay to allow dynamic offers for arbitrary protocols, we simply provide the tools and let the drivers and security teams figure it out.

Abdulla made a point that stuck with me. In the notes it's captured as: "We are still describing (today, and this won't change) what leaves the collection and what goes into the collection". The way I interpret this is: if we treat a collection as a black box kind of like we treat a single component as a black box, do we care what internal routing is done? For a single component, the answer is "no": that single component can have arbitrarily complex internals.

To continue the devil's advocate position: how much do we care to expose information about and limit what internal routing occurs within a collection?

It's a really interesting question. To me it seems wrong to say a collection is encapsulated. We already have a unit of encapsulation in CF: the component itself. You shouldn't need to understand what goes on within a component in order to use it.

I think the drivers-as-components design could make better use of this. For instance, there's a pattern of "platform dependent vs platform independent" drivers. One (platform dependent) driver implements the ETHERNET_IMPL protocol, and another (platform independent) driver adapts that to the ETHERNET protocol. In drivers-as-components, these are two drivers, and therefore two components. In this case, you could imagine the two of them working together to be considered "one component", in which case the platform-dependent part would be hidden from view. If those could be merged together, maybe drivers that speak third party protocols could also be merged together into a single component? Perhaps one day we could turn off our wildcard dynamic offer declarations?

I'm not sure whether there's anything there, but if there is, I think drivers-as-components still puts us on the path towards it.

I think the major disconnect is how we judge security properties of a component. For driver components, I believe bind programs are a big part of that story, and the component framework cannot reason about them, and therefore can't enforce policy by itself. For instance, I imagine we don't want to allow non-platform drivers to bind in place of the ethernet driver. Similarly, I imagine we will have restrictions at some point in terms of which drivers are capable of binding to usb devices based on generic properties such as the USB Class. We may also prefer binding a driver for a vendor's device written by that vendor over one written by a third party. Other contemporary systems solve this problem via signing - if signed by the OS vendor's cert, they get more "capabilities" than other drivers. I'm not entirely sure how we will implement such policies in Fuchsia (and if we will rely on CF primitives to do so), but today we treat all drivers equally.

As far as merging drivers into singular components go, we always have the ability to implement some drivers as libraries rather than components to achieve this. The problem with that approach is that we lose control of ability to independently update the code, as well as reduce the composability of the system. For instance in audio and camera, it can be normal for a single driver in Linux to do what we decomposed into several drivers. I consider this an architectural improvement and I don't want to somehow give incentives to third parties to regress. The other alternative is to keep drivers the same but take CF out of the role of dealing with routing services between things which are "coupled". I find that equally unattractive as it's more complicated to understand how routing occurs, and hard to justify why CF ever plays a role in routing at that point.

Gabe Schine

unread,
Apr 30, 2021, 4:59:09 PM4/30/21
to Gary Bressler, Hunter Freyer, Bryan Henry, Abdulla Kamar, Suraj Malhotra, Ben Wright, Yegor Pomortsev, drive...@fuchsia.dev, component-framework-dev
I agree with defining components as the "unit of encapsulation". There is not necessarily any intrinsic grouping between components in a collection. In fact, the components within may be quite diverse.

I don't want to derail things (too much) so I'll deposit this thought experiment here and then retire my line of thinking:

Consider the case of a collection with one child. From a routing perspective, it is equivalent to statically routing to a single child. The difference between a one-element collection and a statically declared child is that the one component is not known a-priori. If I'm wrong, I apologize and I'll accelerate my CF university curriculum. The contents of the collection are opaque in this case.

Using a type of inductive proof, consider the above a base case. If the collection's parent now creates two children instead and defines a dynamic offer relationship between those two children, what has changed? The contents are still opaque: we still do not know the component(s) a-priori. The routing to/from the collection has not changed: those are still statically routed and available to all children. The only difference that I can see is that the opaque contents of the collection are different. In that way, I see the collection as a type of encapsulation boundary.

I'm not proposing any changes that formalize the above line of thinking into policy or anything like that, I'm simply using it to frame my understanding of/conceptualize the problem. I thought it might also be helpful to others.
Reply all
Reply to author
Forward
0 new messages