RealmBuilder out-of-tree

[Shai Barack]

Hi folks,

Please stop me if this is something that we're already planning on addressing in the upcoming RealmBuilder RFC.

With the current and upcoming work to extend RealmBuilder to offer more (thin) client libs across more languages, as well as the desire to do so via the Fuchsia SDK, I went looking at realm_builder.fidl and fuchsia_component_test.shard.cml to learn how stuff works. There's lots to like, and some matters of some concern for OOT.

Good:

• Clients are thin: obviously great as we aim to decouple the platform from specific languages, and support many languages. Inclusion and all that. "Protocols, not libraries" is a good tenet to practice, though client libs are ok when they're very thin.
• No specialized platform features for testing: once again I love how the same concepts of capability routing, realms, etc are used to enable testing without specialized "testonly" framework features. The framework intermediary component is implemented entirely in terms of the Launcher and Environment capabilities, which are "free" in that we hand them around willy-nilly with no concern. Launcher and Environment are hierarchically contained (the mess you might make is isolated to your realm) and don't offer any escalation paths.
• Abstractions: we're hiding the implementation details well behind client libs and shards. Props.

Needs work:

• Monikers in the ABI: in order to use realmbuilder you need to spawn the framework intermediary as a child component of the test, which will manage a child collection for you. You spawn it by launching a component via a URL. This is problematic for SDK users in several ways. The framework intermediary component doesn't currently exist in SDK images (I believe), and even if it did, promoting the launch URL fuchsia-pkg://fuchsia.com/fuchsia-component-test#meta/framework-intermediary.cm into a hard contract is not great for reasons we've discussed before.
• Limiting usage: there's very little in the way of people using realmbuilder outside of tests. I guess we can't really stop that, seeing as this is all achievable using table-stakes fuchsia.sys.* capabilities, but it's still concerning to be making something with appmgr/sysmgr-like qualities so readily available. Perhaps if the framework intermediary became a feature of being managed by test_manager we'd have more safety in knowing that this won't come back to haunt us?

I have some ideas for solutions, but I wanted to hear people's take on the problem first.

[Gabe Schine]

Good:

• Clients are thin: obviously great as we aim to decouple the platform from specific languages, and support many languages. Inclusion and all that. "Protocols, not libraries" is a good tenet to practice, though client libs are ok when they're very thin.
• No specialized platform features for testing: once again I love how the same concepts of capability routing, realms, etc are used to enable testing without specialized "testonly" framework features. The framework intermediary component is implemented entirely in terms of the Launcher and Environment capabilities, which are "free" in that we hand them around willy-nilly with no concern. Launcher and Environment are hierarchically contained (the mess you might make is isolated to your realm) and don't offer any escalation paths.
• Abstractions: we're hiding the implementation details well behind client libs and shards. Props.

+1

 

Needs work:

• Monikers in the ABI: in order to use realmbuilder you need to spawn the framework intermediary as a child component of the test, which will manage a child collection for you. You spawn it by launching a component via a URL. This is problematic for SDK users in several ways. The framework intermediary component doesn't currently exist in SDK images (I believe), and even if it did, promoting the launch URL fuchsia-pkg://fuchsia.com/fuchsia-component-test#meta/framework-intermediary.cm into a hard contract is not great for reasons we've discussed before.

We could distribute the component itself and ask people to package it with their tests. It really is an implementation detail of the test, even today.

• Limiting usage: there's very little in the way of people using realmbuilder outside of tests. I guess we can't really stop that, seeing as this is all achievable using table-stakes fuchsia.sys.* capabilities, but it's still concerning to be making something with appmgr/sysmgr-like qualities so readily available. Perhaps if the framework intermediary became a feature of being managed by test_manager we'd have more safety in knowing that this won't come back to haunt us?

Since Realm Builder is a rational abstraction on CF, people can recreate it anywhere. I'm not sure what restricting the usage of RealmBuilder the library/component buys us. I'm probably missing something, though. Is it that it gives people an easier path to building realms using code rather than .cml, and we lose the benefits of having everything in .cml?

One option is to do something like you suggested: host the intermediary in such a way that only tests get the relevant realm_builder.fidl capabilities routed to them.

 

I have some ideas for solutions, but I wanted to hear people's take on the problem first.

--
All posts must follow the Fuchsia Code of Conduct https://fuchsia.dev/fuchsia-src/CODE_OF_CONDUCT or may be removed.
---
You received this message because you are subscribed to the Google Groups "component-framework-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to component-framewo...@fuchsia.dev.
To view this discussion on the web visit https://groups.google.com/a/fuchsia.dev/d/msgid/component-framework-dev/CAK0PkCEt95sNa-1cPQGujYtbh4pSYsdQ5G9RSsqUKM5179ZOdg%40mail.gmail.com.

--

Gabe Schine

Software Engineer / Manager

🔵 Blue Dot Ally

go/thatguy-readme

go/where-is-thatguy

[Yegor Pomortsev]

On Mon, Aug 30, 2021 at 4:29 PM 'Shai Barack' via component-framework-dev <component-f...@fuchsia.dev> wrote:

Hi folks,

Please stop me if this is something that we're already planning on addressing in the upcoming RealmBuilder RFC.

With the current and upcoming work to extend RealmBuilder to offer more (thin) client libs across more languages, as well as the desire to do so via the Fuchsia SDK, I went looking at realm_builder.fidl and fuchsia_component_test.shard.cml to learn how stuff works. There's lots to like, and some matters of some concern for OOT.

Good:

• Clients are thin: obviously great as we aim to decouple the platform from specific languages, and support many languages. Inclusion and all that. "Protocols, not libraries" is a good tenet to practice, though client libs are ok when they're very thin.
• No specialized platform features for testing: once again I love how the same concepts of capability routing, realms, etc are used to enable testing without specialized "testonly" framework features. The framework intermediary component is implemented entirely in terms of the Launcher and Environment capabilities, which are "free" in that we hand them around willy-nilly with no concern. Launcher and Environment are hierarchically contained (the mess you might make is isolated to your realm) and don't offer any escalation paths.
• Abstractions: we're hiding the implementation details well behind client libs and shards. Props.

Needs work:

• Monikers in the ABI: in order to use realmbuilder you need to spawn the framework intermediary as a child component of the test, which will manage a child collection for you. You spawn it by launching a component via a URL. This is problematic for SDK users in several ways. The framework intermediary component doesn't currently exist in SDK images (I believe), and even if it did, promoting the launch URL fuchsia-pkg://fuchsia.com/fuchsia-component-test#meta/framework-intermediary.cm into a hard contract is not great for reasons we've discussed before.

Exposing components that implement a particular set of SDK APIs is something we'll need for Session Framework. We'd like developers to include platform-provided components that we know implement the fuchsia.element APIs correctly, and we can keep them updated between SDK revisions. These components use and expose the same APIs that we expose in the SDK, so I don't see it limiting our ability to make internal changes. They're still free to implement the protocols themselves, of course.

This is also a different problem than exposing monikers. The test would not depend on the intermediary to be in a specific part of the platform topology, but only within its own topology, so that's good.

Version pinning would help a lot. If a test depends on a specific version of framework-intermediary to work, then the test author updates the SDK, it's on them to make sure their test is compatible with the new component.

• Limiting usage: there's very little in the way of people using realmbuilder outside of tests. I guess we can't really stop that, seeing as this is all achievable using table-stakes fuchsia.sys.* capabilities, but it's still concerning to be making something with appmgr/sysmgr-like qualities so readily available. Perhaps if the framework intermediary became a feature of being managed by test_manager we'd have more safety in knowing that this won't come back to haunt us?

I'm actually interested to see how people will "misuse" RealmBuilder. My suspicion is that CF doesn't have enough features to satisfy all use cases with static topologies, and users will fall back on RealmBuilder to make things work.

It's still easier to define things statically than write code. That will be a motivator to not use RealmBuilder for everything.

 

I have some ideas for solutions, but I wanted to hear people's take on the problem first.

--

[Shai Barack]

I'm hearing three ways for OOT developers to use the framework intermediary:

1. Similar to how this works today in-tree. Users will need to know not only the launch URL for the intermediary, it's also suggested that they'd need version pinning. Not a fan since we don't have evolution mechanisms for moniker-based identifiers. We haven't (formally!) introduced monikers into the FSI, and I don't think we should start.
2. Distribute a prebuilt framework intermediary component via the SDK, and let developers include it in their test packages. The mechanics for how this would work are TBD. The same mechanics would likely be useful to solve other known OOT development needs, such as using isolated devmgr or fake Scenic in tests, or having OOT test runners. Hermetic packaging is cool.
3. Make the framework intermediary part of the testing platform, a component managed by test_manager or just fold the functionality into test_manager code, then offer the FrameworkIntermediary capability to test components. Exposing RealmBuilder as a FIDL protocol would work, and FIDL offers a full toolkit for managing evolution.

[Hunter Freyer]

On Mon, Aug 30, 2021 at 11:41 PM 'Shai Barack' via component-framework-dev <component-f...@fuchsia.dev> wrote:

I'm hearing three ways for OOT developers to use the framework intermediary:

1. Similar to how this works today in-tree. Users will need to know not only the launch URL for the intermediary, it's also suggested that they'd need version pinning. Not a fan since we don't have evolution mechanisms for moniker-based identifiers. We haven't (formally!) introduced monikers into the FSI, and I don't think we should start.

I think the way we do it in-tree is problematic because the URL is kind of a magic string, but I don't really get how monikers come into it. Can you elaborate?

1. Distribute a prebuilt framework intermediary component via the SDK, and let developers include it in their test packages. The mechanics for how this would work are TBD. The same mechanics would likely be useful to solve other known OOT development needs, such as using isolated devmgr or fake Scenic in tests, or having OOT test runners. Hermetic packaging is cool.

I think this is the only viable option today, but putting multiple components from different authors into the same package is going to be messy, unless we have specific guidance for how to do it. Even with that guidance, +Bryan Henry's Component-scoped executability RFC gets into some of the security problems that approach entails.

This is a real gap in the CF story, and I'd love to see more active progress towards hermetic and portable component distribution. Testing is the most obvious use case, but I think it'll be useful everywhere. 

1. Make the framework intermediary part of the testing platform, a component managed by test_manager or just fold the functionality into test_manager code, then offer the FrameworkIntermediary capability to test components. Exposing RealmBuilder as a FIDL protocol would work, and FIDL offers a full toolkit for managing evolution.

-1 on this. Each test should be able to have its own, self-contained framework intermediary. Making the framework intermediary "multi-tenant" so tests can be isolated from each other sounds like a lot of work of limited value. We want to make it easy for components to depend on other helper components; let's put our efforts towards that.

Hunter

 

To view this discussion on the web visit https://groups.google.com/a/fuchsia.dev/d/msgid/component-framework-dev/CAK0PkCHJr9VjR_A7M7OOVepmD4oYfy%3DdLZ5FWfBjmnW4TzqHvg%40mail.gmail.com.

[Derek Gonyeo]

I'm a big fan of option 2, where we distribute a prebuilt through the SDK and expect developers to include it in their packages. I was unaware that the mechanics for this are TBD, but it sounds like the RFC might be a great opportunity to figure those out!

Option 3 would also work, but I'm a fan of the current architecture where the provider of the realm builder fidl is something you bring along with your test package. It cleanly solves things like version skew and guarantees hermeticity between different tests.

And on preventing misuse, I agree with Yegor that it'll be interesting to see what this ends up getting used for. As Shai called out, everything realm builder is doing is already possible using the common framework features, so if we truly want to prevent usage of this outside of testing scenarios I think we should start chatting about things like "descendents of this realm are no longer allowed to configure their environment", instead of limiting realm builder's availability. If we move the framework intermediary to be managed by test manager, there's nothing stopping someone OOT who's determined from copying the realm builder code to their repository, building it, and putting their own realm builder elsewhere in the topology.

To view this discussion on the web visit https://groups.google.com/a/fuchsia.dev/d/msgid/component-framework-dev/CAK0PkCHJr9VjR_A7M7OOVepmD4oYfy%3DdLZ5FWfBjmnW4TzqHvg%40mail.gmail.com.

[Shai Barack]

Great, I also prefer what's behind door #2.

I've read Bryan's RFC that Hunter linked to back when it was published. I'm not seeing the connection. What's the security issue introduced by packaging test support components like the intermediary or a fake Scenic in your test package? 

[Hunter Freyer]

+Bryan Henry 

On Tue, Aug 31, 2021 at 12:05 PM 'Shai Barack' via component-framework-dev <component-f...@fuchsia.dev> wrote:

Great, I also prefer what's behind door #2.

I've read Bryan's RFC that Hunter linked to back when it was published. I'm not seeing the connection. What's the security issue introduced by packaging test support components like the intermediary or a fake Scenic in your test package? 

Bryan's RFC generally talks about the fact that components have access to "too many executable bytes". Part of that is, "there's images and other such data that nobody should ever execute, and we should make that non-executable." But even if we do that, when two components are bundled together into a single package, they can read each other's files and execute each other's binaries. Abstractly speaking this is "unhygienic". Concretely, it means attackers have more raw material to work with in building exploits, and that some components may become dependent on the implementation details of the other components. (I can totally imagine a test executing the fake scenic binary directly, rather than instantiating it as a component, if it means they can accomplish something they can't do otherwise.)

Hunter

To view this discussion on the web visit https://groups.google.com/a/fuchsia.dev/d/msgid/component-framework-dev/CAK0PkCGq6YbtzNAAsx%3DYjoTOXiZ%3DLp0cavPOVwPzy_r_fPvYjw%40mail.gmail.com.

[Yegor Pomortsev]

Does Option #2 limit us from updating the prebuilt components with non-API breaking changes independently of the parent package, like for security updates?

[Suraj Malhotra]

FWIW, I think it's worth differentiating robustness from security here. Co-packaging may increase the number of things a test or platform component may do, but given the context it's likely not a security issue. It's just a hazard for robustness as the test author may accidentally do something we wouldn't like for them to do, but I'm not sure there needs to be a technical solution to that problem.

On Tue, Aug 31, 2021 at 10:22 AM 'Yegor Pomortsev' via component-framework-dev <component-f...@fuchsia.dev> wrote:

Does Option #2 limit us from updating the prebuilt components with non-API breaking changes independently of the parent package, like for security updates?

 

That feels like it would defeat the purpose. Letting the test author maintain control of the specific version being used seems core to option #2. It's akin to pinning the version of the shared library the test is using, which is something we allow.

(I'm a big fan of option #2 if it wasn't clear 🙂)

 

To view this discussion on the web visit https://groups.google.com/a/fuchsia.dev/d/msgid/component-framework-dev/CAMmMHD4i%2B5_xuLdHPNdZNNeRVOtSoz-X-_rTMa_PMaZJ6%3D7uvg%40mail.gmail.com.

[Hunter Freyer]

On Tue, Aug 31, 2021 at 1:38 PM Suraj Malhotra <surajm...@google.com> wrote:

FWIW, I think it's worth differentiating robustness from security here. Co-packaging may increase the number of things a test or platform component may do, but given the context it's likely not a security issue. It's just a hazard for robustness as the test author may accidentally do something we wouldn't like for them to do, but I'm not sure there needs to be a technical solution to that problem.

Agreed that there's no blinking-marquee saying "THIS IS A VULNERABILITY THAT NEEDS FIXING", but robustness issues can also grow up to become security issues if you're unlucky.

 

On Tue, Aug 31, 2021 at 10:22 AM 'Yegor Pomortsev' via component-framework-dev <component-f...@fuchsia.dev> wrote:

Does Option #2 limit us from updating the prebuilt components with non-API breaking changes independently of the parent package, like for security updates?

 

That feels like it would defeat the purpose. Letting the test author maintain control of the specific version being used seems core to option #2. It's akin to pinning the version of the shared library the test is using, which is something we allow.

+1

 

(I'm a big fan of option #2 if it wasn't clear 🙂)

100% agreed that #2 is the correct direction. But I don't want to say, "go with option #2 and we're done here." It's worth considering how we can make that option more robust.

[Adam Barth]

On Tue, Aug 31, 2021 at 5:51 PM 'Hunter Freyer' via component-framework-dev <component-f...@fuchsia.dev> wrote:

On Tue, Aug 31, 2021 at 1:38 PM Suraj Malhotra <surajm...@google.com> wrote:

FWIW, I think it's worth differentiating robustness from security here. Co-packaging may increase the number of things a test or platform component may do, but given the context it's likely not a security issue. It's just a hazard for robustness as the test author may accidentally do something we wouldn't like for them to do, but I'm not sure there needs to be a technical solution to that problem.

Agreed that there's no blinking-marquee saying "THIS IS A VULNERABILITY THAT NEEDS FIXING", but robustness issues can also grow up to become security issues if you're unlucky.

 

On Tue, Aug 31, 2021 at 10:22 AM 'Yegor Pomortsev' via component-framework-dev <component-f...@fuchsia.dev> wrote:

Does Option #2 limit us from updating the prebuilt components with non-API breaking changes independently of the parent package, like for security updates?

 

That feels like it would defeat the purpose. Letting the test author maintain control of the specific version being used seems core to option #2. It's akin to pinning the version of the shared library the test is using, which is something we allow.

+1

 

(I'm a big fan of option #2 if it wasn't clear 🙂)

100% agreed that #2 is the correct direction. But I don't want to say, "go with option #2 and we're done here." It's worth considering how we can make that option more robust.

For shared libraries, we load them indirectly through a system service.  If there were a critical vulnerability in a shared library, we could "patch" the library by recognizing the vulnerable blob name and having the system service supply a different VMO.

A similar thing happens in #2 for prebuilt components.  The executables are loaded indirectly through a system service: fuchsia.process.Launcher.  If there were a critical vulnerability, we could "patch" the executable using the system service.

Neither of those steps are particularly desirable, but they exist and can be used to mitigate this kind of issue if needed.

Adam

 

To view this discussion on the web visit https://groups.google.com/a/fuchsia.dev/d/msgid/component-framework-dev/CAMkOBfuuWnjp542OW46uCoTtghHPB2HjyDTTs9evJWe82Ogvug%40mail.gmail.com.

[Gary Bressler]

Agree with the consensus around Option #2. Do we think this is something that should gate the launch, or would it be alright for the initial customers to consume it through an absolute URL if that helps us release earlier?

I'm not clear what the challenge is in limiting usage of RealmBuilder to tests. What's stopping us from marking the realmbuilder FIDLs testonly?

To view this discussion on the web visit https://groups.google.com/a/fuchsia.dev/d/msgid/component-framework-dev/CAP%3D28cf1YB8rhLGG1N41xnst-KWW-vy0hCmBv8eXDqX3WJsm5A%40mail.gmail.com.

[Gabe Schine]

Agree with the consensus around Option #2. Do we think this is something that should gate the launch, or would it be alright for the initial customers to consume it through an absolute URL if that helps us release earlier?

Since the component is included through a CML shard that we also own, I'm comfortable not blocking on any of the proposals here.

 

To view this discussion on the web visit https://groups.google.com/a/fuchsia.dev/d/msgid/component-framework-dev/CAK1yh2kuwzrXNOw8o4EhS7fkS-fTte-69t5c5LFGd8zuFDjOhg%40mail.gmail.com.

[Derek Gonyeo]

On Tue, Aug 31, 2021 at 11:02 PM Gary Bressler <g...@google.com> wrote:

Agree with the consensus around Option #2. Do we think this is something that should gate the launch, or would it be alright for the initial customers to consume it through an absolute URL if that helps us release earlier?

I'm not clear what the challenge is in limiting usage of RealmBuilder to tests. What's stopping us from marking the realmbuilder FIDLs testonly?

Does "testonly" exist out of tree? I would have guessed that once something has passed through the SDK we lose that level of control over the build system, and couldn't stop someone using something intended for tests in a non-test scenario.

[Shai Barack]

Testonly is a build system concept that applies to build targets. We publish blobs, so we can assume that such annotations are lost. 

[Gary Bressler]

True, but I assume that we'll provide guidance as to what parts of the SDK are semantically test-only and what are not? Then integrators are responsible for making sure that the 'test-only' parts are marked as such in their build system (if it supports that). Presumably RealmBuilder won't be the only testing API in the SDK, for example if we provide fakes those should be designated as only for testing.

[Shai Barack]

Yes, sorry, I was really referring to the IDK which is build system agnostic. In any SDK we produce today we have some targets that are testonly.