Considerations for compilable service manifests

[Ken Rockot]

I'm working on getting rid of JSON service manifests in favor of compiled C++ manifest structures. See below** for rationale.

I have narrowed this change down to two competing approaches, and I think I know which one I want to use. I'd like to get feedback from other people who do service development before I move forward with either of them. In particular I would like people to think about what they will have to read and write when examining and maintaining manifests.

1. Lazy Runtime Construction

The naive approach is to establish a pattern where each service defines a GetManifest() API in their public C++ client library.

The main advantage is that this approach is not constrained by static initialization concerns because we will use function-local statics. This increases our freedom to provide nice helpers for building some of the nested metadata you find in manifests, and it is generally easier to understand and maintain than approach #2. It also means that if you want to do some goofy run-time logic in your manifest construction (like checking a command line switch), you can actually do that.

The main drawback I see is that it may feel like somewhat of a regression to go from having a single manifest.json file to having two files (manifest.h and manifest.cc) with some pretty rote boilerplate in each, and having the manifest feel more like code than data.

  // *** services/foo/public/cpp/manifest.h

  #include "services/service_manager/public/cpp/manifest.h"

  

  namespace foo {

  const service_manager::Manifest& GetManifest();

  }  // namespace foo

  // *** services/foo/public/cpp/manifest.cc

  #include "base/no_destructor.h"

  #include "services/foo/public/cpp/manifest.h"

  namespace foo {

  const service_manager::Manifest& GetManifest() {

    static base::NoDestructor<service_manager::Manifest> manifest =

        service_manager::ManifestBuilder()

           .WithServiceName("my_pretty_cool_service")

           .WithOptions(service_manager::ManifestOptionsBuilder()

               .CanRegisterOtherServiceInstances(true)

               .WithSandboxType(sandbox::SandboxType::kNone))

           .WithExposedCapabilities({

                 {"gpu_access",

                  InterfaceList<gpu::mojom::GpuHost>()},

                 {"device_access",

                  Interfaces<

                      device::mojom::BluetoothSystem,

                      device::mojom::UsbDeviceManager,

                      device::mojom::WakeLockProvider>()}})

           .Build();

    return *manifest;

  }

  }  // namespace foo

2. Static Constexpr Manifests

With this approach, you would only need to provide a manifest.h with a static definition of a Manifest object. However due to C++ language limitations, you would also need intermediate constants for storage of all the variable-length lists of things a manifest contains, like capability lists, interface lists within each capability, etc. So it would look more like:

  // *** services/foo/public/cpp/manifest.h

  #include "services/service_manager/public/cpp/manifest.h"

  namespace foo {

  constexpr service_manager::Manifest::InterfaceList<

      gpu::mojom::GpuHost>

      kInterfacesExposedForGpuAccess;

  constexpr service_manager::Manifest::InterfaceList<

      device::mojom::BluetoothSystem,

      device::mojom::UsbDeviceManager,

      device::mojom::WakeLockProvider>

      kInterfacesExposedForDeviceAccess;

  constexpr service_manager::Manifest::CapabilityList kExposedCapabilities[] = {

      {"gpu_access", kInterfacesExposedForGpuAccess},

      {"device_access", kInterfacesExposedForDeviceAccess}};

  constexpr service_manager::Manifest kManifest = service_manager::ManifestBuilder()

      .WithServiceName("my_pretty_cool_service")

      .WithOptions(service_manager::ManifestOptionsBuilder()

          .CanRegisterOtherServiceInstances(true)

          .WithSandboxType(sandbox::SandboxType::kNone))

      .WithExposedCapabilities(kExposedCapabilities)

      .Build();

  }  // namespace foo

As you can see, we can still provide a fairly nice builder-style API even in the constexpr case, but you still have to provide explicit storage for each list of things.

On one hand this may be easier to parse, visually; on the other hand it may feel like too much boilerplate to write.

It also means that the ManifestBuilder implementation requires a bunch of gnarly template goop to maintain constexpr-ness, and it means the actual Manifest data structure cannot use complex types like STL containers, making it harder to use at run-time if anyone has cause to do so.

One final thing to note is that in the constexpr case, there is essentially no code generated and that whole blob of stuff essentially compiles down to static data. This is quite a nice property, but I'm not sure it's nice enough to justify everything else.

** The advantages of scrapping JSON are numerous, with the common theme being to make the system less complex and easier to use and understand:

• Manifest definitions can reference actual interface symbols, so no more manifest typos only getting caught at run-time
• Manifest definitions for different services can share string constants (e.g. service names, capability names)
  
• Much easier to see what options a manifest supports and what each field means when it's just a regular data structure indexed by codesearch
• Preprocessor conditionals can be used to conditionally modify manifest contents
• We can migrate away from manifest "overlays" in favor of explicitly composing manifests in code in the few cases where overlays are used today
  
• We can use string resource IDs for display names where available
• We can use enums (like SandboxType) in lieu of string literals
• I could probably keep adding things to this list but it's already long enough

[Albert J. Wong (王重傑)]

We've talked a bunch so you know my opinion, but stating here for the record. Between the two, the function static version is likely superior. There is some amount of runtime overhead, but with the pure constexpr version, it's using enough C++ m magic that some configuration of some optimizer on some platform is likely to blow it up at some point.

-Albert

[Scott Violet]

Yay for taking this on!

I find (1) slightly easier to read, but I don't really feel strongly either way.

  -Scott

On Thu, Dec 13, 2018 at 3:20 PM Ken Rockot <roc...@google.com> wrote:

[James Cook]

I also prefer (1). And thanks for killing JSON manifests!

--
You received this message because you are subscribed to the Google Groups "services-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to services-dev...@chromium.org.
To post to this group, send email to servic...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/services-dev/CAKARY_kXg45Qxik3UiCgYUB15b6Ymn%2Be5dqLhWw5NRnGDNFiAA%40mail.gmail.com.

[John Abd-El-Malek]

I have a slight preference for 1 because of the less magic and simpler to understand code for non C++ gurus.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/services-dev/CAKLzqjFgBih6_pbk8%2BFjTwUYtHO4ZbkEicKi6FKz2y9-71LN-w%40mail.gmail.com.

[Colin Blundell]

Huge +1 to the overall project.

I prefer (1).

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/services-dev/CALhVsw1-rziGwoD1MS_LVSS2biASSJJdywGz7wQV8-OUost9pA%40mail.gmail.com.

[Olga Sharonova]

This will be very handy for finch experiments as well.

I also prefer (1).

Could you give an example of how "requires" part of the manifest will look like?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/services-dev/CAMGE5NEn5YjSYKBGMs-%2Bab8dVLOFuFWF6zim-AeDnQMN1vQnVA%40mail.gmail.com.

[Ken Rockot]

I love resounding consensus!

On Fri, Dec 14, 2018, 1:43 AM Olga Sharonova <ol...@chromium.org wrote:

This will be very handy for finch experiments as well.

I also prefer (1).

Could you give an example of how "requires" part of the manifest will look like?

In the  prototype it looks like:

    .WithRequiredCapabilities(

        {{"service_name", "capability_name"},

         {"service_name", "capability_name"}});

So if you require multiple capabilities from the same service, you'd just repeat that service's name for each one.

We could allow for something like:

  {{"service_name", "cap1", "cap2", ...}}

But I am thinking the simpler list of pairs will be easier for people to parse.

[Olga Sharonova]

Thanks! Both "required" options look nice to me.

Thank you for making manifests better!

Olga