Splitting platform into smaller targets

56 views
Skip to first unread message

Daniel Cheng

unread,
Sep 24, 2017, 10:31:15 AM9/24/17
to platform-architecture-dev
I was doing some refactoring and I noticed that we have dependency cycles for some of the code in platform that uses Mojo. Today, there are several places in the code where:

1. //third_party/WebKit/Source/platform depends on a Mojo service.
2. The Mojo service uses string or url.mojom.Url.
3. Which causes the Blink variant to use WTF::String or KURL.
4. Which are, of course, defined in platform.

This is problematic: we get away with it today by inlining all the code in the header and just not declaring the dependency on //third_party/WebKit/Source/platform in the typemap: see KURL.typemap as an example. I tried moving some code in the KURL struct traits to a separate source file, and the build promptly exploded: https://chromium-review.googlesource.com/c/chromium/src/+/680255

I would like to propose that:
- //third_party/WebKit/Source/platform may not depend on Mojo services.
- if code in //third_party/WebKit/Source/platform wants to use a Mojo service, it needs to be a separate build target.
- existing code that uses Mojo services in //third_party/WebKit/Source/platform needs to be moved to separate targets to follow this rule. This requires updating graphics, instrumentation, loader, mojo, network, and text/hyphenation to be their own targets.

An alternate strategy is to just pull weborigin into its own library; however, when I tried this, I ended up having to put WebString.cpp and runtime enabled features in yet another target, so that the weborigin library could depend on it. It felt pretty arbitrary to group WebString.cpp and runtime enabled features together like this just to get code to build.

Daniel

Daniel Bratell

unread,
Sep 25, 2017, 3:46:03 AM9/25/17
to platform-architecture-dev, Daniel Cheng
If the problem is KURL in particular I think there should be a specific plan for that one since its layering today with GURL as the backend is artificial, and, I assume, a remnant of the WebKit in Chromium time. Last I checked it was also in need of work to become compliant with https://url.spec.whatwg.org, work that at the time was not worth it because of the layering.

Pulling out weborigin into its own library, as you tried, sounds like a good thing to do even if it requires some massaging of types.

/Daniel
--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architect...@chromium.org.
To post to this group, send email to platform-arc...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/platform-architecture-dev/CAF3XrKqoaSLkkZVrQkYyK2gS7ErJe1gzNBuO%2BxFhSPKgZGMDAw%40mail.gmail.com.



--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

Colin Blundell

unread,
Sep 25, 2017, 4:27:15 AM9/25/17
to Daniel Bratell, platform-architecture-dev, Daniel Cheng
Can you remind me of what the vision is for the role of //third_party/WebKit/Source/platform in the post-BOS 2.0 world? I'm wondering what the right *conceptual* relationship between that directory and the services world is, before getting to the question of technical complexities.

Best,

Colin

Daniel Cheng

unread,
Sep 25, 2017, 4:52:16 AM9/25/17
to Daniel Bratell, platform-architecture-dev
On Mon, Sep 25, 2017 at 4:46 PM Daniel Bratell <bra...@opera.com> wrote:
If the problem is KURL in particular I think there should be a specific plan for that one since its layering today with GURL as the backend is artificial, and, I assume, a remnant of the WebKit in Chromium time. Last I checked it was also in need of work to become compliant with https://url.spec.whatwg.org, work that at the time was not worth it because of the layering.

Do you know what was missing? I think that it uses the same URL canonicalization routines under the hood, so in theory, the behavior should be the same. The main issues I would see with combining the implementation is that:
- we assume that KURL is cheap to copy
- we probably share the memory for the url spec with URLs returned via the v8 bindings
 

Pulling out weborigin into its own library, as you tried, sounds like a good thing to do even if it requires some massaging of types.

I had to run before I got it fully compiling, but I think that's actually what I'm going to do. There's too many interdependencies between things in platform to easily split it apart. For example, a lot of random files depend on the blob code which depends on Mojo.

Daniel

Daniel Bratell

unread,
Sep 25, 2017, 5:54:40 AM9/25/17
to platform-architecture-dev, Daniel Cheng
On Mon, 25 Sep 2017 10:52:02 +0200, Daniel Cheng <dch...@chromium.org> wrote:

On Mon, Sep 25, 2017 at 4:46 PM Daniel Bratell <bra...@opera.com> wrote:
If the problem is KURL in particular I think there should be a specific plan for that one since its layering today with GURL as the backend is artificial, and, I assume, a remnant of the WebKit in Chromium time. Last I checked it was also in need of work to become compliant with https://url.spec.whatwg.org, work that at the time was not worth it because of the layering.

Do you know what was missing? I think that it uses the same URL canonicalization routines under the hood, so in theory, the behavior should be the same. The main issues I would see with combining the implementation is that:
- we assume that KURL is cheap to copy
- we probably share the memory for the url spec with URLs returned via the v8 bindings

The problems I recall were mostly related to how to resolve urls relative other urls that were not perfect. Basically things you can trigger with an interesting <base> or <applet codebase> (This was before <applet was dropped>).

This was also before the repo merger (best thing to happen to the dev environment in recent years!)  which was probably the main hurdle to doing work.

/Daniel

--
/* Opera Software, Linköping, Sweden: CET (UTC+1) */

Kentaro Hara

unread,
Sep 25, 2017, 7:59:08 AM9/25/17
to Colin Blundell, Daniel Bratell, platform-architecture-dev, Daniel Cheng
On Mon, Sep 25, 2017 at 5:27 PM, Colin Blundell <blun...@chromium.org> wrote:
Can you remind me of what the vision is for the role of //third_party/WebKit/Source/platform in the post-BOS 2.0 world? I'm wondering what the right *conceptual* relationship between that directory and the services world is, before getting to the question of technical complexities.

After Onion Soup 2.0, //third_party/WebKit/Source/platform/ becomes a Blink-specific library. If a given library is used by Chromium and Blink, it should go to //base/, //cc/, //third_party/WebKit/common/ etc. If a given library is specific to Blink, it should go to //third_party/WebKit/Source/platform/.

Regarding the dependencies, I was thinking about the following model:

- platform/wtf/ does not depend on any other platform/xxx/.
- platform/xxx/ may depend on other platform/yyy/, but the dependencies are controlled by DEPS.
- platform/*/ is linked into one link target.

In other words, I wasn't thinking about the dependency problem Daniel pointed out.


 

Best,

Colin


On Mon, Sep 25, 2017 at 9:46 AM Daniel Bratell <bra...@opera.com> wrote:
If the problem is KURL in particular I think there should be a specific plan for that one since its layering today with GURL as the backend is artificial, and, I assume, a remnant of the WebKit in Chromium time. Last I checked it was also in need of work to become compliant with https://url.spec.whatwg.org, work that at the time was not worth it because of the layering.

Pulling out weborigin into its own library, as you tried, sounds like a good thing to do even if it requires some massaging of types.

/Daniel

On Sun, 24 Sep 2017 16:31:01 +0200, Daniel Cheng <dch...@chromium.org> wrote:

I was doing some refactoring and I noticed that we have dependency cycles for some of the code in platform that uses Mojo. Today, there are several places in the code where:

1. //third_party/WebKit/Source/platform depends on a Mojo service.
2. The Mojo service uses string or url.mojom.Url.
3. Which causes the Blink variant to use WTF::String or KURL.
4. Which are, of course, defined in platform.

This is problematic: we get away with it today by inlining all the code in the header and just not declaring the dependency on //third_party/WebKit/Source/platform in the typemap: see KURL.typemap as an example. I tried moving some code in the KURL struct traits to a separate source file, and the build promptly exploded: https://chromium-review.googlesource.com/c/chromium/src/+/680255

I would like to propose that:
- //third_party/WebKit/Source/platform may not depend on Mojo services.
- if code in //third_party/WebKit/Source/platform wants to use a Mojo service, it needs to be a separate build target.
- existing code that uses Mojo services in //third_party/WebKit/Source/platform needs to be moved to separate targets to follow this rule. This requires updating graphics, instrumentation, loader, mojo, network, and text/hyphenation to be their own targets.

An alternate strategy is to just pull weborigin into its own library; however, when I tried this, I ended up having to put WebString.cpp and runtime enabled features in yet another target, so that the weborigin library could depend on it. It felt pretty arbitrary to group WebString.cpp and runtime enabled features together like this just to get code to build.

Daniel
--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architecture-dev+unsub...@chromium.org.
To post to this group, send email to platform-architecture-dev@chromium.org.
--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architecture-dev+unsub...@chromium.org.
To post to this group, send email to platform-architecture-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architecture-dev+unsub...@chromium.org.
To post to this group, send email to platform-architecture-dev@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/platform-architecture-dev/CAMGE5NG_vsMhHJTcNrP%2Bc--L5af-itBG8WxryNr_yN-1Uuni6Q%40mail.gmail.com.



--
Kentaro Hara, Tokyo, Japan

Kentaro Hara

unread,
Sep 25, 2017, 8:02:39 AM9/25/17
to Daniel Cheng, Daniel Bratell, platform-architecture-dev
On Mon, Sep 25, 2017 at 5:52 PM, Daniel Cheng <dch...@chromium.org> wrote:
On Mon, Sep 25, 2017 at 4:46 PM Daniel Bratell <bra...@opera.com> wrote:
If the problem is KURL in particular I think there should be a specific plan for that one since its layering today with GURL as the backend is artificial, and, I assume, a remnant of the WebKit in Chromium time. Last I checked it was also in need of work to become compliant with https://url.spec.whatwg.org, work that at the time was not worth it because of the layering.

Do you know what was missing? I think that it uses the same URL canonicalization routines under the hood, so in theory, the behavior should be the same. The main issues I would see with combining the implementation is that:
- we assume that KURL is cheap to copy
- we probably share the memory for the url spec with URLs returned via the v8 bindings
 

Pulling out weborigin into its own library, as you tried, sounds like a good thing to do even if it requires some massaging of types.

I had to run before I got it fully compiling, but I think that's actually what I'm going to do. There's too many interdependencies between things in platform to easily split it apart. For example, a lot of random files depend on the blob code which depends on Mojo.

Would you help me understand how this will work? Won't it create the following cycle?

//third_party/WebKit/platform/ => //third_party/WebKit/weborigin/
//third_party/WebKit/weborigin/ => //third_party/WebKit/platform/ (because weborigin depends on WTF::String, which is defined in //third_party/WebKit/platform/wtf/)

Regarding whether we should unify KURL and GURL or not, see this thread.


 

Daniel


/Daniel

On Sun, 24 Sep 2017 16:31:01 +0200, Daniel Cheng <dch...@chromium.org> wrote:

I was doing some refactoring and I noticed that we have dependency cycles for some of the code in platform that uses Mojo. Today, there are several places in the code where:

1. //third_party/WebKit/Source/platform depends on a Mojo service.
2. The Mojo service uses string or url.mojom.Url.
3. Which causes the Blink variant to use WTF::String or KURL.
4. Which are, of course, defined in platform.

This is problematic: we get away with it today by inlining all the code in the header and just not declaring the dependency on //third_party/WebKit/Source/platform in the typemap: see KURL.typemap as an example. I tried moving some code in the KURL struct traits to a separate source file, and the build promptly exploded: https://chromium-review.googlesource.com/c/chromium/src/+/680255

I would like to propose that:
- //third_party/WebKit/Source/platform may not depend on Mojo services.
- if code in //third_party/WebKit/Source/platform wants to use a Mojo service, it needs to be a separate build target.
- existing code that uses Mojo services in //third_party/WebKit/Source/platform needs to be moved to separate targets to follow this rule. This requires updating graphics, instrumentation, loader, mojo, network, and text/hyphenation to be their own targets.

An alternate strategy is to just pull weborigin into its own library; however, when I tried this, I ended up having to put WebString.cpp and runtime enabled features in yet another target, so that the weborigin library could depend on it. It felt pretty arbitrary to group WebString.cpp and runtime enabled features together like this just to get code to build.

Daniel
--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architecture-dev+unsub...@chromium.org.
To post to this group, send email to platform-architecture-dev@chromium.org.



--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architecture-dev+unsub...@chromium.org.
To post to this group, send email to platform-architecture-dev@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/platform-architecture-dev/CAF3XrKrC8S06OfGxPLo5yiyx5o68ornczG2o6s0WjjVph7aNMQ%40mail.gmail.com.

Daniel Cheng

unread,
Sep 25, 2017, 9:33:05 AM9/25/17
to Kentaro Hara, Daniel Bratell, platform-architecture-dev
On Mon, Sep 25, 2017 at 9:02 PM Kentaro Hara <har...@chromium.org> wrote:
On Mon, Sep 25, 2017 at 5:52 PM, Daniel Cheng <dch...@chromium.org> wrote:
On Mon, Sep 25, 2017 at 4:46 PM Daniel Bratell <bra...@opera.com> wrote:
If the problem is KURL in particular I think there should be a specific plan for that one since its layering today with GURL as the backend is artificial, and, I assume, a remnant of the WebKit in Chromium time. Last I checked it was also in need of work to become compliant with https://url.spec.whatwg.org, work that at the time was not worth it because of the layering.

Do you know what was missing? I think that it uses the same URL canonicalization routines under the hood, so in theory, the behavior should be the same. The main issues I would see with combining the implementation is that:
- we assume that KURL is cheap to copy
- we probably share the memory for the url spec with URLs returned via the v8 bindings
 

Pulling out weborigin into its own library, as you tried, sounds like a good thing to do even if it requires some massaging of types.

I had to run before I got it fully compiling, but I think that's actually what I'm going to do. There's too many interdependencies between things in platform to easily split it apart. For example, a lot of random files depend on the blob code which depends on Mojo.

Would you help me understand how this will work? Won't it create the following cycle?

//third_party/WebKit/platform/ => //third_party/WebKit/weborigin/
//third_party/WebKit/weborigin/ => //third_party/WebKit/platform/ (because weborigin depends on WTF::String, which is defined in //third_party/WebKit/platform/wtf/)

wtf is a separate library, so weborigin will depend on that directly to avoid this dependency cycle.

Daniel
 

Regarding whether we should unify KURL and GURL or not, see this thread.


 

Daniel


/Daniel

On Sun, 24 Sep 2017 16:31:01 +0200, Daniel Cheng <dch...@chromium.org> wrote:

I was doing some refactoring and I noticed that we have dependency cycles for some of the code in platform that uses Mojo. Today, there are several places in the code where:

1. //third_party/WebKit/Source/platform depends on a Mojo service.
2. The Mojo service uses string or url.mojom.Url.
3. Which causes the Blink variant to use WTF::String or KURL.
4. Which are, of course, defined in platform.

This is problematic: we get away with it today by inlining all the code in the header and just not declaring the dependency on //third_party/WebKit/Source/platform in the typemap: see KURL.typemap as an example. I tried moving some code in the KURL struct traits to a separate source file, and the build promptly exploded: https://chromium-review.googlesource.com/c/chromium/src/+/680255

I would like to propose that:
- //third_party/WebKit/Source/platform may not depend on Mojo services.
- if code in //third_party/WebKit/Source/platform wants to use a Mojo service, it needs to be a separate build target.
- existing code that uses Mojo services in //third_party/WebKit/Source/platform needs to be moved to separate targets to follow this rule. This requires updating graphics, instrumentation, loader, mojo, network, and text/hyphenation to be their own targets.

An alternate strategy is to just pull weborigin into its own library; however, when I tried this, I ended up having to put WebString.cpp and runtime enabled features in yet another target, so that the weborigin library could depend on it. It felt pretty arbitrary to group WebString.cpp and runtime enabled features together like this just to get code to build.

Daniel
--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architect...@chromium.org.
To post to this group, send email to platform-arc...@chromium.org.



--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architect...@chromium.org.
To post to this group, send email to platform-arc...@chromium.org.

Daniel Cheng

unread,
Sep 25, 2017, 9:37:25 AM9/25/17
to Kentaro Hara, Colin Blundell, Daniel Bratell, platform-architecture-dev
On Mon, Sep 25, 2017 at 8:59 PM Kentaro Hara <har...@chromium.org> wrote:
On Mon, Sep 25, 2017 at 5:27 PM, Colin Blundell <blun...@chromium.org> wrote:
Can you remind me of what the vision is for the role of //third_party/WebKit/Source/platform in the post-BOS 2.0 world? I'm wondering what the right *conceptual* relationship between that directory and the services world is, before getting to the question of technical complexities.

After Onion Soup 2.0, //third_party/WebKit/Source/platform/ becomes a Blink-specific library. If a given library is used by Chromium and Blink, it should go to //base/, //cc/, //third_party/WebKit/common/ etc. If a given library is specific to Blink, it should go to //third_party/WebKit/Source/platform/.

I think of platform as something of a glue layer between Blink and non-Blink code. In the past, it was to allow Blink to directly consume things from //content via a layer of indirection. Today, it is also a glue layer for using things like //net directly in Blink. I'm not sure what it will look like going forward: we're allowing more base dependencies in the rest of Blink now, but I think we probably still don't want to expose things like std::string to the rest of Blink?

Daniel
 

Best,

Colin


To unsubscribe from this group and stop receiving emails from it, send an email to platform-architect...@chromium.org.
To post to this group, send email to platform-arc...@chromium.org.
--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architect...@chromium.org.
To post to this group, send email to platform-arc...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architect...@chromium.org.
To post to this group, send email to platform-arc...@chromium.org.

Kentaro Hara

unread,
Sep 25, 2017, 9:47:47 AM9/25/17
to Daniel Cheng, Daniel Bratell, platform-architecture-dev
On Mon, Sep 25, 2017 at 10:32 PM, Daniel Cheng <dch...@chromium.org> wrote:
On Mon, Sep 25, 2017 at 9:02 PM Kentaro Hara <har...@chromium.org> wrote:
On Mon, Sep 25, 2017 at 5:52 PM, Daniel Cheng <dch...@chromium.org> wrote:
On Mon, Sep 25, 2017 at 4:46 PM Daniel Bratell <bra...@opera.com> wrote:
If the problem is KURL in particular I think there should be a specific plan for that one since its layering today with GURL as the backend is artificial, and, I assume, a remnant of the WebKit in Chromium time. Last I checked it was also in need of work to become compliant with https://url.spec.whatwg.org, work that at the time was not worth it because of the layering.

Do you know what was missing? I think that it uses the same URL canonicalization routines under the hood, so in theory, the behavior should be the same. The main issues I would see with combining the implementation is that:
- we assume that KURL is cheap to copy
- we probably share the memory for the url spec with URLs returned via the v8 bindings
 

Pulling out weborigin into its own library, as you tried, sounds like a good thing to do even if it requires some massaging of types.

I had to run before I got it fully compiling, but I think that's actually what I'm going to do. There's too many interdependencies between things in platform to easily split it apart. For example, a lot of random files depend on the blob code which depends on Mojo.

Would you help me understand how this will work? Won't it create the following cycle?

//third_party/WebKit/platform/ => //third_party/WebKit/weborigin/
//third_party/WebKit/weborigin/ => //third_party/WebKit/platform/ (because weborigin depends on WTF::String, which is defined in //third_party/WebKit/platform/wtf/)

wtf is a separate library, so weborigin will depend on that directly to avoid this dependency cycle.

Ah, I was misunderstanding this.

When we moved wtf/ to platform/wtf/, we were talking about removing hacks around HeapVector and HeapHashTable that were needed because Oilpan and WTF existed in different link units. So I thought that platform/wtf/ was already merged to the platform/ link unit. (And now I realized that it's important to keep platform/wtf/ as a separate library.)

Do you think there's any option to move weborigin, KURL etc to platform/wtf/?



Daniel
 

Regarding whether we should unify KURL and GURL or not, see this thread.


 

Daniel


/Daniel

On Sun, 24 Sep 2017 16:31:01 +0200, Daniel Cheng <dch...@chromium.org> wrote:

I was doing some refactoring and I noticed that we have dependency cycles for some of the code in platform that uses Mojo. Today, there are several places in the code where:

1. //third_party/WebKit/Source/platform depends on a Mojo service.
2. The Mojo service uses string or url.mojom.Url.
3. Which causes the Blink variant to use WTF::String or KURL.
4. Which are, of course, defined in platform.

This is problematic: we get away with it today by inlining all the code in the header and just not declaring the dependency on //third_party/WebKit/Source/platform in the typemap: see KURL.typemap as an example. I tried moving some code in the KURL struct traits to a separate source file, and the build promptly exploded: https://chromium-review.googlesource.com/c/chromium/src/+/680255

I would like to propose that:
- //third_party/WebKit/Source/platform may not depend on Mojo services.
- if code in //third_party/WebKit/Source/platform wants to use a Mojo service, it needs to be a separate build target.
- existing code that uses Mojo services in //third_party/WebKit/Source/platform needs to be moved to separate targets to follow this rule. This requires updating graphics, instrumentation, loader, mojo, network, and text/hyphenation to be their own targets.

An alternate strategy is to just pull weborigin into its own library; however, when I tried this, I ended up having to put WebString.cpp and runtime enabled features in yet another target, so that the weborigin library could depend on it. It felt pretty arbitrary to group WebString.cpp and runtime enabled features together like this just to get code to build.

Daniel
--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architecture-dev+unsubsc...@chromium.org.
To post to this group, send email to platform-architecture-dev@chromium.org.



--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architecture-dev+unsubsc...@chromium.org.
To post to this group, send email to platform-architecture-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architecture-dev+unsubsc...@chromium.org.
To post to this group, send email to platform-architecture-dev@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/platform-architecture-dev/CAF3XrKpP8vzG_eXBPgVK9dGDM904F-KTnvb1MSx2%3DWMjBZ%2BUKQ%40mail.gmail.com.

Kentaro Hara

unread,
Sep 25, 2017, 9:56:09 AM9/25/17
to Daniel Cheng, Colin Blundell, Daniel Bratell, platform-architecture-dev
On Mon, Sep 25, 2017 at 10:37 PM, Daniel Cheng <dch...@chromium.org> wrote:
On Mon, Sep 25, 2017 at 8:59 PM Kentaro Hara <har...@chromium.org> wrote:
On Mon, Sep 25, 2017 at 5:27 PM, Colin Blundell <blun...@chromium.org> wrote:
Can you remind me of what the vision is for the role of //third_party/WebKit/Source/platform in the post-BOS 2.0 world? I'm wondering what the right *conceptual* relationship between that directory and the services world is, before getting to the question of technical complexities.

After Onion Soup 2.0, //third_party/WebKit/Source/platform/ becomes a Blink-specific library. If a given library is used by Chromium and Blink, it should go to //base/, //cc/, //third_party/WebKit/common/ etc. If a given library is specific to Blink, it should go to //third_party/WebKit/Source/platform/.

I think of platform as something of a glue layer between Blink and non-Blink code. In the past, it was to allow Blink to directly consume things from //content via a layer of indirection. Today, it is also a glue layer for using things like //net directly in Blink. I'm not sure what it will look like going forward: we're allowing more base dependencies in the rest of Blink now, but I think we probably still don't want to expose things like std::string to the rest of Blink?

Right.

My mental model is something like:

//base/, //cc/, //v8/ etc <---- Libraries shared between Chromium and Blink
↑
platform/ <---- Blink specific libraries
↑
core/, modules/ <---- The implementation of Web Platform (the specs and the IDL files).

core/ and modules/ are NOT expected to use random things from //base/ etc. core/ and modules/ are expected to use only platform/ and whitelisted //base/ things (i.e., //base/ things that are not abstracted at the platform/ layer).

Am I making sense?


 

Daniel
 

Best,

Colin


To unsubscribe from this group and stop receiving emails from it, send an email to platform-architecture-dev+unsub...@chromium.org.
To post to this group, send email to platform-architecture-dev@chromium.org.
--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architecture-dev+unsub...@chromium.org.
To post to this group, send email to platform-architecture-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architecture-dev+unsub...@chromium.org.
To post to this group, send email to platform-architecture-dev@chromium.org.

Daniel Cheng

unread,
Sep 25, 2017, 9:59:54 AM9/25/17
to Kentaro Hara, Daniel Bratell, platform-architecture-dev
On Mon, Sep 25, 2017 at 10:47 PM Kentaro Hara <har...@chromium.org> wrote:
On Mon, Sep 25, 2017 at 10:32 PM, Daniel Cheng <dch...@chromium.org> wrote:
On Mon, Sep 25, 2017 at 9:02 PM Kentaro Hara <har...@chromium.org> wrote:
On Mon, Sep 25, 2017 at 5:52 PM, Daniel Cheng <dch...@chromium.org> wrote:
On Mon, Sep 25, 2017 at 4:46 PM Daniel Bratell <bra...@opera.com> wrote:
If the problem is KURL in particular I think there should be a specific plan for that one since its layering today with GURL as the backend is artificial, and, I assume, a remnant of the WebKit in Chromium time. Last I checked it was also in need of work to become compliant with https://url.spec.whatwg.org, work that at the time was not worth it because of the layering.

Do you know what was missing? I think that it uses the same URL canonicalization routines under the hood, so in theory, the behavior should be the same. The main issues I would see with combining the implementation is that:
- we assume that KURL is cheap to copy
- we probably share the memory for the url spec with URLs returned via the v8 bindings
 

Pulling out weborigin into its own library, as you tried, sounds like a good thing to do even if it requires some massaging of types.

I had to run before I got it fully compiling, but I think that's actually what I'm going to do. There's too many interdependencies between things in platform to easily split it apart. For example, a lot of random files depend on the blob code which depends on Mojo.

Would you help me understand how this will work? Won't it create the following cycle?

//third_party/WebKit/platform/ => //third_party/WebKit/weborigin/
//third_party/WebKit/weborigin/ => //third_party/WebKit/platform/ (because weborigin depends on WTF::String, which is defined in //third_party/WebKit/platform/wtf/)

wtf is a separate library, so weborigin will depend on that directly to avoid this dependency cycle.

Ah, I was misunderstanding this.

When we moved wtf/ to platform/wtf/, we were talking about removing hacks around HeapVector and HeapHashTable that were needed because Oilpan and WTF existed in different link units. So I thought that platform/wtf/ was already merged to the platform/ link unit. (And now I realized that it's important to keep platform/wtf/ as a separate library.)

I think we could actually do this: basically, we'd have a "platform_base" that can't depend on Mojo, and foundational types like Oilpan, WTF, et cetera would all be in there.
 

Do you think there's any option to move weborigin, KURL etc to platform/wtf/?

For now, I made it a separate target (but it's a very rough draft). I have a CL uploaded, but I'm still tweaking it: I tried to fix PDF mojom dependencies and it ended up causing a completely unrelated mojom to stop linking.

Daniel
 



Daniel
 

Regarding whether we should unify KURL and GURL or not, see this thread.


 

Daniel


/Daniel

On Sun, 24 Sep 2017 16:31:01 +0200, Daniel Cheng <dch...@chromium.org> wrote:

I was doing some refactoring and I noticed that we have dependency cycles for some of the code in platform that uses Mojo. Today, there are several places in the code where:

1. //third_party/WebKit/Source/platform depends on a Mojo service.
2. The Mojo service uses string or url.mojom.Url.
3. Which causes the Blink variant to use WTF::String or KURL.
4. Which are, of course, defined in platform.

This is problematic: we get away with it today by inlining all the code in the header and just not declaring the dependency on //third_party/WebKit/Source/platform in the typemap: see KURL.typemap as an example. I tried moving some code in the KURL struct traits to a separate source file, and the build promptly exploded: https://chromium-review.googlesource.com/c/chromium/src/+/680255

I would like to propose that:
- //third_party/WebKit/Source/platform may not depend on Mojo services.
- if code in //third_party/WebKit/Source/platform wants to use a Mojo service, it needs to be a separate build target.
- existing code that uses Mojo services in //third_party/WebKit/Source/platform needs to be moved to separate targets to follow this rule. This requires updating graphics, instrumentation, loader, mojo, network, and text/hyphenation to be their own targets.

An alternate strategy is to just pull weborigin into its own library; however, when I tried this, I ended up having to put WebString.cpp and runtime enabled features in yet another target, so that the weborigin library could depend on it. It felt pretty arbitrary to group WebString.cpp and runtime enabled features together like this just to get code to build.

Daniel
--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architect...@chromium.org.
To post to this group, send email to platform-arc...@chromium.org.



--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architect...@chromium.org.
To post to this group, send email to platform-arc...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architect...@chromium.org.
To post to this group, send email to platform-arc...@chromium.org.
--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architect...@chromium.org.
To post to this group, send email to platform-arc...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/platform-architecture-dev/CABg10jy-n%3D0fsjwm%2BSErwzXKdC%3D0zE_ZAJzWfKAh%2BoXSQP%2BBjw%40mail.gmail.com.

Colin Blundell

unread,
Sep 25, 2017, 10:09:17 AM9/25/17
to Daniel Cheng, Kentaro Hara, Daniel Bratell, platform-architecture-dev
On Mon, Sep 25, 2017 at 3:59 PM Daniel Cheng <dch...@chromium.org> wrote:
On Mon, Sep 25, 2017 at 10:47 PM Kentaro Hara <har...@chromium.org> wrote:
On Mon, Sep 25, 2017 at 10:32 PM, Daniel Cheng <dch...@chromium.org> wrote:
On Mon, Sep 25, 2017 at 9:02 PM Kentaro Hara <har...@chromium.org> wrote:
On Mon, Sep 25, 2017 at 5:52 PM, Daniel Cheng <dch...@chromium.org> wrote:
On Mon, Sep 25, 2017 at 4:46 PM Daniel Bratell <bra...@opera.com> wrote:
If the problem is KURL in particular I think there should be a specific plan for that one since its layering today with GURL as the backend is artificial, and, I assume, a remnant of the WebKit in Chromium time. Last I checked it was also in need of work to become compliant with https://url.spec.whatwg.org, work that at the time was not worth it because of the layering.

Do you know what was missing? I think that it uses the same URL canonicalization routines under the hood, so in theory, the behavior should be the same. The main issues I would see with combining the implementation is that:
- we assume that KURL is cheap to copy
- we probably share the memory for the url spec with URLs returned via the v8 bindings
 

Pulling out weborigin into its own library, as you tried, sounds like a good thing to do even if it requires some massaging of types.

I had to run before I got it fully compiling, but I think that's actually what I'm going to do. There's too many interdependencies between things in platform to easily split it apart. For example, a lot of random files depend on the blob code which depends on Mojo.

Would you help me understand how this will work? Won't it create the following cycle?

//third_party/WebKit/platform/ => //third_party/WebKit/weborigin/
//third_party/WebKit/weborigin/ => //third_party/WebKit/platform/ (because weborigin depends on WTF::String, which is defined in //third_party/WebKit/platform/wtf/)

wtf is a separate library, so weborigin will depend on that directly to avoid this dependency cycle.

Ah, I was misunderstanding this.

When we moved wtf/ to platform/wtf/, we were talking about removing hacks around HeapVector and HeapHashTable that were needed because Oilpan and WTF existed in different link units. So I thought that platform/wtf/ was already merged to the platform/ link unit. (And now I realized that it's important to keep platform/wtf/ as a separate library.)

I think we could actually do this: basically, we'd have a "platform_base" that can't depend on Mojo, and foundational types like Oilpan, WTF, et cetera would all be in there.

What would the required dependencies of platform_base be? Presumably //base, as we continue to unify things like the callback systems? It would make sense for platform_base to depend only on things that are also "strictly lower than Mojo" in the food chain; that's not too many things though really I think.
 

Kentaro Hara

unread,
Sep 25, 2017, 7:45:56 PM9/25/17
to Colin Blundell, Daniel Cheng, Daniel Bratell, platform-architecture-dev
(I'm not planning to randomize this discussion but...) is there any option to make Mojo's Blink variant part of the platform link target?

Conceptually it looks strange that non-Web-Platform code (i.e., things other than core/ and modules/) depends on platform/. Mojo's Blink variant is doing that and it seems like the root cause of the problem.




 



Daniel
 

Daniel

To unsubscribe from this group and stop receiving emails from it, send an email to platform-architecture-dev+unsub...@chromium.org.
To post to this group, send email to platform-architecture-dev@chromium.org.



--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architecture-dev+unsub...@chromium.org.
To post to this group, send email to platform-architecture-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architecture-dev+unsub...@chromium.org.
To post to this group, send email to platform-architecture-dev@chromium.org.



--
Kentaro Hara, Tokyo, Japan

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architecture-dev+unsub...@chromium.org.
To post to this group, send email to platform-architecture-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architecture-dev+unsub...@chromium.org.
To post to this group, send email to platform-architecture-dev@chromium.org.

Daniel Cheng

unread,
Sep 25, 2017, 8:09:20 PM9/25/17
to Kentaro Hara, Colin Blundell, Daniel Bratell, platform-architecture-dev, Yuzhu Shen, Ken Rockot
On Tue, Sep 26, 2017 at 8:45 AM Kentaro Hara <har...@chromium.org> wrote:
(I'm not planning to randomize this discussion but...) is there any option to make Mojo's Blink variant part of the platform link target?

Conceptually it looks strange that non-Web-Platform code (i.e., things other than core/ and modules/) depends on platform/. Mojo's Blink variant is doing that and it seems like the root cause of the problem.

I think we probably can't do that for code that lives in //services that might be shared, such as resource coordinator and viz (?). I noted in https://chromium-review.googlesource.com/c/chromium/src/+/681616 that it felt awkward that we had to explicitly allow dependencies from Blink variants... it does feel like that should be automatically propagated?

I ended up solving my problem in a different way: I moved KURLStructTraits.* into the sources list for the platform target... which is of course, the opposite of making it smaller, but it fixes the build =)

To me, the bigger problem is while it's possible to make the build work in a number of different ways [1][2][3][4], it's not actually clear what the right way is. So I think we need to set clear guidelines, and then we can figure out what (if anything) to change about how Mojo bindings and dependencies work in Blink.

Daniel

 

 



Daniel
 

Daniel

To unsubscribe from this group and stop receiving emails from it, send an email to platform-architect...@chromium.org.
To post to this group, send email to platform-arc...@chromium.org.



--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architect...@chromium.org.
To post to this group, send email to platform-arc...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architect...@chromium.org.
To post to this group, send email to platform-arc...@chromium.org.



--
Kentaro Hara, Tokyo, Japan

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architect...@chromium.org.
To post to this group, send email to platform-arc...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architect...@chromium.org.
To post to this group, send email to platform-arc...@chromium.org.

Victor Costan

unread,
Oct 4, 2017, 9:45:20 AM10/4/17
to Daniel Cheng, Kentaro Hara, Colin Blundell, Daniel Bratell, platform-architecture-dev, Yuzhu Shen, Ken Rockot
I ran into this problem while trying to move a mojo interface in the network service so it can be shared between the implementation on Blink. I've given it some thought, and here's what I came up with. The e-mail is rather long so, to make up for it, I've put the conclusions in each section at the top.


Layering: Conceptually, we should have 3 layers: a WTF-like layer with types and their allocators, a layer that holds the Blink variants of the mojo clients, and a layer that holds the rest of Blink's platform. The argument is below.

I think it's fair to assume that we'd want Blink-specific types like KURL and WTF::Vector in the Blink variant of mojo's typemaps (Blink mojo typemaps from here on). The alternatives seem to be (a) using Chromium's typemaps, which'd introduce std::vector and GURL in Blink, or (b) using mojo's raw interfaces. Neither seems particularly appealing.

I assume that we want the dependency graphs between Chromium's build targets (and associated mental layering) to form a DAG. At the very least, I think GN will complain if there are cycles in the dependency graph that it sees. Also, having cycles in the mental layer model makes everything more difficult to reason about.

I also assume we still want the client code for each mojo interface into its own build target. Combined with the desire for Blink mojo typemaps, this translates into a requirement that each Blink variant of the client code is in its own build target.

Given that we want (at least) Vector and KURL in Blink mojo typemaps, it's inevitable that the Blink variants of the mojo interface clients (Blink mojo clients from here on) will depend on some layer in Blink. 

I assume that the Blink mojo clients belong in the Platform layer (note I didn't write "build target"), because Platform is the interface for the browser-side functionality used by Blink. Furthermore, (while some services will be consumed directly by the code in core and modules) some mojo services will be used by other Platform code. So, Blink mojo clients can't depend on Platform, to avoid a circular dependency. (Daniel mentioned this earlier)

Based on the above, I think the only solution is setting up a layer containing all the types that Blink typemaps can use. This seems similar to WTF, and I'd guess it might be the WTF types that don't move to //base. The immediate concerns could be addressed by moving KURL to WTF, and having the Blink mojo clients depend on WTF.

My last assumption is the most questionable one. It might also be the case that each Blink layer (platform, core, modules) would to have its mojo clients, so we'd have each layer divided into sub-layers (e.g. Blink core types, Blink core mojo clients, the rest of Blink core) and each sub-layer would be able to depend on previous layers (platform <- core <- modules). An example I've heard of recently that'd justify this approach is 


Component Build: This admits more than one solution, but all solutions end up requiring GN changes. I think that the simplest solution is to augment source sets with a "home" build target. When the source set is linked into the home target, its .o files are used. When the source set is linked into any other target, the home target is substituted as a dependency. This is a slightly more generalized form of the currently used overridden_* tricks (which only works in the mojo template). Argument below.

The component build docs say that a static library or source set target can't be depended on from multiple components. For this argument, it's more useful to think of this rule as no object file can be linked into two components. When this rule is broken, the target's code is duplicated in multiple components, and negative consequences range from build breakages to incorrect behavior (such as duplicate singletons, when using the static init pattern).

The component build docs suggest that the optimum component size is in the hundreds of compilation units. This implies that it's not acceptable to have each Blink mojo client be its own component. While per-client components would still require addressing the layering problem described above, they wouldn't require GN changes.

I assume that we want each Blink mojo client in a separate interface (there's a bit more reasoning in the previous argument), and that we want to be able to declare a dependency on a mojo interface in any Blink build target, by depending on the Blink variant of the mojo target.

A more specific form of the above: I assume that it's not feasible to require that Blink build targets have separate dependencies (hypothetical example: //service/foo:blink_interface vs //service/foo/blink_interface_impl) depending on what component they'll end up in. Breaking this assumption results in a solution that doesn't require GN modifications, but adds maintenance burden to Blink, as editing a BUILD.gn build target would require knowing which component it'd land in.

I also assume that the number of service mojo interfaces consumed by Blink will be in the high tens or low hundreds, so it is desirable to have all of them live in the same component. Breaking this assumption changes the simplicity ranking of alternatives, but not the fact that we'll need GN changes.

Last assumption, for the purpose of simplifying the discussion: Blink mojo clients will be represented as source sets, not as static libraries. The argument can be updated to account for static libraries with reasonable ease.

It's reasonable to expect that there will be at least one service that needs to be consumed by multiple Blink components. The example I'm familiar with is the network service, which needs to be used from core (because of the loader) and modules (fetch, async cookies). This means we'll have build targets that belong to multiple components depend on the Blink mojo client for the service, which is a separate target. Reconciling this reality with the component build requires that the dependency on the Blink mojo client is interpreted differently depending on which target it's coming from -- the mojo client's code must be instantiated in exactly one component, and all the other components that depend on the mojo client must depend on the one component that has the code. This requires GN changes.

The simplest dependency rewrite system I can think of would tie each source set to a component that I'll call the "home" component. When linked inside the home component, the source set dependency is treated as usual -- each source becomes an object file, all object files are linked together with the consumer's objects. When linked outside its home component, the source set dependency is rewritten into a dependency on the home component.

I assume that what I have said is not achievable with GN macros, given the current workarounds. I claim that the proposal above makes GN better even in the absence of the mojo problem, because it makes it easier to reason about dependencies in a component build.


I hope these thoughts help get to a sustainable solution for consuming mojo interfaces in Blink, and look forward to your feedback.

Thank you,
    Victor

Sam McNally

unread,
Oct 4, 2017, 11:09:22 PM10/4/17
to Victor Costan, Daniel Cheng, Kentaro Hara, Colin Blundell, Daniel Bratell, platform-architecture-dev, Yuzhu Shen, Ken Rockot
https://docs.google.com/document/d/1pLsOZ2PQ_FNZHx_U94ZJprDfGLQUgUkCssCoyC9TVVY/edit?usp=sharing is what I ended up with last time this came up.

Fixing this properly probably requires separating build target dependencies from components. That is, specifying dependencies for a target should be possible without consideration of same/different component. Until then, the confusion will continue to crop up.

For the original question, I don't see a problem with individual build targets for wtf, WebString, runtime_enabled_features, KURL and SecurityOrigin, modulo the problems with dependencies and components.

--
You received this message because you are subscribed to the Google Groups "platform-architecture-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-architect...@chromium.org.
To post to this group, send email to platform-arc...@chromium.org.
Reply all
Reply to author
Forward
0 new messages