Intent to Implement: V8 context snapshot

2,213 views
Skip to first unread message

Hitoshi Yoshida

unread,
Jul 12, 2017, 12:12:30 AM7/12/17
to blink-dev, Kentaro Hara, Yuki Shiino

Contact emails

pe...@chromium.org


Summary

Create V8 contexts from a snapshot file. It will make the initializations of V8 contexts for 1.6-3 times faster. (design doc, issue 617892)


Motivation

LocalWindowProxy::initialize() is a very performance sensitive method, because it is called per frame per extension. If Chromium has 10 extensions and opens a page including 10 iframes, it is called more than 100 times.

V8 context snapshot is a set of snapshot images of function templates and V8 contexts. We will make it at build time, and then create new function templates or contexts from the image in client processes. On our experiments, this routine runs 1.6-3 times faster than creating contexts from scratch, and it allows us to improve performance in new ways.


Interoperability and Compatibility Risk
None.


Ongoing technical constraints

None.


Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

It does not support Chrome OS for now, but we will support it soon.


Requesting approval to ship?

No


PhistucK

unread,
Jul 12, 2017, 2:01:50 AM7/12/17
to Hitoshi Yoshida, blink-dev, Kentaro Hara, Yuki Shiino
Is this user-facing at all (beside too fast page loads, of course ;))?


PhistucK

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJnxdXA%2BESgZCrtMnX-REqViTRwAuXJNOA%3DYuZT0%3DF%2BkGMq%2Bew%40mail.gmail.com.

Kentaro Hara

unread,
Jul 12, 2017, 3:26:08 AM7/12/17
to PhistucK, Hitoshi Yoshida, blink-dev, Yuki Shiino
Not user-facing. It just makes the frame creation 3x faster and make the world a better place :)


--
Kentaro Hara, Tokyo, Japan

PhistucK

unread,
Jul 12, 2017, 4:32:19 AM7/12/17
to Kentaro Hara, Hitoshi Yoshida, blink-dev, Yuki Shiino
Woo hoo! Congratulations!


PhistucK

Nico Weber

unread,
Jul 12, 2017, 7:39:54 AM7/12/17
to Hitoshi Yoshida, Kentaro Hara, blink-dev, Yuki Shiino, Andrew Grieve
It sounds like this adds 1.6MB to binary size, which is precious on Android. Is it possible to reduce the size hit, or to mitigate it by removing some existing system no longer needed with this new tech?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

Chen Zhixiang

unread,
Jul 12, 2017, 7:59:36 PM7/12/17
to blink-dev, har...@chromium.org, yukis...@chromium.org
Wow, it's very impressive!
I've done similar work, to enable PageCache on M56, Which need save the v8 context and restore it.
But i didn't see why sometimes it fails here:

//#if ENABLE_PAGE_CACHE void JSReceiver::SetCreationContext(Context* env) {   JSReceiver* receiver = this;   while (receiver->IsJSBoundFunction()) {     receiver = JSBoundFunction::cast(receiver)->bound_target_function();   }   Object* constructor = receiver->map()->GetConstructor();   JSFunction* function;   if (constructor->IsJSFunction()) {     function = JSFunction::cast(constructor);   } else {     // Functions have null as a constructor,     // but any JSFunction knows its context immediately.     CHECK(receiver->IsJSFunction()); //<--- Why this CHECK fails here?     function = JSFunction::cast(receiver);   }   function->set_context(env); } //#endif


and Oilpan, OOPIF, `pointer to const T&` frequently causes fails ... Hope this design doc can clarify something important....

Alexandre Elias

unread,
Jul 12, 2017, 8:32:34 PM7/12/17
to Nico Weber, Hitoshi Yoshida, Kentaro Hara, blink-dev, Yuki Shiino, Andrew Grieve
It's not entirely obvious to me from the data presented that this is an overall win on Android.  10ms is not an enormous win in the context of loading, and it's gaining that via tradeoff rather than Pareto optimization.  The APK size concern alone is a major concern for us, and also I'm also concerned about a possible effect on RAM, and also that we might be waiting on disk I/O on a separate thread you didn't measure (which might more than cancel the win).

It seems like the main goal of this project is to address the pathological extensions problem.  Since Chrome for Android doesn't support extensions, I suggest decoupling the decision to ship on desktop and Android.  Likely the current data is good enough to ship on desktop, but Android will require more debate/data-gathering and it's possible we will never want to ship this there.

Kentaro Hara

unread,
Jul 12, 2017, 8:38:36 PM7/12/17
to Alexandre Elias, Nico Weber, Hitoshi Yoshida, blink-dev, Yuki Shiino, Andrew Grieve
On Thu, Jul 13, 2017 at 9:32 AM, Alexandre Elias <ael...@chromium.org> wrote:
It's not entirely obvious to me from the data presented that this is an overall win on Android.  10ms is not an enormous win in the context of loading, and it's gaining that via tradeoff rather than Pareto optimization.  The APK size concern alone is a major concern for us, and also I'm also concerned about a possible effect on RAM, and also that we might be waiting on disk I/O on a separate thread you didn't measure (which might more than cancel the win).

It seems like the main goal of this project is to address the pathological extensions problem.  Since Chrome for Android doesn't support extensions, I suggest decoupling the decision to ship on desktop and Android.  Likely the current data is good enough to ship on desktop, but Android will require more debate/data-gathering and it's possible we will never want to ship this there.

Yeah, this is a good point. The context snapshot is a clear win on desktops but it's not clear on Android for the reasons you mentioned (APK size, disk I/O cost, no extensions).

+1 to limiting the context snapshot to desktops in short term. Then we can collect more data on Android and see if it makes sense on Android as well.



On Wed, Jul 12, 2017 at 4:39 AM, Nico Weber <tha...@chromium.org> wrote:
It sounds like this adds 1.6MB to binary size, which is precious on Android. Is it possible to reduce the size hit, or to mitigate it by removing some existing system no longer needed with this new tech?

On Jul 12, 2017 12:12 AM, "Hitoshi Yoshida" <pe...@chromium.org> wrote:

Contact emails

pe...@chromium.org


Summary

Create V8 contexts from a snapshot file. It will make the initializations of V8 contexts for 1.6-3 times faster. (design doc, issue 617892)


Motivation

LocalWindowProxy::initialize() is a very performance sensitive method, because it is called per frame per extension. If Chromium has 10 extensions and opens a page including 10 iframes, it is called more than 100 times.

V8 context snapshot is a set of snapshot images of function templates and V8 contexts. We will make it at build time, and then create new function templates or contexts from the image in client processes. On our experiments, this routine runs 1.6-3 times faster than creating contexts from scratch, and it allows us to improve performance in new ways.


Interoperability and Compatibility Risk
None.


Ongoing technical constraints

None.


Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

It does not support Chrome OS for now, but we will support it soon.


Requesting approval to ship?

No


--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJnxdXA%2BESgZCrtMnX-REqViTRwAuXJNOA%3DYuZT0%3DF%2BkGMq%2Bew%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMGbLiFkddj_Eu6vP-PKuDZizZ8_o%2BCbc1PZ9p%3D1-Qj_jCU3pA%40mail.gmail.com.

Hitoshi Yoshida

unread,
Jul 12, 2017, 10:43:31 PM7/12/17
to Kentaro Hara, Alexandre Elias, Nico Weber, blink-dev, Yuki Shiino, Andrew Grieve
Agreed. I'll change the CL under review to take it down on Android.
--
Hitoshi Yoshida (Peria)

Jochen Eisinger

unread,
Jul 13, 2017, 5:39:46 AM7/13/17
to Hitoshi Yoshida, Kentaro Hara, Alexandre Elias, Nico Weber, blink-dev, Yuki Shiino, Andrew Grieve
Note that V8 starts from a snapshot in any case, and the new snapshot is not considerably larger than the old one (we do ship both, however, as we also need the old snapshot e.g. for network proxy auto config), so I don't think that the disk IO time matters.

PhistucK

unread,
Jul 13, 2017, 5:50:31 AM7/13/17
to Jochen Eisinger, Hitoshi Yoshida, Kentaro Hara, Alexandre Elias, Nico Weber, blink-dev, Yuki Shiino, Andrew Grieve
Regarding the proxy use - is this loaded once (on startup)? If so, is it really necessary to keep that snapshot as well? Perhaps just let it load without a snapshot.
Then the resulting binary size would not even really change, even on Android.

I mean, does it use the snapshot for real performance reasons, or just because the code already did that and it is fast, so it was used there as well?


PhistucK

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

Jochen Eisinger

unread,
Jul 13, 2017, 5:58:41 AM7/13/17
to PhistucK, Hitoshi Yoshida, Kentaro Hara, Alexandre Elias, Nico Weber, blink-dev, Yuki Shiino, Andrew Grieve
The network was just an example, there are several other instances where we need a context fast, but it doesn't contain an Window object, e.g. for workers.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

Kentaro Hara

unread,
Jul 28, 2017, 5:10:06 AM7/28/17
to Jochen Eisinger, yangguo, Toon Verwaest, PhistucK, Hitoshi Yoshida, Alexandre Elias, Nico Weber, blink-dev, Yuki Shiino, Andrew Grieve
I saw your CL finally landed on ToT! Combined with other optimizations we've already landed, the V8 context initialization became >3 times faster :D

Thanks Yang and Toon for working on the V8 side and Peria for working on the Blink side. This was a great collaboration project!



To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

Yang Guo

unread,
Jul 28, 2017, 5:37:54 AM7/28/17
to Kentaro Hara, Jochen Eisinger, Toon Verwaest, PhistucK, Hitoshi Yoshida, Alexandre Elias, Nico Weber, blink-dev, Yuki Shiino, Andrew Grieve

Congrats on landing!

I don't think I was aware of this particular email thread. There seems to be a misconception about snapshot size.

The full snapshot takes about 1.6mb, but that's mostly the startup snapshot to initialize the isolate. The context snapshot merely takes 60kb. There can be more than one context snapshot, with each Additional one taking 60kb, or more depending on its content.

I think that an additional 60kb is acceptable even for low end devices and Android. To put into perspective: the startup snapshot contains some builtin code that are in the same order of magnitude. The impact of the snapshot is even more pronounced on low end devices.

Cheers,

Yang


To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

Colin Blundell

unread,
Jul 28, 2017, 5:48:03 AM7/28/17
to Yang Guo, Kentaro Hara, Jochen Eisinger, Toon Verwaest, PhistucK, Hitoshi Yoshida, Alexandre Elias, Nico Weber, blink-dev, Yuki Shiino, Andrew Grieve
On Fri, Jul 28, 2017 at 11:37 AM Yang Guo <yan...@chromium.org> wrote:

Congrats on landing!

I don't think I was aware of this particular email thread. There seems to be a misconception about snapshot size.

The full snapshot takes about 1.6mb, but that's mostly the startup snapshot to initialize the isolate. The context snapshot merely takes 60kb. There can be more than one context snapshot, with each Additional one taking 60kb, or more depending on its content.

I think that an additional 60kb is acceptable even for low end devices and Android. To put into perspective: the startup snapshot contains some builtin code that are in the same order of magnitude. The impact of the snapshot is even more pronounced on low end devices.

Cheers,

Yang


On Fri, Jul 28, 2017, 11:10 Kentaro Hara <har...@chromium.org> wrote:
I saw your CL finally landed on ToT! Combined with other optimizations we've already landed, the V8 context initialization became >3 times faster :D

BTW, the CL unfortunately is being reverted due to an MSAN problem.
 

Hitoshi Yoshida

unread,
Jul 29, 2017, 12:43:48 PM7/29/17
to Yang Guo, Kentaro Hara, Jochen Eisinger, Toon Verwaest, PhistucK, Alexandre Elias, Nico Weber, blink-dev, Yuki Shiino, Andrew Grieve
2017-07-28 18:37 GMT+09:00 Yang Guo <yan...@chromium.org>:

Congrats on landing!

I don't think I was aware of this particular email thread. There seems to be a misconception about snapshot size.

The full snapshot takes about 1.6mb, but that's mostly the startup snapshot to initialize the isolate. The context snapshot merely takes 60kb. There can be more than one context snapshot, with each Additional one taking 60kb, or more depending on its content.

I think that an additional 60kb is acceptable even for low end devices and Android. To put into perspective: the startup snapshot contains some builtin code that are in the same order of magnitude. The impact of the snapshot is even more pronounced on low end devices.

Yeah, but my current CL makes an independent snapshot fie, and it requires 1.6MB for 10 function templates and 3 contexts (including a defalut context).
I'll try to reduce the size with collaborating with V8's snapshot to work for low end Android devices. (I'm sorry, I will need your help more.)

Anyway, thank you for your help and also for your future helps. :P
I'll fix the issues on some builds soon.

Cheers,

Yang


To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.



--
Kentaro Hara, Tokyo, Japan

Yang Guo

unread,
Jul 29, 2017, 2:08:44 PM7/29/17
to Hitoshi Yoshida, Kentaro Hara, Jochen Eisinger, Toon Verwaest, PhistucK, Alexandre Elias, Nico Weber, blink-dev, Yuki Shiino, Andrew Grieve

I'm surprised to hear that you are using an independent snapshot file. You should be able to simply extend the existing snapshot file with additional contexts and templates. These should not take a lot of space in comparison to the startup snapshot, which contains a lot of context-independent things, for example close to 1mb of code objects. These things are redundant if you ship two snapshot files.

Anyways, if you have questions, please let me know!

Cheers,

Yang



Cheers,

Yang


To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.



--
Kentaro Hara, Tokyo, Japan

zheny...@intel.com

unread,
Sep 14, 2017, 11:01:23 PM9/14/17
to blink-dev, har...@chromium.org, yukis...@chromium.org
Very happy to see this feature! And I have some questions about this:

I tried it on Linux but seems failed. I generated v8_context_snapshot.bin in out/Default running ninja generate_v8_context_snapshot, but when I collected some trace file, I didn't observe performance improvement on LocalWindowProxy::Initialize.
Do we need some extra work like additional args/flags to make this feature work?

And what about ChromeOS? Do you have some kind of plan or people working on it?



On Wednesday, July 12, 2017 at 12:12:30 PM UTC+8, Hitoshi Yoshida wrote:

Yuki Shiino

unread,
Sep 15, 2017, 3:21:31 AM9/15/17
to zheny...@intel.com, Hitoshi Yoshida, blink-dev, Kentaro Hara, Yuki Shiino
+To: peria@ (somehow he's dropped off)

Krishnan Ramanathan

unread,
Sep 15, 2017, 3:35:51 AM9/15/17
to blink-dev
Yes

Hitoshi Yoshida

unread,
Sep 19, 2017, 1:21:17 AM9/19/17
to Shan, Zhenyu, blink-dev, Kentaro Hara, Yuki Shiino
I'm sorry for missing your message.

Though I already sent a response to Zhenyu directly, let me answer questions again.

I believe you can build v8_context_snapshot.bin in building chrome.
use_v8_context_snapshot handles if the file should be built or not, so I guess your environment or gn args turn it false.

When you build the file with chrome, you'll find "V8ContextSnapshot" option in chrome://flags.
Please change it to Enabled and relaunch Chrome.
(Note: chome://flags says the feature is available on ChromeOS, but it is unfortunately not, because of build dependencies.)

We would like to enable the feature on all environment (currently disabled on ChromeOS and Android),
but now we are not working for ChromeOS.


thanks,

2017-09-15 12:01 GMT+09:00 <zheny...@intel.com>:

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

zheny...@intel.com

unread,
Oct 11, 2017, 5:39:05 AM10/11/17
to blink-dev, zheny...@intel.com, har...@chromium.org, yukis...@chromium.org, pe...@chromium.org
Here are some discussions in emails between peria & me, moving to this thread:

Me:

Following your instruction I successfully tried the feature, and found the improvement very promising. Nice work done!
 
I’d like to have a try enabling this on ChromeOS. Before I start I have two more questions, which hope you can kindly answer:
 
1.       Have you considered generating snapshot at browser’s first startup, instead of build time?
This can resolve many issues, like ChromeOS building/cross building/Android binary size…etc., in exchange of an extended first startup time, which is not very obvious on desktop (<0.1s).
Is there anything blocking doing this, or a slow startup not acceptable, or building snapshot generator into Chrome too complex?
 
2.       What do you think could be the better way to resolve the building dependency? Here’s what’s on my mind:
a.       Adding required library into ChromeOS chroot build environment to make it work like a normal Linux building environment
b.      Reducing deps in tools/v8_context_snapshot/BUILD.gn, as I’m pretty sure it doesn’t need all of Blink & V8



Peria:

1. Yes, it was an option to build the snapshot at each user-side, and I think it a doable way.
We decided to deliver a built snapshot because it is easy to make a program.
 
AFAIK, we have no blocking issues, but I feel it a heavy task. Action items are following.
- Check if a snapshot already exists. If it doesn't or it is old enough, we have to remake it.
- We have to make a routine to communicate between browser process and a renderer process to write down a snapshot file.
  - Only browser process can create a file.
  - A renderer is needed to work with Blink component.
 
2. if you would like to use snapshot on ChromeOS with current system, I feel (b) is better, while (a) is easy to work for.
I have no other idea to resolve the issue.



I'll go with (b), reducing context generator's dependencies first.
I can't promise any progress though, at least for now. But I guess it won't hurt as no one else is currently working on this...

zheny...@intel.com

unread,
Nov 23, 2017, 1:37:04 AM11/23/17
to blink-dev, zheny...@intel.com, pe...@chromium.org, hong....@intel.com
Sorry for lack of update as I was busy on other things. As we continue to work on this, what do you think if we enable this on IA platform chromebooks at first? Currently we may not have enough resources for enabling & validation on ARM chromebooks

在 2017年10月11日星期三 UTC+8下午5:39:05,zheny...@intel.com写道:

Hitoshi Yoshida

unread,
Nov 28, 2017, 12:36:05 AM11/28/17
to Shan, Zhenyu, blink-dev, hong....@intel.com
I don't think it depends on architectures, but I think it acceptable.
You may test ARM behaviors on trybots.

Nico Weber

unread,
Jan 18, 2018, 10:52:08 AM1/18/18
to Hitoshi Yoshida, blink-dev, Kentaro Hara, Yuki Shiino
I noticed that this made it necessary to run blink bits as part of the build process. This means that in cross builds, we now build blink twice, once for the host and once for the target. This dramatically slows down builds. Would it be possible to use these full snapshots only in is_official_build builds, so that it doesn't slow down most developers?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

Hitoshi Yoshida

unread,
Jan 19, 2018, 2:01:52 AM1/19/18
to Nico Weber, blink-dev, Kentaro Hara, Yuki Shiino
Welcome back, Nico.

Yes, your concern is correct. This requires to build Blink twice on cross builds and hence slows down build time.
I agree to disable it in some environments to unblock developers, but I also hope this feature be tested on more
environments than is_official_builds.

I'd keep this on non-cross builds and (at least) one android trybot in CQ.
What do you think?

Dirk Pranke

unread,
Jan 23, 2018, 8:34:54 PM1/23/18
to Hitoshi Yoshida, Nico Weber, blink-dev, Kentaro Hara, Yuki Shiino
I think it could make sense to have a flag to turn this off, assuming the v8 and bindings teams are willing to support that, but I don't think this should be official-build-only. If this is what we're shipping, it should be on and tested by default and devs should have to opt-out of it, like they do today with NaCl.

Though, I'd be curious what "dramatically" actually means (and possibly open to changing my position), and I'd be curious to know if this had an impact on test cycle time as well ...

-- Dirk 

Hitoshi Yoshida

unread,
Jan 24, 2018, 1:12:48 AM1/24/18
to Dirk Pranke, Nico Weber, blink-dev, Kentaro Hara, Yuki Shiino
Currently v8_context_snapshot.bin is created if use_v8_context_snapshot is true.
It is set true by default depending on the build environment (see v8_context_snapshot.gni) but you can set it false in gn args.

IIUC, most trybots do not build blink twice, because they are not cross-build bots or setting use_v8_context_snapshot false.
However, in near future, we want to enable this feature also on Android, and it must increase compile time for x1.5.
(Some my WIP CLs show that compile time will become 46 min. from 29 min on android_n5x_swarming_rel.)
So it worth discussing what the default value of use_v8_context_snapshot should be.


Hitoshi Yoshida

unread,
Jan 25, 2018, 2:38:23 AM1/25/18
to Dirk Pranke, Nico Weber, blink-dev, Kentaro Hara, Yuki Shiino
2018-01-24 15:12 GMT+09:00 Hitoshi Yoshida <pe...@chromium.org>:
Currently v8_context_snapshot.bin is created if use_v8_context_snapshot is true.
It is set true by default depending on the build environment (see v8_context_snapshot.gni) but you can set it false in gn args.

IIUC, most trybots do not build blink twice, because they are not cross-build bots or setting use_v8_context_snapshot false.
However, in near future, we want to enable this feature also on Android, and it must increase compile time for x1.5.
(Some my WIP CLs show that compile time will become 46 min. from 29 min on android_n5x_swarming_rel.)
So it worth discussing what the default value of use_v8_context_snapshot should be.

Follow-up: "compile time" in my last message figures time of a task labeled "compile" in a build,
and it does not include time for tests nor setting up.

Yuki Shiino

unread,
Jan 25, 2018, 4:32:07 AM1/25/18
to Hitoshi Yoshida, dpr...@chromium.org, Nico Weber, blink-dev, Kentaro Hara, Yuki Shiino
Assuming that the time spent for testing is the same at all, the increase of compile time wouldn't be a huge impact, I guess?  I think that a variety of tests are much more time consuming.

Cheers,
Yuki Shiino



2018年1月25日(木) 16:38 Hitoshi Yoshida <pe...@chromium.org>:

Dirk Pranke

unread,
Jan 25, 2018, 12:30:34 PM1/25/18
to Yuki Shiino, Hitoshi Yoshida, Nico Weber, blink-dev, Kentaro Hara
We spend a lot of time running tests, but because of swarming can be run basically completely in parallel, and so we actually try to target them all completing in 10-15 minutes. So, a compile time increase of 50% from 29 to 46 minutes is a *substantial* regression in comparison :(.

-- Dirk

Primiano Tucci

unread,
Jan 25, 2018, 12:48:11 PM1/25/18
to Dirk Pranke, Yuki Shiino, Hitoshi Yoshida, Nico Weber, blink-dev, Kentaro Hara
(Note that even if you make this an is_official_build only thing, the perf waterfall builds each individual CL without batching, for bisect reasons, with is_official_build=true)

Dirk Pranke

unread,
Jan 25, 2018, 10:12:45 PM1/25/18
to Yuki Shiino, Hitoshi Yoshida, Nico Weber, blink-dev, Kentaro Hara
I've belatedly realized that thakis@ wrote "in cross builds", and the feature isn't enabled for Android or ChromeOS (or iOS, obviously), which means that you're either talking about Fuchsia, or  linux->win or mac->win or something like that. Which isn't "most developers" :). And, I see that there is already a flag to turn off the feature. So, this looks mostly fine.

It would be interesting to see if there was some way to avoid building *all* of Blink for this, but I don't know how feasible that is?

-- Dirk

Daniel Bratell

unread,
Jan 29, 2018, 12:14:30 PM1/29/18
to Dirk Pranke, Hitoshi Yoshida, Nico Weber, blink-dev, Kentaro Hara, Yuki Shiino
On Wed, 24 Jan 2018 07:12:02 +0100, Hitoshi Yoshida <pe...@chromium.org> wrote:

Currently v8_context_snapshot.bin is created if use_v8_context_snapshot is true.
It is set true by default depending on the build environment (see v8_context_snapshot.gni) but you can set it false in gn args.

IIUC, most trybots do not build blink twice, because they are not cross-build bots or setting use_v8_context_snapshot false.
However, in near future, we want to enable this feature also on Android, and it must increase compile time for x1.5.
(Some my WIP CLs show that compile time will become 46 min. from 29 min on android_n5x_swarming_rel.)
So it worth discussing what the default value of use_v8_context_snapshot should be.

29 -> 46 minutes, that would be with goma? What would be the change for a local build?

I'm not sure exactly where this will apply but 50% higher compilation times in general would be painful when it can already take hours.

/Daniel

--
/* Opera Software, Linköping, Sweden: CET (UTC+1) */

Hitoshi Yoshida

unread,
Jan 31, 2018, 11:01:05 PM1/31/18
to Daniel Bratell, Dirk Pranke, Nico Weber, blink-dev, Kentaro Hara, Yuki Shiino
I'm sorry. The number 29->46 minutes is not reliable, because trybots builds incrementally.
I tried to build everything on my machine using goma.
  $ ninja -j 500 -C out/Android
Then the compile time changes from 17'30" to 19'16", and hence its regression is ~10%.

Daniel Bratell

unread,
Feb 1, 2018, 6:14:13 AM2/1/18
to Hitoshi Yoshida, Dirk Pranke, Nico Weber, blink-dev, Kentaro Hara, Yuki Shiino
From a non-Googlers point of view, I am curious about the effect when you don't have access to 500 parallel cores to compile on. Not everyone have access to that kind of hardware. If the added targets end up using 200 cores that would otherwise have been idle, I can see why it only took 10% more time, but it would look completely different for someone who uses all available cores already.

/Daniel
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

Kentaro Hara

unread,
Feb 2, 2018, 12:48:30 AM2/2/18
to Daniel Bratell, Hitoshi Yoshida, Dirk Pranke, Nico Weber, blink-dev, Yuki Shiino
Here are a couple of my thoughts:

- I don't think it's acceptable to significantly regress build time on non-Google developers.

- In the first place, I don't yet understand why we have to build Blink twice in the cross-compiling environment. Given that the context snapshot is used by the target machine, the snapshot should be taken at the target machine -- then why do we need to build Blink on the host machine?

- Either way, I think we need to come up with a plan that 1) doesn't significantly affect build time of trybots or non-Google developers and 2) doesn't regress test coverage. Let's discuss the details among experts offline and get back to this thread.




To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.



--
/* Opera Software, Linköping, Sweden: CET (UTC+1) */

Hitoshi Yoshida

unread,
Mar 19, 2018, 3:49:17 AM3/19/18
to Kentaro Hara, Daniel Bratell, Dirk Pranke, Nico Weber, blink-dev, Yuki Shiino
I'm sorry not to reply this thread so long.

As an internal discussion, we decide to enable this feature on official build Android Chrome.
(To move this decision public forward, we also need other agreements. And I'll send another Intent-to-Ship for it, with adding some metrics.)

So the current situation about this feature on Android Chromium is
- Disabled by default.
- Each developer can enable it with having "use_v8_context_snapshot=true" in their arg.gn. (Yes, you already can try it locally.)
- It needs to build Blink twice (once for Android, and the other for the host machine), so it takes time to build.
and we'd like to update as
- Enable it on official build (is_official_build=true)
- Still need to put "use_v8_context_snapshot=true" if each developer wants to enable it.
- The change will not affect on general developers environment, i.e. in local build and CQ.
- General bugs of V8ContextSnapshot feature will be caught on other platforms' trybots in CQ.
- Android specific bugs of V8ContextSnapshot feature will be caught in an official bot, which is out of CQ. But the case should be rare, and it should be reproduced on local machines.

stay tuned for the intent-to-ship.

2018年2月2日(金) 14:48 Kentaro Hara <har...@chromium.org>:
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.



--
/* Opera Software, Linköping, Sweden: CET (UTC+1) */



--
Kentaro Hara, Tokyo, Japan

Ojan Vafai

unread,
Mar 19, 2018, 2:12:58 PM3/19/18
to Hitoshi Yoshida, Kentaro Hara, Daniel Bratell, Dirk Pranke, Nico Weber, blink-dev, Yuki Shiino
Is there an eventual path towards enabling this for release builds and desktop? A lot of people do local performance testing on release builds, so it's best if the differences between release and official are as minimal as possible to avoid performance testing on something too different from what we see in the wild.


Dirk Pranke

unread,
Mar 19, 2018, 2:20:47 PM3/19/18
to Ojan Vafai, Hitoshi Yoshida, Kentaro Hara, Daniel Bratell, Nico Weber, blink-dev, Yuki Shiino
Isn't it already enabled for release desktop builds? I thought from up-thread it was only disabled for Android, CrOS, and cross-builds.

If this requires us to compile Blink twice, I think it's probably not worth it to enable for Android release builds, just like we wouldn't enable PGO or LTO by default.

-- Dirk

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.



--
/* Opera Software, Linköping, Sweden: CET (UTC+1) */



--
Kentaro Hara, Tokyo, Japan


--
Hitoshi Yoshida (Peria)

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

Kentaro Hara

unread,
Mar 19, 2018, 2:22:14 PM3/19/18
to Dirk Pranke, Ojan Vafai, Hitoshi Yoshida, Daniel Bratell, Nico Weber, blink-dev, Yuki Shiino
On Mon, Mar 19, 2018 at 7:20 PM, Dirk Pranke <dpr...@chromium.org> wrote:
Isn't it already enabled for release desktop builds? I thought from up-thread it was only disabled for Android, CrOS, and cross-builds.

This is correct.

Hitoshi Yoshida

unread,
Mar 19, 2018, 10:26:59 PM3/19/18
to Dirk Pranke, oj...@chromium.org, Kentaro Hara, Daniel Bratell, Nico Weber, blink-dev, Yuki Shiino

2018年3月20日(火) 3:20 Dirk Pranke <dpr...@chromium.org>:
Isn't it already enabled for release desktop builds? I thought from up-thread it was only disabled for Android, CrOS, and cross-builds.
Yes, it's already enabled on desktop platforms.
We would like to additionally enable it on (a part of) Android builds.
CrOS and other cross-build platforms are not supported yet, and not the target of this update.


If this requires us to compile Blink twice, I think it's probably not worth it to enable for Android release builds, just like we wouldn't enable PGO or LTO by default.

-- Dirk

On Mon, Mar 19, 2018 at 11:12 AM, Ojan Vafai <oj...@chromium.org> wrote:
Is there an eventual path towards enabling this for release builds and desktop? A lot of people do local performance testing on release builds, so it's best if the differences between release and official are as minimal as possible to avoid performance testing on something too different from what we see in the wild.

I believe it does not affect most metrics, but yes, it's better to care performance tests.
Thank you for pointing it out.
 

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.



--
/* Opera Software, Linköping, Sweden: CET (UTC+1) */



--
Kentaro Hara, Tokyo, Japan


--
Hitoshi Yoshida (Peria)

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

Yuki Shiino

unread,
Sep 7, 2021, 8:27:59 AM9/7/21
to Rahul Patel, blink-dev, Hitoshi Yoshida, Ojan Vafai, Kentaro Hara, Daniel Bratell, Nico Weber, Yuki Shiino, Dirk Pranke
You can disable the feature by setting a GN flag to false.

You can set `use_v8_context_snapshot` to false.

Cheers,
Yuki Shiino


2021年9月7日(火) 21:09 Rahul Patel <patelrah...@gmail.com>:
Nice feature, i have a quick questions on enabled this feature?

Just for my learning purpose Can we disabled it for linux too? 

Rahul Patel

unread,
Sep 7, 2021, 11:54:56 AM9/7/21
to blink-dev, Hitoshi Yoshida, Ojan Vafai, Kentaro Hara, Daniel Bratell, Nico Weber, blink-dev, Yuki Shiino, Dirk Pranke
Nice feature, i have a quick questions on enabled this feature?

Just for my learning purpose Can we disabled it for linux too? 

On Tuesday, March 20, 2018 at 7:56:59 AM UTC+5:30 Hitoshi Yoshida wrote:

Yuki Shiino

unread,
Sep 8, 2021, 6:00:04 AM9/8/21
to Rahul Patel, Yuki Shiino, blink-dev, Hitoshi Yoshida, Ojan Vafai, Kentaro Hara, Daniel Bratell, Nico Weber, Dirk Pranke
2021年9月8日(水) 18:30 Rahul Patel <patelrah...@gmail.com>:
Thanks for updating Yuki, i see after using the  use_v8_context_snapshot = false flag,  the v8_context_snapshot.bin file not getting generated in the out/Release.

1. But when I am running the content_shell without this file getting the following error?

#

# Fatal error in , line 0

# Failed to deserialize the V8 snapshot blob. This can mean that the snapshot blob file is corrupted or missing.


Have you built `content_shell` with the flag set to false?

On my env (GNU/Linux), content_shell works fine without v8_context_snapshot.bin.

 


Also not seeing any reduction in the size of my .so file, that means disabling v8_context_snapshot will help not depending on v8_context_snapshot.bin file any more, and this is the only win?


.so doesn't contain v8_context_snapshot.bin, so it's no wonder.  The shipping size should be different between with and without the snapshot.  We've not enabled the flag on Android because of the APK size, IIUC.

Yuki Shiino

unread,
Sep 8, 2021, 6:09:59 AM9/8/21
to Rahul Patel, Yuki Shiino, blink-dev, Hitoshi Yoshida, Ojan Vafai, Kentaro Hara, Daniel Bratell, Nico Weber, Dirk Pranke
I've tried repro on GNU/Linux, and content_shell runs fine without the snapshot file.


2021年9月8日(水) 19:03 Rahul Patel <patelrah...@gmail.com>:
Yes, I am building it with that flag, i am able to build the code but it's failing to run.

Yuki Shiino

unread,
Sep 8, 2021, 6:35:57 AM9/8/21
to Rahul Patel, Yuki Shiino, blink-dev, Hitoshi Yoshida, Ojan Vafai, Kentaro Hara, Daniel Bratell, Nico Weber, Dirk Pranke
My understanding is that the snapshot file is bundled in the shipping package but outside of the executable binary.  IIUC, content_shell binary file doesn't contain the snapshot image.


2021年9月8日(水) 19:33 Rahul Patel <patelrah...@gmail.com>:
did you see any size reduction in the content_shell binary? after disabling it.

Yuki Shiino

unread,
Sep 8, 2021, 6:39:25 AM9/8/21
to Rahul Patel, Yuki Shiino, blink-dev, Hitoshi Yoshida, Ojan Vafai, Kentaro Hara, Daniel Bratell, Nico Weber, Dirk Pranke
That's my understanding.


2021年9月8日(水) 19:37 Rahul Patel <patelrah...@gmail.com>:
ok so after disabling the  snapshot image we no longer depending on 

v8_context_snapshot.bin ? Is that correct?

Reply all
Reply to author
Forward
0 new messages