Avoiding spare renderer based on memory pressure

[Łukasz Anforowicz]

Hello,

We are observing (https://crbug.com/852905#c3) that the spare renderer hurts performance at 99th percentile.  We hypothesize that this is because at 99th percentile the system might be thrashing.  We'd like to try verifying this hypothesis by experimenting with suppressing spawning of the spare renderer when under memory pressure.

Does avoiding the spare renderer based on memory pressure seem like a sensible approach?  Is is okay to detect memory pressure based on MemoryPressureMonitor::GetCurrentPressureLevel() returning MEMORY_PRESSURE_LEVEL_MODERATE or higher?

Thanks,

Lukasz

PS. I guess memory pressure might not be the only scenario where a spare renderer might hurt performance.  There might be other resources (CPU, disk IO) that a spare renderer pushes into "over-subscribed" area.  Is there a way to take these scenarios into account as well (e.g. is there a generic GetSystemLoad API somewhere in Chromium)?

[Erik Chen]

> Does avoiding the spare renderer based on memory pressure seem like a sensible approach?

This seems like a good starting point. 

Let's try it out, look at the results, and iterate.  :)

[Chris Hamilton]

The memory pressure seems like a reasonable signal to use, although be forewarned that it's known broken right now (doesn't work at all on Linux, hit and miss on Windows, terrible and way too late on Mac, quite reasonable on ChromeOS). We have ongoing plans to try to fix this, but it's been on the back-burner for a while. That being, said it's likely not a bad place to start.

We don't yet track other types of "resource pressure", but that might be worth exploring.

[Łukasz Anforowicz]

Thanks!  Do I need to worry about tests (unit tests and browser tests) staying deterministic (i.e. not flaky) even if they tickle product code that calls MemoryPressureMonitor::GetCurrentPressureLevel when deciding whether to spawn a spare renderer?  In other words - will my tests become flaky on busy test bots?  I've tried to skim the code to see if the memory pressure listener is fakeable, but I didn't see any obvious leads...  OTOH, I hope that somebody else already did the work of insulating the tests (i.e. the memory pressure listener used inside browser/unit tests) from the state of the bot.

[Erik Chen]

> In other words - will my tests become flaky on busy test bots?

I don't know! I'd recommend running the CQ with a test-version of your CL that always returns CRITICAL memory pressure, and see if that causes test failures.

[Kentaro Hara]

+1 to the proposal % the current memory pressure is broken in many ways and not reliable as Chris pointed out... :)

If the existing memory pressure signal does not work, you might want to think about using (Total.PMF / system RAM size) >= X% as a signal. This is not a good signal either but looks more reliable at least on Android. (e.g., the OOM intervention is using that.)

--
You received this message because you are subscribed to the Google Groups "memory-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+...@chromium.org.
To post to this group, send email to memor...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/memory-dev/CAEYHnr2t3rbARG9zoJjK%2BrGPniwN2Le9_-yTZ4%2BJ7ds%2B_bM45g%40mail.gmail.com.

--

Kentaro Hara, Tokyo, Japan

[Erik Chen]

I would like to see the implementation use memory pressure just to be consistent with all the other places that already use system memory pressure.

> (Total.PMF / system RAM size) >= X% as a signal.

I have a design doc circulating to send memory pressure signals using exactly this heuristic. 

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+unsubscribe@chromium.org.

To post to this group, send email to memor...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/memory-dev/CAEYHnr2t3rbARG9zoJjK%2BrGPniwN2Le9_-yTZ4%2BJ7ds%2B_bM45g%40mail.gmail.com.

[Chris Hamilton]

Yes, I'd prefer we continue using the system as is (at least it's a single place where to register these things), and fix it in place, rather than having multiple folks roll their own heuristics in various places :)

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+...@chromium.org.

To post to this group, send email to memor...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/memory-dev/CAEYHnr2t3rbARG9zoJjK%2BrGPniwN2Le9_-yTZ4%2BJ7ds%2B_bM45g%40mail.gmail.com.

[Kentaro Hara]

Agreed :)

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/memory-dev/CAA2pCLdNOKU4tHfJbqBECdzc4o0Kjxvea4inAG9aYA2_2QOrSg%40mail.gmail.com.

[Gabriel Charette]

Are we still seeing this? I think https://bugs.chromium.org/p/chromium/issues/detail?id=560446#c99 might have been at fault +Nasko Oskov and I noticed that having a spare process was one of three factors required to enable the bug.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/memory-dev/CABg10jx2WhYz7WyN-eT-NKfMQhRSkYe4XnWh5eb912VcQ3kh1w%40mail.gmail.com.

[Łukasz Anforowicz]

Negative impact of the spare renderer process on higher percentiles seems to have gone away in M69 - see https://uma.googleplex.com/p/chrome/variations?sid=5b3cf053f71996516f21dfea506d9809 (I've also updated https://crbug.com/852905).  Woo-hoo!

Agreed :)

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+unsubscribe@chromium.org.

To post to this group, send email to memor...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/memory-dev/CAEYHnr2t3rbARG9zoJjK%2BrGPniwN2Le9_-yTZ4%2BJ7ds%2B_bM45g%40mail.gmail.com.

--

Kentaro Hara, Tokyo, Japan

--
You received this message because you are subscribed to the Google Groups "memory-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+unsubscribe@chromium.org.

To post to this group, send email to memor...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/memory-dev/CAA2pCLdNOKU4tHfJbqBECdzc4o0Kjxvea4inAG9aYA2_2QOrSg%40mail.gmail.com.

--

Kentaro Hara, Tokyo, Japan

--
You received this message because you are subscribed to the Google Groups "memory-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+unsubscribe@chromium.org.

To post to this group, send email to memor...@chromium.org.

[Gabriel Charette]

\o/ :)!

Agreed :)

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+...@chromium.org.

To post to this group, send email to memor...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/memory-dev/CAEYHnr2t3rbARG9zoJjK%2BrGPniwN2Le9_-yTZ4%2BJ7ds%2B_bM45g%40mail.gmail.com.

--

Kentaro Hara, Tokyo, Japan

--
You received this message because you are subscribed to the Google Groups "memory-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+...@chromium.org.

To post to this group, send email to memor...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/memory-dev/CAA2pCLdNOKU4tHfJbqBECdzc4o0Kjxvea4inAG9aYA2_2QOrSg%40mail.gmail.com.

--

Kentaro Hara, Tokyo, Japan

--
You received this message because you are subscribed to the Google Groups "memory-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+...@chromium.org.

To post to this group, send email to memor...@chromium.org.

[Chris Hamilton]

Nice!

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/memory-dev/CAJTZ7LLxGheJSATzkUSGXG8dpVhHFLPRbJfaYCgzP%2BOdO5nMCw%40mail.gmail.com.