It can be seen in devtools Performance profiler after clicking a "Compile script" block in the flamechart:
It was always disabled for extensions due to security concerns, but apparently it was only applicable to content scripts (executed in the web page process) and not to chrome-extension:// pages (executed in their own separate process).
Huge scripts won't be recompiled now each time a chrome-extension:// page loads, instead the result is loaded from cache.
I wonder though if the cache contains "hot code" optimizations that are getting discovered at a later time? Those can improve performance substantially on occasion.
I also saw one script wasn't cached even though it's arguably just like the cached ones, so maybe the feature is still incomplete or bugged.
Sometimes a very long "devtools profiling overhead" block may be shown in the flamechart underneath, skewing the results completely. Anyway, to measure the actual improvements you need to load the page normally (without the profiler) with and without the cache. I don't know how to clear this cache just for one page, but you can change the script(s) trivially to force re-compilation e.g. add a random number at the beginning or end of the code like ;1.