How does the Dart VM work?

[Benjamin Summerton (@def-pri-pub)]

I posted this question here on the GitHub repo for the SDK a while back, but they redirected me here to ask it.

So, how exactly does the Dart VM work (under the hood)?  I've read a few of the documents on isolates and whatnot, but I'm wondering about things like how does my code get JIT'd at runtime (for platforms like x86 or ARM), possible performance hangups, etc.  So maybe I should ask, "Hoes does my 'Hello World' program go from source code to stdout in the terminal?"   How does it run through the Dart VM?

[Filipe Morgado]

Hey :)

I have nothing to add, except more doubts.

I've been using the VM without knowing exactly what's going on (lack of time/incentive to investigate).

Sorry to hijack the thread, but there has been a lot of stories.

Implemented in Dart, We have dart2js, DevCompiler and Kernel. It seems Kernel is a foundation to target any external environment, specially JS, still being working on.

DevCompiler, base on Kernel?, may supplant dart2js.

Then we have the regular VM and AoT compilation implemented in C++. Are those always bundled together?

When I run the VM, it seems to JITs my code directly to unoptimized machine code, then it optimizes code for hot loops. It runs well.

If my server-side VM bundles with AoT compilation, can can I take advantage of it, considering I'm (currently) on a x64?

Is all the work being invested in Kernel and AoT ever gonna produce performance results in a regular VM?

Thank you :)

[Joao Pedrosa]

Hi,

On Fri, Sep 30, 2016 at 4:14 PM, Benjamin Summerton (@def-pri-pub) <def.p...@gmail.com> wrote:

I posted this question here on the GitHub repo for the SDK a while back, but they redirected me here to ask it.

So, how exactly does the Dart VM work (under the hood)?  I've read a few of the documents on isolates and whatnot, but I'm wondering about things like how does my code get JIT'd at runtime (for platforms like x86 or ARM), possible performance hangups, etc.  So maybe I should ask, "Hoes does my 'Hello World' program go from source code to stdout in the terminal?"   How does it run through the Dart VM?

The Dart VM borrowed many ideas from the JavaScript VM used by Chrome called V8. Recall that the Dart VM

was supposed to be included in the browser too, so it had to support features that were friendly to the browser

and that is why it ended up sharing some of the same concerns of the JS VM.

For example as you mentioned the isolates is an idea from the browser. At the beginning, they wanted to

support lightweight isolates in case Dart needed even more concurrency. But once async got in, they seemed

to settle on heavier isolates with the idea of managing at least 1 isolate per CPU thread. The isolates used

more memory as a result, so if say we wanted thousands of them, we could run into memory issues instead.

Once the Dart developers took up async, they started adding support for Zones. Where each isolate was also

a Zone that handled the async transparently. One of the ideas they introduced later with Zone was that they

supported a capturing of the async exception at the last minute to help developers handle it and potentially

survive the exception. With async screwing up the stacktraces, they had other projects that helped to build

a more developer friendly stacktrace to help with debugging, but I think it used a separate package for the

frontend.

In the JS VM, they used as a first step the compilation to unoptimized native code. I think the Dart VM did

the same. For example, the Dart VM has only recently started adopting some more interpretation to help

Dart in devices that disallow JIT or have less memory for the JIT'ed code. So that the Dart VM has started

to add more layers to its execution.

One of the major differences between the JS and Dart optimization strategies revolves around the static

methods. In Dart, classes and objects don't change at runtime, so once compiled the calling the methods

is close to the optimized native speed. This trade-off allows Dart to be quicker with less work for the runtime.

The Dart VM seems to be register based. Dart has some optimizations for each CPU that it targets,

with some layer of assembly code to make it faster I think. Otherwise, a lot of the higher level APIs are either

written in C++ or in Dart itself.

Since Dart code is compiled to native code right away making it fast enough out of the box, further optimizations

would come from being able to remove dead code, inlining method calls or moving code out of the hot loops

if possible (hoisting.) And so on? :-) Those extra steps can be done by the JIT after it notices that code

has become hot after say being called hundreds of times. Say call a function 100 times and the JIT may

try to optimize it further. If the JIT decided that it messed it up, it could try to deoptimize it by running the

old code that it had just compiled to native code when it loaded up the source code. Ah! This reminds me

though. In JS they seem to keep the JS source code around in case they need to visit it again after an 

optimization or deoptimization call. Dart may be similar, it is possible that the VM keeps the Dart's AST

around in case it needs it later. One of the differences between a bytecode and a source code VM is that

the source code VM may be visiting the code again and again since it provides more information for the

compilers.

That brings us to where Dart has been heading, though. The Dart VM is changing. Dart has started to adopt

ahead-of-time (AoT) compilation as another deployment strategy helpful for the mobile platforms. Also, Dart

still needs the interpreter just in case, perhaps. There are developers working on something in Dart called

the Kernel. The Kernel would be a reduced version of Dart that would be a better target for compilation.

So, how would a Hello World type of application go?

1. $ dart hello_world.dart

2. Dart VM parses the AST and compiles it to native code, wrapping it inside the isolate and Zone.

3. Dart VM runs the code's main method. Sees the print("Hello World!") command which would

be in native code at this point. This would be a call to the C++ print API with the static string converted

to a C++ string by now. Runs it and exits the main method and the program.

When Dart code interfaces with C++ code, it is possible that some transformation may be necessary.

The compiler can spare some of it when it does its job. That's one of the differences between an

interpreter and a compiler. But JS by comparison is dynamic, and in the JS's case it is possible that

some code could have changed the print API (console.log) so they have to guard it against changes

when they are running it, whereas in Dart changing the APIs at runtime is largely forbidden. So that's

why Dart lends itself to compiling a bit better than JS does.

How does the JIT work? I think that on function calls (native code) they have a counter variable for each

function. If the function gets called say 100 times, they could run the JIT on it. But before they do, they

would need to take the time it took to run the function and store it for later, and when they run its

JIT version they can compare it to decide whether they need to deoptimize it and try again.

The Dart VM was supposed to be much simpler than the JS one. Since some of the core Dart VM

developers also worked on the JS one and learned from it. Unfortunately the Dart VM didn't make

it into the browser though. And on mobile it seems that concerns like battery life, memory usage

and security create a hassle for VMs.

Cheers,

Joao

 

[Benjamin Summerton (@def-pri-pub)]

Thanks!  That's pretty interesting.  Is there a Dart VM developer we could ping to add some more to the discussion?

[Joao Pedrosa]

Hi,

On Sat, Oct 1, 2016 at 9:48 PM, Benjamin Summerton (@def-pri-pub) <def.p...@gmail.com> wrote:

Thanks!  That's pretty interesting.  Is there a Dart VM developer we could ping to add some more to the discussion?

They hang around here. I'm not aware of any articles or videos talking about the Dart VM in detail, though.

You could try to compare what they have done in the Dart VM by studying up the JS VM which seems to

be talked about more.

On the JS VM I have watched some recent videos on youtube:

https://www.youtube.com/user/blinkontalks/videos

Specifically this one about the new interpreter of JS called Ignition:

https://youtu.be/r5OWCtuKiAk?t=193

Like I said before, the Dart VM and the JS VM share many properties and concerns. They have started

diverging more as of late once the JS VM introduced a new engine called TurboFan which is said to

deal with a Sea of Nodes. But TurboFan was more targeted at high performance math and will have to

catch up to the "traditional" engine used by both JS and Dart that does a better job at "normal" code. If

you watch the video about Ignition, you will know that the challenge they have is kind of interesting.

The Dart VM had maybe around 5 core designers. A couple of them may be left maintaining it now. They

may be helped by some new developers though. Since the interest in the Dart VM died down a little once

they could not have it on the browser, some of them may have moved on a little bit. They tried their hands

at other VMs instead like Dartino for IoT which didn't pan out.

As Anders Hejlsberg from Microsoft alluded to in a recent video when talking about modern day compilers,

some of these compilers are not found in the books yet. They are being invented by these top technological

companies. When Chrome innovated the JS VM, all the other browser companies took notice and started

borrowing ideas so that they could improve their own engines too. These days, Firefox and Edge (IE) have

caught up. That may partly explain why these developers are not in the teaching business.

If you want, you can try to talk to one of the JS and Dart VM developers over on Twitter:

https://twitter.com/mraleph

The JS TurboFan developer is this one:

https://twitter.com/bmeurer

Beware though that the VM has both the lower level tidbits and the higher level stuff to make it work. A single

developer may not cover all of it.

I share your curiosity though. On the NodeJS project, they have debugging tools that help to explain how it

all works. The Dart VM may have some debug APIs too. You could try to find some of those. Be on the

lookout for material explaining how the JS VM JIT works. It should be similar.

Here's a curiosity. The Dartino VM used bytecodes: https://github.com/dartino/sdk

But the Dartino VM was still interpreted. So it was relatively slow. But Dart kind of needed an Interpreter.

The Ignition VM for JavaScript is also based on bytecodes and interpreted. JavaScript also needs an

interpreter for devices that don't allow for JIT or that have restrictions. An interpreter supposedly helps with

the faster startup too. With the Ignition VM, they will use the TurboFan compiler to generate optimized

code. Maybe they wanted to have an extra layer of optimization on top of Dartino too, but they didn't get

to that point unfortunately. My feeling is that they are on that treadmill of we have 5 standards, we need to

fix them, we create a new standard and we now have 6 standards instead. :-)

Cheers,

Joao

[Daniel Morilha]

This question comes once in a while and unfortunately I have never found a good high level document explaining the vm internals.

Dart is more than its VM. It is a programming language project led by google.

As pointed out, the original vm was originally designed to replace javascript on the browser, the language however is purpose free. Dartino implements a very different vm yet relying on the same language.

The dart vm which runs on the server and most recently has been adapted to run on mobile consists of a C++ runtime (memory manager, scheduler, profiler, snapshots, libraries...) and the compiler part (parser, compiler, optimizer, assembler). The core of the VM[1] can be easily embedded with only limited dependencies into their host OS. When running on the server (aka standalone[2]) dart exposes additional libraries to deal with IO. There is a considerable amount of dart written libraries as well[3]. Libraries written in dart use the same infrastructure as the code that the user writes.

the DartVM implements AOT, JIT and a Dart Byte Code (DBC) interpreter for a mix of architectures. The language was designed to rely on JIT techniques so performance for AOT may not compare. However keep in mind AOT part of the project is new and work has been done to speed it up[4]. In fact the whole introduction for AOT started when the project started pursuing the mobile platform rather than web.

The compiler infrastructure within the VM is robust and massive. IMO it is what makes reading the source code complicated. If you are not into compilers, it is very difficult to keep up with all the concepts (intrinsics, arity, canonicalization, class hierarchy, different call conventions). Nevertheless it is a very well designed project yet pragmatic given all it does.

The hello world program, gets parsed and compiled to the machine's architecture. Your console calls tie with the libraries for your operating system. In the end it is just the same kernel call you would see with your C code. There are lots of abstractions and layers for that to happen. The key for performance is to remove the abstraction as much as possible for heavily executed code. Once you have machine code the C++ runtime jumps into it and your compiled code takes over the CPU, for certain operations it goes back to the C++ runtime or even to something external to the vm.

There is a short video which helps to understand the single compiler[5]. Bear in mind that at that time there was a lot of comparisons with javascript, which IMO should not make sense at this point given their different goals.

HTH

[1]: https://github.com/dart-lang/sdk/tree/master/runtime/vm

[2]: https://github.com/dart-lang/sdk/tree/master/runtime/bin

[3]: https://github.com/dart-lang/sdk/tree/master/sdk/lib

[4]: https://codereview.chromium.org/2379733002/#msg11

[5]: https://youtu.be/bNESI-d-tBQ?t=14m46s

--
For other discussions, see https://groups.google.com/a/dartlang.org/
 
For HOWTO questions, visit http://stackoverflow.com/tags/dart
 
To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups "Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.

--

Daniel Morilha (dmor...@gmail.com)

[Vyacheslav Egorov]

If you are curious about how Dart VM compiles your Dart code into native code then you can start by watching my talk "Building an Optimizing (JIT) Compiler for Dart" from Strange Loop 2013:

Video: https://www.infoq.com/presentations/dart-compiler

Slides: http://mrale.ph/s3/StrangeLoop2013.pdf

It outlines most important ideas that Dart VM uses to JIT compile Dart code. (spoiler alert: most of these ideas are coming from Self VM from 90s).

AOT is a completely different story. There will be a talk about Dart VM AOT mode on the Dart Summit this year. Make sure you see it - if you are interested in the challenges that AOT compilation is facing. 

Cheers, 

// Vyacheslav Egorov

[kc]

I made a similar point here:

https://groups.google.com/a/dartlang.org/d/msg/misc/hVUGMpf9x9w/yvlv4w-3AgAJ

The Dart VM is way too much of a black box. 
Some info on how the the Dart VM is engineered - especially to interact with the Flutter engine (with a nice surface syntax) - would help make the sale.

K.