Dart VM performance

[George Moschovitis]

A look at the charts on http://dartlang.org/performance paints a worrisome picture: it seems like DartVM performance has hit a wall.

No material performance improvements in the last 12 months (in fact no performance gains whatsoever in 2015).

Can we expect something more?

-g.

[Joao Pedrosa]

Hi,

I happen to be watching some of the Dart conference videos again, and Lars Bak said that it is important for people to send them benchmarks showing some real-world performance problems that they are having.

Regarding Dart performance, one of their priorities seems to be to balance the code size or the generated code so that the startup or the memory footprint can be improved.

Fletch can load its generated snapshot faster than the Dart VM can its snapshot. I'm not sure why that is. Perhaps given all of the features that the Dart VM has, it ended up making the startup noticeably slower. But the Dart VM is full featured compared to most VMs including Fletch. So it's not entirely fair comparing their performance at this point.

They did let a hint out that they could experiment with pre-compiling some of the program's code so that it was faster, perhaps only a feature for the iOS at the beginning since that was the context of the conversation.

But if we compare the Dart VM to the Java VM, one of the concerns that they tried to tackle since the beginning was startup time, just because the Java VM always had a problem with that, but also because mobile requires that your application always be starting up.

Perhaps they think that the Dart VM is already plenty able to drive most UI needs that people have. If their UI needs is like loading a native UI, presenting and collecting some data with basic input fields, etc. Before they dropped support for the Dart VM on Chrome, some users were also willing to code their games with the Dart VM, implying that the Dart VM was also relatively good for game development.

Since mobile is in its infancy despite its huge popularity, anything could happen concerning mobile development tools. Something that could happen to Dart is that they would start demanding "implementation types" as a subset that would get compiled ahead-of-time to some mobile platforms. An alternative view is that people would try to bring the ease of development (relative) of the web to mobile so that the clients remained "thin" and most of everything remained on the servers. And a "thin" client could be just interpreted like JavaScript is on the web.

iOS forbidding "JIT" also throws a wrench at just using the Dart VM for everything.

They cannot promise much at this point. But here is what could happen... Fletch could produce a standard bytecode set that they would try to run through the LLVM compilers which would compile Dart ahead-of-time. It seems that many high level languages have been dreaming of something like that.

Cheers,

Joao

--
For other discussions, see https://groups.google.com/a/dartlang.org/
 
For HOWTO questions, visit http://stackoverflow.com/tags/dart
 
To file a bug report or feature request, go to http://www.dartbug.com/new

To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.

[George Moschovitis]

Perhaps they think that the Dart VM is already plenty able to drive most UI needs that people have. 

But DartVM is now server-only (or embedded-only, cli-only, whatever). Extra performance can always help.

Anyway, the question still stands. Can we expect any improvements on those benchmarks in 2015?

-g.

PS: I am aware about improvements like the new regexp engine, passing connections to isolates, etc. I am curious about these benchmarks 

(supposedly they are representative of common use cases).

[Don Olmstead]

I don't really see much on the V8 side either.

--

[George Moschovitis]

I don't really see much on the V8 side either.

AFAICS, V8 is closing the gap in two benchmarks. 

But really, V8 is not the focus of this forum. Unless, you imply that 'DartVM is still faster than V8, mission accomplished'.

-g.

[Don Olmstead]

DeltaBlue has some movement in the graph zoomed all the way out. Fluid Motion has been a plateau since fall of 2014, with v8 seeing a jump then a plateau early this year. Havlak has been pretty steady since around that time as well while V8 has been on a downswing there. Richards plateaued end of 2014 with v8 hitting a plateau around middle of 2014. Tracer plateaued fall of 2014 with V8 hitting a similar plateau.

I'm not sure what ones you consider closing a gap. Not trying to be rude I would probably be a bit alarmed if SoundScript was right up there with Dart but I just don't see a cause for alarm as V8 follows a similar trend. Plateau then some breakthrough then plateau.

[Joao Pedrosa]

Hi,

But DartVM is now server-only (or embedded-only, cli-only, whatever). Extra performance can always help.

Anyway, the question still stands. Can we expect any improvements on those benchmarks in 2015?

They say that in many regards the Dart VM is already about as fast as the Java VM which has had over a decade of improvements. And the Dart VM is faster than the JavaScript VMs which also had at least 7 years of major improvements. So if we don't see major improvements it could be that they are a little out of ideas and resources to improve it much further. :-)

I think that the Firefox JavaScript VM has been borrowing ideas that were first used on the V8 VM. So now the Firefox JS VM is about as fast in many regards. The research that they have been pouring into the JS VMs could give us ideas for the Dart VM. But the problem is that say, they are trying to make the JS VMs faster for games, which would not necessarily apply to enterprise software development. I'm thinking for example about huge loops that never stop executing, huge switch statements etc. Enterprise software development don't generally have those kinds of code. One code-word they use for some optimization techniques regarding those is OSR = On-Stack Replacement. So even if the long running method is called just once, it would still get optimized. It is comparably much easier to optimize methods that are called multiple times and on the next call you can send the execution into the optimized path.

For enterprise software development Dart may already be super optimized for. They could tweak it, improve it... They could say optimize isolates, Async programming, stuff like that, that would apply more on a library level.

Cheers,

Joao

[George Moschovitis]

They say that in many regards the Dart VM is already about as fast as the Java VM which has had over a decade of improvements. And the Dart VM is faster than the JavaScript VMs which also had at least 7 years of major improvements. So if we don't see major improvements it could be that they are a little out of ideas and resources to improve it much further. :-)

Well, that might be an answer then. I am not *expecting* any further improvements. I am just *curious* if we will see any major improvements.

Just an informal estimation from the VM team would do (e.g. "we aim for 5% (or 10%, 50%, or 100%) improvement within the next 24 months").

-g.

[Mark H]

I'm also concerned, now that the Dart VM in Chrome is no more, that the performance of dart2js is way off that of the VM in all of the benchmarks. It also, doesn't seem to be closing the gap and is somewhat behind hand-crafted JS. Does this mean that performance in the browser with Dart is no longer a likely prospect?

Numerical performance is pretty important in my app and I get nowhere near the speed when compiled to JS as when running in the VM.

[Srdjan Mitrovic]

The VM team is quite satisfied with the current performance of the tracked benchmarks, improving their performance is not our focus any longer.

The Dart VM has been designed as a client VM (fast startup, quick compilations, small footprint). Very happy that it performs so well in a server setting.

We are still working hard on improving the performance and usability of the VM in all dimensions (startup, footprint, pauses, code quality, ..). That work may benefit at some point the tracked benchmarks as well. 

The VM team has still the same resources and commitment as before, and we have tons of ideas :-). Once a VM reaches certain level, the visible progress becomes slower.

Dart VM should be an excellent choice for writing games as well, let me know if and why you disagree.

Cheers,

- Srdjan

[Mark H]

I agree that the Dart VM should be a good choice for writing games (or computationally intensive apps). It has always seems pretty fast to me. The problem is that, now it's not going to be in the browser, I don't really see what platforms it can be used on?

[Filipe Morgado]

On Wednesday, 13 May 2015 10:14:06 UTC+1, Mark H wrote:

I don't really see what platforms it can be used on?

Maybe Sky?

I really hope they make it cross-platform. I'm tired of vendor-lock BS. 

[George Moschovitis]

Thank you for the detailed and straightforward answer. 

Any plans for server-specific optimizations now that the DartVM is no longer applicable for client-side (i.e. browsers) use?

-g.

[Benjamin Strauß]

You can always embed the vm into your native c/c++ application, write bindings for your engine and program your whole game logic in dart. Similar to what Lua is doing.

[Don Olmstead]

Yea I did an example of that awhile back probably really outdated though.

https://youtu.be/M1qJbcZKud8

https://github.com/donny-dont/DartEmbeddingDemo

[Mark H]

I guess so but there are many different ways this can be achieved in a native app. I was really looking for "somewhere near native" like performance in the browser with Dart, which now does not look likely.

[Cogman]

@Benjamin  Wow, I never thought about embedding dart in a game engine, but it makes a lot of sense.  Dart is really fast (something like 30x faster than lua, I'm not sure about lua-jit), dart is really expressive, and dart is quite feature rich.

[Alex Tatumizer]

Dart VM performance advantage over javascript narrows (if Microsoft's data is to be believed):
http://blogs.windows.com/msedgedev/2015/05/20/delivering-fast-javascript-performance-in-microsoft-edge/

Rough calculations show that Edge is ~13% faster than Chrome on Octane tests; on the other hand, dart is faster than V8 (according to https://www.dartlang.org/performance) by ~37%. Which still gives (roughly) 24% advantage to dart VM vs Edge, but javascript is clearly catching up. [ from another perspective: java is currently faster than dart by greater margin than dart vs (Edge) javascript ]

Another observation from same data is that Firefox is getting very close to Chrome in terms of performance: difference is just about 8% on Octane.

FWIW.




 

[Thomas Schranz]

The microsoft article is interesting, thanks for sharing.

Performance improvements of Edge over IE11 also mean that existing dart applications deployed using dart2js now run faster :)