Re: [dart-misc] The Dart compiler is so bad that it is 150 times slower than Javascript and is not computationally efficient in comparison to other languages

[Daniel Varga]

I am subscribed to this mailing list in the hopes that I get very important deprecation warnings, important announcements, breaking changes. Eg. mirrors

I noticed a trend recently that the tone of some of the messages are very aggressive and extremely provocative.

One of the things I enjoy about the Dart community has always been the approachability of its members and the maintainers. There seems to be a certain "we are in it together" sense that I appreciate very much.

I strongly believe that the manner of communication matters and that we can achieve more if we pay attention to the fact that our public messages are read by other humans.

Thank you for reading!

Daniel

On Thu, Jun 10, 2021, 08:09 Andrew Mezoni <andrew...@gmail.com> wrote:

In the 21st century, the Dart compiler is so primitive that it cannot properly compile elementary code for which other compilers generate efficient code?

I am ashamed that I use Dart for computing.

Dart is completely unsuitable for this purpose.

Dart Pad results:

Time passed: 0.000, Test 'good': 2.399 ms

Time passed: 0.003, Test 'bad': 3.8 ms

Time passed: 0.006, Test 'good': 1.4 ms

Time passed: 0.008, Test 'bad': 2 ms

Time passed: 0.010, Test 'good': 1.5 ms

Time passed: 0.011, Test 'bad': 1.801 ms

Time passed: 0.013, Test 'good': 0.901 ms

Time passed: 0.014, Test 'bad': 0.901 ms

Time passed: 0.015, Test 'good': 0.899 ms

Time passed: 0.016, Test 'bad': 0.899 ms

Time passed: 0.017, Test 'good': 1 ms

Time passed: 0.018, Test 'bad': 0.9 ms

Dart VM results:

Time passed: 0.000, Test 'good': 1.731 ms

Time passed: 0.004, Test 'bad': 159.304 ms

Time passed: 0.163, Test 'good': 2.185 ms

Time passed: 0.166, Test 'bad': 156.398 ms

Time passed: 0.322, Test 'good': 0.614 ms

Time passed: 0.323, Test 'bad': 156.95 ms

Time passed: 0.480, Test 'good': 0.696 ms

Time passed: 0.481, Test 'bad': 157.634 ms

Time passed: 0.638, Test 'good': 0.687 ms

Time passed: 0.639, Test 'bad': 157.56 ms

Time passed: 0.797, Test 'good': 0.728 ms

Time passed: 0.798, Test 'bad': 157.279 ms

Source code:

void main() {

  final count = 1000000;

  final names = ['good', 'bad'];

  final tests = [_test1, _test2];

  final sw = Stopwatch();

  var repeat = 150;

  sw.start();

  while (repeat-- > 0) {

    for (var i = 0; i < tests.length; i++) {

      final name = names[i];

      final test = tests[i];

      final seconds = (sw.elapsedMilliseconds / 1000).toStringAsFixed(3);

      final title = 'Time passed: $seconds, Test \'$name\'';

      _measure(title, 1, () => test(count));

    }

  }

}

void _measure(String name, int count, Function() f) {

  final sw = Stopwatch();

  sw.start();

  for (var i = 0; i < count; i++) {

    f();

  }

  sw.stop();

  final time = sw.elapsedMicroseconds / 1000;

  print('$name: $time ms');

}

int add(int x, int y) => x + y;

int add2(int Function(int x, int y) f, int x, int y) => f(x, y);

void _test1(int count) {

  for (var i = 0; i < count; i++) {

    add(1, 2);

    add(1, 2);

    add(1, 2);

    add(1, 2);

    add(1, 2);

    add(1, 2);

    add(1, 2);

    add(1, 2);

    add(1, 2);

    add(1, 2);

  }

}

void _test2(int count) {

  for (var i = 0; i < count; i++) {

    add2(add, 1, 2);

    add2(add, 1, 2);

    add2(add, 1, 2);

    add2(add, 1, 2);

    add2(add, 1, 2);

    add2(add, 1, 2);

    add2(add, 1, 2);

    add2(add, 1, 2);

    add2(add, 1, 2);

    add2(add, 1, 2);

  }

}

Rust programmers use this kind of computation all over the place. This is in case you want to say that this is not actual.

But Rust is not Javascript, but they are smart people too (Rust developers).

I am very surprised and cannot understand what the Dart developers are doing at work, while other developers are developing normal compilers for normal calculations.

Sorry to be rude, but how is this possible in 2021?

--
For more ways to connect visit https://dart.dev/community
---
You received this message because you are subscribed to the Google Groups "Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.
To view this discussion on the web visit https://groups.google.com/a/dartlang.org/d/msgid/misc/5aefa980-54e3-4b39-8d0f-3361a2147a56n%40dartlang.org.

[Jacob Bang]

I have tried to run your code with both Dart VM and compiled to an exe
file with "dart compile exe" with:
Dart SDK version: 2.13.0 (stable) (Wed May 12 12:45:49 2021 +0200) on
"windows_x64"

And I cannot get the same result as you where "bad" has such a worse
performance:

>dart program.dart
Time passed: 0.000, Test 'good': 1.55 ms
Time passed: 0.003, Test 'bad': 1.819 ms
Time passed: 0.005, Test 'good': 1.006 ms
Time passed: 0.006, Test 'bad': 2.079 ms
Time passed: 0.008, Test 'good': 0.234 ms
Time passed: 0.009, Test 'bad': 0.468 ms
Time passed: 0.010, Test 'good': 0.233 ms
Time passed: 0.010, Test 'bad': 0.467 ms
Time passed: 0.011, Test 'good': 0.233 ms
Time passed: 0.011, Test 'bad': 0.467 ms
Time passed: 0.012, Test 'good': 0.235 ms
Time passed: 0.012, Test 'bad': 0.467 ms
Time passed: 0.013, Test 'good': 0.235 ms
Time passed: 0.013, Test 'bad': 0.466 ms
Time passed: 0.014, Test 'good': 0.235 ms
Time passed: 0.014, Test 'bad': 0.467 ms
Time passed: 0.015, Test 'good': 0.235 ms
Time passed: 0.015, Test 'bad': 0.466 ms
Time passed: 0.016, Test 'good': 0.235 ms
Time passed: 0.016, Test 'bad': 0.466 ms
Time passed: 0.017, Test 'good': 0.233 ms
Time passed: 0.017, Test 'bad': 0.483 ms
Time passed: 0.019, Test 'good': 0.236 ms
Time passed: 0.019, Test 'bad': 0.467 ms
Time passed: 0.020, Test 'good': 0.235 ms
Time passed: 0.021, Test 'bad': 0.469 ms
Time passed: 0.021, Test 'good': 0.233 ms
Time passed: 0.022, Test 'bad': 0.468 ms
Time passed: 0.022, Test 'good': 0.234 ms
Time passed: 0.023, Test 'bad': 0.468 ms
Time passed: 0.023, Test 'good': 0.234 ms
Time passed: 0.024, Test 'bad': 0.469 ms
Time passed: 0.024, Test 'good': 0.233 ms
Time passed: 0.024, Test 'bad': 0.468 ms
Time passed: 0.025, Test 'good': 0.233 ms
Time passed: 0.025, Test 'bad': 0.467 ms
Time passed: 0.026, Test 'good': 0.236 ms
Time passed: 0.026, Test 'bad': 0.468 ms
...
Time passed: 0.163, Test 'good': 0.233 ms
Time passed: 0.163, Test 'bad': 0.466 ms

>dart compile exe program.dart
>program.exe
Time passed: 0.000, Test 'good': 0.466 ms
Time passed: 0.000, Test 'bad': 0.466 ms
Time passed: 0.001, Test 'good': 0.494 ms
Time passed: 0.001, Test 'bad': 0.484 ms
Time passed: 0.002, Test 'good': 0.468 ms
Time passed: 0.003, Test 'bad': 0.468 ms
Time passed: 0.003, Test 'good': 0.466 ms
Time passed: 0.004, Test 'bad': 0.468 ms
Time passed: 0.005, Test 'good': 0.479 ms
Time passed: 0.005, Test 'bad': 0.466 ms
...
Time passed: 0.172, Test 'good': 0.477 ms
Time passed: 0.172, Test 'bad': 0.471 ms
Time passed: 0.173, Test 'good': 0.466 ms
Time passed: 0.173, Test 'bad': 0.47 ms
Time passed: 0.174, Test 'good': 0.466 ms
Time passed: 0.174, Test 'bad': 0.47 ms
Time passed: 0.175, Test 'good': 0.467 ms
Time passed: 0.175, Test 'bad': 0.471 ms
Time passed: 0.176, Test 'good': 0.466 ms
Time passed: 0.176, Test 'bad': 0.471 ms
Time passed: 0.177, Test 'good': 0.466 ms
Time passed: 0.177, Test 'bad': 0.47 ms
Time passed: 0.178, Test 'good': 0.466 ms
Time passed: 0.178, Test 'bad': 0.469 ms
Time passed: 0.179, Test 'good': 0.466 ms
Time passed: 0.180, Test 'bad': 0.483 ms

Can you add more details about what Dart version you are using and if
you are running your tests in some specific way?

--
Jacob Bang / julemand101

[Lasse R.H. Nielsen]

I restructured the benchmark to have a consistent run-time and be proof against dead-code elimination.

  https://dartpad.dev/a16b850405f249408ff8cb877adadd0a

There is a significant difference in the two calls. (Like 2000 ops/us vs 50 ops/us on the VM)

That difference, as already mentioned, appears to be down to the difference between inlining the direct, statically resolved call `add(1, 2)` and making it just be `3`,

and actually calling functions. Even if we always inline `add2` as well, there is still a call to a function value which isn't statically recognized and inlined.

I can get the same slow behavior for "good" by changing `add(1, 2)` to `(add)(1, 2)`. It changes a "static method invocation" to a "closurization"+"function value invocation". The closurization is a constant, but the function value invocation is still slow.

You're falling off an optimization cliff here. Some code is recognized and optimized better than the general case (a statically resolvable function invocation), and in this particular example that code is so abnormally trivial that it can be reduced to the constant `3`. Anything else is going to be orders of magnitude slower than a constant small integer.

What we can discuss is why the VM gets 50 ops/us for a function value invocation (which is completely monomorphic in this example) while dart2js gets ~1000 ops/us for the same code.

That seems to be a failure of JIT-compiling rather than an inherent problem, because the same code compiled with dart2native gets 2000 ops/us for both "good" and "bad".

TL;DR: The VM JIT fails you on this particular code. The VM can do better, and does better when AoT compiled to native.

/L

--

Lasse R.H. Nielsen - l...@google.com  

'Faith without judgement merely degrades the spirit divine'
Google Denmark ApS - Frederiksborggade 20B, 1 sal - 1360 København K - Denmark - CVR nr. 28 86 69 84

[Jacob Bang]

Just want to add that the JavaScript behavior is different when
comparing Firefox and Chrome:

DartPad on Firefox 89.0:
...
Time passed: 31.142, Test 'bad': 209 ms
Time passed: 31.351, Test 'good': 7 ms
Time passed: 31.358, Test 'bad': 208 ms
Time passed: 31.566, Test 'good': 7 ms
Time passed: 31.573, Test 'bad': 208 ms
Time passed: 31.781, Test 'good': 7 ms
Time passed: 31.788, Test 'bad': 208 ms
Time passed: 31.996, Test 'good': 7 ms
Time passed: 32.003, Test 'bad': 212 ms
Time passed: 32.215, Test 'good': 7 ms
Time passed: 32.222, Test 'bad': 208 ms

Chrome 91.0:
...
Time passed: 1.412, Test 'bad': 4.7 ms
Time passed: 1.416, Test 'good': 4.8 ms
Time passed: 1.421, Test 'bad': 4.7 ms
Time passed: 1.426, Test 'good': 4.601 ms
Time passed: 1.431, Test 'bad': 4.8 ms
Time passed: 1.436, Test 'good': 4.599 ms
Time passed: 1.440, Test 'bad': 4.7 ms
Time passed: 1.445, Test 'good': 4.7 ms
Time passed: 1.450, Test 'bad': 4.701 ms

So I would not conclude that the code in general "runs fine in JS".
But it runs fine in Chrome :)

Den tor. 10. jun. 2021 kl. 11.17 skrev 'Lasse R.H. Nielsen' via Dart
Misc <mi...@dartlang.org>:

> To view this discussion on the web visit https://groups.google.com/a/dartlang.org/d/msgid/misc/CA%2BeWuVAfsE323kwbvxn8yJ_i_EOW%3DBAzq-d6QOE1AS3mGV6jKw%40mail.gmail.com.

[Jacob Bang]

My browser test was with your latest posted code. For Dart VM and
compiled Dart i can reproduce your issue:

Dart VM:
Time passed: 11.353, Test 'good': 4.678 ms
Time passed: 11.358, Test 'bad': 248.803 ms
Time passed: 11.607, Test 'good': 4.669 ms
Time passed: 11.612, Test 'bad': 248.164 ms
Time passed: 11.861, Test 'good': 4.726 ms
Time passed: 11.866, Test 'bad': 247.966 ms
Time passed: 12.114, Test 'good': 4.7 ms
Time passed: 12.119, Test 'bad': 248.386 ms

Compiled exe:
Time passed: 2.610, Test 'good': 7.041 ms
Time passed: 2.618, Test 'bad': 364.376 ms
Time passed: 2.982, Test 'good': 7.017 ms
Time passed: 2.989, Test 'bad': 359.865 ms
Time passed: 3.349, Test 'good': 7.006 ms
Time passed: 3.357, Test 'bad': 372.814 ms
Time passed: 3.730, Test 'good': 7.035 ms
Time passed: 3.737, Test 'bad': 361.844 ms
Time passed: 4.099, Test 'good': 7.02 ms
Time passed: 4.106, Test 'bad': 357.714 ms

Den tor. 10. jun. 2021 kl. 11.30 skrev Andrew Mezoni <andrew...@gmail.com>:
>
> >> Just want to add that the JavaScript behavior is different when
> comparing Firefox and Chrome:
>

> Thank you!
> Has anyone ever said that Firefox is good JS.
> What about Dart results?
>
> чт, 10 июн. 2021 г. в 14:26, Jacob Bang <julem...@gmail.com>:

>> To view this discussion on the web visit https://groups.google.com/a/dartlang.org/d/msgid/misc/CADK_2AWkso247e%3DXvGhCNLk6%2BMCOvmkO0M9ZDhWmuC8gSh%2BQSQ%40mail.gmail.com.

>
> --
> For more ways to connect visit https://dart.dev/community
> ---
> You received this message because you are subscribed to the Google Groups "Dart Misc" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.

> To view this discussion on the web visit https://groups.google.com/a/dartlang.org/d/msgid/misc/CAAfHsy9EPNBZ%2B18iA%2B3G02a3%3DGZZfdocOizkbYHnMcWSihZ7OQ%40mail.gmail.com.

[Lasse R.H. Nielsen]

On Thu, Jun 10, 2021 at 11:17 AM Lasse R.H. Nielsen <l...@google.com> wrote:

I restructured the benchmark to have a consistent run-time and be proof against dead-code elimination.

  https://dartpad.dev/a16b850405f249408ff8cb877adadd0a

There is a significant difference in the two calls. (Like 2000 ops/us vs 50 ops/us on the VM)

Ouch, my bad. I forgot I was also passing `--observe` to the VM. Without that I see no significant difference in performance.

So, the Heisenberg JIT is fine, as long as you're not observing it.

/L

[Slava Egorov]

Try changing your benchmark in a way that passes a different closure to sum and measure on JavaScript. Suddenly you will observe things fall of the cliff there as well, e.g. https://gist.github.com/mraleph/3ee317f68f1215faa2c471142143a707 produces 

Time passed: 0.000, Test 'good': 2 ms
Time passed: 0.002, Test 'bad': 4.8 ms
Time passed: 0.007, Test 'whatever': 109.699 ms
Time passed: 0.117, Test 'good': 1.1 ms
Time passed: 0.118, Test 'bad': 1.9 ms
Time passed: 0.120, Test 'whatever': 100.5 ms
Time passed: 0.221, Test 'good': 1 ms
Time passed: 0.222, Test 'bad': 94.5 ms

Time passed: 0.316, Test 'whatever': 102.2 ms  

The reason for that is you no longer hit a nice monomorphic case which made it look fast before. 

Dangers of microbenchmarks. 

As for your second benchmark we currently can't optimize it to nothing because that requires inlining followed by load forwarding followed by more inlining and we only run inlining once in the compiler pipeline. 

So of course if you compare doing something to not doing anything - doing nothing is going to be much faster.

Realistically though you should look at the performance of your own parser end-to-end rather than look at some microbenchmarks you extract from it. Because microbenchmarks are going to mislead you.

 

On Thu, Jun 10, 2021 at 1:22 PM Andrew Mezoni <andrew...@gmail.com> wrote:

Sorry for the spam.

But I was surprised to learn that this approach is now quite popular in Rust (as well as in Rust itself).

Make it as simple as possible. However, this does not mean that many intensive computations will not be done behind the scenes.

Again, with the same approach of coding (do code it simple).

Hexadecimal color parser:

https://github.com/Geal/nom#example

Just a few lines of code.  Parser !

This parser code can be written on your knees  and it will runs  very fast.

I can (anyone can) implement an approach like this (style, call it whatever you want). It is very easy.

But not шт Dart. Dart is ungodly slow for that.

Unlike modern JS, Rust and other compilers . 

My hands are discouraged from such events.

четверг, 10 июня 2021 г. в 15:54:36 UTC+5, Andrew Mezoni:

This is obsolete code and not correct.

Don't refer to it, please.

Correct issue  as below.

void _test2(int count) {

  final sumIt = sum(add);

  for (var i = 0; i < count; i++) {

    sumIt(1, 2);    

  }

}

int Function(int, int) sum(int Function(int, int) f) {

  return (x, y) {

    return x + y + f(x, y);

  };

}

https://gist.github.com/mezoni/c0a0a651ae7fa36b9d038b0f21a0ba2a

The "addition" operation is just for example. In fact, the computations are more intensive.

But not in one place. Constantly calling code like this.

 final digits = many(digitParserFunc); // Run once

final d = digits(input); // Run many times

четверг, 10 июня 2021 г. в 15:40:09 UTC+5, Lasse Reichstein Holst Nielsen:

To view this discussion on the web visit https://groups.google.com/a/dartlang.org/d/msgid/misc/0255f968-2550-4634-8fff-506d1581ca8en%40dartlang.org.

[Martin Kustermann]

Regarding the huge performance drop Lasse mentioned when passing --observe: I've filed sdk/issues/46320

To view this discussion on the web visit https://groups.google.com/a/dartlang.org/d/msgid/misc/CAPD5VRsSSQqba-PVSgC%3D5N%2BY4FEUPVTCGaOnmAHYMOmo1yoXqg%40mail.gmail.com.

[Vijay Menon]

On Thu, Jun 10, 2021 at 12:09 AM Andrew Mezoni <andrew...@gmail.com> wrote:

>>  I strongly believe that the manner of communication matters and that we can achieve more if we pay attention to the fact that our public messages are read by other humans.

Yes, I'm rude. And this is a fact.
And I excuse myself in front of everyone for intemperance.

I am sorry, but you are not excused here.  You are welcome to post benchmarks and ask questions.  You are not welcome to violate our code of conduct: https://dart.dev/code-of-conduct.

 

But there is still one well-known fact.
Dart developers ignore the community.

Dart developers are under no obligation to respond to you, particularly when you are being purposefully rude.

 

Everyone has known this for a long time.
Will you be pleased when you are ignored?
This is not pleasant for many, but they are well-mannered.
And just, out of decency, they say that everything is fine.
What's good? The fact that developers constantly ignore the wishes of the community?
I don't see anything good in this.

>> I am subscribed to this mailing list in the hopes that I get very important deprecation warnings, important announcements, breaking changes

You are deluded.

This is a place for discussion.

And here a dead silence has been standing for a long time. No discussion whatsoever.

чт, 10 июн. 2021 г. в 11:55, Daniel Varga <dvarg...@gmail.com>:

To view this discussion on the web visit https://groups.google.com/a/dartlang.org/d/msgid/misc/CA%2BTxJqBSv_rTExPaYU9pUu67vuZxHqVrKxNtCbHMAQVj4HWrXw%40mail.gmail.com.

--
For more ways to connect visit https://dart.dev/community
---
You received this message because you are subscribed to the Google Groups "Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.

To view this discussion on the web visit https://groups.google.com/a/dartlang.org/d/msgid/misc/CAAfHsy-zbZBxU8uKtM440mQqKV9Vq3xoaqbQUXQWRoJiae1rBQ%40mail.gmail.com.

[Filip Hracek]

Yes, I'm rude. And this is a fact.
And I excuse myself in front of everyone for intemperance.

I am sorry, but you are not excused here.  You are welcome to post benchmarks and ask questions.  You are not welcome to violate our code of conduct: https://dart.dev/code-of-conduct.

I agree. Everyone should feel free to bring issues to others' attention, but please check your emotions while doing so. Don't be rude, don't accuse others, don't use big red lettering, etc. We should all expect nothing less than professionalism and kindness in communication.

— Filip