AOT compile strong mode dart on linux

[Hochhaus, Andy]

Hello,

Is it possible to AOT compile strong mode dart on linux?

I see AOT is possible using flutter for iOS/android and some llvm experiments have  also been performed. However, nothing pops out as readily usable on Linux/server side. Did I miss anything?

Best,

Andy

[Matan Lurey]

Hi Andy

The reason for AOT, specifically for Flutter, is that it impermissible on some mobile platforms to compile code (JIT), and fast-startup is a huge priority.

For running Linux applications via the command-line, I'd just recommend using the normal VM/JIT.

You can improve startup a bit by creating snapshots (similar to the AOT process):

https://github.com/dart-lang/sdk/wiki/Snapshots 

Cheers!

Matan

--
For other discussions, see https://groups.google.com/a/dartlang.org/
 
For HOWTO questions, visit http://stackoverflow.com/tags/dart
 
To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups "Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.

[Hochhaus, Andy]

Thanks Matan.

On Mon, Nov 27, 2017 at 12:30 PM, Matan Lurey <ma...@lurey.org> wrote:
> The reason for AOT, specifically for Flutter, is that it impermissible on some mobile platforms to compile code (JIT), and fast-startup is a huge priority.

Apple's App Store JIT policy makes perfect sense as the initial
motivation for implementing AOT. Still, other use cases (eg: ease of
distribution of "binaries", confidentiality/obscurity of source code,
etc) might exist on linux.

Given that AOT has been implemented (for iOS), is it possible to
perform AOT on Linux?

> For running Linux applications via the command-line, I'd just recommend using the normal VM/JIT.
>
> You can improve startup a bit by creating snapshots (similar to the AOT process):
> https://github.com/dart-lang/sdk/wiki/Snapshots

Thanks for the pointers.

Best,
Andy

[Matan Lurey]

On Mon, Nov 27, 2017 at 12:53 PM Hochhaus, Andy <ahoc...@samegoal.com> wrote:

Thanks Matan.

On Mon, Nov 27, 2017 at 12:30 PM, Matan Lurey <ma...@lurey.org> wrote:
> The reason for AOT, specifically for Flutter, is that it impermissible on some mobile platforms to compile code (JIT), and fast-startup is a huge priority.

Apple's App Store JIT policy makes perfect sense as the initial
motivation for implementing AOT. Still, other use cases (eg: ease of
distribution of "binaries", confidentiality/obscurity of source code,
etc) might exist on linux.

Given that AOT has been implemented (for iOS), is it possible to
perform AOT on Linux?

Yes, but for the reasons I described it's probably not beneficial.

The VM team is focusing on the Flutter use case for now, so there is a danger if you choose to use AOT for Linux you'll gain little benefit and lots of pain. I'd focus on the snapshot case below for now.

 

> For running Linux applications via the command-line, I'd just recommend using the normal VM/JIT.
>
> You can improve startup a bit by creating snapshots (similar to the AOT process):
> https://github.com/dart-lang/sdk/wiki/Snapshots

Thanks for the pointers.

Best,
Andy

[Michael Francis]

If this is something you absolutely feel you need. You could always setup a workflow where you take the Dart source code, and embed your application snapshot so it automatically loads the snapshot you embedded instead of running a script.

I'd start by looking at
//sdk/runtime/bin/main.cc

Around line 1021 you see the following.

AppSnapshot* app_snapshot = Snapshot::TryReadAppSnapshot(script_name);

  if (app_snapshot != NULL) {

    vm_run_app_snapshot = true;

    app_snapshot->SetBuffers(&vm_snapshot_data, &vm_snapshot_instructions,

                             &app_isolate_snapshot_data,

                             &app_isolate_snapshot_instructions);

[Vyacheslav Egorov]

> Is it possible to AOT compile strong mode dart on linux?

It is possible to AOT compile (or precompile) *any* Dart on Linux/Windows/MacOS (on x64, ARM or ARM64 architectures) as long as your code does not use mirrors. 

Unfortunately you can't do this purely by using the SDK, because binaries you need are not distributed with it at the moment. 

You would need to build Dart from sources. Make sure to build dart_precompiled_runtime and dart_bootstrap targets. 

Then you simply do:

$ out/ReleaseX64/dart_bootstrap --snapshot-kind=app-aot --use-blobs --snapshot=app.snapshot app.dart 

$ out/ReleaseX64/dart_precompiled_runtime app.snapshot

Note: AOT compiled code has different performance characteristics from JIT compiled code, and at the moment might be considerably slower for

some specific use cases. 

// Vyacheslav Egorov

[Florian Loitsch]

On Tue, Nov 28, 2017 at 7:37 AM 'Vyacheslav Egorov' via Dart Misc <mi...@dartlang.org> wrote:

> Is it possible to AOT compile strong mode dart on linux?

It is possible to AOT compile (or precompile) *any* Dart on Linux/Windows/MacOS (on x64, ARM or ARM64 architectures) as long as your code does not use mirrors. 

Unfortunately you can't do this purely by using the SDK, because binaries you need are not distributed with it at the moment. 

You would need to build Dart from sources. Make sure to build dart_precompiled_runtime and dart_bootstrap targets. 

 

This might sound more complicated than it actually is.

Follow the instructions here to get a build on your machine:

https://github.com/dart-lang/sdk/wiki/Building#getting-the-source

Then do

tools/build.py --mode=release dart_precompiled_runtime dart_bootstrap

Then you should be able to continue with Slava's instructions.

[Hochhaus, Andy]

Thanks Vyacheslav and Florian! That meets my needs perfectly.

[Adrian Mercieca]

Hi all,

Well, I tried all this as per the instructions given; built the SDK from source and then did some tests.

Vyacheslav was right actually; in the web server test I did (code below), the AOT compiled version ran considerably slower than the JIT one.

 

#!/usr/bin/dart

library hw2;

import 'dart:async';

import 'dart:isolate';

import 'dart:io';

Future<Null> serve(final SendPort sendport) async {

  final HttpServer httpd =

      await HttpServer.bind(InternetAddress.ANY_IP_V4, 8777, shared: true);

  print('Serving on port ${httpd.port}');

  await for (final HttpRequest req in httpd) {

    req.response

      ..statusCode = 200

      ..write("Hello World from Dart")

      ..close();

  }

}

Future<Null> main(final List<String> args) async {

  final int cpuCount = Platform.numberOfProcessors;

  for (int i = 0; i < cpuCount; ++i) {

    await Isolate.spawn(serve, new ReceivePort().sendPort);

  }

}

On my humble machine, I was getting about 12K requests per second with the JIT version; only managed about 8.5K (max) with the AOT version.

Quite disappointing actually. (Perhaps my code is wrong?)

- Adrian.

[Matan Lurey]

It's not that surprising, most servers are either written in languages that JIT (i.e. Java, Node) or are much lower-level (Rust, C++) with less overhead.

In the default (JIT) versus of Dart, the compiler is able to make many optimizations over time that can near the level of C++ in some cases, but in AOT it can (only) make some pessimistic optimizations ahead-of-time. Future versions of the AOT compiler will be able to take more advantage of strong mode and other upcoming language changes to improve, but imagine for long-running server-side applications you will almost always want to use the JIT, regardless.

Cheers!

--

[Adrian Mercieca]

Hi Matan,

Thanks for your answer (and sorry for my delay in answering).

Well, I have experienced Java improved performance of server apps over time (after considerable warm-up), so I can correlate with your view.

But on the other hand, I have seen C++ REST frameworks delivering 320K requests per second (doing the thing above - nothing complicated, just returning 'Hello World' just as my Dart code quoted was doing). The best I could manage with Dart (JIT-ed) on my new (same spec) machine was about 55K requests per second. Why would Dart be about 4 times slower than C++ even after considerable warm-up running in JIT mode?

Regards.

[Zach Anderson]

HttpResponse.write() does a conversion to utf8 and one or more copies and/or allocations, which are probably not too great for benchmarking.

The benchmark here: https://github.com/costajob/app-servers/blob/master/servers/dart_server.dart seems more apples-to-apples to what you're doing with C++, but you'd probably also want to pre-allocate the ContentType object in that benchmark instead of allocating a new one for every request.

Cheers,

Zach

--

[Adrian Mercieca]

Hi Zach, et al

Well, I amended the code as below (based on your instructions):

#!/usr/bin/dart

import 'dart:convert';

import 'dart:io';

import 'dart:isolate';

const int _PORT = 9292;

const String _HOST = '0.0.0.0';

const String _GREET = 'Hello World';

final ContentType _CONTENT_TYPE = new ContentType('text', 'plain');

_startServer(arg) {

final List<int> response = UTF8.encode(_GREET);

HttpServer.bind(_HOST, _PORT, shared: true).then((server) {

server.listen((HttpRequest request) {

request.response

..headers.contentType = _CONTENT_TYPE

..add(response)

..close();

});

});

}

void main() {

final int _CORES = Platform.numberOfProcessors ~/ 2;

print("Cores: $_CORES");

for (int i = 0; i < _CORES - 1; ++i) {

Isolate.spawn(_startServer, null);

}

_startServer(null);

}

and ran an ab test on it, executing: 

ab -n 786432 -c 128 -k http://localhost:9292/

Results obtained (3 runs):

• Requests per second:    33469.39 [#/sec] (mean)
• Requests per second:    35338.14 [#/sec] (mean)
• Requests per second:    34385.53 [#/sec] (mean)

Then I built this C++ equivalent (using pistache: http://pistache.io/), code:

#include "pistache/endpoint.h"

using namespace Pistache;

class HelloHandler : public Http::Handler {

public:

HTTP_PROTOTYPE(HelloHandler)

void onRequest(const Http::Request& request, Http::ResponseWriter response) {

response.send(Http::Code::Ok, "Hello, World");

}

};

int main() {

Address addr(Ipv4::any(), Port(9080));

auto opts = Http::Endpoint::options().threads(4);

Http::Endpoint server(addr);

server.init(opts);

server.setHandler(std::make_shared<HelloHandler>());

server.serve();

}

Compiled the code with:

g++ -pthread -s -o x -Ofast hw.cpp -lpistache 

and ran ab test on it, executing:

ab -n 786432 -c 128 -k http://localhost:9080/

Results obtained (3 runs):

• Requests per second:    253391.44 [#/sec] (mean)
  
• Requests per second:    252958.08 [#/sec] (mean)
  
• Requests per second:    251081.52 [#/sec] (mean)

I ran these tests on an Ubuntu Linux 17.04 64-bit machine with 32GB RAM having Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz (4 cores with hyper-threading).

As you can see, the best I could get with JIT-ed Dart was about 35K requests per second; using C++, I was getting an average of about 251K requests per second. So basically, C++ is about 7 times faster than JIT-ed Dart! This is miles away from the 'near the level of C++' mentioned. What could I be doing wrong? Why can't I get anywhere close to the level of C++? 

- Adrian.

You received this message because you are subscribed to a topic in the Google Groups "Dart Misc" group.
To unsubscribe from this topic, visit https://groups.google.com/a/dartlang.org/d/topic/misc/BCfmQ6WTfNU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to misc+uns...@dartlang.org.

[Istvan Soos]

Run your script with dart --checked --enable-vm-service:60156 (port is
just a random port), check the web interface
(http://127.0.0.1:60156/), enable cpu profiling, do your load test,
and check back on the resulting profile.

Very little time is spent in the actual dart code
`request.response[...].close()`. Most of the time is spent in native
socket handling and in the event notification loop (notifying
listeners, running the microtask loop). It is slower than your C++
benchmark not because of the JIT, but because it does a bit more stuff
(which you don't use in your benchmark), and with such short-lived
requests, that overhead dominates the performance.

The more you move away from these synthetic microbenchmarks, the
overhead would become much less in CPU-percentage. Add some backend/DB
calls, JSON processing, validation logic, and soon enough the JIT
performance is really comparable to native. For example this benchmark
shows when the Dart VM can win by a good margin over C++:
https://www.techempower.com/benchmarks/#section=data-r15&hw=ph&test=query

[Adrian Mercieca]

Hi Istvan.

I see your point; probably, when accessing DB's, etc - as you say, performance will be close.

We're so tempted to switch to using Dart on the back-end as well because it is such a fluid, productive and expressive language. We already use it for all our front-end work (web + mobile, using flutter); perhaps we need to do some real research/study on using it server side.

Thanks for your help.

- Adrian.

p.s. by the way... can one interface easily to C from Dart? Perhaps in the really performance critical stuff, we could call C from Dart; is that possible?

[Florian Loitsch]

The Dart VM supports C interop: https://www.dartlang.org/articles/dart-vm/native-extensions

That article is quite old, though. I'm not sure if things still work the way they are described in that document.

[cc "maco" young]

Speaking of server-side Dart: mailing list, user group?

[ravi teja]

@young https://gitter.im/dart-lang/server

[Istvan Soos]

It seems to be easy to interface, but I've just read the source code,
haven't tried to do it myself. One example is in the leveldb package:
https://github.com/adamlofts/leveldb_dart

pub.dartlang.org is using Dart and Appengine on the server-side, the
performance bottleneck is usually the latency to the Appengine backend
(or more precisely: the accumulated latency of not entirely concurrent
requests), and not the Dart code.

I also have a server-side data processing codebase using Postgresql,
ElasticSearch and Dart, and so far I'm happy with the performance. It
is usually IO-limited, I can saturate an octa-core Ryzen server with
SSDs only with one or two parallel Dart isolates (the Dart code doing
10-20 concurrent connections to the database and to elastic).

Cheers,
Istvan