UX: dart2dart & pub build (questions & thoughts)

131 views
Skip to first unread message

Thomas Schranz

unread,
Aug 22, 2014, 3:10:37 PM8/22/14
to mi...@dartlang.org
I've written a hop (https://github.com/kevmoo/hop) powered command line tool
that I would like to 'give/distribute' to end users.

Use-case

It is a command line tool that can do various dev related things like building stuff, starting processes etc.
It is used inside of a linux virtual machine (our dev environment set up using vagrant).

=> I want my end users to be able to just 'use' it like it was self-contained without needing to know anything
about dart, pub or how to build dart projects and so on.

dart2dart to the rescue

If I understand it correctly the thing I need to do to pull this off is to transpile (?) my dart project
into a dart file that is self-contained with all its dependencies inlined.

It took me quite some time to figure out that 'dart2dart' as command line tool does not really exist
but that it is a rather oddly named option for the 'dart2js' command line tool (this was not intuitive to me).

I learned that I also need to call it like this:

dart2js --output-type=dart --categories=Server path/to/my/small/app

As ignorant end-user it also wasn't intuitive for me that I have to specify 'Server' as category here.
The output I got before I did this wasn't very clear:

Error: Library not found 'dart:io'.

import 'dart:io';

With that knowledge I did expect that I can use the same settings in pubspec.yaml (see https://code.google.com/p/dart/issues/detail?id=19978)
so I can get the same result by just executing 'pub build tool'.

Maybe I've overlooked something but I was not able to get to that behaviour
using the following settings added to my pubspec.yaml:

transformers:
- $dart2js:
  minify: true
  commandLineOptions: ['--output-type=dart', '--categories=Server']

Questions

Am I using the right tools here for what I want to achieve? (being able to distribute a self-contained command line tool)
or am I going about it the wrong way?

How do you people currently do this? A build.dart file? A custom transformer?

Is there a way to rename/rearrange dart2js and the dart2dart story to make it a bit less confusing?
(I absolutely understand that the command line tool functionality grew and evolved over time)

Strawman

I understand that there might not be an easy solution to this as many use-cases have to be kept in mind
and I certainly don't see the whole picture here but here are some thoughts re what I'd naively expect as a user:

One command line tool 'dart' to rule them all

* without options => an awesome REPL (see python)
* a/file/path => runs / executes the file/app (+ options)
* build => generate js or dart files, (+ options like category/context/environment (?), minification, …)
* analyze => dart analyzer output (+ options)
* fmt/format => formating
* version => …

as well as a pub specific command line tool (very much like the current one) that is delegating to 'dart' wherever it makes sense.
Here I would expect/dream that since pub knows more about the context (directory name convention, pubspec file, …)
and what sensible defaults might be I'd basically just would execute pub build and 'the right thing' (heh)
happens depending on what kind of app I have (web, bin, tool — or a combination of them).

I'm certain I missed some obvious things and I'm still trying to wrap my head around various permutations of environments
and potential output scenrarios but the experience I had was a bit surprising/unintuitive.

hth & sorry for the wall of text, I'm thankful for any pointers :)

Thomas Schranz

unread,
Aug 22, 2014, 3:30:20 PM8/22/14
to mi...@dartlang.org
Some (related) links I found and forgot to mention in the above email (for future reference):




:)

Alexandre Ardhuin

unread,
Aug 22, 2014, 3:32:28 PM8/22/14
to General Dart Discussion


--
For other discussions, see https://groups.google.com/a/dartlang.org/
 
For HOWTO questions, visit http://stackoverflow.com/tags/dart
 
To file a bug report or feature request, go to http://www.dartbug.com/new

To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.

Thomas Schranz

unread,
Aug 22, 2014, 4:49:13 PM8/22/14
to mi...@dartlang.org
Thanks for the pointers!

Bob Nystrom

unread,
Aug 22, 2014, 5:01:57 PM8/22/14
to General Dart Discussion
On Fri, Aug 22, 2014 at 12:10 PM, Thomas Schranz <tho...@blossom.io> wrote:
I've written a hop (https://github.com/kevmoo/hop) powered command line tool
that I would like to 'give/distribute' to end users.

Use-case

It is a command line tool that can do various dev related things like building stuff, starting processes etc.
It is used inside of a linux virtual machine (our dev environment set up using vagrant).

=> I want my end users to be able to just 'use' it like it was self-contained without needing to know anything
about dart, pub or how to build dart projects and so on.

We're hoping that global packages in pub will be the canonical answer for this, but we've still got some work to do. Mainly:
  1. Putting executables from globally activated packages directly on your PATH, à la npm -g.
  2. Startup perf.
dart2dart to the rescue

If I understand it correctly the thing I need to do to pull this off is to transpile (?) my dart project
into a dart file that is self-contained with all its dependencies inlined.

Sure, but keep in mind that only covers dependencies on .dart files. If your app uses other assets from those packages, dart2dart won't help. (Most don't, but it's worth keeping in mind.)
 
It took me quite some time to figure out that 'dart2dart' as command line tool does not really exist
but that it is a rather oddly named option for the 'dart2js' command line tool (this was not intuitive to me).

Yeah, it's not very intuitive. :( Ideally, the command would just be something output-file-independent like "dartc" but that name was already taken by a previous incarnation of the compiler.
 
I learned that I also need to call it like this:

dart2js --output-type=dart --categories=Server path/to/my/small/app

As ignorant end-user it also wasn't intuitive for me that I have to specify 'Server' as category here.
The output I got before I did this wasn't very clear:

The category stuff is weird. I don't understand why seeing an import of "dart:io" doesn't imply that category automatically.
 
With that knowledge I did expect that I can use the same settings in pubspec.yaml (see https://code.google.com/p/dart/issues/detail?id=19978)
so I can get the same result by just executing 'pub build tool'.

Pub build doesn't currently support dart2dart. It did for a while, but we disabled it because the dart2js folks said dart2dart was still being developed and wasn't ready for real usage. If that's not still the case, they haven't told us.
 
Maybe I've overlooked something but I was not able to get to that behaviour
using the following settings added to my pubspec.yaml:

transformers:
- $dart2js:
  minify: true
  commandLineOptions: ['--output-type=dart', '--categories=Server']

That might work, or it may just confuse pub given the above.
 
Am I using the right tools here for what I want to achieve? (being able to distribute a self-contained command line tool)
or am I going about it the wrong way?

You're doing the best you can given that we don't have mature answers for this stuff yet. (It's only been the past few months that we've started to put real attention into the command-line side of Dart.)

My hope is that eventually the canonical solution will be you tell the end user:

$ pub global activate your_awesome_thing
$ your_awesome_thing

...and that's it.
 
Is there a way to rename/rearrange dart2js and the dart2dart story to make it a bit less confusing?

I'd file a bug (or two). That will at least ensure the right people look at it.
 
(I absolutely understand that the command line tool functionality grew and evolved over time)

Strawman

I understand that there might not be an easy solution to this as many use-cases have to be kept in mind
and I certainly don't see the whole picture here but here are some thoughts re what I'd naively expect as a user:

One command line tool 'dart' to rule them all

* without options => an awesome REPL (see python)
* a/file/path => runs / executes the file/app (+ options)
* build => generate js or dart files, (+ options like category/context/environment (?), minification, …)
* analyze => dart analyzer output (+ options)
* fmt/format => formating
* version => …

I know some people on the team are in favor of this, but, as you can imagine, a lot of people on the team have strong opinions about how this should be organized which makes it hard to make progress in any direction.
 
Here I would expect/dream that since pub knows more about the context (directory name convention, pubspec file, …)
and what sensible defaults might be I'd basically just would execute pub build and 'the right thing' (heh)
happens depending on what kind of app I have (web, bin, tool — or a combination of them).

When you say "build" what goal are you trying to accomplish?

Cheers!

- bob

Thomas Schranz

unread,
Aug 22, 2014, 6:26:16 PM8/22/14
to mi...@dartlang.org
Great answers Bob. Thank you.

Without having thought this through much I'd expect `pub build` (or maybe explicitly call it like `pub build all`)
to look at my directory structure and create:

build/web/
build/bin/
build/tool/

(depending on what directories I have (I think this is already the current behaviour))

and by default put an `out.dart` as well as an `out.js` file into each of them.
Those files would be self-sustaining (?) and have all their dependencies inlined.

I'd like things in 'build' as simple and self-contained by default as possible.
No 'packages' directories (also no symlinks etc). It gets more complicated with assets obviously,
I don't have strong opinions about those yet (still need to look into transformers more).

If I only want to build a specific directory (like bin) I'd explicitly call it like 'pub build bin'.

In pubspec.yaml I'd expect that I can add build options (and/or/equal to various transformers?)
like minification, or in the 'command line use case' maybe an option/transformer to declare
that not only do I want out.dart to include all dependencies inlined but also include (?) the vm itself (gen_snapshot (?)).

Ideally I could give a certain set/configuration of transformers a name so I can refer to the specific build I want to do.
This way I can have builds like 'release/production' that might take more time but create an optimized output
vs builds that are fast and more suited for development. I think it makes sense to have a 'default' namespace that will be used if you do 'pub build'
without specifying a namespace (as well as for simple projects that don't have a need for different build configurations).

Right now it feels a bit odd to only be able to specify _one_ specific set of transformers (or am I missing something?).


Not sure how supporting build 'names' would play together with having multiple directories like bin, web, tool etc in an intuitive way.

I guess the build names thing would bring pub a bit far into the territory of more complex build tools which might be bad or great depending
on what the goals are.


If I understand pub and its intentions correctly it almost behaves like what I expect already
and the rest (like support for dart2dart is coming eventually).

(OT: pub serve seems like a kind of pub build --watch (+ simple server) (?) => awesome)


I hope the above makes sense and isn't too confusing,
take it with a grain of salt since I might have some misconceptions about what 'pub' is intended to do.
Apparently the `snapshot` functionality of `dart` was exactly what I needed earlier today (and I was expecting `pub build` or `dart2dart` to do that for me).

:)

Bob Nystrom

unread,
Aug 25, 2014, 4:05:48 PM8/25/14
to General Dart Discussion
On Fri, Aug 22, 2014 at 3:26 PM, Thomas Schranz <tho...@blossom.io> wrote:
Great answers Bob. Thank you.

Without having thought this through much I'd expect `pub build` (or maybe explicitly call it like `pub build all`)
to look at my directory structure and create:

build/web/
build/bin/
build/tool/

(depending on what directories I have (I think this is already the current behaviour))

Yup!
 

and by default put an `out.dart` as well as an `out.js` file into each of them.
Those files would be self-sustaining (?) and have all their dependencies inlined.

Yup!
 
I'd like things in 'build' as simple and self-contained by default as possible.
No 'packages' directories (also no symlinks etc). It gets more complicated with assets obviously,
I don't have strong opinions about those yet (still need to look into transformers more).

Yup!
 
If I only want to build a specific directory (like bin) I'd explicitly call it like 'pub build bin'.

Yup!
 
In pubspec.yaml I'd expect that I can add build options (and/or/equal to various transformers?)

Right, we expect transformers to be what you'd normally do here.
 
like minification, or in the 'command line use case' maybe an option/transformer to declare
that not only do I want out.dart to include all dependencies inlined but also include (?) the vm itself (gen_snapshot (?)).

We don't currently have any plans to build a completely self-contained executable that also includes the VM. You'd probably have to write a little script that does this yourself.
 
Ideally I could give a certain set/configuration of transformers a name so I can refer to the specific build I want to do.
This way I can have builds like 'release/production' that might take more time but create an optimized output
vs builds that are fast and more suited for development.

Pub build/serve support a "mode" argument. Transformers can see the mode and tailor their behavior for it. So, you could define a transformer that does lots of complex optimizations in release mode but not in debug.
 
Right now it feels a bit odd to only be able to specify _one_ specific set of transformers (or am I missing something?).

You can specify as many transformers as you want in your pubspec. You can even specify the same transformer multiple times if, for example, you want to run it on different assets with different configurations. For example:

transformers:
- [foo, bar]
- [baz, bang]

Here we're specifying one phase with two transformers that run in parallel, foo and bar. The outputs of those (and any other assets that they didn't modify) are then available to another phase of transformers containing baz and bang.
 
(OT: pub serve seems like a kind of pub build --watch (+ simple server) (?) => awesome)

Exactly right. :)

Cheers!

- bob

Thomas Schranz

unread,
Aug 25, 2014, 5:37:22 PM8/25/14
to mi...@dartlang.org
Awesome :) thanks
Reply all
Reply to author
Forward
0 new messages