Dart Language and Library Newsletter (2017-08-11)

[Florian Loitsch]

Github link: https://github.com/dart-lang/sdk/blob/1daab67666ce4f7fe5f04b1349000a5492af6302/docs/newsletter/20170811.md

Earlier newsletters:

https://github.com/dart-lang/sdk/blob/d94cecba3ceaf6578315764a07292ea0d5e082c3/docs/newsletter/20170804.md

https://github.com/dart-lang/sdk/blob/d94cecba3ceaf6578315764a07292ea0d5e082c3/docs/newsletter/20170728.md

Dart Language and Library Newsletter

Welcome to the Dart Language and Library Newsletter.

Follow Ups

Void Arrow Functions

As mentioned in an earlier newsletter, void arrow functions with non-void expressions (as in void foo() => x++) are supported with Dart 1.24. However, this feature still has to be used with care. Due to a temporary limitation of the type inference in strong mode, returning a non-void expression might not work as expected.

For example:

var f = new Future(() { doSomethingAsynchronously(); };
f.catchError((e) => errorCounter++);

The type-inference algorithm currently infers Null for the generic type of the Future. Functions without return indeed return null, so technically, that type is correct. However, the catchError signature requires the provided function to return the same type as the function it is attached to. In this case, f is a Future<Null>, but errorCounter++ is an int. Since int is not Null this throws at runtime.

As mentioned in earlier newsletters, we are actively working on generalizing void, and once it is supported the inferred type of f will be Future<void>. The catchError closure then would just need to be a subtype of void Function() which would work fine for (e) => errorCounter++. Until then, be careful where you use the void arrow function syntax.

Deferred Loading

Last time we discussed our plans to allow the use of deferred types even when the deferred libraries haven't been loaded yet. This makes programs, like the following, possible:

/// lib1.dart
class A {}

/// lib2.dart
export "lib1.dart" show A;

/// main.dart
import "lib1.dart";
import "lib2.dart" deferred as def;

main() {
  print(new A() is def.A);  // Requires knowledge of `def.A`.
}

A follow-up mail questioned the need for such a big hammer. In reality, most programs just want to use the deferred type as a type annotations:

main() async {
  def.A a; // <-- Illegal today.
  await def.loadLibrary();
  a = new def.A();
}

Is there a simpler / better solution that would allow patterns like these, but not require full knowledge of the deferred types?

It turns out, that the answer is likely "no". In fact, we find that, because of type inference, even the current behavior is already counterintuitive and should be fixed. That is, even without allowing more uses of deferred types, programs don't behave as expected:

// ------ def.dart
class Box<T> {
  T value;
  Box(this.value);
}

// ------ main.dart
import "def.dart" deferred as def;

main() async {
  await def.loadLibrary();
  var box = new def.Box(499);
  var list = [box.value];
}

With type inference, users expect three things to happen:

1. box is of type def.Box<int>.
2. the generic type of new def.Box(499) is Box<int>, as if the user had written new def.Box<int>(499).
3. list is of type List<int>.

Without access to the deferred sources, none of these expectations is met. Since type inference runs at compile-time, boxhas to be treated like dynamic. There is simply not more information available. For similar reasons, box must be of type Box<dynamic>. Since the invocation of the constructor happens at runtime (where no type-inference happens), the missing generic type is dynamically filled with dynamic.

Finally, list must be of type List<dynamic> since box.value is a dynamic invocation, and the type inference doesn't know that the returned value will be of type int.

This small example shows that type inference requires knowledge of the deferred types to do its job. This means that all sources must be available when compiling individual libraries. Once that's the case it doesn't make sense to restrict the use of deferred types. They don't take up much space (which is the usual reason for deferring libraries), and giving full access to them removes a lot of boilerplate or dynamic code.

Const Functions

The language team discussed the possibility of supporting const functions.

class A {
  final Function(e) callback;
  const A(this.callback);
}

// Provide a `const` function to `A`'s constructor.
const x = const A(const (e) { print(e); });

// Default values have to be `const`.
void sort(List<int> list, [int compare(int x, int y) = const (x, y) => x - y) {
  ...
}

This feature doesn't add new functionality. Users can already now write a static function with the same body and use its tear-off (which is guaranteed to be const) in all of these locations. However, it's more convenient to write functions closer to where they are needed. For example, the classic map.putIfAbsent(x, () => []) allocates a new function (a cheap operation, but still), whereas map.putIfAbsent(x, const () => []) would always reuse the same function.

Sidenote: in dart2js, many const values (not functions) are allocated at initialization, which shifts some execution time to the beginning of the program where many teams already struggle with performance. In the current dart2js version it's thus not always beneficial to make objects const.

Shadowing of Core Libraries

When deprecating core library classes (like SplayTreeMap) we intend to minimize the cost to our users. We copy the deprecated classes to packages (in this case collection) so that users just need to change their imports from dart:collection to package:collection. However, that means that programs that import dart:collection and package:collection at the same time now see the same class twice; once from each import. Which class should Dart now use? Is this an error?

For "normal" imports (not dart:), the rules are simple: an ambiguous reference is an error. There is no good way to decide between class A of package pkg1 or pkg2. With core libraries, things get a bit more complicated: whereas upgrading packages is a user-triggered action (with the fallback to revert to the previous pubspec.lock), upgrading the SDK should generally be safe. As a consequence, Dart considers core libraries as less important. That is, shadowing a class from any dart: library is ok. Importing dart:collection and package:collection/collection.dart is thus fine and will not lead to errors. It's still good practice to use show and hide to make the intention completely clear.

We are still unsure how to handle cases when the user explicitly used show to import a specific core library type:

import 'dart:collection` show SplayTreeMap;
import 'package:collection/collection.dart';

[Danny Tuppeny]

On Fri, 11 Aug 2017 at 16:35 'Florian Loitsch' via Dart Misc <mi...@dartlang.org> wrote:

Shadowing of Core Libraries

We are still unsure how to handle cases when the user explicitly used show to import a specific core library type:

import 'dart:collection` show SplayTreeMap;
import 'package:collection/collection.dart';

If it's not clear what the user wants, reject it! :-)

I'd much rather have errors in my editor than ambiguous code (or unexpected runtime behaviour if it picks the "wrong" one). The fix seems easy; remove show from the dart: import or add hide to the collection import) - it could even have code fixes in the analyzer to make it just a few keypresses to update?

That said; I still think it's a bit ambiguous shadowing without the show.. It seems weird that some random third party package I pull in could silently replace some built-in types and it not be totally obvious unless I step in/jump to definition (this might sound quite ridiculous but only last week I spent several hours investigating an issue that was caused by a NuGet package declaring its own version of System.Runtime.CompilerServices.ExtensionAttribute!).

[Lasse R.H. Nielsen]

The reason to have the rule to begin with is that it allows us to add members to platform libraries without breaking existing code.

Packages are versioned, so if a new version is incompatible, you can stay on the old version until it's fixed.

An upgrade in the SDK isn't as easy to roll back, so we want there to be some way to grow the platform libraries without causing too much breakage.

In this case, having a `show` on the platform library definitely means that the collision was not caused by a platform library addition, so it must have been caused by a package change. So, yes, rejecting is probably the right thing to do, it won't unduly affect platform library development.

The largest argument against is that it complicates the rule - it's not just "if multiple imports introduce different declarations for a name, then it's a conflict, unless there is exactly one non-platform library declaration being introduced, in which case that one wins" (which is already long), but you have to add, "unless one of the platform library declarations is introduced by an import declaration with a show clause".

(well, it's not that bad :)

/L

--
For other discussions, see https://groups.google.com/a/dartlang.org/
 
For HOWTO questions, visit http://stackoverflow.com/tags/dart
 
To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups "Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.

--

Lasse R.H. Nielsen - l...@google.com  

'Faith without judgement merely degrades the spirit divine'
Google Denmark ApS - Frederiksborggade 20B, 1 sal - 1360 København K - Denmark - CVR nr. 28 86 69 84

[Danny Tuppeny]

On Mon, 14 Aug 2017 at 14:08 'Lasse R.H. Nielsen' via Dart Misc <mi...@dartlang.org> wrote:

The largest argument against is that it complicates the rule - it's not just "if multiple imports introduce different declarations for a name, then it's a conflict, unless there is exactly one non-platform library declaration being introduced, in which case that one wins" (which is already long), but you have to add, "unless one of the platform library declarations is introduced by an import declaration with a show clause".

Yeah, ISWYM but I don't think it's too bad. Maybe you could describe that show clauses are used first, then packages, then platform library, with an ambiguous error reported if any stage has multiple results (I think that's the same?).

I'm probably in a minority but I wouldn't care if adding a new type to the SDK colliding with an imported package was just an error and I had to fix it (eg. don't support shadowing; we have prefixes for name collisions). I know it can make SDK upgrades more breaking; but in reality I suspect it won't happen a lot and I don't think anyone should be upgrading the SDK on a production server without testing it locally first! :D IMO the SDK also shouldn't be getting lots of new types, it should be made leaner and most stuff made in packages (.NET has a big fat BCL and seems to be trying to move away from it).

Avoiding dev-time breaks but potentially introducing runtime breaks isn't a good trade IMO.. If we want to let the client find our bugs we can use JavaScript instead of Dart! The compiler shouldn't make assumptions about my code, it something is not clear it should force me to make it clear! :D

[Günter Zöchbauer]

Breaking the source you're currently working on is usually the minor problem, but what if a package you depend on gets broken because of core lib changes. You'd need to wait for the maintainer to fix it, or to merge your PR and publish a new version, or you create a fork that you patch.

When such a broken package is an indirect dependency, you might need to get a whole chain of packages updated to get unblocked.

[Danny Tuppeny]

Valid point; I'd somewhat forgotten that packages are shipped as source (I'm used to C#)! ;o(

--

[Stephen Adams]

​Will it reuse the same function as const () => [] in some other place?

If the answer is Yes, this does make everything harder for all implementations, and opens the question of whether const (x) => x and const (y) => y are the same, or const (_x) => _x and const (_x) => _x in different libraries, and how does the canonicalization work efficiently for separate compilation in DDC?

If the answer is No, and each textual occurrence of a const closure is a separate object, could we not simply make the language create the shared function whenever possible? If a closure captures no variables, then it is shared. Most users would expect to not pay for repeated allocations of trivial closures. The analysis that says 'const is valid for this function' is (almost) the same analysis that says 'use the same closure object here each time'.

In the weird edge case of wanting separate closure objects (for example, as an unbounded number of keys in an identity hash map), we could force that with the keyword new. If this idea seems to 'not pay its way', it shows we are just throwing away performance for no good reason.

​

Sidenote: in dart2js, many const values (not functions) are allocated at initialization, which shifts some execution time to the beginning of the program where many teams already struggle with performance. In the current dart2js version it's thus not always beneficial to make objects const.

This is easily fixed provided const or implicitly const closures are not canonicalized across libraries.

BTW const does not help when there are type parameters. e.g. https://github.com/dart-lang/sdk/commit/dc8a4f21b8f32b4bfd73b51e86412b27a2c0f45d#diff-aaa4279b60e9df6da1c0805bd016374d

Shadowing of Core Libraries

When deprecating core library classes (like SplayTreeMap) we intend to minimize the cost to our users. We copy the deprecated classes to packages (in this case collection) so that users just need to change their imports from dart:collection to package:collection. However, that means that programs that import dart:collection and package:collection at the same time now see the same class twice; once from each import. Which class should Dart now use? Is this an error?

For "normal" imports (not dart:), the rules are simple: an ambiguous reference is an error. There is no good way to decide between class A of package pkg1 or pkg2. With core libraries, things get a bit more complicated: whereas upgrading packages is a user-triggered action (with the fallback to revert to the previous pubspec.lock), upgrading the SDK should generally be safe. As a consequence, Dart considers core libraries as less important. That is, shadowing a class from any dart: library is ok. Importing dart:collection and package:collection/collection.dart is thus fine and will not lead to errors. It's still good practice to use show and hide to make the intention completely clear.

We are still unsure how to handle cases when the user explicitly used show to import a specific core library type:

import 'dart:collection` show SplayTreeMap;
import 'package:collection/collection.dart';

--

[Florian Loitsch]

The answer is No: we consider source-location a part of the identity of a function. Debugging, stack-traces, ... all become completely unmanageable if we merge functions that have `const`. Also, as you mention, we would also run into equivalence questions (and you didn't even mention comments and `=>` vs `return`).

Can compilers automatically canonicalize closures if they don't capture any non-const state?: it's something we think about. This is similar to the question whether `Duration(milliseconds: 499)` should be a const object once we make `new` and `const` optional. That is: should we always insert `const` automatically, when there is no explicit `new` (and `const` works, of course)?

This is still under discussion.

 

​

Sidenote: in dart2js, many const values (not functions) are allocated at initialization, which shifts some execution time to the beginning of the program where many teams already struggle with performance. In the current dart2js version it's thus not always beneficial to make objects const.

This is easily fixed provided const or implicitly const closures are not canonicalized across libraries.

BTW const does not help when there are type parameters. e.g. https://github.com/dart-lang/sdk/commit/dc8a4f21b8f32b4bfd73b51e86412b27a2c0f45d#diff-aaa4279b60e9df6da1c0805bd016374d

Correct. 

[Günter Zöchbauer]

That is: should we always insert `const` automatically, when there is no explicit `new` (and `const` works, of course)?
This is still under discussion.

I'd prefer `const` to be required when not in a context where only const is allowed anyway, to make it more obvious what's going on.

myMethod({MyClass a = MyClass()}) {
}

class MyClass {
   static const a = MyClass();
}

class MyClass {
  final a = const MyClass({'foo': 'bar'});
}

[Bob Nystrom]

On Wed, Aug 16, 2017 at 10:31 PM, 'Florian Loitsch' via Dart Misc <mi...@dartlang.org> wrote:

On Wed, Aug 16, 2017 at 11:52 PM 'Stephen Adams' via Dart Misc <mi...@dartlang.org> wrote:

If the answer is No, and each textual occurrence of a const closure is a separate object, could we not simply make the language create the shared function whenever possible? If a closure captures no variables, then it is shared. Most users would expect to not pay for repeated allocations of trivial closures. The analysis that says 'const is valid for this function' is (almost) the same analysis that says 'use the same closure object here each time'.

In the weird edge case of wanting separate closure objects (for example, as an unbounded number of keys in an identity hash map), we could force that with the keyword new. If this idea seems to 'not pay its way', it shows we are just throwing away performance for no good reason.

The answer is No: we consider source-location a part of the identity of a function. Debugging, stack-traces, ... all become completely unmanageable if we merge functions that have `const`. Also, as you mention, we would also run into equivalence questions (and you didn't even mention comments and `=>` vs `return`).

I thought one of the key motivations for the const function proposal is that Angular code ends up with () => null and other similar tiny functions all over the place. Those contribute a lot of code size to the resulting JS and they wanted a way to have those be automatically collapsed to avoid the redundancy. If const functions don't do that, what's the main customer use case pushing for them?

– bob

[Matan Lurey]

FWIW, the specification can say anything it wants, as long as dart2js implements it differently (dart2js isn't spec complaint anyway).