Analysis engine ‘strict’ mode - a proposal

[Paul Jolly]

The following document proposes a ‘strict’ mode for the Dart Analysis Engine, a mode which when enabled generates ‘strict typing hints’ about Dart code above and beyond the static warnings dictated by the Dart Language Specification:

https://docs.google.com/document/d/14CPEUzLmF__q3SDZX7H4w4Z_YV6Jeyzu_gdU3k9BOpc/edit?usp=sharing

The proposal presents some motivating examples, as well as highlighting when such a 'strict' mode might be useful (e.g. enforcing code quality in a continuous integration). It also highlights where changes would be required to dartanalyzer and the Dart Analysis Server.

Anyone looking to register support for/interest in this proposal should do so by voting for issue 20443:

https://code.google.com/p/dart/issues/detail?id=20443

Please use this thread on dart-misc for any discussion. Feedback/thoughts greatly appreciated

Paul

[Filipe Morgado]

I understood that the 'strict' mode enforces the use of type annotations.

I always thought that the way to go would be to statically infer types of 'var'.

[Alex Tatumizer]

> A strict type hint is generated if a type annotation is omitted anywhere the spec defines one could be accepted

I think this is way too strict. E.g. you write 

var f=new SimpleBeanFactoryAwareAspectInstanceFactory();

is it really necessary to still annotate f?

Probably you meant "if annotation is omitted ... and cannot be inferred from context"?

[Günter Zöchbauer]

It's great news that you consider adding strict mode!

Can you allow comments in the doc?

I don't think this is the desired behavior (at least not for me)

SE1

Enforce the use of type annotations

A strict type hint is generated if a type annotation is omitted anywhere the spec defines one could be accepted. Corollary: a strict type hint is generated anywhere the unknown type dynamic is found during static analysis (Object should be used instead)

instead a hint should be generated when the type inferred from the right hand side would generate a warning (as when this type had been added explicitly).
Dynamic on the RHS should only be valid when the LHS was declared as dynamic too.

void main() {

 var i = blah(); // this should not produce a hint

// var should allow dynamic

 print(i);

}

// blah() is missing a return type annotation

// a hint is fine assuming that flow analysis isn't supported // otherwise the type could be inferred from the return type

// the least common denominator would be fine

blah() {

 return 4;

}

How would this be handled?

        void main() {

 var i = blah();  

 print(i);

}

dynamic blah() { // <= added type annotation

 return 4;

}

[Paul Jolly]

It's great news that you consider adding strict mode!

Glad you are interested! Thank you for the feedback.

 

Can you allow comments in the doc?

I decided against that; managing discussion here in this thread leaves the document 'clean' for someone reading it for the first time. We can, and of course will, produce revisions as required.

 

I don't think this is the desired behavior (at least not for me)

SE1

Enforce the use of type annotations

A strict type hint is generated if a type annotation is omitted anywhere the spec defines one could be accepted. Corollary: a strict type hint is generated anywhere the unknown type dynamic is found during static analysis (Object should be used instead)

instead a hint should be generated when the type inferred from the right hand side would generate a warning (as when this type had been added explicitly).
Dynamic on the RHS should only be valid when the LHS was declared as dynamic too.

void main() {

 var i = blah(); // this should not produce a hint

// var should allow dynamic

 print(i);

}

// blah() is missing a return type annotation

// a hint is fine assuming that flow analysis isn't supported // otherwise the type could be inferred from the return type

// the least common denominator would be fine

blah() {

 return 4;

}

As I explain in the proposal, the overarching goal behind is to try and help improve code quality and reduce the effort required to achieve higher quality code. Missing off the return type annotation from blah() doesn't help me, as a human, understand the code any better. In fact it makes it worse. Computers might be good at flow analysis; I am not.

Use of var is equivalent to omitting a type annotation, hence is equivalent to use of dynamic. The proposal (currently) precludes the use of dynamic as a type annotation (see my comments further down). Instead, it is proposed Object be used. 

i.e. the following would generate a strict type hint:

var i = 5;

The following would not however:

Object i = 5;

How would this be handled?

        void main() {

 var i = blah();  

Strict type hint generated: missing type annotation.

 

 print(i);

}

dynamic blah() { // <= added type annotation

 return 4;

}

Strict type hint generated: cannot use dynamic as return type annotation. 

Rereading the proposal, I think I need to make SE1 stronger because it allows for dynamic to be used as a type identifier, even though the corollary suggests otherwise (I've just added some markup to that effect)

Your example would need to be rewritten as:

void main() {

  Object i = blah();   

  print(i);

}

Object blah() {

  return 4;

} 

[Paul Jolly]

Thanks for the feedback Filipe.

As I mentioned in my response to Günter, the overarching goal is to

try and help improve code quality and reduce the effort required to

achieve higher quality code. Missing off type annotations doesn't help
me, as a human, understand the code any better, even if it is possible
for a computer (and eventually me as a human) to statically infer a
type.

> --
> For other discussions, see https://groups.google.com/a/dartlang.org/
>
> For HOWTO questions, visit http://stackoverflow.com/tags/dart
>
> To file a bug report or feature request, go to http://www.dartbug.com/new
>
> To unsubscribe from this group and stop receiving emails from it, send an
> email to misc+uns...@dartlang.org.

[Paul Jolly]

Alex - thanks for the feedback.

As I mentioned in my response to Günter, yes, the type annotation is (in the view of the 'strict' proposal) still necessary.

Coming from a Go background I like what (I think) you are proposing because it is close to short variable declarations:

http://golang.org/ref/spec#Short_variable_declarations

However, the specifications of the two languages are quite different. In Go a variable carries its type around with it, hence the following generates a compile error:

package main
import "fmt"

func main() {
  i := doIt() // i's type becomes int
  s := "test" // s's type becomes string (compile time constant)
  s = i  // ** cannot use i (type int) as type string in assignment

  fmt.Printf("We have i = %v and s = %v\n", i, s)
}

func doIt() int {
  return 5
}

 

However in Dart, use of var is equivalent to omitting a type annotation. Hence the following is permitted:

void main() {

  var i = 5;

  i = new List<int>();

  print(i);

}

Under the 'strict' proposal, the above Dart code would be invalid, and would instead need to be written as:

void main() {

  Object i = 5;

  i = new List<int>();

  print(i);

}

Your example would, therefore, need an explicit type annotation on the LHS. 

[Günter Zöchbauer]

Thanks for your feedback.

On Thursday, August 14, 2014 12:19:26 PM UTC+2, Paul Jolly wrote:

It's great news that you consider adding strict mode!

As I explain in the proposal, the overarching goal behind is to try and help improve code quality and reduce the effort required to achieve higher quality code. Missing off the return type annotation from blah() doesn't help me, as a human, understand the code any better. In fact it makes it worse. Computers might be good at flow analysis; I am not.

Redundant type information everywhere don't help much either to make the code easier to understand.

SomeType<SomeOtherType> x = new SomeType<SomeOtherType>();

Who wants to read this? This doesn't help to improve quality.

This doesn't reduce effort. What I save in debugging I pay in writing during development and by replacing type annotations during refactoring. 

I still think the only helpful feature would be that the analyzer produces hints when it can't verify the code with the existing type annotations.

With 'dynamic' I explicitly tell the analyzer that it shouldn't care.

Do you plan to use strict mode internally?

• Build tools and continuous integration - dartanalyzer, with modifications as proposed, could be used as part of a continuous integration setup. For example, a Travis build of, say, AngularDart could be made to fail if someone committed code that omitted a return type annotation

Is strict mode only checked for the package currently analyzed but ignored for dependencies?

I guess this would be fine for check-in-hooks when each package is in its own repository 

but I wonder how this would work for CI when the application is split into several packages.

I would then need to analyze each package separately instead of just the entry point I guess?

[Alex Tatumizer]

If the goal is to help human read the code, then I would argue that extra Cucumber in

final Cucumber cucumber = new Cucumber();

makes things worse, not better, compared with

final cucumber = new Cucumber();

Maybe "final" deserves special treatment in your proposal?

[Paul Berry]

On 14 August 2014 03:19, Paul Jolly <pa...@myitcv.org.uk> wrote:

It's great news that you consider adding strict mode!

Glad you are interested! Thank you for the feedback.

 

Can you allow comments in the doc?

I decided against that; managing discussion here in this thread leaves the document 'clean' for someone reading it for the first time. We can, and of course will, produce revisions as required.

 

I don't think this is the desired behavior (at least not for me)

SE1

Enforce the use of type annotations

A strict type hint is generated if a type annotation is omitted anywhere the spec defines one could be accepted. Corollary: a strict type hint is generated anywhere the unknown type dynamic is found during static analysis (Object should be used instead)

instead a hint should be generated when the type inferred from the right hand side would generate a warning (as when this type had been added explicitly).
Dynamic on the RHS should only be valid when the LHS was declared as dynamic too.

void main() {

 var i = blah(); // this should not produce a hint

// var should allow dynamic

 print(i);

}

// blah() is missing a return type annotation

// a hint is fine assuming that flow analysis isn't supported // otherwise the type could be inferred from the return type

// the least common denominator would be fine

blah() {

 return 4;

}

As I explain in the proposal, the overarching goal behind is to try and help improve code quality and reduce the effort required to achieve higher quality code. Missing off the return type annotation from blah() doesn't help me, as a human, understand the code any better. In fact it makes it worse. Computers might be good at flow analysis; I am not.

Use of var is equivalent to omitting a type annotation, hence is equivalent to use of dynamic. The proposal (currently) precludes the use of dynamic as a type annotation (see my comments further down). Instead, it is proposed Object be used. 

Yes, from the language point of view, var is equivalent to dynamic.  But I don't think that necessarily means we have to treat it as equivalent for the purposes of the type checking in strict mode.

I would also be willing to consider if, for the purposes of strict mode type checking, we treated:

var i = 5;

as shorthand for:

int i = 5;

However, if we go down that path, I think we'd need to sharply delineate exactly what type inference strict mode will do, and what type inference it won't.  I would propose that we only allow 'var' on a variable declaration when the RHS is a literal or an instance creation expression.  So for example:

var i = 5;   =>   int i = 5;

var s = 'foo';   =>   String s = 'foo';

var d = 5.0;   =>   double d = 5.0;

var c = new Cow();   =>   Cow c = new Cow(); *

var l = <int>[];   =>   List<int> l = <int>[]; *

var m = <int, String>{};   =>   Map<int, String> m = <int, String>{}; *

Note: the three examples I marked with "*" above are my motivation for suggesting this.  I find these patterns come up frequently, and it really annoying to have to repeat the type when it appears elsewhere on the same line.

But in the cases below, type inference would not be done, and hence the user would be required to replace "var" with the desired type:

var x = [1, 2, 3];   // type is ambiguous; it could be List<int>, List<num>, or List<Object>.

var x = {'foo': 'bar'};   // also ambiguous; could be Map<String, String>, Map<String, Object>, etc.

var x = 3 + 5;   // RHS is neither a literal nor an instance creation expression.

var x = blah();   // RHS is neither a literal nor an instance creation expression.

That's my two cents, anyway.

PaulB

[Paul Jolly]

PaulB - thanks for the feedback. Good to get some Dart Analysis Server perspective.

Yes, from the language point of view, var is equivalent to dynamic.  But I don't think that necessarily means we have to treat it as equivalent for the purposes of the type checking in strict mode.

I would also be willing to consider if, for the purposes of strict mode type checking, we treated:

var i = 5;

as shorthand for:

int i = 5;

However, if we go down that path, I think we'd need to sharply delineate exactly what type inference strict mode will do, and what type inference it won't.  I would propose that we only allow 'var' on a variable declaration when the RHS is a literal or an instance creation expression.  So for example:

var i = 5;   =>   int i = 5;

var s = 'foo';   =>   String s = 'foo';

var d = 5.0;   =>   double d = 5.0;

var c = new Cow();   =>   Cow c = new Cow(); *

var l = <int>[];   =>   List<int> l = <int>[]; *

var m = <int, String>{};   =>   Map<int, String> m = <int, String>{}; *

Note: the three examples I marked with "*" above are my motivation for suggesting this.  I find these patterns come up frequently, and it really annoying to have to repeat the type when it appears elsewhere on the same line.

I was just in the process of responding to Günter and Alex, but given that we're all talking about the same point I'll respond with some thoughts here.

As I said in my response to Alex, given my Go background I'm very much minded to agree:

var m = <int, String>{}; // OR the theoretical

m := <int, String>{};    // Go-like, but not valid Dart

is indeed much cleaner to write and read than:

Map<int, String> m = <int, String>{};

However does this not introduce some ambiguity with respect to the spec? For example if I were allowed to write:

var m = <int, String>{};  // [1]

m = new List<Dog>();      // [2]

with strict analysis of [1] essentially attributing the type Map<int, String> to m, line [2] would generate a strict type hint. I think we're agreed on that point.

But according to the spec, use of var is equivalent to dynamic.  So someone unaware of this special 'strict' analysis rule (but yet subject to it, perhaps as part of a team decision) would perhaps be understandably confused.

Don't get me wrong, I would much prefer the shorter versions you all propose. But, acting as devil's advocate, are we getting too close to the spec here?

But in the cases below, type inference would not be done, and hence the user would be required to replace "var" with the desired type:

var x = [1, 2, 3];   // type is ambiguous; it could be List<int>, List<num>, or List<Object>.

Indeed. According to the proposal this would generate a strict type hint because a type annotation is missing on the RHS.

 

var x = {'foo': 'bar'};   // also ambiguous; could be Map<String, String>, Map<String, Object>, etc.

Ditto.

 

var x = 3 + 5;   // RHS is neither a literal nor an instance creation expression.

Is the RHS not a compile time constant here?

 

var x = blah();   // RHS is neither a literal nor an instance creation expression.

But if blah() has a return type annotation this is surely permissible? 

[Paul Jolly]

Redundant type information everywhere don't help much either to make the code easier to understand.

Please see my response to PaulB. I'm minded to agree.

Do you plan to use strict mode internally?

Not sure what you mean by internally?

• Build tools and continuous integration - dartanalyzer, with modifications as proposed, could be used as part of a continuous integration setup. For example, a Travis build of, say, AngularDart could be made to fail if someone committed code that omitted a return type annotation

Is strict mode only checked for the package currently analyzed but ignored for dependencies?

You raise a good point. I think such details however fall under further more detailed proposals for both dartanalyzer and the Dart Analysis Server, but that certainly shouldn't stop discussion here and now (see below)

I guess this would be fine for check-in-hooks when each package is in its own repository 

but I wonder how this would work for CI when the application is split into several packages.

I would then need to analyze each package separately instead of just the entry point I guess?

I can certainly see a situation as follows:

• I want code I write to be subject to 'strict' analysis
• I use AngularDart, they also subject their code to 'strict' analysis; great
• However, package banana that I'm using doesn't use 'strict' analysis

Problems would arise if package banana exposed an API that didn't pass 'strict' analysis. For example if it exposed functions:

class Fruit {

  Slice() {

    // ...

  }

}

class Apple extends Fruit {

} 

CreateNewFruit() {

  return new Apple();

}

CutUpFruit(f) {

  f.Slice();

}

We would probably want 'strict' analysis of our code to avoid returning errors for the banana package: there may be hundreds of them. But how then to deal with the above in our code? I think we need to resort to casting:

class Fruit {

  Slice() {

    // ...

  }

}

class Apple extends Fruit {

} 

CreateNewFruit() {

  return new Apple();

}

CutUpFruit(f) {

  f.Slice();

}

void main() {

  Apple a = CreateNewFruit() as Apple;

  a.Slice(); // OR

  CutUpFruit(a);

}

Object o = CreateNewFruit();

Far easier would be to ask the maintainer of banana to adopt 'strict' analysis. It makes things clearer for everyone.

[Günter Zöchbauer]

On Thursday, August 14, 2014 4:26:56 PM UTC+2, Paul Jolly wrote:

Redundant type information everywhere don't help much either to make the code easier to understand.

Please see my response to PaulB. I'm minded to agree.

Do you plan to use strict mode internally?

Not sure what you mean by internally?

This was in relation to the topic mentioned below (CI/dependencies). 

Would the packages shipped by the Dart team be updated to comply with strict mode.

[Paul Jolly]

On Thursday, August 14, 2014 4:26:56 PM UTC+2, Paul Jolly wrote:

Redundant type information everywhere don't help much either to make the code easier to understand.

Please see my response to PaulB. I'm minded to agree.

Do you plan to use strict mode internally?

Not sure what you mean by internally?

This was in relation to the topic mentioned below (CI/dependencies). 

Would the packages shipped by the Dart team be updated to comply with strict mode.

We would need someone from Google/Dart team to comment on that: I don't fall into either category!

[Günter Zöchbauer]

Yes, from the language point of view, var is equivalent to dynamic.  But I don't think that necessarily means we have to treat it as equivalent for the purposes of the type checking in strict mode.

I would also be willing to consider if, for the purposes of strict mode type checking, we treated:

var i = 5;

as shorthand for:

int i = 5;

However, if we go down that path, I think we'd need to sharply delineate exactly what type inference strict mode will do, and what type inference it won't.  I would propose that we only allow 'var' on a variable declaration when the RHS is a literal or an instance creation expression.  So for example:

var i = 5;   =>   int i = 5;

var s = 'foo';   =>   String s = 'foo';

var d = 5.0;   =>   double d = 5.0;

var c = new Cow();   =>   Cow c = new Cow(); *

var l = <int>[];   =>   List<int> l = <int>[]; *

var m = <int, String>{};   =>   Map<int, String> m = <int, String>{}; *

I would add function/method calls that have a type annotation for the return type

 

Note: the three examples I marked with "*" above are my motivation for suggesting this.  I find these patterns come up frequently, and it really annoying to have to repeat the type when it appears elsewhere on the same line.

But in the cases below, type inference would not be done, and hence the user would be required to replace "var" with the desired type:

var x = [1, 2, 3];   // type is ambiguous; it could be List<int>, List<num>, or List<Object>.

var x = {'foo': 'bar'};   // also ambiguous; could be Map<String, String>, Map<String, Object>, etc.

for these two at least `List` and `Map` could be inferred.
When all elements have the same type I would also infer the generic type

if the developer wants a more general type he can change to <num>[1, 2, 3] or <dynamic>[1,2,3]

similar for map 

 

var x = 3 + 5;   // RHS is neither a literal nor an instance creation expression.

as already mentioned this is a constant expression, these should always be inferred.

 

var x = blah();   // RHS is neither a literal nor an instance creation expression.

when blah() has a type annotation for the return type it should be inferred

[Günter Zöchbauer]

Sorry, didn't notice.  

[Paul Berry]

Yes, agreed.

 

But according to the spec, use of var is equivalent to dynamic.  So someone unaware of this special 'strict' analysis rule (but yet subject to it, perhaps as part of a team decision) would perhaps be understandably confused.

Don't get me wrong, I would much prefer the shorter versions you all propose. But, acting as devil's advocate, are we getting too close to the spec here?

I'll grant you that someone might certainly find this confusing.  In fact I'm glad you brought that up because I think it's a good way to think about the tradeoffs involved in my proposal: I'm proposing a change that allows the user to reduce annoying redundancy at the expense of some additional risk of confusion.  IMHO it's worth it, but I certainly understand if someone else feels otherwise :)

 

But in the cases below, type inference would not be done, and hence the user would be required to replace "var" with the desired type:

var x = [1, 2, 3];   // type is ambiguous; it could be List<int>, List<num>, or List<Object>.

Indeed. According to the proposal this would generate a strict type hint because a type annotation is missing on the RHS.

 

var x = {'foo': 'bar'};   // also ambiguous; could be Map<String, String>, Map<String, Object>, etc.

Ditto.

 

var x = 3 + 5;   // RHS is neither a literal nor an instance creation expression.

Is the RHS not a compile time constant here?

Correct.  But my proposal was to do type inference only for literals and instance creation expressions, not necessarily for all compile time constants.

You might be able to talk me into allowing type inference for compile time constants as well, but the downside is that it forces more complexity on analysis (there are some odd corner cases in the evaluation of compile time constants in the analyzer today, and I worry that allowing types to be inferred from compile time constants would make those corner cases even worse).  My bet is that this sort of declaration occurs rarely enough in practice that it wouldn't really benefit enough people to be worth the added complexity.

 

 

var x = blah();   // RHS is neither a literal nor an instance creation expression.

But if blah() has a return type annotation this is surely permissible? 

I haven't decided how I feel about this.  One of the benefits, IMHO, of strict mode is that it allows me to change a method in my code and be confident that warnings and hints will lead me to all of the other code that's potentially affected by that change.  For example, let's say I change the return type of blah().  Under Paul Jolly's definition of strict mode, nearly* all uses of blah() would immediately be flagged, and then I could go through them one by one to see if they merely needed to have their type annotation changed or if they needed to be more extensively reworked.  However, if we allow "var x = blah();" to do type inference based on the return type of blah(), then I'd lose that benefit; I would only get a warning/hint if some later code happened to use x in a way that conflicts with the new type.  Which means I would miss out on some important opportunities to avoid making mistakes.  Also, any warnings/hints I did get would be located at the site where x was used rather than at the site of the call to blah(), so it would require more mental effort to figure out what needed to be changed.

*Of course, some uses of blah() would still not be flagged, e.g. 'blah().foo' would only be flagged if the new type lacked a member called 'foo'.

This sort of process (make a change and then chase down warnings/hints to make sure I've addressed all affected code) was a really useful part of my workflow back when I was programming in C++ and C#, and I've been frustrated by the fact that I can't do this in Dart.  So that's the reason I didn't want to do type inference on 'var x = blah();'--I was looking for a way to reduce the amount of redundant typing I'd have to do in strict mode without weakening that workflow.

PaulB

[Günter Zöchbauer]

is indeed much cleaner to write and read than:

Map<int, String> m = <int, String>{};

However does this not introduce some ambiguity with respect to the spec? For example if I were allowed to write:

var m = <int, String>{};  // [1]

m = new List<Dog>();      // [2]

This is where I expect to get a hint.

I than have to explicitly state that I want a more generic type.

             var m = <int, String>{} as dynamic;

             dynamic m = <int, String>{};

 

with strict analysis of [1] essentially attributing the type Map<int, String> to m, line [2] would generate a strict type hint. I think we're agreed on that point.

But according to the spec, use of var is equivalent to dynamic.  So someone unaware of this special 'strict' analysis rule (but yet subject to it, perhaps as part of a team decision) would perhaps be understandably confused.

but it would work when used in non-strict mode without any hint/warning as required.

 

Don't get me wrong, I would much prefer the shorter versions you all propose. But, acting as devil's advocate, are we getting too close to the spec here?

But in the cases below, type inference would not be done, and hence the user would be required to replace "var" with the desired type:

var x = [1, 2, 3];   // type is ambiguous; it could be List<int>, List<num>, or List<Object>.

either replace 

- var with a type 

- or specify the type for the RHS <num>[1,2,3];

- or as mentioned above infer the type when all elements have the same type (this can be tricky when the values are not primitives (what if it is a list of comparable types or other values that implement more than one interface)

 

Indeed. According to the proposal this would generate a strict type hint because a type annotation is missing on the RHS.

 

var x = {'foo': 'bar'};   // also ambiguous; could be Map<String, String>, Map<String, Object>, etc.

a map without generic type parameters could be treated as Map<String, dynamic>.  I think this is what you get with `new Map();`

[Paul Berry]

On 14 August 2014 07:49, Günter Zöchbauer <gzo...@gmail.com> wrote:

Yes, from the language point of view, var is equivalent to dynamic.  But I don't think that necessarily means we have to treat it as equivalent for the purposes of the type checking in strict mode.

I would also be willing to consider if, for the purposes of strict mode type checking, we treated:

var i = 5;

as shorthand for:

int i = 5;

However, if we go down that path, I think we'd need to sharply delineate exactly what type inference strict mode will do, and what type inference it won't.  I would propose that we only allow 'var' on a variable declaration when the RHS is a literal or an instance creation expression.  So for example:

var i = 5;   =>   int i = 5;

var s = 'foo';   =>   String s = 'foo';

var d = 5.0;   =>   double d = 5.0;

var c = new Cow();   =>   Cow c = new Cow(); *

var l = <int>[];   =>   List<int> l = <int>[]; *

var m = <int, String>{};   =>   Map<int, String> m = <int, String>{}; *

I would add function/method calls that have a type annotation for the return type

 

Note: the three examples I marked with "*" above are my motivation for suggesting this.  I find these patterns come up frequently, and it really annoying to have to repeat the type when it appears elsewhere on the same line.

But in the cases below, type inference would not be done, and hence the user would be required to replace "var" with the desired type:

var x = [1, 2, 3];   // type is ambiguous; it could be List<int>, List<num>, or List<Object>.

var x = {'foo': 'bar'};   // also ambiguous; could be Map<String, String>, Map<String, Object>, etc.

for these two at least `List` and `Map` could be inferred.

Except that List is short for List<dynamic> and Map is short for Map<dynamic, dynamic>.  I thought we were trying to disallow dynamic everywhere, so automatically inferring a type that includes dynamic as a type parameter seems problematic.

 

When all elements have the same type I would also infer the generic type

if the developer wants a more general type he can change to <num>[1, 2, 3] or <dynamic>[1,2,3]

similar for map 

What would you propose we infer for:

var x = [];

var x = {};

 

 

var x = 3 + 5;   // RHS is neither a literal nor an instance creation expression.

as already mentioned this is a constant expression, these should always be inferred.

 

var x = blah();   // RHS is neither a literal nor an instance creation expression.

when blah() has a type annotation for the return type it should be inferred

 

That's my two cents, anyway.

PaulB

--

[Paul Berry]

On 14 August 2014 08:05, Günter Zöchbauer <gzo...@gmail.com> wrote:

is indeed much cleaner to write and read than:

Map<int, String> m = <int, String>{};

However does this not introduce some ambiguity with respect to the spec? For example if I were allowed to write:

var m = <int, String>{};  // [1]

m = new List<Dog>();      // [2]

This is where I expect to get a hint.

I than have to explicitly state that I want a more generic type.

             var m = <int, String>{} as dynamic;

             dynamic m = <int, String>{};

 

with strict analysis of [1] essentially attributing the type Map<int, String> to m, line [2] would generate a strict type hint. I think we're agreed on that point.

But according to the spec, use of var is equivalent to dynamic.  So someone unaware of this special 'strict' analysis rule (but yet subject to it, perhaps as part of a team decision) would perhaps be understandably confused.

but it would work when used in non-strict mode without any hint/warning as required.

 

Don't get me wrong, I would much prefer the shorter versions you all propose. But, acting as devil's advocate, are we getting too close to the spec here?

But in the cases below, type inference would not be done, and hence the user would be required to replace "var" with the desired type:

var x = [1, 2, 3];   // type is ambiguous; it could be List<int>, List<num>, or List<Object>.

either replace 

- var with a type 

- or specify the type for the RHS <num>[1,2,3];

- or as mentioned above infer the type when all elements have the same type (this can be tricky when the values are not primitives (what if it is a list of comparable types or other values that implement more than one interface)

 

Indeed. According to the proposal this would generate a strict type hint because a type annotation is missing on the RHS.

 

var x = {'foo': 'bar'};   // also ambiguous; could be Map<String, String>, Map<String, Object>, etc.

a map without generic type parameters could be treated as Map<String, dynamic>.  I think this is what you get with `new Map();`

Actually, new Map() gives you Map<dynamic, dynamic>.  Unlike Javascript, Dart does not require map keys to be strings. 

 

 

Ditto.

 

var x = 3 + 5;   // RHS is neither a literal nor an instance creation expression.

Is the RHS not a compile time constant here?

 

var x = blah();   // RHS is neither a literal nor an instance creation expression.

But if blah() has a return type annotation this is surely permissible? 

--

[Günter Zöchbauer]

On Thursday, August 14, 2014 5:06:45 PM UTC+2, Paul Berry wrote:

On 14 August 2014 07:49, Günter Zöchbauer <gzo...@gmail.com> wrote:

Yes, from the language point of view, var is equivalent to dynamic.  But I don't think that necessarily means we have to treat it as equivalent for the purposes of the type checking in strict mode.

I would also be willing to consider if, for the purposes of strict mode type checking, we treated:

var i = 5;

as shorthand for:

int i = 5;

However, if we go down that path, I think we'd need to sharply delineate exactly what type inference strict mode will do, and what type inference it won't.  I would propose that we only allow 'var' on a variable declaration when the RHS is a literal or an instance creation expression.  So for example:

var i = 5;   =>   int i = 5;

var s = 'foo';   =>   String s = 'foo';

var d = 5.0;   =>   double d = 5.0;

var c = new Cow();   =>   Cow c = new Cow(); *

var l = <int>[];   =>   List<int> l = <int>[]; *

var m = <int, String>{};   =>   Map<int, String> m = <int, String>{}; *

I would add function/method calls that have a type annotation for the return type

 

Note: the three examples I marked with "*" above are my motivation for suggesting this.  I find these patterns come up frequently, and it really annoying to have to repeat the type when it appears elsewhere on the same line.

But in the cases below, type inference would not be done, and hence the user would be required to replace "var" with the desired type:

var x = [1, 2, 3];   // type is ambiguous; it could be List<int>, List<num>, or List<Object>.

var x = {'foo': 'bar'};   // also ambiguous; could be Map<String, String>, Map<String, Object>, etc.

for these two at least `List` and `Map` could be inferred.

Except that List is short for List<dynamic> and Map is short for Map<dynamic, dynamic>.  I thought we were trying to disallow dynamic everywhere, so automatically inferring a type that includes dynamic as a type parameter seems problematic.

 

I wouldn't disallow dynamic. 

I would only produce a hint when a value is used in a way that can not be verified using the provided type annotations or inferred types.

for 

var x = {'foo': 'bar'}; 

I would expect a hint when I do

x["foo"].moo();

because it's not known if x["foo"] provides this method

Whether x["foo"] should already produce a hint depends on what type exactly should be derived from {'foo': 'bar'}

Maybe strict should require generic type annotations for instance creation.

[Günter Zöchbauer]

I haven't decided how I feel about this.  One of the benefits, IMHO, of strict mode is that it allows me to change a method in my code and be confident that warnings and hints will lead me to all of the other code that's potentially affected by that change.  For example, let's say I change the return type of blah().  Under Paul Jolly's definition of strict mode, nearly* all uses of blah() would immediately be flagged, and then I could go through them one by one to see if they merely needed to have their type annotation changed or if they needed to be more extensively reworked.  However, if we allow "var x = blah();" to do type inference based on the return type of blah(), then I'd lose that benefit; I would only get a warning/hint if some later code happened to use x in a way that conflicts with the new type.  Which means I would miss out on some important opportunities to avoid making mistakes.  Also, any warnings/hints I did get would be located at the site where x was used rather than at the site of the call to blah(), so it would require more mental effort to figure out what needed to be changed.

*Of course, some uses of blah() would still not be flagged, e.g. 'blah().foo' would only be flagged if the new type lacked a member called 'foo'.

This sort of process (make a change and then chase down warnings/hints to make sure I've addressed all affected code) was a really useful part of my workflow back when I was programming in C++ and C#, and I've been frustrated by the fact that I can't do this in Dart.  So that's the reason I didn't want to do type inference on 'var x = blah();'--I was looking for a way to reduce the amount of redundant typing I'd have to do in strict mode without weakening that workflow.

PaulB

If the callers use the returned value according to the contract (only access methods/getters/setters this type actually provides) I wouldn't want to modify the code on the each callers site just because I changed the return type to some more generic or more restrictive type.

Only when the caller depends on methods the new type doesn't provide would I want to get a hint.

[Alex Tatumizer]

How about keeping things simple, and relying on inferred type whenever variable is initialized?

That is,

1. var x=0;

always equivalent in strict mode to

int x=0;

2. var x=[1,2,3];

is equivalent to

List<dynamic> x=[1,2,3]

Etc.

Rule SE1 should be fixed by excluding these cases.

BTW, at least to my taste, this is better than golang's special syntax with :=
 

[Paul Jolly]

But according to the spec, use of var is equivalent to dynamic.  So someone unaware of this special 'strict' analysis rule (but yet subject to it, perhaps as part of a team decision) would perhaps be understandably confused.

Don't get me wrong, I would much prefer the shorter versions you all propose. But, acting as devil's advocate, are we getting too close to the spec here?

I'll grant you that someone might certainly find this confusing.  In fact I'm glad you brought that up because I think it's a good way to think about the tradeoffs involved in my proposal: I'm proposing a change that allows the user to reduce annoying redundancy at the expense of some additional risk of confusion.  IMHO it's worth it, but I certainly understand if someone else feels otherwise :)

I too would prefer to reduce annoying redundancy so we're agreed on that. So let's leave aside the devil's advocate argument for now.

 

But in the cases below, type inference would not be done, and hence the user would be required to replace "var" with the desired type:

var x = [1, 2, 3];   // type is ambiguous; it could be List<int>, List<num>, or List<Object>.

Indeed. According to the proposal this would generate a strict type hint because a type annotation is missing on the RHS.

 

var x = {'foo': 'bar'};   // also ambiguous; could be Map<String, String>, Map<String, Object>, etc.

Ditto.

 

var x = 3 + 5;   // RHS is neither a literal nor an instance creation expression.

Is the RHS not a compile time constant here?

Correct.  But my proposal was to do type inference only for literals and instance creation expressions, not necessarily for all compile time constants.

You might be able to talk me into allowing type inference for compile time constants as well, but the downside is that it forces more complexity on analysis (there are some odd corner cases in the evaluation of compile time constants in the analyzer today, and I worry that allowing types to be inferred from compile time constants would make those corner cases even worse).  My bet is that this sort of declaration occurs rarely enough in practice that it wouldn't really benefit enough people to be worth the added complexity.

Dealing with compile time constants could be a future enhancement as opposed to day 1 requirement if the implementation were sufficiently more complex? 

 

var x = blah();   // RHS is neither a literal nor an instance creation expression.

But if blah() has a return type annotation this is surely permissible? 

I haven't decided how I feel about this.  One of the benefits, IMHO, of strict mode is that it allows me to change a method in my code and be confident that warnings and hints will lead me to all of the other code that's potentially affected by that change.  For example, let's say I change the return type of blah().  Under Paul Jolly's definition of strict mode, nearly* all uses of blah() would immediately be flagged, and then I could go through them one by one to see if they merely needed to have their type annotation changed or if they needed to be more extensively reworked.  However, if we allow "var x = blah();" to do type inference based on the return type of blah(), then I'd lose that benefit; I would only get a warning/hint if some later code happened to use x in a way that conflicts with the new type.  Which means I would miss out on some important opportunities to avoid making mistakes.  Also, any warnings/hints I did get would be located at the site where x was used rather than at the site of the call to blah(), so it would require more mental effort to figure out what needed to be changed.

*Of course, some uses of blah() would still not be flagged, e.g. 'blah().foo' would only be flagged if the new type lacked a member called 'foo'.

Looking at this the other way round, if we were to not permit this then we would need to write:

MyReallyReallyReallyReallyReallyLongClassName x = blah();

This is not a counter argument to your point about wanting to avoid mistakes, rather just looking to make the implications clear. 

 

This sort of process (make a change and then chase down warnings/hints to make sure I've addressed all affected code) was a really useful part of my workflow back when I was programming in C++ and C#, and I've been frustrated by the fact that I can't do this in Dart.  So that's the reason I didn't want to do type inference on 'var x = blah();'--I was looking for a way to reduce the amount of redundant typing I'd have to do in strict mode without weakening that workflow.

It's a good point. I'm going to think a bit more about this. 

[Paul Jolly]

Note: the three examples I marked with "*" above are my motivation for suggesting this.  I find these patterns come up frequently, and it really annoying to have to repeat the type when it appears elsewhere on the same line.

But in the cases below, type inference would not be done, and hence the user would be required to replace "var" with the desired type:

var x = [1, 2, 3];   // type is ambiguous; it could be List<int>, List<num>, or List<Object>.

var x = {'foo': 'bar'};   // also ambiguous; could be Map<String, String>, Map<String, Object>, etc.

for these two at least `List` and `Map` could be inferred.

Except that List is short for List<dynamic> and Map is short for Map<dynamic, dynamic>.  I thought we were trying to disallow dynamic everywhere, so automatically inferring a type that includes dynamic as a type parameter seems problematic.

 

I wouldn't disallow dynamic. 

To be clear on why I propose we disallow dynamic. dynamic is referred to in the spec as "the unknown type". To my mind this could easily lead to confusion if permitted in strict analysis; after all the aim here is to make clearer what types are being passed around.

If you want a mixed list, one can use Object which represents the root of the type hierarchy:

var x = <Object> [1, 2, 3];

 

I would only produce a hint when a value is used in a way that can not be verified using the provided type annotations or inferred types.

for 

var x = {'foo': 'bar'};

The proposal (in this thread) to permit the omitting of the type annotation for the LHS during variable declaration does not extend to allowing the RHS to omit the type annotation. 

Hence:

var x = {'foo': 'bar'};

would generate a strict type hint because the RHS omits the type annotation.

 

I would expect a hint when I do

x["foo"].moo();

because it's not known if x["foo"] provides this method

See above.

 

Whether x["foo"] should already produce a hint depends on what type exactly should be derived from {'foo': 'bar'}

And that's the point. That, in the absence of a type annotation on the RHS, there is ambiguity regarding the type. Hence why the RHS type annotation should not be optional in 'strict' analysis.

 

Maybe strict should require generic type annotations for instance creation.

It does already in SE1 by requiring the RHS to have a type annotation.

[Paul Jolly]

How about keeping things simple, and relying on inferred type whenever variable is initialized?

That is,

1. var x=0;

always equivalent in strict mode to

int x=0;

Yes, agreed, this is where a proposal on this thread is leading us. I'm going to write up something more formal to add to the proposal.

 

2. var x=[1,2,3];

is equivalent to

List<dynamic> x=[1,2,3]

See my most recent reply to Günter.

I'm proposing that we do not permit dynamic for strict analysis; Object should be used where required.

The proposal on this thread does not extend to permitting the type annotation on the RHS to be omitted. Hence you would need to be explicit:

var x= <Object> [1,2,3]; // OR

var x= <int> [1,2,3];

 

Etc.

Rule SE1 should be fixed by excluding these cases.

As mentioned above, the proposal is to loosen SE1 to allow the LHS to omit a type annotation during variable declaration. 

 

BTW, at least to my taste, this is better than golang's special syntax with :=

We should start a different thread for that discussion :) 

[Alex Tatumizer]

> I'm proposing that we do not permit dynamic for strict analysis; Object should be used where required

I would agree with that if instead of Object you require *explicit* dynamic.

That is

var x=[1,2,3]; // produces a warning

var y=<dynamic>[1,2,3]; // no warning

I have a feeling that Object as universal substitute is persona non grata in dart, so your proposal will have greater chance to succeed if you don't insist on Object.

[Paul Berry]

I have been wondering about this too.

On 14 August 2014 10:15, Alex Tatumizer <tatu...@gmail.com> wrote:

> I'm proposing that we do not permit dynamic for strict analysis; Object should be used where required

I would agree with that if instead of Object you require *explicit* dynamic.

That is

var x=[1,2,3]; // produces a warning

var y=<dynamic>[1,2,3]; // no warning

I have a feeling that Object as universal substitute is persona not grata in dart, so your proposal will have greater chance to succeed if you don't insist on Object.

[Paul Jolly]

I have been wondering about this too.

On 14 August 2014 10:15, Alex Tatumizer <tatu...@gmail.com> wrote:

> I'm proposing that we do not permit dynamic for strict analysis; Object should be used where required

I would agree with that if instead of Object you require *explicit* dynamic.

That is

var x=[1,2,3]; // produces a warning

var y=<dynamic>[1,2,3]; // no warning

I have a feeling that Object as universal substitute is persona not grata in dart, so your proposal will have greater chance to succeed if you don't insist on Object. 

Certainly happy to defer on this point - I am relatively new to Dart. 

Also if you feel things might be more readily adopted/understand by leaving it in then all the better.

Critically (off the top of my head), I don't think allowing dynamic as a type annotation changes the semantics of what's proposed, i.e. a dynamic type annotation can appear anywhere that Object can appear. Unless I'm missing a corner case?

So very happy to modify the proposal to reflect this and the optionality of the LHS type annotations for variable declarations.

[Alex Tatumizer]

Concerning "dynamic" vs "Object" - the official position is something like this: Object tells you that it can be ANY type.

Dynamic doesn't carry this "any" meaning. It's often used when you mean it can be either Foo or Bar, and nothing else. This is not equivalent to saying that it can be just anything. There's a subtle distinction.

On top of that, there's one thing that nobody says aloud, but at least for me, "dynamic" *looks* better than Object.

[Paul Jolly]

Concerning "dynamic" vs "Object" - the official position is something like this: Object tells you that it can be ANY type.

Yes, I'm clear on that, hence why I was in favour of adopting it over dynamic, particularly when the latter is referred to in the spec as being "the unknown type." My worry was that this would have unhelpful connotations as far as trying to explain 'strict' analysis was concerned.

 

Dynamic doesn't carry this "any" meaning. It's often used when you mean it can be either Foo or Bar, and nothing else. This is not equivalent to saying that it can be just anything. There's a subtle distinction.

Not sure what you mean by "...you mean can be either Foo or Bar, and nothing else"? Can you give an example?

[Justin Fagnani]

On Thu, Aug 14, 2014 at 10:39 AM, Paul Jolly <pa...@myitcv.org.uk> wrote:

I have been wondering about this too.

On 14 August 2014 10:15, Alex Tatumizer <tatu...@gmail.com> wrote:

> I'm proposing that we do not permit dynamic for strict analysis; Object should be used where required

I would agree with that if instead of Object you require *explicit* dynamic.

That is

var x=[1,2,3]; // produces a warning

var y=<dynamic>[1,2,3]; // no warning

I have a feeling that Object as universal substitute is persona not grata in dart, so your proposal will have greater chance to succeed if you don't insist on Object. 

Certainly happy to defer on this point - I am relatively new to Dart. 

Also if you feel things might be more readily adopted/understand by leaving it in then all the better.

Critically (off the top of my head), I don't think allowing dynamic as a type annotation changes the semantics of what's proposed, i.e. a dynamic type annotation can appear anywhere that Object can appear. Unless I'm missing a corner case?

Object means that the only thing you know is that it's an Object - the tools warn on any call that's not defined in Object.

dynamic means that what you know isn't describable by the type system - the tools will not warn about any calls.

So very happy to modify the proposal to reflect this and the optionality of the LHS type annotations for variable declarations.

--

[Paul Jolly]

Certainly happy to defer on this point - I am relatively new to Dart. 

Also if you feel things might be more readily adopted/understand by leaving it in then all the better.

Critically (off the top of my head), I don't think allowing dynamic as a type annotation changes the semantics of what's proposed, i.e. a dynamic type annotation can appear anywhere that Object can appear. Unless I'm missing a corner case?

Object means that the only thing you know is that it's an Object - the tools warn on any call that's not defined in Object.

dynamic means that what you know isn't describable by the type system - the tools will not warn about any calls.

As of dart1.6.0-dev.8.0, I see the following:

 

What's the distinction between the grey and yellow listings in the 'Problems' pane?

Given your comments Justin, I'm guessing the yellow entry is a static warning per the spec but the grey entry is the result of some additional, non-spec specified analysis and hence is a hint or some such?

[Brian Wilkerson]

Paul,

What's the distinction between the grey and yellow listings in the 'Problems' pane?

Given your comments Justin, I'm guessing the yellow entry is a static warning per the spec but the grey entry is the result of some additional, non-spec specified analysis and hence is a hint or some such?

Correct. The yellow is a warning (static or static type). The grey is a hint based on type propagation.

Brian

[Justin Fagnani]

Exactly 

[Alex Tatumizer]

> Object means that the only thing you know is that it's an Object - the tools warn on any call that's not defined in Object.

> dynamic means that what you know isn't describable by the type system - the tools will not warn about any calls.

Oops, my explanation was incorrect then.

Which means that in the context of proposal, requirement should be that type for things like

var x=[1,2,3]

should be explicitly written as

var x=<dynamic>[1,2,3],

and <Object>[1,2,3] will be a mistake as long as you want to treat these ints as ints (as opposed to just Object).

I am not sure though it really makes much sense to force adding <dynamic>. It's dynamic by default. Let it be inferred as default as it is now.

For initialized variables, we can simply say that everything in strict mode behaves as if variable was explicitly declared with inferred type. I think we discussed that at some point, with inconclusive results (there was even some proposal to generate explicit definitions based on inferred ones and add them to the source, which IMO would be an overkill).

[Alex Tatumizer]

... Which reduces the proposal to a single line: "in strict mode, inferred type works like explicit type declaration". No?

[Bob Nystrom]

On Thu, Aug 14, 2014 at 7:44 AM, Paul Jolly <pa...@myitcv.org.uk> wrote:

Would the packages shipped by the Dart team be updated to comply with strict mode.

We would need someone from Google/Dart team to comment on that: I don't fall into either category!

The Dart style guide explicitly discourages type annotating local variables, so it's unlikely that we'd be changing that.

I don't personally see any value in adding more type annotations to my code. As far as I'm concerned a smarter type checker should do more work for me, not make me do more work for it.

- bob

[Justin Fagnani]

I don't see much value here for myself either. The analyzer already does type inference to offer help in places where this document asserts that the tools can't - it gives hints, suggests code completions, and performs refactoring based on inferred types. Also, some of the code in the document doesn't give a static warning, but doesn't give a runtime error either. The soundness of the type system was relaxed (as I understand it) exactly because that happens and we don't want to force casts.

What I personally want out of analysis tools seems to be a bit different than what I've seen suggested so far: mainly enforcing or materializing what type inferencing already determines. I want the analyzer to tell me when it's having problems inferring types, so that I know to either refactor my code or help out by adding select type annotations (or file a bug on the analyzer!). I specifically don't want to add types everywhere.

[Alex Tatumizer]

>  I want the analyzer to tell me when it's having problems inferring types

Any example of situation where analyzer has difficulty inferring types right now (excluding cases of uninitialized var)?

[Günter Zöchbauer]

On Thursday, August 14, 2014 10:01:44 PM UTC+2, Justin Fagnani wrote:

I don't see much value here for myself either. The analyzer already does type inference to offer help in places where this document asserts that the tools can't - it gives hints, suggests code completions, and performs refactoring based on inferred types. Also, some of the code in the document doesn't give a static warning, but doesn't give a runtime error either. The soundness of the type system was relaxed (as I understand it) exactly because that happens and we don't want to force casts.

What I personally want out of analysis tools seems to be a bit different than what I've seen suggested so far: mainly enforcing or materializing what type inferencing already determines. I want the analyzer to tell me when it's having problems inferring types, so that I know to either refactor my code or help out by adding select type annotations (or file a bug on the analyzer!). I specifically don't want to add types everywhere.

I guess this is exactly what I want too.

I hate it when autocompletion won't pop up on ctrl+space. The analyzer should show where it can't infer a type so I know that an additional type annotation might be necessary to allow the analyzer to support me.

[Justin Fagnani]

You can trigger it very easily with any method where the analyzer can't see the call site. Take Polymer's xChanged() or event handler methods, closures used as callbacks. Here's an example:

f(g) => g('hello');

main() { 

  f((s) => print(s.toUppercase()));

}

It'd be very nice if the analyzer could infer that s is a String, but until then if I add the type annotation I'll see that I had a typo.

On Thu, Aug 14, 2014 at 1:19 PM, Alex Tatumizer <tatu...@gmail.com> wrote:

>  I want the analyzer to tell me when it's having problems inferring types

Any example of situation where analyzer has difficulty inferring types right now (excluding cases of uninitialized var)?

[Alex Tatumizer]

Your example fits in a (broad) definition of "variable not initialized at the point of declaration". "s" is not initialized as formal parameter. Had it had a default (e.g. being on optional with non-null default), compiler could infer type.

Another common category  function that doesn't specify return type.

Regardless of how smart analyzer is, these 2 categories probably account for the bulk of type inference problems.

I agree that having some hint would help a lot.

 

[Filipe Morgado]

+1

[Arron Washington]

This, too, is what I desire.

[Lasse R.H. Nielsen]

On Thu, Aug 14, 2014 at 4:41 PM, Günter Zöchbauer <gzo...@gmail.com> wrote:
>
> This was in relation to the topic mentioned below (CI/dependencies).

> Would the packages shipped by the Dart team be updated to comply with strict
> mode.

I don't speak for the Dart team, but personally I'd not use strict
mode as defined here, and I would argue against making it policy. It's
too strict.

Downcasting is used deliberately when we know which subclass an object
belongs to, in order to give it a name. This isn't used as much as
before we got type inference from is-checks, but it is still needed in
some cases, e.g., when casting to an interface, or when casting
depending on a test other than an is-test:
if (foo.isSpecial) {
SpecialFoo special = foo;
...
}

We also use dynamic as return type deliberately.
There are functions where the type system can't express the actual
type of a function, like Iterable.reduce, or JSON.parse. That is
usually a case where the user *do* know the return type.
We could return Object instead (and in some other cases we decide to
do that instead), but for *usability* we allow you to write:
print(JSON.parse('["a","b","c"]').join(""));
This would probably go away if we had generic functions, but we don't.

I personally tend to add type annotations everywhere, even on local
variables, because I rely on language warnings to catch my typos (but
a go-like "x:=foo;" declaration would be great!).

/L
--
Lasse R.H. Nielsen - l...@google.com
'Faith without judgement merely degrades the spirit divine'
Google Denmark ApS - Frederiksborggade 20B, 1 sal - 1360 København K -
Denmark - CVR nr. 28 86 69 84

[Günter Zöchbauer]

I think so too.

[Matthew Butler]

On Thursday, August 14, 2014 4:38:32 PM UTC-3, Bob Nystrom wrote:

The Dart style guide explicitly discourages type annotating local variables, so it's unlikely that we'd be changing that.

I don't personally see any value in adding more type annotations to my code. As far as I'm concerned a smarter type checker should do more work for me, not make me do more work for it.

- bob

+1 

[Paul Jolly]

The Dart style guide explicitly discourages type annotating local variables, so it's unlikely that we'd be changing that.

Interesting. What tools does Google use for checking/linting Dart code against the style guide? Not necessarily enforcing, but at least checking to see what 'violations' exist.

[Paul Jolly]

I don't see much value here for myself either. The analyzer already does type inference to offer help in places where this document asserts that the tools can't - it gives hints, suggests code completions, and performs refactoring based on inferred types. Also, some of the code in the document doesn't give a static warning, but doesn't give a runtime error either. The soundness of the type system was relaxed (as I understand it) exactly because that happens and we don't want to force casts.

What I personally want out of analysis tools seems to be a bit different than what I've seen suggested so far: mainly enforcing or materializing what type inferencing already determines. I want the analyzer to tell me when it's having problems inferring types, so that I know to either refactor my code or help out by adding select type annotations (or file a bug on the analyzer!). I specifically don't want to add types everywhere.

This is an interesting thought. The reason I started down the 'strict' proposal route was to try and avoid a situation where the analyzer would get into such a situation. Hence SE1 (with a nod to the more relaxed version of this rule proposed earlier in this thread). But I concede that if a hint could be generated in such a situation then this brings your suggestion and the proposal a bit closer together (at least as far as SE2 and the more relaxed SE1 are concerned - ignore invariant generics for now). 

Just so that I'm clear, would you suggestion also throw up a hint on the line comment with [1] below?

class A {

  String doA() => "test";

}

class B extends A {

  int doB() => 5;

}

void main() {

  var a = new A();

  var b = new B();

  b.doB();

  b = a;

  b.doB();  // [1] 

}

No static warning/hint is given at the moment.

[Paul Jolly]

> This was in relation to the topic mentioned below (CI/dependencies).
> Would the packages shipped by the Dart team be updated to comply with strict
> mode.

I don't speak for the Dart team, but personally I'd not use strict
mode as defined here, and I would argue against making it policy. It's
too strict.

Thanks for the feedback. This is and remains only a proposal so very much designed to be knocked over as a straw man. Certainly not advocating we make it policy!

 

I personally tend to add type annotations everywhere, even on local
variables, because I rely on language warnings to catch my typos (but
a go-like "x:=foo;" declaration would be great!).

I do the same. Hence my kicking off this proposal.

Do you have a tool to help check for the absence of type annotations?

How do you handle situations where you use packages that don't use type annotations?

[Lasse R.H. Nielsen]

On Fri, Aug 15, 2014 at 3:20 PM, Paul Jolly <pa...@myitcv.org.uk> wrote:

>> I personally tend to add type annotations everywhere, even on local
>> variables, because I rely on language warnings to catch my typos (but
>> a go-like "x:=foo;" declaration would be great!).
>
>
> I do the same. Hence my kicking off this proposal.
>
> Do you have a tool to help check for the absence of type annotations?

No. Missing annotations kind of stand out to me, to the point where I
preferred writing "dynamic" to not writing a return type (but that was
too far from the style guide, I've promised not to do that again :)

> How do you handle situations where you use packages that don't use type
> annotations?

I'm probably uncharacteristic here because I write mainly in the core
libraries, which can't use any packages, or on packages built just on
top of the core library.

If I use a package without type annotations, I'll 1) curse a lot, and
2) assign return values to variables with types, so I can see what I'm
doing. Most of the time it isn't really a problem - the interface
between my code and another package's code is likely to be small and
clearly abstracted. Meaning that I also 3) abstract any foreign code
into as few of my own functions as possible, with appropriate names
from my domain, and with types :)

[Alex Tatumizer]

> Meaning that I also 3) abstract any foreign code into as few of my own functions as possible, with appropriate names
> from my domain, and with types :)

This is a good idea regardless of the language. I do the same in java.

I had a sudden revelation upon reading this thread: there can be a secret plan in dart (which was devised by the language itself, not necessarily by people :-) to make type inference a centerpiece of type system.

Consider:

foo(a) {

   if (a is String) {

      // do something with String

   } else if (a is int) {

     // do something with int

   } else {

      throw ("what?");

   }

}

Analyzer can be made smart enough to recognize that parameter is either String or int, and if you write foo(3.14), it will warn about wrong type. However, it's impossible to define type of "a" as (String | int) because there's no support for this syntax - probably, out of principle, just because same can be achieved by analyzing the code. I have no way to know whether my speculation is correct, but it might be.

[Bob Nystrom]

On Fri, Aug 15, 2014 at 12:58 AM, 'Lasse R.H. Nielsen' via Dart Misc <mi...@dartlang.org> wrote:

I personally tend to add type annotations everywhere, even on local
variables, because I rely on language warnings to catch my typos (but
a go-like "x:=foo;" declaration would be great!).

Hints in the analyzer have come a long way. If you haven't relied on them in a while, you may be surprised how often "var" works just as well for this.

I personally tend to write lots of mistakes in my code and the analyzer does a great job of helping me find them even when I don't annotate locals.

- bob

[Bob Nystrom]

On Fri, Aug 15, 2014 at 5:54 AM, Paul Jolly <pa...@myitcv.org.uk> wrote:

What tools does Google use for checking/linting Dart code against the style guide? Not necessarily enforcing, but at least checking to see what 'violations' exist.

We don't have any tooling for it beyond the in-progress formatter. We'd like to, but it hasn't been a high enough priority to put resources on it.

Cheers!

- bob

[Bob Nystrom]

On Fri, Aug 15, 2014 at 6:16 AM, Paul Jolly <pa...@myitcv.org.uk> wrote:

class A {

  String doA() => "test";

}

class B extends A {

  int doB() => 5;

}

void main() {

  var a = new A();

  var b = new B();

  b.doB();

  b = a;

  b.doB();  // [1] 

}

No static warning/hint is given at the moment.

Personally, I think static analysis could be better here. It's possible in many cases to do simple flow analysis and handle variables changing their type and would handle this.

In practice, though, users rarely change the type of value assigned to a variable like you do here.

- bob

[Justin Fagnani]

On Fri, Aug 15, 2014 at 7:12 AM, 'Lasse R.H. Nielsen' via Dart Misc <mi...@dartlang.org> wrote:

On Fri, Aug 15, 2014 at 3:20 PM, Paul Jolly <pa...@myitcv.org.uk> wrote:

>> I personally tend to add type annotations everywhere, even on local
>> variables, because I rely on language warnings to catch my typos (but
>> a go-like "x:=foo;" declaration would be great!).
>
>
> I do the same. Hence my kicking off this proposal.
>
> Do you have a tool to help check for the absence of type annotations?

No. Missing annotations kind of stand out to me, to the point where I
preferred writing "dynamic" to not writing a return type (but that was
too far from the style guide, I've promised not to do that again :)

I haven't. Explicitly declaring the return type, even as void or dynamic, enables the editor to be nice and tell you when you forgot to return a value. I think the style guide needs to evolve with the tools here.

 

> How do you handle situations where you use packages that don't use type
> annotations?

I'm probably uncharacteristic here because I write mainly in the core
libraries, which can't use any packages, or on packages built just on
top of the core library.

If I use a package without type annotations, I'll 1) curse a lot, and
2) assign return values to variables with types, so I can see what I'm
doing. Most of the time it isn't really a problem - the interface
between my code and another package's code is likely to be small and
clearly abstracted. Meaning that I also 3) abstract any foreign code
into as few of my own functions as possible, with appropriate names
from my domain, and with types :)

/L
--
Lasse R.H. Nielsen - l...@google.com
'Faith without judgement merely degrades the spirit divine'
Google Denmark ApS - Frederiksborggade 20B, 1 sal - 1360 København K -
Denmark - CVR nr. 28 86 69 84

[Günter Zöchbauer]

I haven't. Explicitly declaring the return type, even as void or dynamic, enables the editor to be nice and tell you when you forgot to return a value. I think the style guide needs to evolve with the tools here.

 

+1 

The same for type annotations on local variables.

When the analyzer can't infer the type the annotation should be allowed. 

[Brian Wilkerson]

I personally tend to add type annotations everywhere, even on local
variables, because I rely on language warnings to catch my typos (but
a go-like "x:=foo;" declaration would be great!).

Hints in the analyzer have come a long way. If you haven't relied on them in a while, you may be surprised how often "var" works just as well for this.

I personally tend to write lots of mistakes in my code and the analyzer does a great job of helping me find them even when I don't annotate locals.

One of the things analyzer can't do is tell the difference between a dynamic variable that is expected to change its type and one that is not. Similar to the previous example that was given:

main() {

  var a = 'One';

  a = 2; // Is this intentional?

}

Analyzer can't know whether whether the commented line is a problem. Providing a type annotation would solve that, but at a cost.

Brian

[Günter Zöchbauer]

Usually it works fine (for the simple cases) but the cases where it doesn't are more likely to contain the bugs.

Its nice that the analyzer provides hints here and there but one can't rely on them.

If the hints are not provided everywhere its often worse than if the analyzer wouldn't exist at all (false security).

The analyzer doesn't have to be perfect but it should at least tell where it needs support (type annotations) to be able to do its work in a reliable way.

[Bob Nystrom]

On Fri, Aug 15, 2014 at 9:42 AM, 'Justin Fagnani' via Dart Misc <mi...@dartlang.org> wrote:

I haven't. Explicitly declaring the return type, even as void or dynamic, enables the editor to be nice and tell you when you forgot to return a value. I think the style guide needs to evolve with the tools here.

The style guide certainly does tell you to annotate "void" if your function is public.

I'm agnostic about "dynamic". I'm personally fine with it because it clarifies "this function returns a dynamic type" from "I forgot to annotate this function", but others pushed for a more terse style. The consensus we reached was the latter.

- bob

[Bob Nystrom]

On Fri, Aug 15, 2014 at 9:49 AM, 'Brian Wilkerson' via Dart Misc <mi...@dartlang.org> wrote:

One of the things analyzer can't do is tell the difference between a dynamic variable that is expected to change its type and one that is not. Similar to the previous example that was given:

main() {

  var a = 'One';

  a = 2; // Is this intentional?

}

Analyzer can't know whether whether the commented line is a problem. Providing a type annotation would solve that, but at a cost.

Since Dart is a dynamically typed language and variables can change type, I think the analyzer pretty much has to allow the above code. Instead, I really like what it does now:

    var a = 'One';

    a = 2;

    a.length; // Warning, int has no length getter.

- bob

[Alex Tatumizer]

This example shows that type inference cannot be trusted. This is a major drawback.

Bob, when you see the line var a = 'One', what is your best guess: is this a String, or we don't know? In 99.9% cases, it's a String, and never be assigned anything else. However, remaining 0.1% is treated as default by a language. There should a way to utilize type inference, without the need to explicitly declare String a = 'One'

[Bob Nystrom]

On Fri, Aug 15, 2014 at 10:49 AM, Alex Tatumizer <tatu...@gmail.com> wrote:

This example shows that type inference cannot be trusted. This is a major drawback.

I don't know if my example shows what you think it shows:

    var a = 'One';

    a.length; // No warning here, a is a String.

    a = 2;

    a.length; // Warning, int has no length getter.

 

Bob, when you see the line var a = 'One', what is your best guess: is this a String, or we don't know?

When I see that line, I'm certain it's a string. And when I see the line "a = 2", now I'm certain it's an int.

Of course, that does mean you have to keep previous assignments in mind when looking at how a variable is used. In practice, code rarely assigns new types to variables (and, in fact, code rarely re-assigns local variables at all), so it usually doesn't require close scrutiny.

- bob

[Alex Tatumizer]

> In practice, code rarely assigns new types to variables

Then why extremely rare case , probably a bug, doesn't cause warnings, until you invoke a.length? And what if length is there, but the type is wrong?

Rare case dominates decisions, I don't think it's the best choice,

[Alex Tatumizer]

To put it more bluntly: what if dart makes it a bit harder to change type? How much will we lose?

In a program like

var b=0;
var a="abc"
...

a = b;

when you really intended to change type, you would probably put a comment:

a = b; // not-a-bug!

This comment (not-a-bug) is for a programmer (maybe yourself); analyzer doesn't understand it. But what if dart introduces some standard way to put such comment, e.g. a =  (some special syntax) b;

Yes, in 0.1% of cases you will have to write more; but in 99.9% of cases you will get a warning you deserve.

An argument can be made that current situation is surprising, while dart, overall, tries hard to minimize surprise.

It's not late to introduce such syntax - nothing will be really broken, it affects warnings only. I think this is a major point of contention, discussions on this mailing list occur periodically.

[Frank Pepermans]

That'd be better than some ugly @suppresswarnings syntax.

It could be like the current int division, var a = 'bla';
a = 1; // nok
a ~= 1; // ok

[Arron Washington]

+1 to what Bob just said. Even in a language like Ruby I've very rarely changed the type of a variable. There's generally not much utility to it and usually the loss of clarity isn't worth it.

--
- AW

[Alex Tatumizer]

Too many subtleties in current concept of types in dart. Consider:

  var obj='123';

  1+obj; // ok - why?

  obj+1;  // ok - why?

  obj.foo; // hint (barely visible) "there's no such getter 'foo' in String"

  '123'.foo; // warning: "there's no such getter 'foo' in String"

  obj as int; // ok - why?

  '123' as int; // ok, too. 

I find this all quite perplexing. Same warning is sometimes a hint, sometimes a warning; obj.foo is a hint, but obj+1 is not. 

'123'.foo is flagged, but absurd '123' as int - is not.

Because of various subtleties, it's hard to know whether something is a bug in analyzer or not. E.g. obj+1 - should it be flagged? 

I had many situations where I tried to refactor/rename, and these subtleties suddenly started coming up in a major way. Editor insisted on interactive refactoring, which I hate (it's easier to use CTRL/F and rename selectively then) - but I couldn't see the reason; maybe because of vars instead of explicit types - I don't know. And I don't even know if I should complain or not, because I'm confused and don't trust myself.

[Lasse R.H. Nielsen]

On Sat, Aug 16, 2014 at 5:29 AM, Alex Tatumizer <tatu...@gmail.com> wrote:
> Too many subtleties in current concept of types in dart. Consider:
> var obj='123';
> 1+obj; // ok - why?
> obj+1; // ok - why?
> obj.foo; // hint (barely visible) "there's no such getter 'foo' in String"
> '123'.foo; // warning: "there's no such getter 'foo' in String"
> obj as int; // ok - why?
> '123' as int; // ok, too.
>
> I find this all quite perplexing. Same warning is sometimes a hint,
> sometimes a warning; obj.foo is a hint, but obj+1 is not.
> '123'.foo is flagged, but absurd '123' as int - is not.
> Because of various subtleties, it's hard to know whether something is a bug
> in analyzer or not. E.g. obj+1 - should it be flagged?

It probably should be a hint (since it could), but it differs from obj.foo.
For obj.foo, there is no foo getter on the derived type of obj.
For obj+1 there is a + operator on the derived type of obj, but its
argument type is String.
There is no specification for hints, so not giving one isn't an error,
but it could be a feature request.

[Eric Clayberg (Google)]

Errors and warnings are stricting defined by the spec. Omitting any or adding additional ones would be an error in Analyzer. Additional "warnings" based on type propagation are surfaced as hints. The lack of a hint is not a bug, but could certainly be a feature request and should be reported to the Analyzer section of the issue tracker.

 

  var obj='123';

  1+obj; // ok - why?

  obj+1;  // ok - why?

OK from the POV of the spec, but not really OK in practice, so a good place for a new hint. If you type the first line, and then see actual warnings show up, but no corresponding hints when typed as 'var', those are good places for us to add parallel hints.

 

  obj.foo; // hint (barely visible) "there's no such getter 'foo' in String"

When typed as 'var', the spec says no error/warming... thus the hint.

 

  '123'.foo; // warning: "there's no such getter 'foo' in String"

Definitely a warning per the spec.

 

  obj as int; // ok - why?

  '123' as int; // ok, too. 

Those are allowed per the spec. I don't know whether we would even want a hint in those cases.

Because of various subtleties, it's hard to know whether something is a bug in analyzer or not.

This is how we would characterize Analyzer issues as bugs or not:

• Missing error or warning defined in the spec = bug
  
• Extra error or warning not defined in the spec = bug
  
• Inappropriate hint (bad type propagation?) = bug
  
• Missing hint = not a bug / feature request opportunity
  

[Lasse R.H. Nielsen]

On Sat, Aug 16, 2014 at 2:21 PM, Eric Clayberg (Google)
<clay...@google.com> wrote:

>> obj as int; // ok - why?
>> '123' as int; // ok, too.
>
>
> Those are allowed per the spec. I don't know whether we would even want a
> hint in those cases.

It's a guaranteed error, so I think a hint would be fine.
If you really want to always throw a CastError, I'd recommend "throw
new CastError()" instead.

[Alex Tatumizer]

I know, warnings are defined in the spec, and hints are not. The problem, however, is that no one (except maybe Gilad) knows spec by heart, and things are not very intuitive to guess what should be there. and whether something is a bug in analyzer, or bug in spec, or in the program, or in the head.

E.g. yesterday, we discussed the type of var o="abc", and Bob and myself come to consensus (which is rare, because normally our views are diametrically opposite) that this must be a String.

However, when we write

var o="abc";

o.foo;

we get a hint. But when you write

String o="abc";

o.foo;

we get a warning.

I know, this is all in full compliance with the spec, so I don't claim this is a bug in conventional sense.

But: if var o="abc" is CERTAINLY a String, then o.foo is CERTAINLY a bug. What difference does it make from user's viewpoint that something is defined in the spec, and something is not?

Maybe spec should provide a loophole for analyzer allowing warnings in places where bugs are obvious?

But the root cause IMO is this: spec pays not enough attention to type inference, which could be promoted much higher. The problems above are due to the fact that type inference is not trusted by the spec, so var o='foo', though obviously String, is treated rather as kind of "maybe String"., 

[Lasse R.H. Nielsen]

On Sat, Aug 16, 2014 at 4:47 PM, Alex Tatumizer <tatu...@gmail.com> wrote:
> I know, warnings are defined in the spec, and hints are not. The problem,
> however, is that no one (except maybe Gilad) knows spec by heart, and things
> are not very intuitive to guess what should be there. and whether something
> is a bug in analyzer, or bug in spec, or in the program, or in the head.
>
> E.g. yesterday, we discussed the type of var o="abc", and Bob and myself
> come to consensus (which is rare, because normally our views are
> diametrically opposite) that this must be a String.
> However, when we write
> var o="abc";
> o.foo;
> we get a hint. But when you write
> String o="abc";
> o.foo;
> we get a warning.
> I know, this is all in full compliance with the spec, so I don't claim this
> is a bug in conventional sense.
>
> But: if var o="abc" is CERTAINLY a String, then o.foo is CERTAINLY a bug.
> What difference does it make from user's viewpoint that something is defined
> in the spec, and something is not?
> Maybe spec should provide a loophole for analyzer allowing warnings in
> places where bugs are obvious?

There are (obviously) differing opinions about this.
The designers of Dart wants that someone writing a library without
warnings should not cause warnings for another person importing that
library.
That means that all tools should *agree* on what is a warning, and to
ensure this, the warnings must be specified centrally in the spec.

Needing to specify the warnings exactly puts an upper bound on the
complexity of the static analysis that the spec can use. It could
specify some extremely complex, but precise, analysis, but then nobody
would be able to implement it correctly anyway (or specify it without
making errors).
As a trade-off, the static analysis is fairly simple, and any use of
"var" or "dynamic" turns it completely off for that variable.

The other opinion is that tools should be allowed to give as much help
to the programmer as it possibly can.
This is what makes the analyzer give "hints" (aka. a warning by
another name) when it thinks it can do better than what the spec
requires.
That is basically the loophole you are asking for: you can say
anything as long as you don't call it a "warning".

For the case "o.foo" being certainly a bug, that is only possible to
say if you know exactly the values for "o" that reach this statement.
In general, it requires a value propagation analysis which is more
complex than what the spec wants to do - specifying that correctly is
significantly harder than just saying "the type of o is the declared
type from its declaration".

>
> But the root cause IMO is this: spec pays not enough attention to type
> inference, which could be promoted much higher. The problems above are due
> to the fact that type inference is not trusted by the spec, so var o='foo',
> though obviously String, is treated rather as kind of "maybe String".,

The spec doesn't want to specify a full type inference. Even if it
did, it would just give the same problem if someone else came up with
an even more precise type inference. It had to draw a line somewhere,
and that was at the simple analysis currently in the spec. Some tools
can definitely do better, but that is always the case.

[Brian Wilkerson]

Alex,

I know, warnings are defined in the spec, and hints are not. The problem, however, is that no one (except maybe Gilad) knows spec by heart, and things are not very intuitive to guess what should be there. and whether something is a bug in analyzer, or bug in spec, or in the program, or in the head.

As a member of the team that maintains the analyzer, I would much rather that you assumed it is a bug in the analyzer. I'm happy to pass it along as a bug in the spec if that's the case, or convert it to an enhancement request if that's more appropriate. But without feedback we won't know where we need to improve the tooling.

But: if var o="abc" is CERTAINLY a String, then o.foo is CERTAINLY a bug. What difference does it make from user's viewpoint that something is defined in the spec, and something is not?

It doesn't make any difference.

But the analyzer, rightly enough, must adhere to the specification. So sometimes, as unhelpful as it might be to our users, the only thing I can tell you is why he tool is doing things the way it is.

The language tries to walk a fine line between two sets of users. On the one hand, there are users who want the tools to be able to use all of the power of a type system to help catch bugs during development. On the other hand, there are users who want to be able to write code quickly and don't want the tools to impose unnecessary restrictions or require them to write unnecessary code (for example, see the discussions about having to write a type annotation on the left-hand-side when the variable initializer consists of an instance creation expression).

That's why types are optional. If you want as much static checking as possible, you should fully specify the type of everything. If you want the tools to completely leave you alone, you should not specify the type of anything. The style guide assumes that most users fall in the middle and tries to find a good compromise between the extremes.

But that opens up a hard question (one of the questions being wrestling with on this thread): what should happen when you mix those two styles? The language generally takes the approach that there shouldn't be an error or warning unless it's always going to be a real problem. That implies that some code containing problems will not produce errors or warnings.

Maybe spec should provide a loophole for analyzer allowing warnings in places where bugs are obvious?

As Lasse pointed out, there is a loophole. Hints are warnings that are not included in the specification.

Unfortunately, the set of hints is not nearly as complete as I would like it to be. If it were, I suspect that this discussion would be a lot shorter. But the examples being raised here will feed into decisions about what additional hints to provide, and I'll encourage you again to file feature requests for places where you think there ought to be a hint when there isn't.

I'll also point out that when deciding whether to create a hint we also look at the probability of false positives, and are heavily biased toward not creating hints that will generate false positives. The reason is that when a tool produces too many false positives users tend to ignore the results of that tool, including the real problems that it's reporting. (This is also one of the reasons why the language doesn't produce errors or warnings in places where it might not always be a problem.)

But the root cause IMO is this: spec pays not enough attention to type inference, which could be promoted much higher. The problems above are due to the fact that type inference is not trusted by the spec, so var o='foo', though obviously String, is treated rather as kind of "maybe String".

Lasse pointed out some of the difficulties with defining type promotion in the specification, but even if it were defined there, there would eventually be some place where tools could do a better job of promoting types than by strictly following the specification and we'd be right back where we are today. I think the right approach is to ask how we can present the information in the tools in such a way that it's more helpful and less confusing.

Brian

[Günter Zöchbauer]

The language tries to walk a fine line between two sets of users. On the one hand, there are users who want the tools to be able to use all of the power of a type system to help catch bugs during development. On the other hand, there are users who want to be able to write code quickly and don't want the tools to impose unnecessary restrictions or require them to write unnecessary code (for example, see the discussions about having to write a type annotation on the left-hand-side when the variable initializer consists of an instance creation expression).

 

I have a hard time to understand why hints would cause troubles for the 2nd group. 

This is not like in Go where an unused import prevents the app to compile. They are just hints.
And when the 2nd group sees a hint in the code they currently care about (and would cause a runtime error when executed anyway) I'm pretty sure they would be glad to see the problem immediately instead of wasting time trying to reproduce the runtime state where this code is actually executed only to see that it fails.
Do you really think that there are people out there which are offended by hints produced by the static analyzer and accept only runtime errors as indication that their code doesn't work?

To satisfy people who want the compilation to halt when the analyzer finds any problem (for example for deployment, CI, ...) it would be sufficient to treat hints and warnings as errors.

In my opinion more in-depth analysis and more hints would improve the experience for everyone.

[Alex Tatumizer]

Lasse, Brian,

I fully understand your points.

I just want to mention one particular issue that didn't receive enough attention: refactoring.

Not sure how refactoring algo works, but it looks like it doesn't fully utilize inferred types either. Which brings it into confrontation with style guide.

Consider refactor/rename. I am not particularly good at choosing right names, so I often need to change them. This proved to be difficult: the tool often required dialog.

Eventually, I came to the same conclusion as Lasse's: type-annotate everything, including locals. The variable can be local, but in terms of refactor/rename, it's very much global,  

I'd like to illustrate what I mean by "global":

class Person {

   String name;

   ...

}  

main () {

   var p=new Person("John");

   print(p.name);

}

You want to rename "name" to "lastName". If tool fails to recognize right type of p (no matter how local it is), refactoring will require dialog.

This particular example works fine with current refactoring implementation, but I had ones that didn't (can't remember particular scenarios)

.

Filing a bug would be easier if refactoring tool printed an explanation as to WHY it is in doubt and wants a dialog. Then I could say exactly whose fault it is.

Otherwise, user, after encountering a couple of problems here and there, out of desperation decides to use type annotations everywhere, even where they are redundant, and never files any bugs.

Additionally, a good experiment might be to take pub (written reportedly in accordance with style guide) and try to randomly rename just everything and see whether it works and where it breaks.

[Brian Wilkerson]

Günter,

... On the other hand, there are users who want to be able to write code quickly and don't want the tools to impose unnecessary restrictions or require them to write unnecessary code ...

...

 

Do you really think that there are people out there which are offended by hints produced by the static analyzer and accept only runtime errors as indication that their code doesn't work?

I don't think anyone is offended when the tools provide valid feedback related to problems in the code, but there are plenty of people who are offended when the tools are not as smart as they expect them to be. There was a point in the evolution of the language when the following code produced a warning:

int f(Object x) {

  if (x is String) {

    return x.length; // warning: There is no such getter 'length' in 'Object'

  }

  return 0;

}

Lots of people were offended. The warning is easy to work around (in several ways), but nobody wanted to have to add code to turn off an obviously false warning. The language specification was updated to take this code pattern into account and the warning is no longer produced. But there is a limit to how far the specification can/should be taken, and the following code still produces a warning:

int f(Object x) {

  if (x is! String) {

    return 0;

  }

  return x.length; // warning: There is no such getter 'length' in 'Object'

}

So I think that we need to be careful about adding hints. They should help find problems in the code without becoming a hinderance to developers.

In my opinion more in-depth analysis and more hints would improve the experience for everyone.

I agree, as long as we're careful about the hints we add.

Brian

[Brian Wilkerson]

Alex,

I just want to mention one particular issue that didn't receive enough attention: refactoring.

I did notice that comment in your previous post, but unintentionally neglected to respond to it.

Not sure how refactoring algo works, but it looks like it doesn't fully utilize inferred types either. Which brings it into confrontation with style guide.

...

 

This particular example works fine with current refactoring implementation, but I had ones that didn't (can't remember particular scenarios).

If the example you gave works, then refactoring is taking into account the propagated type information (which matches my understanding of the code).

The problems you're seeing are likely the result of incomplete analysis and should be addressed if they can be. There will always be cases where we cannot infer any type information, but part of why we do type propagation is to make tools like refactoring and code completion better. The next time you run into a case where it couldn't figure out what was going on and should have, please open an issue to let us know.

Filing a bug would be easier if refactoring tool printed an explanation as to WHY it is in doubt and wants a dialog. Then I could say exactly whose fault it is.

I agree. In fact, the question came up in an internal conversation on Friday. I don't know when we would have time to tackle something like this, but it would be fun to do and possibly very helpful to users. (How useful, of course, depends on how good a job we could do at explaining it.)

Additionally, a good experiment might be to take pub (written reportedly in accordance with style guide) and try to randomly rename just everything and see whether it works and where it breaks.

That's an interesting idea.

Brian

[Alex Tatumizer]

Brian,

Though half of my head accepts the arguments in defense of status quo, another half finds the overall result confusing,

Too many subtleties,

E.g. even

final s='abc';

s.foo;

produces just a hint, though it's an obvious bug, it's a string, and it's final, so no false positives will ever occur if we flag it as warning.

To keep it short, I have 2 vaguely defined suggestions (which unfortunately will involve some changes in the spec, but no backward compatibility issues)

1. Promote type inference to a higher position (warnings, not hints in cases where analyzer is 100% sure).

2. Make it harder to change the type of variable dynamically.

I don't know how exactly to formalize these suggestions, the only thing I know is that the status quo is too subtle for average Joe (where I belong myself).

Please give it some consideration, obviously this is not something that can be decided in 5 minutes.

 

[Anders Holmgren]

+1 I'm in this camp. I'm in general disappointed with the refactoring results. Yesterday I changed the name of a class and it only seemed to make the changes in that file. So I ended up opening intellij and using replace in path. I don't tend to file bugs related to refactoring as my expectations have been lowered to the point where I expect it to refactor badly :-(

[Anders Holmgren]

This discussion made me realise that my code style has evolved in the last 6 months to where I no longer put types on all my locals.  That suggests to me that the analyzer has improved substantially in that time so kudos to the team.

Admittedly I always use final rather than var which removes a whole set of the issues discussed on this thread as you can't do

final s = 'One';
s = 2;

I think the biggest source of time wasting left now for me is due to the lack of generic functions. The analyzer doesn't seem to have made any attempts to cover this language deficiency with any sort of hints from what I can tell.

e.g

 final Future<int> foo = new Future.value("One"); // no hint :-(

 final Iterable<int> fum = [1].map((i) => '$i');  // no hint :-(

 final Stream<int> bar = new Stream.fromIterable(['s']);  // no hint :-(

Even including types on locals doesn't help this case. Sadly a lot of my code runs into this as I favour the functional style. This leaves me catching way to many things in unit tests for my liking. And tracking the bugs down is typically tricky

A

[Paul Jolly]

> Do you have a tool to help check for the absence of type annotations?

No. Missing annotations kind of stand out to me, to the point where I
preferred writing "dynamic" to not writing a return type (but that was
too far from the style guide, I've promised not to do that again :)

I'm sure they would probably stand out to me too, but I would prefer a 'tool' to help me verify this. Hence part of the motivation behind the 'strict' proposal.

 

> How do you handle situations where you use packages that don't use type
> annotations?

I'm probably uncharacteristic here because I write mainly in the core
libraries, which can't use any packages, or on packages built just on
top of the core library.

This is a notable point. The situation I'm imaging in my 'strict' proposal is one where I'm using lots of packages written by lots of people. Again, to stress, 'strict' would not be mandated, rather entirely optional and at the user's discretion.

 

If I use a package without type annotations, I'll 1) curse a lot, and
2) assign return values to variables with types, so I can see what I'm
doing. Most of the time it isn't really a problem - the interface
between my code and another package's code is likely to be small and
clearly abstracted. Meaning that I also 3) abstract any foreign code
into as few of my own functions as possible, with appropriate names
from my domain, and with types :)

Thanks - that's also interesting to know. I think your perspective supports my point that (and this is, I grant you, a personal preference possibly shared by others) having type annotations helps me understand code more quickly

[Paul Jolly]

class A {

  String doA() => "test";

}

class B extends A {

  int doB() => 5;

}

void main() {

  var a = new A();

  var b = new B();

  b.doB();

  b = a;

  b.doB();  // [1] 

}

No static warning/hint is given at the moment.

Personally, I think static analysis could be better here. It's possible in many cases to do simple flow analysis and handle variables changing their type and would handle this.

Understood. 

 

In practice, though, users rarely change the type of value assigned to a variable like you do here.

Whilst it may be the case that Dart users rarely change the type of a value assigned to a variable (I don't have any real body of evidence to turn to), if there is nothing stopping them from doing so (as you say, the language allows it) then it could happen. And unless you had a particularly sharp eye you wouldn't spot a check in that does just that. That said, you might be quite happy to permit such a style of coding. Again, lots of flexibility via the language spec.

The thinking behind the 'strict' proposal was not, as I have said, to mandate that everyone adopts it. Rather, it was to express a stricter style of coding in Dart that might appeal to others, in which case they too can adopt it. A style that it is hoped is familiar to those from Java/C# etc backgrounds. It won't be to everyone's taste.

I personally would not want to allow the above and the rules in the 'strict' proposal help enforce that. Certainly not the only way - as you say simple flow analysis could also do the trick. 

[Paul Jolly]

I haven't. Explicitly declaring the return type, even as void or dynamic, enables the editor to be nice and tell you when you forgot to return a value. I think the style guide needs to evolve with the tools here.

 

+1 

The same for type annotations on local variables.

When the analyzer can't infer the type the annotation should be allowed. 

I might be missing something here, but when can the analyzer know that it has been unsuccessful in inferring a type? 

Surely the spec precludes there being such a situation?

And we have to include dynamic here because the spec is specific on when dynamic is inferred. For example if I write var i = 2 then the analyzer can't tell me that's wrong (even with a hint) because I could well have intended the type of i to be dynamic.

What examples do you have in mind? 

[Paul Jolly]

I personally tend to write lots of mistakes in my code and the analyzer does a great job of helping me find them even when I don't annotate locals.

One of the things analyzer can't do is tell the difference between a dynamic variable that is expected to change its type and one that is not. Similar to the previous example that was given:

main() {

  var a = 'One';

  a = 2; // Is this intentional?

}

Analyzer can't know whether whether the commented line is a problem. Providing a type annotation would solve that, but at a cost.

 

I think this is a variation of the point I just developed in my response to Günter. 

Just so that I'm clear, the 'cost' you are referring to here is the effort of me typing (or using code completion) to type annotate the LHS?

Paul

[Paul Jolly]

One of the things analyzer can't do is tell the difference between a dynamic variable that is expected to change its type and one that is not. Similar to the previous example that was given:

main() {

  var a = 'One';

  a = 2; // Is this intentional?

}

Analyzer can't know whether whether the commented line is a problem. Providing a type annotation would solve that, but at a cost.

Since Dart is a dynamically typed language and variables can change type, I think the analyzer pretty much has to allow the above code. Instead, I really like what it does now:

    var a = 'One';

    a = 2;

    a.length; // Warning, int has no length getter.

But a 'strict' mode could give you some sort of notification (strict type hint, call it whatever we like!)

The proposal does not challenge the spec or indeed the analyzer's compliance with the spec. It is designed to complement it. 

[Paul Jolly]

(replying off list because I would be repeating myself too much :)

"This example shows that type inference cannot be trusted. This is a major drawback"

This is the crux of my argument behind why a 'strict' proposal would be nice (for me, and potentially others). I don't know, hence why I asked the question on list, whether it's possible to ever prove that the analyzer can be 'trusted'. Having a 'strict' mode as proposed would be trustable, at the tiny expense of having to type annotate code.

On 15 August 2014 18:49, Alex Tatumizer <tatu...@gmail.com> wrote:

This example shows that type inference cannot be trusted. This is a major drawback.

Bob, when you see the line var a = 'One', what is your best guess: is this a String, or we don't know? In 99.9% cases, it's a String, and never be assigned anything else. However, remaining 0.1% is treated as default by a language. There should a way to utilize type inference, without the need to explicitly declare String a = 'One'

[Paul Jolly]

(replying off list because I would be repeating myself too much :)

Except I replied to the list.... sorry for the noise. 

[Paul Jolly]

But the analyzer, rightly enough, must adhere to the specification. So sometimes, as unhelpful as it might be to our users, the only thing I can tell you is why he tool is doing things the way it is.

The language tries to walk a fine line between two sets of users. On the one hand, there are users who want the tools to be able to use all of the power of a type system to help catch bugs during development. On the other hand, there are users who want to be able to write code quickly and don't want the tools to impose unnecessary restrictions or require them to write unnecessary code (for example, see the discussions about having to write a type annotation on the left-hand-side when the variable initializer consists of an instance creation expression).

Nicely put. The 'strict' proposal is trying to help (along with the existing implementation of the analyzer) provide for those "who want the tools to be able to use all of the power of a type system to help catch bugs during development". 

But that opens up a hard question (one of the questions being wrestling with on this thread): what should happen when you mix those two styles? The language generally takes the approach that there shouldn't be an error or warning unless it's always going to be a real problem. That implies that some code containing problems will not produce errors or warnings.

This is indeed a hard question given the spectrum of users that exist between the two extremes you described earlier. However, I don't believe it prevents development of a 'strict' proposal, whether in the analyzer, as a plugin for the analyzer, or as a separate tool. The question of how to deal with the mix of styles can start to be tackled when one can clearly see (in an automated way) that we are dealing with a mix of styles.

[Günter Zöchbauer]

On Monday, August 18, 2014 1:22:38 PM UTC+2, Paul Jolly wrote:

I haven't. Explicitly declaring the return type, even as void or dynamic, enables the editor to be nice and tell you when you forgot to return a value. I think the style guide needs to evolve with the tools here.

 

+1 

The same for type annotations on local variables.

When the analyzer can't infer the type the annotation should be allowed. 

I might be missing something here, but when can the analyzer know that it has been unsuccessful in inferring a type? 

When dynamic is inferred then there is no specific type information.

 

Surely the spec precludes there being such a situation?

And we have to include dynamic here because the spec is specific on when dynamic is inferred. For example if I write var i = 2 then the analyzer can't tell me that's wrong (even with a hint) because I could well have intended the type of i to be dynamic.

with var i = 2 is nothing wrong. It should be treated as integer.

What examples do you have in mind? 

someMethod(x) {

  if(x) {

    return "foo";

  } else {

    return 12.34;

  }

}

var i = someMethod(y);

 

[Paul Jolly]

I don't think anyone is offended when the tools provide valid feedback related to problems in the code, but there are plenty of people who are offended when the tools are not as smart as they expect them to be.

This thread has been fantastic in helping me to understand more about Dart, the decisions made by the spec authors, tool developers etc. Certainly not intending to cut off that discussion, rather just to say thanks to everyone for their feedback, thoughts, corrections etc.

In Brian's words, I fit very much into the camp of someone "who want the tools to be able to use all of the power of a type system to help catch bugs during development". Hence the 'strict' proposal.

I've taken a look at the example code provided with the analyzer to try and pull together a very rough proof of concept of a tool that helps (partially) check SE1. Thanks to the analyzer team for providing such a complete example.

It can be found here (along with instructions on how to run it):

https://github.com/myitcv/dart_strict

The code is very rough, probably broken and definitely incomplete:

• It runs as a separate tool from the analyzer, and hence is not part of the warnings/hints returned in the Dart Editor
• Right now it would only be useful as part of a CI build process
• It is very slow
• It (partially) enforces the original SE1, namely that type annotations should appear everywhere. I say partially, because I haven't even completed this rule
• If a type annotation is missing, the [line:col] is printed along with a helpful message, and the process exits with a non-zero exit code

For example running the tool against one of the provided test files:

$ dart dart_strict.dart /home/myitcv/darts/dart-sdk _test_files/test1.dart

[1:1] Missing a return type for function 'blah1'

[1:7] Parameter 'a' is missing a type annotation

[10:3] Variable declaration is missing a type annotation

[11:3] Variable declaration is missing a type annotation

As a standalone tool, the idea would be that dart_strict be used in conjunction with the analyzer as part of a CI.

Clearly this can (and will) be improved a lot, to become tidier, enforce the looser version of SE1 and also bring in SE2 and SE3. 

My question to Brian and the team is, with due reference to Gilad's comments on issue #20443, how could this be integrated or otherwise with the analyzer?

Thanks

[Paul Jolly]

The same for type annotations on local variables.

When the analyzer can't infer the type the annotation should be allowed. 

I might be missing something here, but when can the analyzer know that it has been unsuccessful in inferring a type? 

When dynamic is inferred then there is no specific type information.

But the spec allows for dynamic to be inferred. How can you distinguish between a situation when I intended this and one where I didn't?

And what about if I deliberately use dynamic?

This is why I proposed the use of Object in the original version of the 'strict' proposal, to help distinguish. 

 

Surely the spec precludes there being such a situation?

And we have to include dynamic here because the spec is specific on when dynamic is inferred. For example if I write var i = 2 then the analyzer can't tell me that's wrong (even with a hint) because I could well have intended the type of i to be dynamic.

with var i = 2 is nothing wrong. It should be treated as integer.

Per the relaxed version of SE1 it would be treated as an integer so I agree in the context of the 'strict' proposal.

What examples do you have in mind? 

someMethod(x) {

  if(x) {

    return "foo";

  } else {

    return 12.34;

  }

}

var i = someMethod(y);

What is the return type of this method? I would suggest it is Object. 

[Günter Zöchbauer]

On Monday, August 18, 2014 4:13:57 PM UTC+2, Paul Jolly wrote:

The same for type annotations on local variables.

When the analyzer can't infer the type the annotation should be allowed. 

I might be missing something here, but when can the analyzer know that it has been unsuccessful in inferring a type? 

When dynamic is inferred then there is no specific type information.

But the spec allows for dynamic to be inferred. How can you distinguish between a situation when I intended this and one where I didn't?

It's allowed but it also indicates that there isn't any type information.

I also think there should some distinction between dynamic a no type information.
Object would be fine.

In my opinion the developer should have a way to explicitly tell where he wants dynamic and where he wants to be explicit about types and the analyzer should handle the code accordingly

but the recent comments have again convinced me that the language designers have no interest to change anything in this regard and I therefore think that the discussion doesn't lead anywhere.

 

And what about if I deliberately use dynamic?

This is why I proposed the use of Object in the original version of the 'strict' proposal, to help distinguish. 

 

Surely the spec precludes there being such a situation?

And we have to include dynamic here because the spec is specific on when dynamic is inferred. For example if I write var i = 2 then the analyzer can't tell me that's wrong (even with a hint) because I could well have intended the type of i to be dynamic.

with var i = 2 is nothing wrong. It should be treated as integer.

Per the relaxed version of SE1 it would be treated as an integer so I agree in the context of the 'strict' proposal.

What examples do you have in mind? 

someMethod(x) {

  if(x) {

    return "foo";

  } else {

    return 12.34;

  }

}

var i = someMethod(y);

What is the return type of this method? I would suggest it is Object. 

The method is just a method with no specific return type (dynamic/object) to make clear where the initial value for the variable comes from in the example.

This is an example where I think type annotations for local variables should be allowed per style guide.

[Bob Nystrom]

This thread is long!

I just want to reply to one point. (As usual, Brian, I think you did a great job articulating your perspective here, and I'm in agreement with everything else.)

On Sat, Aug 16, 2014 at 9:05 AM, 'Brian Wilkerson' via Dart Misc <mi...@dartlang.org> wrote:

The language tries to walk a fine line between two sets of users. On the one hand, there are users who want the tools to be able to use all of the power of a type system to help catch bugs during development. On the other hand, there are users who want to be able to write code quickly and don't want the tools to impose unnecessary restrictions or require them to write unnecessary code (for example, see the discussions about having to write a type annotation on the left-hand-side when the variable initializer consists of an instance creation expression).

You may be implying these are disjoint sets of users, but I don't believe they are. I think you have people with a range of preferences about:

• How many type annotations they like to read and write in their code.
  
• How many problems and potential problems they want static analysis to draw their attention to.

Even if you get each of those continua and chop them into two buckets (which I think is valid to do), I still believe that these are orthogonal axes. I think you're considering:

• People who want lots of static checking and lots of type annotations. They want their code to be very explicit and want tools to catch as many bugs as possible. These people are likely fans of "defensive coding". They feel most comfortable nested snugly in the embrace of fully-typed code. Java, C++, and maybe Eiffel cater to them.
  
  
• People who want terse code and tools that don't get in their way. They just want to run their program and if there's an error at runtime, they'll deal with it then. These people like dynamic languages and won't suffer a stupid tool that tries to make them feel bad. JavaScript, Ruby, and Python cater to them.

These buckets exist, but so do the other corners of 2x2 matrix:

• People who want few type annotations and lots of static checking. These people want to lean heavily on tooling. They like lots of type inference and want lots of analysis based on it. They want very smart tools and have little patience for boilerplate or bugs that could easily be caught by tools. ML, C#, and Scala cater to them. I'm in this set.
  
  
• People who want lots of type annotations and little static checking. These people like seeing annotations in their code to help them understand it, but they believe those annotations are purely a communication tool between programmers. Gilad calls this a "documentary type system". Users here don't want tooling that looks at those and then gets in the way of running their program. As far as I can tell, Dart caters best to this group. My belief is that this is the least populated group of the four options.

I think it's important to tease these two axes apart because when you hear people clamoring for better static type checking, remember many of those people are not asking to have to write more type annotations. Likewise, when you hear people railing against type annotations, they may not be saying they want to ignore errors in their code.

Cheers,

- bob

[Brian Wilkerson]

Bob,

I hadn't thought of them being orthogonal issues, but you're right, they are.

I'll also point out that I don't believe that any given person always has the same requirements for comfort.

I prefer having the extra safety of static checking and lots of type annotations when I'm writing production code, and the more clients that code is likely to have the more I want it. But when I'm writing test code I'm happy to have few annotations (I still like static checking because I think it shortens the development cycle). In part that's because by nature no test should depend on another, so if the test code is wrong there's less impact.

Brian

[Alex Tatumizer]

So maybe "strict mode" is not a bad idea after all? It just covers another corner of the matrix. But I think it should be based on type inference (group 3 in Bob's classification).

[Günter Zöchbauer]

Would it be so difficult to make this configurable to some degree?

[Günter Zöchbauer]

good write-up :-)

[Alex Tatumizer]

I'm afraid if strict mode is introduced (under any definition), then all libraries will have to be written in strict mode. Otherwise, corporate users will refuse to rely on them. As a result, current "free mode" will be applicable only to tests, etudes etc.

What follows is that strict mode becomes a default.