Type annotations in bzl files

[Laurent Le Brun]

Type annotations are useful for many reasons: readability, type
checking, refactoring, code completion in IDEs, etc. Although we don't
want to make them part of the language, we should standardize a way to
write them.

The current recommendation is to document the code using docstrings
like this: https://github.com/google/styleguide/blob/gh-pages/pyguide.md#38-comments-and-docstrings

We observe that "Args" and "Returns" sections typically start with a
type information, e.g.

Returns:
A dict mapping keys to [...]

This means that tools can extract the types of functions arguments and
return values (the word after an optional article). Shall we make more
precise recommendations for writing types in docstrings, to ensure
they can be properly parsed?

--
Laurent

[Klaus Aehlig]

> Shall we make more
> precise recommendations for writing types in docstrings, to ensure
> they can be properly parsed?

I know that bzl files are a bit more restricted than python, but my
experience with python is that there is little value to have parsable
doc strings, if the type system is not specified as part of the language.
Actual python programs have type constraints that that need a type
system well beyond simple types to adequately express their constraints.
This is done either to gain a some flexibility, as for the delay
argument in
http://git.ganeti.org/?p=ganeti.git;a=blob;f=lib/utils/retry.py;h=6ff45e78ab40c02967767cd364090f5ab9997850;hb=HEAD#l115
or because dependent types are used to work around the absence of a
sum type in python, as the return type in
http://git.ganeti.org/?p=ganeti.git;a=blob;f=lib/backend.py;h=58f4ebde5c46f9678a1157b19e97e7c09f0d5c0d;hb=HEAD#l5706

So I would leave it at the human-readable level. Should we go
for parsable types, we should at least make sure we chose a type
system that is capable of expressing the actual use cases in a
more meaningful way than "aribtrary types in, arbitrary type out".

Thanks,
Klaus

--
Klaus Aehlig
Google Germany GmbH, Erika-Mann-Str. 33, 80636 Muenchen
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschaeftsfuehrer: Paul Terence Manicle, Halimah DeLaine Prado

[Alexandre Rostovtsev]

Instead of parseable docstrings, maybe it makes more sense to support some subset of Python 3 type annotation hints in Skylark.

See https://docs.python.org/3/library/typing.html

--
You received this message because you are subscribed to the Google Groups "bazel-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-dev+...@googlegroups.com.
To post to this group, send email to baze...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-dev/20180816145555.GA175927%40google.com.
For more options, visit https://groups.google.com/d/optout.

[Tony Aiuto]

On Thu, Aug 16, 2018 at 11:05 AM 'Alexandre Rostovtsev' via bazel-dev <baze...@googlegroups.com> wrote:

Instead of parseable docstrings, maybe it makes more sense to support some subset of Python 3 type annotation hints in Skylark.

See https://docs.python.org/3/library/typing.html

+1 If we are going to do it, we might as well use Python 3 syntax.

Even if we want an interim experiment with parsable doc strings, it should be a

python3 spec in a well defined place. So I would not have the free form example,

     Returns:

          A dict mapping keys to [...]

Instead, it would be something like:

     Returns:

        (Mapping[str, str]): A map of keys to ...

Where the ( ): before the comment clearly mark the type spec.

[Laurent Le Brun]

We want to avoid syntax changes to the language (so we can't use the
inline Python 3 annotations), but it's a good idea to use the notation
inside docstrings.

> I know that bzl files are a bit more restricted than python, but my
> experience with python is that there is little value to have parsable
> doc strings, if the type system is not specified as part of the language.

Like in Python 3, the language doesn't use the type annotations, but
tools might.
If we want to have type annotations inside the language grammar in the
future, it will be easy to update the code, assuming the docstrings
can be parsed.

--
Laurent

[Alexandre Rostovtsev]

We want to avoid breaking syntactical changes. But I don't understand the reason for avoiding adding new syntactical constructions (type annotations) which were previously not syntactically valid.

[Laurent Le Brun]

Extending the syntax means changing the language specification and
updating all the tools that parse/manipulate BUILD/bzl files. So it's
a much more substantial change.

The list of tools also includes legacy tools that are based on Python
2 (and assume bzl files are a syntactic subset of Python 2).

--
Laurent

[Tony Aiuto]

On Fri, Aug 17, 2018 at 11:12 AM Laurent Le Brun <laur...@google.com> wrote:

Extending the syntax means changing the language specification and
updating all the tools that parse/manipulate BUILD/bzl files. So it's
a much more substantial change.

But it is possible to change the language spec once there is an 18 month (2 years+ even better) roadmap.

We could build docstring parsers now, but implemented with the intent that they can be reused in the

language later.

 

The list of tools also includes legacy tools that are based on Python
2 (and assume bzl files are a syntactic subset of Python 2).

If we know of tools outside of bazelbuild, we should invite their authors directly into this conversation.

[Laurent Le Brun]

From https://docs.bazel.build/versions/master/skylark/language.html#syntax
"Starlark is syntactically a subset of both Python 2 and Python 3, and
will remain so through at least the 1.x release lifecycle. This
ensures that Python-based tooling can at least parse Starlark code.
Although Starlark is not semantically a subset of Python, behavioral
differences are rare (excluding cases where Starlark raises an
error)."

> To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-dev/CACpMnus2rrB%2BSKRFpVYyAGQXnCMUjU3ZeeJRp6%3Dz%2Bm9MzVO16g%40mail.gmail.com.

[Marco Farrugia]

Starlark has a lot of awkwardness around typing, and would benefit from non-comment type annotations for all the same reasons python 3 adopted it (https://www.python.org/dev/peps/pep-0526/). It's not clear if this would change things like rule or provider attr declarations, but there could be benefits in making those aspects look more like Python. 

Rules repositories grow ever larger and more complicated, perhaps we would benefit from more stricter / type checking as well.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-dev/CAFtxR%2BkKBiGqKZi7-%3DAxh9zfCy31VqKj%3DzaJS2LudZfz8182mw%40mail.gmail.com.

[Philipp Wollermann]

I'd just like to point out that "let's add type annotations to comments" hasn't worked out well in the past for other languages.

It reminds me a lot of Closure Compiler (hiding type annotations in JSDoc) vs. TypeScript (making them an explicit feature of the language, but being backwards compatible to existing code without annotations).

If we decide that we want type support in Starlark, I'd be in favor of making it an explicit feature.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-dev/CAEMOC3n0_VnZHrUu1B6BDC8J_M_%2BDahtKT6q7s8J3uYT8DuoAw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Philipp Wollermann
Software Engineer
phi...@google.com

Google Germany GmbH
Erika-Mann-Straße 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg

Diese E-Mail ist vertraulich. Falls sie diese fälschlicherweise erhalten haben sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail an die falsche Person gesendet wurde.

This e-mail is confidential. If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person.

[pauld...@gmail.com]

I'd just like to point out that "let's add type annotations to comments" hasn't worked out well in the past for other languages.

It reminds me a lot of Closure Compiler (hiding type annotations in JSDoc) vs. TypeScript (making them an explicit feature of the language, but being backwards compatible to existing code without annotations).

I don't really have a dog in this fight, but counterpoint: Flow.

Closure annotations aren't less popular than Flow, despite's Flow's TypeScript-like syntax.

IMO, the success or lack of success had less to do with syntax and more to do with tooling. Typescript had first-class support from a world-class IDE. Neither Closure nor Flow had that.

And by that measure, keeping using Python type hints is the better choice for Starlark.

[peter.l...@gmail.com]

Python 2 is supposed to meet "end of life" in 2020, so maybe it's OK to not support tools that only handle Python 2?

It's not difficult to make tools that support both Python 2 and Python 3. Python's lib2to3 has a grammar that handles both, with a few minor tweaks (e.g., removing "print" and "exec" as reserved words); in general, Python3's grammar is a superset of Python2.

And if you do add type annotations to Skylark that match Python 3's annotations, you might be able to leverage pytype and mypy.

>> >>> To unsubscribe from this group and stop receiving emails from it, send an email to baze...@googlegroups.com.