PEP 484 type hinting in Django

[Alexander Hill]

Hi all,

I like the plan to include PEP 484 type hinting in Django, outlined in the PyCharm promotion blog post. [1]

Has this proposal been fleshed out much? Is there any extra information available anywhere that I've missed? I think this will be great for Django, and I'd happily contribute to the effort.

Cheers,

Alex

[1] https://www.djangoproject.com/weblog/2016/jun/30/pycharm-and-django-fundraiser/

[Markus Holtermann]

Hi Alex,

I haven't heard of any discussion on that topic. I'd certainly like to
have a DEP before we start implementing it, though.

Cheers,

Markus

>--
>You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
>To post to this group, send email to django-d...@googlegroups.com.
>Visit this group at https://groups.google.com/group/django-developers.
>To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CA%2BKBOKzHPhsqeZGF4ebcc2eCuMjpSsN4MfWFAMxzFXjU%3D2-isw%40mail.gmail.com.
>For more options, visit https://groups.google.com/d/optout.

[Florian Apolloner]

I personally do not think that this is something belonging into Django itself -- at least not in the current state. The requests (I think) maintainers have a good blog post iirc on the current issues.

[Daniel Moisset]

Hi 

@Alex,

I wasn't aware of the fellowship program, but I've been getting started to work at this and I already have some minimal things up

I have a repo with type hints (currently just covering the HttpRequest object and other minor stuff) at https://github.com/machinalis/mypy-django ; I'm currently working on the HttpResponse object (with that I can already have some typehinting at my views). To work on it I'm actually doing annotations on the source code (I have a forked django at https://github.com/dmoisset/django/tree/typing-requests ), and then converting those to stub files, In that way I can run the checker against the actual code and ensure the annotations are consistent.

I was planning to make a comment about these here when they were a bit more mature with some minimal example, but given that the discussion already got here I think they may help.

@Markus

If there's interest in the community, I volunteer for the process of getting in charge of the discussion and drafting of the DEP. I wanted to know first how the community feels about these.

@Florian

Would you care to ellaborate? I couldn't find the post you mention (although requests is one of the few 3rd party projects that have support at the official typeshed repository, https://github.com/python/typeshed )

* 

@all

For those of you wondering what all these is about, it's about adding static type hints to the arguments/result types of the results in Django. Currently there are 2 ways to do this:

A) As a 3rd party project, providing separate "interface files" that mimic the ones with django, only with the function/method signatures

B) As inline annotation in the django code (think of it as structured docstrings)

Option (A) has the benefit of not interfering at all with the project, with the burden that it's harder to maintain and keep in sync when django makes new releases; it only helps django users

Option (B) is easy to keep in sync and can help with django development itself, but it may imply some process changes (integrating a checker into CI, and in some very specific cases making contributors aware of it). Another drawback is that given that django still supports python 2, we need to use the "#type" comments instead of python3 annotations that are quite uglier.

I've been working doing this for some internal projects at Machinalis and for a couple open source projects. So let me add a few comments that I already know that are frequently asked questions:

* This has minimal to zero impact on runtime (probably adding a few imports). It's comparable to adding docstrings to a module that doesn't have it. The annotation is for outside analyzer tools (checkers and linters) running on CI, not on the django project itself

* This does not have to be done project-wide, so it's not a huge undertaking; the point of gradual typing is that you can work at function granularity and build up. That's why I've started in small but visible modules like HttpRequest/HttpResponse.

* In fact, there are some parts of django that will probably not be covered for a long time (I thinking QuerySets here) because they would need substantial support from the tool to make sense

* The main goal of this is NOT detecting bugs (tests generally work ok). The goal is providing accurate documentation about function inputs and outputs that can help humans (for example people asking themselves "are the keys of request.FILES bytes or unicode?"), and tools (for example documentation generators that could expose this info, code editors that can add autocompletion, etc)

* In a codebase with builtin annotations, it may help detecting simple bugs more quickly than the test suite; it works well integrated with an editor (in the same way that tools like pep8/flake8 detect simple things there). The power here increases as you get more core annotated (but again, it may work gradually).

I also wanted to reinforce that I'm not asking somebody else to do it... I need this, so I'm already working on it, but I welcome anyone to join :)

Best,

    D. 

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CA%2BKBOKzHPhsqeZGF4ebcc2eCuMjpSsN4MfWFAMxzFXjU%3D2-isw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Daniel F. Moisset - UK Country Manager

www.machinalis.com

Skype: @dmoisset

[Florian Apolloner]

On Wednesday, August 17, 2016 at 11:06:47 AM UTC+2, dmoisset wrote:

@Florian

Would you care to ellaborate? I couldn't find the post you mention (although requests is one of the few 3rd party projects that have support at the official typeshed repository, https://github.com/python/typeshed )

https://lwn.net/Articles/643269/ and https://lwn.net/Articles/643399/ -- might be that things changed by now.

[Tim Graham]

The JetBrains announcement that they want to fund the project isn't a guarantee that it'll be implemented. The feature needs to go through the normal feature acceptance process, which as Markus said, might involve a DEP.

Assuming the idea is accepted, my sense on timing would be to wait until January when Django drops support for Python 2.7 and 3.4 in master. Then we could use inline annotations rather than the stub files.

Past discussions of type hinting:
https://groups.google.com/d/topic/django-developers/z_P1TvJ6QG8/discussion
https://groups.google.com/d/topic/django-developers/xOTmq93YZuQ/discussion

[Daniel Moisset]

Thanks for the replies,

As I mentioned, I have already started implementation (and I'm willing to go through with it, having even some time from my work allocated to do it); I wasn't aware of the JetBrains plan (it's a nice plus, but I don't depend on it), and I'll probably do it as external files if the django core team itself is not interested, but I think it's more beneficial to do it inline.

One clarification is that there is no need to drop 2.7 support to use inline annotations; there's a python2 compatible way to annotate described now in PEP484. 

What I'd like to do is to get a reading of the room to see if there's some degree of interest (at least a "I'm curious to see how it looks if you do the work") before following up with the DEP process. I made a very quick summary of what I think the benefits would be, but if it's not clear I'd be happy to ellaborate.

Some key things that have changed since last year (regarding the links posted by Florian and Tim) are:

* PEP-484 is now approved and standard part of python. The mypy checker is now under the python project umbrella and getting active maintainance and backed by key people in the python community

* Having a standard (instead of just a 3rd party tool supporting this) means that this now annotations can help to interoperate with many tools (type checkers, editors, documentation generators, refactoring tools), so the impact in the ecosystem is larger

* There's some evidence that this works on production (people in dropbox have been using it for ~ a year, according to [1])

* There are several complaints telling that "this won't be actually optional", but I see no evidence to support it. And in any case those are arguments around deciding to include this in the language, and that decision has been made already. But in my experience, annotations help more in some particular modules/APIs and not in others, so an abvious option is to add them only where they add value (i.e. increase readbility and clarity of interfaces)

* Cory Benfield points at some complex types, that (from a quick look) with new type aliases and overload semantics can probably be described in a much simpler and readable way. And again, if they don't that function (or that argument) shouldn't be annotated.

* There's someone volunteering to do the work (me and some colleagues at Machinalis) :)

I've already been looking at some interfaces in django and I feel that a lot of them are as not dynamic and polymorphic as requests is, so some success can be achieved here.

So, how do you guys feel about this? what are the risks/fears that you'd like to have addressed? do you share my opinion that this will be positive for both the framework and its users?

Best,

[1] http://pythonpodcast.com/david-greg-mypy.html

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/b5dcb500-4706-407f-8e42-65944d30ccf5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[ludovic coues]

Dropping support for python 2.7 has been planned for some time. Django
1.11 will be the last one supporting python 2. The following release
will be django 2 and will only support django 3.5+. See
https://docs.djangoproject.com/en/dev/releases/1.11/

That's what Tim was talking about. You can target inclusion in django
2 and forget about the quite uglier #type comment.

>> email to django-develop...@googlegroups.com.
>> To post to this group, send email to django-d...@googlegroups.com.

>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/b5dcb500-4706-407f-8e42-65944d30ccf5%40googlegroups.com.
>>
>> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> --
> Daniel F. Moisset - UK Country Manager
> www.machinalis.com
> Skype: @dmoisset
>

> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.

> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/django-developers/CALuYSZUopy0YHPgPZ%2BNgSk%2BKzNf8CpP-gdj2-CLXpicSNJ86Dw%40mail.gmail.com.

>
> For more options, visit https://groups.google.com/d/optout.



--

Cordialement, Coues Ludovic
+336 148 743 42

[Josh Smeaton]

Agree. January is not that far out, and then you get to build the annotations as designed in the PEP. You can begin work earlier of course, but keeping your patch up to date with all of the work going on may become annoying. The HttpRequest/Response objects don't really get changed that often so it merge conflicts might not be that big of a deal in practise.

I'm probably in favour of the idea in general. There are many internals in django that would benefit from documenting inputs and outputs - if only for the people working deep within the guts. I understand those parts probably wouldn't be targeted first, but as new patches come in, we can begin making those annotations gradually.

I do have some concerns about providing typing information where importing type names would normally cause import cycles. How is that addressed? I admit to not really following the typing hint features very much so I'm a bit naive here.

Thanks,

[Daniel Moisset]

Hi,

On Sat, Aug 20, 2016 at 6:44 AM, Josh Smeaton <josh.s...@gmail.com> wrote:

Agree. January is not that far out, and then you get to build the annotations as designed in the PEP. You can begin work earlier of course, but keeping your patch up to date with all of the work going on may become annoying. The HttpRequest/Response objects don't really get changed that often so it merge conflicts might not be that big of a deal in practise.

OK, I'll probably go for a hybrid approach (external annotations like I'm doing now), then convert to inline python3 style closer to the switch. It'll be easier to maintain in syn that way, I hope. I've also noticed that assuming python3 simplifies a lot of the work (like library code that is copied into django for python 2 compatibility, six-related stuff, text conversions, etc)

 

I'm probably in favour of the idea in general. There are many internals in django that would benefit from documenting inputs and outputs - if only for the people working deep within the guts. I understand those parts probably wouldn't be targeted first, but as new patches come in, we can begin making those annotations gradually.

Actually I've found it better to work from the bottom up, so I end up annotating the internal modules and work my way up to the APIs. For example, to annotate HttpRequest, I actually started from django.utils.datastructures to have MultiValueDict, which then is inherited by QueryDict, which is an attribute of request objects. So it's easier to do it in a way that will end up covering internals. (I could do it right away from the API, but I'd have less certainty that my annotations are right)

I do have some concerns about providing typing information where importing type names would normally cause import cycles. How is that addressed? I admit to not really following the typing hint features very much so I'm a bit naive here.

On the code I have covered I haven't found any case of this, but if it happens the standard solution is to write

if typing.TYPE_CHECKING:

    from module import SomeType

the TYPE_CHECKING constant is defined to be False, so no import actually occurs in runtime, but the typechecker assumes it as True and finds the relevant type.

 Best,

To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/f3be2364-0a42-4b56-b274-662cb0833e76%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Graham Wideman]

First -- three thumbs up for Daniel's initiative and advocacy!  Count me in as a random enthusiastic would-be user.

Question: I've looked at Daniels repo and the pyi files therein, and one issue is ascertaining which version of Django they apply to. Are there some ideas about how that's to be done?

Graham

On Wednesday, August 17, 2016 at 7:13:51 AM UTC-7, dmoisset wrote:

Thanks for the replies,

As I mentioned, I have already started implementation ...

[Daniel Moisset]

Hi Graham, thanks for the support.

I'm aiming at Django 1.10 right now, but given that APIs are more stable than implementation, you may have some success with other versions. This is just a guess, I haven't tried

Best,

    D.

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/ac0102f7-9106-40dd-be62-17fcb30c0f1f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Graham Wideman]

Hi Daniel, thanks for replying. 

My question was less about what version of django you were targeting, and more about how a type-hints library such as this should be annotated to indicate what version(s) it targets. Since a type hints library may well be distributed separately from the code that it hints, it won't necessarily be obvious.

Graham

On Monday, September 5, 2016 at 6:08:30 AM UTC-7, dmoisset wrote:

Hi Graham, thanks for the support.

I'm aiming at Django 1.10 right now, but given that APIs are more stable than implementation, you may have some success with other versions. This is just a guess, I haven't tried

Best,

    D.

On Sat, Aug 27, 2016 at 7:29 AM, Graham Wideman <graham....@gmail.com> wrote:

First -- three thumbs up for Daniel's initiative and advocacy!  Count me in as a random enthusiastic would-be user.

Question: I've looked at Daniels repo and the pyi files therein, and one issue is ascertaining which version of Django they apply to. Are there some ideas about how that's to be done?

Graham

On Wednesday, August 17, 2016 at 7:13:51 AM UTC-7, dmoisset wrote:

Thanks for the replies,

As I mentioned, I have already started implementation ...

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/ac0102f7-9106-40dd-be62-17fcb30c0f1f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Graham Wideman]

Just making note of the typeshed project https://github.com/python/typeshed, "Typeshed models function types for the Python standard library and Python builtins, as well as third party packages."

Pertaining to my earlier question about how pyi files should tell what version of a library they pertain to:FWIW, typeshed uses directory branches for different pyi files pertaining to different versions of Python, and version-range conditionals within pyi files. These version-selective devices pertain to version of Python. The interest in the discussion here would relate to version of Django, of course, where it seems similar measures might apply.

[Daniel Moisset]

Hey Graham. mypy has a structure to support multiple python versions as you noticed (and allows it to select them from the command line), but there's no mechanism for selecting library version. For me, supporting "the latest stable version" is a reasonable goal now. Flagging that appriopriately may become a problem in the future, but only if this succeeds, so I'm focusing on making this useful right now :).

I already mentioned a naive approach (use different branches in the typeshed repo to support different django versions), but I'm keen to hear better ideas!

And of course, if we get this into the django codebase itself the problem goes away (you get the annotations when installing django).

Best,

   D.

On Sun, Sep 18, 2016 at 9:00 AM, Graham Wideman <graham....@gmail.com> wrote:

Just making note of the typeshed project https://github.com/python/typeshed, "Typeshed models function types for the Python standard library and Python builtins, as well as third party packages."

Pertaining to my earlier question about how pyi files should tell what version of a library they pertain to:FWIW, typeshed uses directory branches for different pyi files pertaining to different versions of Python, and version-range conditionals within pyi files. These version-selective devices pertain to version of Python. The interest in the discussion here would relate to version of Django, of course, where it seems similar measures might apply.

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/5bf15a4b-8360-4ee0-a985-01f42e7e27fd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Daniel Moisset]

Just a small update on this; I've published a first usable (well, I hope) version at https://github.com/machinalis/mypy-django/ . It's stubs only, although I'm basing it on an annotated django fork at https://github.com/dmoisset/django/tree/typing . I covered some high-visibility APIs only (request/responses, url routing, generic views), and some helper modules that hose are built on (core.files, timezones, data structures).

I think it works, although while annotating the original code I struggled with a few things:

 - Some classes (most notably MultiValueDict/QueryDict) use inheritance then break a bit the original interface. In that case I changed the implementation a bit to use delegation instead of inheritance, so it's more than "just adding annotations" in places like this. It still passes tests, and I haven't run benchmarks but GvR suggested that the delegation version is probably faster.

 - There are some warnings thrown by the checker in the timezone module that I'm not sure if they're because a bad specification or if those are actual bugs (some methods like ReferenceLocalTimezone.utcoffset crash if passed a "None" argument, but some examples seem to imply that a tzinfo should accept it); there's no clear spec there. In the end I think this is a good sign (this work will help identify some tricky/underspecified corner cases in the API).

 - The type stubs (annotations in separate files) allow to tell some "white lies" about the API (for example saying @classmethod instead of @classonlymethod because the latter confuses the typechecker). Those make it more usable, but something will have to be done with that if integrating annotations on the code is desired.

In general, the APIs in django seem to be much more structured than other python stuff (like the requests example Florian mentioned

Feel free to comment and follow-up details on the project issue tracker. I posted this update here in case it helps in the future to follow the discussion about adding inline annotations.

Best, 

    D.

[Eddy C]

Hi Tim, do you know if JetBrains still willing to fund the project as the upcoming django 2.1 will only support python 3.5+ that pave the way for inline annotations.

[Florian Apolloner]

To the best of my knowledge JetBrains fundled the money to Django for that specific purpose -- so yes the funding should be here if needed. That said, there is no decission on a) whether we actually want type hints and b) if the should be inline or stubs. Those two points have to be cleared first.

Cheers,
Florian

[Daniel Moisset]

A long due update on this, given that the question popped up recently:

I worked at some time in type annotations and published some for Django 1.10 ; I'm currently not working on them, given that my current work has not been very close to Django development

After trying a few things, the best way to get this would be:

1) Add annotations upstream. Having to maintain a separate set of annotations is probably not sustainable and will lead to errors. This of course needs annotations accepted by the team.

1.1) Integrate typechecking into Django's build/CI process. This would ensure that annotations are consistent with the implementation.

1.2) Annotations do not need to be exhaustive (and there are certainly parts of Django which could benefit from them much more than others)

2) Generate stubs from the annotated source. This was not possible when I wrote my stubs, but I've commited the mypy improvements with that feature. 

2.1)A few stubs may need to be overridden manually from the autogenerated (I found a few examples of this when I wrote my stubs)

3) Distribute annotated files separately (not with typeshed, and probably not with django) so people can install the version they need. Distribution of stubs has been problematic, PEP 561 should solve it but an implementation for it has been just merged last week into mypy, and probably released soon.

Before PEP561 and the ability to generate stubs, I don't believe it was practical to make this project sustainable. At this time, it might be and it can be a good time to restart this discussion.

Best,

D

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/6f2b5dd9-f8e6-4b67-8e25-585cbd802413%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Daniel F. Moisset - UK Country Manager - Machinalis Limited

www.machinalis.co.uk

Skype: @dmoisset T: + 44 7398 827139

1 Fore St, London, EC2Y 9DT

Machinalis Limited is a company registered in England and Wales. Registered number: 10574987.

[Andreas Galazis]

To me one approach would be to put a cut off for any merged code /PR  start inlining type hints/annotations for all new code. This seems to simple to be a solution but at the end of the day as code gets updated even bigger part of the codebase will have type hints. The question is whether partial type-hinting is actually useful, but at least it supports heading towards the right direction.

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/6f2b5dd9-f8e6-4b67-8e25-585cbd802413%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Daniel Moisset]

On 11 April 2018 at 11:21, Andreas Galazis <agal...@gmail.com> wrote:

To me one approach would be to put a cut off for any merged code /PR  start inlining type hints/annotations for all new code. This seems to simple to be a solution but at the end of the day as code gets updated even bigger part of the codebase will have type hints. The question is whether partial type-hinting is actually useful, but at least it supports heading towards the right direction.

I don't think that approach will work. Partial type hinting is useful and viable, but not randomly.... you need to do it bottom up (covering basic abstractions first). So the place to start is probable the parts of django that change less and are more solid foundations (in my case I started with requests, views and URL resolvers, etc)

[Andreas Galazis]

I agree with you, but at some point, we could combine solid annotated core with a cut off for non annotated code? Otherwise, this will end up being a loop.

[李海珍]

Wow, time fly. I had taken part in the promotion activity 2 years ago.

while haven't seen any progress on this topic.

Is there on any progress on this topic?

在 2016年8月17日星期三 UTC+8下午12:08:48，Alex Hill写道：

[Maxim Kurnikov]

I'm working on stubs / mypy plugin for Django here

https://github.com/mkurnikov/django-stubs

I'm going to release an alpha version sometime around New Year's. I've integrated machinalis version of stubs. There's also generated stubs from Monkeytype execution over the django testsuite in the repository, I'm moving those to the main directory little by little. 

Still emits a lot of false positives though. Also, I test it on an internal project with python3.7/django2.1, don't know about other versions. 

With PEP561 there could be only one stubs package in PyPI (named "django-stubs"), so I'm postponing release till it'll be as useful as possible. I want to know core devs opinion too, whether it's safe to release or not.

[Carlton Gibson]

Hi all. 

Where are we with this Type Hinting work? 

I just closed https://code.djangoproject.com/ticket/30019 as needsinfo pointing back to this thread. 

As far as I can see: 

* There's keenness for this. 

* There's a number of people who are prepared to put in the effort. 

* But we just need a strategy. 

It looks then like a coordination problem. Do we just need to get said people in a (virtual) room...? 

Maybe one of you — anyone — opening a DEP to begin the conversation would be enough.. 

(That's just an idea.) 

Looking here: https://github.com/django/deps/blob/master/final/0001-dep-process.rst#dep-submission-workflow

I'd guess "Forming the Team" is relevant. 

Happy to help if I can. 

(I recall Frank mentioned it no too long ago...) 

Kind Regards,

Carlton

[Andreas Galazis]

Just stumbled on this blog post: https://www.willmcgugan.com/blog/tech/post/adding-type-hints-to-the-django-orm/. An interesting but too theoretical suggestion: We could even get rid of field/ field configuration. But I guess that's far ahead of what we are trying to achieve xD.

[Maxim Kurnikov]

Hi, I'm the original creator of https://github.com/typeddjango/django-stubs.

I guess it's my duty to start writing a DEP, so here is some preliminary thoughts/questions to gather early feedback: 

Main discussion points: 

1. There's two ways to make Django compatible with type hinting and PEP484: 

    - Separate repo with `.pyi` files and mypy plugin (how it's done currently with django-stubs). 

    Pros: 

        - no backwards-incompatible changes. Only required change from Django is adding the `__class_getitem__` method to `QuerySet` https://docs.python.org/3/reference/datamodel.html#object.__class_getitem__ to support generic parameters in 3.7+

        - could be released separately

        - no type hints code clutter

    Cons: 

        - (as of now) there's no way to use type hints to typecheck Django codebase itself. I think the official answer to that is to use retype tool https://github.com/ambv/retype (https://github.com/python/mypy/issues/5028#issuecomment-495942769), but so far I didn't have any positive experience with that

    - Inline type hints in the codebase. 

    

    Pros/Cons are exact reverse. 

2. Whether to use Django own machinery to extract info for the models/fields, or not. 

Current implementation of django-stubs utilises `Apps` registry of Django itself to extract some data for models/fields.

 So, basically it executes codes first, then performs static analysis of it. 

Pros: 

    - lots of models/fields introspection could be extracted from the `_meta` API => lower maintenance burden, more accurate types. In the pre-1.0.0 versions of the django-stubs it was all done statically, and it was very painful to implement. 

Cons: 

    - mypy daemon mode (dmypy) is broken. It could be fixed, but it requires some changes to mypy itself, and as `django.setup()` could not run incrementally, it's not clear whether dmypy would provide any real benefits over plain incremental mode. 

    - there could be some side effects in `django.setup()` invocation

    - cannot typecheck invalid code

    - slower to execute

Carlton, is it possible to move discussion to Github somehow?

[Carlton Gibson]

On Monday, 26 August 2019 14:05:19 UTC+2, Maxim Kurnikov wrote:

Carlton, is it possible to move discussion to Github somehow?

Drafting a DEP begins by making a PR against the drafts folder in the DEPs repo. I guess that could be a WIP PR, if felt beneficial/necessary. 

[Lukas Meier]

I'm writing here with the risk of you guys obviously already knowing about it

there is a project called monkeytype written by instagram that does analysis similar to what  @Maxim said

they actually have specific answers to django related questions here

https://monkeytype.readthedocs.io/en/stable/faq.html

[Maxim Kurnikov]

I actually used MonkeyType to generate first version of `django-stubs` back in the day. 

Best regards, 

Maxim Kurnikov. 

--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/trTEbURFhEY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/2273ec96-8ad8-4ae3-9802-1e0663bdd9b6%40googlegroups.com.

[Carlton Gibson]

Maxim Kurnikov has begun a draft DEP for adding hints into Django. 

https://github.com/django/deps/pull/65

Please contribute your thoughts there. 

Kind Regards,

Carlton