Sounding out for GSoC 2020.

339 views
Skip to first unread message

Carlton Gibson

unread,
Dec 10, 2019, 10:25:22 AM12/10/19
to Django developers (Contributions to Django itself)
Hi all. 

It's time to start thinking about Google Summer of Code (GSoC). If we're going to participate and projects we might propose. 

This year was interesting. Sage in particular did well putting together a cross-db JSONField, but he was probably under-mentored, 
since Mariusz has spent quite a bit of time reworking the PR, and still has a bit to go, before we can pull it in, hopefully for 3.1

So, one consideration we need to think about seriously is our capacity for mentoring. (This isn't just about the candidate's ability — Sage was able to implement all suggestions — we just didn't have as much capacity as we might have liked to think about the requirement implementation — and there were four of us actively giving some time each... — Anyhow, to think about.) 

Then it's projects. There are three that I have on my list that would require a "competent candidate":

1. Work on the migrations. Markus mentioned a particular ticket here but... 
2. Make the parallel test runner work on Windows. ("fork" vs "spawn")

And 3, and this is the big one: 

3. Add 2FA to Django. 

This has been raised a few times: 

* ... others... 

If I'm honest, in 2020, it's the one "battery" I feel a little bit embarrassed we haven't got a story for. Maybe it's not possible in a GSoC type scope but... 
— What would it look like? What can we leverage? Is it worth a go? 

I'm looking at James, Florian, Joe, ... — who else has been keen here?
I'm also looking at the Technical Board, which I'm thinking has (will have) a new guiding role to come up with suggestions for the direction of Django. 


Other Projects: Are there other ideas? (Do you have one?) Are there easier things that we could take "weaker" candidates for? But with that is there a commitment for the mentoring help they'd need? 

Anyhow, we have until January, so I'm just starting the discussion here. 

Kind Regards,

Carlton



Carlton Gibson

unread,
Dec 10, 2019, 10:34:41 AM12/10/19
to Django developers (Contributions to Django itself)
This thread was good on 2FA: linked to some prior art.

Andrew Godwin

unread,
Dec 10, 2019, 1:14:51 PM12/10/19
to Django developers (Contributions to Django itself)
I agree that 2FA could be a good choice - some of the async support work would also have been good, had I made more progress in the latter half of this year.

A couple of other ideas for big projects:

* A secrets manager abstraction and built-in support for Vault, KMS, and other common ones
* A proper story and hooks for running Django as a service (i.e. outside of a traditional HTTP request/response cycle)

And some ludicrous projects I'd still consider:

* Evented/event-sourcing database work on top of the ORM
* A GraphQL-to-ORM mapper

Andrew


--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/af075d59-ece3-4044-9204-b690c746b9e0%40googlegroups.com.

Carlton Gibson

unread,
Dec 10, 2019, 1:51:49 PM12/10/19
to Django developers (Contributions to Django itself)
Hey Andrew. 

Yes, Async, of course. It's not until next July so I'd still think we should list this. Worst case is there's exactly zero progress between now and decision day, in which case you could say that we could only accept a clearly capable candidate. (Jannis was once our GSoC person, so... :) 

The other ideas seem good: the secrets manager abstraction is a nice one (!)

No rush, but when you have a moment, perhaps you could draft a couple of paragraphs for each? (Even if they don't go to GSoC directly, it would come in handy for the TB's direction guidance...) 

Kind Regards,

Carlton


On Tuesday, 10 December 2019 19:14:51 UTC+1, Andrew Godwin wrote:
I agree that 2FA could be a good choice - some of the async support work would also have been good, had I made more progress in the latter half of this year.

A couple of other ideas for big projects:

* A secrets manager abstraction and built-in support for Vault, KMS, and other common ones
* A proper story and hooks for running Django as a service (i.e. outside of a traditional HTTP request/response cycle)

And some ludicrous projects I'd still consider:

* Evented/event-sourcing database work on top of the ORM
* A GraphQL-to-ORM mapper

Andrew


To unsubscribe from this group and stop receiving emails from it, send an email to django-d...@googlegroups.com.

Andrew Godwin

unread,
Dec 14, 2019, 12:31:20 PM12/14/19
to Django developers (Contributions to Django itself)
Here's my take on each of these, leaving out async as that'd be more dependent on where we were:

Secrets Manager
Many environments Django is deployed in use a separate secrets manager to store and provide secrets per environment - either as environment variables, files, or via a direct HTTP API. The project would be to design and add an abstraction interface over secrets managers that allows users to easily map to an external secret in a settings file. There's also room for giving Django a better, built-in per-environments settings option too, pulling from popular third-party patterns.

Django Service Hooks
Django has a lot tied to the HTTP lifecycle, and if you run it outside of this, you miss out on some key things - like database connections being closed properly, or logging, or some middleware. The project would be to design and implement a separate way of calling Django from a service framework - even if that framework was itself HTTP based (basically, don't rely on Django having to run through WSGI/ASGI). Initial idea is to provide a context manager that replaces the wrapping of a request, but the project would have to include requirements-gathering and interviewing people who use Django to power microservices to work out what they want.

Evented Datastores
Evented, or event-sourcing, database schemas are growing in popularity - essentially, a way to implement an append-only pattern on top of a relational database, where every single event that has happened is stored, but you can still efficiently retrieve the current state of an item. Given the way the pattern works, this would be possible to implement on top of Django models - likely as a whole new Model subclass - and implement with a matching QuerySet/Model API. It would require someone very skilled in database design, and it's also arguable if it should be in Django itself, or comprise of developing a third-party app alongside adding hooks/fixes to the Django ORM where needed to make it work well.

GraphQL
Basically, work on an "official" graphql-to-ORM mapper for Django, probably re-using a considerable amount of third-party work. This one could really vary in its application, based on how much other "API-ish" work the project includes (given that Django REST Framework is still separate, we should follow that rough pattern).

Plus, some bonus ones that aren't Django code itself:

Django Case Studies
Our long-talked-about website which brings together case studies and white papers about why people choose Django, aimed at companies working out what they want to choose. This is likely only some coding and a lot of writing/legwork, so may not be a good fit for GSOC, but should be on any future plan we have.

Django Benchmarking
Django has had a benchmarking suite for a while, but we've generally got more slack at tracking the effects each commit and release has. We should institute a set of top-level benchmarks aimed at various areas (the current set in djangobench are weak in, say, the HTTP-serving-speed area) and then run them against pull requests and each release, so we can make sure Django doesn't get slower while nobody is looking.

I'm sure there's a lot more, but these are top of mind at the moment!

Andrew


To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/e1818353-9fa0-4e49-a6e9-4757f06c7e19%40googlegroups.com.

Adam Johnson

unread,
Dec 18, 2019, 5:51:06 PM12/18/19
to django-d...@googlegroups.com
This year was interesting. Sage in particular did well putting together a cross-db JSONField, but he was probably under-mentored, 
since Mariusz has spent quite a bit of time reworking the PR, and still has a bit to go, before we can pull it in, hopefully for 3.1

So, one consideration we need to think about seriously is our capacity for mentoring. (This isn't just about the candidate's ability — Sage was able to implement all suggestions — we just didn't have as much capacity as we might have liked to think about the requirement implementation — and there were four of us actively giving some time each... — Anyhow, to think about.)

I think your analysis is pretty accurate Carlton. Sage was a pretty self-motivated candidate, writing some great blog posts about his code, and we still didn't manage to merge his PR. Perhaps the project was larger than we can reasonably expect?

Nice work on the descriptions there Andrew.

Django Service Hooks

I think a key use case of this is background task processing. Nearly all the Django integrated libraries need to remember to "connection.close()" / "close_old_connections" at the end of requests. I've seen a couple in-house ones that needed this.

That said I can't think of much else to go inside the "service hook"? There's not much else to "the end of a Django context" at the moment is there?

Evented Datastores

I'm not sure if other databases have it but MariaDB recently implemented "temporal tables" which are an SQL standard way of doing historical views on data:  https://mariadb.com/kb/en/library/temporal-data-tables/

Admittedly not quite the same as event sourcing but perhaps easier to implement given it would be limited to the ORM layer.

Django Benchmarking

I love djangobench, and do wish it was maintained and expanded. Running on CI would be amazing.


2FA also sounds like a great idea.

Other ideas:
  • Support MySQL/MariaDB storage engines. I think other databases tend to support some table-level customizations to.
  • Official CORS middleware/view decorator. I took over maintenance of https://github.com/adamchainz/django-cors-headers and its design is a bit pants.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFwN1uqaniLZayw19gKOzEi3KT2EqD8VD-3%3Dd77FExQaYEUVYw%40mail.gmail.com.


--
Adam

Sage M. Abdullah

unread,
Dec 19, 2019, 1:31:15 AM12/19/19
to Django developers (Contributions to Django itself)
Thank you for the kind comments!

I see some very interesting ideas here. I haven't been able to come up with a good idea, but I can give a hand as a mentor next year, so I thought I'd join on this conversation :)

Andrew, I think the Django Case Studies will be a great idea for Season of Docs. It's a similar program from Google, but it's targeted towards technical writers. Carlton has talked about it on another thread. You might want to check that out as well.

Regards,
Sage

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-d...@googlegroups.com.

Asif Saif Uddin

unread,
Dec 19, 2019, 3:09:04 AM12/19/19
to Django developers (Contributions to Django itself)
As a long term user of django, I would love to see the DB/ORM + HTTP related improvements more in django and the schema migration improvements as I use django mostly for ORM and API/CMS based projects.

I like the proposals of Adams related to ORM and inclusion of django-cors-headers to core.

Beside the Idea of supporting 2FA, I think django should think about providing oauth2 support out of the box based on oauthlib and django-oauth-toolkit :)

Definitely there could be better packages :)

Just my thoughts as an long term user of django framework.

Thanks,
Asif

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-d...@googlegroups.com.

vineeth sagar

unread,
Dec 19, 2019, 3:46:45 AM12/19/19
to django-d...@googlegroups.com
Hi Carlton,

I don't know if this is appropriate here or not. I will be starting Master's this spring 2020(Jan)


I love the idea of 2FA. For first iteration I have an idea. While creating a user we can ask for a 6 digit pin and whenever someone tries to login we can make them input this pin. I have seen this implemented by a few companies like Zerodha, UPI in India, these have a big customer base as well. We can move towards more complicated ones with this as a starting point.

Moreover is there a reading list on what's expected of the 2FA, So I can start drafting a proposal for this. I'll look at the rfc mentioned in the thread for motivation and how it can be integrated to django. If there are other latest sources it will be helpful.


Best,
Vineeth


To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/8e15ba2f-4c84-47de-974d-27a36147e426%40googlegroups.com.

Carlton Gibson

unread,
Dec 19, 2019, 5:29:14 AM12/19/19
to Django developers (Contributions to Django itself)
Hi Vineeth. 

There are a few third-party options in the space already. (Some linked from the thread I mentioned above. Others easily found via Google, "django 2fa".) I'd begin looking at those. 

I will write up a proper page for GSoC with the ideas here over the holiday period, but if you want an early start look at the "Students" section (and the following ones) from last year's page: 


Make sure you're up and running with a development environment. Get to know Trac, even if only a little bit. 

And hang-out here and on the Forum https://forum.djangoproject.co — I want to leverage that for comms in 2020. 

Above all have fun! :)
Welcome aboard

Kind Regards,

Carlton

Aaryan Dewan

unread,
Dec 23, 2019, 8:01:52 AM12/23/19
to Django developers (Contributions to Django itself)
Hi Carlton.

As I'm a Freshman in College, I'm totally new to the Open Source community. I have been using Python for two years and recently, I learned the Django Framework so I thought that it would be a great idea to contribute to it during my summer break through GSoC. 

I have read the Django contribution guide and have looked at the previous year projects. 

This year (2020) I only seem to understand 2 of the proposed projects (in this thread) namely:
  • Two Factor authentication
  • Django case studies
Since you said in your post "Are there easier things that we could take "weaker" candidates for?...", are there any proposed projects yet that are of low complexity as I'm unable to judge the complexity of the proposed projects here. If so, please let me know so that I could start working on them ASAP.

Also, you gave a great presentation at DjangoCon US :-)

Regards,

Aaryan

Carlton Gibson

unread,
Jan 2, 2020, 4:25:50 AM1/2/20
to Django developers (Contributions to Django itself)
Hi Aaryan, 

Happy New Year! Thanks for your interest, and your comment on my presentation :)

I will write up the project ideas shortly, and try to give some guidance, but I imaging the "secrets abstraction" idea wouldn't be too hard: look at, say, Vault, and the AWS offering, what are the common operations? What's the simplest API that abstracts them? (c.f. the cache API: it's just set, get, delete really...)

Have a play: it's the idea that appeals to you that will have most value. The distinction between "stronger" and "weaker" candidates is largely, how self-directed are you? Yes, some programming tasks are genuinely difficult, but for many, it's amazing how far folks can get if they are able to roll up their sleeves and try to work it out.

If you do want input early post on forum.djangoproject.com. 

Kind Regards,

Carlton
Reply all
Reply to author
Forward
0 new messages