Rename request.GET and POST, and lowercase COOKIES, FILES, and META

[Adam Johnson]

request.GET and request.POST are misleadingly named:

• GET contains the URL parameters and is therefore available whatever the request method. This often confuses beginners and “returners” alike.
  
• POST contains form data on POST requests, but not other kinds of data from POST requests. It can confuse users who are posting JSON or other formats.

Additionally both names can lead users to think e.g. "if request.GET:" means "if this is a GET request", which is not true.

I believe the CAPITALIZED naming style was inherited from PHP's global variables $_GET, $_POST, $_FILES etc. ( https://www.php.net/manual/en/reserved.variables.get.php ). It stands out as unpythonic, since these are instance variables and not module-level constants (as per PEP8 https://www.python.org/dev/peps/pep-0008/#constants ).

I therefore propose these renames:

• request.GET -> request.query_params (to match Django Rest Framework - see below)
  
• request.POST -> request.form_data
• request.FILES -> request.files
• request.COOKIES -> request.cookies
• request.META -> request.meta

Since these are very core attributes and the change would cause a huge amount of churn, I propose not deprecating the old aliases immediately, but leaving them in with a documentation note that they "may be deprecated." Perhaps they can be like url() or ifequal which came up on the mailing list recently - put through the normal deprecation path after a few releases with such a note.

Django Rest Framework already aliases GET as request.query_params in its request wrapper: https://www.django-rest-framework.org/api-guide/requests/#query_params . Therefore the name request.query_params should not be surprising. DRF also provides request.data which combines request.POST and request.FILES, and flexibly parses from different content types, but I'm not sure that's feasible to implement in Django core.

For reference, other Python web frameworks have more "Pythonic" naming:

• Bottle: request.url_args, request.forms, request.files, request.cookies, request.environ
  
• Flask: request.args, request.form, request.files, request.cookies, request.environ
• Starlette: request.query_params, request.form(), request.form()[field_name], request.cookies, scope

One more note for those who might think such core attributes should be left alone: Django 2.2 added request.headers as a way of accessing headers by name. This is convenient as it avoids the need to transform the header to the WSGI environ name. makes the code more readable, and in the process reduces the potential for bugs. I believe this proposal is in the same vein.

[Carlton Gibson]

Hmmm. I have to say I think there are areas where we could get a better ROI on our time than this.

I always took the capitalisation of GET &co to come from HTTP — I'd say that's where PHP took it from too but 🤷‍♀️

That they're a bit "unpythonic" (Meh™) is OK: they serve to remind that request was ultimately mapped from an HTTP request, and most of that is still available if you care to dig.

<form method=GET ...>

Then request.form_data -> Oh, where's my form data, if not in "form_data"? Well it's in "query_params"... Hmmm

That's no better learning experience. Folks still have to learn how HTTP maps to request properties.

> ... better ROI...

There's lots we might take from DRF. The one that's come up before, and which for work was began, but only began, was content negotiation.

I'd rather see work/energy/effort spent there.

Maybe we'd have different names if we began today. But this would be very disruptive.

We've just had a discussion re url() suggesting that "deprecated but not really" is an error on our part, and having two ways to do things definitely isn't "pythonic".

So I'm inclined towards the range between -1 and -0 — but I haven't had my coffee yet. 😝

Kind Regards,

Carlton

[Jure Erznožnik]

I agree. If anything, I've always had issues with GET & POST being different entities in the first place, but never with their names. I would really like to see an entity combining both of them. THAT one, maybe the preferred way of doing so, would be lowercase, but RAW representations of incoming data, I for one like them being upper case. Always makes me think twice before playing with them.

The content negotiation thingy is also something I would like to see more time invested in: I have a project where I have to hack & slash DRF's implementation in order to get what I want, but perhaps I'm tackling the issue incorrectly. But that's beside the point. People will always try to do stuff framework developers didn't think of. What's important is to give them a platform where they can do so easily.

LP,
Jure

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/bc60e809-6390-44fc-a07a-ed6e0f9eef86%40googlegroups.com.

[Raffaele Salmaso]

For me it's a good move (the best would be to have request.data as with DRF).

I've done this kind of "upgrade" from Django request.{GET,POST} to DRF request.{query_params,data} and it was quite trivial (search and replace).

At the same time it forced me to check the code for correctnes, and I found a couple of (unrelated) bugs.

For me I'll add the alias as soon as possible (3.1?), use them in documentation, and start the deprecation after an LTS release (4.0? 5.0?), so there would be plenty of time to upgrade.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM3%2BUucViDezhkWrFsk6ZsKViWjOgtA5aBm9pnzozdc%2Beg%40mail.gmail.com.

--

| Raffaele Salmaso
| https://salmaso.org
| https://bitbucket.org/rsalmaso
| https://github.com/rsalmaso

[Steven Mapes]

I also thought it was a taken more form HTTP rather than PHP but if this is changed please do not use request.form_data as that is also misleading as you can POST many more sources than form data such as with APIs. post_data would be much clearer.

[Steven Mapes]

Combined them would be very very bad and you would have the same problems as with $_REQUEST in PHP where you have to decide which one wins as you can make a POST to a URL with querystring where one of the parameters uses the same name as a element of the POST and but where the value is different. It's bad practice but it's valid

To unsubscribe from this group and stop receiving emails from it, send an email to django-d...@googlegroups.com.

[Adam Johnson]

I always took the capitalisation of GET &co to come from HTTP — I'd say that's where PHP took it from too but 🤷‍♀️

Yes GET and POST are capitalized in HTTP, but then COOKIES is not (Set-Cookie / Cookies headers), and FILES and META aren't direct references to anything in HTTP.

Also I'm not sure it's a particularly good argument. Capitalization inside Python should be based upon the conventions of Python, not the conventions of the system the code talks to. For example, we have SELECT ... FOR UPDATE in SQL, but .select_for_update() in the ORM.

 

Hmmm. I have to say I think there are areas where we could get a better ROI on our time than this. 

I think if we took the amount of time spent by users due to the confusion of GET/POST, divided by the amount of time to make the code and docs changes, it would come out with a relatively good ratio.

My inspiration (or "the final straw") for making this proposal was a recent forum question with confusion on the meaning of POST: https://forum.djangoproject.com/t/ajax-post-to-view-not-displaying-content/2034 . I'm pretty sure I saw another one in the past month but can't find it But it's an issue I've seen repeatedly. I know I've even spent time figuring out how I'd get the query params during a POST request.

please do not use request.form_data as that is also misleading as you can POST many more sources than form data such as with APIs. post_data would be much clearer.

Steven - I think you're confused by the current name. request.POST *only* contains POST'd form data. It does not contain other POST'd data, such as JSON bodies. So perahps that's another point for the name change?

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/b23d3b4a-9ff4-42ed-a08b-594f185b8d3b%40googlegroups.com.

--

Adam

[Steven Mapes]

Ah yes that's true, totally forgot about request.body  just then even though I was using it a few days ago. So yes renaming it would help in that regard as I do remember seeing a few S/O question where it's confused people in the past.

Mr Steven Mapes Software Development and Solutions
	E: st...@jigsawtech.co.uk P: 07974220046 W: https://uk.linkedin.com/in/stevemapes/

This E-Mail and its contents are confidential, protected by law and legally privileged. Only access by the addressee is authorised. Any liability (in negligence, contract or otherwise) arising from any third party taking any action or refraining from taking any action on the basis of any of the information contained in this E-Mail is hereby excluded to the fullest extent of the law. In the event that you are not the addressee, please notify the sender immediately. Do not discuss, disclose the contents to any person or store or copy the information in any medium or use it for any purpose whatsoever.

You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/Kx8BfU-z4_E/unsubscribe.

To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM19Ay4oAf26QaA8EyixrzDXHpXmFgtWijVf_%3D30W-gVBA%40mail.gmail.com.

[Tom Carrick]

> This often confuses beginners and “returners” alike.

I'm neither a beginner nor returner and I still have to remind myself that request.POST doesn't contain my JSON data.

I think it's a positive change. I agree there are things that would provide a better ROI, but people have to be willing to do them. I also think there's no harm in tackling things with a lower ROI.

Tom

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/137322AE-3BF9-4B5F-B80D-7AC7EF298A6C%40getmailspring.com.

[Ryan Hiebert]

I can agree that there might be better things to do, but this falls into the category of warts for me, so I'm +1. Having the name be pythonic is a plus, but avoiding a beginner footgun is better.

And I believe the conventions used did indeed come from PHP. There even used to be a parallel to `$_REQUEST`. https://code.djangoproject.com/ticket/18659

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAHoz%3DMZgb-6o0-BMoYHrjwnF6YGBBcFO8A%3DBDSRJFvNi%2BL1eWw%40mail.gmail.com.

[Riccardo Magliocchetti]

Hello,

On 05/05/20 23:26, Adam Johnson wrote:
> request.GET and request.POST are misleadingly named:
>

> - GET contains the URL parameters and is therefore available whatever

> the request method. This often confuses beginners and “returners” alike.

> - POST contains form data on POST requests, but not other kinds of data

> from POST requests. It can confuse users who are posting JSON or other
> formats.

[...]>

> I therefore propose these renames:
>

> - request.GET -> request.query_params (to match Django Rest Framework -
> see below)
> - request.POST -> request.form_data
> - request.FILES -> request.files
> - request.COOKIES -> request.cookies
> - request.META -> request.meta

Yes please, query_params to match DRF makes a lot of sense to me!
These days i use mostly DRF views, still have to maintain plain Django code so
anything reducing the difference is much appreciated

> Since these are very core attributes and the change would cause a huge
> amount of churn, I propose not deprecating the old aliases immediately, but
> leaving them in with a documentation note that they "may be deprecated."
> Perhaps they can be like url() or ifequal which came up on the mailing list
> recently - put through the normal deprecation path after a few releases
> with such a note.

Given these are on most tutorial / book ever published on paper and on the
internet I'd just make them an alias of the new proposed names and kept for
quite a few.

Thanks

--
Riccardo Magliocchetti
@rmistaken

http://menodizero.it

[אורי‎]

Hi,

I also prefer lowercase names since uppercase is usually reserved for constants in Python. The names of the requests ("GET" / "POST") can be used in strings to check the request method (uppercase). But there is no sense in using uppercase names for variables.

אורי

u...@speedy.net

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM3%2BUucViDezhkWrFsk6ZsKViWjOgtA5aBm9pnzozdc%2Beg%40mail.gmail.com.

[Fran Hrženjak]

+1 request.query_params

If request.method == "POST":

    thing = request.GET.get("thing")

That’s just silly.

+1 request.data

We shouldn’t be POSTists, there is also PUT and PATCH.

And apparently GET requests can also have a body:  

https://stackoverflow.com/questions/978061/http-get-with-request-body

+1 No combining

Because.

+1 For 27 years deprecation cycle.

But in reality vast majority of codebases will only need a simple search and replace, no? (Famous last words I guess...)

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CABD5YeEFsbneB%3DfPKpE90cNukOUAKZejfp4Yo1N712mkmNVQHQ%40mail.gmail.com.

--

LP,
Fran

[Adam Johnson]

+1 request.data

We shouldn’t be POSTists, there is also PUT and PATCH.

Would just like to point out - that's not the proposal. The proposal is to rename the existing request.POST to request.form_data, which is based on parsing application/x-www-form-urlencoded data from request.body. Browsers only send such data in the body on POST.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAM2o%3DwPf%2B_p2vh%3Ds9C_O03jebgQ%2Bd%2Be9oODnLoJ_HdM58HSJqg%40mail.gmail.com.

--

Adam

[Florian Apolloner]

On Wednesday, May 6, 2020 at 5:49:40 PM UTC+2, Adam Johnson wrote:

Would just like to point out - that's not the proposal. The proposal is to rename the existing request.POST to request.form_data, which is based on parsing application/x-www-form-urlencoded data from request.body. Browsers only send such data in the body on POST.

I am not convinced. As Carlton point out, forms with method="GET" are legitimate and wouldn't end up in `form_data` which is imo quite confusing. Do we have any other options? (Hard I know, but…)

[Fran Hrženjak]

On Wed, 6 May 2020 at 17:49, Adam Johnson <m...@adamj.eu> wrote:

+1 request.data

We shouldn’t be POSTists, there is also PUT and PATCH.

Would just like to point out - that's not the proposal. The proposal is to rename the existing request.POST to request.form_data, which is based on parsing application/x-www-form-urlencoded data from request.body. Browsers only send such data in the body on POST.

That is a fair point, but also AJAX/JS can do those other things, including POSTing JSON, as mentioned above.

Let’s use something more generic then `form_data`, more focused on the actual mechanics of data transfer (data in body vs data in query params) rather than focused on the intent (form, query, navigation, API…) Because, also as mentioned above, users will have to understand these things anyway.

Something like `request.data` can hold decoded fom data just like `request.POST` does now. In the future, we can include decoded JSON in there too, so we don't have to `json.loads(request.body)` any more. Also XML anyone much? :) Basically what DRF does with parsers.

This is, of course, way outside the scope here, but perhaps it is worth keeping the names in line with the long term goals (if something like decoding JSON and other formats, DRF-style, is a goal, IMHO it shold be).

request.data is also somewhat wague, but at least not misleadng. Maybe request.body_params, request.decoded_body, request.body_data, request.content? An argument for `request.data` is that is will end up in `form.data` in the usual use case, so a nice symmerty there.

[Carlton Gibson]

On 7 May 2020, at 22:03, Florian Apolloner <f.apo...@gmail.com> wrote:

forms with method="GET" are legitimate and wouldn't end up in `form_data` forms with method="GET" are legitimate and wouldn't end up in `form_data` which is imo quite confusing. Do we have any other options?

Would there be an issue with `.post`? 

Bar the suggested `query_params` change, the proposal would then amount to “lowercase these names”.

Then there’s the additional “Also rename `get` to `query_params`, to match DRF”. 

[Ryan Hiebert]

It seems that many of the people who want a different name are not understanding that this name is going to be exclusively for dealing with POST forms that are using the standard `x-www-form-urlencoded` media type, and won't be attempting to interpret any other media type. Some generic names, like "data" might be really good for a generic interface that interprets various content types, like Django Rest Framework has, but this isn't where Django is right now, and shouldn't be part of this question. All we are trying to accomplish is to give access to POST form data with the normal form submission content type that is not already accessible by the `query_params`, which are of the url parameters (like a GET form). This is just matching the current behavior of request.POST with a different name.

It is my opinion that the suggested `query_params` and `form_data` are great. While `form_data` can be slightly ambiguous if you're thinking of GET forms, once you realize that GET forms will show up in `query_params`, and are available even in a POST request, then all `form_data` would have left are the default POST forms. So +1 to the names as well.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/66AB2042-8344-436B-A86A-2B9060BFAA17%40gmail.com.

[Aymeric Augustin]

Hello,

This is quite disruptive — says the guy who suggested to remove request.REQUEST a few years back — but I think we could go ahead and do it. The names proposed by Adam are good.

I read the concerns about <form method="GET"> (e.g. for search forms) where form data ends up in query_params and not form_data. I'd call this a feature :-) Here's why:

- Users who don't know that GET queries mustn't have side effects and who don't want to bother will have an incentive to use a POST query to get the data in form_data; this is the safe option with regards to CSRF;

- Users who understand the safety model of browsers and who can determine that their view isn't vulnerable to CSRF will understand that the form data is in query_params, because that's where it is really.

Also, looking at various names between `form_data` and `form_data_decoded_from_x_www_form_urlencoded_post_body` (the technically correct name), I believe that `form_data` is the best compromise between clarity and terseness.

Best regards,

-- 

Aymeric.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM3%2BUucViDezhkWrFsk6ZsKViWjOgtA5aBm9pnzozdc%2Beg%40mail.gmail.com.

[Florian Apolloner]

On Saturday, May 9, 2020 at 1:41:41 PM UTC+2, Aymeric Augustin wrote:

- Users who don't know that GET queries mustn't have side effects and who don't want to bother will have an incentive to use a POST query to get the data in form_data; this is the safe option with regards to CSRF;

This is imo the best argument in this thread and convinces me a little bit :D But I still think we should accompany this with documentation that explains why GET-powered forms do not show up in form_data. What irks me a little bit is that request.GET/POST nicely stood out in code -- ie I could find user input handling on a first glance.

As for disruption, can we just alias them for now and drop GET/POST from the docs and not immediately start with warnings?

Cheers,

Florian

[Tim Allen]

I'm tentatively +1 on this change... but...

While it may be a straightforward operation to do a case sensitive search and replace in most code bases, it is not trivial to do so on the rest of the web. If we're going to do this, we must realize that this will invalidate thousands of examples, tutorials, and answers in communities such as Stack Overflow, the Django Forum, the Django Girls Tutorial, and the archive of this Google Group. It has been a few years now since Django 2.0, and I still find myself frequently helping junior developers amend their PRs to move to `path` and `re_path` instead of `url`, which they copypasta'd from somewhere. I was also for that change, but as a community, we need to be cognizant of the fact that as its ambassadors, we are going to have a fair amount of work to do to properly spread the word about a change like this.

There are many possible responses to why that shouldn't be happening in a perfect world, unfortunately, we have to exist in this reality! (I'm hard at work at the unreality PR for Django 5.0.)

So let's make sure we consider the full impact of this change before we do it. I have some pet peeves about Django naming choices, but where do we stop? Two examples:

* createsuperuser -> create_super_user, queryset -> query_set,  for readability and to be more Pythonic

* HttpResponse should be HTTPResponse, like DjangoJSONEncoder, for example. (From PEP-8: "Note: When using acronyms in CapWords, capitalize all the letters of the acronym. Thus HTTPServerError is better than HttpServerError.")

Something to consider for the future about when the churn is worth it, and when it isn't. Thanks, y'all.

[Tobias McNulty]

Tim your point is well made. I'm left  with the feeling that an imperfect convention is really not so bad, and possibly not worth the huge number of global developer hours it would cost to effect such a change.

Plus (I make this argument mostly in jest), the new names, because they are more Pythonic, probably would not stick out so well / not be as easy to search for. :D

Tobias

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/5022af59-4280-4dca-a33e-d7ec79bc131b%40googlegroups.com.

[Adam Johnson]

I finally got around to creating a ticket: https://code.djangoproject.com/ticket/32259#ticket

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMGFDKQwPiaS2UtCnaTQ-Q0HL7ZAP1iQyWDraECEiDKgj1gnsA%40mail.gmail.com.

--

Adam