Should Django handle uploaded files in PUT as well as POST?
Malcolm Box
I'm working on a REST-full API that allows an uploaded file to be PUT to a URL. I also need to support the X-Http-Method-Override header to turn a POST request into a PUT for clients (Flash, mobile etc) that can't do PUT.
At the moment I've got the API working with POST requests, and am now moving over to PUT + implementing the override.
I'm running into several places where it seems Django processes things differently if they come from a POST or a PUT. The most obvious is the HttpRequest (and derived classes) handling of _load_post_and_files(). This parses a multipart upload into a set of files in request.FILES and the rest of the POST data into request.POST, whenever request.POST is accessed.
However a similar transformation isn't applied to PUT requests - so accessing request.FILES finds nothing there, and there's no equivalent request.PUT.
Django-rest-interface (http://code.google.com/p/django-rest-interface/wiki/RestifyDjango) gets round this by looking for PUT requests, then calling request._load_post_and_files() directly to populate the FILES member.
This seems very brittle if anything goes near request.POST before the rest interface runs, since the request.POST access will parse the uploaded data which is then not available for reparsing - error messages such as "Cannot read more than the available bytes from the HTTP incoming data" then pop up.
It seems to me that Django should process POST and PUT requests the same - i.e. the request.FILES attribute should be initialised for either if a multipart content-type is detected.
Have I fundamentally misunderstood how this stuff should work?
Cheers,
Malcolm
Russell Keith-Magee
On first inspection, I don't think you've got anything fundamentally
wrong. Support for PUT and the other 'exotic' HTTP request methods is
one area where Django support is a little bit patchy.
This is mostly an artefact of the vast majority of browsers providing
exactly zero support for requests other that GET and POST. As a result
Django is (by necessity) really good at handling GET and POST; PUT,
DELETE et al are all essential parts of the full HTTP spec, but
support in Django isn't quite as comprehensive.
This is one of those areas where patches are definitely welcome.
Yours,
Russ Magee %-)
Malcolm Box
On Sat, Jan 16, 2010 at 6:32 AM, Malcolm Box <malco...@gmail.com> wrote:
<snip>
> It seems to me that Django should process POST and PUT requests the same -
> i.e. the request.FILES attribute should be initialised for either if a
> multipart content-type is detected.
>
> Have I fundamentally misunderstood how this stuff should work?
On first inspection, I don't think you've got anything fundamentally
wrong. Support for PUT and the other 'exotic' HTTP request methods is
one area where Django support is a little bit patchy.
This is mostly an artefact of the vast majority of browsers providing
exactly zero support for requests other that GET and POST. As a result
Django is (by necessity) really good at handling GET and POST; PUT,
DELETE et al are all essential parts of the full HTTP spec, but
support in Django isn't quite as comprehensive.
This is one of those areas where patches are definitely welcome.
Which raises the question of what the correct behaviour should be, so the bug can report that it's not happening and the patch can implement.
As a first pass:
"On a PUT request, Django should populate request.PUT and request.FILES in the same way as for POST requests. FILES will be initialised if the content is multipart, and the PUT dictionary will have key/value pairs if the uploaded content included url-encoded form entries.
If the format of the datastream is not understood, it will be left in _raw_post_data as previously, and req.FILES and req.PUT initialised to empty hashes."
Does this sound reasonable as a bug/feature description?
Cheers,
Malcolm
Russell Keith-Magee
> On Sat, Jan 16, 2010 at 4:58 AM, Russell Keith-Magee
> <freakb...@gmail.com> wrote:
>>
>> On Sat, Jan 16, 2010 at 6:32 AM, Malcolm Box <malco...@gmail.com>
>> wrote:
>> <snip>
>> > It seems to me that Django should process POST and PUT requests the same
>> > -
>> > i.e. the request.FILES attribute should be initialised for either if a
>> > multipart content-type is detected.
>> >
>> > Have I fundamentally misunderstood how this stuff should work?
>>
>> On first inspection, I don't think you've got anything fundamentally
>> wrong. Support for PUT and the other 'exotic' HTTP request methods is
>> one area where Django support is a little bit patchy.
>>
> Well that's good to know!
>
>>
>> This is mostly an artefact of the vast majority of browsers providing
>> exactly zero support for requests other that GET and POST. As a result
>> Django is (by necessity) really good at handling GET and POST; PUT,
>> DELETE et al are all essential parts of the full HTTP spec, but
>> support in Django isn't quite as comprehensive.
>>
>> This is one of those areas where patches are definitely welcome.
>>
> I'm happy to provide some patches - presumably a bug report + patch is the
> way to go.
As always.
> Which raises the question of what the correct behaviour should be, so the
> bug can report that it's not happening and the patch can implement.
>
> As a first pass:
>
> "On a PUT request, Django should populate request.PUT and request.FILES in
> the same way as for POST requests. FILES will be initialised if the content
> is multipart, and the PUT dictionary will have key/value pairs if the
> uploaded content included url-encoded form entries.
>
> If the format of the datastream is not understood, it will be left in
> _raw_post_data as previously, and req.FILES and req.PUT initialised to empty
> hashes."
>
> Does this sound reasonable as a bug/feature description?
Sounds reasonable to me. I can't think of any obvious reason that the
handling for PUT should differ from POST.
Yours,
Russ Magee %-)
Malcolm Box
On Mon, Jan 18, 2010 at 8:14 AM, Malcolm Box <malco...@gmail.com> wrote:
> On Sat, Jan 16, 2010 at 4:58 AM, Russell Keith-Magee> Which raises the question of what the correct behaviour should be, so the
> bug can report that it's not happening and the patch can implement.
>
> As a first pass:
>
> "On a PUT request, Django should populate request.PUT and request.FILES in
> the same way as for POST requests. FILES will be initialised if the content
> is multipart, and the PUT dictionary will have key/value pairs if the
> uploaded content included url-encoded form entries.
>
> If the format of the datastream is not understood, it will be left in
> _raw_post_data as previously, and req.FILES and req.PUT initialised to empty
> hashes."
>
> Does this sound reasonable as a bug/feature description?
Sounds reasonable to me. I can't think of any obvious reason that the
handling for PUT should differ from POST.
Masklinn
Does that mean the request object could have arbitrary HTTP_METHOD attributes when it can parse the content as keys:values? At least if the MIME of the request is `application/x-www-form-urlencoded`?
I mean as far as I know nothing forbids creating custom HTTP methods (indeed, that's what e.g. WEBDAV does) and some of these custom methods could very well have the same properties as POST.
Malcolm Box
On 18 Jan 2010, at 03:04 , Russell Keith-Magee wrote:
>
> On Mon, Jan 18, 2010 at 8:14 AM, Malcolm Box <malco...@gmail.com> wrote:
>> On Sat, Jan 16, 2010 at 4:58 AM, Russell Keith-Magee
>> Which raises the question of what the correct behaviour should be, so the
>> bug can report that it's not happening and the patch can implement.
>>
>> As a first pass:
>>
>> "On a PUT request, Django should populate request.PUT and request.FILES in
>> the same way as for POST requests. FILES will be initialised if the content
>> is multipart, and the PUT dictionary will have key/value pairs if the
>> uploaded content included url-encoded form entries.
>>
>> If the format of the datastream is not understood, it will be left in
>> _raw_post_data as previously, and req.FILES and req.PUT initialised to empty
>> hashes."
>>
>> Does this sound reasonable as a bug/feature description?
>
> Sounds reasonable to me. I can't think of any obvious reason that the
> handling for PUT should differ from POST.
Does that mean the request object could have arbitrary HTTP_METHOD attributes when it can parse the content as keys:values? At least if the MIME of the request is `application/x-www-form-urlencoded`?
I mean as far as I know nothing forbids creating custom HTTP methods (indeed, that's what e.g. WEBDAV does) and some of these custom methods could very well have the same properties as POST.
I don't see why not, although that's not what I'm planning to submit patches for - I'm just aiming to cover PUT ie to complete the support for the core HTTP methods that are allowed entity bodies.
So the HttpRequest object will grow a new PUT attribute, but nothing else.
Malcolm
Russell Keith-Magee
FYI, Section 5.1.1 of RFC 2616 specifically allows the definition of
extension methods.
I'm open to suggestions on how Django's handling of arbitrary
extension methods can be improved. As it stands, request.REQUEST will
contain all the query arguments that have been provided in a request,
regardless of the request method. This can already be used to support
extension request methods.
If you're looking for something more than this, feel free to make a
specific proposal. Personally I'm not sure I agree that having
request.FOOBAR is a good idea. There are many names that are legal
request methods, but aren't legal variable names, so we would need to
have a mapping scheme. We would also end up with a situation where
extension methods would only produce attributes when used, so
hasatttr(request, 'POST') would always succeed, but hasattr(request,
'FOOBAR') would only succeed if request.method == 'FOOBAR'.
However, as Malcolm notes - this is a separate issue to handling FILES
on a PUT request, so it should be handled as a separate ticket and
mailing list thread.
Yours,
Russ Magee %-)