Add support for multiple file fields

[Johannes Hoppe]

Hi there!

I already created a ticket regarding this matter, but I think this thicket requires some discussion prior to crafting a solution.

https://code.djangoproject.com/ticket/28554

The django.db.models.FileField currently allows only to store a single file URL.
This behavior seems outdated considering that input[type=file] supports a multiple attribute, which is supported by all major Browsers.
See: ​https://www.w3schools.com/tags/att_input_multiple.asp

I would suggest to have a similar attribute on the database field itself, as well as on the form field and widget.

The major point for discussion would be how to store the data. The easiest would be to coma separate the paths in a single char field. The major concern here would be the lack of atomicity. For databases like Postgres I would be sensical to use the ArrayField as an underlying structure. Another way for serialisation would be the use of JSONs in a char field.

All solutions based on a text or char field have an issue related to the compatibility with the current FileField which is limited to 100 characters. The type would need to be migrated to a text type or the length would need to be increased. The latter options bares the question to which point.

Another approach would be to defer the storage issue and only provide support for multiple files on form level.
This would mean to add a multiple attribute to the form field and widget. Users would be able to create forms with multiple files in a single field and handle storage according to their preference.

[Johannes Hoppe]

I tried to exclude my personal opinion and preference from the ticket description, to have an open discussion.
So here goes my personal opinion:

It seems odd to me that the FileField is limited to 100 characters. I could not find any reference to why the field was limited in the first place. Furthermore I do not know of any file system with a 100 char limitation nor are URLs limited to only 100 chars.

Therefore I would suggest basing the FileField upon the TextField.

I would recommend splitting the issue tho. I would first add support form multiple files to form. This is a nice feature in itself and requires little work and good documentation. I would presume that such a feature would spawn a larger discussion on how to store those files. As multiple model instances? As an Array in Postgres? As CSV or JSON in a TextField? I think we can figure this out later or not at all if we don't come to a conclusion.

I would love to work on that. I do maintain [django-s3file](​https://github.com/codingjoe/django-s3file/) and [django-stdimage](​https://github.com/codingjoe/django-stdimage/) and have some experience with custom the FileInput and FileField. Let me know if I can be of any help.

Cheers
-joe

[Tom Forbes]

(I wrote a reply earlier and sent it, but it appears to have disappeared into /dev/null. Apologies if it comes through at some later date.)

It seems adding a multiple kwarg and retrofitting it onto the existing FileField might not be the best idea. It might cause confusion with existing FileField APIs like size and name which don't make sense with multiple files.

We could add a MultiFileField that had a files attribute, which returns a list of individual objects analogous to a single FileField?

I guess that MultiFileField could be hidden between a 'multiple' keyword argument, but I'm not sure a migration from true to false would be possible which could also be confusing.

 

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/2b9bb49f-1cbc-4ea6-94df-774dfad43114%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Melvyn Sopacua]

I don't think a MultiFileField belongs in the model layer.
You're violating normalization principles making updates,
removals, counts etc, needlessly complex.

From the ModelForm perspective, I can certainly appreciate
a multi file form field that I can tell to store each file as a new
model instance referencing 'app_label.modelname.filefieldname'.

What could belong in the model layer is an ArchiveField, that
accepts multiple files, puts them in a single archive and stores
metadata (contents+count+whatever) in it's metadata_field
(similar approach as width and height for ImageField).

> email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.

> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/2b9bb49f-1cbc-4ea6-94df-774dfad43114%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.

> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/django-developers/CAFNZOJPrZZDJHhXpGKi9nbWA6RKnZBfxnHP1GFnt0Bf1HhFjmA%40mail.gmail.com.

>
> For more options, visit https://groups.google.com/d/optout.

--
Melvyn Sopacua

[Adam Johnson]

I agree with Melvyn on the first point. I don't think Django should include model fields that encourage denormalization by default (apart from JSONField etc which are built into the DB).

A forms.MultiFileField would make sense but to make storage easy it should somehow translate into multiple rows in the DB.

ArchiveField sounds a bit too specific for Django core, the most common case for uploading multiple files would probably be to access them individually, which it would prevent.

> email to django-developers+unsubscribe@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.

> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/2b9bb49f-1cbc-4ea6-94df-774dfad43114%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to django-developers+unsubscribe@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.

> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAFNZOJPrZZDJHhXpGKi9nbWA6RKnZBfxnHP1GFnt0Bf1HhFjmA%40mail.gmail.com.
>
> For more options, visit https://groups.google.com/d/optout.

--
Melvyn Sopacua

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CA%2Bgw1GWrv3pKTva9SxzycO%2Bc88EuLkKXkG%3DvDUdU2uoCHAt87g%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Adam

[Johannes Hoppe]

Interesting, I love the discussion. I anticipated, that there are may different solutions and opinions regarding this topic. I will limit the implementation to `django.forms` for now and hope, that more people will voice their opinions regarding a model implementation once this becomes available to a broader audience.

Thanks everyone :)

--

Johannes Hoppe

Fon: +49 331 2812 9869 1

Fax: +49 331 2812 9869 9

www.johanneshoppe.com

Lennéstr. 19

14469 Potsdam

USt-IdNr.: DE284754038

You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/IU8L9Gc6LUI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM2jX7JjiF3T%2Br0jza_ThYa0_fdvftPMhLgnXE-PQ2aoCA%40mail.gmail.com.

[Melvyn Sopacua]

On Sat, Sep 2, 2017 at 10:37 AM, Adam Johnson <m...@adamj.eu> wrote:

> ArchiveField sounds a bit too specific for Django core, the most common case
> for uploading multiple files would probably be to access them individually,
> which it would prevent.

Yeah, was on the fence on that one myself. The use cases are very domain
specific, the obvious one being a backup application, but also design specs
or legal documents that site staff downloads and never actually views on site
individually. The upside of the field being that you have control of the archive
format and/or can encrypt the archive with user specific key before sending.

I researched it for an upcoming project and all the hooks are in place to
create such a model and you're right, it goes beyond "batteries included".
--
Melvyn Sopacua

[Johannes Hoppe]

There is another thing I just come across while working on the form implementation.

I realised, that there are multiple ways to extend the ClearableFileInput to support multiple files.

More specifically the clear checkbox. Should there be only one checkbox to clear all files or individual once to clear each file of the set of files inside contained by the input.

I think there are two think to keep in mind. This case is only for initial values. This case might be rare, since there is no model representation for that case. Hence, once would need to manually build a view and load those images as initial value for once input. This seems an odd UX choice.

Secondly there is the question of consistency. If you upload a single of a list of files into a file widget, it would replace the current list of files. So it is all or nothing. I would recommend not to deviate from this approach when it comes to clearing initial values. Have separate checkboxes might also prove to be a UX issue, since users would need to check them all individually.

What do you guys think? Am I on the right track regarding this issue?

--

Johannes Hoppe

Fon: +49 331 2812 9869 1

Fax: +49 331 2812 9869 9

www.johanneshoppe.com

Lennéstr. 19

14469 Potsdam

USt-IdNr.: DE284754038

--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/IU8L9Gc6LUI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CA%2Bgw1GWTn%2B8f_bUV7mmJ8gEFodwUrtK_yzF_zEWA%2Bv5ew332gA%40mail.gmail.com.

[Johannes Hoppe]

OK, I drafted an implementation for django.forms.FileField to support `multiple` as an argument.

https://github.com/django/django/pull/9011

I would appreciate some feedback. If you like the design, I’ll try to add documentation and better tests tomorrow.

Thanks

-Joe :)

--

Johannes Hoppe

Fon: +49 331 2812 9869 1

Fax: +49 331 2812 9869 9

www.johanneshoppe.com

Lennéstr. 19

14469 Potsdam

USt-IdNr.: DE284754038

On 2. Sep 2017, 11:10 +0200, Melvyn Sopacua <m.r.s...@gmail.com>, wrote:

--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/IU8L9Gc6LUI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CA%2Bgw1GWTn%2B8f_bUV7mmJ8gEFodwUrtK_yzF_zEWA%2Bv5ew332gA%40mail.gmail.com.

[Johannes]

Ok,

My patch is ready. Anyone cares to review to get it in before the deadline this weekend?

https://github.com/django/django/pull/9011

Thanks

-Joe

--

Johannes Hoppe

Fon: +49 331 2812 9869 1

Fax: +49 331 2812 9869 9

www.johanneshoppe.com

Lennéstr. 19

14469 Potsdam

USt-IdNr.: DE284754038