[Django] #32326: Proposing a more concrete example of 'Streaming large CSV files'

Django

unread,

Jan 5, 2021, 10:37:38 PM1/5/21

to django-...@googlegroups.com

#32326: Proposing a more concrete example of 'Streaming large CSV files'
-------------------------------------+-------------------------------------
Reporter: niauah | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: | Version: 3.1
Documentation |
Severity: Normal | Keywords: streamingresponse
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
In the current document, the 'large CSV' is demonstrated through a simple
65536-element list, while the streaming feature is useful combined with a
Python generator function. Here I propose a slightly modified example with
a Python `yield` function.

--
Ticket URL: <https://code.djangoproject.com/ticket/32326>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,

Jan 7, 2021, 1:10:37 AM1/7/21

to django-...@googlegroups.com

#32326: Proposing a more concrete example of 'Streaming large CSV files'
-------------------------------------+-------------------------------------
Reporter: niauah | Owner: nobody

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Mariusz Felisiak):

* status: new => closed
* resolution: => wontfix

Comment:

Thanks for this proposition, however the current example is sufficient for
Django documentation, IMO.

--
Ticket URL: <https://code.djangoproject.com/ticket/32326#comment:1>

Django

unread,

Jan 7, 2021, 3:36:30 AM1/7/21

to django-...@googlegroups.com

#32326: Proposing a more concrete example of 'Streaming large CSV files'
-------------------------------------+-------------------------------------
Reporter: niauah | Owner: nobody

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by niauah):

Replying to [comment:1 Mariusz Felisiak]:

> Thanks for this proposition, however the current example is sufficient

for Django documentation, IMO. We already have an extensive docs and we
cannot document each use case and non-Django caveats.

Thanks for the reply. I understand that the document cannot cover detailed
non-Django caveats (e.g. issues on Python methods). If necessary, I can
revise the patch and delete them.

However, in the original example:
{{{
rows = (["Row {}".format(idx), str(idx)] for idx in range(65536))
pseudo_buffer = Echo()
writer = csv.writer(pseudo_buffer)
response = StreamingHttpResponse((writer.writerow(row) for row in
rows),
content_type="text/csv")
}}}

the 65536-row list is fully traversed at declaration, not in a 'call-by-
need' fashion.

IMO, providing an example combined with yield method would be more
illustrative of the 'streaming' nature.

--
Ticket URL: <https://code.djangoproject.com/ticket/32326#comment:2>

Django

unread,

Jan 7, 2021, 3:37:00 AM1/7/21

to django-...@googlegroups.com

#32326: Proposing a more concrete example of 'Streaming large CSV files'
-------------------------------------+-------------------------------------
Reporter: niauah | Owner: nobody

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------