Can't get Uploaded File into DictReader
jeff
I'm trying to upload a CSV file specified by the user in a form and
then parse it using DictReader. Everything seems to be working until
I initialize DictReader with the uploaded file. As soon as I try to
access any value in the row, it throws an odd exception. Am I doing
something wrong or is the UploadedFile object incompatible with
DictReader?
if request.POST:
form = ImportFileForm( request.POST, request.FILES )
if form.is_valid():
try
csvfile = request.FILES['filename'].read() # reads entire file
reader = csv.DictReader( csvfile, 'r' )
for row in reader:
group_name = str.strip( row['Group Name'] )
#
throws exception "Group Name"
except Exception, e:
print e
else:
form = ImportFileForm()
return render_to_response('admin_import_from_excel.html', {'form':
form,}, RequestContext(request) )
Thanks in advance for any assistance.
Jeff
Bill Freeman
not a string. It is also not like open(), so it doesn't take the 'r' argument.
You could try:
...
reader = csv.DictReader(request.FILES['filename'])
...
but you may have to normalize the line endings.
I suspect that DictReader process the file as an iterator of lines, so
you can probably fix line endings, if necessary, using a generator, or
perhaps even a generator expression.
Bill
> --
> You received this message because you are subscribed to the Google Groups "Django users" group.
> To post to this group, send email to django...@googlegroups.com.
> To unsubscribe from this group, send email to django-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
>
>
bruno desthuilliers
>
> I suspect that DictReader process the file as an iterator of lines
Don't suspect, make sure:
http://docs.python.org/library/csv.html#csv.reader
"""
csvfile can be any object which supports the iterator protocol and
returns a string each time its next() method is called — file objects
and list objects are both suitable
"""
bruno desthuilliers
On Feb 10, 2:50 am, jeff <jwils2...@gmail.com> wrote:
> I'm trying to upload a CSV file specified by the user in a form and
> then parse it using DictReader. Everything seems to be working until
> I initialize DictReader with the uploaded file. As soon as I try to
> access any value in the row, it throws an odd exception.
Exceptions tracebacks are here to help sorting out what went wrong and
where it went wrong. So please post the full traceback.
> Am I doing
> something wrong or is the UploadedFile object incompatible with
> DictReader?
>
> if request.POST:
> form = ImportFileForm( request.POST, request.FILES )
> if form.is_valid():
> try
> csvfile = request.FILES['filename'].read() # reads entire file
> reader = csv.DictReader( csvfile, 'r' )
Did you read the FineManual(tm) for the csv module ?
http://docs.python.org/library/csv.html#csv.reader
"""
csvfile can be any object which supports the iterator protocol and
returns a string each time its next() method is called — file objects
and list objects are both suitable.
"""
Here you are passing a (possibly huge) string. Did you ever try to
iterate over a string ? If yes, you should by now understand what
happens. Else, well, fire up a Python shell and run the following
snippet:
s = "A string is indeed a sequence of characters\nSuprising, isn't
it ?"
for char in s:
print char
Starting to understand what happens ?-)
As a side note, reading the whole file in memory is a bad idea anyway
- unless you're ok to eat most of your ram.
> for row in reader:
> group_name = str.strip( row['Group Name'] )
A bit OT, but "row['Group Name'].strip()" works as well (assuming
'row' has a 'Group Name' key of course), and is more idiomatic.
> #
> throws exception "Group Name"
I don't think the exception is of type "Group Name". I guess what you
have is a KeyError. Did you try to inspect what "row" actually looked
like ? I bet you'd have kind of a surprise !-)
> except Exception, e:
> print e
By all mean never do something like this in production code. If you
can't properly handle an exception, just leave it alone - or at least
re-raise it.
Ok, to make a long story short, you can pass the UploadedFile object
itself to the DictReader, but as Bill mentioned, beware of line-ending
issues. Another (more robust) solution might be to wrap the uploaded
file in some custom file-like object that is able to handle the issue,
reading the uploaded file by chunks (cf the UploadedFile doc), fixing
the newlines chars if necessary, and yielding the resulting lines.
This is a bit more involved, but IMHO the RightThing(tm) to do.
HTH
jeff
the right track and it turned out to be pretty simple. For the
benefit of others, here's the basic code to select a CSV file on the
local machine and process it on the server using DictReader to manage
the columns:
FORM:
class ImportFileForm( forms.Form ):
filename = forms.FileField(
label='Select File',
required=True,
help_text="Click Browse to find the file on your computer",
)
(Template is just a standard form display)
VIEW:
def import_from_csv( request ):
# =================================
# Select File
# =================================
if request.POST:
form = ImportFileForm( request.POST, request.FILES )
if form.is_valid():
pass # Fall through to process the file
else:
form = ImportFileForm()
return render_to_response('admin_import_from_csv.html', {'form':
form,}, RequestContext(request) )
errors = [] # List
# =================================
# Upload From File
# =================================
reader = csv.DictReader( request.FILES['filename'] )
try:
for row in reader:
group_name = str.strip( row['Group Name'] )
try:
job_name = str.strip( row['Job Name'] )
except KeyError:
job_name = ''
try:
shift_location = str.strip( row['Shift Location'] )
except KeyError:
shift_location = ''
try:
shift_date = str.strip( row['Shift Date'] )
except KeyError:
shift_date = ''
try:
start_time = str.strip( row['Start Time'] )
except KeyError:
start_time = ''
try:
end_time = str.strip( row['End Time'] )
except KeyError:
end_time = ''
try:
min_volunteers = str.strip( row['Min Volunteers'] )
except KeyError:
min_volunteers = ''
try:
max_volunteers = str.strip( row['Max Volunteers'] )
except KeyError:
max_volunteers = ''
try:
shift_notes = str.strip( row['Shift Notes'] )
except KeyError:
shift_notes = ''
# Do something with the row values here.....
except Exception, e:
errors += [( reader.line_num, "Serious Error Importing File: " +
str( e ) )]
# Done! Render a success screen or ?