Error importing csv file in python3.6

[Maurice Waka]

I shifted to web2py/python3. In python2.7 I was able to upload csv files but using the same code below I get an error.

For example:

def import_csv():
    form = FORM(INPUT(_type = 'file', _name = 'csv_file'),
             INPUT(_type = 'submit', _value = T('Import')))
    return dict(form=form)
@mobilize
@auth.requires_login()
def index1():
    if request.vars.csvfile != None:
        # set values
        table = db[request.vars.table]
        file = request.vars.csvfile.file
        # import csv file
        table.import_from_csv_file(file)
        # update who imported
        query = db.food_data.protein==""
        db(query).update(protein="")
        response.flash = 'Data uploaded'
    return dict()

The error is this

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.

Traceback (most recent call last):
  File "/home/mauricewaka/web2py/gluon/restricted.py", line 219, in restricted
    exec(ccode, environment)
  File "/home/mauricewaka/web2py/applications/Hestque_Wellness/controllers/default.py", line 479, in <module>
  File "/home/mauricewaka/web2py/gluon/globals.py", line 421, in <lambda>
    self._caller = lambda f: f()
  File "/home/mauricewaka/web2py/gluon/contrib/user_agent_parser.py", line 695, in __call__
    return self.func()
  File "/home/mauricewaka/web2py/gluon/tools.py", line 3867, in f
    return action(*a, **b)
  File "/home/mauricewaka/web2py/applications/Hestque_Wellness/controllers/default.py", line 329, in index1
    table.import_from_csv_file(file)
  File "/home/mauricewaka/web2py/gluon/packages/dal/pydal/objects.py", line 980, in import_from_csv_file
    for lineno, line in enumerate(reader):
_csv.Error: iterator should return strings, not bytes (did you open the file in text mode?)

How can I solve this

[sandeep patel]

I had the same issue. I have solved this way.

Please try this

You have to changing line 926 of gluon/packages/dal/pydal/objects.py from

reader = csv.reader(csvfile, delimiter=delimiter,

quotechar=quotechar, quoting=quoting)

to

reader = csv.reader(codecs.iterdecode(csvfile,'utf-8'), delimiter=delimiter,

quotechar=quotechar, quoting=quoting)

After changes done, you have to execute the  web2py.py

Thanks

SP

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[sandeep patel]

In fact, this is an open issue for the same problem. You can check that as well

[Lovedie JC]

Works.

But I can't access the data. 

[sandeep patel]

@Lovedie

Can you explain a little bit more? What you mean by can't access the data?

If you getting the error to import data from CSV. You can try this as well.

 form = FORM(DIV(LABEL('File input',_for='exampleInputFile'),INPUT(_type='file',_name='csvsheet',_id="exampleInputFile"),_class='from-group'),BUTTON('Submit',_type='submit',_class="btn btn-primary"))

    if form.process().accepted:

        try:

            for i in csv.DictReader(codecs.iterdecode(request.vars.csvsheet.file,'utf-8')):

                db['table name'].insert(**i)

        except Exception as e: 

            print(e)

            form.errors.csvfile = 'Invalid file format.'

Thanks

SP

[Maurice Waka]

When I search for an item it returns blank. However I see that the upload to the db took place.

When I try to add data to the database, where other data was uploaded, I get this error

1. 2. 3. 4. 5. 6.

Traceback (most recent call last):

File "/home/mauricewaka/web2py/gluon/restricted.py", line 219, in restricted exec(ccode, environment

) File "/home/mauricewaka/web2py/applications/Hestque_Wellness/views/generic.html", line 81, in <module> TypeError: 'dict_values' object does not support indexing

You received this message because you are subscribed to a topic in the Google Groups "web2py-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py/t2SEQC9o-ic/unsubscribe.
To unsubscribe from this group and all its topics, send an email to web2py+un...@googlegroups.com.

[Maurice Waka]

After much study, I noted that the import was successful. Both in python 2.7 and 3.6. Its in python 3.6 that I get problems.

 

Getting to select the items from the database was the problem unless i knew the item's fill name in full. The code below helps to 'auto suggest' and helps in picking the items from the DB but its not working.

I imported a food file, and want to use it but it gives me a blank page. For example, I have to type in 'BUTTER,WITH SALT', and submit to successfully retrieve the item.

All I want is a list of suggested items once i start typing.

I'm using mongoDB: db = DAL('mongodb://localhost/wellness')

This code from https://github.com/shatalovdm/food_diary Is what I thing is where the problem is.:

def search_selector():
    if not request.vars.name:
        return 'Not found'
    selected = []
    for row in db(db.food_data.name.contains(request.vars.name)).select():
        if len(selected) <50:
            selected.append(row.name)
        else:
            break
    return ''.join([DIV(k,
                 _onclick="jQuery('#name').val('%s')" % k,
                 _onmouseover="this.style.backgroundColor='lightblue'",
                 _onmouseout="this.style.backgroundColor='white'"
                 ).xml() for k in selected])

You received this message because you are subscribed to a topic in the Google Groups "web2py-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py/t2SEQC9o-ic/unsubscribe.
To unsubscribe from this group and all its topics, send an email to web2py+un...@googlegroups.com.

[Maurice Waka]

Narrowed down to the xml().

Can you help?

You received this message because you are subscribed to a topic in the Google Groups "web2py-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py/t2SEQC9o-ic/unsubscribe.
To unsubscribe from this group and all its topics, send an email to web2py+un...@googlegroups.com.

[Leonel Câmara]

xml() returns bytes, You need to do b''.join(... instead of ''.join

[Maurice Waka]

Still not working. 

On Fri, 1 Mar 2019, 21:33 Leonel Câmara <leonel...@gmail.com> wrote:

xml() returns bytes, You need to do b''.join(... instead of ''.join

--

[Kevin Keller]

There an issue filed on github already?

[Maurice Waka]

"xml() in python 3.6 · Issue #2096 · web2py/web2py" https://github.com/web2py/web2py/issues/2096

On Sat, 2 Mar 2019, 14:19 Kevin Keller <kell...@gmail.com> wrote:

There an issue filed on github already?

[nbac...@bancointernacional.cl]

I had a similar issue.

I solve it doing this:

f = request.vars.csvfile.file

import io

file = io.TextIOWrapper(f, encoding='utf-8')

CONFIDENCIALIDAD La información contenida en este mensaje y/o en los archivos adjuntos es de carácter confidencial o privilegiada y está destinada al uso exclusivo del emisor y/o de la persona o entidad a quien va dirigida. Si usted no es el destinatario, cualquier almacenamiento, divulgación, distribución o copia de esta información está estrictamente prohibido y sancionado por la ley. Si recibió este mensaje por error, por favor infórmenos inmediatamente respondiendo este mismo mensaje y borre éste y todos los archivos adjuntos. Gracias.

CONFIDENTIAL NOTE The information transmitted in this message and/or attachments is confidential and/or privileged and is intented only for use of the person or entity to whom it is addressed. If you are not the intended recipient, any retention, dissemination, distribution or copy of this information is strictly prohibited and sanctioned by law. If you received this messagge in error, please reply us this same message and delete this message and all attachments. Thank you. 

[Dan Carroll]

It Works! YESSS!!!

Worked in my code. Now to fix the import csv function in appadmin. Using version 2.18.5 of web2py with python 3.7.0 on pythonanywhere.

[Dan Carroll]

This works in your own code using import_from_csv_file as well as in appadmin, all for python 3.

On Friday, April 26, 2019 at 1:56:39 AM UTC-4, nbac...@bancointernacional.cl wrote: