scheduler: imported unicodedata module in model not visible in a view

[Adi]

unicodedata module is accessible in a view when i test code directly from an application, but get the error when run the same code from scheduler. I can move the code up to a model (db_scheduler), but just wanted to know if imports from models should be visible to Views when run through scheduler?

Thanks,
Adnan

db_scheduler.py
        import unicodedata

        def send_order_confirmation_email():
            ...
             context=dict(order=order)
             message=response.render(msg_file, context)
          ...send email
            return

Views:
msg_file.html
    {{=XML('<p>%s %s</p>' % \
                                                (row.orderItem.Qty, unicodedata.normalize('NFKD', \
                                                unicode(row.orderItemDetail.ProductDesc, 'utf8')).encode('ascii','ignore')))
    }}

Traceback (most recent call last):
  File "/opt/web-apps/web2py/gluon/scheduler.py", line 218, in executor
    result = dumps(_function(*args,**vars))
  File "applications/crm/models/db_scheduler.py", line 179, in send_order_confirmation_email
    message=response.render(msg_file, context)
  File "/opt/web-apps/web2py/gluon/globals.py", line 217, in render
    run_view_in(self._view_environment)
  File "/opt/web-apps/web2py/gluon/compileapp.py", line 654, in run_view_in
    context=environment)
  File "/opt/web-apps/web2py/gluon/restricted.py", line 222, in restricted
    raise RestrictedError(layer, code, output, environment)
RestrictedError: <type 'exceptions.NameError'> name 'unicodedata' is not defined

[Massimo Di Pierro]

Where is unicodedata.py located?

[Adi]

this is the one that comes from python: http://docs.python.org/library/unicodedata.html

(using it to clean out trademark and other symbols in order to send text as email)

[Niphlod]

it's odd, but seems unrelated to scheduler. To help on further investigation, I managed to reproduce in shell.
Append to standard db.py

import unicodedata
def testfunction():
     return response.render('message.html', context=dict())

then, create views/message.html

{{=XML('<p>%s</p>' % (unicodedata.normalize('NFKD', u'hi')))}}

then, controllers/default.py

def test():
    return testfunction()

go to testapp/default/test , no errors, "hi" is returned correctly

but, in shell....

web2py -M -S testapp
>>> myvar = testfunction()

Traceback (most recent call last):

  File "<console>", line 1, in <module>
  File "applications/un/models/db.py", line 86, in testfunction
    return response.render('message.html', context=dict())
  File "/home/niphlod/Scrivania/web2py_source/web2py/gluon/globals.py", line 218, in render
    run_view_in(self._view_environment)
  File "/home/niphlod/Scrivania/web2py_source/web2py/gluon/compileapp.py", line 656, in run_view_in
    restricted(ccode, environment, layer)
  File "/home/niphlod/Scrivania/web2py_source/web2py/gluon/restricted.py", line 222, in restricted
    raise RestrictedError(layer, code, output, environment)

RestrictedError: <type 'exceptions.NameError'> name 'unicodedata' is not defined

>>> unicodedata.normalize('NFKD', u'hi')
u'hi'

[Massimo Di Pierro]

There is a known problem with the custom_import, it does not work when import mymodule is called from the shell. Perhaps this is also a problem with the scheduler. It is in my todo list to fix it by the end of the week.

[Niphlod]

Ok, look into that when you can. Just notice that the "run_view_in" can't see the unicodedata module, but in shell environment unicodedata is present, otherwise this

>>> unicodedata.normalize('NFKD', u'hi')
u'hi'

would have raised an exception