py27 branch - Error (with FIX) in local admin console

[Paolo Casciello]

I noticed that in the py27 branch of Kay the local admin console (/_ah/admin) stopped working.

Below is the error.

I tried with a pure GAE app and it works.. 

ERROR    2011-11-23 16:38:52,472 cgi.py:121] Traceback (most recent call last):

  File "/usr/local/google_appengine/google/appengine/ext/admin/__init__.py", line 75, in <module>

    from google.appengine.ext import webapp

  File "/usr/local/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 640, in Decorate

    return func(self, *args, **kwargs)

  File "/usr/local/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1756, in load_module

    return self.FindAndLoadModule(submodule, fullname, search_path)

  File "/usr/local/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 640, in Decorate

    return func(self, *args, **kwargs)

  File "/usr/local/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1628, in FindAndLoadModule

    description)

  File "/usr/local/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 640, in Decorate

    return func(self, *args, **kwargs)

  File "/usr/local/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1571, in LoadModuleRestricted

    description)

  File "/usr/local/google_appengine/google/appengine/ext/webapp/__init__.py", line 190, in <module>

    from webapp2 import *

ImportError: No module named webapp2

INFO     2011-11-23 16:38:52,489 dev_appserver.py:2753] "GET /_ah/admin HTTP/1.1" 500 -

The error raised in GAE and seemed not related to Kay but happens only using Kay. I suspected something related to sys.path.

I've found in the setup_env function in kay/__init__.py a set of instructions to recreate the sys.path created by dev_appserver

Logging in GAE I've found that it includes only all the subdirectories in lib/ except for yaml so the following code in kay is useless (lines 101-113):

        # SDK 1.4.2 introduced Django 1.2, and renamed django to django_0_96

        print 'autodiscovery %s' % dir

        if dir == 'django_0_96':

         EXTRA_PATHS.append(os.path.join(lib, dir))

         continue

        path = os.path.join(lib, dir)

        # Package can be under 'lib/<pkg>/<pkg>/' or 'lib/<pkg>/lib/<pkg>/'

        detect = (os.path.join(path, dir), os.path.join(path, 'lib', dir))

        for path in detect:

          print "\tdetect %s" % path

          if os.path.isdir(path):

            EXTRA_PATHS.append(os.path.dirname(path))

            break

( i left my debug prints.. i'm lazy :P )

Instead we could just include all the lib and specify the only exception:

      # Automatically add all packages in the SDK's lib folder:

      for dir in os.listdir(lib):

        EXTRA_PATHS.append(os.path.join(lib, dir))

      

      EXTRA_PATHS.append(os.path.join(lib, 'yaml', 'lib'))

      sys.path = EXTRA_PATHS + sys.path

And admin console works perfectly now!

Handling this exception is fair because we are bound to the appengine SDK and this type of things rarely change.

And just because we're in topic of coding rules, using "dir" as a variable name is not good as "dir" is a builtin so in my patch i've changed this name.

Following is the diff. Too short to make a patch request I think.. :)

diff -r ed3ad77a6afb kay/__init__.py

--- a/kay/__init__.py Mon Nov 21 16:42:55 2011 +0900

+++ b/kay/__init__.py Wed Nov 23 18:39:56 2011 +0100

@@ -97,18 +97,10 @@

       EXTRA_PATHS = [SDK_PATH]

       lib = os.path.join(SDK_PATH, 'lib')

       # Automatically add all packages in the SDK's lib folder:

-      for dir in os.listdir(lib):

-        # SDK 1.4.2 introduced Django 1.2, and renamed django to django_0_96

-        if dir == 'django_0_96':

-         EXTRA_PATHS.append(os.path.join(lib, dir))

-         continue

-        path = os.path.join(lib, dir)

-        # Package can be under 'lib/<pkg>/<pkg>/' or 'lib/<pkg>/lib/<pkg>/'

-        detect = (os.path.join(path, dir), os.path.join(path, 'lib', dir))

-        for path in detect:

-          if os.path.isdir(path):

-            EXTRA_PATHS.append(os.path.dirname(path))

-            break

+      for subdir in os.listdir(lib):

+        EXTRA_PATHS.append(os.path.join(lib, subdir))

+      

+      EXTRA_PATHS.append(os.path.join(lib, 'yaml', 'lib'))

       sys.path = EXTRA_PATHS + sys.path

 

     # corresponds with another google package

Bye,

 Paolo

[Paolo Casciello]

Anyone? Takashi? Ian?

Do you prefer an issue with the attached patch?

[Prateek Malhotra]

Probably best to submit a patch.

Alternatively I suggest just importing the "EXTRA_PATHS" variable given by the actual app_engine SDK appcfg.py (this way we stay consistent with the SDK's environmental imports):

Replace:
    else:

      EXTRA_PATHS = [SDK_PATH]
      lib = os.path.join(SDK_PATH, 'lib')
      # Automatically add all packages in the SDK's lib folder:

      for dir in os.listdir(lib):

        # SDK 1.4.2 introduced Django 1.2, and renamed django to django_0_96

        if dir == 'django_0_96':
         EXTRA_PATHS.append(os.path.join(lib, dir))
         continue
        path = os.path.join(lib, dir)
        # Package can be under 'lib/<pkg>/<pkg>/' or 'lib/<pkg>/lib/<pkg>/'
        detect = (os.path.join(path, dir), os.path.join(path, 'lib', dir))
        for path in detect:

          if os.path.isdir(path):
            EXTRA_PATHS.append(os.path.dirname(path))
            break

      sys.path = EXTRA_PATHS + sys.path

With:

    else:
      sys.path = SDK_PATH + sys.path
      from appcfg import EXTRA_PATHS as appcfg_EXTRA_PATHS
      sys.path = appcfg_EXTRA_PATHS + sys.path

This is the change I made and it seems to be working well. I'll submit a bug and corresponding patch at a later date.

Thanks,
Prateek

On Sat, Nov 26, 2011 at 8:54 AM, Paolo Casciello <paolo.c...@gmail.com> wrote:

Anyone? Takashi? Ian?

Do you prefer an issue with the attached patch?

--
You received this message because you are subscribed to the Google Groups "kay-users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/kay-users/-/1ZzXuLPFJ5sJ.

To post to this group, send email to kay-...@googlegroups.com.
To unsubscribe from this group, send email to kay-users+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/kay-users?hl=en.

[Prateek Malhotra]

Sorry it was meant to be:

    else:
      sys.path = [SDK_PATH] + sys.path
      from appcfg import EXTRA_PATHS as appcfg_EXTRA_PATHS
      sys.path = appcfg_EXTRA_PATHS + sys.path

[Paolo Casciello]

Yes this is even more elegant!

[Takashi Matsuo]

Thanks Paolo and Prateek, and sorry to chime in late.

The change seems good and I've just incorporated it in the py27 branch.

Thanks a lot!

-- 
Takashi Matsuo
matsuo....@gmail.com
Kay's daddy

2011/11/28 Paolo Casciello <paolo.c...@gmail.com>

Yes this is even more elegant!

--

You received this message because you are subscribed to the Google Groups "kay-users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/kay-users/-/fIWG32q19a8J.