2.10.1 / Windows errors / cache

[Kiran Subbaraman]

• Downloaded and extracted web2py src for Windows: http://www.web2py.com/examples/static/web2py_src.zip. Did the same with the windows binary too: http://www.web2py.com/examples/static/web2py_win.zip
• Performed the following steps with both the instances of web2py, but one at a time.
  
• Installed a test app (which uses postgres DB, and web2py cache). The test app is included as attachment.
• After installing the test app, I visit the app, and the output is as expected - a simple page listing the contents of the db
• I refresh the browser, and I see this error

<type 'exceptions.IOError'> [Errno 13] Permission denied
Version
web2py™     Version 2.10.1-stable+timestamp.2015.03.31.22.17.00
Python     Python 2.7.9: D:\programs\open\python27\python.exe (prefix: D:\programs\open\python27)
Traceback

Traceback (most recent call last):
  File "C:\Users\subbaraman\Downloads\web2py_src\web2py\gluon\restricted.py", line 227, in restricted
    exec ccode in environment
  File "C:/Users/subbaraman/Downloads/web2py_src/web2py/applications/sd/controllers/default.py", line 75, in <module>
  File "C:\Users\subbaraman\Downloads\web2py_src\web2py\gluon\globals.py", line 393, in <lambda>
    self._caller = lambda f: f()
  File "C:/Users/subbaraman/Downloads/web2py_src/web2py/applications/sd/controllers/default.py", line 22, in index
    rows = db(db.t1.id > 0).select(**cache_settings)
  File "C:\Users\subbaraman\Downloads\web2py_src\web2py\gluon\packages\dal\pydal\objects.py", line 2093, in select
    return adapter.select(self.query,fields,attributes)
  File "C:\Users\subbaraman\Downloads\web2py_src\web2py\gluon\packages\dal\pydal\adapters\base.py", line 1237, in select
    time_expire)
  File "C:\Users\subbaraman\Downloads\web2py_src\web2py\gluon\cache.py", line 459, in __call__
    item = self.storage.get(key)
  File "C:\Users\subbaraman\Downloads\web2py_src\web2py\gluon\cache.py", line 413, in get
    return self[key]
  File "C:\Users\subbaraman\Downloads\web2py_src\web2py\gluon\cache.py", line 356, in __getitem__
    value = pickle.load(recfile.open(key, 'rb', path=self.folder))
IOError: [Errno 13] Permission denied



Function argument list

(self=<gluon.cache.PersistentStorage object>, key='GUZWGOJXGAZWGMBTGA3DSNJRHEZTGOJZHA4GMNJSGQ4WINRTMNSQ====')
Code listing

                val_file = recfile.open(key, mode='rb', path=self.folder)
            except IOError:
                raise KeyError

            self.wait_portalock(val_file)
value = pickle.load(recfile.open(key, 'rb', path=self.folder))

            val_file.close()
            return value

Variables
global pickle     <module 'cPickle' (built-in)>
self     <gluon.cache.PersistentStorage object>
value     undefined
pickle.load     <built-in function load>
recfile.open     <function open>
self.folder     r'C:\Users\subbaraman\Downloads\web2py_src\web2py\applications\sd\cache'
key     'GUZWGOJXGAZWGMBTGA3DSNJRHEZTGOJZHA4GMNJSGQ4WINRTMNSQ===='
path     undefined
global recfile     <module 'gluon.recfile' from 'C:\Users\subbaraman\Downloads\web2py_src\web2py\gluon\recfile.pyc'>

I try to clean the cache and errors, via the admin app, and see this error:

Error ticket for "admin"
Ticket ID

127.0.0.1.2015-04-01.05-01-31.706d8e9c-ad53-4b25-9e56-5dab5a36bbbb
<type 'exceptions.WindowsError'> [Error 32] The process cannot access the file because it is being used by another process: 'C:/Users/subbaraman/Downloads/web2py_src/web2py/applications/sd/cache/088\\1f4\\GUZWGOJXGAZWGMBTGA3DSNJRHEZTGOJZHA4GMNJSGQ4WINRTMNSQ===='
Version
web2py™     Version 2.10.1-stable+timestamp.2015.03.31.22.17.00
Python     Python 2.7.9: D:\programs\open\python27\python.exe (prefix: D:\programs\open\python27)
Traceback


Traceback (most recent call last):
  File "C:\Users\subbaraman\Downloads\web2py_src\web2py\gluon\restricted.py", line 227, in restricted
    exec ccode in environment
  File "C:/Users/subbaraman/Downloads/web2py_src/web2py/applications/admin/controllers/default.py", line 1944, in <module>
  File "C:\Users\subbaraman\Downloads\web2py_src\web2py\gluon\globals.py", line 393, in <lambda>
    self._caller = lambda f: f()
  File "C:/Users/subbaraman/Downloads/web2py_src/web2py/applications/admin/controllers/default.py", line 448, in cleanup
    clean = app_cleanup(app, request)
  File "C:\Users\subbaraman\Downloads\web2py_src\web2py\gluon\admin.py", line 124, in app_cleanup
    if f[:1] != '.': recursive_unlink(os.path.join(path, f))
  File "C:\Users\subbaraman\Downloads\web2py_src\web2py\gluon\fileutils.py", line 181, in recursive_unlink
    recursive_unlink(os.path.join(f, s))
  File "C:\Users\subbaraman\Downloads\web2py_src\web2py\gluon\fileutils.py", line 181, in recursive_unlink
    recursive_unlink(os.path.join(f, s))
  File "C:\Users\subbaraman\Downloads\web2py_src\web2py\gluon\fileutils.py", line 184, in recursive_unlink
    os.unlink(f)
WindowsError: [Error 32] The process cannot access the file because it is being used by another process: 'C:/Users/subbaraman/Downloads/web2py_src/web2py/applications/sd/cache/088\\1f4\\GUZWGOJXGAZWGMBTGA3DSNJRHEZTGOJZHA4GMNJSGQ4WINRTMNSQ===='

Error snapshot help

<type 'exceptions.WindowsError'>([Error 32] The process cannot access the file because it is being used by another process: 'C:/Users/subbaraman/Downloads/web2py_src/web2py/applications/sd/cache/088\\1f4\\GUZWGOJXGAZWGMBTGA3DSNJRHEZTGOJZHA4GMNJSGQ4WINRTMNSQ====')

[Leonel Câmara]

Awww dammit we forgot to update the admin module.

I attached an admin.py file you can replace the one in the gluon with to get this to work until we make a proper fix.

[Massimo Di Pierro]

2.10.2 fixes it as Leonel suggested.

[Kiran Subbaraman]

Just cloned the repo, and noticed the 2.10.2 tag.
Tried this again, in this codebase, and the errors remain.
Please note, I have highlighted two issues that occur:

• Install the sample app, view the default page. The page comes up just fine. Click page-refresh, and you see the first error: "<type 'exceptions.IOError'> [Errno 13] Permission denied...."
• Go to the Admin page, and select "clean", for this app. The second error: "<type 'exceptions.WindowsError'> [Error 32] The process cannot access the file because it is being used by another process"
  

________________________________________
Kiran Subbaraman
http://subbaraman.wordpress.com/about/

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Niphlod]

Straight from Peter Pan....

“Everytime a user says 'I can't clean the cache,' there's a a little developer somewhere that falls down dead"

muahhaha. We'd need a regression test for it.

[Leonel Câmara]

Kiran I had some trouble replicating with 2.10.2 but after putting 6 tabs auto refreshing in the initial page and another tab in the admin cleaning I started seeing the error both with the cache and session. I patched app_cleanup so it would not throw an error when it encountered locked files as these are files that are being used by other processes and cannot be deleted, it's not a problem anyway since fileutils/w2p_pack ignores these directories.

Can you verify that this also fixes stuff for you:

https://github.com/web2py/web2py/pull/891

[Kiran Subbaraman]

Leonel,
Thank you for trying to replicate the issue - am guessing you replicated the "cleaning up of the cache issue", and not the "issue while using the cache itself"?
I went ahead and refreshed the web2py repo (with your latest PR in it), and tried to test for these issues.
I can confirm that the "cleaning" issue seems to be sorted out, except once in a while I see a message on the admin screen, while trying to clean the app, that "some files were not removed / cleaned". I do not have  screenshot, will share the next time I come across it.
The issue with the usage of the cache itself is now an issue that I have raised on github: https://github.com/web2py/web2py/issues/895

Please let me know if any more information will help in sorting out this issue. I'll see if I can take a look at the code, today, to track down this issue.

[Kiran Subbaraman]

Here is the screenshot of the message I see, when I try to clean the app, via the admin app. This is not something that I see all the time, but does occur frequently enough.

[Leonel Câmara]

Thank you for trying to replicate the issue - am guessing you replicated the "cleaning up of the cache issue", and not the "issue while using the cache itself"?

No no, I solved both issues that's why I had to have a multitude of browser tabs opened auto-refreshing to replicate the issue.

Here is the screenshot of the message I see, when I try to clean the app, via the admin app. This is not something that I see all the time, but does occur frequently enough

Kiran that's ok, it's supposed to happen. In an application that's online and being used, sometimes you won't be able to delete every file.