cache.disk.clear() crashes in appadmin

[Eric]

Hi All,

When I press the 'empty cache' button on the Appadmin I get a nice error:

Traceback (most recent call last):
 File "/home/eric/Projects/Web2Py/site/gluon/restricted.py", line 224, in restricted
 exec ccode in environment
 File "/home/eric/Projects/Web2Py/site/applications/cms/controllers/appadmin.py", line 654, in <module>
 File "/home/eric/Projects/Web2Py/site/gluon/globals.py", line 392, in <lambda>
 self._caller = lambda f: f()
 File "/home/eric/Projects/Web2Py/site/applications/cms/controllers/appadmin.py", line 389, in ccache
 cache.disk.clear()
 File "/home/eric/Projects/Web2Py/site/gluon/cache.py", line 394, in clear
 storage.clear()
 File "/home/eric/Projects/Web2Py/site/gluon/cache.py", line 334, in clear
 del self[key]
 File "/home/eric/Projects/Web2Py/site/gluon/cache.py", line 316, in __delitem__
 recfile.remove(key, path=self.folder)
 File "/home/eric/Projects/Web2Py/site/gluon/recfile.py", line 36, in remove
 raise IOError
IOError

Is this a known issue? Web2Py crashes also when there are no files and directories in the "cache" folder.

I'm running on Web2Py version 2.9.11

Tnx!

[Paolo Valleri]

Open a ticket on google code

[Leonel Câmara]

Hey, 

It's not a known issue, the disk cache implementation is completely new in this version. I managed to replicate the error and I'm investigating the cause. I'll get back to you in this thread soon.

[Eric]

@Paolo: Done (http://code.google.com/p/web2py/issues/detail?id=1996)

@Leonel: Thanks!

[Leonel Câmara]

Hey, I have found the problem and it's not due to a bug (more like there needs to be a bit of a cleanup before upgrading to this version and this should have probably been documented). In you applications cache folder you have a cache.lock and cache.shelve, delete them. Or just delete the entire folder if you want as cache will create a new one.

[Leonel Câmara]

Humm nevermind, it seems these files that cause the problem are being created automatically by appadmin. So deleting them only temporarily solves the problem.

[Leonel Câmara]

I have found the problem in appadmin.py and fixed it. You can put it in your applications controllers replacing the one in controllers/appadmin.py to fix the problem.

Here are the steps:

1. Replace appadmin.py with the one attached here.

2. Delete everything in the cache folder.

3. Problem Solved!

I'll make a pull request later so we can close the issue.

[Robin Manoli]

Hey!
I'm having the same issue here, however not in appadmin, but using cache.disk.clear in a controller instead. What did you change to fix this?

Traceback (most recent call last):

  File "/web2py/gluon/restricted.py", line 224, in restricted
    exec ccode in environment
  File "/web2py/applications/myapp/controllers/test.py", line 145, in <module>
  File "/web2py/gluon/globals.py", line 392, in <lambda>

    self._caller = lambda f: f()

  File "/web2py/gluon/tools.py", line 3439, in f
    return action(*a, **b)
  File "/web2py/applications/myapp/controllers/test.py", line 57, in cache_clear
    cache.disk.clear(None)
  File "/web2py/gluon/cache.py", line 394, in clear
    storage.clear()
  File "/web2py/gluon/cache.py", line 334, in clear
    del self[key]
  File "/web2py/gluon/cache.py", line 316, in __delitem__
    recfile.remove(key, path=self.folder)
  File "/web2py/gluon/recfile.py", line 36, in remove
    raise IOError
IOError

[Leonel Câmara]

Hey Robin,

This was my fix for this issue

https://github.com/web2py/web2py/commit/9d4b2e66c4736c218033e8be70d135cdf0fc718f

If you show me some code I can try to solve your issue.

[Robin Manoli]

Thanks Leonel!

Actually the code is just this:
def cache_clear():
    cache.disk.clear(None)

[Leonel Câmara]

Hey Robin,

If the code is just that you probably have stuff in your cache folder from the previous way of using it that shouldn't be there anymore. If you manually delete all files and folders in your cache folder you should stop seeing this error in the future. 

[Eric Christiaanse]

Hi,

It took me a LONG time to check the update, but I just did. I've updated to Web2Py 2.9.12 (this version contains the fix), and copied the appadmin.py to my project.

When I call cacke.disk.clear() I still get an error :( To be exact this one:

Traceback (most recent call last):

  File "/home/eric/Projects/Web2Py/web/gluon/restricted.py", line 224, in restricted

    exec ccode in environment

  File "/home/eric/Projects/Web2Py/web/applications/cms/controllers/maintenance.py", line 415, in <module>

  File "/home/eric/Projects/Web2Py/web/gluon/globals.py", line 393, in <lambda>

    self._caller = lambda f: f()

  File "/home/eric/Projects/Web2Py/web/gluon/tools.py", line 3444, in f

    return action(*a, **b)

  File "/home/eric/Projects/Web2Py/web/applications/cms/controllers/maintenance.py", line 396, in edit

    flush_cache()

  File "/home/eric/Projects/Web2Py/web/gluon/tools.py", line 3444, in f

    return action(*a, **b)

  File "/home/eric/Projects/Web2Py/web/applications/cms/controllers/maintenance.py", line 406, in flush_cache

    cache.disk.clear(None)

  File "/home/eric/Projects/Web2Py/web/gluon/cache.py", line 427, in clear

    storage.clear()

  File "/home/eric/Projects/Web2Py/web/gluon/cache.py", line 365, in clear

    del self[key]

  File "/home/eric/Projects/Web2Py/web/gluon/cache.py", line 343, in __delitem__

    recfile.remove(key, path=self.folder)

  File "/home/eric/Projects/Web2Py/web/gluon/recfile.py", line 36, in remove

    raise IOError

IOError

Yes, I've cleared the cache folder :)

When I go to cache page in the appadmin.py in the admin folder I get the following error (even when the cache folder is empty and I did a fresh restart of Web2Py):

<type 'exceptions.KeyError'>

Traceback (most recent call last):

  File "/home/eric/Projects/Web2Py/web/gluon/restricted.py", line 224, in restricted

    exec ccode in environment

  File "/home/eric/Projects/Web2Py/web/applications/cms/controllers/appadmin.py", line 700, in <module>

  File "/home/eric/Projects/Web2Py/web/gluon/globals.py", line 393, in <lambda>

    self._caller = lambda f: f()

  File "/home/eric/Projects/Web2Py/web/applications/cms/controllers/appadmin.py", line 466, in ccache

    value = cache.disk.storage[key]

  File "/home/eric/Projects/Web2Py/web/gluon/cache.py", line 333, in __getitem__

    raise KeyError

KeyError

Any ideas?

2015-03-02 16:26 GMT+01:00 Leonel Câmara <leonel...@gmail.com>:

Hey Robin,

If the code is just that you probably have stuff in your cache folder from the previous way of using it that shouldn't be there anymore. If you manually delete all files and folders in your cache folder you should stop seeing this error in the future. 

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to a topic in the Google Groups "web2py-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py/3usaQxAFY0k/unsubscribe.
To unsubscribe from this group and all its topics, send an email to web2py+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Leonel Câmara]

Hey Eric,

web2py 2.9.12 does contain the fix, however your application's appadmin probably doesn't as the error is in appadmin which is a controller in your own application. You need to copy the one with the fix from the 2.9.12 welcome application and overwrite it.

[Eric Christiaanse]

Hi Leonel,

Done that and cleaned the cache folder. When I call /appadmin/ccache the code breaks here:

try:

     disk['ratio'] = disk['hits'] * 100 / value['hit_total']

The first time everything goes great with the from cache.disk.storage[key]:

{'hit_total': 0, 'misses': 0}

The second time it hits my breakpoint on the try/except is crashes on this code:

value = cache.disk.storage[key]

The key it is looking for is: 'all-wcprops'

Hmmm... that's an SVN file in a 'hidden' folder (.svn) :) 

When I download Web2Py and run it, than everything works.

2015-03-03 13:14 GMT+01:00 Leonel Câmara <leonel...@gmail.com>:

Hey Eric,

web2py 2.9.12 does contain the fix, however your application's appadmin probably doesn't as the error is in appadmin which is a controller in your own application. You need to copy the one with the fix from the 2.9.12 welcome application and overwrite it.

--

[Leonel Câmara]

Why is the cache folder under svn control? Files are key-values in the current cache disk implementation so you really can't have anything else putting files there. 

[Eric Christiaanse]

Hi,

The folder itself is under svn control, the data in it isn't.

It the cache folder created 'on demand'? When that's the case we can remove all svn control on it. 

2015-03-03 15:01 GMT+01:00 Leonel Câmara <leonel...@gmail.com>:

Why is the cache folder under svn control? Files are keys values in the current cache disk implementation so you really can't have anything else putting files there. 

[Leonel Câmara]

Yes, it is created on demand if it isn't there, go ahead and svn delete it.

[Eric Christiaanse]

Hi Leonel,

I've removed the cache folder from svn. Everything works fine now with cache removal :)

Tnx!

2015-03-03 16:21 GMT+01:00 Leonel Câmara <leonel...@gmail.com>:

Yes, it is created on demand if it isn't there, go ahead and svn delete it.

--