too many open files

[bblais]

Hello,

I have started using buzhug, and ran into this problem. I have made a
database with a number of parts, and open it with:

basedir='db/'

users=Base(basedir+'users').open()
proposals=Base(basedir+'proposals').open()
revisions=Base(basedir+'revisions').open()
documents=Base(basedir+'documents').open()
actions=Base(basedir+'actions').open()

After the last .open() attempt, I get the error:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/site-packages/IPython/ultraTB.py", line 497, in text
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/site-packages/IPython/ultraTB.py", line 124, in
_fixed_getinnerframes
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/inspect.py", line 877, in getinnerframes
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/inspect.py", line 837, in getframeinfo
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/inspect.py", line 393, in getsourcefile
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/inspect.py", line 419, in getmodule
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/inspect.py", line 403, in getabsfile
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/posixpath.py", line 403, in abspath
OSError: [Errno 24] Too many open files

Unfortunately, your original traceback can not be constructed.

I can open each individually, or any 4 of them, but the 5th puts it
over the edge. Has anyone ever seen this?

Brian Blais

[Nicolas Pinault]

Hi,

> basedir='db/'
>
> users=Base(basedir+'users').open()
> proposals=Base(basedir+'proposals').open()
> revisions=Base(basedir+'revisions').open()
> documents=Base(basedir+'documents').open()
> actions=Base(basedir+'actions').open()
>
>
> After the last .open() attempt, I get the error:
>

Can you give us more details ?
Wich OS ?
A more complete code sample.
What are bases structures ?

Nicolas

[bblais]

> Can you give us more details ?

> Which OS ?

> A more complete code sample.
> What are bases structures ?

I'll try.

OS: Mac OS X

yeah,I should have put in the bases structure. sorry! (I list it
below). As for code sample, I tried to make "fake" data so I could
post a small example, but buzhug works fine! there seems to be
something specific to my case, the way I am assigning or reading
data. I just don't see anything obvious. it's not really possible to
post the entire code, because it is pulling in data from different
directories, and some data from some excel spreadsheets, etc...

is there any obvious stupid thing I could have done to make buzhug
give a too many open files error?

thanks,

bb

users=Base(basedir+'users')
users.create(('name',str),
('email',str),
('department',str))

proposals=Base(basedir+'proposals')
proposals.create( ('description',str),
('label',str),
('date',datetime),
('submitter',users),
('alternate_submitter1',users),
('alternate_submitter2',users),
('alternate_submitter3',users),
('contact',users))

items=Base(basedir+'items')
items.create(('type',str),
('viewable',int),
('proposal',proposals))

documents=Base(basedir+'documents')
documents.create(('filename',str),
('label',str),
('date',datetime),
('viewable',int),
('submitter',users),
('item',items))

actions=Base(basedir+'actions')
actions.create( ('who',str),
('type',str),
('date',datetime),
('proposal',proposals)) # this is here, so that
# each proposal may have more
than
# one action

[Nicolas Pinault]

> OS: Mac OS X
>
> yeah,I should have put in the bases structure. sorry! (I list it
> below). As for code sample, I tried to make "fake" data so I could
> post a small example, but buzhug works fine! there seems to be
> something specific to my case, the way I am assigning or reading
> data. I just don't see anything obvious. it's not really possible to
> post the entire code, because it is pulling in data from different
> directories, and some data from some excel spreadsheets, etc...
>
> is there any obvious stupid thing I could have done to make buzhug
> give a too many open files error?
>

I tried your code sample on a win xp machine without any problem.
Your problem is surely located in your original code.

Nicolas

[bblais]

> I tried your code sample on a win xp machine without any problem.
> Your problem is surely located in your original code.

ok, I came up with a small example which still crashes. Any ideas on
what I am doing wrong?

I can even send the entire db folder, zipped up, because it's only
96k. I have no idea how to debug this.

thanks,

bb

# =====================================code which loads the database
from buzhug import Base
from datetime import date,datetime
basedir='db/'

users=Base(basedir+'users').open()
print len(users)
proposals=Base(basedir+'proposals').open()
print len(proposals)
items=Base(basedir+'items').open()
print len(items)
documents=Base(basedir+'documents').open()
print len(documents)
actions=Base(basedir+'actions').open()
print len(actions)

#===========================================================

# ==================================code which creates bad buzhug
database
import os,sys

from buzhug import Base
from datetime import date,datetime
import glob
import shutil

reset=True

basedir='db/'

if reset:
cmd='rm -r '+basedir
print "Executing %s" % cmd
os.system(cmd)

cmd='mkdir '+basedir
print "Executing %s" % cmd
os.system(cmd)

try:

actions=Base(basedir+'actions')
actions.create( ('who',str), # should be VPAA or Committee or Dean

('type',str),
('date',datetime),
('proposal',proposals)) # this is here, so that
# each proposal may have more
than
# one action

except IOError:

users=Base(basedir+'users').open()
proposals=Base(basedir+'proposals').open()

items=Base(basedir+'items').open()

documents=Base(basedir+'documents').open()
actions=Base(basedir+'actions').open()

print "Parsing users"

users.insert(name='Nobody')
nobody=users[0]

dt2=datetime(2000,1,1,0,0)

print "Parsing proposals"

submitter0=nobody
submitter1=nobody

submitter2=nobody

submitter3=nobody

prop=proposals.insert(description='',
label='',
date=dt2,
submitter=submitter0,
alternate_submitter1=submitter1,
alternate_submitter2=submitter2,
alternate_submitter3=submitter3,
contact=submitter0)