Bug: homedisco

[afancy]

Hi, when I use homedisco to run locally, it creates the output path "/
srv/disco/data/localhost/65/disco_tut@1254822448", however in
homedisco.py, the path is /srv/disco/f1/disco_tut@1254822815/part-
disco-000000000:1

############### Error thrown #################
OSError: [Errno 2] No such file or directory: '/srv/disco/f1/
disco_tut@1254822815/part-disco-000000000:1'
File "/home/xiliu/workspace/etlonmapred/pygrametlexa/
pygrametlmapreduce.py", line 371, in <module>
main()
File "/home/xiliu/workspace/etlonmapred/pygrametlexa/
pygrametlmapreduce.py", line 358, in main
params=Params(pgcopybulkloader=pgcopybulkloader)
File "/usr/lib/python2.5/site-packages/homedisco.py", line 53, in
new_job
disco_worker.op_map(req)
File "/usr/lib/python2.5/site-packages/disconode-0.1-py2.5.egg/
disconode/disco_worker.py", line 402, in op_map
File "/usr/lib/python2.5/site-packages/disco-0.1-py2.5.egg/disco/
util.py", line 13, in msg
(c, t, job_name, job_input, m)
File "/usr/lib/python2.5/site-packages/homedisco.py", line 25, in
write
self.out = parse_dir(addr)
File "/usr/lib/python2.5/site-packages/homedisco.py", line 9, in
parse_dir
for f in os.listdir(LOCAL_PATH + path):



############# Output path ########################
afancy@afancy-fedora:/srv/disco/data/localhost/65/
disco_tut@1254822448$ pwd
/srv/disco/data/localhost/65/disco_tut@1254822448
afancy@afancy-fedora:/srv/disco/data/localhost/65/
disco_tut@1254822448$ ls
part-disco-000000000 part-disco-000000001
afancy@afancy-fedora:/srv/disco/data/localhost/65/
disco_tut@1254822448$


########## it throws error when os.listdir(LOCAL_PATH + path)
###################
def parse_dir(dir):
x, x, LOCAL_PATH = util.load_conf()
x, x, x, mode, path = dir.split("/", 4)
res = []
for f in os.listdir(LOCAL_PATH + path):
if not (f.startswith(mode) or "." in f):
continue
proto = f.split("-")[1]
if proto == "chunk":
res.append("chunkfile://%s%s" % (path, f))
elif proto == "disco":
res.append("disco://localhost/%s%s" % (path,
f))
return res


>>> os.listdir(LOCAL_PATH +path)
Traceback (most recent call last):
File "<string>", line 1, in <fragment>
OSError: [Errno 2] No such file or directory: '/srv/disco/65/
disco_tut@1254822448/part-disco-000000000:1'
>>>

[eric.fredine]

Yes. I have the same problem -- running on OS X (my system is just
set up as a single cluster).

On Oct 6, 3:55 am, afancy <toxiuf...@gmail.com> wrote:
> Hi, when I usehomediscoto run locally, it creates the output path "/
> srv/disco/data/localhost/65/disco_tut@1254822448", however inhomedisco.py, the path is /srv/disco/f1/disco_tut@1254822815/part-

[杨晓瑜]

Er,I also have the same problem

2009/10/21 eric.fredine <eric.f...@gmail.com>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Disco-development" group.
To post to this group, send email to disc...@googlegroups.com
To unsubscribe from this group, send email to disco-dev+...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/disco-dev?hl=en
-~----------~----~----~----~------~----~------~--~---