word_count.py执行错误

[Casa Zhang]

hi all,
   我安装了dpark，按照wiki上写了一个程序word_count.py如下：
  
from dpark import DparkContext

def word_count(file_path, word):
    dpark = DparkContext()
    f = dpark.textFile(file_path, splitSize=16<<20)

    print word, 'count:', f.map(lambda line: line.strip()).filter(lambda line: word in line).count()

word_count("/home/casa/data","a")

但是，结果是对的，但是出现了如下错误：

a count: 1
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib64/python2.6/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/home/casa/git/dpark/dpark/context.py", line 267, in stop
    env.stop()
  File "/home/casa/git/dpark/dpark/env.py", line 100, in stop
    self.trackerServer.stop()
  File "/home/casa/git/dpark/dpark/tracker.py", line 49, in stop
    sock.connect(self.addr)
  File "socket.pyx", line 459, in zmq.core.socket.Socket.connect (zmq/core/socket.c:4201)
ZMQError: Invalid argument
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/usr/lib64/python2.6/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/home/casa/git/dpark/dpark/context.py", line 267, in stop
    env.stop()
  File "/home/casa/git/dpark/dpark/env.py", line 100, in stop
    self.trackerServer.stop()
  File "/home/casa/git/dpark/dpark/tracker.py", line 49, in stop
    sock.connect(self.addr)
  File "socket.pyx", line 459, in zmq.core.socket.Socket.connect (zmq/core/socket.c:4201)
zmq.error.ZMQError: Invalid argument

a count: 1是运行结果，是对的，但是会出现下面的错误。

[Davies Liu]

从出错信息来看，是计算完成后退出时清理环境是出错了，具体的错误与zmq有关，你的zmq是什么版本？

在下面这一行前加一个 “print self.addr” 看能不能找到问题的原因

File "/home/casa/git/dpark/dpark/tracker.py", line 49, in stop

--
You received this message because you are subscribed to the Google Groups "DPark Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dpark-users...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
 - Davies

[张磊]

pyzmq 13.1.0

2013/10/3 Davies Liu <davie...@gmail.com>

--
You received this message because you are subscribed to a topic in the Google Groups "DPark Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dpark-users/Si_ON_mRjqo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dpark-users...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

 

 

Best Wishes!

                                                                                                                                       zhanglei

[muxueqz(张明源)]

貌似dpark在python 2.6情况会有问题，我是用python 2.7的

2013/10/3 张磊 <zhangl...@gmail.com>

--

明源
http://muxueqz.gitcafe.com
Twitter: @muxueqz

[张磊]

你在eclipse里面运行过dpark吗？

2013/10/4 muxueqz(张明源) <zhangmin...@gmail.com>

--

 

 

Best Wishes!

                                                                                                                                       zhanglei

[muxueqz(张明源)]

没，同事有用，不过他在Windows下已知有很多地方是不兼容的了

建议你试试不在IDE里运行，直接在终端下运行，IDE可能会修改你的一些环境变量

2013/10/5 张磊 <zhangl...@gmail.com>