大家是怎么部署Dpark的?

97 views
Skip to first unread message

muxueqz(张明源)

unread,
Jan 7, 2014, 4:11:14 AM1/7/14
to dpark...@googlegroups.com
Hi, 大家好:
      因为Dpark集群要求路径/内容一致,所以想问一下大家是怎么部署Dpark应用的?
      先说一下我们自己吧,每个项目有独立的virtualenv,日志和程序都放在moosefs中
      PS: 刚刚提交一个进程模式(-m process)无法退出的issue

--

xiaobo lee

unread,
Mar 5, 2014, 10:21:02 PM3/5/14
to dpark...@googlegroups.com
我也有同样的问题,用 virtualenv怎么部署? 实验了下两台服务器基于 mesos + dpark + virtualenv ,在slave virtualenv 环境下执行如下测试代码无法运行:

b = dpark.parallelize([(1, 11), (1, 12), (2, 22)])
r1 = b.map(lambda x: (x[0], x)).groupByKey().collect()
print r1

slave 日志报错:
E0306 11:02:17.247473 10494 slave.cpp:1356] Failed to handle status update acknowledgement (UUID: 332e4233-9627-7b1a-17d2-cc1d5c283ac6) for task 1:1:1 of framework 201403061047-689246218-5050-31705-0002: Cannot find the status update stream for task 1:1:1 of framework 201403061047-689246218-5050-31705-0002

在 2014年1月7日星期二UTC+8下午5时11分14秒,muxueqz写道:

muxueqz(张明源)

unread,
Mar 5, 2014, 10:30:57 PM3/5/14
to dpark...@googlegroups.com
我这里多virtualenv的问题已经解决,
你可以看看mesos slave日志stderr/stdout中具体的内容



--
You received this message because you are subscribed to the Google Groups "DPark Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dpark-users...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Reply all
Reply to author
Forward
0 new messages