ValueError: unsupported pickle protocol: 4
Adam G
/usr/local/lib/python3.5/site-packages/disco/fileutils.py:141: UserWarning: Duplicate name: 'usr/local/lib/python3.5/site-packages/disco/job.py'
self.write(file, file)
/usr/local/lib/python3.5/site-packages/disco/fileutils.py:141: UserWarning: Duplicate name: 'usr/local/lib/python3.5/site-packages/disco/worker/classic/worker.py'
self.write(file, file)
Job@5ab:90e3b:8314c:
Status: [map] 0 pending, 0 waiting, 1 running, 0 done, 0 failed
2015/12/31 15:24:27 master New job initialized!
2015/12/31 15:24:27 master Created job "Job@5ab:90e3b:8314c"
2015/12/31 15:24:27 master Stage map scheduled with 1 tasks
2015/12/31 15:24:27 master map:0 assigned to localhost
2015/12/31 15:24:27 master FATAL: [map:0] Traceback (most recent call last):
File "/Users/adam/py/disco/root/data/localhost/ee/Job@5ab:90e3b:8314c/usr/local/lib/python3.5/site-packages/disco/worker/__init__.py", line 343, in main
task = cls.get_task()
File "/Users/adam/py/disco/root/data/localhost/ee/Job@5ab:90e3b:8314c/usr/local/lib/python3.5/site-packages/disco/worker/__init__.py", line 428, in get_task
return Task(**dict((str(k), v) for k, v in cls.send('TASK').items()))
File "/Users/adam/py/disco/root/data/localhost/ee/Job@5ab:90e3b:8314c/usr/local/lib/python3.5/site-packages/disco/task.py", line 70, in __init__
self.jobobjs = dPickle.loads(self.jobpack.jobdata)
ValueError: unsupported pickle protocol: 4
2015/12/31 15:24:27 master WARN: Job killed
Status: [map] 0 pending, 1 waiting, 0 running, 0 done, 1 failed
Traceback (most recent call last):
File "examples/util/count_words.py", line 16, in <module>
for word, count in result_iterator(job.wait(show=True)):
File "/usr/local/lib/python3.5/site-packages/disco/core.py", line 369, in wait
timeout, poll_interval * 1000)
File "/usr/local/lib/python3.5/site-packages/disco/core.py", line 329, in check_results
raise JobError(Job(name=jobname, master=self), "Status {0}".format(status))
disco.error.JobError: Job Job@5ab:90e3b:8314c failed: Status dead
Adam G
Temporarily fixed by altering the python path in the scripts.
Also I fixed estimate_pi.py for Python 3
diff --git a/examples/util/estimate_pi.py b/examples/util/estimate_pi.py
index 1efab29..8897ddf 100644
--- a/examples/util/estimate_pi.py
+++ b/examples/util/estimate_pi.py
@@ -14,5 +14,5 @@ if __name__ == '__main__':
job = Job().run(input=["raw://0"] * COUNT , map=map)
tot = 0
for k, v in result_iterator(job.wait()):
- tot += v
- print (4.0 * tot) / COUNT
+ tot += int(v)
+ print(4.0 * tot / COUNT)
diff --git a/lib/disco/compat.py b/lib/disco/compat.py
index 4630d5f..4139234 100644
--- a/lib/disco/compat.py
+++ b/lib/disco/compat.py
@@ -12,7 +12,7 @@ if sys.version_info[0] == 3:
def str_to_bytes(s):
if isinstance(s, bytes):
return s
- return s.encode('utf-8')
+ return str(s).encode('utf-8')
def bytes_to_str(b):
if isinstance(b, str):
return b
diff --git a/lib/disco/worker/__init__.py b/lib/disco/worker/__init__.py
index bfb74bc..4974cd7 100644
--- a/lib/disco/worker/__init__.py
+++ b/lib/disco/worker/__init__.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
"""
:mod:`disco.worker` -- Python Worker Interface
==============================================
diff --git a/lib/disco/worker/classic/worker.py b/lib/disco/worker/classic/worker.py
index ff477d8..d7ee37d 100644
--- a/lib/disco/worker/classic/worker.py
+++ b/lib/disco/worker/classic/worker.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
"""
:mod:`disco.worker.classic.worker` -- Classic Disco Runtime Environment
=======================================================================
diff --git a/lib/disco/worker/pipeline/worker.py b/lib/disco/worker/pipeline/worker.py
index 51d4e3d..bdc2159 100644
--- a/lib/disco/worker/pipeline/worker.py
+++ b/lib/disco/worker/pipeline/worker.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
"""
:mod:`disco.worker.pipeline.worker` -- Pipeline Disco Runtime Environment
===========================================================================
diff --git a/lib/disco/worker/simple.py b/lib/disco/worker/simple.py
index 078f605..c0a37e8 100644
--- a/lib/disco/worker/simple.py
+++ b/lib/disco/worker/simple.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
"""
:mod:`disco.worker.simple` -- Simple Worker
===========================================
diff --git a/lib/setup.py b/lib/setup.py
index c93fd94..9c61fa3 100644
--- a/lib/setup.py
+++ b/lib/setup.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
import os
from distutils.core import setup