"socket closed!!" in standard out

[Ariel]

Hello,

I am trying to run multiple computations on the same data-set, which
is stored in a rather large numpy array (it's data from an MRI scan).
After I discovered that it would be impossible to pass the data as an
argument to cloud calls, I saved the data into a .pkl and then used
cloud.files.put to store it in the cloud. Then, in the definition of
the function that gets called using cloud.map, I use the following
pattern:

fname = 'data.pkl'
dafa_f = could.files.getf(fname)
data = pickle.load(data_f)

At this point, this function should proceed to analyze the data that
was stored in the array and return the result of the analysis into the
code run on the client side. But, the jobs I ran are failing and I am
getting the following standard output:

socket closed!!

And the following exception:

Traceback (most recent call last):
File "/root/.local/lib/python2.7/site-packages/cloudserver/workers/
employee/child.py", line 583, in run
File "/biac4/wandell/biac2/wandell6/users/arokem/projects/wmret/
boot_dwi.py", line 172, in booter
File "/afs/picloud.com/environment/base/2/usr/lib/python2.7/
pickle.py", line 1378, in load
return Unpickler(file).load()
File "/afs/picloud.com/environment/base/2/usr/lib/python2.7/
pickle.py", line 858, in load
dispatch[key](self)
File "/afs/picloud.com/environment/base/2/usr/lib/python2.7/
pickle.py", line 880, in load_eof
raise EOFError
EOFError

What does this mean? Should I be using some other strategy to perform
this kind of computation?

Thanks,

Ariel

[Aaron Staley]

Hi Ariel,

Could you give us a sense of how large of an object you are trying to unpack?  Also, how long does it take (locally) to unpickle your object?

Our cloud.files system sits behind a load balancer that may terminate a connection after 60 seconds.  If the unpickling route takes longer than this, the connection can get cut.

Assuming your unpickling is taking a long time, some suggestions:

1) Use cPickle i(http://docs.python.org/library/pickle.html#module-cPickle) instead of pickle if you aren't already.  cPickle runs significantly faster than pickle.

2) Consider using a c2 instance type which will further allow unpickling to run faster.

3) Consider reading the entire object into memory and then unpickling it:

fname = 'data.pkl'
dafa_f = could.files.getf(fname)

pickled_data = data_f.read()
data = cPickle.loads(pickled_data)

(The disadvantage of this option is that it can run a bit slower and take more memory.)

Regards,

Aaron Staley

PiCloud, Inc.