upgrading pandas on ubuntu 11.04 environment

[Richard Styron]

Hi,

I'm running some models on an Ubuntu 11.04 environment, and I am getting an error that I think is related to an obsolete version of Pandas installed on the environment (at the end of the email).  Note that the code works fine on my local machine, and if I do things manually with IPython after ssh-ing into the environment (Ipython must have a different search path for modules).

I have tried to upgrade Pandas on the environment using Pip and have 0.9.1 installed.  If I ssh into the environment and import Pandas with the terminal, it accesses the new version (e.g. if I run a python script that just says: 

import pandas

print pandas.version.version

)

But if I run my picloud scripts, it imports Pandas v. 0.3.0.  I have tried to upgrade using apt but it doesn't find pandas, so I don't know how this version of Pandas was installed or what I can do to upgrade it.  

There seem to be two ways to deal with this: either upgrade the Pandas version in python's path, or specify which version to use.  I prefer the first, but if the second is quick, I would be happy to do that.  Unfortunately I don't know how to do either.  Any help would be appreciated.

Thanks,

Richard

Pandas error:

 Traceback (most recent call last):

  File "/usr/local/picloud/.employee/pimployee/job_util.py", line 119, in process_job

    result = func(*args, **kwargs)

  File "/home/itchy/ecopetrol/ec-working/pecube_scripts/run_eceS1_cloud.py", line 300, in run_pecube_map

  File "/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py", line 125, in __init__

    sdict, columns, index = self._init_dict(data, index, columns, dtype)

  File "/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py", line 176, in _init_dict

    v = Series(v, index=index)

  File "/usr/local/lib/python2.7/dist-packages/pandas/core/series.py", line 172, in __new__

    subarr.index = index

  File "/usr/local/lib/python2.7/dist-packages/pandas/core/series.py", line 193, in _set_index

    raise AssertionError('Lengths of index and values did not match!')

AssertionError: Lengths of index and values did not match!

[Ken Elkabany]

Hi Richard,

You should be able to run the following commands in your environment to install the latest pandas:

$ sudo pip uninstall pandas

$ sudo pip install pandas

Be sure to save our environment.

Also, if you use our latest base environment "base/precise" (from Ubuntu 12.04 Precise), you'll have access to pandas 0.9.0 by default. Unfortunately, you would have to re-create your custom environment.

Best,

Ken

[Richard Styron]

Ken,

Thanks for the quick reply.  I had tried that and it didn't help, so I tried it again to make sure, with no obvious effect.  I then simply uninstalled pandas, and re-ran my script, after changing it so that 'import pandas' is in the function that is called by cloud.map (i.e. it's not being imported on my machine).  It still runs the script, and imports pandas just fine (though the 0.3.0 version).  

So I put:

'print pandas.__file__'

in the function called by cloud.map and it results in this: 

'/usr/local/lib/python2.7/dist-packages/pandas/__init__.pyc'

However, if I ssh into the environment, 'locate pandas' turns up nothing, and pandas is not found in that path, either.  Nor is it there on my machine, so it doesn't seem to be importing it from my machine.

It's also strange because if I ssh into the environment and type 'which python' I get '/usr/bin/python' not '/usr/local/lib/python' as I would expect if that's where pandas was being imported from.  Pip also installed things into /usr/bin/python2.7/...

I am wondering if A) saving the environment is not working properly, or B) there is some repository of auxiliary python modules that is accessed by the cloud module (though this seems a little unlikely to me, although I really don't know).

I'd prefer not to make a new environment because it's a pretty big pain to get the finite element modeling software I use on the environment, and I've heard it has problems with fresh installs on 12.04 because of some weird dependency problems, although I haven't seen for myself.

Thanks,

Richard 

[Ken Elkabany]

The behavior is definitely unexpected. Could you please double check that the environment you're specifying with the _env flag is the correct one? If so, please file a support ticket, and we'll take a look into your account with your permission.

Ken