[newb] custom c modules

[nbecker]

I'm interested in picloud. My app uses a lot of custom c++ modules
that are based on boost::python, numpy, and pyublas. Would I be able
to use picloud? Any hints how?

[Ken Elkabany]

Hi there,

Sorry we didn't get to your post sooner! It slipped under our radar. We do support custom Python C/C++ extensions that require compilation. There are two ways to do so through our web interface:

1. You can upload an archive (zip, gzip) of your package.

2. You can sync a subversion repository that holds your package.

We will automatically compile your package, and install it on our cluster. A report is generated to assist in debugging compilation errors. We're happy to help you through the process if you run into any problems, as users do tend to have issues with including header files and linking to libraries.

Best,

Ken

[nbecker]

Is there a description of the procedure?

Does this assume a standard python distribution package (using setuptools) to compile?  I need to install several collections of software, which are built with various tools (e.g., scons).  Would I ssh into some host and manually build the c++ modules?

[Ken Elkabany]

Glad you followed up again! We've completely revamped the system. The new feature for deploying custom c++ modules (or anything else non-Python) is called Environments. You can see a quick how to on our blog: http://blog.picloud.com/2011/09/26/introducing-environments-run-anything-on-picloud/

Let us know if you need any clarification.

Best,

Ken

[Neal Becker]

Trying it out.  I got as far as ssh 
ssh -i ~/picloud/privatekey.pem pic...@ec2-184-73-54-48.compute-1.amazonaws.com

But it puts me in a directory that I can't write to:

$ cd

picloud@domU-12-31-39-0C-75-A1:~$ pwd

/home/picloud

picloud@domU-12-31-39-0C-75-A1:~$ touch stuff

touch: cannot touch `stuff': Permission denied

picloud@domU-12-31-39-0C-75-A1:~$ cd /home/setup

-bash: cd: /home/setup: No such file or directory

[Ken Elkabany]

That's a bug that we recently introduced. /home/picloud is incorrectly owned by root, instead of user picloud on the setup server. For the time being, please just use sudo access.

For example,

$ sudo touch stuff

Ken

[Neal Becker]

I'm trying to copy files to the guest.  Unfortunately, I think rsync does dot have a -i <identity.pem> flag.  scp seems to work.  What is your recommended method?

[Ken Park]

Hi Neal,

Try including this option with rsync:

    -e "ssh -i path-to-your-keyfile"

This should tell rsync to use ssh as the remote shell with the given keyfile.

If this doesn't get through, scp is known to work for the setup servers.

Ken Park,

PiCloud, Inc.

[Neal Becker]

Working directory.

I have my application installed into a subdirectory of /home/picloud.  Is this setup workable?  I don't see a way to change my working directory before invoking cloud.call.

There are probably any number of places in my app where I assume that cwd is set to that directory (e.g., c++ code that opens some data files...)

[Ken Elkabany]

Yes, if your application is installed in /home/picloud, your function will be able to access it by either using the absolute path ('/home/picloud/yourprogram'), or, preferably, by its relative path ('yourprogram').

You can change your working directly after invoking cloud.call(), by including in your function:

def your_function():

    if cloud.running_on_cloud():

        import os

        os.chdir('new_dir')

    # the rest of what your function does

Using cloud.running_on_cloud(), we can ensure that the working directory is only changed when your function is being run on PiCloud. Also, you can see how you could install your software to somewhere like '/usr/local/yourprogram', and simply chdir into it when necessary.

Ken