Problem installing biolinux tools

[Zhi-Wei Lu]

I have tried to install some cloudbiolinux tools for Galaxy-P dependencies, but I had quite some difficulties.

I was able to install the OS dependencies for cloudbiolinux on Ubuntu 12.04. 

fab -f fabfile.py -H root@localhost install_biolinux:target=packages,flavor=proteomics/galaxyp

but when I tried to install the custom tools for galaxy, it finished but didn't install any tool!

fab -f fabfile.py -H root@localhost install_custom:galaxy_tools

[root@localhost] Executing task 'install_custom'

INFO: This is a Base Flavor - no overrides

INFO: Reading default fabricrc.txt

DBG [config.py]: Using config file /opt/cloudbiolinux/cloudbio/../config/fabricrc.txt

DBG [__init__.py]: CloudBioLinux Edition 1.6.1

INFO: This is a cloudbiolinux

INFO: Distribution ubuntu

INFO: Ubuntu setup

DBG [distribution.py]: Debian-shared setup

DBG [distribution.py]: Source=precise

DBG [distribution.py]: NixPkgs: Ignored

[root@localhost] run: echo $HOME

[root@localhost] out: /root

[root@localhost] out:

[root@localhost] run: uname -m

[root@localhost] out: x86_64

[root@localhost] out:

INFO: Now, testing connection to host...

INFO: Connection to host appears to work!

DBG [utils.py]: Expand paths

DBG [config.py]: Using config file /opt/cloudbiolinux/cloudbio/../config/custom.yaml

INFO: Reading /opt/cloudbiolinux/cloudbio/../config/custom.yaml

DBG [shared.py]: Packages to install: anaconda,emboss,pgdspider,bowtie,bowtie2,bwa,gmap,lastz,mosaik,snap,stampy,bamtools,bedtools,cram,dw\

gsim,fastq_screen,fastqc,fastx_toolkit,ogap,varianttools,vcftools,bfast,novoalign,novosort,plink_seq,ucsc_tools,cufflinks,picard,sambamba,\

samtools,shrec,tophat,vep,hydra,bcbio_variation,crisp,freebayes,gatk,gemini,grabix,snpeff,stacks,tabix,tassel,varscan,vcflib,macs,abyss,co\

rtex_var,ray,transabyss,trinity,velvet,directag,idpqonvert,myrimatch,mzmine,omssa,pepitome,percolator,tagrecon,transproteomic_pipeline,mor\

pheus,proteomics_wine_env,proteowizard,nginx,novnc,proftpd,sge,gnu_parallel,pydoop,seal,galaxy_tools,galaxy_webapp,proteomics_tools,protkg\

em,protvis,leiningen,beast,tracer,bx-python,netsa-python,rpy,s3fs,viralassembly,viralassembly_cleanall,viralvigor,viralvigor_cleanall,vira\

lvigor_test,viralvigor_validate

INFO: Custom install for 'galaxy_tools' start time: 2013-07-10 17:10:17.130441

DBG [fabfile.py]: Import galaxy_tools

[root@localhost] sudo: useradd -d /home/sgeadmin --create-home --shell /bin/bash -c"Galaxy-required user"  --user-group sgeadmin

DBG [__init__.py]: Done setting up CloudMan users

INFO: Custom install for 'galaxy_tools' end time: 2013-07-10 17:10:17.688372; duration: 0:00:00.557931

Done.

Disconnecting from root@localhost... done.

That was about three weeks ago, I tried again today by cloning the latest cloudbiolinux from github, now I am getting more errors right at the beginning,

I updated the fabric today as well, but nothing helped.

fab -f fabfile.py -H root@localhost install_biolinux:target=packages,flavor=proteomics/galaxyp

would fail, with 

[root@localhost] Executing task 'install_biolinux'

INFO: Config start time: 2013-07-31 19:02:05.769718

INFO: This is a Base Flavor - no overrides

INFO: Reading default fabricrc.txt

DBG [config.py]: Using config file /opt/cloudbiolinux/cloudbio/../config/fabricrc.txt

DBG [__init__.py]: CloudBioLinux Edition 1.7.0

INFO: This is a cloudbiolinux

INFO: Distribution ubuntu

INFO: Ubuntu setup

DBG [distribution.py]: Debian-shared setup

DBG [distribution.py]: Source=precise

Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/fabric/main.py", line 743, in main

    *args, **kwargs

  File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 368, in execute

    multiprocessing

  File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 264, in _execute

    return task.run(*args, **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 171, in run

    return self.wrapped(*args, **kwargs)

  File "/opt/cloudbiolinux/fabfile.py", line 71, in install_biolinux

    and target in ["libraries", "custom"]))

  File "/opt/cloudbiolinux/cloudbio/utils.py", line 95, in _configure_fabric_environment

    _setup_distribution_environment(ignore_distcheck=ignore_distcheck)

  File "/opt/cloudbiolinux/cloudbio/distribution.py", line 32, in _setup_distribution_environment

    configure_runsudo(env)

  File "/opt/cloudbiolinux/cloudbio/fabutils.py", line 156, in configure_runsudo

    env.safe_put = put

NameError: global name 'put' is not defined

but 

fab -f fabfile.py -H localhost install_biolinux:target=packages,flavor=proteomics/galaxyp

works! (without root@localhost above)

but 

fab -f fabfile.py -H localhost install_custom:galaxy_tools

[localhost] Executing task 'install_custom'

INFO: This is a Base Flavor - no overrides

INFO: Reading default fabricrc.txt

DBG [config.py]: Using config file /opt/cloudbiolinux/cloudbio/../config/fabricrc.txt

DBG [__init__.py]: CloudBioLinux Edition 1.6.1

INFO: This is a cloudbiolinux

INFO: Distribution ubuntu

INFO: Get local environment

INFO: Ubuntu setup

DBG [distribution.py]: Debian-shared setup

DBG [distribution.py]: Source=precise

DBG [distribution.py]: NixPkgs: Ignored

[localhost] local: echo $HOME

[localhost] local: uname -m

INFO: Now, testing connection to host...

INFO: Connection to host appears to work!

DBG [utils.py]: Expand paths

DBG [config.py]: Using config file /opt/cloudbiolinux/cloudbio/../config/custom.yaml

INFO: Reading /opt/cloudbiolinux/cloudbio/../config/custom.yaml

DBG [shared.py]: Packages to install: anaconda,emboss,pgdspider,bowtie,bowtie2,bwa,gmap,lastz,mosaik,snap,bamtools,bedtools,cram,dwgsim,fa\

stq_screen,fastqc,fastx_toolkit,ogap,varianttools,vcftools,bfast,novoalign,novosort,plink_seq,ucsc_tools,cufflinks,picard,sambamba,samtool\

s,shrec,tophat,vep,hydra,bcbio_variation,crisp,freebayes,gatk,gemini,grabix,mutect,snpeff,stacks,tabix,tassel,varscan,vcflib,macs,abyss,co\

rtex_var,ray,transabyss,trinity,velvet,directag,idpqonvert,myrimatch,mzmine,omssa,pepitome,percolator,tagrecon,transproteomic_pipeline,mor\

pheus,proteomics_wine_env,proteowizard,nginx,novnc,proftpd,sge,gnu_parallel,pydoop,seal,galaxy_webapp,cbl_galaxy_tools,galaxy_protk,proteo\

mics_tools,protvis,leiningen,beast,tracer,bx-python,netsa-python,rpy,s3fs,viralassembly,viralassembly_cleanall,viralvigor,viralvigor_clean\

all,viralvigor_test,viralvigor_validate

INFO: Custom install for 'galaxy_tools' start time: 2013-07-31 18:29:25.353365

DBG [fabfile.py]: Importing module cloudbio.custom.galaxy_tools

DBG [fabfile.py]: Looking for custom install function cloudbio.custom.galaxy_tools.install_galaxy_tools

Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/fabric/main.py", line 739, in main

    *args, **kwargs

  File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 317, in execute

    multiprocessing

  File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 213, in _execute

    return task.run(*args, **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 123, in run

    return self.wrapped(*args, **kwargs)

  File "/opt/cloudbiolinux/fabfile.py", line 259, in install_custom

    fn = _custom_install_function(env, p, pkg_to_group)

  File "/opt/cloudbiolinux/fabfile.py", line 303, in _custom_install_function

    % (p, pkg_to_group[p]))

KeyError: 'galaxy_tools'

I would appreciate to get some help resolving the problem of installing cloudbiolinux tools. Actually, I have install most of those tools by hand before, but

using cloudbiolinux would save me a lot of time I thought.

Thank you.

[Brad Chapman]

Zhi-Wei;
Thanks for trying out CloudBioLinux and sorry about the issues. I can
help with one of your issues but hopefully John Chilton can stop by with
more information about the Galaxy-P specific problems.

> I have tried to install some cloudbiolinux tools for Galaxy-P dependencies,
> but I had quite some difficulties.

[...]

> NameError: global name 'put' is not defined

Apologies, this was a bug in CloudBioLinux. I pushed a fix so if you
re-get the code from GitHub this will work correctly now.

> fab -f fabfile.py -H localhost install_custom:galaxy_tools

[...]
> KeyError: 'galaxy_tools'

Hopefully John can help with the right invocation for this one. Sorry
I'm not as familiar with his GalaxyP setup as I should be.

Thanks again for the patience getting this going,
Brad

[John Chilton]

Hey Zhi-Wei,

We made some changes to CloudBioLinux at the CodeFest earlier this
month and I had not updated the Galaxy-P installation instructions to
reflect these changes. The command you ran should now be:

fab -f fabfile.py -H localhost install_custom:cbl_galaxy_tools

In tracking this down though I noticed another bug in the install
instructions. It says to run that command, but that command does not
do anything unless fabricrc.txt also contains the following override:

galaxy_install_dependencies = True

Thanks for the interest in Galaxy-P and I am sorry the install
instructions are so half baked (or completely unbaked), this was
really rushed out the door before ASMS and I have not had a good
opportunity to revisit them or even test them.

I will be running through the install instructions on fresh Ubuntu and
CentOS images this week and weekend. So you may want to wait as I work
through the bugs. Brad also built a sort of installer framework on top
of CloudBioLinux at the CodeFest that I hope to utilize to build a
proper installer for Galaxy and Galaxy-P.

Thanks and sorry again,
-John

> --
> You received this message because you are subscribed to the Google Groups
> "cloudbiolinux" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cloudbiolinu...@googlegroups.com.
> To post to this group, send email to cloudb...@googlegroups.com.
> Visit this group at http://groups.google.com/group/cloudbiolinux.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>