Figured out how to get BioPerl work at MSI machines

[Liangliang Gao]

Using (1) Perlbrew (2) cpanm (3) local::lib

(1) Perlbrew
curl -kL http://install.perlbrew.pl | bash

(2)cpanm install
curl -L http://cpanmin.us | perl - App::cpanminus
(3)local::lib install

cd
wget http://search.cpan.org/CPAN/authors/id/A/AP/APEIRON/local-lib-1.005001.tar.gz
tar zxf local-lib-1.005001.tar.gz
cd ~/local-lib-1.005001
perl Makefile.PL --bootstrap
make test && make install
echo 'eval $(perl -I$HOME/perl5/lib/perl5 -Mlocal::lib)' >>~/.bashrc

(4) So finally install BioPerl
cpanm BioPerl


 
######################
Without any prompt or configuration it will just install BioPerl to your MSI account! 
Cheers!
#################

[Joseph Guhlin]

Hmm, did you run perlbrew and install a perl version? I wonder if we
can streamline this.

with
perlbrew init

add
source ~/perl5/perlbrew/etc/bashrc
to your ~/.bash_profile and ~/.bashrc using:
cat "source ~/perl5/perlbrew/etc/bashrc" >> ~/.bash_profile
cat "source ~/perl5/perlbrew/etc/bashrc" >> ~/.bashrc

then:

perlbrew install perl-5.16.0

I think that will set up local::lib for you ?
It also gets you the newest version of perl
perl -v
(which is a little bit faster too)

If not then I think you can skip perlbrew too, but I think installing
it is better.

Awesome job though getting it on MSI. :)

--Joey

> --
> You received this message because you are subscribed to the Google Groups
> "SPADA" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/spada/-/y0vcQFZlZ7AJ.
> To post to this group, send email to sp...@googlegroups.com.
> To unsubscribe from this group, send email to
> spada+un...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/spada?hl=en.

[Liangliang Gao]

Strangely, when I use Bioperl in my own MSI account, it works fine.
However, when I submit jobs to HPC using qsub, error message comes saying that

"Can't locate Bio/SeqIO.pm in @INC (@INC contains: /home/msi/gaoll/perl5/perlbrew/perls/perl-5.12.3/lib/site_perl/5.12.3/x86_64-linux /home/msi/gaoll/perl5/perlbrew/perls/perl-5.12.3/lib/site_perl/5.12.3 /home/msi/gaoll/perl5/perlbrew/perls/perl-5.12.3/lib/5.12.3/x86_64-linux /home/msi/gaoll/perl5/perlbrew/perls/perl-5.12.3/lib/5.12.3 .) at /home/msi/gaoll/bin/convert.fq.pl line 4.
BEGIN failed--compilation aborted at /home/msi/gaoll/bin/convert.fq.pl line 4."

[Joseph Guhlin]

The lab computers for MSI use /home/msi/(yourname) but the HPC uses
/home/elmo1/(yourname) as your home directory.

Because you have installed perl into your local home directory you
would have to re-do your steps on elmo1 (or other HPC). I recommend
running this on the lab queue instead of the elmo queue as the lab
queue is all Redhat CentOS Linux and elmo is SUSE Unix, and they are
not binary compatible. SPADA does not use more than 1 or 2 cores
anyways so the lab computers are generally fine to run this on.

How are you submitting your job for spada?

Best,
--Joseph

> --
> You received this message because you are subscribed to the Google Groups
> "SPADA" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/spada/-/mKRHBzrz0pcJ.

[Liangliang Gao]

#PBS -l mem=2gb,walltime=72:00:00
#pBS -q lab

/home/msi/gaoll/perl5/perlbrew/build/perl-5.12.3/perl /home/msi/gaoll/bin/convert.fq.pl /home/msi/gaoll/unix.perl.c/perl/convert.fq/test.fastq

[Joseph Guhlin]

Ah, that is the lab queue, sorry about the other stuff(ignore
everything I said about elmo).

PBS does not run your bash file when it starts.

Change it to this:

#PBS -l mem=2gb,walltime=72:00:00
#pBS -q lab

bash -l -c "/home/msi/gaoll/perl5/perlbrew/build/perl-5.12.3/perl
/home/msi/gaoll/bin/convert.fq.pl
/home/msi/gaoll/unix.perl.c/perl/convert.fq/test.fastq "

> --
> You received this message because you are subscribed to the Google Groups
> "SPADA" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/spada/-/e26jvVyP2tMJ.

[Liangliang Gao]

#PBS -l mem=2gb,walltime=72:00:00

#PBS -q lab

bash -l -c "/home/msi/gaoll/perl5/perlbrew/build/perl-5.12.3/perl
/home/msi/gaoll/bin/convert.fq.pl
/home/msi/gaoll/unix.perl.c/perl/convert.fq/test.fastq "

######################################################################################

Can't locate Bio/SeqIO.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /home/msi/gaoll/bin/convert.fq.pl line 4.

BEGIN failed--compilation aborted at /home/msi/gaoll/bin/convert.fq.pl line 4.

bash: line 2: /home/msi/gaoll/unix.perl.c/perl/convert.fq/test.fastq: Permission denied

[Joseph Guhlin]

Hmm, do you have anything to set up local lib again in your .bashrc or
.bash_profile files in your home directory?

Going to be on this side of campus today, I can take a look.

Best,
--Joseph

> --
> You received this message because you are subscribed to the Google Groups
> "SPADA" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/spada/-/2U_nMTVR2WYJ.

[Liangliang Gao]

I think that BioPerl is clearly functional at gaoll@lab005. See below: 

##################################################################

gaoll@lab005: ~/unix.perl.c/perl/convert.fq$ ls -rtl
total 12
-rw-r--r--. 1 gaoll bradeenj 276 Jul  5 00:03 test.fastq
-rwxr-xr-x. 1 gaoll bradeenj 541 Jul  5 00:34 convert.fq.batch.pl
-rw-r--r--. 1 gaoll bradeenj 141 Jul  5 11:38 test.fastq.convert.j
gaoll@lab005: ~/unix.perl.c/perl/convert.fq$ cat test.fastq.convert.j

#PBS -l mem=2gb,walltime=72:00:00
#PBS -q lab

perl /home/msi/gaoll/bin/convert.fq.pl /home/msi/gaoll/unix.perl.c/perl/convert.fq/test.fastq
gaoll@lab005: ~/unix.perl.c/perl/convert.fq$ perl /home/msi/gaoll/bin/convert.fq.pl /home/msi/gaoll/unix.perl.c/perl/convert.fq/test.fastq
gaoll@lab005: ~/unix.perl.c/perl/convert.fq$ ls -rtl
total 16
-rw-r--r--. 1 gaoll bradeenj 276 Jul  5 00:03 test.fastq
-rwxr-xr-x. 1 gaoll bradeenj 541 Jul  5 00:34 convert.fq.batch.pl
-rw-r--r--. 1 gaoll bradeenj 141 Jul  5 11:38 test.fastq.convert.j
-rw-r--r--. 1 gaoll bradeenj 276 Jul  5 11:45 test.fastq.converted
gaoll@lab005: ~/unix.perl.c/perl/convert.fq$ cat test.fastq
@HWI-ST261:1:1:1199:2075#ACTTGA/1
...C...................TA........................C
+
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
@HWI-ST261:1:1:1043:2097#ACTTGA/1
........................A.........................
+
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
gaoll@lab005: ~/unix.perl.c/perl/convert.fq$ cat test.fastq.converted
@HWI-ST261:1:1:1199:2075#ACTTGA/1
...C...................TA........................C
+
##################################################
@HWI-ST261:1:1:1043:2097#ACTTGA/1
........................A.........................
+
##################################################
############################################################################################


gaoll@lab005: ~/unix.perl.c/perl/convert.fq$ cat ~/bin/convert.fq.pl

#!/home/msi/gaoll/perl5/perlbrew/build/perl-5.12.3/perl -w

use strict;
use Bio::SeqIO;

my $file=shift @ARGV;

my $seqio_in=Bio::SeqIO->new(-file => "< $file", -format=>'fastq', -variant=>'illumina', -alphabet=>'dna');
my $seqio_out=Bio::SeqIO->new(-file=>"> $file.converted",-format=>'fastq',-variant=>'sanger',-alphabet=>'dna');
while (my $seq=$seqio_in->next_seq){
        $seqio_out->write_seq($seq);
}

[Joseph Guhlin]

Yes, I think it is working too, the problem I think is the environment
for perlbrew is not fully set up, and it is not set up automatically
in PBS.

In your ~/.bashrc file you should have this line: (or ~/.bash_profile)
source ~/perl5/perlbrew/etc/bashrc

and calling bash above should re-enable it. Not sure why it isn't. Can
you check that that line is there?

--Joseph

> --
> You received this message because you are subscribed to the Google Groups
> "SPADA" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/spada/-/GktU3Lrs-UcJ.

[Liangliang Gao]

I have that line on both ~/.bashrc and ~/.bash_profile

gaoll@lab005: ~/unix.perl.c/perl/convert.fq$ grep source ~/.bashrc
source /home/msi/gaoll/perl5/perlbrew/etc/bashrc
gaoll@lab005: ~/unix.perl.c/perl/convert.fq$ grep source ~/.bash_profile
source ~/perl5/perlbrew/etc/bashrc

[Joseph Guhlin]

Hm, then you shouldn't be getting the bioperl error from your pbs script.

Make sure not to put everything on different lines if it is just one command.

Maybe try this and let me know what you get:

#PBS -l mem=2gb,walltime=72:00:00
#PBS -q lab

bash -l -c "source /home/msi/gaoll/perl5/perlbrew/etc/bashrc;
/home/msi/gaoll/perl5/perlbrew/build/perl-5.12.3/perl
/home/msi/gaoll/bin/convert.fq.pl
/home/msi/gaoll/unix.perl.c/perl/convert.fq/test.fastq "

--Joseph

> --
> You received this message because you are subscribed to the Google Groups
> "SPADA" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/spada/-/kv9qVhE4pDkJ.

[Liangliang Gao]

Thank you Joseph!
use lib "/home/msi/gaoll/perl5/lib/perl5"
worked!

I tried to add
export PERL5LIB="/home/msi/gaoll/perl5/lib/perl5"
to my .bashrc without further luck.

I am woundering in SPADA, maybe I could simply add one more line to the code(s).

Or I can simply run it on isub instead of qsub.

Cheers!