Can't connect

[Diego Herrera]

Hello,

When I try to connect to ami-06845b6f through NX, it keeps on saying
connection refused even though I gave my instance permission to allow
connections though both HTTP and SSH in the security group. The user
manual on the CloudBioLinux site suggested to first log in through
terminal, which was not in the Youtube video, but still I could not
connect.

I am using OS 10.6.8 and have been able to connect to instances I have
created in the past. However, this is my first time trying to connect
to a public image.

Diego

[Enis Afgan]

Hi Diego,
What type/size of an instance are you using?

Diego

--
You received this message because you are subscribed to the Google Groups "cloudbiolinux" group.
To post to this group, send email to cloudb...@googlegroups.com.
To unsubscribe from this group, send email to cloudbiolinu...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cloudbiolinux?hl=en.

[Diego Herrera]

I am using the free tier, so it is a micro instance.

Pretty much, all I want to do is transfer one genome in 1000 Genomes
Public Data Set to my S3 bucket to then visualize my output data on
the NX remote desktop.

Since I have not been able to connect to the Bio-Linux image, I have
not looked into what features it offers besides the data set, so what
I am trying to do may not be necessary. I want to run my own
application, Real Time Genomics, with a cluster I created with
Starcluster.

Diego

On Apr 15, 4:12 pm, Enis Afgan <eaf...@emory.edu> wrote:
> Hi Diego,
> What type/size of an instance are you using?
>
> On Mon, Apr 16, 2012 at 5:25 AM, Diego Herrera

> <diego.herrera...@gmail.com>wrote:

[Enis Afgan]

I forgot to mention this in the previous email, maybe give biocloudcentral.org a shot when starting an instance - this will ensure all of the security groups++ are all setup properly. 

Can you ssh to the instance? If so, run the following command after loggin in:

sudo service cloudman start

Then wait a few seconds for the system to complete the configuration and try connecting via NX again. I ran into an issue of this kind with small instance types (micro/small). I'll work on having it fixed permanently in the next few days but a new AMI will have to be built for that to take effect so don't wait on it quite today.

On a side note, considering you've mentioned the scaling and customization aspects of your analysis/tool - are you familiar with CloudMan (usecloudman.org)? It's part of the CloudBioLinux image and it allows you to scale the size of your cloud platform via a backend cluster, customize the set of available tools, and also share what you've created?

Enis

[Diego Herrera]

Awesome, yeah this is for a class a project so it won't be a problem
to share what I have made. I will get back to you within a couple of
days, if not sooner since I have to study for some exams.

Diego

On Apr 15, 4:49 pm, Enis Afgan <afg...@gmail.com> wrote:
> I forgot to mention this in the previous email, maybe give
> biocloudcentral.org a shot when starting an instance - this will ensure all
> of the security groups++ are all setup properly.
>
> Can you ssh to the instance? If so, run the following command after loggin
> in:
> sudo service cloudman start
> Then wait a few seconds for the system to complete the configuration and
> try connecting via NX again. I ran into an issue of this kind with small
> instance types (micro/small). I'll work on having it fixed permanently in
> the next few days but a new AMI will have to be built for that to take
> effect so don't wait on it quite today.
>
> On a side note, considering you've mentioned the scaling
> and customization aspects of your analysis/tool - are you familiar with
> CloudMan (usecloudman.org)? It's part of the CloudBioLinux image and it
> allows you to scale the size of your cloud platform via a backend
> cluster, customize the set of available tools, and also share what you've
> created?
>
> Enis
>
> On Mon, Apr 16, 2012 at 9:35 AM, Diego Herrera

> <diego.herrera...@gmail.com>wrote:

[Diego Herrera]

Also, I am not familiar with usecloudman.org but will look into it. I
wish there was a well written manual out there
that explains step by step how to use Amazon EC2, how build your
cluster and run applications on your data using the
best and most simple packages. Everything I have seen has been only
to start an instance, but never from start
to end on how to complete a real genomics project.

[Enis Afgan]

On Mon, Apr 16, 2012 at 11:17 AM, Diego Herrera <diego.he...@gmail.com> wrote:

Also, I am not familiar with usecloudman.org but will look into it.  I
wish there was a well written manual out there
that explains step by step how to use Amazon EC2, how build your
cluster and run applications on your data using the
best and most simple packages.  Everything I have seen has been only
to start an instance, but never from start
to end on how to complete a real genomics project.

 

Take a look at this link for more complete documentation: http://wiki.g2.bx.psu.edu/Admin/Cloud 

(also, some of the presentations from http://usecloudman.org/publications/ should give you a quick and visual idea of the topics available). 

I would hope the above gives you all the components "to complete a real genomics project" :)

Let us know if you run into problems or have any questions,

Enis

[Diego Herrera]

I started reading a little bit and watched the screencast (very nice)
and I was wondering if there was a command line interface within the
galaxy module? The tool the person used was bwa, which is already
configured in Galaxy. I will be using my own mapping program instead.

Diego

On Apr 15, 6:23 pm, Enis Afgan <afg...@gmail.com> wrote:
> On Mon, Apr 16, 2012 at 11:17 AM, Diego Herrera

> <diego.herrera...@gmail.com>wrote:

>
> > Also, I am not familiar with usecloudman.org but will look into it.  I
> > wish there was a well written manual out there
> > that explains step by step how to use Amazon EC2, how build your
> > cluster and run applications on your data using the
> > best and most simple packages.  Everything I have seen has been only
> > to start an instance, but never from start
> > to end on how to complete a real genomics project.
>
> Take a look at this link for more complete documentation:http://wiki.g2.bx.psu.edu/Admin/Cloud

> (also, some of the presentations fromhttp://usecloudman.org/publications/shouldgive you a quick and visual

[Enis Afgan]

On Tue, Apr 17, 2012 at 6:13 AM, Diego Herrera <diego.he...@gmail.com> wrote:

I started reading a little bit and watched the screencast (very nice)

Thank you!

 

and I was wondering if there was a command line interface within the
galaxy module?

You mean for the tools run through Galaxy or Galaxy itself?

For the first option - Galaxy itself is wrapper for the command line tools so any of the tools available within Galaxy can simply be run through the command line.

For the second option - Galaxy has an API that allows you to script many of the steps: http://wiki.g2.bx.psu.edu/Admin/API

 The tool the person used was bwa, which is already
configured in Galaxy.  I will be using my own mapping program instead.

You can also add your own tool to Galaxy any make if part of the complete analysis: http://wiki.g2.bx.psu.edu/Admin/Tools/Adding%20Tools

Cheers,

Enis

[Diego Herrera]

I was able to connect to the cloud through terminal after following
the links you sent me (building my cluster and all). I plan on
mounting my S3 bucket to my EC2 instance, will I be able to access the
data in that bucket seamlessly?

Also, since my alignment software is in that bucket, can I simply type
the commands to execute the program? It would be nice if I could
upload my own genome browser such as IGV or Savant as well so that I
would not require a remote desktop such as NX. The less things I can
use, the better.

I don't know if you are familiar with Sabalcore, but they were a lot
easier to understand than EC2 in just about everything. The only
thing I didn't like was the cost. Galaxy has definitely made AWS
closer to achieving the ease of use as Sabalcore does.

Diego

On Apr 16, 1:13 pm, Diego Herrera <diego.herrera...@gmail.com> wrote:
> I started reading a little bit and watched the screencast (very nice)
> and I was wondering if there was a command line interface within the
> galaxy module?  The tool the person used was bwa, which is already
> configured in Galaxy.  I will be using my own mapping program instead.
>
> Diego
>
> On Apr 15, 6:23 pm, Enis Afgan <afg...@gmail.com> wrote:
>
> > On Mon, Apr 16, 2012 at 11:17 AM, Diego Herrera
> > <diego.herrera...@gmail.com>wrote:
>
> > > Also, I am not familiar with usecloudman.org but will look into it.  I
> > > wish there was a well written manual out there
> > > that explains step by step how to use Amazon EC2, how build your
> > > cluster and run applications on your data using the
> > > best and most simple packages.  Everything I have seen has been only
> > > to start an instance, but never from start
> > > to end on how to complete a real genomics project.
>
> > Take a look at this link for more complete documentation:http://wiki.g2.bx.psu.edu/Admin/Cloud

> > (also, some of the presentations fromhttp://usecloudman.org/publications/shouldgiveyou a quick and visual

> > > > > CloudMan (usecloudman.org)? It's part of theCloudBioLinuximage and

> > > > > > > > manual on theCloudBioLinuxsite suggested to first log in

> > > > > > To post to this group, send email tocloud...@googlegroups.com.

> > > > > > To unsubscribe from this group, send email to
> > > > > >cloudbiolinu...@googlegroups.com.
> > > > > > For more options, visit this group at
> > > > > >http://groups.google.com/group/cloudbiolinux?hl=en.
>
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "cloudbiolinux" group.

> > > To post to this group, send email tocloud...@googlegroups.com.

[agbiotec]

Hi Diego,

did you manage to ssh into the ami-06845b6f - did you also get NX desktop connection ?

The 1000 human genomes S3 buckets are mounted under /mnt/1000genomes. 

To mount your bucket so you can copy the data you can do the following after you connect:

-- on the command line, just ssh login:

   sudo mkdir /mnt/diegobucket

   if your bucket is private:

   export AWSACCESSKEYID=your_amazon_key

   export AWSSECRETACCESSKEY=your_amazon_secret_key

   sudo s3fs your_bucket_name -o allow_other,readwrite_timeout="120" /mnt/diegobucket

   if your bucket is public (all access) just do this:

   sudo s3fs your_bucket_name -o allow_other,public_bucket=1,readwrite_timeout="120" /mnt/diegobucket

   then you can copy data as you please between /mnt/1000genomes and /mnt/diegobucket

-- if you have remote desktop connection with NX

   you will see the 1000 human genomes buckets as a folder on the desktop

   open firefox from the Applications->Internet menu and install the s3fox plugin  http://www.s3fox.net/

   set your_amazon_key and your_amazon_secret_key in s3fox

   within s3fox navigate to the desktop so you can see the 1000 human genomes folder, and copy whichever

   subfolders you want to your bucket

Please let us know if you have any more questions !

  Ntino

[Diego Herrera]

I have been able to connect by ssh, but was not able to connect by NX. It says that I have the wrong password.  Terminal asks me to set up NX and type a new UNIX password
but it always says that my passwords do not match.  I will follow your recommendations and keep you updated.

[Diego Herrera]

I am getting this error now

root@ip-10-190-46-141:/mnt/diegobucket# ls -la
ls: reading directory .: Input/output error
total 0

What should I do?

Diego

On Sunday, April 15, 2012 12:25:01 PM UTC-7, Diego Herrera wrote:

[Enis Afgan]

What are the permissions on your bucket?

If you're mounting a bucket whose objects you've made public but the bucket itself is not public, you will see this error. Set the bucket itself as public (ie, everyone can read/list it) and things should be OK. 

That's the only situation where I've seen this case. However, if you're mounting a private bucket with creds, not sure what's going on...

Enis

--

You received this message because you are subscribed to the Google Groups "cloudbiolinux" group.

To view this discussion on the web visit https://groups.google.com/d/msg/cloudbiolinux/-/z99TYkK7ljIJ.