RE: can the SEEK storage platform be installed in the cloud? - answer is yes

[Taylor, Ronald C]

Hi Stuart,

 

Thanks! Good to know. Our Linux/cloud/supercomputing gurus here at PNNL are seriously considering putting a copy of SEEK in a cloud environment for us. If that proceeds, they might want to contact Mr. Haines at your U of Manchester SEEK team, if they run into a problem or have a question.  Stay tuned.

 

Ron

 

Ronald Taylor, Ph.D.

Computational Biology & Bioinformatics Group

Pacific Northwest National Laboratory (U.S. Dept of Energy/Battelle)

Richland, WA 99352

phone: (509) 372-6568

email: ronald...@pnnl.gov

web page:  http://www.pnnl.gov/science/staff/staff_info.asp?staff_num=7048

 

From: Stuart Owen [mailto:stuar...@manchester.ac.uk]
Sent: Monday, November 17, 2014 3:17 AM
To: seek4s...@googlegroups.com; Quyen Nguyen
Cc: Ronald Taylor; Wiley, Steven; Taylor, Ronald C; Robert Haines
Subject: Re: can the SEEK storage platform be installed in the cloud?

 

Hi,
    Yes. This is the Biovel Portal - https://portal.biovel.eu/ which although it isn't immediately apparent, but is actually the SEEK platform with a different skin on it.
    This runs on an Amazon AWS. It's a large one, but this is mainly to facilitate the running of Taverna workflows, as there is also a Taverna workflow execution server running on it.
    They do have a couple of problems related to authorization performance, due to the large number of workflow runs they store, which we are looking into. But other than that I don't believe they have any drawbacks.
    I've CC'd Robert Haines who worked on setting up this AWS image.

Stuart.

On 14/11/2014 19:57, Taylor, Ronald C wrote:

Hi Quyen,

 

Got another question for you:  has the SEEK platform ever been installed in the cloud? That is, rather than installing a production copy on a single Linux server, has anybody ever done an install on one of the cloud platforms like the Google Cloud or Amazon AWS? Is that even doable? If doable, what would be the drawbacks?

 

Ron

 

Ronald Taylor, Ph.D.

Computational Biology & Bioinformatics Group

Pacific Northwest National Laboratory (U.S. Dept of Energy/Battelle)

Richland, WA 99352

phone: (509) 372-6568

email: ronald...@pnnl.gov

web page:  http://www.pnnl.gov/science/staff/staff_info.asp?staff_num=7048

 

--
If you have installed SEEK - please take a moment to fill out our registration form at http://www.seek4science.org/seek-registration
---
You received this message because you are subscribed to the Google Groups "SEEK" group.
To unsubscribe from this group and stop receiving emails from it, send an email to seek4science...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

 

[Robert Haines]

Hi Ronald,

Sorry for not replying earlier.

> Thanks! Good to know. Our Linux/cloud/supercomputing gurus here at PNNL
> are seriously considering putting a copy of SEEK in a cloud environment
> for us. If that proceeds, they might want to contact Mr. Haines at your
> U of Manchester SEEK team, if they run into a problem or have a
> question. Stay tuned.

As Stuart says we have installed a very simple SEEK setup in AWS. It
doesn't auto load-balance or anything like that as we've never had the
need to do that. In theory all you'd need to do to load-balance SEEK is
make sure that all your balanced SEEKs connect to the same database and
have access to a shared file-system (such as S3 in AWS).

Happy to help if I can.

Best wishes,
Rob

>
> Ronald Taylor, Ph.D.
>
> Computational Biology & Bioinformatics Group
>
> Pacific Northwest National Laboratory (U.S. Dept of Energy/Battelle)
>
> Richland, WA 99352
>
> phone: (509) 372-6568
>
> email: ronald...@pnnl.gov
>
> web page: http://www.pnnl.gov/science/staff/staff_info.asp?staff_num=7048
>

> *From:*Stuart Owen [mailto:stuar...@manchester.ac.uk]
> *Sent:* Monday, November 17, 2014 3:17 AM
> *To:* seek4s...@googlegroups.com; Quyen Nguyen
> *Cc:* Ronald Taylor; Wiley, Steven; Taylor, Ronald C; Robert Haines
> *Subject:* Re: can the SEEK storage platform be installed in the cloud?

> email: ronald...@pnnl.gov <mailto:ronald...@pnnl.gov>

>
> web page:
> http://www.pnnl.gov/science/staff/staff_info.asp?staff_num=7048
>
> --
> If you have installed SEEK - please take a moment to fill out our
> registration form at http://www.seek4science.org/seek-registration
> ---
> You received this message because you are subscribed to the Google
> Groups "SEEK" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to seek4science...@googlegroups.com

> <mailto:seek4science...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.
>

--
Robert Haines
myGrid Team
School of Computer Science
University of Manchester

[minn...@gmail.com]

Hi, 

We have installed SEEK and we are assessing the tool to see if it satisfies our requirements list one of which being storing the files in AWS S3.

We have an S3 bucket and I have changed the amazon_s3.yml file in the config directory to point to our bucket.

Now I need to change the configurations to use the S3 and this is were I am struggling.

I am hoping to use your help with this as well as initializing Taverna workflows through seek which falls under a different thread.

Thank you for your help in advance.

Mahsa

>     <mailto:seek4science+unsub...@googlegroups.com>.

[Stuart Owen]

HI Masha,

I think that amazon_s3.yml file came from a plugin at some point, and I'm not sure that alone it will be all that you need to do.

SEEK was once run using an S3 store, but not for some time.

SEEK uses both an SQL database for storing basic information, and also the local drive for storing files, such as uploaded assets. The location of the stored file is by default "filestore/" relative to the root  of the SEEK directory, but can also be reconfigured to point to another disk or folder.

Would a simple solution of mounting an S3 store as a folder, and then using this as the SEEK filestore be a suitable solution? There are a number of linux solutions for mounting S3 stores, some are described in: https://cloud.netapp.com/blog/amazon-s3-as-a-file-system

regards,

Stuart

To unsubscribe from this group and stop receiving emails from it, send an email to seek4science...@googlegroups.com.

[MIN GEIN]

Hi Stuart, 

Thank you for the comment. 

Your solution to mount S3 store as a folder is good for us since we are just evaluating the software now.

We learnt there are also other ways of syncing the "filetore" folder with  S3 that will be good enough for now.

Thanks for your help.

Regards,

Masha

>     <mailto:seek4science+unsubscrib...@googlegroups.com>.

>     For more options, visit https://groups.google.com/d/optout.
>


--
Robert Haines
myGrid Team
School of Computer Science
University of Manchester

--
If you have installed SEEK - please take a moment to fill out our registration form at http://www.seek4science.org/seek-registration
---
You received this message because you are subscribed to the Google Groups "SEEK" group.

To unsubscribe from this group and stop receiving emails from it, send an email to seek4science+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

If you have installed SEEK - please take a moment to fill out our registration form at http://www.seek4science.org/seek-registration
---

You received this message because you are subscribed to a topic in the Google Groups "SEEK" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/seek4science/SAFsr3WvKb4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to seek4science+unsubscribe@googlegroups.com.