OSU OpenSource Labs (OSL) Planning Work - CORE SERVICES FOUNDATION
deborah shaddon
for OSL:
As you probably read in the CrisisCommons announcement, we will be
partnering with Oregon State Universities OSL for hosting of some of
our key CrisisCommons capabilities (outlined in the sloan proposal as
well as listed here). This is expected to be a long term engagement,
and our migration to this will happen 'over time'.
http://osuosl.org/about/news/crisis-commons-joins-osl
We've started some initial conversations with the OSL work to
establish the partnership and begin to understand their processes.
Essentially, we will supply some 'hardware' that they can virtualize
and manage for us (and allow us to manage if we want too), and support
capabilities for hosting of opensource projects (such as crisiscommons
itself as well as other 'true' opensource projects), and development/
staging servers, etc. I see this as a model of they will manage the
OS level resources, we can still manage our application level
resources and configurations. This will also allow us to put some
structure into our design and deployment processes by staging into
production, to achieve some stability.
To plan to utilize their environment properly and build a platform to
support our future efforts, some thoughts and considerations, and I
wanted to pose some of this to the community (again, this aligns with
some of the infrastructure-strategic model in the sloan proposal):
Our first order will be to establish and migrate some 'core' services
over, including
1) crisiscommons.org,
2) wiki.crisiscommons.org and
3) crisiscommons.org email.
*(crisiscamp.org will remain a redirect at this time).
But rather than take these 'as is', I want to incorporate the
following high level changes, as building block to the future. PLEASE
provide some thoughts and steps that you can think of for this, this
is meant to be JUST A START, but is focused on some
SystemAdministration sort of activities:
Highlevel Plan/Approach - CORE SERVICES FOUNDATION:
1. The Hardware/OS allocation: Planning for level of virtualization
and capacity (the hardware approach):
*Match of some capacity models against hardware we need to purchase
and how we allocate this (with consideration for future scalability)
*Decision on which linux variation we want (preferences? what do we
have today?)
*basic administration model we want at the OS level (we need to retain
some, but look to delegate as much as possible to them, but retain
some 'merit based' model of administration and control.
*Will identify monitoring and backup strategies.
*Leveraging some models of other OS projects in the OSL.
2. Some application 'services':
*We spoke of application management and common-ids. As such, we want
to leverage (most likely) an LDAP repository that our other components
(via apache basic auth configuration and plug-in integration
components for wordpress and mediawiki to sync ids). This will allow
SSO, and single-id, and we can leverage this for future components
*We can design this appropriately to allow 'seeding' (ie, conversion)
of the IDs from our current systems.
*Consideration also for LDAP and OpenID integration as well (ie, your
LDAP can be your openID).
*crisiscommons.org emails hosted as well, and ALSO tied into the LDAP
(?), but not everyone here would have an email (so policies as well).
*Anybody have any thoughts or conderations for this?
3. The 'Application Software' allocation - SETUP:
*So, we would have a central LDAP above, our own email hosted services
with integration, as initial platform.
*We spoke of breaking up DB servers from App-Servers (a common
practice, even of other OS projects in this lab).
*Continue with Wordpress and MediaWiki environment to start. Plugins
for LDAP, and Apache specific configurations for basic-auth and ldap.
*Do we want to do SSL on some of the pages, namely the login? Need
certificate management then too (and verisign, etc.).
4. CONVERSION and MIGRATION PLANNING:
*We will have to plan on how to do the conversion properly without
loosing content, and allowing for the integration of all this.
*database or export level conversions? What about other content or
settings (non-db?)
*We will want to setup a 'sandbox' of this environment above, to
represent our staging environment, and discover how to get to stage
from pre-prod to prod.
*And integrate into SCM system (which they also provide some basic
procedures and processes for locally).
5. ACTUAL CUTOVER.
*We should do just one system at a time instead of all-at-once (get
our feet wet with their processes and policies).
*How to 'cut off' content updates, etc.
6.ETC
Chris Foote (Spike)
My ideas and comments in-line below:
deborah shaddon wrote:
> Looking for group feedback and thoughts on the proposed move approach
> for OSL:
>
> As you probably read in the CrisisCommons announcement, we will be
> partnering with Oregon State Universities OSL for hosting of some of
> our key CrisisCommons capabilities (outlined in the sloan proposal as
> well as listed here). This is expected to be a long term engagement,
> and our migration to this will happen 'over time'.
> http://osuosl.org/about/news/crisis-commons-joins-osl
>
> We've started some initial conversations with the OSL work to
> establish the partnership and begin to understand their processes.
> Essentially, we will supply some 'hardware'
"we will supply some hardware"? Really?
> that they can virtualize
> and manage for us (and allow us to manage if we want too), and support
> capabilities for hosting of opensource projects (such as crisiscommons
> itself as well as other 'true' opensource projects), and development/
> staging servers, etc.
If there is hardware, what is the point of "virtualizing" it?
> I see this as a model of they will manage the
> OS level resources, we can still manage our application level
> resources and configurations. This will also allow us to put some
> structure into our design and deployment processes by staging into
> production, to achieve some stability.
>
>
OK for them to manage at OS level, but if it's "virtualized" it would be
nice to have view (even if not control) of config and logs and things.
Still not sure why we need to buy hardware? Can we not lease/borrow
virtual server from OSU? Or is the deal that they are homing our
hardware? Sorry, need clarification of this point before we go any further.
---(Rest of good stuff snipped) ---
Spike
Vivek Lakshmanan
Apologies for the long radio silence. Thanks Deborah for the details
on details on the OSU-OSL move! My $0.02 on your proposal:
On Thu, Dec 16, 2010 at 1:02 PM, deborah shaddon
<deborah...@gmail.com> wrote:
...
>
>
> Highlevel Plan/Approach - CORE SERVICES FOUNDATION:
>
> 1. The Hardware/OS allocation: Planning for level of virtualization
> and capacity (the hardware approach):
> *Match of some capacity models against hardware we need to purchase
> and how we allocate this (with consideration for future scalability)
> *Decision on which linux variation we want (preferences? what do we
> have today?)
> *basic administration model we want at the OS level (we need to retain
> some, but look to delegate as much as possible to them, but retain
> some 'merit based' model of administration and control.
> *Will identify monitoring and backup strategies.
> *Leveraging some models of other OS projects in the OSL.
I agree with Chris that the need for purchasing hardware seems a bit
odd. The little snooping around on the OSU-OSL website did not yield
any information as to what their current capacity/utilization is. Do
you know what their status is on this front? Do they really
need/recommend us buying our own hardware? I think it would impede
their ability to administer and manage our infrastructure. Looks like
the plan is to use virtual machines but I suggest we host them on
OSL's own hardware.
I think that delegating monitoring/administrative tasks to OSL with
members of CCIWG providing oversight will help greatly.
It would be good to know how much of the support and administration
burden can be delegated to OSU-OSL. If it is determined that they are
most likely to take over day-to-day maintenance, then I suggest we let
them have input on the software environment within the virtual
machines. But CCIWG members should be aware of the choices made and be
able to step in when needed.
>
> 2. Some application 'services':
> *We spoke of application management and common-ids. As such, we want
> to leverage (most likely) an LDAP repository that our other components
> (via apache basic auth configuration and plug-in integration
> components for wordpress and mediawiki to sync ids). This will allow
> SSO, and single-id, and we can leverage this for future components
> *We can design this appropriately to allow 'seeding' (ie, conversion)
> of the IDs from our current systems.
> *Consideration also for LDAP and OpenID integration as well (ie, your
> LDAP can be your openID).
> *crisiscommons.org emails hosted as well, and ALSO tied into the LDAP
> (?), but not everyone here would have an email (so policies as well).
> *Anybody have any thoughts or conderations for this?
>
I think going the OpenID route makes a lot of sense. It can allow you
to use the same identity across all the core services and provides SSO
support. I would suggest punting on crisiscommons.org hosting in
favour of google apps etc. One less service to have to manage IMO, but
perhaps there are good reasons to have hosted e-mail..
> 3. The 'Application Software' allocation - SETUP:
> *So, we would have a central LDAP above, our own email hosted services
> with integration, as initial platform.
> *We spoke of breaking up DB servers from App-Servers (a common
> practice, even of other OS projects in this lab).
> *Continue with Wordpress and MediaWiki environment to start. Plugins
> for LDAP, and Apache specific configurations for basic-auth and ldap.
> *Do we want to do SSL on some of the pages, namely the login? Need
> certificate management then too (and verisign, etc.).
>
> 4. CONVERSION and MIGRATION PLANNING:
> *We will have to plan on how to do the conversion properly without
> loosing content, and allowing for the integration of all this.
> *database or export level conversions? What about other content or
> settings (non-db?)
> *We will want to setup a 'sandbox' of this environment above, to
> represent our staging environment, and discover how to get to stage
> from pre-prod to prod.
> *And integrate into SCM system (which they also provide some basic
> procedures and processes for locally).
>
I am not sufficiently aware of the current setup to comment on the
migration strategy. Is this documented somewhere? Or perhaps someone
with deeper insight into this can give us a refresher.
> 5. ACTUAL CUTOVER.
> *We should do just one system at a time instead of all-at-once (get
> our feet wet with their processes and policies).
> *How to 'cut off' content updates, etc.
>
Seems quite pragmatic to me.
In closing I think the OSL alliance makes a lot of sense. Thanks for
pushing forward on this!
- Vivek
Monika Adamczyk
Here are my comments. Keep in mind that I haven't read all the
proposal documents so if certain decisions were made already or a well
defined road map exists, I am not be aware of the information.
On Dec 16, 1:02 pm, deborah shaddon <deborahshad...@gmail.com> wrote:
> Looking for group feedback and thoughts on the proposed move approach
> for OSL:
>
> As you probably read in the CrisisCommons announcement, we will be
> partnering with Oregon State Universities OSL for hosting of some of
> our key CrisisCommons capabilities (outlined in the sloan proposal as
> well as listed here). This is expected to be a long term engagement,
could utilize for CrisisCommons infrastructure and whether the lab
already uses VM (chances are, the are). If that already exists, my
next guess would be, they have already OS installations available.
Regardless which OS we pick (I would certainly vote for Linux over
Windows, a least for servers installation), we definitively would like
to have root access to the OS instances on which CrisisCommons
software would be running.
This way we would leave low level system management up to OSL and high
level admin up to CrisisCommons. Think about OSL being like our own
IaaS cloud. If we went to a commercial provider (e.g. Amazon WS), that
is exactly what we would get at IaaS level.
If on the other hand, the only thing that OSL provides is physical
space and racks, we may need to look into hardware / software
solutions for our own VM environment but if that was the case, I
would rather look to an external infrastructure provider. I mentioned
yesterday directly to Deborah that using VMware is one of the options
(have no direct interest, just happen to know some people from VMware
who I maybe to convinced to donate some of their products) but on a
second thought, we are too small of organization to have our own data
center even if we are part of the OSL facility.
> 2. Some application 'services':
> *We spoke of application management and common-ids. As such, we want
> to leverage (most likely) an LDAP repository that our other components
> (via apache basic auth configuration and plug-in integration
> components for wordpress and mediawiki to sync ids). This will allow
> SSO, and single-id, and we can leverage this for future components
> *We can design this appropriately to allow 'seeding' (ie, conversion)
> of the IDs from our current systems.
> *Consideration also for LDAP and OpenID integration as well (ie, your
> LDAP can be your openID).
> *crisiscommons.org emails hosted as well, and ALSO tied into the LDAP
> (?), but not everyone here would have an email (so policies as well).
> *Anybody have any thoughts or conderations for this?
storming session on what services/functionality we want and need both
to maintain internal CrisisCommons operations and provide services to
larger community. Once what know what we want, selection of technology
(including SSO) will be much easier.
>
> 3. The 'Application Software' allocation - SETUP:
> *So, we would have a central LDAP above, our own email hosted services
> with integration, as initial platform.
> *We spoke of breaking up DB servers from App-Servers (a common
> practice, even of other OS projects in this lab).
> *Continue with Wordpress and MediaWiki environment to start. Plugins
> for LDAP, and Apache specific configurations for basic-auth and ldap.
> *Do we want to do SSL on some of the pages, namely the login? Need
> certificate management then too (and verisign, etc.).
functional design. What we need and why. Just because almost everybody
does things certain way is not always the best argument in itself. I
am not saying that the suggestions made by Deborah are wrong, but if
we choose to make any changes, than let first make sure we know why we
want to make them and if the investment in them will pay off.
>
> 4. CONVERSION and MIGRATION PLANNING:
> *We will have to plan on how to do the conversion properly without
> loosing content, and allowing for the integration of all this.
> *database or export level conversions? What about other content or
> settings (non-db?)
> *We will want to setup a 'sandbox' of this environment above, to
> represent our staging environment, and discover how to get to stage
> from pre-prod to prod.
> *And integrate into SCM system (which they also provide some basic
> procedures and processes for locally).
>
do in real life so yes, we need careful planning to make sure the
switch from one to another system is worth the pain and effort. Having
a staging environment is helpful because it allows for testing before
going into production but that is usually more common in software
development environment.
I am not 100% certain what you mean by SCM (software control or
configuration management?), but if my assumption is correct, than that
would apply to customer code development. Before we get into it, we
should see if we can get by with existing software (be it open source
or free COTS) so we can minimize custom implementations. In this case
I would rather look for something similar to CSM (certified software
management) system so can have control over third party software
upgrades. Of course if we have needs for custom application
development, than I will be the first one to advocate not only SCM but
the entire continuous integration environment and test driven
methodologies. :-)
> 5. ACTUAL CUTOVER.
> *We should do just one system at a time instead of all-at-once (get
> our feet wet with their processes and policies).
> *How to 'cut off' content updates, etc.
>
up one big system at once. Again, proper planning and design is
required to figure out proper architecture for our system and release
plan.
> 6.ETC
To sum things up, my recommendation is to establish a small group of
people with the proper technical/design/planning background that can
focus on the functional requirements and design before we dive into
building the system. 0Given my background, I can volunteer in this
area but I wouldn't want to be the only person working on it since I
am neither familiar with the existing infrastructure nor I would feel
comfortable making all decisions myself.
Monika Adamczyk
-- MACO TECH INC
-- http://www.maco-tech.com
-- mon...@maco-tech.com
-- +1 781-816-9817
-- Twitter @macotech
George Heake
Since I come from the accessibility side of technology I would advocate
including platforms such as HiSoftware's enterprise level system
Compliance Sheriff that monitors websites by scanning their pages for
compliance either to section 508 compliance or to WCAG compliance. It
also has the ability to scan pdf's for accessibility and alert the admin
there are "violations" to they can be fixed etc... There are also tools
for the developers to verify the content etc.. I am in the process of
seeing if they will sponsor us or come up with an affordable pricing
structure we can live with...There are also hardware server appliances
that could facilitate captioning of video etc..
Food for thought.
George
George Heake
Executive Director
No Borders Communication Inc (501c3)
NOBOCOMM
210-870-8618 (Cell)
210-210-491-6901 (direct)
210-590-7487 (office)
210-590-7203 (fax)
george...@nobocomm.org
http://www.nobocomm.org/
Follow us on Twitter
http://twitter.com/nobo_comm
deborah shaddon
consideration and I'll try to answer the questions, I don't have all
the answers either yet. I'll also try to setup a conference call over
next week (I'll work around the Holiday's) to discuss this and as some
key details emerge...It's really not going to be that hard to get this
figured out if we don't try to pork-barrel it too much or change
direction on ourselves at the moment. And under the Wilson center, we
will need to demonstrate sustainability of some capabilities before
going too far with anything new. And we'll ensure that we have a
workplan to track our tasks and steps that we want to do before
'starting' anything.
Remember, we are just talking about moving our core services over to
the OSL (to start), without changing them too much (SSO for example,
we've said was a foundational requirement for awhile and allows us to
achieve some future state capabilities too, is incremental). Not
changing direction on any technology at the moment, just moving and
continuing to learn and observe, and some of the 'process' changes to
making future technology decisions will have to come over time. And
beginning to establish our long-term partnership here too.
Sahana is also in the process of moving to OSL. In working with Mark
from Sahana, here are some reasons they provide for why OSUOSL over
just paying a hosting company, and you'll see that this is similar to
us and our goals. This is also about 'partnership'. They are moving
a) their website (drupal, we are wordpress) b) their wiki (they are
docuwiki, we are mediawiki) c) some of their development services (for
us this is still to be defined). Sahana is a good model for us, and
they have been at this for 5 years, so we can leverage their lessons
as well:
1) they are open source, and have a greater degree of experience with
the environments we require
2) they are providing services we don't get from EC2 or Slicehost or
GoDaddy - maintenance, upgrades, backups - and that is hard to get
from any volunteer infrastructure support committee (we've been
through this pain for five years now - trust me - moving to OSUOSL is
a godsend and a huge relief and allows me to sleep better at night).
3) they are providing a professionally managed service with dedicated
support, a ticketing system, and again, people that I know from the
Open Source community who I can escalate problems to if I need to.
4) I hope folks at OSU will become partners in our projects through
providing support and services for our test environments, hosting our
developers tools, etc. It's a way of broadening our own community.
5) The co-location is important for our partnership building.
Evidence of that is your note and our conversation (CrisisCommons)....
we are both to be hosted by OSUOSL, so we have common interests in
FOSS... so we work together on other things too. That is a good
thing.
*************************
So, with that said, some additional CrisisCommons infrastructure
points:
1. The hardware question is a tricky one. We still have to work this
out with them in terms of what can be provided for how much we pay for
it. We have bare-bones funding for infrastructure (and this has to
cover at a minimum domain names and other types of support services
for the core). So, hardware allocation to be figured out, but there
is no 'standard image', per se, this is based on our requirements or
preferences. There is currently no 'space' available to us or funding
that we could give them at the moment for this, so we have time to
'think it through'.....the feeling I get is this is sort of a sliding-
scale arrangement.
2. Why we wouldn't look to host externally is I think summed up here,
and part of a strategic governance and partnership decision already
made.
3. Access and control. We should be able to get access if we need to
to logs, OS, etc. But I think we want a model where we don't 'need'
to do that as much, and rely on them and their professional services
to the best extent possible. This has been our own experience this
year as well, and also highlighted by Sahana in Mark's point #2 above
(and why we need to 'learn' from others before pushing too far ahead
with volunteer infrastructure committees for this type of support).
And part of next year will be establishing our processes around this,
and having a merit-based system in place before people are given
certain types of access (this protects the community), similar to
apache....
4. Working on getting access to the latest infra-model to sloan from
Heather, it was previously available for public review and comments
over last month not sure where it is now. But as fyi, a lot of this
was gathered through many many working sessions and notes and
observations throughout the past year, some in this google group,
others on the wiki, so you can find some context here without that
document to start 'looking around'. In general, we want to be very
cautious, and careful at the moment to not completely open up and
change direction from an infrastructure working group level because of
the sloan foundation grant, a lot of previous work has been completed
and established, and we need to leverage that for now. In fact, now
trying to maintain course and stability through this transition period
will be key, and there will be finite set of resources (both people
and money and hardware), so criteria and process will be key. This is
why simply 'moving' our current sites and doing some work at the ID
management level is an initial step, without trying to take on too
much redesign (more consolidation design), allows us to observe and
define the process, and establish some key partnerships. There are a
lot of requirements for 'services' needed which the Tech and other
leads will drive (starting mid-year 2011-2012), which will get to the
'requirements' for future infrastructure, not necessarily building it
in that timeframe either. One of the key points of the sloan proposal
is of infrastructure not existing for the sake of itself....but in
support of the other crisiscommons capabilities (why we had a
capability model), so those need time to grow and get established as
well.
5. VM is something they can provide for us, and have, it is based on
our requirements not theirs. Check out Apache's infrastructure page,
you can see they use it (http://apache.org/dev/machines.html).
Virtualization is most commonly used in instances where we need to
provision more widely across a finite set of resources. Achieving
funding from sloan is still very very bare bones for infrastructure,
and we need to provide some allocation to them for their work, and
clear criteria when to 'add more', but this gives us some leverage to
have 'more unique instances of stuff, for instance, if we decide to
spin up an ethernet, we don't need to acquire new hardware to do so
(http://en.wikipedia.org/wiki/Virtualization). Given the proposed
future use of this for things like 'on-demand sites for disaster
apps', and 'spin up some development', and given we don't know what
our requirements will be (except that they will be finite), we need a
way to provision more dynamically within these finite resources.
However, we won't be in a position for any of this for 6-12 months, or
more, this is just an initial way to take some finite resources and be
able to chop the up for more specific project purposes and isolation,
rather than trying to stack too much on individual machines and deal
with complexities of dependencies. A common practice.
6. The don't have standard 'OS', this is depicted by our
requirements. The host many linux (and other) variety instances.
Linux is what our current environment runs on, we are not changing
that environment at this time. That is where that requirement came
from.
7. We've had many discussions internally about criteria and selection
of technologies, and that is why we are not changing course now on
this at the moment (and a task is to formalize this based on
observations and lessons). The roadmap outlines key services and
integration roadmap over next couple years, so we are sticking with
wordpress and wiki and a few supporting services for now. 2011 will
figure out some of the future state. The biggest thing in the roadmap
was not to rip out the foundation, but work on focusing on user
experience (a common dashboard, ie portal, and sso experience that
brought different disparate tools together), so things like the ldap/
openID are first steps in that.
8. I think accessibility for CC work is key, but some of those would
be future capabilities in the roadmap. Would like to look initial for
low-impact (and no cost or infrastructure) solutions if we can now.
There is no reason why our current wiki and wordpress site couldn't be
subjected to non-invasive accessiblility-testing evalaution. This
would be good input to the design-and-branding firm that will help
with the public facing image and web-site redesign of the commons for
next year.
Thanks!!
Deborah
On Dec 17, 2:37 pm, "George Heake" <george.he...@nobocomm.org> wrote:
> Hi Everyone,
> Since I come from the accessibility side of technology I would advocate
> including platforms such as HiSoftware's enterprise level system
> Compliance Sheriff that monitors websites by scanning their pages for
> compliance either to section 508 compliance or to WCAG compliance. It
> also has the ability to scan pdf's for accessibility and alert the admin
> there are "violations" to they can be fixed etc... There are also tools
> for the developers to verify the content etc.. I am in the process of
> seeing if they will sponsor us or come up with an affordable pricing
> structure we can live with...There are also hardware server appliances
> that could facilitate captioning of video etc..
> Food for thought.
> George
>
> George Heake
> Executive Director
> No Borders Communication Inc (501c3)
> NOBOCOMM
> 210-870-8618 (Cell)
> 210-210-491-6901 (direct)
> 210-590-7487 (office)
> 210-590-7203 (fax)
> Follow us on Twitterhttp://twitter.com/nobo_comm
>
>
> -----Original Message-----
> From: crisiscom...@googlegroups.com
>
> [mailto:crisiscom...@googlegroups.com] On Behalf Of deborah
> shaddon
> Sent: Thursday, December 16, 2010 12:03 PM
> To: CrisisCommons Infrastructure Working Group
> Subject: CCIWG: 52 OSU OpenSource Labs (OSL) Planning Work - CORE
> SERVICES FOUNDATION
>
> Looking for group feedback and thoughts on the proposed move approach
> for OSL:
>
> As you probably read in the CrisisCommons announcement, we will be
> partnering with Oregon State Universities OSL for hosting of some of
> our key CrisisCommons capabilities (outlined in the sloan proposal as
> well as listed here). This is expected to be a long term engagement,
>
> - Show quoted text -
Andrew Turner
a cruise for 3 days with no 'net. I'd love to setup a time the week
after Christmas to talk in depth about the Infra and even handle some
migration of services since it will be a very slow work week.
Quick notes in planning:
- cc.org and the wiki run on a single Linux server. Standard LAMP
stack - or even postgresql is often my preference :) I turned on
caching so processor usage is low but memory 'high' (1gb ram across my
VPS). Depends if we're doing shared host, vps or dedicated box.
Obviously is great/necessary to have shell/root access.
- ubuntu is probably a good distro choice. Though we use centos for
our geoiq servers.
- we can pick up other systems as necessary when we identify social
network/crm platform. Likely soemthing like Redmine will be great for
project issue tracking et al. That's Ruby on Rails. Not difficult to
setup, just a little more than standard LAMP.
Have a great Holiday!
Andrew
(via mobile)
On Dec 17, 2010, at 5:08 PM, deborah shaddon
Andrew Turner
migration.
We are using Google Apps (free for up to 50) and would like to stay on
that for longer. It gives a lot of additional capability/reliability
for no cost. It also provides OpenID support so we could simply enable
WP and MediaWiki openid via plugins and people could use their cc.org
accounts or third party openid providers. So we could push back any
development of LDAP (fun!)
Virtualization is great. It's straightforward to imagine Camps and
projects building on local VMs and then we can deploy those to our
infrastructure after testing.
Amazon is beginning to support VM upload to AMI a well, so it gives us
and projects flexibility. Projects could migrate out of 'incubation'
and deploy to their own hosting very easily. So definitely good
direction in planning that process.
One piece of technology we don't have now very well (minor built into
the VPS I'm hosting on now), and I would like to know what options OSU
can provide are server monitoring such as Monit and Munin for
understanding server health, load, and characteristics. This would
probably be the only new piece we would want to bring in right away.
Andrew
(via mobile)
On Dec 19, 2010, at 10:35 AM, Andrew Turner <and...@crisiscommons.org>
wrote:
Monika Adamczyk
<and...@crisiscommons.org> wrote:
> Some more quick notes on our current infastructure and thoughts of
> migration.
>
> We are using Google Apps (free for up to 50) and would like to stay on that
> for longer. It gives a lot of additional capability/reliability for no cost.
> It also provides OpenID support so we could simply enable WP and MediaWiki
> openid via plugins and people could use their cc.org accounts or third party
> openid providers. So we could push back any development of LDAP (fun!)
>
As paying user of Google Apps, I second the idea to get maximum
benefit out of everything that Google has to offer. One thing I would
suggest (if that is not done already), to convert crisiscommons.org
account into full service account. That means that in addition to the
primary services such as email, calendar,docs,website, the domain gets
access to all services that are available to free account.
> Virtualization is great. It's straightforward to imagine Camps and projects
> building on local VMs and then we can deploy those to our infrastructure
> after testing.
>
It just occurred to me that VMware offers software for windows that
allows to create a VM from a physical drive for easy import into their
virtual environment. I will try to find out if equivalent product
exists for Linux. If it does, moving existing infrastructure to OSL
should be relatively easy.
> Amazon is beginning to support VM upload to AMI a well, so it gives us and
> projects flexibility. Projects could migrate out of 'incubation' and deploy
> to their own hosting very easily. So definitely good direction in planning
> that process.
>
> One piece of technology we don't have now very well (minor built into the
> VPS I'm hosting on now), and I would like to know what options OSU can
> provide are server monitoring such as Monit and Munin for understanding
> server health, load, and characteristics. This would probably be the only
> new piece we would want to bring in right away.
>
I took a look at the list of the hosting services OSL offers and they
mention Nagios and Cacti SNMP monitoring:
http://osuosl.org/services/hosting/details
Jeff Sheltren
believe it or not, I was not online or within cell phone range! Yes,
it was as wonderful as it sounds :)
I'm Jeff Sheltren, Operations Manager at the OSU Open Source Lab. I
haven't had much time to spend with Deborah and others to discuss
details about the infrastructure, so we've really just gone over some
general information and options at this point. I've joined this
google group now and Lance Albertson, our lead system administrator
has as well so that we can help answer some questions and chime in
where needed. Deborah has done a good job summarizing the questions,
but I'll addressa couple of the points that have been brought up
inline below.
It's worth noting that the OSUOSL is not your typical ISP or hosting
company. I'll take a minute to describe the OSUOSL in hopes that it
will help everyone understand who we are and what we do. We're part
of Oregon State University (not a commercial entity) and our mission
can be simplified to something along the lines of "help open source
projects succeed". The majority of our funding comes through
donations from industry and individuals. We try to be as flexible as
possible when providing infrastructure and system administration
services so that we can meet the needs of many different projects and
communities. However, there are many applications and technologies
that we are more experienced and comfortable with than others, so we
will tend to suggest those if and when it makes sense. That doesn't
mean that we will restrict projects to only certain applications/
technologies, simply that we may not be able to support & scale
everything to the same extent that we can do with those that we use
often. I'll keep it at that for now, but feel free to check out our
website - http://osuosl.org - or contact me directly if you have more
questions which you don't feel are entirely relevant to this list.
-Jeff
On Dec 17, 2:08 pm, deborah shaddon <deborahshad...@gmail.com> wrote:
> 1. The hardware question is a tricky one. We still have to work this
> out with them in terms of what can be provided for how much we pay for
> it. We have bare-bones funding for infrastructure (and this has to
> cover at a minimum domain names and other types of support services
> for the core). So, hardware allocation to be figured out, but there
> is no 'standard image', per se, this is based on our requirements or
> preferences. There is currently no 'space' available to us or funding
> that we could give them at the moment for this, so we have time to
> 'think it through'.....the feeling I get is this is sort of a sliding-
> scale arrangement.
>
one or two virtual machines for CC. In my talk with Deborah I got the
impression that while this may be enough for the very short-term, this
would not be sufficient to host all of the services you are looking to
host moving forward. This is why I suggest that Crisis Commons
provide some hardware and we will be able to create a dedicated
virtualization cluster just for CC use where OSL could (if desired)
provide low-level system administration and maintenance and allow CC
to manage administration of certain apps/services. If CC doesn't have
resources to purchase hardware, we may have some hardware we could
allocate at least temporarily -- remember, we are a non-profit and
most of our hardware has been donated to us, we don't have money to
provide every project with dedicated hardware. One other thing to
note is that we do provide a large number of centralized (shared)
services which means that you won't need to run your own instance of
everything on a virtual machine. For example, we provide centralized
mysql/postgresql servers, mailman, email relays, DNS, etc. The MySQL
servers, for example, are running on very high-end machines which we
received as a donation and are able to provide database services much
better than a VM could.
>
> 6. The don't have standard 'OS', this is depicted by our
> requirements. The host many linux (and other) variety instances.
> Linux is what our current environment runs on, we are not changing
> that environment at this time. That is where that requirement came
> from.
>
needed (related to OS, tools, scaling for large sites, whatever...),
but we're not going to force our views/tools on you.
Chris Foote (Spike)
Hi Jeff!
I'm Spike from London and I'm one of the current admins for the some of
the web-facing services. Thanks for your introduction and welcome to you
and to Lance to this Google Group.
As you can imagine most of us are still recovering from the excitement
of getting the grant the other day not to mention Christmas and the New
Year - normal service will (probably) be resumed in the next few days!
Best regards
Spike