IU Server Move - Need response

[Michael Caisse]

Hi all -

---

TL;DR - All of the Boost servers are moving. Are there bad dates for you? Are you concerned about possible monthly cost?

---

== Why ==

In case you missed it previously, IU's sponsor of the open source resources is moving to Digipen. We thank IU for their very generous support over the years. As such, projects formerly hosted at IU are moving to new homes.

Ciere's IT department has been working with DongInn at IU to complete the migration. Servers and functionality was migrated in November. Tests have been performed to verify that the new system is operating properly. The new server resources are syncing with IU's data on a regular basis.

== When ==

That is one of the main questions of this email. We can move the websites immediately. They have been migrated and tested and are sync'ing regularly. The mail lists need some administrative (credit card) love but have also been migrated, tested, and have been sync'ing regularly for some time.

We prefer not to make the move during the middle of a release. Are there sets of days that we should black-out?

== What is moving ==

IU hosts mail lists and web sites. The following resources have been migrated:

Mail Lists :

wowbagger is the IU instance that hosted mail lists. It has a 200GB storage volume attached to maintain the strorage of the mailing list archives and other functions. Specific lists that are part of wowbagger and have been migrated to a cloud instance by the same name are:

• Boost - Boost developers' mailing list
• Boost-announce - Boost announce-only mailing list
• Boost-bugs - Bugs reported/updated through Boost's Trac
• Boost-build - Boost.Build developer's and user's list
• Boost-cmake - Discussion of the CMake-based build system for Boost
• Boost-commit - Boost Subversion commit messages
• Boost-docs - Discussion of Boost Documentation
• Boost-Interest - Moderated announcements of interest to the Boost community.
• Boost-maint - Boost community maintenance mailing list
• Boost-mpi - Discussion of Boost.MPI development
• Boost-test - Test list for the boost mailing lists
• Boost-Testing - Running Boost regression tests
• Boost-users - Boost Users mailing list
• Geometry - Boost.Geometry library mailing list
• glas - Generic Linear Algebra Software
• Noreply - [no description available]
• osl-mailman - [no description available]
• Osl-test2 - [no description available]
• proto - Discussions about Boost.Proto and DSEL design
• test - [no description available]
• test-list - [no description available]
• threads-devel - Discussions about the boost.thread library
• ublas - ublas mailing list

Web sites:

The following web sites have been migrated to a cloud instance.

• beta.boost.org
• boostcon.boost.org
• lists.boost.org
• svn.boost.org
• wiki.boost.org
• www.boostcon.org
• www.boost.org

== Where is it moving? ==

Rackspace has donated cloud resources to the Software Freedom Conservancy for their member projects. Boost receives $2000/month of donated cloud services which were mostly being unused. Ciere spends about $25k/month with Rackspace and our account and technical managers have been helpful in navigating the SFC donation.

In addition to the normal cloud services, we are utilizing MailGun (a Rackspace service similar to SendGrid and the ilk) for sending of mass emails. Through the SFC donation, we receive 50k free emails a month. More on this in the mail list cost details below.

== Web Server Costs ==

The web server cost should easily fall into the resources donated by Rackspace. The exact sizing for bandwidth is still a little "unknown" and we are refining our estimates; however, we currently expect the 

== Mail List Costs ==

The FSC relationship gets us 50k of free emails per month. A heavy month sees just under 2M emails. If the lists maintain that high volume, it would cost ~$670/month for mail list traffic. Based on a quick look at the logs, some months will be within the 50k emails.

Boost can likely support the monthly expense without doing any promotion. Optionally, we can solicit for corporate sponsorship which would include a one-line footer in the outgoing emails. I suspect Ciere might be willing to support this.

== Other ==

Ciere purchased a multi-year wildcard certificate for Boost sometime ago. There has been some complaint about our web site not being https ... we can change that over post migration.

== What I need to know ==

Are there black-out dates for the migration of the MLs or web sites or both?

We need formal authorization from the steering committee for the ~$700/month email expense to maintain the MLs.

Ciere will continue to donate resources to perform administration.

[Beman Dawes]

On Mon, Jan 2, 2017 at 2:56 PM, Michael Caisse <mca...@boost.org> wrote:

 

== What I need to know ==

Are there black-out dates for the migration of the MLs or web sites or both?

Traffic is light on the lists right not, so the sooner that gets done the better IMO.

OTOH, right after a release may not be the best time to migrate the web sites. Maybe wait a couple of weeks.

 

We need formal authorization from the steering committee for the ~$700/month email expense to maintain the MLs.

I'm wondering how many subscribers are zombies? I.E. the mail isn't bouncing, but there is no longer a human at the other end reading it. No sure how to deal with that concern - has anyone else run into this before and figured out how to deal with it in a way that doesn't scare off actual readers of the list?

Ciere will continue to donate resources to perform administration.

Thanks you so much!

--Beman

[Michael Caisse]

I was also interested in filtering out zombie users.

Some lists have a yearly "Press this link to verify you want to stay subscribed." The downfall is that email can get lost in high volume lists. We could implement something like that and send monthly emails if no response is received on the first yearly email. After 4-6 months without a response, we can then trim the email address.

I'm interested in hearing other advice!

--
You received this message because you are subscribed to the Google Groups "Boost Steering Committee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to boost-steering+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rene Rivera]

On Mon, Jan 2, 2017 at 1:56 PM, Michael Caisse <mca...@boost.org> wrote:

Ciere's IT department has been working with DongInn at IU to complete the migration. Servers and functionality was migrated in November. Tests have been performed to verify that the new system is operating properly. The new server resources are syncing with IU's data on a regular basis.

Are the user account moving also? If yes, are the log in credentials the same? If not, what is the alternative access to the file system?

Have you migrated and tested the test results system?

Have you migrated & tested that the test results reports web pages work the same?

Have you migrated & tested the test report uploads?

What is the disk space availability of the new web server location after all the mirroring?

 

== Web Server Costs ==

The web server cost should easily fall into the resources donated by Rackspace. The exact sizing for bandwidth is still a little "unknown" and we are refining our estimates; however, we currently expect the 

You forgot to say something there ;-) 

== What I need to know ==

Are there black-out dates for the migration of the MLs or web sites or both?

Not during: 2-4 weeks before the start of a release cycle through 2 weeks after a release to accommodate possible emergency patch release.

The next release is scheduled for 2017-04-12. Which means the release cycle starts on 2017-02-22. Which means that the switch could to do it before the next release would need to happen:

* Ideally from 2017-01-09 through 2017-01-24.

* Workable from 2017-01-09 through 2017-02-07.

Warning, there is at least one new library inbound which needs to happen before anything gets moved as it needs some time to iron out the integration. So I would shorten the front end of a move to after 1-15 (assuming I start on the integration Wednesday when I'm back in town).

And of course this all depends on the answers to my questions above. As the critical item for releases is the uptime of the testing system.

--

-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net
-- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

[Michael Caisse]

On Mon, Jan 2, 2017 at 11:01 PM, Rene Rivera <grafi...@gmail.com> wrote:

On Mon, Jan 2, 2017 at 1:56 PM, Michael Caisse <mca...@boost.org> wrote:

Ciere's IT department has been working with DongInn at IU to complete the migration. Servers and functionality was migrated in November. Tests have been performed to verify that the new system is operating properly. The new server resources are syncing with IU's data on a regular basis.

Are the user account moving also? If yes, are the log in credentials the same? If not, what is the alternative access to the file system?

• IU did not have any user accounts directly tied to the machine.  IU personnel logged in via ldap.  Any console access accounts would need to be added.
• Other access to items such as TRAC is handled via Apache’s .htaccess and .htpasswd files.  That migrates over and is unchanged.
• Any PSKs for ssh access would still be present today.

 

Have you migrated and tested the test results system?

Have you migrated & tested that the test results reports web pages work the same?

Have you migrated & tested the test report uploads?

The short answer is I still need to test this. I was hoping to have had enough time last weekend to report back but that didn't occur. I'll get back to you on this ... and probably ask a few questions on IRC.

 

What is the disk space availability of the new web server location after all the mirroring?

There is currently about 100GB available; however, it is more complicated than that of course. I think it is safe to say that the server(s) will be sized based on our needs and additional storage added when needed. If you have some concerns or targets at this time, let us know!

 

 

== Web Server Costs ==

The web server cost should easily fall into the resources donated by Rackspace. The exact sizing for bandwidth is still a little "unknown" and we are refining our estimates; however, we currently expect the 

You forgot to say something there ;-) 

Funny. I was trying to get a closer estimate and then didn't add it in. We are currently estimating that the cost will be about $300/month. This is covered by the Rackspace donation and there will be no out-of-pocket expense to Boost.

 

== What I need to know ==

Are there black-out dates for the migration of the MLs or web sites or both?

Not during: 2-4 weeks before the start of a release cycle through 2 weeks after a release to accommodate possible emergency patch release.

The next release is scheduled for 2017-04-12. Which means the release cycle starts on 2017-02-22. Which means that the switch could to do it before the next release would need to happen:

* Ideally from 2017-01-09 through 2017-01-24.

* Workable from 2017-01-09 through 2017-02-07.

Warning, there is at least one new library inbound which needs to happen before anything gets moved as it needs some time to iron out the integration. So I would shorten the front end of a move to after 1-15 (assuming I start on the integration Wednesday when I'm back in town).

Makes sense.

 

And of course this all depends on the answers to my questions above. As the critical item for releases is the uptime of the testing system.

Thank you Rene for taking time to respond. I'll coordinate more with you on IRC since it seems like you are the main person who understands the concerns related to moving many of the web services.

Take care -

michael

[Michael Caisse]

All -

DongInn at IU (who has been administrating the Boost systems for years) has notified us that we must migrate next week. There is some uncertainty with changes at IU and we need to complete the move while we are insured full access.

I will work with Rene on validating that test reporting does the "right thing". We will push as late as possible to help improve confidence; however, we might have to fix some things post migration.

We (Ciere) will clear our schedules for the transition day to address any problems that might arise.

[Marshall Clow]

On Feb 9, 2017, at 10:57 AM, Michael Caisse <mca...@boost.org> wrote:

All -

DongInn at IU (who has been administrating the Boost systems for years) has notified us that we must migrate next week. There is some uncertainty with changes at IU and we need to complete the move while we are insured full access.

I will work with Rene on validating that test reporting does the "right thing". We will push as late as possible to help improve confidence; however, we might have to fix some things post migration.

We (Ciere) will clear our schedules for the transition day to address any problems that might arise.

Thank you, Michael.

It’s a shame that IU has handed this (very short) deadline.

— Marshall

--
You received this message because you are subscribed to the Google Groups "Boost Steering Committee" group.

To unsubscribe from this group and stop receiving emails from it, send an email to boost-steerin...@googlegroups.com.

[Michael Caisse]

To be honest, we have taken longer than IU wanted (I think). They have been very gracious along the way. We had a drop-dead date in March but some things changed this week and I think DongInn notified us right away.

I missed Rene's ideal window before the release because I still haven't verified full round trip with the testing reports. That is my fault.

Anyhow ... while unfortunate, I think it is very manageable.

To unsubscribe from this group and stop receiving emails from it, send an email to boost-steering+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Boost Steering Committee" group.

To unsubscribe from this group and stop receiving emails from it, send an email to boost-steering+unsubscribe@googlegroups.com.

[Rene Rivera]

On Thu, Feb 9, 2017 at 11:04 AM, Marshall Clow <mc...@boost.org> wrote:

On Feb 9, 2017, at 10:57 AM, Michael Caisse <mca...@boost.org> wrote:

All -

DongInn at IU (who has been administrating the Boost systems for years) has notified us that we must migrate next week. There is some uncertainty with changes at IU and we need to complete the move while we are insured full access.

I will work with Rene on validating that test reporting does the "right thing". We will push as late as possible to help improve confidence; however, we might have to fix some things post migration.

We (Ciere) will clear our schedules for the transition day to address any problems that might arise.

Thank you, Michael.

It’s a shame that IU has handed this (very short) deadline.

Indeed.. But it is what it is.. Michael, I'm busy through Friday. I'll be flying and arriving back home after midnight on Friday. I likely have time in the weekend to test stuff and help migrate testing then. During the week I only get small snippets of time during work hours. So if it's something prolonged it would have to be at night then.