sage infrastructure at UW

[William Stein]

Hi,

Yesterday a machine at UW (the one hosting *.sagenb.org) was attacking
other computers, so the UW shut it down.
Today trac, the wiki, and many other machines crashed.

I'm traveling, so can't look into the hardware anytime soon. Morever,
the situation is that there is nobody else who will either. I ran out
of money, and had to focus on other things, so what's left of the UW
sage-related compute infrastructure is basically dead now.

I've created two VM's on Google compute engine (running the same OS --
Ubuntu 12.04 -- as trac and wiki) and pointed

http://trac.sagemath.org/ and http://wiki.sagemath.org/

at them with a message that both sites are down.

Thierry said he has a recent backup of the wiki data on the VM hosting
wiki.sagemath.org, but will wait a day or two before providing it.

I have a backup of trac.sagemath.org from May 26, which might be
valid. However, I think the only person who understands the trac
setup is Andrew Ohana, and he hasn't been involved in a very long
time...

The options I can think of right now:

1. Wait an indefinite amount of time to see if somehow magically the
machines come back.

2. Move all Sage development to github: https://github.com/sagemath/sage
I just re-enabled the github issues page, etc.
There is also a wiki here https://github.com/sagemath/sage/wiki
At some point migrate the data on wiki.sagemath.org to github (maybe
remove all images if that is a problem).
Eventually, bring trac.sagemath.org back in read only mode and copy
over all open tickets to github issues,
via some python script that goes through the trac database and creates
github tickets.

3. Configure trac.sagemath.org and wiki.sagemath.org so they work
again using the most recent backups.

I am personally *not* going to do anything further except give people
ssh access to the above machines and/or point DNS somewhere.

Ultimately, Volker is the release manager and whatever he wants to do
to continue making releases will hopefully happen...

-- William

[Ralf Stephan]

On Monday, June 6, 2016 at 5:10:16 AM UTC+2, William wrote:

There is also a wiki here https://github.com/sagemath/sage/wiki
At some point migrate the data on wiki.sagemath.org to github (maybe
remove all images if that is a problem).

Images are no problem (put them in the same repo). LaTeX is more (but just display it verbatim).

 

Eventually, bring trac.sagemath.org back in read only mode and copy
over all open tickets to github issues,

Solutions at

http://stackoverflow.com/questions/6671584/how-to-export-trac-to-github-issues

However, the participants of the tickets get notified according to their github settings, i.e. mail storm.

How to work around this?

[Erik Bray]

On Mon, Jun 6, 2016 at 5:09 AM, William Stein <wst...@gmail.com> wrote:
> Hi,
>
> Yesterday a machine at UW (the one hosting *.sagenb.org) was attacking
> other computers, so the UW shut it down.
> Today trac, the wiki, and many other machines crashed.
>
> I'm traveling, so can't look into the hardware anytime soon. Morever,
> the situation is that there is nobody else who will either. I ran out
> of money, and had to focus on other things, so what's left of the UW
> sage-related compute infrastructure is basically dead now.
>
> I've created two VM's on Google compute engine (running the same OS --
> Ubuntu 12.04 -- as trac and wiki) and pointed
>
> http://trac.sagemath.org/ and http://wiki.sagemath.org/
>
> at them with a message that both sites are down.
>
> Thierry said he has a recent backup of the wiki data on the VM hosting
> wiki.sagemath.org, but will wait a day or two before providing it.
>
> I have a backup of trac.sagemath.org from May 26, which might be
> valid. However, I think the only person who understands the trac
> setup is Andrew Ohana, and he hasn't been involved in a very long
> time...
>
> The options I can think of right now:
>
> 1. Wait an indefinite amount of time to see if somehow magically the
> machines come back.

Probably not.

> 2. Move all Sage development to github: https://github.com/sagemath/sage
> I just re-enabled the github issues page, etc.
> There is also a wiki here https://github.com/sagemath/sage/wiki
> At some point migrate the data on wiki.sagemath.org to github (maybe
> remove all images if that is a problem).
> Eventually, bring trac.sagemath.org back in read only mode and copy
> over all open tickets to github issues,
> via some python script that goes through the trac database and creates
> github tickets.

-1

I think one of the best things about Sage's development is that it's
using Trac. GitHub's issue tracker is awful by comparison, especially
for the volume of issues Sage generates.

> 3. Configure trac.sagemath.org and wiki.sagemath.org so they work
> again using the most recent backups.

If anyone needs help with the Trac configuration I used to be a
developer of Trac and know it rather well. As long as somehow has a
backup of the database and the config file it should be easy enough to
get up and running again.

Best,
Erik

[Dima Pasechnik]

On Monday, June 6, 2016 at 7:03:42 AM UTC+1, Ralf Stephan wrote:

On Monday, June 6, 2016 at 5:10:16 AM UTC+2, William wrote:

There is also a wiki here https://github.com/sagemath/sage/wiki
At some point migrate the data on wiki.sagemath.org to github (maybe
remove all images if that is a problem).

Images are no problem (put them in the same repo). LaTeX is more (but just display it verbatim).

 

Eventually, bring trac.sagemath.org back in read only mode and copy
over all open tickets to github issues,

by the way, there is https://groups.google.com/forum/#!forum/sage-trac

with all the trac ticket messages (last one dating from some 17 hours ago (from my 9:30am BST (London) time))

Perhaps more urgent would be to get a good recent snapshot of g...@trac.sagemath.org repo. 

Do buildbots/patchbots pull the whole tree?

Solutions at

http://stackoverflow.com/questions/6671584/how-to-export-trac-to-github-issues

However, the participants of the tickets get notified according to their github settings, i.e. mail storm.

How to work around this?

well, it's a per project setting, so that's not such a big deal, as it's possible to get notifications on a particular issue;  after you started participating, you'd get email notifications 

One development model would be to review forks of the main sage tree not as pull requests, but in the individual repos, and only after a positive review create a pull request.

But I agree with Erik that trac is much more convenient, and perhaps we should not pursue github workflows now.

Erik, would you be happy to bring trac.sagemath.org back online on a William's GCE host?

I cc this to William, hopefully he can give you all the creds and backups he has.

I have very little time this week due to teaching (last teaching week of the term) and being in single-parent mode till Sat...

-----------------------------------

Somehow I was under impression that trac backups are made on a regular basis, not just ad hoc. 

Oh well, this is embarrassing.

The most pressing need to get this back up and running concerns running GSoC projects.

Best,

Dima

 

[Jeroen Demeyer]

On 2016-06-06 10:33, Erik Bray wrote:
> I think one of the best things about Sage's development is that it's
> using Trac. GitHub's issue tracker is awful by comparison, especially
> for the volume of issues Sage generates.

I agree.

>> 3. Configure trac.sagemath.org and wiki.sagemath.org so they work
>> again using the most recent backups.
>
> If anyone needs help with the Trac configuration I used to be a
> developer of Trac and know it rather well. As long as somehow has a
> backup of the database and the config file it should be easy enough to
> get up and running again.

Totally +1, let's do this! Hopefully, OpenDreamKit can help with this.

[mmarco]

Would it be technically possible to host all these services in different physical locations (syncronized, of course), and use some kind of load balancer to handle the falldown of one of them?

[William Stein]

I've sent invites to the Google compute engine project to the following people.

- Andrew O
- Dima P
- Volker B
- Erik B
- Samuel L

> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.



--
William (http://wstein.org)

[William Stein]

and Ralf Stephan.

NOTE: the backups are in /orig on each machine.

--
William (http://wstein.org)

[Thierry]

Hi,

i am currently working on the wiki (my backup is <2days so there is almost
no lose, and it will be easy to merge the very few changes once we can
access the current sate).

For the git/trac, i am not sure that it is a good idea to put a 2-weeks
backup read/write, because it will be awful to merge with the current
state. It is imho much better to have some downtime since people can still
work on their local branch.

Ciao,
Thierry

[Jeroen Demeyer]

On 2016-06-06 13:09, Thierry wrote:
> For the git/trac, i am not sure that it is a good idea to put a 2-weeks
> backup read/write, because it will be awful to merge with the current
> state. It is imho much better to have some downtime since people can still
> work on their local branch.

Sure, but the question is: is the data still available somewhere?

[Ralf Stephan]

On Monday, June 6, 2016 at 2:48:47 PM UTC+2, William wrote:

I've sent invites to the Google compute engine project to the following people.

How do people coordinate usually in this setting?  Mail cc? chat? group?

[Thierry]

Is there a way to contact UW sysadmins ? The VM are located somewhere and
the disks should be at least physically reachable.

Ciao,
Thierry

[Frédéric Chapoton]

http://patchbot.sagemath.org/ is now also affected, even if it was working at some point after trac's failure.

Frédéric

[Dima Pasechnik]

The conversations and ticket descriptions are available in form of messages to sage-trac@googlegroups, for sure. 

I already asked whether the trac git tree is available on some patch/build-bots.

Or perhaps someone pulled a full private copy recently?

Any idea about this?

 

[Frédéric Chapoton]

I do not think that the patchbots clients have a copy of the full git repo. They only pull branches. But they have the branch for 7.3.beta3, at least.

Frederic

[Dima Pasechnik]

Also flint/mpir/nemo sites are down (flint people know this), due to being on the same cluster of machines...

[Bill Hart]

It's a devastating blow. But we will rebuild.

https://imgflip.com/i/15dnsh

[Dima Pasechnik]

On Monday, June 6, 2016 at 3:30:55 PM UTC+1, Frédéric Chapoton wrote:

I do not think that the patchbots clients have a copy of the full git repo. They only pull branches. But they have the branch for 7.3.beta3, at least.

this one is just here:  https://github.com/sagemath/sage/tree/develop

[William Stein]

Hello,

The sysadmin just explained what happened -- namely the room has been
cooking all weekend due to an AC failure: This means that everything
might -- or might not -- come back in the next few days. There is no
way to know the extent of the damage, if any.

---
"The PDL C28 server room AC unit died some point this weekend. The
ambient temperature in the room was about 100 degrees F this morning.

I've physically unplugged power to all the Math and Sage machines for
the time being until the HVAC repair is done.

Hopefully there isn't too much hardware damage from the heat.

Steve Sheetz
Manager, Dept Computing

University of Washington
Dept of Mathematics
Padelford Hall, C-32
Box 354350"
---

[Ralf Stephan]

On Monday, June 6, 2016 at 4:22:02 PM UTC+2, Dima Pasechnik wrote:

Or perhaps someone pulled a full private copy recently?

Around end of April, I think, someone better?

ralf@ark:~/sage/.git> find logs/refs/remotes/trac/| wc
  7231    7231  368968

[Dima Pasechnik]

On Monday, June 6, 2016 at 3:57:41 PM UTC+1, William wrote:

Hello,

The sysadmin just explained what happened -- namely the room has been
cooking all weekend due to an AC failure:  This means that everything
might -- or might not -- come back in the next few days.  There is no
way to know the extent of the damage, if any.

OK, +37 degrees Celsius does not sound too bad.

Could you perhaps ask when they did the shutdown?

We know, roughly, when the last trac comment went through.

Perhaps some nodes shut themselves down, due to overheating?

[Volker Braun]

IMHO thats not a big deal, some orgs are experimenting with permanently running datacenters at 90+F...

The best course of action is probably to wait and get the machines back up. Then slowly move the infrastructure out of UW.

[Thierry]

Hi,

the wiki is back online at University Paris North :
http://sagewiki.lipn.univ-paris13.fr/

It should work with wiki.sagemath.org domain name so please do not
hesitate to switch the DNS to 194.254.163.84. I will setup SSL once the
domain name is correct (not before tomorrow).

Once the data will be recovered, i will update the few edits that happened
yesterday.

I moved from apache to nginx+uwsgi and set up a similar firewall than
ask.sagemath.org so that the other admins (Niles, Vincent, Samuel,
KarlDieter,...) can give a hand. Some of them are already OK to
participate to the admin of the wiki already, so this will go smoothly.
Also, we are on the way to set up a dedicated backup service at Universite
Paris South.

I agree with Volker: regarding trac, it is better to wait for the data to
be back, otherwise it will be hard to merge trac comments and all the
stuff.

I have to leave my keyboard now, i will tell more about the details later.

Ciao,
Thierry

[Samuel Lelievre]

Please someone switch the DNS to point to the resurrected sage wiki.

(cc-ing Harald in case he can do it).

Thanks Thierry!

Samuel

[Eric Gourgoulhon]

Le lundi 6 juin 2016 19:08:44 UTC+2, Thierry (sage-googlesucks@xxx) a écrit :

Hi,

the wiki is back online at University Paris North :
http://sagewiki.lipn.univ-paris13.fr/

Thank you very much Thierry!
I've checked it on recently modified pages: the Sage Days 74 ones.
Everything is fine; in particular one can recover the attached pdf and ipynb files.

Best regards,

Eric. 

[William Stein]

On Mon, Jun 6, 2016 at 1:21 PM, Eric Gourgoulhon <egourg...@gmail.com> wrote:
>
> Le lundi 6 juin 2016 19:08:44 UTC+2, Thierry (sage-googlesucks@xxx) a écrit
> :
>>
>> Hi,
>>
>> the wiki is back online at University Paris North :
>> http://sagewiki.lipn.univ-paris13.fr/
>>
>
> Thank you very much Thierry!

Thank you!

I've put a link from the static page at http://wiki.sagemath.org/ to
http://sagewiki.lipn.univ-paris13.fr/.

I am flat out strongly against hosting the Sage wiki, trac, etc.,.
They should be properly hosted in the cloud, where anybody can be
added as admins, we can make global backups, move the machines to have
a presence anywhere in the world (due to many data centers all over),
etc.

I feel like I've been way too hesitant regarding expressing my
opinions about the Sage infrastructure for the last year.

-- William

> I've checked it on recently modified pages: the Sage Days 74 ones.
> Everything is fine; in particular one can recover the attached pdf and ipynb
> files.
>
> Best regards,
>
> Eric.
>

[mmarco]

 I will setup SSL once the
domain name is correct (not before tomorrow).

Will you need the old certificate or plan to generate a new one?  

[Clement Pernet]

[pernet@nisqually] :~/Logiciels/sage/.git > find logs/refs/remotes/trac/|wc
7382 7382 376919

This was updated on May 30.

--
Clément

[Thierry]

Hi,

On Mon, Jun 06, 2016 at 02:35:22PM -0400, William Stein wrote:
[...]

> I am flat out strongly against hosting the Sage wiki, trac, etc.,.
> They should be properly hosted in the cloud, where anybody can be
> added as admins,

FUD

> we can make global backups,

FUD

> move the machines to have a presence anywhere in the world (due to many
> data centers all over),

FUD

> etc.

FUD

> I feel like I've been way too hesitant regarding expressing my
> opinions about the Sage infrastructure for the last year.
>
> -- William

Following the "don't feed the troll" principle, i did not answer all your
(and Dima's) previous attacks against myself and the people who currently
work on the Sage infrastructure, but this is going too far. Some examples
include:

- spreading wrong and racist statements about french universities and how
they manage access to VMs [1]
- suggesting that nobody was working on the migration while you know that
we are working to set things up correctly (e.g. remote backup system),
and considering that workers are at your disposal, this is extremly
patronizing [2]
- maintaining the confusion between university hosting and the very bad
practices that led us to this situation [3] [4]

If you and Dima deign to apologize and consider the people who work for
Sage with some respect, perhaps might we start a constructive discussion
on the benefits of having a collectively-managed decentralized
University-hosted infrastructure for the Sage services.

Ciao,
Thierry

[1] http://article.gmane.org/gmane.comp.mathematics.sage.devel/84915
[2] http://article.gmane.org/gmane.comp.mathematics.sage.devel/86960
[3] http://article.gmane.org/gmane.comp.mathematics.sage.devel/86954
[4] http://article.gmane.org/gmane.comp.mathematics.sage.devel/84894

[William Stein]

Hi Thierry,

I don't really know who you are, and I've never met you. But given
your claims that I'm "spreading wrong and racist statements about
french universities"... etc., I guess I do now. Many thanks for the
clarification.

-- William

[Thierry]

On Tue, Jun 07, 2016 at 10:35:19AM -0400, William Stein wrote:
> Hi Thierry,
>
> I don't really know who you are, and I've never met you. But given
> your claims that I'm "spreading wrong and racist statements about
> french universities"... etc., I guess I do now. Many thanks for the
> clarification.

If you deign read the links, you will notice this point was about Dima's
post. Anyway, some other claims concern your posts.

Thierry

[William Stein]

Hi,

The Air Conditioning and power at UW have been fixed, and everything
turned back on. I've pointed DNS back at trac.sagemath.org and
wiki.sagemath.org so you should see them shortly.

I'm setting up very frequent periodic automatic backups right now to
/orig on the GCE machines. I eagerly await somebody to volunteer to
migrate trac and wiki there. Perhaps Volker can start when he gets
back from travel. Volker -- whose opinion on our dev infrastructure I
value the highest of all right now, as he is the release manager --
has said two things:

-- tier one cloud hosting provider
-- Docker containers

so I'll do whatever I can to support that.

-- William

On Sun, Jun 5, 2016 at 11:09 PM, William Stein <wst...@gmail.com> wrote:
> Hi,
>
> Yesterday a machine at UW (the one hosting *.sagenb.org) was attacking
> other computers, so the UW shut it down.
> Today trac, the wiki, and many other machines crashed.
>
> I'm traveling, so can't look into the hardware anytime soon. Morever,
> the situation is that there is nobody else who will either. I ran out
> of money, and had to focus on other things, so what's left of the UW
> sage-related compute infrastructure is basically dead now.
>
> I've created two VM's on Google compute engine (running the same OS --
> Ubuntu 12.04 -- as trac and wiki) and pointed
>
> http://trac.sagemath.org/ and http://wiki.sagemath.org/
>
> at them with a message that both sites are down.
>
> Thierry said he has a recent backup of the wiki data on the VM hosting
> wiki.sagemath.org, but will wait a day or two before providing it.
>
> I have a backup of trac.sagemath.org from May 26, which might be
> valid. However, I think the only person who understands the trac
> setup is Andrew Ohana, and he hasn't been involved in a very long
> time...
>
> The options I can think of right now:
>
> 1. Wait an indefinite amount of time to see if somehow magically the
> machines come back.
>
> 2. Move all Sage development to github: https://github.com/sagemath/sage
> I just re-enabled the github issues page, etc.
> There is also a wiki here https://github.com/sagemath/sage/wiki
> At some point migrate the data on wiki.sagemath.org to github (maybe
> remove all images if that is a problem).
> Eventually, bring trac.sagemath.org back in read only mode and copy
> over all open tickets to github issues,
> via some python script that goes through the trac database and creates
> github tickets.

>
> 3. Configure trac.sagemath.org and wiki.sagemath.org so they work
> again using the most recent backups.
>

> I am personally *not* going to do anything further except give people
> ssh access to the above machines and/or point DNS somewhere.
>
> Ultimately, Volker is the release manager and whatever he wants to do
> to continue making releases will hopefully happen...
>
> -- William



--
William (http://wstein.org)

[Dima Pasechnik]

On Tuesday, June 7, 2016 at 3:24:43 PM UTC+1, Thierry (sage-googlesucks@xxx) wrote:

Hi,

On Mon, Jun 06, 2016 at 02:35:22PM -0400, William Stein wrote:
[...]
> I am flat out strongly against hosting the Sage wiki, trac, etc.,.
> They should be properly hosted in the cloud, where anybody can be
> added as admins,

FUD

> we can make global backups,

FUD

> move the machines to have a presence anywhere in the world (due to many
> data centers all over),

FUD

> etc.

FUD

> I feel like I've been way too hesitant regarding expressing my
> opinions about the Sage infrastructure for the last year.
>
>  -- William


Following the "don't feed the troll" principle, i did not answer all your
(and Dima's) previous attacks against myself and the people who currently
work on the Sage infrastructure, but this is going too far. Some examples
include:

- spreading wrong and racist statements about french universities and how
  they manage access to VMs [1]

As [1] refers to a post by me, I think I must clarify that [1] is based on my personal experience

mostly in Uni. Rennes II about 7-10 years ago. Perhaps I should say that I spent a number of months there

on a number of occasions; as well as spending several more months visiting Uni. Brest, IHP, and ENS.

None of them offered (or agreed) to keep my computer account open for more than a couple of months after my departure.

And in most places local system admin was almost non-existent.

Well, not that I know many places where

it's considerably better, and I did visit few dozens of universities and research institutes for more or less

extended periods, that involved dealing with IT. And this is why I think that confining things

to hosting at a particular university is not a good idea.

And for "rasist", I don't even have a clue what you meant. Really.

I think it's just a plain insult in my address.

Dima

[William Stein]

On Tue, Jun 7, 2016 at 11:12 AM, William Stein <wst...@gmail.com> wrote:
> Hi,
>

> The Air Conditioning and power at UW have been fixed, and everything
> turned back on. I've pointed DNS back at trac.sagemath.org and
> wiki.sagemath.org so you should see them shortly.
>
> I'm setting up very frequent periodic automatic backups right now to
> /orig on the GCE machines. I eagerly await somebody to volunteer to
> migrate trac and wiki there. Perhaps Volker can start when he gets
> back from travel. Volker -- whose opinion on our dev infrastructure I
> value the highest of all right now, as he is the release manager --
> has said two things:
>
> -- tier one cloud hosting provider
> -- Docker containers
>
> so I'll do whatever I can to support that.

Erik has volunteered to setup trac, and will do so soon.

Still waiting for somebody to setup the wiki.

William

[Volker Braun]

On Tuesday, June 7, 2016 at 11:22:10 AM UTC-4, William wrote:

Erik has volunteered to setup trac, and will do so soon. 

There are a bunch of caveats to that one as far as I remember:

* The ssh key management plugin for trac shares the keys over the local file system with gitolite authentication system

* The cgit repo viewer accesses the gitolite repos over the local file system.

* The branch diff viewer (part of our trac plugin) does merges on /tmp and is prone to leaving large stale temp directories lying around.

Its rather annoying that trac, git repo, and git viewer are so tightly coupled and that has caused some issues in the past (most notably disk full -> trac down). 

A somewhat related issue is authentication. The trac authentication plugin hasn't been too much of a pleasure to work with, part of our users can't reset their passwords apparently because the database format changed at one point. Also I don't remember if / how the wiki authentication synchronizes with it. It would be great if we could use github authentication, say.

[Dima Pasechnik]

There is no need to do the git authentication on a trac instance, if its git server is read-only (and gets updates from another git server, on which the real pushing is done).

Here is such a scheme for trac+github integration:  https://github.com/trac-hacks/trac-github

(imho it's a good idea, as it also does a real-time backup of git repo along the way)...

[Volker Braun]

Our git server is read/write, of course. Authentication is so that only you can push to branches matching u/dimpase/.*, and that is handled by gitolite.

There is no authentication for the cgit repo viewer, as you said.

Ideally, everything but the git server and the trac database would be separate (and able to scale horizontally), but thats not the way it is right now. 

[Dima Pasechnik]

On Tuesday, June 7, 2016 at 5:37:47 PM UTC+1, Volker Braun wrote:

Our git server is read/write, of course. Authentication is so that only you can push to branches matching u/dimpase/.*, and that is handled by gitolite.

There is no authentication for the cgit repo viewer, as you said.

Ideally, everything but the git server and the trac database would be separate (and able to scale horizontally), but thats not the way it is right now. 

by the way, this trac-github is a standard trac plugin.

[Erik Bray]

Thanks. I haven't even looked at the git stuff yet, but that will be
good to know when I do. Just trying to get Trac back up for now (at
*least* the tickets). The other bits will follow shortly.

Do you (or anyone) know what server was being used previously for
Trac? Was it using tracd? Apache? nginx?
Not that it terribly matters I can just set up whatever works for now,
but it would be helpful to know especially if it was one of the latter
two, and if there is any useful configuration for them I should go
looking for before starting from scratch.

Thanks,
Erik

[Volker Braun]

Right now its apache; You should really go and look around on the trac.sagemath.org vm for how its set up right now. If you don't have an account there I can probably set you up if you send me your private key.

[Erik Bray]

Also does anyone know what version of Trac it was running? The
db_version is 29, so that seems to suggest 1.0.x. But it would be
nice to be sure.

I'll work on documenting this as well, and will put the documentation
somewhere that isn't sage trac or sage wiki :) (though it can go
there too).

Erik

[Jan Groenewald]

On 7 June 2016 at 18:54, Volker Braun <vbrau...@gmail.com> wrote:

I can probably set you up if you send me your private key.

private key or public key?

Regards,

Jan

[Erik Bray]

On Tue, Jun 7, 2016 at 12:54 PM, Volker Braun <vbrau...@gmail.com> wrote:
> Right now its apache; You should really go and look around on the
> trac.sagemath.org vm for how its set up right now. If you don't have an
> account there I can probably set you up if you send me your private key.

Oh, it's back up again! I didn't realize that. Well in that case I'm
going to go make dinner now and will return to this tomorrow as it
feels less urgent now. But I will get the new server setup ASAP.

Erik

[Volker Braun]

Sholud have been public key, of course. Though you all can send me your private keys, too, if you want ;-)

[Erik Bray]

I'll let you know tomorrow. It's possible someone already added me,
but if so they used a key that's only on a computer I'm not using
right now

Erik

[kcrisman]

Having now read the comment [1] I agree that while it may not be true about French universities in general, it was certainly phrased as "my experience" and was limited to the relatively narrow technical comment of who administrators might feel comfortable allowing access to.  I have already been told multiple times here (not in France) that I wouldn't be allowed to host things (even a patchbot) so while one might disagree on the wisdom of such policies, it isn't limited to any particular nationality, and hopefully we can move any further comments about [1] to sage-flame.

However, these comments do touch on something substantive:

- suggesting that nobody was working on the migration while you know that
  we are working to set things up correctly (e.g. remote backup system),
  and considering that workers are at your disposal, this is extremly
  patronizing [2]
- maintaining the confusion between university hosting and the very bad
  practices that led us to this situation [3] [4]

Namely that GCE might also have some drawbacks (notably, not being provided free of charge? perhaps others) and that there were people hoping to make a university system work.  If there is a technical disagreement (or more than technical) on cloud versus uni, I'm not getting into that, but I don't see why we couldn't host in one place and backup in the other (indeed, multiple).  What does R do? 

 

[Dima Pasechnik]

R has so much commercial (and university - cf. CRAN) backing, it's meaningless to compare it to Sage. 

 

 

[Erik Bray]

I had suggested in private that maybe we use the university systems at
our disposal as a primary service, but with a replicated setup on GCE
for backup. Nobody I suggested that to seemed interested.

Erik

[kcrisman]

Namely that GCE might also have some drawbacks (notably, not being provided free of charge? perhaps others) and that there were people hoping to make a university system work.  If there is a technical disagreement (or more than technical) on cloud versus uni, I'm not getting into that, but I don't see why we couldn't host in one place and backup in the other (indeed, multiple).  What does R do? 

R has so much commercial (and university - cf. CRAN) backing, it's meaningless to compare it to Sage. 

True but one can wish. 

[Bill Hart]

I don't think that anyone is making a case specifically against French Universities any more than they are making a case against American univerisities.

Some of us now have now had bad experiences on both sides of the Atlantic (specifically France because they do host a lot of Free Software infrastructure, not because they are French). For example, I have had tickets and emails ignored for almost a year in one case.

University admins the world over are not equipped to support large Open Source international projects as reliably as we'd like.

William is 100% right here. We should not be relying on specific universities, especially when that would add to the load of sysadmins who will just end up ignoring tickets, even when there is a system explicit set up to deal with such problems.

Bill.

[Thierry]

Such calls for a one-day-hero are part of the problem, not the solution (at
least not a viable one as we could observe previously).

Ciao,
Thierry


> William
>

[William Stein]

If you're going to flame me don't do it here.  I really want this list to be civil again. 

 

Ciao,
Thierry


> William

>

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

--
Sent from my massive iPhone 6 plus.

[Vincent Delecroix]

On 18/06/16 22:48, William Stein wrote:
> On Saturday, June 18, 2016, Thierry <sage-goo...@lma.metelu.net> wrote:
>
>> On Tue, Jun 07, 2016 at 11:21:29AM -0400, William Stein wrote:
>>> On Tue, Jun 7, 2016 at 11:12 AM, William Stein <wst...@gmail.com
>> <javascript:;>> wrote:
>>>> Hi,
>>>>
>>>> The Air Conditioning and power at UW have been fixed, and everything
>>>> turned back on. I've pointed DNS back at trac.sagemath.org and
>>>> wiki.sagemath.org so you should see them shortly.
>>>>
>>>> I'm setting up very frequent periodic automatic backups right now to
>>>> /orig on the GCE machines. I eagerly await somebody to volunteer to
>>>> migrate trac and wiki there. Perhaps Volker can start when he gets
>>>> back from travel. Volker -- whose opinion on our dev infrastructure I
>>>> value the highest of all right now, as he is the release manager --
>>>> has said two things:
>>>>
>>>> -- tier one cloud hosting provider
>>>> -- Docker containers
>>>>
>>>> so I'll do whatever I can to support that.
>>>
>>> Erik has volunteered to setup trac, and will do so soon.
>>>
>>> Still waiting for somebody to setup the wiki.
>>
>> Such calls for a one-day-hero are part of the problem, not the solution (at
>> least not a viable one as we could observe previously).
>
>
> If you're going to flame me don't do it here. I really want this list to
> be civil again.

Why are you so odious? I see no agression in the preceding message and
it makes a lot of sense: "To achieve something viable it is better to do
it together and with reasonable time schedule".

For me, the flame applies much more to your two messages.

Not mentioning the fact that Thierry took the time to set up the wiki on
an alternative sever...

Cheers,
Vincent