Ultimatum: sage's trac and wiki

[William Stein]

Hi Sage Developers,

Can somebody *PLEASE* volunteer to move trac.sagemath.org and
wiki.sagemath.org to a VM on GCE and maintain it for a while?

I've setup a complete Google compute engine project specifically for
this and I'll pay for this indefinitely. The project has a machine on
it with ssh keys that can ssh to the root account of trac.sagemath.org
and wiki.sagemath.org. I can easily add anybody with a Google
account to this project (though I only want to add people that at
least enable 2-factor auth, since you can waste a lot of money if you
get hacked).

Why move to GCE from UW? (1) the machines at UW are not backed up at
all! -- I assumed a certain student had them backed up, but nope; in
fact, we can't even ssh into the VM host right now (obviously, I could
get physical access). (2) With GCE it's trivially to snapshot the
whole VM's hundreds of times for very, very cheap, and things are very
easy to scale, and every person who I add to the project has full
access to everything through a web browser. It's just vastly better.

The other option is that we move *everything* to Github. I think
that would be way too disruptive to do right now.

-- William

[Dima Pasechnik]

Hi William,

I have already volunteered; Thierry set me up  accounts on trac and wiki hosts, but I (still) lack sudo rights.

Perhaps for moving the thing it's not needed, I didn't check.

(Thierry replies to my emails very sparingly, like once a week?)

Moving wiki as it is to a VM does not look like a good idea, as it's running on a very old Ubuntu,

and it'd need testing to see if its setup can be moved to an up-to date Ubuntu (or whatever).

Perhaps it's indeed better to move the wiki to github (there were discussions about better/best wikis

here, I didn't pay much attention to them).

I do have 2-step auth enabled for my google account.

Cheers,

Dima

[William Stein]

On Fri, Feb 12, 2016 at 1:53 AM, Dima Pasechnik <dim...@gmail.com> wrote:
> Hi William,
> I have already volunteered; Thierry set me up accounts on trac and wiki
> hosts, but I (still) lack sudo rights.

Dima -- excellent -- I've added you to the Google Compute Engine project.

I'll send you another email offlist.

> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.



--
William (http://wstein.org)

[Thierry]

Hi,

On Thu, Feb 11, 2016 at 05:42:55PM -0800, William Stein wrote:
> Hi Sage Developers,
>
> Can somebody *PLEASE* volunteer to move trac.sagemath.org and
> wiki.sagemath.org to a VM on GCE and maintain it for a while?

I already volonteered for this so i guess it is useless to answer again.

Actually, the following is currently done:
- contact people working on ask.sagemath.org and mmarco (who set up
letsencrypt for the wiki), most seemed to be in as well (+dimpase now)
- ask for 2 fresh VM to my university:
http://sagewiki.lipn.univ-paris13.fr/
http://sagetrac.lipn.univ-paris13.fr/
- work on understanding the existing setup and try to reproduce a separate
working environment on a personal VM (dpkg -l, debsums, diff, and all
the like) with nonduplicated tools between services (e.g. apache+nginx,
mysql+postgres,...), wiki is almost done, there are still things to
understand for git/trac, i might have to ask some questions later.

Note that the wiki (trac as well) runs Ubuntu 12/04 and according to the
welcome message of the wiki:

Your current Hardware Enablement Stack (HWE) is no longer supported
since 2014-08-07. Security updates for critical parts (kernel and
graphics stack) of your system are no longer available.

As nothing is documented, it seems dangerous to just upgrade the
production VM on itself.

I think the current situation is the consequence of the kind of
"ultimatum" that is sent in such situation. While it looks easier to ask
students to do the job, or developpers to quick-and-dirty fix when some
problem happens, this costs actually a lot more time to the people that
eventually do the job. When ask.sagemath.org went down, i personally spent
a few days to just figure out how were the things configured. This kind of
archeology wastes way more time than a serious documented installation
from the beginning (at least it is more frustrating).

> I've setup a complete Google compute engine project specifically for
> this and I'll pay for this indefinitely. The project has a machine on
> it with ssh keys that can ssh to the root account of trac.sagemath.org
> and wiki.sagemath.org. I can easily add anybody with a Google
> account to this project (though I only want to add people that at
> least enable 2-factor auth, since you can waste a lot of money if you
> get hacked).
>
> Why move to GCE from UW? (1) the machines at UW are not backed up at
> all! -- I assumed a certain student had them backed up, but nope; in
> fact, we can't even ssh into the VM host right now (obviously, I could
> get physical access). (2) With GCE it's trivially to snapshot the
> whole VM's hundreds of times for very, very cheap, and things are very
> easy to scale, and every person who I add to the project has full
> access to everything through a web browser. It's just vastly better.

I am not sure the arguments are related to the kind of hosting: it is more
about setting up scheduled backups! Having physical access to the machines
and having direct contact with the sysadmins can be very handy in some
circumstances too.

Moreover, the previous instance of ask.sagemath.org was hosted on GCE as
well, and i remember we finally had to move to a more permanent place at
Niles university (OSU), which is definitely more stable.

As in the "emergency" thread in may 2015, i asked to my lab and it is OK
to host those two servers, and they went just set up. I asked for similar
performance than the existing VM's, it seems they are lacking some RAM
right now but i guess we could ask for more when needed.

I did a quick backup (though not recurrent) of trac and wiki so that we
should not lose much in case of crash.

What is the deadline to leave UW for git/trac/wiki ? I will be almost
completely offline for the next 3 days, moreover i guess it is better to
have some time to do things collectively and documented, or we will face
the same issue again and again.

Ciao,
Thierry

> The other option is that we move *everything* to Github. I think
> that would be way too disruptive to do right now.
>
> -- William
>

[Dima Pasechnik]

On Mon, Feb 15, 2016 at 1:27 AM, Thierry
<sage-goo...@lma.metelu.net> wrote:
> Hi,
>
> On Thu, Feb 11, 2016 at 05:42:55PM -0800, William Stein wrote:
>> Hi Sage Developers,
>>
>> Can somebody *PLEASE* volunteer to move trac.sagemath.org and
>> wiki.sagemath.org to a VM on GCE and maintain it for a while?
>
> I already volonteered for this so i guess it is useless to answer again.
>
> Actually, the following is currently done:
> - contact people working on ask.sagemath.org and mmarco (who set up
> letsencrypt for the wiki), most seemed to be in as well (+dimpase now)
> - ask for 2 fresh VM to my university:
> http://sagewiki.lipn.univ-paris13.fr/
> http://sagetrac.lipn.univ-paris13.fr/

This would seem to imply that noone outside Paris 13 would be able to log in
to these VMs, or one would be at mercy of often unpredictable university
administration in this regard.
My experience with French universities is that they are often not at all happy
to give meaningful computer access to me as an official visitor, them
allowing arbitrary people having root access on their computers sounds
very unusual.
In this sense choice of GCE seems to make much better sense.

> - work on understanding the existing setup and try to reproduce a separate
> working environment on a personal VM (dpkg -l, debsums, diff, and all
> the like) with nonduplicated tools between services (e.g. apache+nginx,
> mysql+postgres,...), wiki is almost done, there are still things to
> understand for git/trac, i might have to ask some questions later.
>
> Note that the wiki (trac as well) runs Ubuntu 12/04 and according to the
> welcome message of the wiki:
>
> Your current Hardware Enablement Stack (HWE) is no longer supported
> since 2014-08-07. Security updates for critical parts (kernel and
> graphics stack) of your system are no longer available.
>
> As nothing is documented, it seems dangerous to just upgrade the
> production VM

IMHO it's inviting trouble to post this in a open forum, do you see why?

well, see above. It appears to severely limit the range of people who have any
access to the systems at all.

Dima

> You received this message because you are subscribed to a topic in the Google Groups "sage-devel" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-devel/ed_ya-d-k_E/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to sage-devel+...@googlegroups.com.

[Thierry Dumont]

Le 15/02/2016 11:50, Dima Pasechnik a écrit :
> On Mon, Feb 15, 2016 at 1:27 AM, Thierry
> <sage-goo...@lma.metelu.net> wrote:
>> Hi,
>>
>> On Thu, Feb 11, 2016 at 05:42:55PM -0800, William Stein wrote:
>>> Hi Sage Developers,
>>>
>>> Can somebody *PLEASE* volunteer to move trac.sagemath.org and
>>> wiki.sagemath.org to a VM on GCE and maintain it for a while?
>>
>> I already volonteered for this so i guess it is useless to answer again.
>>
>> Actually, the following is currently done:
>> - contact people working on ask.sagemath.org and mmarco (who set up
>> letsencrypt for the wiki), most seemed to be in as well (+dimpase now)
>> - ask for 2 fresh VM to my university:
>> http://sagewiki.lipn.univ-paris13.fr/
>> http://sagetrac.lipn.univ-paris13.fr/
>
> This would seem to imply that noone outside Paris 13 would be able to log in
> to these VMs, or one would be at mercy of often unpredictable university
> administration in this regard.
> My experience with French universities is that they are often not at all happy
> to give meaningful computer access to me as an official visitor, them
> allowing arbitrary people having root access on their computers sounds
> very unusual.
> In this sense choice of GCE seems to make much better sense.
>

Maybe we can ask to put sage related virtual machines on the French
*CNRS* (Math. Institute/ Mathrice) machines, and have root access on
them. Nicolas (Thiéry)(and others), what do you think of that?

We already have a Jupyter/sage CNRS server. It would have sense.

T.

[William Stein]

On Mon, Feb 15, 2016 at 2:50 AM, Dima Pasechnik <dim...@gmail.com> wrote:
> On Mon, Feb 15, 2016 at 1:27 AM, Thierry
> <sage-goo...@lma.metelu.net> wrote:
>> Hi,
>>
>> On Thu, Feb 11, 2016 at 05:42:55PM -0800, William Stein wrote:
>>> Hi Sage Developers,
>>>
>>> Can somebody *PLEASE* volunteer to move trac.sagemath.org and
>>> wiki.sagemath.org to a VM on GCE and maintain it for a while?
>>
>> I already volonteered for this so i guess it is useless to answer again.
>>
>> Actually, the following is currently done:
>> - contact people working on ask.sagemath.org and mmarco (who set up
>> letsencrypt for the wiki), most seemed to be in as well (+dimpase now)
>> - ask for 2 fresh VM to my university:
>> http://sagewiki.lipn.univ-paris13.fr/
>> http://sagetrac.lipn.univ-paris13.fr/
>
> This would seem to imply that noone outside Paris 13 would be able to log in
> to these VMs, or one would be at mercy of often unpredictable university
> administration in this regard.
> My experience with French universities is that they are often not at all happy
> to give meaningful computer access to me as an official visitor, them
> allowing arbitrary people having root access on their computers sounds
> very unusual.
> In this sense choice of GCE seems to make much better sense.

Thierry -- many thanks for getting a backup setup and doing work on
this. I really appreciate your work.

I agree with Dima though. We will host on GCE. I've already setup the
GCE project and added Dima, I'm willing to pay for it indefinitely,
and I can add numerous people to the project from all over the world,
and they will have full access. By hosting on GCE, we get a lot of
advantages beyond just making it easy for any project member to add
other admins. For example, it is very, very easy to make snapshots
of disk images (and/or tarballs) that are globally distributed. It's
also easier for us to scale up or down or move the geographic location
of the servers.

Thierry -- I'm also happy with adding you if you can send me an
@gmail.com address (and confirm that you setup 2-factor auth on it).
However, if you have a fundamental opposition to using any Google
technology (your email is "sage-googlesucks@..."), then I understand,
and will work to find other support.

William (wst...@gmail.com)

William (http://wstein.org)

[Nicolas M. Thiery]

On Mon, Feb 15, 2016 at 10:14:51AM -0800, William Stein wrote:
> Thierry -- I'm also happy with adding you if you can send me an
> @gmail.com address (and confirm that you setup 2-factor auth on it).
> However, if you have a fundamental opposition to using any Google
> technology (your email is "sage-googlesucks@..."), then I understand,
> and will work to find other support.

Trivial note: there are two Thierry's involved here: Monteil
(LIPN/CNRS) and Dumont (Lyon/CNRS).

And this is not even including me. But I have my accent to make the
difference :-)

Cheers,
Nicolas
--
Nicolas M. Thiéry "Isil" <nth...@users.sf.net>
http://Nicolas.Thiery.name/

[Martin Vahi]

reede, 12. veebruar 2016 9:53.23 UTC kirjutas Dima Pasechnik:

Hi William,

I have already volunteered; Thierry set me up  accounts on trac and wiki hosts, but I (still) lack sudo rights.

Perhaps for moving the thing it's not needed, I didn't check.

(Thierry replies to my emails very sparingly, like once a week?)

Moving wiki as it is to a VM does not look like a good idea, as it's running on a very old Ubuntu,

and it'd need testing to see if its setup can be moved to an up-to date Ubuntu (or whatever).

Perhaps it's indeed better to move the wiki to github (there were discussions about better/best wikis

here, I didn't pay much attention to them).

I do have 2-step auth enabled for my google account.

I do not know about the trac part, because the trac might be fancier than the

https://www.fossil-scm.org/

bugtracking feature set, but if the amount of Sage developers is less than 100 and the amount of edits is as low as it seems to me, then the Fossil might very well run the Wiki. The main issue with Fossil Wiki is the difficulty of displaying images and WebM movies, but for textual parts it's probably fine. Fossil stores everything, Wiki history, bug-track history, code commits and their history to a single file. Fossil itself is a single binary that gets that file as an input and can run as a CGI-program, a stand-alone web server, also as something wrapped to PHP shell execs. 

If that sounds appealing to You, then You might want to try the Fossil out at

http://chiselapp.com/repositories/

Fossil history:
https://www.youtube.com/watch?v=jPSOxVjK8A0
https://www.youtube.com/watch?v=O0sP9Qw23JA

Thank You for reading my comment. :-)