incremental migration to github?

[Ralf Stephan]

Can you think of ways to move development step-by-step to github?

What would be wrong with accepting pull requests for *some parts of Sage, then review, followed by submission to trac by an intermediary?

This would need a second repository I guess. Permissions to merge could be given on request.

Other ideas?

[William Stein]

Hi,

For what it is worth I'm highly supportive of Sage development moving to github.   However, I think the release manager should be completely 100% in charge of where Sage dev happens.  It's much more important that we have a solid process for doing sage releases than anything else.  

Robert Bradshaw once wrote some sort of github <---> trac bot.  Maybe he can say something about that. 

Anyway what github have accomplished in the last few years is very impressive.

William

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

--
Sent from my massive iPhone 6 plus.

[Bill Page]

On 11 January 2016 at 11:26, William Stein <wst...@gmail.com> wrote:
>
> For what it is worth I'm highly supportive of Sage development moving to
> github.

>...

> Anyway what github have accomplished in the last few years is very
> impressive.
>

+1

[Volker Braun]

As William already said, there is the github<->trac bot. Even without that, you can just copy branches over. So if you want to do the review on github and then stick it into trac thats easy to do.

But if you want to use the github issue tracker then that wouldn't work as easily. I don't think we even can import our current trac database, not to mention that some fields (e.g. Reviewer) are missing.

I also think that one of the nice features of the current workflow is that ticket information is summarized in the merge commit. If you use github's merge button then it won't be there, making the history more difficult to understand.

Also there is no comparable free CI to our buildbot instance, e.g. travis-ci doesn't have any platform diversity (ok, the paid version also has OSX in addition to their linux docker instance).

[kcrisman]

But if you want to use the github issue tracker then that wouldn't work as easily. I don't think we even can import our current trac database, not to mention that some fields (e.g. Reviewer) are missing.

There's also the non-trivial (though not blocker, probably) issue that zillions of links to trac.sagemath.org would instantly be obsolete, and we'd be dependent upon a private company to maintain our bug tracker - one which might not be easily recreatable in the event GH gets bought or they go out of business or the service is taken down for whatever reason.

On a curiosity note, would GH be able to import the many cross-references within Trac itself to its native xref link creation?  (I mean things like the [comment:56:ticket:100 this link to a really vital comment] and #12345 syntax, presumably those are themselves some kind of plugin to Trac.)

- kcrisman

[William Stein]

On Mon, Jan 11, 2016 at 10:43 AM, kcrisman <kcri...@gmail.com> wrote:
>
>>
>>
>> But if you want to use the github issue tracker then that wouldn't work as
>> easily. I don't think we even can import our current trac database, not to
>> mention that some fields (e.g. Reviewer) are missing.
>>
>
> There's also the non-trivial (though not blocker, probably) issue that
> zillions of links to trac.sagemath.org would instantly be obsolete,

No. We wouldn't shut down running trac for a long, long time.

> and we'd
> be dependent upon a private company to maintain our bug tracker - one which
> might not be easily recreatable in the event GH gets bought or they go out
> of business or the service is taken down for whatever reason.

Nothing is black and white like you make it out above. It's a
tradeoff. Having used GH, it's definitely way, way more than worth
that tradeoff. Not making this tradeoff could easily lead to the
death of Sage, since Sage development is definitely WAY WAY too hard
and full of friction. If somebody else came along with something like
Sage that used GH and provided a much more pleasant development
experience, the Sage project would be in very serious trouble.
Waiting until it is too late is no solution. I'm so glad that
certain people (not me!) had the foresight to switch away from
Mercurial!

> On a curiosity note, would GH be able to import the many cross-references
> within Trac itself to its native xref link creation? (I mean things like
> the [comment:56:ticket:100 this link to a really vital comment] and #12345
> syntax, presumably those are themselves some kind of plugin to Trac.)

GH is amazing at automatically cross referencing. As are the many
integrations, e.g. with gitter, slack, etc.

William

>
> - kcrisman

>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.



--

William (http://wstein.org)

[William Stein]

On Mon, Jan 11, 2016 at 10:52 AM, William Stein <wst...@gmail.com> wrote:
> On Mon, Jan 11, 2016 at 10:43 AM, kcrisman <kcri...@gmail.com> wrote:
>>
>>>
>>>
>>> But if you want to use the github issue tracker then that wouldn't work as
>>> easily. I don't think we even can import our current trac database, not to
>>> mention that some fields (e.g. Reviewer) are missing.
>>>
>>
>> There's also the non-trivial (though not blocker, probably) issue that
>> zillions of links to trac.sagemath.org would instantly be obsolete,
>
> No. We wouldn't shut down running trac for a long, long time.
>
>> and we'd
>> be dependent upon a private company to maintain our bug tracker - one which
>> might not be easily recreatable in the event GH gets bought or they go out
>> of business or the service is taken down for whatever reason.
>
> Nothing is black and white like you make it out above. It's a
> tradeoff. Having used GH, it's definitely way, way more than worth
> that tradeoff. Not making this tradeoff could easily lead to the
> death of Sage, since Sage development is definitely WAY WAY too hard
> and full of friction. If somebody else came along with something like
> Sage that used GH and provided a much more pleasant development
> experience, the Sage project would be in very serious trouble.
> Waiting until it is too late is no solution. I'm so glad that
> certain people (not me!) had the foresight to switch away from
> Mercurial!

Following up on this, that we don't fully support people doing
development for Sage by creating independent pip-installable packages
(which depend on sage) is a *major* point of friction for our project.
The sage dev process is very heavy and confusing compared to what it
should be. This friction could kill us.

William



--
William (http://wstein.org)

[Jeroen Demeyer]

On 2016-01-11 17:26, William Stein wrote:
> Hi,
>
> For what it is worth I'm highly supportive of Sage development moving to
> github.

I like trac (especially the way Sage uses it) a lot better than github.

What I mostly dislike about github is that "issues" and "pull requests"
are different things. I very much prefer the trac model where you create
a ticket, discuss things, then have multiple people work together on a
branch all on the same page. With github, it sometimes happens that you
have one issue and several pull requests by different people which are
all about the same thing. I get lost more easily in the github forest.

Another useful thing is that everything on trac is in one git tree. I
can do "git fetch" and have all tickets ready to check out without any
hassle. I don't know if you can easily checkout a pull request from github.

Finally a stupid thing: I don't get why github discussions don't have a
"reply" button.

I must admit that I don't use github that heavily, so maybe there are
things that I am missing.


Jeroen.

[Volker Braun]

On Monday, January 11, 2016 at 8:20:45 PM UTC+1, William wrote:

Following up on this, that we don't fully support people doing
development for Sage by creating independent pip-installable packages 

Where is the problem, I did that before and it works just fine. 

Of course sage isn't on pypi so you can't auto-download it as a dependency, but then its doubtful that this would work anyways. There are way too may specialized shared-library dependencies, and if there is one thing that really sucks then that's (pip,npm,rvm,...)-packages that start compiling gobs of third-party C/C++ code when installing. Neither is pip/wheel/... made for distributing binaries of third-party code. So realistically there should always be a "sage runtime" to compile the dependencies before installing Sage-the-python-library. But you can just use pip to install packages depending on Sage on top of that, no problem.

[William Stein]

On Mon, Jan 11, 2016 at 11:39 AM, Volker Braun <vbrau...@gmail.com> wrote:
> On Monday, January 11, 2016 at 8:20:45 PM UTC+1, William wrote:
>>
>> Following up on this, that we don't fully support people doing
>> development for Sage by creating independent pip-installable packages
>
>
> Where is the problem, I did that before and it works just fine.

Technically there is no problem. The problem is mostly one of
documentation and culture. How many sage dependent packages are on
pypi?

>
> Of course sage isn't on pypi so you can't auto-download it as a dependency,
> but then its doubtful that this would work anyways. There are way too may
> specialized shared-library dependencies, and if there is one thing that
> really sucks then that's (pip,npm,rvm,...)-packages that start compiling
> gobs of third-party C/C++ code when installing. Neither is pip/wheel/...
> made for distributing binaries of third-party code. So realistically there
> should always be a "sage runtime" to compile the dependencies before
> installing Sage-the-python-library. But you can just use pip to install
> packages depending on Sage on top of that, no problem.

Yes, the latter. We could have a simple dependency, e.g., "sagelib"
or something, that pip pulls in, and it simply checks that sage is
installed, and if not, explains the situation. Then in a few years
when I prove you wrong regarding "you can't auto-download it as a
dependency, but then its doubtful that this would work anyways." then
we can replace that with sage itself.

William

[William Stein]

Why Python moved to github:

http://www.snarky.ca/the-history-behind-the-decision-to-move-python-to-github

[Jeroen Demeyer]

On 2016-01-15 16:00, William Stein wrote:
> Why Python moved to github:
>

> http://www.snarky.ca/the-history-behind-the-decision-to-move-python-to-github<http://www.snarky.ca/the-history-behind-the-decision-to-move-python-to-github#toc_3>

It seems to boil down to "we use GitHub because everybody uses GitHub".
I see the point, but I would certainly miss Trac :-(

[William Stein]

That was the argument for github versus gitlab.  It was  not the argument for switching away from hosting their own infrastructure. 

 

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

--

[Vincent Delecroix]

On 15/01/16 15:29, William Stein wrote:
> On Friday, January 15, 2016, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:
>
>> On 2016-01-15 16:00, William Stein wrote:
>>
>>> Why Python moved to github:
>>>
>>>
>>> http://www.snarky.ca/the-history-behind-the-decision-to-move-python-to-github
>>> <
>>> http://www.snarky.ca/the-history-behind-the-decision-to-move-python-to-github#toc_3
>>>>
>>>
>>
>> It seems to boil down to "we use GitHub because everybody uses GitHub". I
>> see the point, but I would certainly miss Trac
>
>
>
> That was the argument for github versus gitlab. It was not the argument
> for switching away from hosting their own infrastructure.

Their infrastructure was terrible. This is not the case of Sage. It is
not clear to me what would be better with github? Does anybody has a
serious proposal for a github workflow? If you do so, then open a trac
ticket with it! I only see a potential increase of the number of pull
requests. Which would actually be bad since we have a lot of pending
tickets.

Doing as "Guido says" or "as everybody does" is by far the worse
arguments I can imagine. And I agreed with something in the post: just
pay attention to people who are clearly willing to do the job.

Vincent

[Ralf Stephan]

On Monday, January 11, 2016 at 8:25:32 PM UTC+1, Jeroen Demeyer wrote:

What I mostly dislike about github is that "issues" and "pull requests"
are different things. I very much prefer the trac model where you create
a ticket, discuss things, then have multiple people work together on a
branch all on the same page. With github, it sometimes happens that you
have one issue and several pull requests by different people which are
all about the same thing. I get lost more easily in the github forest.

On the other hand you can unsubscribe from each of these branches of

conversation. Also, since a new code branch is a separate pull request

the old branch stays visible until it's explicitly closed. It's up to you to add

# links if you want. Moreover, backlinks from any of the conversation branches

are shown automatically ("user referenced this pull request 5 days ago").

Another useful thing is that everything on trac is in one git tree. I
can do "git fetch" and have all tickets ready to check out without any
hassle. I don't know if you can easily checkout a pull request from github.

Of course you can.

 

Finally a stupid thing: I don't get why github discussions don't have a
"reply" button.

That is easily enhanced by custom scripts:

https://github.com/jerone/UserScripts

Can you do that with trac? Are there people interested, at all? 

[Ralf Stephan]

On Friday, January 15, 2016 at 7:45:23 PM UTC+1, vdelecroix wrote:

And I agreed with something in the post: just
pay attention to people who are clearly willing to do the job.

Right. No one in this thread has even looked at the original questions,

apparently. I was explicitly not proposing moving completely to github.

[Ralf Stephan]

On Friday, January 15, 2016 at 7:45:23 PM UTC+1, vdelecroix wrote:

I only see a potential increase of the number of pull
requests. Which would actually be bad since we have a lot of pending 

tickets.

 You're assuming that tickets are still pending only because of the number

of tickets. This is wrong. They are pending for various reasons, for example

because there is no matching reviewer. In what way is having more tickets

bad, then? You will only increase matches and so, contributions. No one

will punish anyone for having more tickets open. They will only increase the

code available.

[Jeroen Demeyer]

On 2016-01-16 08:15, Ralf Stephan wrote:
> Another useful thing is that everything on trac is in one git tree. I
> can do "git fetch" and have all tickets ready to check out without any
> hassle. I don't know if you can easily checkout a pull request from
> github.
>
> Of course you can.

Please tell me how! I want this feature.

[Jeroen Demeyer]

On 2016-01-16 08:15, Ralf Stephan wrote:

> On the other hand you can unsubscribe from each of these branches of
> conversation. Also, since a new code branch is a separate pull request
> the old branch stays visible until it's explicitly closed. It's up to
> you to add
> # links if you want. Moreover, backlinks from any of the conversation
> branches
> are shown automatically ("user referenced this pull request 5 days ago

> <https://github.com/symengine/symengine/pull/752#ref-issue-123534774>").

Adding links isn't nearly as practical as just having the discussion on
one page in the first place.

[Ralf Stephan]

https://gist.github.com/piscisaureus/3342247 

[Jeroen Demeyer]

On 2016-01-16 14:29, Ralf Stephan wrote:
> https://gist.github.com/piscisaureus/3342247

Sorry, but something which involves manually editing config files is not
"easily checkout a pull request".

But at least it works.

[Dima Pasechnik]

Dear all,

I'm resurrecting this thread, as the costs of the currect setup of trac hosted on Google Cloud Compute are prohibitive, at the current rate we are spending over US$4500 per year on it.

For trac+wiki+zulip: December 2019 – December 2020 (forecasted total cost)  $4,644.67

These are paid by UW's hosted Sage Foundation, which is projected to run out of money soon.

We've started OpenCollective and GitHub sponsoring, but the rate the donations coming in there would only suffices for perhaps 25-30% of this figure.

Options:

1. find money
2. find cheaper (ideally, free, based in an academic institution) provider
3. migrate to GitHub or another (semi)gratis platform, e.g. GitLab, sw.ht, etc.

Savings also may be made by re-hosting zulip and wiki somewhere else.

They should be very easy to move I suppose.

Dima

[David Roe]

For Zulip, zulipchat.com provides free hosting for open source projects.  I'm fairly confident that we could export our history and import it to a new organization there fairly easily, if there's a consensus to do so.

David

--

You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/95f0cc64-3f5e-494c-83f4-7c1127ae3a36n%40googlegroups.com.

[Vincent Delecroix]

All the services (= trac + wiki + zulip) could plausibly be hosted
by CNRS in France. There are dedicated servers for this purpose. If
it is accepted, the institute will support the cost.

I am available to open the server and discuss with the institute.
However, I won't have time to do any work on the migration.

Best
Vincent

Le 10/12/2020 à 11:41, Dima Pasechnik a écrit :
> Dear all,
>
> I'm resurrecting this thread, as the costs of the currect setup of trac
> hosted on Google Cloud Compute are prohibitive, at the current rate we are
> spending over US$4500 per year on it.
>
> For trac+wiki+zulip: December 2019 – December 2020 (forecasted total cost)
> $4,644.67
>
> These are paid by UW's hosted Sage Foundation, which is projected to run
> out of money soon.
> We've started OpenCollective and GitHub sponsoring, but the rate the
> donations coming in there would only suffices for perhaps 25-30% of this
> figure.
>
> Options:
>
>

> 1. find money
> 2. find cheaper (ideally, free, based in an academic institution)
> provider
> 3. migrate to GitHub or another (semi)gratis platform, e.g. GitLab,

[Dima Pasechnik]

On Thu, Dec 10, 2020 at 10:50 AM Vincent Delecroix

<20100.d...@gmail.com> wrote:
>
> All the services (= trac + wiki + zulip) could plausibly be hosted
> by CNRS in France. There are dedicated servers for this purpose. If
> it is accepted, the institute will support the cost.
>
> I am available to open the server and discuss with the institute.
> However, I won't have time to do any work on the migration.

How reliable are they? Needless to say, something that goes down and
stays down for a week in August (vacation!)
is not what we can agree to.

Are they willing to grant root access to outside persons? (This is
another deal-breaker)

Dima

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/5bb98aee-614b-f391-08e4-1661d8540a87%40gmail.com.

[Dima Pasechnik]

On Thu, Dec 10, 2020 at 10:49 AM David Roe <roed...@gmail.com> wrote:
>
> For Zulip, zulipchat.com provides free hosting for open source projects. I'm fairly confident that we could export our history and import it to a new organization there fairly easily, if there's a consensus to do so.

I don't think we even need a consensus for this. Lack of funds is a
sufficient reason to migrate.
(apart from the history, account data is important...)

We then can point zulip.sagemath.org to the appropriate url at zulipchat.com

Dima

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAChs6_%3DB6GQKi07kr%2BamOgh9xBqoowy57ZxSNF1HvwULCLvm_g%40mail.gmail.com.

[Vincent Delecroix]

Le 10/12/2020 à 11:54, Dima Pasechnik a écrit :
> On Thu, Dec 10, 2020 at 10:50 AM Vincent Delecroix
> <20100.d...@gmail.com> wrote:
>>
>> All the services (= trac + wiki + zulip) could plausibly be hosted
>> by CNRS in France. There are dedicated servers for this purpose. If
>> it is accepted, the institute will support the cost.
>>
>> I am available to open the server and discuss with the institute.
>> However, I won't have time to do any work on the migration.
>
> How reliable are they? Needless to say, something that goes down and
> stays down for a week in August (vacation!)
> is not what we can agree to.

Never had a problem, even in August. Though it does not mean much.

> Are they willing to grant root access to outside persons? (This is
> another deal-breaker)

This needs to go through administrative form filling. But it should
be doable.

Best
Vincent

[Dima Pasechnik]

On Thu, Dec 10, 2020 at 11:06 AM Vincent Delecroix

<20100.d...@gmail.com> wrote:
>
> Le 10/12/2020 à 11:54, Dima Pasechnik a écrit :
> > On Thu, Dec 10, 2020 at 10:50 AM Vincent Delecroix
> > <20100.d...@gmail.com> wrote:
> >>
> >> All the services (= trac + wiki + zulip) could plausibly be hosted
> >> by CNRS in France. There are dedicated servers for this purpose. If
> >> it is accepted, the institute will support the cost.
> >>
> >> I am available to open the server and discuss with the institute.
> >> However, I won't have time to do any work on the migration.
> >
> > How reliable are they? Needless to say, something that goes down and
> > stays down for a week in August (vacation!)
> > is not what we can agree to.
>
> Never had a problem, even in August. Though it does not mean much.
>
> > Are they willing to grant root access to outside persons? (This is
> > another deal-breaker)
>
> This needs to go through administrative form filling. But it should
> be doable.
>

How about starting down this road immediately, and see what we get in
the form of access
and hardware+bandwidth?

I suppose we can always utilise it for something else, less
mission-critical, e.g. as a dedicated
GitLab CI runner.

Dima





> Best
> Vincent

>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/66b1c03e-ee9a-44d0-a9f8-8305b2f085b9%40gmail.com.

[David Roe]

On Thu, Dec 10, 2020 at 6:06 AM Dima Pasechnik <dim...@gmail.com> wrote:

On Thu, Dec 10, 2020 at 10:49 AM David Roe <roed...@gmail.com> wrote:
>
> For Zulip, zulipchat.com provides free hosting for open source projects.  I'm fairly confident that we could export our history and import it to a new organization there fairly easily, if there's a consensus to do so.

I don't think we even need a consensus for this. Lack of funds is a
sufficient reason to migrate.
(apart from the history, account data is important...)

We then can point zulip.sagemath.org to the appropriate url at zulipchat.com

I've created sagemath.zulipchat.com and requested sponsorship as an open source project.   I exported our current Zulip history into a tarball as a test.  We probably want to wait until the current Sage Days is over to complete the migration so there's no disruption (the traffic is low enough that I'll probably pick a convenient time to deactivate it for a day, port the history over to zulipchat.com and then invite all our current users to the new organization).

David

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq1YEG9_8cP0RjTQmDCMKnphNXL15hqXvGwmVLhkXFt7Rg%40mail.gmail.com.

[Samuel Lelievre]

2020-12-10 11:37:41 UTC, David Roe:

>

> On Thu, Dec 10, 2020 at 6:06 AM Dima Pasechnik:

> >

> > On Thu, Dec 10, 2020 at 10:49 AM David Roe:

> > >

> > > For Zulip, zulipchat.com provides free hosting for open source projects.

> > > I'm fairly confident that we could export our history and import it to

> > > a new organization there fairly easily, if there's a consensus to do so.

> >

> > I don't think we even need a consensus for this. Lack of funds

> > is a sufficient reason to migrate.

> > (apart from the history, account data is important...)

> >

> > We then can point zulip.sagemath.org to the appropriate url at zulipchat.com

>

> I've created sagemath.zulipchat.com and requested sponsorship

> as an open source project. I exported our current Zulip history

> into a tarball as a test. We probably want to wait until the current

> Sage Days is over to complete the migration so there's no disruption

> (the traffic is low enough that I'll probably pick a convenient time

> to deactivate it for a day, port the history over to zulipchat.com

> and then invite all our current users to the new organization).

Is there a way to retain the zulip.sagemath.org url

while hosting at zulipchat.com?

[Sébastien Labbé]

On Thursday, December 10, 2020 at 11:54:18 AM UTC+1 dim...@gmail.com wrote:

On Thu, Dec 10, 2020 at 10:50 AM Vincent Delecroix
<20100.d...@gmail.com> wrote:
>
> All the services (= trac + wiki + zulip) could plausibly be hosted
> by CNRS in France. There are dedicated servers for this purpose. If
> it is accepted, the institute will support the cost.
>
> I am available to open the server and discuss with the institute.
> However, I won't have time to do any work on the migration.

How reliable are they? Needless to say, something that goes down and
stays down for a week in August (vacation!)
is not what we can agree to.

Where is the ask.sagemath.org hosted? Was it down in August in recent years?

Sébastien

 

[Samuel Lelièvre]

Le ven. 11 déc. 2020 à 08:56, Sébastien Labbé:
>
> On Thursday, December 10, 2020 at 11:54:18 AM UTC+1 Dima:
>>
>> On Thu, Dec 10, 2020 at 10:50 AM Vincent Delecroix:

>> >
>> > All the services (= trac + wiki + zulip) could plausibly be hosted
>> > by CNRS in France. There are dedicated servers for this purpose. If
>> > it is accepted, the institute will support the cost.
>> >
>> > I am available to open the server and discuss with the institute.
>> > However, I won't have time to do any work on the migration.
>>
>> How reliable are they? Needless to say, something that goes down and
>> stays down for a week in August (vacation!)
>> is not what we can agree to.
>>
>
> Where is the ask.sagemath.org hosted? Was it down in August in recent years?
>
> Sébastien

Ask Sage was hosted on Google Cloud until late June 2014,
then migrated to Ohio State University servers thanks to
Niles Johnson. It is in the process of migrating to France
thanks to Vincent Delecroix. Even though things slow down
in France in August, there's always a hotline for computing
and hosting, so I would not worry about serious down time.

[David Roe]

It took me longer than I wanted, but I've ported over all of our history and accounts to sagemath.zulipchat.com.  For security reasons the passwords haven't been migrated, so you'll need to use the password reset link (let me know if you don't remember what email you used).

I've been struggling with the google cloud interface and can't figure out how to shut down the VM that was running Zulip.  Dima, since you started the thread about costs, do you know how to do this?  Anyone else?

As for the url, I assume it should be possible to set up a redirect, but I don't have easy access to our DNS records.

David

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/05a659d7-48c6-4fd0-9ead-fa48701c2190n%40googlegroups.com.

[Dima Pasechnik]

On Thu, Jan 7, 2021 at 8:13 PM David Roe <roed...@gmail.com> wrote:
>
>
>
> On Thu, Dec 10, 2020 at 10:19 PM Samuel Lelievre <samuel....@gmail.com> wrote:
>>
>> 2020-12-10 11:37:41 UTC, David Roe:
>> >
>> > On Thu, Dec 10, 2020 at 6:06 AM Dima Pasechnik:
>> > >
>> > > On Thu, Dec 10, 2020 at 10:49 AM David Roe:
>> > > >
>> > > > For Zulip, zulipchat.com provides free hosting for open source projects.
>> > > > I'm fairly confident that we could export our history and import it to
>> > > > a new organization there fairly easily, if there's a consensus to do so.
>> > >
>> > > I don't think we even need a consensus for this. Lack of funds
>> > > is a sufficient reason to migrate.
>> > > (apart from the history, account data is important...)
>> > >
>> > > We then can point zulip.sagemath.org to the appropriate url at zulipchat.com
>> >
>> > I've created sagemath.zulipchat.com and requested sponsorship
>> > as an open source project. I exported our current Zulip history
>> > into a tarball as a test. We probably want to wait until the current
>> > Sage Days is over to complete the migration so there's no disruption
>> > (the traffic is low enough that I'll probably pick a convenient time
>> > to deactivate it for a day, port the history over to zulipchat.com
>> > and then invite all our current users to the new organization).
>>
>> Is there a way to retain the zulip.sagemath.org url
>> while hosting at zulipchat.com?
>
>
> It took me longer than I wanted, but I've ported over all of our history and accounts to sagemath.zulipchat.com. For security reasons the passwords haven't been migrated, so you'll need to use the password reset link (let me know if you don't remember what email you used).
>
> I've been struggling with the google cloud interface and can't figure out how to shut down the VM that was running Zulip. Dima, since you started the thread about costs, do you know how to do this? Anyone else?

Great, thanks.
OK, the VM is shut down now. I have not yet deleted the VM, but will
do, once we are happy that everything is properly saved etc.

>
> As for the url, I assume it should be possible to set up a redirect, but I don't have easy access to our DNS records.

Harald - can you take care of this?

Cheers
Dima

> David
>>
>> --
>> You received this message because you are subscribed to the Google Groups "sage-devel" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/05a659d7-48c6-4fd0-9ead-fa48701c2190n%40googlegroups.com.
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAChs6_nBJ8qXH8nrVv_%3DTS3fL9G7TztiWBtfXSqJY5jUL9n7XA%40mail.gmail.com.

[Harald Schilly]

On Thu, Jan 7, 2021 at 9:23 PM Dima Pasechnik <dim...@gmail.com> wrote:
> Harald - can you take care of this?
>

Uhm, what's happening? Could someone please summarize this for me?

[David Roe]

zulip.sagemath.org used to point to a google virtual machine.  We'd like it to redirect to sagemath.zulipchat.org so that people looking for the Zulip server are sent to the right place.

David

 

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAGG4CB5oiSVA3W3AJWfosLuMuzEKPaoHZ_DWNS58H%3DFgast98w%40mail.gmail.com.

[Isuru Fernando]

It should be sagemath.zulipchat.com right? (Instead of .org)

Isuru

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAChs6_mN8PRRt2Ot7YcHqcZGLXPoPcVS0_R_QdjCVYpZHpi5Ng%40mail.gmail.com.

[David Roe]

On Thu, Jan 7, 2021 at 3:49 PM Isuru Fernando <isu...@gmail.com> wrote:

It should be sagemath.zulipchat.com right? (Instead of .org)

Yes, sorry for the typo!

David

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CA%2B01voOy5SXXkyQeqMB-AxeGMXEv5MN%2Bj3FBdYB1DihHHh0inQ%40mail.gmail.com.

[Tobia...@gmx.de]

For what's worth, + 1 for migrating to github.

The interface is cleaner, it has many more features and integrations, and is more active which could attract more contributions. There are a few scripts/tools that allow to migrate from trac to github. But most of them are unmaintained for a few years already, so I'm not sure if they still work (which should be taken as a sign that one should migrate sooner than later).

[Samuel Lelièvre]

Regarding Zulip, the new hosting has some limitations.

For example, trying to access

https://sagemath.zulipchat.com/#narrow/stream/271072-padics/topic/p-adic.20help.20request.20on.20sage-support

the following information gets displayed:

Some older messages are unavailable. Upgrade
your organization to access your full message history.

Not sure whether this means older messages are stored
but not displayed, or not stored at all. --Samuel

[Dima Pasechnik]

On Mon, Jan 18, 2021 at 3:52 PM Samuel Lelièvre
<samuel....@gmail.com> wrote:
>
> Regarding Zulip, the new hosting has some limitations.
>
> For example, trying to access
>
> https://sagemath.zulipchat.com/#narrow/stream/271072-padics/topic/p-adic.20help.20request.20on.20sage-support
>
> the following information gets displayed:
>
> Some older messages are unavailable. Upgrade
> your organization to access your full message history.
>
> Not sure whether this means older messages are stored

This might have been David's oversight as he moved GCE-hosted zulip to
the new, free, hosting.
I've asked him to look this up.

> but not displayed, or not stored at all. --Samuel
>

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAEcArF3h%3DQduPMD41G9iY2gMYtBb-vs6S19y_BjPXaWtARmzqA%40mail.gmail.com.

[David Roe]

There was an oversight in upgrading us to a sponsored account as an open source organization.  They've done so now; let me know if you observe any similar issues going forward.
David

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq3ceNRJoTcEWUCQBTO0cwGk_CRnbwGqBArsLnxM1w7Ctg%40mail.gmail.com.

[E. Madison Bray]

On Tue, Jan 12, 2021 at 11:33 PM Tobia...@gmx.de <Tobia...@gmx.de> wrote:
>
>
> For what's worth, + 1 for migrating to github.
>
> The interface is cleaner, it has many more features and integrations, and is more active which could attract more contributions. There are a few scripts/tools that allow to migrate from trac to github. But most of them are unmaintained for a few years already, so I'm not sure if they still work (which should be taken as a sign that one should migrate sooner than later).

In 2019 Julian Rüth and I, with the help of some others, already put
in some effort to set up an organization for SageMath on GitLab:
https://gitlab.com/sagemath

Between GitHub and GitLab, we felt that the latter would be more
acceptable to the broader Sage community. We also implemented a bot
that can mirror GitLab merge requests as Trac tickets, though it's
been in need of troubleshooting for a while.

This was also done before the advent of GitHub Actions, and the
ability to provide custom CI runners for GitLab Pipelines seemed
advantageous, since we could maintain our own fleet of runners, be it
on Sage developers' personal machines (if they are generous enough to
host them) or any conceivable constellation of cloud computing
platforms.

In practice this has gained little traction, in part due to lack of
advertising. The GitLab Runner solution also proved a bit troublesome
to maintain, as it required some constant attention to make sure there
were always working runners available. I tried to keep that up for a
while myself, but have had other obligations.

In the meantime Matthias and others have been doing really interesting
things with GitHub Actions for our CI. For the time being GitHub is
being *very* generous with computing time available to open source
projects. Though I fear it's only a matter of time before Microsoft's
investors come banging on the door, and they start putting in bigger
limits for free users (as happened with Travis CI).

I would still prefer the GitLab approach for a myriad of reasons, or a
hybrid approach at least for the GitHub Actions stuff. It just needs
to be better advertised, and there needs to be better instructions for
where users and potential developers should go to open issues.

As for the wiki I've always been in favor of dropping Moin Wiki and
migrating the existing wiki pages to Trac (or to GitLab). Someone
just has to do it, as is always the problem.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/e2c237d2-b8aa-4002-8fb4-edeaf03a8d3fn%40googlegroups.com.

[Dima Pasechnik]

On Wed, Mar 10, 2021 at 4:00 PM E. Madison Bray <erik....@gmail.com> wrote:
>
> On Tue, Jan 12, 2021 at 11:33 PM Tobia...@gmx.de <Tobia...@gmx.de> wrote:
> >
> >
> > For what's worth, + 1 for migrating to github.
> >
> > The interface is cleaner, it has many more features and integrations, and is more active which could attract more contributions. There are a few scripts/tools that allow to migrate from trac to github. But most of them are unmaintained for a few years already, so I'm not sure if they still work (which should be taken as a sign that one should migrate sooner than later).
>
> In 2019 Julian Rüth and I, with the help of some others, already put
> in some effort to set up an organization for SageMath on GitLab:
> https://gitlab.com/sagemath
>
> Between GitHub and GitLab, we felt that the latter would be more
> acceptable to the broader Sage community. We also implemented a bot
> that can mirror GitLab merge requests as Trac tickets, though it's
> been in need of troubleshooting for a while.
>
> This was also done before the advent of GitHub Actions, and the
> ability to provide custom CI runners for GitLab Pipelines seemed
> advantageous, since we could maintain our own fleet of runners, be it
> on Sage developers' personal machines (if they are generous enough to
> host them) or any conceivable constellation of cloud computing
> platforms.
>
> In practice this has gained little traction, in part due to lack of
> advertising. The GitLab Runner solution also proved a bit troublesome
> to maintain, as it required some constant attention to make sure there
> were always working runners available. I tried to keep that up for a
> while myself, but have had other obligations.

I think it should be mentioned that GitLab has an analog of GitHub Actions,
and the difference is that its runners may be self-hosted, or provided
by GitLab.
E.g. see https://gitlab.com/sagemath/dev/trac/-/pipelines/266731297

> You received this message because you are subscribed to a topic in the Google Groups "sage-devel" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-devel/ayOL8_bzOfk/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to sage-devel+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAOTD34bSQLz-JBbc5P9WOHYxFUFdLt20A24yTQ48ySfaOiL2jQ%40mail.gmail.com.

[Dima Pasechnik]

I just tried to switch to a "community" runner, and got an error which
is probably
obvious to people versed in Docker:

https://gitlab.com/sagemath/dev/trac/-/jobs/1089520433

[E. Madison Bray]

I think it might be because the Docker builds have been otherwise not
working for a while (due to lack of reliable runners). So a more
recent "build-from-clean" job is needed. These jobs are run when
develop/master are updated as well as on tags. Whereas
"built-from-latest" is run on branches for tickets. It tries to build
the branch on top of the "latest" Docker image e.g. for develop. But
the last one that built successfully is too old, and so trying to make
the diff between that ticket and the version of develop it's based on
fails. Hence the message:

"Could not find commit fbca269f627bf6a8bc6f0a611ed7e26260ebc994 in
your local Git history. Please merge in the latest built develop
branch to fix this: git fetch trac && git merge
fbca269f627bf6a8bc6f0a611ed7e26260ebc994"

But for the automated CI that's not a very useful message...

I know Matthias has done some impressive things to get around GitHub
Actions' time limit on jobs by breaking the build up into "stages"
that can be split across multiple jobs. I see no reason that couldn't
work with GitLab as well.

But it would still be better to have our own fleet of runners--they
would be faster, and we could test on more different custom hardware
configurations. The problem is that this is at a minimum a part-time
job...

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq1RV5bEmq3N_FP-5Eq%3D05d7%2BegZZ6yqp9%2Bs-B0Wbb9y-A%40mail.gmail.com.

[E. Madison Bray]

Well looks like I need to correct the record a bit. Perhaps I've been
a bit too sanguine about the state of the GitLab builds. In fact, the
latest develop commit, 9.3beta8, built quite successfully:
https://gitlab.com/sagemath/sage/-/pipelines/266734885

And it ran on one of the fleet of runners I've been maintaining here
at Paris-Saclay, which I haven't touched in months. So I guess it's
still working after all ^^; Ever since I set this up I had been
having a problem with runners randomly erroring out, and not being
deleted correctly when they do. I have tried many times to fix it to
no avail, and I kind of gave up for a while. I assumed eventually
this caused things to grind to a halt, but apparently not.

Knowing that it's still working at least somewhat gives me motivation
to try again to investigate the problem with the erroring runners and
see if it can't be fixed. Maybe an upgrade of the gitlab-runner
controller is in order...

[Frédéric Chapoton]

Erik, did you stop the Orsay runners for gitlab ? It seems that the docker build there for 9.3.b9 is stuck by lack of runners.

https://gitlab.com/sagemath/sage/-/pipelines

Frédéric

[Dima Pasechnik]

I am resurrecting this thread, as in addition of trac continuing to eat up funds (at a rate of over US$ 10 per day at the moment), it has gotten increasingly broken. In particular, in the last 2 weeks no new developers can really join the project, as there is no normal* way to add new ssh keys into trac accounts, and it's not possible to push/pull with "new" github ssh keys, i.e. keys that were not already "known" to trac, i.e. added to the trac store of ssh keys before the last breakage happened.

As far as funding is concerned, attempts to bring trac to a "free" hosting stalled (see earlier messages in this thread).

A further longer term issue is that trac software is basically on life support, and it's only matter of time it will become totally obsolete. 

Such a move will allow a considerable simplification of our devops, and free up quite a bit of developer time

to do interesting work rather than messing around with semi-obsolete stuff such as trac, gitolite, etc. 

Importantly, Volker, the release manager, is willing to proceed with the move.

Also, various Sage upstream (and downstream) projects have moved away from trac to github, e.g. Cython, or away from another system to github, e.g. CPython, GAP, jupyter, etc...

There is a trac ticket to manage the proposed move, 

https://trac.sagemath.org/ticket/30363 tentatively set for Sage 9.8.

I've conducted few experiments with a tool to import trac sites to github: https://github.com/svigerske/trac-to-github, which in particular allows to import trac tickets as github issues; a result of running it on few tickets

may be inspected here: https://github.com/dimpase/trac_to_gh/issues?q=is%3Aissue+is%3Aclosed

(Here issues 1-10 correspond to trac tickets one to one :-))

Further work on trac-to-github will be needed, in particular to properly link branches in our git tree, but it's doable,

and we have volunteers to do it.

We'd like to hear about serious objections to the move, if any.  

*) normal - i.e. using trac interface; we (probably) still have a way to modify the repository of ssh keys used by trac manually.

[Dima Pasechnik]

On Fri, Sep 9, 2022 at 11:15 AM Frédéric Chapoton <fchap...@gmail.com> wrote:
>
> Dear sage developers and maintainers,
>
> Whereas I agree that we currently have two issues, I do not agree on the necessity to switch to github and certainly not urgently.

it is a disaster that new people can't come aboard easily. It really is urgent.
A convoluted system to get new developers onboard and contributing is
a very bad omen for open-source projects, it really is.

E.g. try to contribute to something like OpenBSD - I'd sure most
potentail contributors run away screaming,
upon learning that they must use CVS and e-mail patches around for approval.

>
> * The first issue is the cost of google compute engine. This is under investigation and can be lowered by creating a new project. This should be do-able and could save us 3 $ per day.

but this is far from free, still, and hosting prices are to go with
the energy prices, up and up.
It's really spending money on a questionable luxury, instead of
something useful.

> * The second issue is about new users entering new ssh keys. There is hope to fix that and in the mean-time one could ask new users to send sshkeys to some of us.
>
> My own preference would be to go on using trac, for some years, as this is serving us quite well. We should not change this for superficial and temporary reasons.

the reasons are not supreficial, in particular, trac+gitolite software
is obsolete.
I cannot imagine a new project that would choose it as a platform.

>
> The serious reasons that I see are : money and the futre of the trac software itself.
>
> In my opinion, money is the only serious issue, and I would like to see trac heberged by some university. There are already several services in France, so another country would be better. Germany ? Somebody must step forward.
>
> About the trac software, it now has a python3-compatible version, available on most linux distributions. We should aim to use that. Once done, the situation will be stable.

Why do you think so? The bus factors of trac and gitolite software are
very, very small.
(https://en.wikipedia.org/wiki/Bus_factor)
As well as the bus factor for our trac instance.

>
> As a side matter, it seems to me that gitlab is much more in the spirit of open source software. We should rather not bow under the power of large private companies.
Let's not get into this argument. I don't see how paying Google's
adware criminals US$4000 per year is more ethical than moving over to
GitHub (which, by the way, gives us a bit of money,
via GitHub sponsors system :-)).
Besides, moving from GitHub to GitLab is rather easy, compared to move
from trac to Git**b.

Dima




>
> Frédéric
>
>
> Le ven. 9 sept. 2022 à 11:55, Dima Pasechnik <dim...@gmail.com> a écrit :

>> --
>> You received this message because you are subscribed to the Google
>> Groups "sage-devel" group.
>> To unsubscribe from this group and stop receiving emails from it, send
>> an email to sage-devel+...@googlegroups.com.
>> To view this discussion on the web visit

>> https://groups.google.com/d/msgid/sage-devel/142912ca-a226-47a7-8ea4-6afe5711376fn%40googlegroups.com.
>>
>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups "sagemath-admins" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to sagemath-admi...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/sagemath-admins/CAAWYfq2mC7yHKP%2B%3DGzvdAo0BrYiv-CMJ3r2xbMfBA-_8Jr-k8Q%40mail.gmail.com.
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups "sagemath-admins" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sagemath-admi...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sagemath-admins/CAL7VZwDQoUVKLrHazy2-%2BzW6LfdJ-zSavzgeBqU4F%3Db59Ub9vQ%40mail.gmail.com.

[Thierry]

Hi,

let me forward the email of Frédéric as a whole, so that the thread remains
complete.

----- Forwarded message from Frédéric Chapoton <fchap...@gmail.com> -----

Date: Fri, 9 Sep 2022 12:15:25 +0200
From: Frédéric Chapoton
To: sagemat...@googlegroups.com
Subject: Re: [sagemath-admins] Fwd: [sage-devel] Re: incremental migration

to github? [prompted by FUNDING issues!!!] + general flakiness of trac

Dear sage developers and maintainers,

Whereas I agree that we currently have two issues, I do not agree on the
necessity to switch to github and certainly not urgently.

* The first issue is the cost of google compute engine. This is under
investigation and can be lowered by creating a new project. This should be
do-able and could save us 3 $ per day.

* The second issue is about new users entering new ssh keys. There is hope
to fix that and in the mean-time one could ask new users to send sshkeys to
some of us.

My own preference would be to go on using trac, for some years, as this is
serving us quite well. We should not change this for superficial and
temporary reasons.

The serious reasons that I see are : money and the futre of the trac
software itself.

In my opinion, money is the only serious issue, and I would like to see
trac heberged by some university. There are already several services in
France, so another country would be better. Germany ? Somebody must step
forward.

About the trac software, it now has a python3-compatible version, available
on most linux distributions. We should aim to use that. Once done, the
situation will be stable.

As a side matter, it seems to me that gitlab is much more in the spirit of
open source software. We should rather not bow under the power of large
private companies.

Frédéric

-----

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq15a%2BnkQBzvZ7FKb2uG%2B01AdP%2BE3nxnVqL%3DbbmgdBb-pg%40mail.gmail.com.

[Kwankyu Lee]

the futre of the trac software itself.

According to their developer mailing list

https://groups.google.com/g/trac-dev

the future of trac seems not so ominous to me.

Perhaps publishing our fork of trac with customized plugins (?) to sagemath github repo may help increase  the bus factor about our own trac

[Dima Pasechnik]

On Fri, 9 Sep 2022, 15:26 Kwankyu Lee, <ekwa...@gmail.com> wrote:

the futre of the trac software itself.

According to their developer mailing list

https://groups.google.com/g/trac-dev

the future of trac seems not so ominous to me.

18 posts (in half a dozen threads) since beginning of 2022 does not look healthy to me.

And on their users group one sees that it's populated by people who have projects running SVN and  mercurial - VCSs not supported by GitHub.

Perhaps publishing our fork of trac with customized plugins (?) to sagemath github repo may help increase  the bus factor about our own trac

maintaining our trac does not bring me (or anyone, I suppose) any joy.

If your objection to the move is strong - you are welcome to take over. 

Fred, Jan, and me spent together perhaps 40 working hours on this totally thankless task in the last 2 weeks. We could instead have done something useful for Sage proper instead.

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/c5050c0a-3bcb-4f01-af40-ff3e7a8cb393n%40googlegroups.com.

[William Stein]

Hi,

By not using GitHub we are losing many potential contributors to Sage.
GitHub is by far the most popular site for hosting of open source
projects, and potential Sage developers are likely to be familiar with
GitHub. Not using GitHub adds a huge barrier to entry for Sage
development.

For example, I don't contribute to Sage anymore because the barrier to
entry via trac is too high. I contributed (and had accepted) a little
pull request yesterday to numpy **because** the barrier was so low.

-- William

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq3jS%2B--h_DUyDk5WAZkiXVy_zLKbiPk9dUhnrqtKNSP6A%40mail.gmail.com.



--
William (http://wstein.org)

[Thierry]

Hi,

"[prompted by FUNDING issues!!!]" ???

Back in 2016, when cloud hosting was imposed over academic hosting,
William promised "to pay for it indefinitely", see
https://groups.google.com/g/sage-devel/c/ed_ya-d-k_E/m/jYoR6opODAAJ

If only for this reason, there is no funding issue.

The fact is that the the magic cloud ideology turned into a disaster,
years after years, from askbot in 2014 (thanks again to Niles and OSU
sysadmins for their involvement in its academic hosting during six
years !), to the wiki a few months ago, and now trac+git.

I have been involved in rescuing those services, and putting them back
in a safe place. What do the facts tell us about academic hosting, in
2022 ?

- askbot is academically hosted and works pretty well.
Fees to the Sage community: 0$ per year

- patchbot server is academically hosted and works pretty well.
Fees to the Sage community: 0$ per year

- I bet that most patchbot clients are run from academic desktops.
Fees to the Sage community: 0$ per year

- wiki is academically hosted and works pretty well.
Fees to the Sage community: 0$ per year

- the backup server (which currently backs up askbot and wiki) is
academically hosted and works pretty well.
Fees to the Sage community: 0$ per year

- most download mirrors are academically hosted and work pretty well.
Fees to the Sage community: 0$ per year

For this reason also, and given the number of universities around the
world, there is no funding issue. The rest is FUD.

Ciao,
Thierry

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/142912ca-a226-47a7-8ea4-6afe5711376fn%40googlegroups.com.

[Matthias Koeppe]

Strong +1 on moving to GitHub as soon as possible. 

Very important to convert the complete Trac ticket history to GitHub issues/PRs.

[Matthias Koeppe]

On Friday, September 9, 2022 at 7:48:01 AM UTC-7 wst...@gmail.com wrote:

By not using GitHub we are losing many potential contributors to Sage.
GitHub is by far the most popular site for hosting of open source
projects, and potential Sage developers are likely to be familiar with
GitHub. Not using GitHub adds a huge barrier to entry for Sage
development.

Indeed.

Lowering these barriers is extremely important to keep the Sage project alive.

[Matthias Koeppe]

On Friday, September 9, 2022 at 7:48:07 AM UTC-7 Thierry (sage-googlesucks@xxx) wrote:

- wiki is academically hosted and works pretty well.
Fees to the Sage community: 0$ per year

Not really relevant for this thread, but: 

The wiki is dead. It has zero community and its only remaining purpose is to keep an archive of Sage days activities. 

https://trac.sagemath.org/ticket/33725

[John Cremona]

To me, as a contributor of code to Sage who has not contributed at all
to the backend support, it seems that there is a clear majority in
favour of moving to github. As an ordinary developer I would be very
happy with that.

It looks to me as if Frédéric's main issue with github is his final
point "...We should rather not bow under the power of large
private companies.". I don't know enough about gitlab to know if it
is a sensible alternative, but I myself have no problem with using
github for this, as I do for just about everything else.

John

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/20220909141000.GA18932%40metelu.net.

[Dima Pasechnik]

On Fri, 9 Sep 2022, 15:48 Thierry, <sage-goo...@lma.metelu.net> wrote:

Hi,

"[prompted by FUNDING issues!!!]" ???

Back in 2016, when cloud hosting was imposed over academic hosting,
William promised "to pay for it indefinitely", see
https://groups.google.com/g/sage-devel/c/ed_ya-d-k_E/m/jYoR6opODAAJ

If only for this reason, there is no funding issue.

this is bus factor 1 funding. Not good at all.

The fact is that the the magic cloud ideology turned into a disaster,
years after years, from askbot in 2014 (thanks again to Niles and OSU
sysadmins for their involvement in its academic hosting during six
years !), to the wiki a few months ago, and now trac+git.

I have been involved in rescuing those services, and putting them back
in a safe place.

OK, so most of these services therefore have bus factor 1.

What do the facts tell us about academic hosting, in
2022 ?

Several people promised to look for academic hosting for trac. Nothing came out of it.

- askbot is academically hosted and works pretty well.

for some value of "well". Last time I checked its logins were screwed up, and dates of posts too.

  Fees to the Sage community: 0$ per year

- patchbot server is academically hosted and works pretty well.

you must be kidding here. It's often down, the trac integration is gone.

Most of Sage CI happens on GitHub Actions, anyway.

  Fees to the Sage community: 0$ per year

- I bet that most patchbot clients are run from academic desktops.
  Fees to the Sage community: 0$ per year

- wiki is academically hosted and works pretty well.

bus factor 1 service.

  Fees to the Sage community: 0$ per year

- the backup server (which currently backs up askbot and wiki) is
  academically hosted and works pretty well.

bus factor 1 service.

 

  Fees to the Sage community: 0$ per year

- most download mirrors are academically hosted and work pretty well.

  Fees to the Sage community: 0$ per year

For this reason also, and given the number of universities around the
world, there is no funding issue. The rest is FUD.

FUD? No, me and other people pointed years ago that trac is about to break, and now it is broken. It's not FUD, it's bad dream that came true, inevitably.

The very fact that there are half a dozen shabby and differently run services with bus factor 1 needed to support Sage (and you didn't mention Google groups, web hosting, DNS, email for sagemath.org - the latter is bus factor 1 too) indicates it is very fragile and breaking all the time.

And for a newcomer to wrap their head around all them is hardly possible - as William (and I) wrote, it's  a huge barrier to enter Sage development.

Hence an urgent need for consolidation. 

Dima

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/20220909144801.GA19110%40metelu.net.

[Dima Pasechnik]

On Fri, Sep 9, 2022 at 4:06 PM John Cremona <john.c...@gmail.com> wrote:
>
> To me, as a contributor of code to Sage who has not contributed at all
> to the backend support, it seems that there is a clear majority in
> favour of moving to github. As an ordinary developer I would be very
> happy with that.
>
> It looks to me as if Frédéric's main issue with github is his final
> point "...We should rather not bow under the power of large
> private companies.".

my reply to Frederic got posted ahead of his post, as he didn't cc to
sage-devel.
I wrote there that I don't see how to consolidate the latter wish with
us currently paying a huge private
corporation, and a rather evil one, a figure of about US$4000 p.a . for hosting.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAD0p0K78WEmRAfC7bPddU2Y1denCS3RbtVdDcOV7zt0Eg9eRsw%40mail.gmail.com.

[Thierry]

On Fri, Sep 09, 2022 at 04:07:48PM +0100, Dima Pasechnik wrote:
[...]

>
> Several people promised to look for academic hosting for trac. Nothing
> came out of it.

Please do not rewrite history, as it adds violence to violence. There
have indeed been offers for academic hosting, the VM were ready, but the
cloud hosting was enforced, and you was promoting this.

Ciao,
Thierry

[Dima Pasechnik]

I don't recall any concrete offer like this. If I recall correctly,
the problem of accessing VMs in French
universities for outsiders have never been resolved in a meaningful
way, leading to an unacceptably small
for the project bus factor.

By the way, another proof of my small bus factor claim is here:
https://trac.sagemath.org/ticket/33725#comment:30
Apparently you want to do everything your way, and your way only.
You are asked to provide the wiki contents in a suitable for import
form, and you just pretend you didn't see the request?


Dima



>
> Ciao,
> Thierry

>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/20220909185835.GA20878%40metelu.net.

[Travis Scrimshaw]

I really dislike Github's decentralized approach with PR and having to have separate clones of the repo within each user. My understanding is if two people have different fixes, then they individually submit PRs that are not explicitly linked with each other, much less with a specific bug report issue. It is not so uncommon that people end up working on competing proposals or on the same branch, both of which are not natural using Githubs PR system (I am not even entirely sure how I should do this either; Do I have to clone from their repo? Is it the same remote? Where is the documentation on this? Can I still access the branch if someone deletes their account?). How do other projects at the scale of Sage deal with this?

Honestly, I doubt we are loosing that many quality contributors by not using Github's system since we can have people with GH login's to access trac. Getting the login credentials was the biggest barrier; everything else is mostly straightforward and based on very simple git commands.

Right now, I find there are way too many practical questions and barriers for how we develop that make moving to Git**b a much bigger pain that people will think it is. Some of this could be my lack of experience with Github (and a bit of "get off my lawn with your facegrams"), but right now I feel like the move to Github is like starting a car to go to the party in the next town without learning how to drive, much less knowing the route.

Best,

Travis

[Matthias Koeppe]

On Friday, September 9, 2022 at 9:34:16 PM UTC-7 Travis Scrimshaw wrote:

I really dislike Github's decentralized approach with PR and having to have separate clones of the repo within each user. My understanding is if two people have different fixes, then they individually submit PRs that are not explicitly linked with each other, much less with a specific bug report issue.

In the PR you would include a comment such as "Fixes #1234", which links it to an Issue (bug report).

Yes, there can be multiple competing PRs in order solve one ticket. Better than edit wars on a Trac ticket.

[Dima Pasechnik]

On Sat, 10 Sep 2022, 05:34 'Travis Scrimshaw' via sage-devel, <sage-...@googlegroups.com> wrote:

I really dislike Github's decentralized approach with PR and having to have separate clones of the repo within each user. My understanding is if two people have different fixes, then they individually submit PRs that are not explicitly linked with each other, much less with a specific bug report issue. It is not so uncommon that people end up working on competing proposals or on the same branch, both of which are not natural using Githubs PR system (I am not even entirely sure how I should do this either; Do I have to clone from their repo? Is it the same remote? Where is the documentation on this? Can I still access the branch if someone deletes their account?). How do other projects at the scale of Sage deal with this?

It seems you never reviewed a non-trivial PR on GitHub, no?

Well, you can go and have a look how e.g. CPython does it, or SciPy. 

The PR process and the underlying structure is very well-documented.

A PR creates a branch in the git repo of the "main"

project. This is really not much different from what we do on trac, except that the branch name

satisfied different rules.

Have a look at https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

E.g. using https://github.com/scipy/scipy/pull/14786 as an example, the underlying branch in the repo of scipy is

named pull/14786. 

To examine it locally on your machine you can use plain git to fetch it, or you can use GitHub's

tool called cli (its functionality is close to what git-trac does, not sure if you use or used git-trac, though).

It stays there even if the user GitHub account is closed (the latter triggers an automatic closure of the PR, but the underlying

branch remains in the repo, it can be worked on just the same using git)

Besides, nobody forces you to use a PR to review something. You can open an issue, and ask for a specific branch somewhere (in a fork, or even in a completely unrelated git branch hosted outside of GitHub) to be reviewed, and only do a PR once the branch in question is stabilized.

Honestly, I doubt we are loosing that many quality contributors by not using Github's system since we can have people with GH login's to access trac.

except that this is broken now, (along with automatic management of ssh keys in general) and one needs

a tricky manual process (modifying a special gitolite git repo used by trac) by one of trac admins

to add/modify ssh keys.

(and I am not even sure whether this manual process does not lead to an eventual breakage of the ssh keystore, it's all tied up

with trac in a rather arcane way).

Getting the login credentials was the biggest barrier; everything else is mostly straightforward and based on very simple git commands.  

Right now, I find there are way too many practical questions and barriers for how we develop that make moving to Git**b a much bigger pain that people will think it is.

 

Travis, many people nowadays never used git without GitHub or GitLab. For such a person it's a major pain to 

learn our workflow.

Some of this could be my lack of experience with Github (and a bit of "get off my lawn with your facegrams"), but right now I feel like the move to Github is like starting a car to go to the party in the next town without learning how to drive, much less knowing the route.

What's the alternative? 

Would you like to contribute your time to do our time-consuming trac etc devops?

Anyway, trac is obsolete software used mainly to maintain legacy projects which don't even use git,

and we'd have to move sooner or later to something that is robust and maintained, and is not an endless

timesink to keep working. And trac is only a part of the general Sage devops nightmare, created largerly by

sticking to semi-obsolete tools for too long.

I think you have pointed at exactly one question - you didn't know how to deal with git branches of GitHub's PR.

I hope you see now that it's trivial. Do you have any other objections?

Dima

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/4e8fe22b-6538-4a1b-a2c4-620127934e48n%40googlegroups.com.

[Dima Pasechnik]

On Sat, 10 Sep 2022, 05:48 Matthias Koeppe, <matthia...@gmail.com> wrote:

On Friday, September 9, 2022 at 9:34:16 PM UTC-7 Travis Scrimshaw wrote:

I really dislike Github's decentralized approach with PR and having to have separate clones of the repo within each user. My understanding is if two people have different fixes, then they individually submit PRs that are not explicitly linked with each other, much less with a specific bug report issue.

In the PR you would include a comment such as "Fixes #1234", which links it to an Issue (bug report).

and this Issue will then automatically get a comment/mention linking to the PR, no manual intervention in the Issue is needed.

Yes, there can be multiple competing PRs in order solve one ticket. Better than edit wars on a Trac ticket.

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/2548301a-19d8-4ab5-ab1c-84f3fdcf5bbcn%40googlegroups.com.

[Vincent Delecroix]

Hello,

I am in the same mood as Travis : if I was to consider a move to
github I would like to have a clear and complete overview of the
changes in the workflow (how do we set ticket dependencies? how
reviews will work? management of releases? etc). For me the discussion
in this thread is very premature as there is no proposal of this sort.
The "moving to github" does not specify how things will work.

I see clear advantages of moving to github. The first being trac
maintenance and a second example that William mentioned is that it
might lower the barrier for newcomers. But there are also plenty of
reasons not to migrate. The first one I think is that we might loose
active developers. Let me recall that the move from mercurial to git
some 10 years ago made many active developers quit.

So I would like to propose that ticket #30363 instead of being
technical (ie how do we do the move) explains
- a concrete proposal for a github workflow (there might be several of them)
- list the pros
- list the cons
Then we could proceed with a reasonable discussion on whether we will
do such move.

Ideally (if we had illimited developer time) I would like to encourage
the possibility of having both trac and github. If I recall correctly,
it was possible to authenticate to trac with github account and make
PR on github that automatically transformed into a ticket on trac.

Best
Vincent

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq3rdAfkj%3DkDLJa0TxMPxLHz4O2_stffO8F-PyiGsO0ZqQ%40mail.gmail.com.

[Aram Dermenjian]

Hi all,

I'm only an occasional developer with sage, but I wanted to also bring up one (minor) positive for moving to github which is less technical. Moving to github would be a motivating factor for people to contribute to sagemath as it gives a pathway to work in industry (programming) if they chose to leave academia. In particular, github shows contributions made by a user to various projects. This could be a valuable tool for someone on the job market who would prefer (or has no choice) to move to industry and get a programming job. Sure, one can already currently say that one develops for sagemath, but by putting it on github it creates a unified location for future employers to look. As mentioned, this is just a minor positive, but could be a motivating factor for some.

Also, I concur with Vincent in that we should have somewhere with a list of pros/cons for either side in some kind of list fashion as currently it's a little hard to keep track of whether people are making the same arguments or new arguments.

Kindly,

Aram Dermenjian

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAGEwAAkYpD4fh%3D_QvuwmZoRSgrF6AgUeg1WWvbXMhX-DJ5hanA%40mail.gmail.com.

[Dima Pasechnik]

On Sat, 10 Sep 2022, 09:35 Vincent Delecroix, <20100.d...@gmail.com> wrote:

Hello,

I am in the same mood as Travis : if I was to consider a move to
github I would like to have a clear and complete overview of the
changes in the workflow (how do we set ticket dependencies? how
reviews will work? management of releases? etc). For me the discussion
in this thread is very premature as there is no proposal of this sort.
The "moving to github" does not specify how things will work.

I am sure we can work out reasonable workflows 

once we have the infrastructure (in particular tickets moved to github's issues) in place. There need not be one single model; as you know, it's perfectly possible to have named branches on GitHub, and use them instead of PRs.

And/or we can merge PRs into integration branches, which are then used by Volker for beta and other releases. 

Many projects merge PRs directly into the master branch, though. We can try this, see if it works.

There are many options and possibilities, and choosing one in advance is not very meaningful.

I don't want to work on a detailed description of a workflow until we have in principle agreement on the move. 

Doing such a detailed proposal without a prior agreement is akin to writing a grant application, with a high probability it is rejected, and all the effort for nothing. Been there many times, and no, thanks, I don't like this model of moving things forward, it is too wasteful.

I see clear advantages of moving to github. The first being trac
maintenance and a second example that William mentioned is that it
might lower the barrier for newcomers. But there are also plenty of
reasons not to migrate. The first one I think is that we might loose
active developers. Let me recall that the move from mercurial to git
some 10 years ago made many active developers quit.

I don't recall who quit just due to unwillingness to learn git, not for other, unrelated to the move reasons, using the move as a pretext to get out.

Moving from a patch-quilt model to a proper system with branches etc was basically a necessity then, there was too much bitrot going on.

Moving away from trac is basically a necessity now, as our devops are breaking down for some time, and it only gets worse.

So I would like to propose that ticket #30363 instead of being
technical (ie how do we do the move) explains
- a concrete proposal for a github workflow (there might be several of them)
- list the pros
- list the cons
Then we could proceed with a reasonable discussion on whether we will
do such move.

Ideally (if we had illimited developer time) I would like to encourage
the possibility of having both trac and github. If I recall correctly,
it was possible to authenticate to trac with github account and make
PR on github

this is semi-broken now - new accounts have to be added manually, and we don't know how to fix this part of trac/gitolite.

that automatically transformed into a ticket on trac.

this was never in place. There is something like this gitlab-based, but it's not in a good shape now, at all.

Dima

Best
Vincent

On Sat, 10 Sept 2022 at 09:54, Dima Pasechnik <dim...@gmail.com> wrote:
>
>
>
> On Sat, 10 Sep 2022, 05:48 Matthias Koeppe, <matthia...@gmail.com> wrote:
>>
>> On Friday, September 9, 2022 at 9:34:16 PM UTC-7 Travis Scrimshaw wrote:
>>>
>>> I really dislike Github's decentralized approach with PR and having to have separate clones of the repo within each user. My understanding is if two people have different fixes, then they individually submit PRs that are not explicitly linked with each other, much less with a specific bug report issue.
>>
>>
>> In the PR you would include a comment such as "Fixes #1234", which links it to an Issue (bug report).
>
>
> and this Issue will then automatically get a comment/mention linking to the PR, no manual intervention in the Issue is needed.
>
>
>> Yes, there can be multiple competing PRs in order solve one ticket. Better than edit wars on a Trac ticket.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups "sage-devel" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/2548301a-19d8-4ab5-ab1c-84f3fdcf5bbcn%40googlegroups.com.
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq3rdAfkj%3DkDLJa0TxMPxLHz4O2_stffO8F-PyiGsO0ZqQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAGEwAAkYpD4fh%3D_QvuwmZoRSgrF6AgUeg1WWvbXMhX-DJ5hanA%40mail.gmail.com.

[Jan Groenewald]

Hi

https://trac.edgewall.org/wiki/TracUsers

In 2020, 21, 22, _only_ Edgewall & Trac-hacks (the trac people), and sagemath are listed as trac users bothering to update their details.

I don't know how accurate the below articles are, but found them intersting to read. Would be interested in

comments from anyone who knows where they are inaccurate:

https://kinsta.com/blog/gitlab-vs-github/#gitlab-vs-github-key-differences

https://www.zdnet.com/article/github-vs-gitlab-the-key-differences/

https://www.incredibuild.com/blog/gitlab-vs-github-comparison (Flow and CI sections)

https://radixweb.com/blog/github-vs-gitlab#difference

I can't tell which workflow is better for sagemath. Remember feature parity betweenm them will also change every few months!

Also see what they say about themselves/each other:

https://about.gitlab.com/devops-tools/github-vs-gitlab/

https://resources.github.com/devops/tools/compare/

It does look like github might offer more contributors. (Of high and low quality? Perhaps the slight hurdle to logging in to gitlab the first time is a good thing...)

I'd be happy to assist with self-hosting (so gitlab) and perhaps sponsor a container (even a separate machine for CI) but I think the sagemath project would probably be better off on git**b. I'm also hesitant to invest my time in learning trac/gitolite. The bus problem Dima mentioned with self-hosting is valid. But nice to have it as an option for a quick exit (from either git**b) when whatever happens to the companies...

I don't think the move is so urgent though, but it should probably happen in the next year or so.

For the lulz:

https://imgs.xkcd.com/comics/git_2x.png

Regards,

Jan

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAKQMtir03oQsrdJb06_%2Bi4cdVDtxAdZ8pszcbw3C0juqi87mXA%40mail.gmail.com.

--

  .~.
  /V\     Jan Groenewald
 /( )\    www.aims.ac.za
 ^^-^^ 

[Kwankyu Lee]

On Saturday, September 10, 2022 at 6:26:29 PM UTC+9 dim...@gmail.com wrote:

> Moving away from trac is basically a necessity now, as our devops are breaking down for some time, and it only gets worse.

I wish that our trac survives the present crisis so that some people can have enough time to plan an organized move to github. After that, we may make an informed decision.

[Dima Pasechnik]

On Sat, 10 Sep 2022, 10:56 Jan Groenewald, <j...@aims.ac.za> wrote:

Hi

https://trac.edgewall.org/wiki/TracUsers

In 2020, 21, 22, _only_ Edgewall & Trac-hacks (the trac people), and sagemath are listed as trac users bothering to update their details.

I don't know how accurate the below articles are, but found them intersting to read. Would be interested in

comments from anyone who knows where they are inaccurate:

https://kinsta.com/blog/gitlab-vs-github/#gitlab-vs-github-key-differences

https://www.zdnet.com/article/github-vs-gitlab-the-key-differences/

https://www.incredibuild.com/blog/gitlab-vs-github-comparison (Flow and CI sections)

https://radixweb.com/blog/github-vs-gitlab#difference

I can't tell which workflow is better for sagemath. Remember feature parity betweenm them will also change every few months!

Also see what they say about themselves/each other:

https://about.gitlab.com/devops-tools/github-vs-gitlab/

https://resources.github.com/devops/tools/compare/

It does look like github might offer more contributors. (Of high and low quality? Perhaps the slight hurdle to logging in to gitlab the first time is a good thing...)

I'd be happy to assist with self-hosting (so gitlab) and perhaps sponsor a container (even a separate machine for CI) but I think the sagemath project would probably be better off on git**b. I'm also hesitant to invest my time in learning trac/gitolite. The bus problem Dima mentioned with self-hosting is valid. But nice to have it as an option for a quick exit (from either git**b) when whatever happens to the companies...

migration gitlhub->gitlab is easy, and the latter can be selfhosted on opensource code, so a migration to selfhosted gitlab should not be a problem, if needed.

I don't think the move is so urgent though, but it should probably happen in the next year or so.

I don't see how the current ssh keys issue is not urgent. I think it is very urgent.

And in general, as I said, potential contributors look at the shabby state of our devops and stay away.

Dima

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAg%3Dp_2KD6ZDiY9VKD7RDzNNne%2B5WLg0Cqt7VMyRDvVepyWS7Q%40mail.gmail.com.

[Dima Pasechnik]

On Sat, 10 Sep 2022, 10:02 Aram Dermenjian, <aram.derme...@gmail.com> wrote:

Hi all,

I'm only an occasional developer with sage, but I wanted to also bring up one (minor) positive for moving to github which is less technical. Moving to github would be a motivating factor for people to contribute to sagemath as it gives a pathway to work in industry (programming) if they chose to leave academia. In particular, github shows contributions made by a user to various projects. This could be a valuable tool for someone on the job market who would prefer (or has no choice) to move to industry and get a programming job. Sure, one can already currently say that one develops for sagemath, but by putting it on github it creates a unified location for future employers to look. As mentioned, this is just a minor positive, but could be a motivating factor for some.

it's an excellent point, thanks.

Also, I concur with Vincent in that we should have somewhere with a list of pros/cons for either side in some kind of list fashion as currently it's a little hard to keep track of whether people are making the same arguments or new arguments.

I've enabled wikis on our GitHub and started

https://github.com/sagemath/sage/wiki/migration-from-trac-to-Git**b

Please feel free to add content there.

Dima

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAKQMtir03oQsrdJb06_%2Bi4cdVDtxAdZ8pszcbw3C0juqi87mXA%40mail.gmail.com.

[AndrewVSutherland]

Just to give a different perspective, from someone who is not a Sage developer but is a Sage donor, the only reason I give money to Sage every month is because I happened to see your appeal on GitHub.  If you had no GitHub presence at all, I likely would not be a donor.  Not because I would not want to support Sage in that scenario, but because I would not know you needed money and there would not be a button I could just press to give you some (and I am not the only one, according to https://github.com/sponsors/sagemath there are 25 of us and you currently receive about $3000/year in donations made through GitHub).

I think the same principle applies to attracting donations of time rather than money, and these are much more important to the health of an open source project.  The pool of potential Sage contributors you might attract on GitHub is vastly larger than the pool of people who are motivated enough to figure out how to make contributions via trac.  Yes, you will get plenty of junk PRs you have to deal with on GitHub, but that does not require a lot of effort and I think it would be well worth it for the sake of the non-junk PRs you might get (whose main value is not the code but the person behind it who might submit more PRs in the future if you make it easy for them to do so).  Indeed, I can even imagine myself fixing the occasional bug I come across in Sage if all I had to do was submit a PR on GitHub, and there are surely many Sage users who might become Sage contributors if the activation cost were low enough.  

I also think Aram Dermenjian made an excellent point -- in today's world the value of being able to point to contributions to projects on GitHub vastly exceeds the value of being able to point to a contribution you made on trac (regardless of the merits of the contribution or the project itself).  This does not apply just to professional software developers, it also applies to students who might be your best source of future Sage developers.

In any case, I hope you can all come to an agreement that everyone is happy with (or at least can live with)!

[kcrisman]

For what it's worth, I think that Jan makes some very salient points, such as

> I'd be happy to assist with self-hosting (so gitlab) and perhaps sponsor a container (even a separate machine for CI) but I think the sagemath project would probably be better off on git**b. I'm also hesitant to invest my time in learning trac/gitolite. The bus problem Dima mentioned with self-hosting is valid. But nice to have it as an option for a quick exit (from either git**b) when whatever happens to the companies... 

The real problem with GH is if we end up locked in.  As much hassle as Trac is (and once again, for what it's worth, let me gratefully acknowledge those who are keeping it up and running in whatever state is possible), not being able to get stuff OFF GH is the real issue.

For me, the comparison is with learning management systems that many people who have higher teaching loads than the typical core Sage developer (!) are required to use.  While there are import-export tools from one to another, they are nontrivial and do not always bring by any means the full content with.  In addition, the internal export (say, to pdf or some other transferable medium) can be tiresome in the extreme, when it's even possible.  So in some sense we don't have very good control of our own content, which is very frustrating when I want to use something from our LMS that is wholly inside of it.

The advantage of Trac is having full control of all the data (among others, but for the sake of this point), as well as the VERY extensive ticket history and seemingly endless discussions (I view those as a strength of the community, on the whole).  One really doesn't want to underestimate that.  The advantage of GH is nearly completely convenience (and there are many conveniences, Aram's point is very well taken).  It sounds like there is a pretty reliable GH -> Gitlab mechanism as well, and people willing to host a GL instance on (French?) academic computers.

So is a possible solution to migrate Trac, *including the entire history of discussions*, to GH, and then have a GL mirror of this?  It was unclear from the comments in this thread whether one can actually real-time mirror GH in GL, or only export  via discrete actions (and presumably not realistically to mirror).  Secondarily, one could imagine an automated action that on (say) a point-release basis exports to GL, and that we can point people to that.  I'm not sure how contributions from folks who on principle do not want to support GH would work, but doubtless we aren't the only project to have that problem.

- kcrisman

[Travis Scrimshaw]

I really dislike Github's decentralized approach with PR and having to have separate clones of the repo within each user. My understanding is if two people have different fixes, then they individually submit PRs that are not explicitly linked with each other, much less with a specific bug report issue. It is not so uncommon that people end up working on competing proposals or on the same branch, both of which are not natural using Githubs PR system (I am not even entirely sure how I should do this either; Do I have to clone from their repo? Is it the same remote? Where is the documentation on this? Can I still access the branch if someone deletes their account?). How do other projects at the scale of Sage deal with this?

It seems you never reviewed a non-trivial PR on GitHub, no?

Not at the scale of some of the tickets for Sage (most of it has been fairly small patches), and that is the problem. It is not at all clear how I should do that unlike our current setup.

 

Well, you can go and have a look how e.g. CPython does it, or SciPy. 

Thank you for the pointer. You wouldn't happen to have specific cases that would make for good examples of the behaviors I mentioned above?

The PR process and the underlying structure is very well-documented.

The basics, yes. I didn't see examples of anything beyond "contributor contributes code, then reviewer with project repo permissions reviews and merges it."

 

A PR creates a branch in the git repo of the "main"

project. This is really not much different from what we do on trac, except that the branch name

satisfied different rules.

There is a big difference you aren't even mentioning: You need to fork the repo locally to your UN, which has some additional setup that needs to be done.

Have a look at https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

That sure looks like what you do when you are a developer for the project, and you get the PRs. It doesn't talk about what to do with someone else"s repo. The key title words are "your repository."

E.g. using https://github.com/scipy/scipy/pull/14786 as an example, the underlying branch in the repo of scipy is

named pull/14786. 

To examine it locally on your machine you can use plain git to fetch it, or you can use GitHub's

tool called cli (its functionality is close to what git-trac does, not sure if you use or used git-trac, though).

It is not clear that if I can pull the branch from origin. There are lots of technical issues and questions I have that are not documented, or at least that I can easily find after skimming through things for a few minutes.

It stays there even if the user GitHub account is closed (the latter triggers an automatic closure of the PR, but the underlying

branch remains in the repo, it can be worked on just the same using git)

Which repo? Either way, this seems like a regression compared to our current setup, where if a user quits, then branch, ticket, and everything remains.

Besides, nobody forces you to use a PR to review something. You can open an issue, and ask for a specific branch somewhere (in a fork, or even in a completely unrelated git branch hosted outside of GitHub) to be reviewed, and only do a PR once the branch in question is stabilized.

At the end of the day, all paths lead to PRs. A rose by any other name still has thorns one needs to be careful of. 

Honestly, I doubt we are loosing that many quality contributors by not using Github's system since we can have people with GH login's to access trac.

except that this is broken now, (along with automatic management of ssh keys in general) and one needs

a tricky manual process (modifying a special gitolite git repo used by trac) by one of trac admins

to add/modify ssh keys.

(and I am not even sure whether this manual process does not lead to an eventual breakage of the ssh keystore, it's all tied up

with trac in a rather arcane way).

This is a definite problem. 

Getting the login credentials was the biggest barrier; everything else is mostly straightforward and based on very simple git commands.  

Right now, I find there are way too many practical questions and barriers for how we develop that make moving to Git**b a much bigger pain that people will think it is.

 

Travis, many people nowadays never used git without GitHub or GitLab. For such a person it's a major pain to 

learn our workflow.

Do you really believe that everyone is using the web interface to make edits to the code and not using some form of git locally (either command line or GUI based)? The web interface has major problems, such as not being able to run tests locally, in addition to being unwieldy with a project on the scale of Sage. Honestly, people really don't use "git pull", "git push", "git commit" when working with Github? You haven't said how people use it as any sort of counter argument.

I also made a very clear point about git commands. Our workflow won't fundamentally change: (1) make ticket (GH lingo is open issue) (2) make changes (3) test code locally (4) polish code and documentation (5) push for review (6) undertake review (7) give offerings to Volker for his majesty and grace of being a great release manager. A little different interface for someone to learn is not really that hard.

Some of this could be my lack of experience with Github (and a bit of "get off my lawn with your facegrams"), but right now I feel like the move to Github is like starting a car to go to the party in the next town without learning how to drive, much less knowing the route.

What's the alternative? 

You mean beyond spending the time to fix it instead of pushing us into the pool and hoping we swim?

 

Would you like to contribute your time to do our time-consuming trac etc devops?

I would be willing, although I expect to be hit by an approximate 3kg (very cute) bus within the next week or two.

You need much more of a plan than simply saying "its easy because other people use it". I definitely don't want to be the dog who caught the car. That will be far more painful than what this move already will be.

Anyway, trac is obsolete software used mainly to maintain legacy projects which don't even use git,

and we'd have to move sooner or later to something that is robust and maintained, and is not an endless

timesink to keep working. And trac is only a part of the general Sage devops nightmare, created largerly by

sticking to semi-obsolete tools for too long.

I think you have pointed at exactly one question - you didn't know how to deal with git branches of GitHub's PR.

This is a horrible oversimplication of my post at best.

 

I hope you see now that it's trivial. Do you have any other objections?

No, it is not. Quit brushing it aside and saying it is trivial while pointing to a massive stack of pages that doesn't answer any of my questions.

I am sorry if my response is coming across harsh, but I don't appreciate a bunch of stawman arguments and condensation as an answer. If it is as easy as you claim it is, then you should be able to actually explain it with examples.

When we switched to git from Mg, we spent a week experimenting with a variety of things with trying to find a good approach. We could also experiment with the community during that time to refine our practices. With this potential migration, it is either the Githubway or the highway and there is no way back.

Best,

Travis

[Travis Scrimshaw]

Just to give a different perspective, from someone who is not a Sage developer but is a Sage donor, the only reason I give money to Sage every month is because I happened to see your appeal on GitHub.

Thank you very much for your donations.

 

  If you had no GitHub presence at all, I likely would not be a donor.  Not because I would not want to support Sage in that scenario, but because I would not know you needed money and there would not be a button I could just press to give you some (and I am not the only one, according to https://github.com/sponsors/sagemath there are 25 of us and you currently receive about $3000/year in donations made through GitHub).

I think the same principle applies to attracting donations of time rather than money, and these are much more important to the health of an open source project.  The pool of potential Sage contributors you might attract on GitHub is vastly larger than the pool of people who are motivated enough to figure out how to make contributions via trac.  Yes, you will get plenty of junk PRs you have to deal with on GitHub, but that does not require a lot of effort and I think it would be well worth it for the sake of the non-junk PRs you might get (whose main value is not the code but the person behind it who might submit more PRs in the future if you make it easy for them to do so).  Indeed, I can even imagine myself fixing the occasional bug I come across in Sage if all I had to do was submit a PR on GitHub, and there are surely many Sage users who might become Sage contributors if the activation cost were low enough. 

If I knew that you could easily access trac, I would ask you for feedback on what you find about our current development process. It might be good to have a bit more outside opinion for this (in particular, without any help). Even just sketching how you think the process would go and where you had some struggles would be useful. Then we could explain about any of the specifics and see how you think it compares to PRs on GH. Only if you have time.

I also think Aram Dermenjian made an excellent point -- in today's world the value of being able to point to contributions to projects on GitHub vastly exceeds the value of being able to point to a contribution you made on trac (regardless of the merits of the contribution or the project itself).  This does not apply just to professional software developers, it also applies to students who might be your best source of future Sage developers.

Indeed, this is a good point.

Best,

Travis

[Matthias Koeppe]

On Saturday, September 10, 2022 at 5:09:04 AM UTC-7 Dima Pasechnik wrote:

I've enabled wikis on our GitHub and started

https://github.com/sagemath/sage/wiki/migration-from-trac-to-Git**b

Please feel free to add content there.

Thanks, Dima!

 

I've added a draft of a proposed workflow on GitHub with the idea to just follow the Trac workflow.

Help is welcome in adding links to documentation for the various bits.

I think we can have a fleshed out Trac to GitHub transition guide by the end of the weekend.

[Matthias Koeppe]

On Saturday, September 10, 2022 at 2:02:03 AM UTC-7 aram.derme...@gmail.com wrote:

I wanted to also bring up one (minor) positive for moving to github which is less technical. Moving to github would be a motivating factor for people to contribute to sagemath as it gives a pathway to work in industry (programming) if they chose to leave academia. In particular, github shows contributions made by a user to various projects. This could be a valuable tool for someone on the job market who would prefer (or has no choice) to move to industry and get a programming job. Sure, one can already currently say that one develops for sagemath, but by putting it on github it creates a unified location for future employers to look. As mentioned, this is just a minor positive, but could be a motivating factor for some.

Yes, this is a very important point.

 

[John H Palmieri]

On Friday, September 9, 2022 at 12:08:38 PM UTC-7 dim...@gmail.com wrote:

On Fri, Sep 9, 2022 at 7:58 PM Thierry <sage-goo...@lma.metelu.net> wrote:
>
> On Fri, Sep 09, 2022 at 04:07:48PM +0100, Dima Pasechnik wrote:
> [...]
> >
> > Several people promised to look for academic hosting for trac. Nothing
> > came out of it.
>
> Please do not rewrite history, as it adds violence to violence. There
> have indeed been offers for academic hosting, the VM were ready, but the
> cloud hosting was enforced, and you was promoting this.

I don't recall any concrete offer like this. If I recall correctly,
the problem of accessing VMs in French
universities for outsiders have never been resolved in a meaningful
way, leading to an unacceptably small
for the project bus factor.

By the way, another proof of my small bus factor claim is here:
https://trac.sagemath.org/ticket/33725#comment:30
Apparently you want to do everything your way, and your way only.
You are asked to provide the wiki contents in a suitable for import
form, and you just pretend you didn't see the request?

Dima, these kinds of personal remarks/attacks are completely inappropriate. Please stop.

--

John

[Marc Mezzarobba]

Matthias Koeppe wrote:
> I've added a draft of a proposed workflow on GitHub with the idea to
> just follow the Trac workflow.

In my eyes at least, it is a defect of our current workflow that tickets
are used for tracking both bugs and proposed enhancements. On git**b,
wouldn't it be more natural to use issues for, well, issues (that is,
bug reports, as opposed to bug *fixes* or enhancements) and pull
requests for proposed changes?

--
Marc Mezzarobba

[Matthias Koeppe]

On Saturday, September 10, 2022 at 10:23:51 AM UTC-7 Marc Mezzarobba wrote:

Matthias Koeppe wrote:
> I've added a draft of a proposed workflow on GitHub with the idea to
> just follow the Trac workflow.

I should have said "... to the extent that makes sense."

 

In my eyes at least, it is a defect of our current workflow that tickets
are used for tracking both bugs and proposed enhancements. On git**b,
wouldn't it be more natural to use issues for, well, issues (that is,
bug reports, as opposed to bug *fixes* or enhancements) and pull
requests for proposed changes?

Yes, of course, and that's what I am documenting at https://github.com/sagemath/sage/wiki/migration-from-trac-to-Git**b

 

[Marc Mezzarobba]

Aram Dermenjian wrote:
> In particular, github shows contributions made by a user to
> various projects.

This is true to some extent (only for commits as opposed to code review
or other kinds of contributions, and maybe only for users who have set
up their account in a certain way) even when the contributions were not
developed on github but simply ended up in a repository mirrored on
github.

--
Marc Mezzarobba

[Matthias Koeppe]

Yes, that's true for commits, which do show up at https://github.com/sagemath/sage/graphs/contributors at the time of merging into a release (except when people have misconfigured their email).

But our Trac tickets are invisible to the world.

[Marc Mezzarobba]

Matthias Koeppe wrote:
> Yes, of course, and that's what I am documenting
> at https://github.com/sagemath/sage/wiki/migration-from-trac-to-Git**b

Sorry, apparently I misunderstood your proposal.

I would suggest the following changes then:

"Instead of opening a Trac ticket"
--> "To report a bug, instead of opening a Trac ticket"

""Bug"/"Enhancement" is mapped to "Labels""
--> "No "Bug"/"Enhancement" distinction (use pull requests to submit
enhancements)"

--
Marc

[Matthias Koeppe]

On Saturday, September 10, 2022 at 10:38:16 AM UTC-7 Marc Mezzarobba wrote:

Matthias Koeppe wrote:
> Yes, of course, and that's what I am documenting
> at https://github.com/sagemath/sage/wiki/migration-from-trac-to-Git**b

Sorry, apparently I misunderstood your proposal.

I would suggest the following changes then:

"Instead of opening a Trac ticket"
--> "To report a bug, instead of opening a Trac ticket"

No, Issues are not just for bugs, they can also be used for planning enhancements.

 

""Bug"/"Enhancement" is mapped to "Labels""
--> "No "Bug"/"Enhancement" distinction (use pull requests to submit
enhancements)"

I'll edit to make it clearer what I intended to say there. 

 

[Marc Mezzarobba]

Matthias Koeppe wrote:
> No, Issues are not just for bugs, they can also be used for planning
> enhancements.

Sure; the changes I was suggesting are an oversimplification. But I
don't think we should keep with the practice of forcing developers to
open a dedicated issue for every proposed change.

--
Marc Mezzarobba

[Matthias Koeppe]

OK, good point, and I agree. I'll work that into the wiki page.

 

[Marc Mezzarobba]

Matthias Koeppe wrote:
> OK, good point, and I agree. I'll work that into the wiki page.

Thanks!

--
Marc

[Matthias Koeppe]

On Saturday, September 10, 2022 at 8:12:48 AM UTC-7 Matthias Koeppe wrote:

I've added a draft of a proposed workflow on GitHub with the idea to just follow the Trac workflow.

Help is welcome in adding links to documentation for the various bits.

I think we can have a fleshed out Trac to GitHub transition guide by the end of the weekend.

A draft of the Trac-to-GitHub transition guide is now available at:

https://github.com/sagemath/sage/wiki/migration-from-trac-to-Git**b

Please let me know what's missing or unclear.

 

[Nathan Dunfield]

I think moving to GitHub makes total sense.  Every other open-source math project I use regularly is on GH, and there are big network-effect benefits to being where everyone else is as others have already pointed out: easier for others to contribute, easier to get credit for contributions, easier cross references to issues up and downstream, etc.  Given the range of projects that use GH, including those like numpy and scipy whose issue/PR counts are similar to the number Sage trac tickets, I'm sure any technical or workflow issues have already been overcome by others and the solutions are likely even documented.  Given the open-source community's reliance on GH, any serious misbehavior on GH's part would result in a massive pushback and, if necessary, a concerted effort by a huge number of people to work out an alternative.

Nathan

[seb....@gmail.com]

 What will be the replacement for the ticket description? Just the first comment of the issue reprectively PR? If so, who can edit it?

[John H Palmieri]

I have heard lots of support for moving to GitHub, and I do not really understand the arguments against. Is there more than "trac is what we're used to"? Can those who are opposed please articulate the reasons?

I appreciate the efforts by Matthias and others to provide a roadmap for the proposed transition.

--

John

[Dima Pasechnik]

yes

If so, who can edit it?

anyone with write access to the repo.

https://docs.github.com/en/communities/moderating-comments-and-conversations/managing-disruptive-comments#editing-a-comment

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/4420889b-a904-4ef8-9743-037648e7640cn%40googlegroups.com.

[Dima Pasechnik]

On Sun, 11 Sep 2022, 00:36 Dima Pasechnik, <dim...@gmail.com> wrote:

On Sun, 11 Sep 2022, 00:22 seb....@gmail.com, <seb....@gmail.com> wrote:

Matthias Koeppe schrieb am Samstag, 10. September 2022 um 21:32:50 UTC+2:

On Saturday, September 10, 2022 at 8:12:48 AM UTC-7 Matthias Koeppe wrote:

I've added a draft of a proposed workflow on GitHub with the idea to just follow the Trac workflow.

Help is welcome in adding links to documentation for the various bits.

I think we can have a fleshed out Trac to GitHub transition guide by the end of the weekend.

A draft of the Trac-to-GitHub transition guide is now available at:

https://github.com/sagemath/sage/wiki/migration-from-trac-to-Git**b

Please let me know what's missing or unclear.

 What will be the replacement for the ticket description? Just the first comment of the issue reprectively PR?

yes

If so, who can edit it?

anyone with write access to the repo.

as well as the author, naturally.

[seb....@gmail.com]

dim...@gmail.com schrieb am Sonntag, 11. September 2022 um 01:36:31 UTC+2:

On Sun, 11 Sep 2022, 00:22 seb....@gmail.com, <seb....@gmail.com> wrote:

Matthias Koeppe schrieb am Samstag, 10. September 2022 um 21:32:50 UTC+2:

On Saturday, September 10, 2022 at 8:12:48 AM UTC-7 Matthias Koeppe wrote:

I've added a draft of a proposed workflow on GitHub with the idea to just follow the Trac workflow.

Help is welcome in adding links to documentation for the various bits.

I think we can have a fleshed out Trac to GitHub transition guide by the end of the weekend.

A draft of the Trac-to-GitHub transition guide is now available at:

https://github.com/sagemath/sage/wiki/migration-from-trac-to-Git**b

Please let me know what's missing or unclear.

 What will be the replacement for the ticket description? Just the first comment of the issue reprectively PR?

yes

If so, who can edit it?

anyone with write access to the repo.

https://docs.github.com/en/communities/moderating-comments-and-conversations/managing-disruptive-comments#editing-a-comment

Thanks!

[seb....@gmail.com]

> I appreciate the efforts by Matthias and others to provide a roadmap for the proposed transition.

+1

Nobody would buy a new house which has not been built yet without a good animation of how it would look like.

> Is there more than "trac is what we're used to"? Can those who are opposed please articulate the reasons?

I think it's natural that people want to know which advantages of the old house will be gone after the move. Therefore, we should not ask this question before the animation has finished.

[Jan Groenewald]

Hi

On Sat, 10 Sept 2022 at 21:32, Matthias Koeppe <matthia...@gmail.com> wrote:

A draft of the Trac-to-GitHub transition guide is now available at:

https://github.com/sagemath/sage/wiki/migration-from-trac-to-Git**b

Please let me know what's missing or unclear.

Thank you for editing that!

The wiki page is titled gitl**b but the content is about github only, not gitlab:

* How would a gitlab workflow work better or worse than github? Especially the CI/Actions. Even if the preferred workflow is possible to implement in either platform, their preferred philosophy will impact future features.

From my previous post, as I don't feel qualified to evaluate this:

"""

 I don't know how accurate the below articles are, but found them intersting to read. Would be interested in

comments from anyone who knows where they are inaccurate:

https://kinsta.com/blog/gitlab-vs-github/#gitlab-vs-github-key-differences

https://www.zdnet.com/article/github-vs-gitlab-the-key-differences/

https://www.incredibuild.com/blog/gitlab-vs-github-comparison (Flow and CI sections)

https://radixweb.com/blog/github-vs-gitlab#difference

I can't tell which workflow is better for sagemath. Remember feature parity betweenm them will also change every few months!

Also see what they say about themselves/each other:

https://about.gitlab.com/devops-tools/github-vs-gitlab/

https://resources.github.com/devops/tools/compare/

"""

Also the point that GL offers free private repos... if that is necessary/useful.

Is someone able to update the wiki with equivalent guide-to-new-workflow for gitlab so they can be compared? Is that necessary or not?

Regards,

Jan

[Tobias Diez]

Thanks, this is already in a pretty nice shape! I've extended some points a bit.

In my opinion, we can also directly drop the develop branch and move to the master/main-branch only model. But that's slightly orthogonal to the migration to Github.