Project metadata "lost" in fork to new organisation

[John Mark Vandenberg]

I emailed Joe and he said there are five maintainers now, and suggested I email this group to express my interest in helping.

fwiw, I only see one public member of the new organisation.

https://github.com/orgs/httplib2/people

Maybe the other four are private members?

The new repo at https://github.com/httplib2/httplib2 is just a fork, which means it is forever linked to the upstream repository at https://github.com/jcgregorio/httplib2 .

Why wasnt the project, with all issues, PRs, wiki, etc transferred to the new organisation?

https://help.github.com/articles/transferring-a-repository/

Was this intentional?  Was this discussed somewhere?  It seems like a very bad choice...

All of the existing commit links (e.g. `#322`) are going to continue to resolve to https://github.com/jcgregorio/httplib2 .

Eventually there will be two meanings for `#322`. :/

I was very surprised to see all of the issues being closed, especially without any comment/link/etc explaining the move.  I expect other users will also be put off by that.

By transferring the repository to the organisation properly, all metadata can be retained, and any of the valid recently closed tickets can be reopened.

I am happy to help out.

If this is to be fixed, it needs to be fixed soon, as people are creating new issues in the new repo, and fixing the repo will mean the current fork needs to be deleted/renamed, with all issues, PRs, etc deleted/renamed.

--

John Vandenberg

[chris...@gmail.com]

On Mon, 11 Apr 2016, John Mark Vandenberg wrote:

> I emailed Joe and he said there are five maintainers now, and suggested I
> email this group to express my interest in helping.
> fwiw, I only see one public member of the new organisation.
> https://github.com/orgs/httplib2/people
> Maybe the other four are private members?

When the new org was created all 5 maintainers were added as
private. I guess I'm the only one who has made themself public
(because why not?).

> The new repo at https://github.com/httplib2/httplib2 is just a fork, which
> means it is forever linked to the upstream repository at
> https://github.com/jcgregorio/httplib2 .
>
> Why wasnt the project, with all issues, PRs, wiki, etc transferred to the
> new organisation?

The five of us who have sort of volunteered to take on maintenance
haven't had a chance to make a plan on what is going to happen with
httplib2 nor how its infrastructure is going to be managed. I'm
hoping that your posting will be a catalyst to get that going, but
in the meantime things are pretty up in the air.

From my standpoint the creation of the new org was a surprise (I
assumed something like it would happen but wasn't sure when or in
what form).

> https://help.github.com/articles/transferring-a-repository/
>
> Was this intentional? Was this discussed somewhere? It seems like a very
> bad choice...

I'm not aware of any discussion. However, from my standpoint not
transferring the repo in full is a good idea for a few reasons:

In the brief discussions that happened with some of the five people
the consensus was to stabilize (as much as possible) httplib2 and then
freeze and deprecate it (in favor of urllib3). Not everybody wanted
this, but it was the outcome that everyone could live with[1].

By not transferring all the issues and pull requests we create a hard
break such that only those issues with an actively concerned audience
get transferred across.

> I was very surprised to see all of the issues being closed, especially
> without any comment/link/etc explaining the move. I expect other users
> will also be put off by that.

I think, but am not certain, that Joe did this in an automatic
fashion as a signal of his "I'm done" which is his perogative.

> By transferring the repository to the organisation properly, all metadata
> can be retained, and any of the valid recently closed tickets can be
> reopened.

Given the number of issues that were closed in the last 24 hours
there were enough open issues that they were pretty much noise. I
think we're more likely to create a quality and stable "final"
release if we allow issues to resurrect.

> I am happy to help out.

Thank you. I'm sure it will take a while to figure out what the plan
actually is. There was some enthusiasm and energy a little earlier
in the year for making something cohesive happen and then there was
some latency getting things moving. That bled off some of that
energy into other areas (switching some projects that depend on
httplib2 over to urllib3[2]) so now will need to gin it up again.

> If this is to be fixed, it needs to be fixed soon, as people are creating
> new issues in the new repo, and fixing the repo will mean the current fork
> needs to be deleted/renamed, with all issues, PRs, etc deleted/renamed.

I'll see if I can find some of the people today and point them at
this thread so we can make a proper plan.

[1] I was one of the people who wanted to keep httplib2 alive and
thriving (collapse the crazy py2-3 split into a single codebase,
address the issues, find a community etc) because I find it is the
only http library that has an interface that I actually like (does
what it says on the tin, limited magic) but other people said given
existing commitments this would be very challenging. And they are right,
this is the first I've had a chance to pay any attention to this, and
diverting time and energy to give it a real go would be challenging. So
I did some [2].

[2] In OpenStack there's a lot of effort put into making sure that
all dependencies (all the way down the tree) are well maintained, so
when when word came round that httplib2's state of un-maintained was
going to go from assumed to official there was a choice to stick with it
or switch to something with a robust community. Most people decided
that switching to urllib3 was the way to go. In my own project,
gabbi, the switch was pretty easy:

https://github.com/cdent/gabbi/commit/8a8af66eee6d5b9936419d037aadaf37e829cedb

A similar switch is in progress for Tempest:

https://review.openstack.org/#/c/295900/

--
Chris Dent http://burningchrome.com/
[...]

[John Mark Vandenberg]

On Tue, Apr 12, 2016 at 4:31 PM, <chris...@gmail.com> wrote:
> On Mon, 11 Apr 2016, John Mark Vandenberg wrote:

>> ..
> ..

>> The new repo at https://github.com/httplib2/httplib2 is just a fork, which
>> means it is forever linked to the upstream repository at
>> https://github.com/jcgregorio/httplib2 .
>>
>> Why wasnt the project, with all issues, PRs, wiki, etc transferred to the
>> new organisation?
>

> ..

>> https://help.github.com/articles/transferring-a-repository/
>>
>> Was this intentional? Was this discussed somewhere? It seems like a very
>> bad choice...
>
>
> I'm not aware of any discussion. However, from my standpoint not
> transferring the repo in full is a good idea for a few reasons:
>
> In the brief discussions that happened with some of the five people
> the consensus was to stabilize (as much as possible) httplib2 and then
> freeze and deprecate it (in favor of urllib3). Not everybody wanted
> this, but it was the outcome that everyone could live with[1].

I agree that httplib2 should be very stable, as it is approaching 1.0,
to the point that I would not like the "crazy" py2/py3 split
'fixed'[1], and IMO a complete feature freeze should be in place until
1.0 is released, and possibly even after that with _soft_
'deprecation' not unreasonable.

But that is a maintenance style decision, and has nothing to do with
forking vs transferring the repository to the new organisation.

> By not transferring all the issues and pull requests we create a hard
> break such that only those issues with an actively concerned audience
> get transferred across.

If their issues are going to be declined, due to the project
maintainers wanting to freeze the project, that should be done with
clear communication.
Disconnecting "old" users intentionally appears to be quite rude.
Given the number of projects depending on httplib2, either currently
or in earlier stable supported releases [2], poor communication is not
going to help. Most users of httplib2 are still users of httplib2
because of its stability. The users don't suddenly disappear; they
just get annoyed.

>> By transferring the repository to the organisation properly, all metadata
>> can be retained, and any of the valid recently closed tickets can be
>> reopened.
>
>
> Given the number of issues that were closed in the last 24 hours
> there were enough open issues that they were pretty much noise. I
> think we're more likely to create a quality and stable "final"
> release if we allow issues to resurrect.

Many were exceedingly old and real severe bugs, with patches,
sometimes several patches ;-)
Especially problems isolated to one of py2 vs py3, which any
reasonable attempt at maintaining would consider to be appropriate to
be fixed, provided sufficient tests exist, etc.
Ignoring history doesnt seem like a good way forward if stability is a goal.

[1] Pywikibot 2.0's httplib2 repo, which exists to simplify stability
for our users, has custom code that depends on this.

https://github.com/wikimedia/pywikibot-externals-httplib2/commits/master/__init__.py

Of course, we could 'fix' that, but that is an annoyance on our stable branch.

[2] This is the case with Pywikibot - our -dev branch has switched to
requests, however we have patches in our queue to re-add support for
httplib2 alongside requests. urllib3/requests is nice, but it is a
rapidly changing beast, and it is desupporting platforms that halt its
progress, or being very noisy about deprecating them. httplib2 had
many advantages.

--
John Vandenberg

[chris...@gmail.com]

On Tue, 12 Apr 2016, John Mark Vandenberg wrote:

> Ignoring history doesnt seem like a good way forward if stability is a goal.

The "goal" is the open question (stability is one option,
resurrection is another, and a graceful death is yet another). If you
hang tight for a few hours a few more people will be able to get
involved in the conversation and we can make a real plan.

[graffatc...@gmail.com]

Hi John,

Long time, no chat.

On Tuesday, April 12, 2016 at 6:01:05 AM UTC-5, John Mark Vandenberg wrote:

On Tue, Apr 12, 2016 at 4:31 PM,  <chris...@gmail.com> wrote:
> On Mon, 11 Apr 2016, John Mark Vandenberg wrote:
>> ..
> ..
>> The new repo at https://github.com/httplib2/httplib2 is just a fork, which
>> means it is forever linked to the upstream repository at
>> https://github.com/jcgregorio/httplib2 .
>>
>> Why wasnt the project, with all issues, PRs, wiki, etc transferred to the
>> new organisation?
>
> ..
>> https://help.github.com/articles/transferring-a-repository/
>>
>> Was this intentional?  Was this discussed somewhere?  It seems like a very
>> bad choice...

No. Joe was entirely responsible for this and did this before adding any maintainers that I'm aware of. I suspect, like many people who have had projects become wildly popular, that they want to keep a "popular" repository on their profile page because it shows they've made very popular projects and builds a "brand".

Please don't brow beat the 5 maintainers who have decided to split their attention, yet again, to help stabilize and maintain this project. This doesn't motivate any of us.
 

> I'm not aware of any discussion. However, from my standpoint not
> transferring the repo in full is a good idea for a few reasons:
>
> In the brief discussions that happened with some of the five people
> the consensus was to stabilize (as much as possible) httplib2 and then
> freeze and deprecate it (in favor of urllib3). Not everybody wanted
> this, but it was the outcome that everyone could live with[1].

I agree that httplib2 should be very stable, as it is approaching 1.0,
to the point that I would not like the "crazy" py2/py3 split
'fixed'[1], and IMO a complete feature freeze should be in place until
1.0 is released, and possibly even after that with _soft_
'deprecation' not unreasonable.

What's the difference between a "soft" deprecation and a "hard" deprecation?

In my mind the way we would deprecate this is:

"Bug fixes only" (which is what I think of when Chris says "freeze")

That might last for a year until the time when we say "Security bug fixes only" which lasts for 6 months to a year and then the project stops seeing any updates.
 

But that is a maintenance style decision, and has nothing to do with
forking vs transferring the repository to the new organisation.

Correct, but they're a little related.
 

> By not transferring all the issues and pull requests we create a hard
> break such that only those issues with an actively concerned audience
> get transferred across.

If their issues are going to be declined, due to the project
maintainers wanting to freeze the project, that should be done with
clear communication.
Disconnecting "old" users intentionally appears to be quite rude.

Again, please don't scorn people who had nothing to do with this decision and have not done any of this. Joe has done all of this on their own repository without our input or help.
 

Given the number of projects depending on httplib2, either currently
or in earlier stable supported releases [2], poor communication is not
going to help.  Most users of httplib2 are still users of httplib2
because of its stability.  The users don't suddenly disappear; they
just get annoyed.

We have only *just* been made maintainers. We haven't had time to discuss maintenance and we haven't really discussed much, as Chris has tried to make abundantly clear.
 

>> By transferring the repository to the organisation properly, all metadata
>> can be retained, and any of the valid recently closed tickets can be
>> reopened.
>
>
> Given the number of issues that were closed in the last 24 hours
> there were enough open issues that they were pretty much noise. I
> think we're more likely to create a quality and stable "final"
> release if we allow issues to resurrect.

Many were exceedingly old and real severe bugs, with patches,
sometimes several patches ;-)
Especially problems isolated to one of py2 vs py3, which any
reasonable attempt at maintaining would consider to be appropriate to
be fixed, provided sufficient tests exist, etc.
Ignoring history doesnt seem like a good way forward if stability is a goal.

[1] Pywikibot 2.0's httplib2 repo, which exists to simplify stability
for our users, has custom code that depends on this.

https://github.com/wikimedia/pywikibot-externals-httplib2/commits/master/__init__.py

Of course, we could 'fix' that, but that is an annoyance on our stable branch.

[2] This is the case with Pywikibot - our -dev branch has switched to
requests, however we have patches in our queue to re-add support for
httplib2 alongside requests.  urllib3/requests is nice, but it is a
rapidly changing beast, and it is desupporting platforms that halt its
progress, or being very noisy about deprecating them.  httplib2 had
many advantages.

You talk a lot about things that are rude or things that are alienating. Here's something rude and alienating: Telling us that how we plan to make maintenance sustainable is unacceptable (e.g., combining the project into a single code-base). It's not as if we're going to haphazardly throw a bunch of code together and call it "httplib2 in one code base". We're not incompetent and I don't appreciate the implication that we wouldn't be able to successfully do that. A single code-base is far simpler to maintain and will allow us to not be frustrated in the maintenance we will be doing (meaning we might be less inclined to stop maintenance earlier).

[John Mark Vandenberg]

On Tue, Apr 12, 2016 at 10:23 PM, <graffatc...@gmail.com> wrote:
> What's the difference between a "soft" deprecation and a "hard" deprecation?

Hard: emitting warnings, or worse, is a commonly used definition
Soft: documentation

> In my mind the way we would deprecate this is:
>
> "Bug fixes only" (which is what I think of when Chris says "freeze")

It would be helpful to know how existing feature differences between
the Python 2 & 3 codebases will be treated.by the new maintainers.

Are they bugs, which may be fixed, or will patches be rejected as new
functionality that hasnt previously existed on that version of Python?

--
John Vandenberg

[John Mark Vandenberg]

On Tue, Apr 12, 2016 at 5:07 AM, John Mark Vandenberg <jay...@gmail.com> wrote:
>..

> All of the existing commit links (e.g. `#322`) are going to continue to
> resolve to https://github.com/jcgregorio/httplib2 .
> Eventually there will be two meanings for `#322`. :/

Actually, it appears to be more complex than that, and this is already
a 'problem' as the hash notation was first used in commits messages
from 2007 when the project was on Google Code, and the numbering was
preserved when the project was transferred to Github.

https://github.com/httplib2/httplib2/commit/0d4a2b8e5c mentions #2,
which github now links to
https://github.com/httplib2/httplib2/issues/2 .

OTOH,

https://github.com/httplib2/httplib2/commit/37060d2d3 mentions #322,
which github links to https://github.com/jcgregorio/httplib2/pull/322
.

My guess is that the # number resolves to the local project if
possible, otherwise it falls back to the upstream repo.
In which case the next link to change will be when #12 is created.
https://github.com/httplib2/httplib2/commit/e202d21

Just an idea to avoid that problem, assuming no transfer is going to
occur, is to create and close boilerplate tickets with a link to the
old repo, so that the new issues raised under the new organisation
repo will be above #328 to avoid number conflicts.

--
John Vandenberg

[chris...@gmail.com]

On Tue, 12 Apr 2016, chris...@gmail.com wrote:
> On Mon, 11 Apr 2016, John Mark Vandenberg wrote:
>> If this is to be fixed, it needs to be fixed soon, as people are creating
>> new issues in the new repo, and fixing the repo will mean the current fork
>> needs to be deleted/renamed, with all issues, PRs, etc deleted/renamed.
>
> I'll see if I can find some of the people today and point them at
> this thread so we can make a proper plan.

So people are aware:

Ian and I will be getting together at the end of this week (the
soonest we both have time) to talk about how we're going to proceed
with httplib2. As I'm sure everyone can understand it is not going
to be as simple and straightforward to get things back on some kind
of even keel as people would like. Them's the breaks. We'll do what
we can to make sure there is a useful artifact.