Migrating PhantomJS issue tracker from Google Code to GitHub

[James Greene]

Hello, everyone!

TL;DR:

We are currently planning to migrate the PhantomJS project issues from the Google Code issue track to a new home integrated with the PhantomJS GitHub repo's issue tracker (which is currently disabled).  I've already done all the ground work to make it quick and mostly painless, so we can proceed with this change within the next few days (or maybe ~1 week max) if there aren't any major concerns.

Please share any thoughts, concerns, encouragements, conversion bugs, etc. on this proposed plan ASAP.  Thank you!

Long version:

As most of you know, we've been slowly migrating all our PhantomJS project assets over to GitHub from Google Code: first source, then some static Wiki pages to our new gh-pages based website, then the rest of the Wiki to GitHub Wiki.  We are now planning to migrate our Issues over to GitHub Issues and maintaining as much parity as possible: IDs, comments, links, tagged users, state, milestones, labels, etc.

However, since we migrated our primary source location first, we've also had hundreds of GH pull requests since then. I thought this would be fine... but, in the fall, I noticed/learned that GitHub PRs and Issues share the same "numberspace" (ID counter)... surprise!  As such, it is now impossible for us to achieve the ID parity we desired when migrating our Issues without creating a new repo (which could cause many more problems: disconnected PR history, severed forks, etc.).  The best we can do now is migrate the issues with an ID offset (we're using 10000), so Google Code Issue #1 will become GitHub Issue #10001, and so on.

Over the last few months, I've been collaborating with Ariya and some GitHub staffers to plan out a way for us to migrate all of our issues from the Google Code issue tracker over to GitHub.  We've worked it out to a pretty slick process such that it can be completed in just a few minutes' time, and all account information (linking), user email privacy, labels, milestones, comments, dates, etc. will be maintained.

A few things that we know will not be maintained:

• On Google Code, all users who "star" an issue receive updates about it. GitHub has a similar process where anyone can choose to "watch" an issue without having to actively participate in it.  However, Google Code does not expose the details on who starred each issue, so we will not be able to automatically carry forward these notifications.
  
  
• For any issues/comments posted on the Google Code issue tracker using an email address which also has a linked GitHub account, those issues/comments will be created on GitHub as that user.  If such a link does not exist, the issue/comment will be created with my GitHub account.  In either case, the body text will begin with "<TheOriginalPoster> commented..."
  
  
• Star counts: there is no equivalent on GitHub at this time.

You can see the dry run result of this process conducted in late January (prior to a number of bug fixes I've made since) here:

    https://github.com/JamesMGreene/phantomjs-issues/issues

Please take a little time to review the dry run's migrated issues.  Although I have already made a number of bug fixes since that migration, please do not hesitate to reply to this thread with any conversion problems you see.  I'd rather get reports of bugs I already fixed than miss real remaining bugs! :)

If there aren't any major concerns, we plan to proceed with this permanent migration within the next few days (or maybe ~1 week max).

Please share any thoughts, concerns, encouragements, conversion bugs, etc. on this proposed plan ASAP.  Thank you!

Sincerely,
    James Greene

---------- Forwarded message ----------
From: James Greene <james.m...@gmail.com>
Date: Mon, Oct 22, 2012 at 9:02 AM
Subject: Special request for migrating high-profile project from Google Code to GitHub
To: support <sup...@github.com>

Hello!
I am contributing to the PhantomJS project (the premier cross-platform headless browser), which is a pretty popular open source project: 3000+ stargazers on GitHub, 50000+ downloads of the previous minor version release (1.6 & 1.6.1) in a 3 month period with the latest release (1.7) well on its way to shattering that number,  many "child projects" as offshoots, and several NPM modules to wrap/proxy/control/install it from Node.js.

We've been slowly migrating all our assets over to GitHub from Google Code: first source, then some static Wiki pages to our new gh-pages based website, then the rest of the Wiki to GitHub Wiki.  We now want to migrate our Issues over to GitHub Issues and obtain as much parity as possible: IDs, comments, links, tagged users, state, milestones, labels, etc.

However, since we migrated our source first (plus 3 other mirrored copies elsewhere but GH is now the primary), we've also had hundreds of GH pull requests since then. I thought this would be fine... but I recently learned that PRs and Issues share the same "numberspace" (ID counter)... surprise!  As such, it is now impossible for us to achieve the ID parity (which is very important to the project owner) we desire when migrating our Issues without creating a new repo (in which case we lose our history, PR history, severe our forks, etc.).

So, a few questions:
1. Is there any git or GitHub magic to achieve this?  Assuming not since the PR history would still have the conflicting IDs.
2. If not, are there any kindly souls at GitHub (perhaps consumers of PhantomJS?) who would help us renumber our PRs while maintaining their hashes (and links to them from commits, if any) in order to allow us to transfer our Issues from Google Code while still achieving ID parity?
3. If neither (), can you make a strong case for why we should still migrate our Issues to GitHub over Google Code despite the loss of parity? In other words, what advantages do GH issues offer over GC issues, other than their close integration with the source, PRs, and wiki?  On the flip side, are there any disadvantages that you can point out to Google Code Issues?

Issue ID parity seems to be a dealbreaker (or very close to it) for the project owner, so if we can't achieve this he will likely not agree to migrate them at all... meaning we will continue using Google Code for our Issues. (Again: )

Please let me know. Thanks in advance!

Sincerely,
    James Greene

[Ivan De Marino]

I think you did an amazing job and managed to cover all the possible basis.

The small issues that you have clearly highlighted are passable to me: ultimately they are 2 different issue trackers.

Question 1: after migration, when do we shut down (or put in read only mode) the GCode one?
Question 2: do the issues in GCode get a link that points to the issue on GitHub in the comments?

Amazing work BTW...

--
You received this message because you are subscribed to the Google Groups "phantomjs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to phantomjs+...@googlegroups.com.
Visit this group at http://groups.google.com/group/phantomjs?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[James Greene]

Thanks, Ivan! Answers below.

Question 1: after migration, when do we shut down (or put in read only mode) the GCode one?

Not 100% positive but we would want to shut it down very soon afterward, hopefully the same day.  Google Code won't allow us to actually lock the issue tracker down (lame) so we will likely just have to update the "New Issue" templates to say something like "DON'T FILE THIS!  Go to https://github.com/ariya/phantomjs/issues".  Google Code will allow you to hide the "Issues" link for the project but you can still jump to its direct URL (which is good and bad).

I believe Ariya wants to leave the Google Code issue tracker there for historical purposes anyway, if I'm not mistaken, though I personally worry a bit about people still getting there via Google searches... which is why the updated issue templates and issue closures w/ migration links will be very important!

The more extreme alternative is deleting the whole Google Code project, which I think Ariya will not want to do.

Question 2: do the issues in GCode get a link that points to the issue on GitHub in the comments?

Yes, I already have another script that will add a new comment to every issue pointing to its equivalent issue on GitHub.  Any open issues will also be closed with a special status of "Migrated". Closed issues will keep their original status.  Much like the issue compilation, the issue closure process will also only take a few minutes.

---

If anyone wants to learn more about the conversion process, you can look at JamesMGreene/gc2gh-issue-migrator.  The two scripts for PhantomJS issue migration are under the "examples" folder, too.

Far warning, through, gc2gh-issue-migrator is pretty raw and ugly code... definitely not proud of the code but it yields useful results and I didn't want to blow any more time on it at this point, especially since the Google Code issue tracker API is being shutdown in June.

Sincerely,
    James Greene

[Ariya Hidayat]

> The more extreme alternative is deleting the whole Google Code project, which I think Ariya will not want to do.

Indeed. Since GitHub deprecated its download support, we still need
Google Code to host the source and binary packages.



--
Ariya Hidayat, http://ariya.ofilabs.com
http://twitter.com/ariyahidayat
http://gplus.to/ariyahidayat

[Ariya Hidayat]

Hi folks,

It's been a week and so far there is no objection on the issue
migration to GitHub. Knowing that James has been working on this for a
while, personally I'd reckon the outcome will be excellent. I am quite
picky on some aspects of the issue tracker and so far the dry-run
shows no problem whatsoever.

I'm pretty excited and I'd say we shall move forward with the
migration as soon as possible.

If any person can show just cause why we should not do this, speak now
or forever hold your peace.

Thank you!

Regards,

[James Greene]

Hey all!

Our issue tracker is now officially migrated to GitHub.  Please use https://github.com/ariya/phantomjs/issues for all issues, new and old. :)

Have a nice weekend!

Sincerely,
    James Greene

[Ariya Hidayat]

Thank you James for your hard work!

Meanwhile, I am also changing any links to the old tracker to point to
the new one. If you see some stale links, please let me know.

Also, some of you might get repo collaborator access so that you can
help managing the issues. Of course, more help will be always
welcomed!



Best regards,

[Ariya Hidayat]

From looking at the new issue tracker at GitHub, one glaring issue (no
pun intended) is the lack of reporting template. It's quite often that
I need to ask for more information, e.g. which PhantomJS version,
which OS version, etc. This is not a showstopper but it's nevertheless
annoying and time-wasting.

<rant>
Sometimes I can't stop wondering how a certain GitHub feature is
designed/rolled out. Between an issue reporting template and a Zen
writing mode (https://github.com/blog/1379-zen-writing-mode), it's an
easy choice for many project maintainers.
</rant>

A possible workaround would be to craft a special web page which
prepares all the fields (like a form). However, I'm sure a lot of
trigger-happy users will simply click on the inviting "Issue" link on
our GitHub project page and thus bypass that special issue reporting
page.

Any other ideas?

Thank you!

Regards,

[James Greene]

Users do get a "warning"-like link to the repo's CONTRIBUTING.md file when filing a new issue... they just choose to ignore it:

   

But yes, I fully agree that we would be better off if GitHub offered issue templates as well.  I've also suggested it to them in the past.

Sincerely,
    James Greene

[Ivan De Marino]

1) maybe we could close automatically what's not compliant
2) maybe we could use a bot (I saw one once)

[James Greene]

Just a random note: I submitted a bug for the Chromium project the other day (on Google Code, not GitHub) and I noticed that they have a "wizard" that walks you through bug details (and grabs some info automatically, like your browser, OS, etc.).  Pretty neat but obviously GitHub doesn't offer any such feature at the moment.

Sincerely,
    James Greene