BitBucket exit plan

[Matt Harbison]

With BitBucket getting rid of Mercurial hosting next month, the migration to Heptapod is nearly complete.  The repositories are not yet public, but will be hosted here:

    https://foss.heptapod.net/mercurial/thg

    https://foss.heptapod.net/mercurial/thg-shellext

    https://foss.heptapod.net/mercurial/thg-build

    https://foss.heptapod.net/mercurial/thg-winbuild

If there are other repos that need to be migrated, please speak up now.

The general plan is to reconvert the `thg` repo next weekend to pick up changes to the issue tracker since the trial run, and then be done with BitBucket.  The coming RC won't be built off of Heptapod, but the 5.4 release likely will.  The download story on www.mercurial-scm.org needs a little more work, but the plan is to have 5.4 hosted there.  (5.3 is currently there, but not advertised on the main page.)

Thanks to Clever Cloud and Octobus for donating the hosting infrastructure.

[Steve Borho]

Hi Folks, I was just about to send an email to the list to ask about this.

Let me know if there is anything you need from me in regard to the move. I can contact the fellow who is paying for the tortoisehg.org domain name to move the URL when the time is right. Do we have any hosting in mind for the web page?

I want to thank Matt a lot for picking up the slack on package building; I hadn’t booted my Windows build machine in about half a year. Do you still need a code signing key for your packages, or is there anything I can help with?

Looking at the current situation with Python3 and virtualenv and pip, I think there is less and less need for a “Batteries Included” binary package for Windows. If the thg project posted wheels with all the right required packages lists, users could pip install tortoisehg quite effectively. The only downside is the loss of shell extensions. I imagine the current binary installer could be trimmed down to only have the shell extension, then it could have its own very sparse release schedule.

Those are just my thoughts.. take them or leave them.

—
Steve Borho

[Matt Harbison]

On Monday, April 27, 2020 at 2:08:57 PM UTC-4, Steve Borho wrote:

Hi Folks, I was just about to send an email to the list to ask about this.

Let me know if there is anything you need from me in regard to the move.  I can contact the fellow who is paying for the tortoisehg.org domain name to move the URL when the time is right.  Do we have any hosting in mind for the web page?

I don't have anything in mind.  I wasn't sure where the webpages were even stored (I did finally find the bitbucket repo for it the other day).  I was thinking worst case scenario would be reducing it to just a wiki page on www.mercurial-scm.org, but it would be nice to keep the site if possible.  Not sure if there's a way to just host it on m-s.o.

I've uploaded a few recent installers to m-s.o. (And I'd kinda like to save the rest of them too, since they're basically impossible to rebuild.)  Augie was going to figure out the automation that updates the version file that offers the latest download, but I don't think he got to that yet.  We also need to figure out where to put the curversion.txt that is checked in the AboutBox.  Hopefully we can get this all figured out before cutting the 5.4 release, so it all just works going forward.  (Windows won't because we don't bundle the SSL libraries, but if there is a domain name that is being kept alive, maybe that's good enough for now.)  Maybe drop in on IRC and we can figure something out?

 

I want to thank Matt a lot for picking up the slack on package building; I hadn’t booted my Windows build machine in about half a year.  Do you still need a code signing key for your packages, or is there anything I can help with?

I don't have a code signing key.  I asked Augie if the Mercurial project was thinking of getting one, and he said they would probably have to get FSF involved.  But if we could figure out a short term way to handle the signing, that would avoid the scary dialog when launching the MSI.  I think long term the Mercurial project should get one, and also one from Apple since it's required for the notarization signing.

 

Looking at the current situation with Python3 and virtualenv and pip, I think there is less and less need for a “Batteries Included” binary package for Windows.  If the thg project posted wheels with all the right required packages lists, users could pip install tortoisehg quite effectively.  The only downside is the loss of shell extensions.  I imagine the current binary installer could be trimmed down to only have the shell extension, then it could have its own very sparse release schedule.

I did jettison hg-git, and tweaked the app to pick up things that are installed with `pip install --user`.  We should go over the extensions that are being bundled for py3 and get rid of the lesser used ones.  I'd have a slight preference for keeping things like evolve and keyring for simplicity.  The setup scripts at work are rather involved for Linux and Mac, but it's a pretty simple bat file on Windows once the MSI is done.  I'm hoping PyOxidizer greatly simplifies some of this stuff, but I haven't tried it yet since setup.py is still py2 only.  It probably could stand to drop the Qt4 stuff during that cleanup, and maybe we will need to care about Qt6 soon?

[Steve Borho]

> On Apr 27, 2020, at 10:15 PM, 'Matt Harbison' via TortoiseHg Developers <thg...@googlegroups.com> wrote:
>
>
> On Monday, April 27, 2020 at 2:08:57 PM UTC-4, Steve Borho wrote:
>
> Hi Folks, I was just about to send an email to the list to ask about this.
>
> Let me know if there is anything you need from me in regard to the move. I can contact the fellow who is paying for the tortoisehg.org domain name to move the URL when the time is right. Do we have any hosting in mind for the web page?
>
> I don't have anything in mind. I wasn't sure where the webpages were even stored (I did finally find the bitbucket repo for it the other day). I was thinking worst case scenario would be reducing it to just a wiki page on www.mercurial-scm.org, but it would be nice to keep the site if possible. Not sure if there's a way to just host it on m-s.o.

Do we know if Bitbucket is going to purge all the hg repos, or just make them read-only?

> I've uploaded a few recent installers to m-s.o. (And I'd kinda like to save the rest of them too, since they're basically impossible to rebuild.) Augie was going to figure out the automation that updates the version file that offers the latest download, but I don't think he got to that yet. We also need to figure out where to put the curversion.txt that is checked in the AboutBox. Hopefully we can get this all figured out before cutting the 5.4 release, so it all just works going forward. (Windows won't because we don't bundle the SSL libraries, but if there is a domain name that is being kept alive, maybe that's good enough for now.) Maybe drop in on IRC and we can figure something out?

Now is a good time to deprecate curversion.txt and move to latest.txt

>
> I want to thank Matt a lot for picking up the slack on package building; I hadn’t booted my Windows build machine in about half a year. Do you still need a code signing key for your packages, or is there anything I can help with?
>
> I don't have a code signing key. I asked Augie if the Mercurial project was thinking of getting one, and he said they would probably have to get FSF involved. But if we could figure out a short term way to handle the signing, that would avoid the scary dialog when launching the MSI. I think long term the Mercurial project should get one, and also one from Apple since it's required for the notarization signing.
>
> Looking at the current situation with Python3 and virtualenv and pip, I think there is less and less need for a “Batteries Included” binary package for Windows. If the thg project posted wheels with all the right required packages lists, users could pip install tortoisehg quite effectively. The only downside is the loss of shell extensions. I imagine the current binary installer could be trimmed down to only have the shell extension, then it could have its own very sparse release schedule.
>
> I did jettison hg-git, and tweaked the app to pick up things that are installed with `pip install --user`. We should go over the extensions that are being bundled for py3 and get rid of the lesser used ones. I'd have a slight preference for keeping things like evolve and keyring for simplicity. The setup scripts at work are rather involved for Linux and Mac, but it's a pretty simple bat file on Windows once the MSI is done. I'm hoping PyOxidizer greatly simplifies some of this stuff, but I haven't tried it yet since setup.py is still py2 only. It probably could stand to drop the Qt4 stuff during that cleanup, and maybe we will need to care about Qt6 soon?

There is also pyqt-deploy which emits similar single-file executables, but if Mercurial is focusing on Rust and PyOxidizer it makes sense for us to follow suit.

PyQt6 is supposed to be released very soon after Qt6 itself is released, so it would be possible to test those waters sooner than what happened with Qt5. I suppose it mostly depends on the nature of the API changes. It’s doubtful they will add any features that we absolutely need.

—
Steve

[Matt Harbison]

On Tuesday, April 28, 2020 at 2:13:31 PM UTC-4, Steve Borho wrote:

> On Apr 27, 2020, at 10:15 PM, 'Matt Harbison' via TortoiseHg Developers <thg...@googlegroups.com> wrote:
>
>
> On Monday, April 27, 2020 at 2:08:57 PM UTC-4, Steve Borho wrote:
>
> Hi Folks, I was just about to send an email to the list to ask about this.
>
> Let me know if there is anything you need from me in regard to the move.  I can contact the fellow who is paying for the tortoisehg.org domain name to move the URL when the time is right.  Do we have any hosting in mind for the web page?
>
> I don't have anything in mind.  I wasn't sure where the webpages were even stored (I did finally find the bitbucket repo for it the other day).  I was thinking worst case scenario would be reducing it to just a wiki page on www.mercurial-scm.org, but it would be nice to keep the site if possible.  Not sure if there's a way to just host it on m-s.o.

Do we know if Bitbucket is going to purge all the hg repos, or just make them read-only?

The BitBucket guy that stopped in IRC the day they announced said it would be more of a "soft delete" initially- there, but not accessible by the public.  So I don't think read-only is planned.  Pierre-Yves mentioned something about a project to archive all of BitBucket, so it may yet survive in a read-only state somewhere.  The hetpapod import needed some settings fiddled with, so I'm not sure what this archive will be able to grab without that.  It seems better not to leave it to chance, and I doubt we'll get help recovering things in the soft-deleted state given the hassle of convincing them to turn obsmarker exchange over http back on.

 

> I've uploaded a few recent installers to m-s.o. (And I'd kinda like to save the rest of them too, since they're basically impossible to rebuild.)  Augie was going to figure out the automation that updates the version file that offers the latest download, but I don't think he got to that yet.  We also need to figure out where to put the curversion.txt that is checked in the AboutBox.  Hopefully we can get this all figured out before cutting the 5.4 release, so it all just works going forward.  (Windows won't because we don't bundle the SSL libraries, but if there is a domain name that is being kept alive, maybe that's good enough for now.)  Maybe drop in on IRC and we can figure something out?

Now is a good time to deprecate curversion.txt and move to latest.txt

Do you know the URL offhand so I can curl it and look?  I'm all for eliminating duplication.

>
> I want to thank Matt a lot for picking up the slack on package building; I hadn’t booted my Windows build machine in about half a year.  Do you still need a code signing key for your packages, or is there anything I can help with?
>
> I don't have a code signing key.  I asked Augie if the Mercurial project was thinking of getting one, and he said they would probably have to get FSF involved.  But if we could figure out a short term way to handle the signing, that would avoid the scary dialog when launching the MSI.  I think long term the Mercurial project should get one, and also one from Apple since it's required for the notarization signing.
>  
> Looking at the current situation with Python3 and virtualenv and pip, I think there is less and less need for a “Batteries Included” binary package for Windows.  If the thg project posted wheels with all the right required packages lists, users could pip install tortoisehg quite effectively.  The only downside is the loss of shell extensions.  I imagine the current binary installer could be trimmed down to only have the shell extension, then it could have its own very sparse release schedule.
>
> I did jettison hg-git, and tweaked the app to pick up things that are installed with `pip install --user`.  We should go over the extensions that are being bundled for py3 and get rid of the lesser used ones.  I'd have a slight preference for keeping things like evolve and keyring for simplicity.  The setup scripts at work are rather involved for Linux and Mac, but it's a pretty simple bat file on Windows once the MSI is done.  I'm hoping PyOxidizer greatly simplifies some of this stuff, but I haven't tried it yet since setup.py is still py2 only.  It probably could stand to drop the Qt4 stuff during that cleanup, and maybe we will need to care about Qt6 soon?

There is also pyqt-deploy which emits similar single-file executables, but if Mercurial is focusing on Rust and PyOxidizer it makes sense for us to follow suit.

I played with that a little bit, hoping to avoid figuring out the right libraries to install into the right locations in the virtualenv used to build, but never got anything useful from it.  The mac build scripts do use the mac variant of qtdeploy, which I think we will want to do going forward as PyOxidizer doesn't support making .app (yet?).

 

PyQt6 is supposed to be released very soon after Qt6 itself is released, so it would be possible to test those waters sooner than what happened with Qt5.  I suppose it mostly depends on the nature of the API changes.  It’s doubtful they will add any features that we absolutely need.

I'm not aware of any things we'd need.  It's more of a chicken and egg problem where *not* cleaning up the Qt4 stuff is useful to identify what needs to be looked at to patch in Qt6, but it's also obsolete code that would be nice to ignore for py3 porting.

[Pierre-Yves David]

On 4/28/20 10:56 PM, 'Matt Harbison' via TortoiseHg Developers wrote:
>
> On Tuesday, April 28, 2020 at 2:13:31 PM UTC-4, Steve Borho wrote:
>
>
>
> > On Apr 27, 2020, at 10:15 PM, 'Matt Harbison' via TortoiseHg

> Developers <thg...@googlegroups.com <javascript:>> wrote:
> >
> >
> > On Monday, April 27, 2020 at 2:08:57 PM UTC-4, Steve Borho wrote:
> >
> > Hi Folks, I was just about to send an email to the list to ask
> about this.
> >
> > Let me know if there is anything you need from me in regard to
> the move.  I can contact the fellow who is paying for the

> tortoisehg.org <http://tortoisehg.org> domain name to move the URL

> when the time is right.  Do we have any hosting in mind for the web
> page?
> >
> > I don't have anything in mind.  I wasn't sure where the webpages
> were even stored (I did finally find the bitbucket repo for it the
> other day).  I was thinking worst case scenario would be reducing it
> to just a wiki page on www.mercurial-scm.org

> <http://www.mercurial-scm.org>, but it would be nice to keep the

> site if possible.  Not sure if there's a way to just host it on m-s.o.
>
> Do we know if Bitbucket is going to purge all the hg repos, or just
> make them read-only?
>
>
> The BitBucket guy that stopped in IRC the day they announced said it
> would be more of a "soft delete" initially- there, but not accessible by
> the public.

Do you have a copy of this conversation anywhere ?

> So I don't think read-only is planned.  Pierre-Yves
> mentioned something about a project to archive all of BitBucket, so it
> may yet survive in a read-only state somewhere.

This would hopefully be covered. more details here:

https://octobus.net/blog/2020-04-23-heptapod-and-swh.html


--
Pierre-Yves David

[Matt Harbison]

On Tuesday, April 28, 2020 at 6:25:43 PM UTC-4, Pierre-Yves David wrote:

On 4/28/20 10:56 PM, 'Matt Harbison' via TortoiseHg Developers wrote:
>
> The BitBucket guy that stopped in IRC the day they announced said it
> would be more of a "soft delete" initially- there, but not accessible by
> the public.

Do you have a copy of this conversation anywhere ?

Yes.  That day's log is ~92K, so I can email it to you if you want.  (I figure posting that attachment here would be frowned on)

 

> So I don't think read-only is planned.  Pierre-Yves
> mentioned something about a project to archive all of BitBucket, so it
> may yet survive in a read-only state somewhere.
This would hopefully be covered. more details here:

     https://octobus.net/blog/2020-04-23-heptapod-and-swh.html

Good info, thanks.  Any idea if the downloads page content is covered by the "collecting ... attachments" reference?  e.g. the installers on https://bitbucket.org/tortoisehg/files/downloads/  (I think it makes more sense to keep all of the downloads together in the new location, but in case I don't get to it in time...)

[Steve Borho]

> On Apr 28, 2020, at 3:56 PM, 'Matt Harbison' via TortoiseHg Developers <thg...@googlegroups.com> wrote:
>
>
> On Tuesday, April 28, 2020 at 2:13:31 PM UTC-4, Steve Borho wrote:
>
>
> > On Apr 27, 2020, at 10:15 PM, 'Matt Harbison' via TortoiseHg Developers <thg...@googlegroups.com> wrote:
> >
> >
> > On Monday, April 27, 2020 at 2:08:57 PM UTC-4, Steve Borho wrote:
> >
> > Hi Folks, I was just about to send an email to the list to ask about this.
> >
> > Let me know if there is anything you need from me in regard to the move. I can contact the fellow who is paying for the tortoisehg.org domain name to move the URL when the time is right. Do we have any hosting in mind for the web page?
> >
> > I don't have anything in mind. I wasn't sure where the webpages were even stored (I did finally find the bitbucket repo for it the other day). I was thinking worst case scenario would be reducing it to just a wiki page on www.mercurial-scm.org, but it would be nice to keep the site if possible. Not sure if there's a way to just host it on m-s.o.
>
> Do we know if Bitbucket is going to purge all the hg repos, or just make them read-only?
>
> The BitBucket guy that stopped in IRC the day they announced said it would be more of a "soft delete" initially- there, but not accessible by the public. So I don't think read-only is planned. Pierre-Yves mentioned something about a project to archive all of BitBucket, so it may yet survive in a read-only state somewhere. The hetpapod import needed some settings fiddled with, so I'm not sure what this archive will be able to grab without that. It seems better not to leave it to chance, and I doubt we'll get help recovering things in the soft-deleted state given the hassle of convincing them to turn obsmarker exchange over http back on.
>
> > I've uploaded a few recent installers to m-s.o. (And I'd kinda like to save the rest of them too, since they're basically impossible to rebuild.) Augie was going to figure out the automation that updates the version file that offers the latest download, but I don't think he got to that yet. We also need to figure out where to put the curversion.txt that is checked in the AboutBox. Hopefully we can get this all figured out before cutting the 5.4 release, so it all just works going forward. (Windows won't because we don't bundle the SSL libraries, but if there is a domain name that is being kept alive, maybe that's good enough for now.) Maybe drop in on IRC and we can figure something out?
>
> Now is a good time to deprecate curversion.txt and move to latest.txt
>
> Do you know the URL offhand so I can curl it and look? I'm all for eliminating duplication.

They are both hosted in the same location, in the root of the web site.

—
Steve

[Matt Harbison]

Is there a documented format somewhere?  When I curl it, I get:

10      5.0.2   .*Windows.*(WOW|x)64.*  http://bitbucket.org/tortoisehg/files/downloads/tortoisehg-5.0.2-x64.msi       TortoiseHg (including Mercurial) 5.0.2 - x64 Windows

I'm not sure what the 10 is or what's hidden behind the wildcards.  And we'll want to figure out how to get Linux, Mac, and Windows x32 shoehorned in there too.

[Pierre-Yves David]

That's the plan :-)

--
Pierre-Yves David

[Pierre-Yves David]

I think you can find document about this somewhere on the Mercurial
wiki. I remember learning about it in 2015 when I started to generate
rpm for centos.

On 4/30/20 1:07 AM, 'Matt Harbison' via TortoiseHg Developers wrote:
>
>
> On Wednesday, April 29, 2020 at 5:53:03 PM UTC-4, Steve Borho wrote:
>
>
>
> > On Apr 28, 2020, at 3:56 PM, 'Matt Harbison' via TortoiseHg

> Developers <thg...@googlegroups.com <javascript:>> wrote:
> >
> >
> > On Tuesday, April 28, 2020 at 2:13:31 PM UTC-4, Steve Borho wrote:
> >
>
> > Now is a good time to deprecate curversion.txt and move to
> latest.txt
> >
> > Do you know the URL offhand so I can curl it and look?  I'm all
> for eliminating duplication.
>
> They are both hosted in the same location, in the root of the web site.
>
>
> Is there a documented format somewhere?  When I curl it, I get:
>
> 10      5.0.2   .*Windows.*(WOW|x)64.*
> http://bitbucket.org/tortoisehg/files/downloads/tortoisehg-5.0.2-x64.msi
> TortoiseHg (including Mercurial) 5.0.2 - x64 Windows
>
> I'm not sure what the 10 is or what's hidden behind the wildcards.  And
> we'll want to figure out how to get Linux, Mac, and Windows x32
> shoehorned in there too.
>

> --
> You received this message because you are subscribed to the Google
> Groups "TortoiseHg Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to thg-dev+u...@googlegroups.com
> <mailto:thg-dev+u...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/thg-dev/abe420e8-b029-465a-a6ff-c29350f2caf1%40googlegroups.com
> <https://groups.google.com/d/msgid/thg-dev/abe420e8-b029-465a-a6ff-c29350f2caf1%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
Pierre-Yves David

[Steve Borho]

I believe it is a browser signature, so the web-page can auto-select the download targets.

> To unsubscribe from this group and stop receiving emails from it, send an email to thg-dev+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/thg-dev/6b042c24-4c0e-a900-633d-b3ef5532cd04%40ens-lyon.org.

[Matt Harbison]

On Monday, April 27, 2020 at 2:08:57 PM UTC-4, Steve Borho wrote:

Hi Folks, I was just about to send an email to the list to ask about this.

Let me know if there is anything you need from me in regard to the move.  I can contact the fellow who is paying for the tortoisehg.org domain name to move the URL when the time is right.  Do we have any hosting in mind for the web page?

I want to thank Matt a lot for picking up the slack on package building; I hadn’t booted my Windows build machine in about half a year.  Do you still need a code signing key for your packages, or is there anything I can help with?

Looking at the current situation with Python3 and virtualenv and pip, I think there is less and less need for a “Batteries Included” binary package for Windows.  If the thg project posted wheels with all the right required packages lists, users could pip install tortoisehg quite effectively.  The only downside is the loss of shell extensions.  I imagine the current binary installer could be trimmed down to only have the shell extension, then it could have its own very sparse release schedule.

Those are just my thoughts.. take them or leave them.

Steve,

Do you need the website repo[1] imported from bitbucket too?  Other than hosting the installers on mercurial-scm.org, I don't have any ideas about the website, other than maybe update it to point to the new hosting for the repo and downloads.  Even though at least some of the info on it is dated, it's probably worse to lose the site entirely.

At this point, all of the repos are imported (though the main one needs to be reimported to pick up bug tracker changes).  So we're pretty close to cutting the cord.

[1] https://bitbucket.org/kuy/thg-web/