Progressing with hg-git?

[Dan Villiom Podlaski Christiansen]

Hi all,

Since I was recently made co-maintainer of the project, I figured I’d share my thoughts for the future, and perhaps we could start a discussion on how to proceed.

My main perspective on hg-git is as a user; given that Git won the VCS “war” it’s rather unusual for me to clone actual Mercurial repositories — apart from those used for Mercurial-related projects. Instead, I use hg-git as a daily driver, both in my spare time and at work. In my experience, hg-git has some rather serious issues when used like that, and I’ve attempted to address them, in particular in one MR.[1]

Since my last message to list, most of my MRs received review and approval — and that's great! Unfortunately however, not much has happened since then, and I now have eight MRs pending again. I'd appreciate some feedback on how to proceed — should I just merge them without review? Should we add other maintainers or reviewers?

It seems to me that the changes already in — and some of those pending — contain useful bug fixes, and it would be nice to get a release out. Who has the keys to do so? Since the changes are relatively minor, I'd propose cutting a 0.9.1, and then getting started on 1.0. Some areas I'd like to improve hg-git are:

* Prune remote references, and outdated bookmarks while at it.

* Make a bare push safe; don't create branches/bookmarks unless requested by the user.

* Create binary archives for Ubuntu — or just binary Python wheels.

* Could we do a `git gc --auto` on pull?

* Can we transfer tags between Mercurial clones?

* Can we transfer the Git commit IDs/map/state between clones?

I'm not sure how to achieve the latter two, and they're sort of longer-term goals for me. But as part of planning a more significant release, I'd also propose dropping support for Python 2.7 and severely pruning the Mercurial versions supported.

Deeper changes, such as migrating away from bookmarks or optimising the core, would be nice, but I don't consider them all that important. In my experience, the core logic essentially works, and works quite well.

Anyway, I hope there's someone out there willing to discuss this 🙂

[1] https://foss.heptapod.net/mercurial/hg-git/-/merge_requests/42

--

Dan Villiom Podlaski Christiansen
dan...@gmail.com — +45 2728 9771

[Uwe Brauer]

>>> "DVPC" == Dan Villiom Podlaski Christiansen <dan...@gmail.com> writes:

> Hi all,
> Since I was recently made co-maintainer of the project, I figured I’d
> share my thoughts for the future, and perhaps we could start a
> discussion on how to proceed.

> My main perspective on hg-git is as a user; given that Git won the VCS
> “war” it’s rather unusual for me to clone actual Mercurial
> repositories — apart from those used for Mercurial-related projects.

That is a rather sad story especially now, that evolve is here with
features such as topics

> * Prune remote references, and outdated bookmarks while at it.

> * Make a bare push safe; don't create branches/bookmarks unless requested by the user.

> * Create binary archives for Ubuntu — or just binary Python wheels.

> * Could we do a `git gc --auto` on pull?

> * Can we transfer tags between Mercurial clones?

> * Can we transfer the Git commit IDs/map/state between clones?

I'd like just to add two changesets that would be great if they could be
merged into stable

1. acc4280e2ab7


2. 3ad09bbcbe3e

From June 2019. They add support of named_branches to hg-git, but alas
are sitting on the (topic) branch export-additional-refs, and
same-ref-for-multiple-changesets

Regards

Uwe Brauer

[Georges Racinet]

Hi Dan,

On 12/20/20 8:36 PM, Dan Villiom Podlaski Christiansen wrote:
> Since I was recently made co-maintainer of the project, I figured I’d share my thoughts for the future, and perhaps we could start a discussion on how to proceed.
>
> My main perspective on hg-git is as a user; given that Git won the VCS “war” it’s rather unusual for me to clone actual Mercurial repositories — apart from those used for Mercurial-related projects. Instead, I use hg-git as a daily driver, both in my spare time and at work. In my experience, hg-git has some rather serious issues when used like that, and I’ve attempted to address them, in particular in one MR.[1]
>
> Since my last message to list, most of my MRs received review and approval — and that's great! Unfortunately however, not much has happened since then, and I now have eight MRs pending again. I'd appreciate some feedback on how to proceed — should I just merge them without review?

Since the work you've done on the CI setup, I've been busy full time
with Heptapod development, in particular to meet tomorrow's deadline for
version 0.18.0. Now that is almost done: the pipeline doing that release
is running as I'm writing this.

I should be able to do some hg-git work tomorrow after install on
foss.heptapod.net and on Tuesday. I know, 10 days ago I was saying
something alike, but there was acceleration with respect to some subjects.

> Should we add other maintainers or reviewers?

We should certainly avoid a repeating pattern of people stepping up
(like you did) and getting tired. Not saying we are there right now, of
course. I am personally open to a bigger pool of maintainers.

> It seems to me that the changes already in — and some of those pending — contain useful bug fixes, and it would be nice to get a release out. Who has the keys to do so?

Currently, Kevin Bullock, Manuel Jacob and me.

I've gained the ability to appoint people on PyPI 6 weeks ago, and
transferred it to Manuel immediately.

> Since the changes are relatively minor, I'd propose cutting a 0.9.1,

Yes, I can do that.

> and then getting started on 1.0. Some areas I'd like to improve hg-git are:
>
> * Prune remote references, and outdated bookmarks while at it.
>
> * Make a bare push safe; don't create branches/bookmarks unless requested by the user.
>
> * Create binary archives for Ubuntu — or just binary Python wheels.

As far as I understand, the question of wheels would be fairly trivial
for hg-git itself, since it is pure Python. But Mercurial does not
publish wheels yet (the question was raised I believe a few times), and
I don't know about dulwich (it has a small amount of C code, too).

> * Could we do a `git gc --auto` on pull?

It seems that the issue for Git garbage collection in dulwich [2] is
still open. But it's very old, so it's not obvious it's still relevant.

> * Can we transfer tags between Mercurial clones?

There was a brainstorming about tags at the November Mercurial sprint.
Maybe new possibilities will arise from there and/or the landscape will
change enough that we need to keep an eye on that, but I'll have to
check if something concrete is in the works.

> * Can we transfer the Git commit IDs/map/state between clones?
>
> I'm not sure how to achieve the latter two, and they're sort of longer-term goals for me.

The last one includes what is in my eyes the most important issue:
changeset stability in the Git to Mercurial conversion. In practice,
that turns hg-git users into isolated islands. I don't want to give you
false hopes, but there's a tiny tiny chance I'd have a lead on that –
call that a hunch at this point.

Exchanging the map would be a mere nice-to-have for comparison, useful
only for bigger repos, I suppose.

I'd like to introduce testing with my mercurial-testhelpers [1] library,
as I think it would help solving these deep questions. Basically, it
should be efficient for hg-git for the same reasons it is for Heptapod
components. For example, last week, I've written a pytest fixture [3]
that manages a pair of Mercurial and Git repos. It could easily be used
in local pull/push scenarios.

> But as part of planning a more significant release, I'd also propose dropping support for Python 2.7 and severely pruning the Mercurial versions supported.

I would be supportive of that move if you feel so, and it really makes
sense if the project maintainers are already too busy.

That being said, I'd like to hear wider feedback on that, in particular,
what would be the minimal version that wouldn't cut off our user base?

> Deeper changes, such as migrating away from bookmarks or optimising the core, would be nice, but I don't consider them all that important. In my experience, the core logic essentially works, and works quite well.
>
> Anyway, I hope there's someone out there willing to discuss this 🙂

I certainly am.

Thanks for all the work you've been doing so far.

Best,


[1] https://pypi.org/project/mercurial-testhelpers/

[2] https://github.com/dulwich/dulwich/issues/92

[3]
https://foss.heptapod.net/heptapod/py-heptapod/-/blob/branch/default/heptapod/testhelpers/gitlab.py#L40
The GitLab specifics are very easy to cut off if we wanted to make a
version for hg-git.

Sample usage:
https://foss.heptapod.net/heptapod/py-heptapod/-/blob/branch/default/hgext3rd/heptapod/tests/git/test_integration.py#L86
and #L121

--
Georges Racinet
https://octobus.net, https://about.heptapod.host, https://heptapod.net
GPG: BF5456F4DC625443849B6E58EE20CA44EF691D39, sur serveurs publics

[Pierre-Yves David]

On 12/20/20 8:36 PM, Dan Villiom Podlaski Christiansen wrote:

> * Can we transfer tags between Mercurial clones?
>
> * Can we transfer the Git commit IDs/map/state between clones?
>
> I'm not sure how to achieve the latter two, and they're sort of longer-term goals for me.

As long as we are talking about exchange between two peers who enables
hg-git, exchanging those should be as simple as shoving the data inside
some dedicated bundle2 parts. I can provide pointer if you need some.

> But as part of planning a more significant release, I'd also propose dropping support for Python 2.7 and severely pruning the Mercurial versions supported.

If I remember correctly, the main mercurial project is planning to drop
2.7 as soon as TortoiseHG is py3 ready, and that was mostly packaging
concerns when we talked about it last month. Matt Harbison will know
what's the status of this is.

For the evolve extensions, my policy is to keep at least 1 years of
compat (4 Mercurial versions). That support often expand further as I
drop >1 years versions when compatibility requires a significant amount
of work.

> Deeper changes, such as migrating away from bookmarks or optimising the core, would be nice, but I don't consider them all that important. In my experience, the core logic essentially works, and works quite well.

I am curious about your plan regarding migrating away from bookmarks.

Cheers,

--
Pierre-Yves David

[Dan Villiom Podlaski Christiansen]

On 21 Dec 2020, at 07.41, Pierre-Yves David <pierre-y...@ens-lyon.org> wrote:

> On 12/20/20 8:36 PM, Dan Villiom Podlaski Christiansen wrote:
>
>> * Can we transfer tags between Mercurial clones?
>> * Can we transfer the Git commit IDs/map/state between clones?
>> I'm not sure how to achieve the latter two, and they're sort of longer-term goals for me.
>
> As long as we are talking about exchange between two peers who enables hg-git, exchanging those should be as simple as shoving the data inside some dedicated bundle2 parts. I can provide pointer if you need some.

Please do — it might well be some time before I look into this, but any pointers would at least allow me to start thinking about it 🙂

Exchanging tags might require help from core Mercurial. Although non-versioned tags feel like an anti-feature and regression compared to the proper, versioned, well-behaved tags supported by Mercurial,[1] the simple fact is that they're the only kind of tag supported by Git. So it would be nice to have something equivalent in Mercurial, even if as an extension.

>> But as part of planning a more significant release, I'd also propose dropping support for Python 2.7 and severely pruning the Mercurial versions supported.
>
> If I remember correctly, the main mercurial project is planning to drop 2.7 as soon as TortoiseHG is py3 ready, and that was mostly packaging concerns when we talked about it last month. Matt Harbison will know what's the status of this is.
>
> For the evolve extensions, my policy is to keep at least 1 years of compat (4 Mercurial versions). That support often expand further as I drop >1 years versions when compatibility requires a significant amount of work.

The current policy, as it was stated in the Makefile and now .gitlab-ci.yml, is only up to the latest Ubuntu LTS. Since that's just this April, I think I'd settle for a compromise: One year or the latest Ubuntu LTS, whichever is older.

It sounds like TortoiseHg will be addressed “soon”, so I'd say dropping Python 2.7 for 1.0 makes sense.

>> Deeper changes, such as migrating away from bookmarks or optimising the core, would be nice, but I don't consider them all that important. In my experience, the core logic essentially works, and works quite well.
>
> I am curious about your plan regarding migrating away from bookmarks.

It's not so much a plan as a wish: Topics are nicer to work with, and might allow stuff like non-fast-forward merges with Git “branches” — as requested by Uwe Brauer on this list.[2] But they're not the only way to address that, and cause issues with stuff like bi-directionality. However, that leads me to another long-term feature I'd love to see:

* Heuristics for automatic obsolescence of force-pushed branches.

That'd be a killer feature in my book 🙂

[1] I don't like Git's stupid non-branches and non-tags. They remind me of Subversion in a way, and just feel like a regression compared even to CVS… </rant>
[2] https://groups.google.com/g/hg-git/c/A4K4UCkWgTY/m/toHbqwNsAAAJ

[Dan Villiom Podlaski Christiansen]

On 20 Dec 2020, at 23.05, Georges Racinet <georges...@octobus.net> wrote:
>
> Hi Dan,
>
> On 12/20/20 8:36 PM, Dan Villiom Podlaski Christiansen wrote:
>
>> Since I was recently made co-maintainer of the project, I figured I’d share my thoughts for the future, and perhaps we could start a discussion on how to proceed.
>>
>> My main perspective on hg-git is as a user; given that Git won the VCS “war” it’s rather unusual for me to clone actual Mercurial repositories — apart from those used for Mercurial-related projects. Instead, I use hg-git as a daily driver, both in my spare time and at work. In my experience, hg-git has some rather serious issues when used like that, and I’ve attempted to address them, in particular in one MR.[1]
>>
>> Since my last message to list, most of my MRs received review and approval — and that's great! Unfortunately however, not much has happened since then, and I now have eight MRs pending again. I'd appreciate some feedback on how to proceed — should I just merge them without review?
>
> Since the work you've done on the CI setup, I've been busy full time
> with Heptapod development, in particular to meet tomorrow's deadline for
> version 0.18.0. Now that is almost done: the pipeline doing that release
> is running as I'm writing this.
>
> I should be able to do some hg-git work tomorrow after install on
> foss.heptapod.net and on Tuesday. I know, 10 days ago I was saying
> something alike, but there was acceleration with respect to some subjects.

I know we all have day jobs and that hg-git usually isn't part of it, but it is a bit frustrating to wait a week or more without any response at all. If you could spare the time, something like a confirmation that you've seen the MR or a high-level comment would be much appreciated!

>> Should we add other maintainers or reviewers?
>
> We should certainly avoid a repeating pattern of people stepping up
> (like you did) and getting tired. Not saying we are there right now, of
> course. I am personally open to a bigger pool of maintainers.

I don't if they're on this list, or even interested, but muxator has been providing good contributions and feedback. Perhaps that's another candidate.

>> It seems to me that the changes already in — and some of those pending — contain useful bug fixes, and it would be nice to get a release out. Who has the keys to do so?
>
> Currently, Kevin Bullock, Manuel Jacob and me.
>
> I've gained the ability to appoint people on PyPI 6 weeks ago, and
> transferred it to Manuel immediately.
>
>> Since the changes are relatively minor, I'd propose cutting a 0.9.1,
> Yes, I can do that.
>> and then getting started on 1.0. Some areas I'd like to improve hg-git are:
>>
>> * Prune remote references, and outdated bookmarks while at it.
>>
>> * Make a bare push safe; don't create branches/bookmarks unless requested by the user.
>>
>> * Create binary archives for Ubuntu — or just binary Python wheels.
> As far as I understand, the question of wheels would be fairly trivial
> for hg-git itself, since it is pure Python. But Mercurial does not
> publish wheels yet (the question was raised I believe a few times), and
> I don't know about dulwich (it has a small amount of C code, too).

I opened an issue with Dulwich — it has Windows wheels, but nothing for Linux. The only discussion I could find relating to Mercurial was from 2017, though.[*] The LaunchPad PPA also seems quite out of date[†] — that was my go-to solution previously.

AFAICT GitLab (and thus Heptapod?) actually has support for PyPI repositories.[‡] Perhaps enabling that as an interim solution would work? Then packages and wheels could be manually published to PyPI.

[*] https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-July/100963.html
[†] https://launchpad.net/~mercurial-ppa/+archive/ubuntu/releases
[‡] https://foss.heptapod.net/help/user/packages/pypi_repository/index.md

>> * Could we do a `git gc --auto` on pull?
> It seems that the issue for Git garbage collection in dulwich [2] is
> still open. But it's very old, so it's not obvious it's still relevant.

Not even libgit2 has `git gc`,[*] so I think it's just too much to ask of Dulwich. I was thinking of checking whether the `git` command was available, and then invoking it after a pull. For large repositories — those containing more than 10k files or commits, for example — we could then add a suppressible warning in its absence. I've personally run into this, and the failure mode is rather annoying: In some cases, Dulwich just hangs.

[*] https://foss.heptapod.net/mercurial/hg-git/-/merge_requests/63

>> * Can we transfer tags between Mercurial clones?
> There was a brainstorming about tags at the November Mercurial sprint.
> Maybe new possibilities will arise from there and/or the landscape will
> change enough that we need to keep an eye on that, but I'll have to
> check if something concrete is in the works.
>> * Can we transfer the Git commit IDs/map/state between clones?
>>
>> I'm not sure how to achieve the latter two, and they're sort of longer-term goals for me.
>
> The last one includes what is in my eyes the most important issue:
> changeset stability in the Git to Mercurial conversion. In practice,
> that turns hg-git users into isolated islands. I don't want to give you
> false hopes, but there's a tiny tiny chance I'd have a lead on that –
> call that a hunch at this point.
>
> Exchanging the map would be a mere nice-to-have for comparison, useful
> only for bigger repos, I suppose.

I have some automatic conversions set up on a server of my own. In most cases, the conversions are indeed stable, but it would be nice to be able to transfer the Git identity with a commit. A personal goal for me is to have mercurial+hg-git reach a state where I could say to co-workers: “Yes, you really should use Mercurial; it's a better Git client than git.” For that to be the case, two clones should _always_ have the same Git state, regardless of configuration, etc., and I don't see how we can provide that guarantee without transferring the commit map.

> I'd like to introduce testing with my mercurial-testhelpers [1] library,
> as I think it would help solving these deep questions. Basically, it
> should be efficient for hg-git for the same reasons it is for Heptapod
> components. For example, last week, I've written a pytest fixture [3]
> that manages a pair of Mercurial and Git repos. It could easily be used
> in local pull/push scenarios.

To be honest, I don't really have any issues with the current testsuite which, unlike the Mercurial tests, do run reasonably quickly, even on a Mac. But making testing easier is always good 🙂

Speaking of which, you might want to look at MR 63[*] — to ensure that cloning over the network, actually, properly worked, I added a test using a CI service exposing a Git repository over the Git, SSH and HTTP protocols, the latter with authentication.

[*] https://github.com/libgit2/libgit2/issues/3247

>> But as part of planning a more significant release, I'd also propose dropping support for Python 2.7 and severely pruning the Mercurial versions supported.
>
> I would be supportive of that move if you feel so, and it really makes
> sense if the project maintainers are already too busy.
>
> That being said, I'd like to hear wider feedback on that, in particular,
> what would be the minimal version that wouldn't cut off our user base?
>
>> Deeper changes, such as migrating away from bookmarks or optimising the core, would be nice, but I don't consider them all that important. In my experience, the core logic essentially works, and works quite well.
>>
>> Anyway, I hope there's someone out there willing to discuss this 🙂
>
> I certainly am.
>
> Thanks for all the work you've been doing so far.

And likewise on Heptapod — it's great 🙂

> Best,
>
>
> [1] https://pypi.org/project/mercurial-testhelpers/
>
> [2] https://github.com/dulwich/dulwich/issues/92
>
> [3]
> https://foss.heptapod.net/heptapod/py-heptapod/-/blob/branch/default/heptapod/testhelpers/gitlab.py#L40
> The GitLab specifics are very easy to cut off if we wanted to make a
> version for hg-git.
>
> Sample usage:
> https://foss.heptapod.net/heptapod/py-heptapod/-/blob/branch/default/hgext3rd/heptapod/tests/git/test_integration.py#L86
> and #L121

--

[Sune Foldager]

Hi Dan

Long time ;)

We actually use hg-git at work for important mirroring, and I’ve put a few patches on top of (what was then) upstream, in order to make it work for us. They are roughly (as far as I remember):

- Fix stuff so “æøå” file names work on Windows (the usual :p). It doesn’t out of the box.
- Add filtering so not all git refs are converted.
- Add a way to point to a different git directory.
- Add a way to skip tags.
- Very important: add tracking of the sha a commit came from on each side, and never convert something from X to Y, if it already came from Y.

I *think* that’s correct, but I’ll go take a closer look later today.

The last point in the last above is very important for us since we convert both ways and the conversion isn’t entirely “double idempotent”. It is for regular commits, but unfortunately not for merges. Adding that feature, of course, means that several features trying to deal with this are not needed. It doesn’t work, and one obvious problem is the weird handling of removed files in merges that hg has.

- Sune

[Dan Villiom Podlaski Christiansen]

On 21 Dec 2020, at 12.40, 'Sune Foldager' via hg-git <hg-...@googlegroups.com> wrote:

> Hi Dan
>
> Long time ;)

Indeed! I'm guessing you're as surprised I'm still active as I am that you are 😄

> We actually use hg-git at work for important mirroring, and I’ve put a few patches on top of (what was then) upstream, in order to make it work for us. They are roughly (as far as I remember):

If you're able, please do create merge requests for all those! Alternatively, perhaps you could publish your changes somewhere, such as on Heptapod or Sourcehut?

> - Fix stuff so “æøå” file names work on Windows (the usual :p). It doesn’t out of the box.

Ooh, this should go into 0.9.1!

> - Add filtering so not all git refs are converted.

Do want — that'd enable me to convert hg-git itself.

> - Add a way to point to a different git directory.

I think that's already supported?

> - Add a way to skip tags.

Sure, why not?

> - Very important: add tracking of the sha a commit came from on each side, and never convert something from X to Y, if it already came from Y.

I'm interested in how you achieved that — are you embedding the commit IDs into the extras or the message? How does that work with history editing, on either side?

> I *think* that’s correct, but I’ll go take a closer look later today.
>
> The last point in the last above is very important for us since we convert both ways and the conversion isn’t entirely “double idempotent”. It is for regular commits, but unfortunately not for merges. Adding that feature, of course, means that several features trying to deal with this are not needed. It doesn’t work, and one obvious problem is the weird handling of removed files in merges that hg has.

Yeah, I can image. It's the thing holding me from recommending hg-git at work. Personally, I think making hg-git truly idempotent is a nice goal that's utterly unachievable in practice.

[Georges Racinet]

On 12/21/20 11:46 AM, Dan Villiom Podlaski Christiansen wrote:
> On 20 Dec 2020, at 23.05, Georges Racinet <georges...@octobus.net> wrote:
>> Hi Dan,
>>
>> On 12/20/20 8:36 PM, Dan Villiom Podlaski Christiansen wrote:
>>
>>> Since I was recently made co-maintainer of the project, I figured I’d share my thoughts for the future, and perhaps we could start a discussion on how to proceed.
>>>
>>> My main perspective on hg-git is as a user; given that Git won the VCS “war” it’s rather unusual for me to clone actual Mercurial repositories — apart from those used for Mercurial-related projects. Instead, I use hg-git as a daily driver, both in my spare time and at work. In my experience, hg-git has some rather serious issues when used like that, and I’ve attempted to address them, in particular in one MR.[1]
>>>
>>> Since my last message to list, most of my MRs received review and approval — and that's great! Unfortunately however, not much has happened since then, and I now have eight MRs pending again. I'd appreciate some feedback on how to proceed — should I just merge them without review?
>> Since the work you've done on the CI setup, I've been busy full time
>> with Heptapod development, in particular to meet tomorrow's deadline for
>> version 0.18.0. Now that is almost done: the pipeline doing that release
>> is running as I'm writing this.
>>
>> I should be able to do some hg-git work tomorrow after install on
>> foss.heptapod.net and on Tuesday. I know, 10 days ago I was saying
>> something alike, but there was acceleration with respect to some subjects.
> I know we all have day jobs and that hg-git usually isn't part of it, but it is a bit frustrating to wait a week or more without any response at all. If you could spare the time, something like a confirmation that you've seen the MR or a high-level comment would be much appreciated!

Yes, I'll try and do better with that.

>
>>> Should we add other maintainers or reviewers?
>> We should certainly avoid a repeating pattern of people stepping up
>> (like you did) and getting tired. Not saying we are there right now, of
>> course. I am personally open to a bigger pool of maintainers.
> I don't if they're on this list, or even interested, but muxator has been providing good contributions and feedback. Perhaps that's another candidate.

Noted, thanks.

>
>>> It seems to me that the changes already in — and some of those pending — contain useful bug fixes, and it would be nice to get a release out. Who has the keys to do so?
>> Currently, Kevin Bullock, Manuel Jacob and me.
>>
>> I've gained the ability to appoint people on PyPI 6 weeks ago, and
>> transferred it to Manuel immediately.
>>
>>> Since the changes are relatively minor, I'd propose cutting a 0.9.1,
>> Yes, I can do that.
>>> and then getting started on 1.0. Some areas I'd like to improve hg-git are:
>>>
>>> * Prune remote references, and outdated bookmarks while at it.
>>>
>>> * Make a bare push safe; don't create branches/bookmarks unless requested by the user.
>>>
>>> * Create binary archives for Ubuntu — or just binary Python wheels.
>> As far as I understand, the question of wheels would be fairly trivial
>> for hg-git itself, since it is pure Python. But Mercurial does not
>> publish wheels yet (the question was raised I believe a few times), and
>> I don't know about dulwich (it has a small amount of C code, too).
> I opened an issue with Dulwich — it has Windows wheels, but nothing for Linux. The only discussion I could find relating to Mercurial was from 2017, though.[*] The LaunchPad PPA also seems quite out of date[†] — that was my go-to solution previously.
>
> AFAICT GitLab (and thus Heptapod?) actually has support for PyPI repositories.[‡]

Yes, it should be rather orthogonal to the changes we've done for
Heptapod, hence I hope it'd work right away, but I haven't tried it yet.

> Perhaps enabling that as an interim solution would work? Then packages and wheels could be manually published to PyPI.

If you have in mind something fully automatic based on tag pipelines,
there's also the question of PGP signatures. Previous releases were
signed with the `gpg` extension, and I've done the same for the ones I
did. Fortunately, the changeset with the signature is after the tag, so
that we can tackle the two questions of pushing a package and of the
signature separately.

> [*] https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-July/100963.html
> [†] https://launchpad.net/~mercurial-ppa/+archive/ubuntu/releases
> [‡] https://foss.heptapod.net/help/user/packages/pypi_repository/index.md
>
>>> * Could we do a `git gc --auto` on pull?
>> It seems that the issue for Git garbage collection in dulwich [2] is
>> still open. But it's very old, so it's not obvious it's still relevant.
> Not even libgit2 has `git gc`,[*] so I think it's just too much to ask of Dulwich. I was thinking of checking whether the `git` command was available, and then invoking it after a pull. For large repositories — those containing more than 10k files or commits, for example — we could then add a suppressible warning in its absence.

I'm not against using the `git` executable for this, and was actually
wondering if you had that in mind. It's nice for hg-git to work without
Git, but it probably shouldn't prevent doing some additional stuff if
`git` is actually available. That's fairly common after all.

To state the obvious, in the case of GC, it should be controlled by
dedicated options. For example, in GitLab (hence Heptapod), Git garbage
collecting is already taken care of by scheduled background jobs with a
locking system to present concurrent writes.

> I've personally run into this, and the failure mode is rather annoying: In some cases, Dulwich just hangs.

Just to make sure: you mean that because presumably of the size of the
`.hg/git` repo, hg-git stopped working at some point?

>
> [*] https://foss.heptapod.net/mercurial/hg-git/-/merge_requests/63
>
>>> * Can we transfer tags between Mercurial clones?
>> There was a brainstorming about tags at the November Mercurial sprint.
>> Maybe new possibilities will arise from there and/or the landscape will
>> change enough that we need to keep an eye on that, but I'll have to
>> check if something concrete is in the works.
>>> * Can we transfer the Git commit IDs/map/state between clones?
>>>
>>> I'm not sure how to achieve the latter two, and they're sort of longer-term goals for me.
>> The last one includes what is in my eyes the most important issue:
>> changeset stability in the Git to Mercurial conversion. In practice,
>> that turns hg-git users into isolated islands. I don't want to give you
>> false hopes, but there's a tiny tiny chance I'd have a lead on that –
>> call that a hunch at this point.
>>
>> Exchanging the map would be a mere nice-to-have for comparison, useful
>> only for bigger repos, I suppose.
> I have some automatic conversions set up on a server of my own. In most cases, the conversions are indeed stable,

In the cases I care about in the Git -> Mercurial direction, it's never
been stable but these are heavy and complicated repos (GitLab…) and I've
been backuping those map files systematically after I noticed this,
hence sidestepped the problem.

> but it would be nice to be able to transfer the Git identity with a commit. A personal goal for me is to have mercurial+hg-git reach a state where I could say to co-workers: “Yes, you really should use Mercurial; it's a better Git client than git.” For that to be the case, two clones should _always_ have the same Git state, regardless of configuration, etc., and I don't see how we can provide that guarantee without transferring the commit map.

Yes, perhaps you're right and doing that is the only way that can be
robust enough, but it raises a few questions of its own: the map file is
mostly appending – except there's a command to clean it. As you were
saying, it's a longer term goal.

>
>> I'd like to introduce testing with my mercurial-testhelpers [1] library,
>> as I think it would help solving these deep questions. Basically, it
>> should be efficient for hg-git for the same reasons it is for Heptapod
>> components. For example, last week, I've written a pytest fixture [3]
>> that manages a pair of Mercurial and Git repos. It could easily be used
>> in local pull/push scenarios.
> To be honest, I don't really have any issues with the current testsuite which, unlike the Mercurial tests, do run reasonably quickly, even on a Mac. But making testing easier is always good 🙂

Yes, in the case of hg-git, running time is not the issue. It's more
about being at the heart of things rather than seeing all the action
from a shell script.

>
> Speaking of which, you might want to look at MR 63[*] — to ensure that cloning over the network, actually, properly worked, I added a test using a CI service exposing a Git repository over the Git, SSH and HTTP protocols, the latter with authentication.

Sound nice

>
> [*] https://github.com/libgit2/libgit2/issues/3247
>
>>> But as part of planning a more significant release, I'd also propose dropping support for Python 2.7 and severely pruning the Mercurial versions supported.
>> I would be supportive of that move if you feel so, and it really makes
>> sense if the project maintainers are already too busy.
>>
>> That being said, I'd like to hear wider feedback on that, in particular,
>> what would be the minimal version that wouldn't cut off our user base?
>>
>>> Deeper changes, such as migrating away from bookmarks or optimising the core, would be nice, but I don't consider them all that important. In my experience, the core logic essentially works, and works quite well.
>>>
>>> Anyway, I hope there's someone out there willing to discuss this 🙂
>> I certainly am.
>>
>> Thanks for all the work you've been doing so far.
> And likewise on Heptapod — it's great 🙂

:-)

Let's get on with it, now.

>
>> Best,
>>
>>
>> [1] https://pypi.org/project/mercurial-testhelpers/
>>
>> [2] https://github.com/dulwich/dulwich/issues/92
>>
>> [3]
>> https://foss.heptapod.net/heptapod/py-heptapod/-/blob/branch/default/heptapod/testhelpers/gitlab.py#L40
>> The GitLab specifics are very easy to cut off if we wanted to make a
>> version for hg-git.
>>
>> Sample usage:
>> https://foss.heptapod.net/heptapod/py-heptapod/-/blob/branch/default/hgext3rd/heptapod/tests/git/test_integration.py#L86
>> and #L121
> --
>
> Dan Villiom Podlaski Christiansen
> dan...@gmail.com — +45 2728 9771
>

[Pierre Augier]

From my point of view of simple user and of someone who needs to teach how to use version control to students and colleagues:

Le dim. 20 déc. 2020 à 20:36, Dan Villiom Podlaski Christiansen <dan...@gmail.com> a écrit :

Some areas I'd like to improve hg-git are:

* Prune remote references, and outdated bookmarks while at it.

Yes, I agree it would be useful!

I'd add two simple issues

1. A very common issue encountered by beginners that don't know about branches / bookmarks is related to the fact that the master bookmark is not activated by default after clone. They clone, create some commits and push. They don't understand at all why Mercurial tells them that there are no new changes. https://foss.heptapod.net/mercurial/hg-git/-/issues/293 It would be good that people do not encounter this problem just at the beginning of their discovery of using hg to interact with Github/Gitlab. I guess a very clear and simple message would be enough.

2. The behaviour when working with 2 remotes is not perfect. Since it is a very standard workflow for many open-source projects hosted on Github, it would be nice to fix https://foss.heptapod.net/mercurial/hg-git/-/issues/334

 

Deeper changes, such as migrating away from bookmarks or optimising the core, would be nice, but I don't consider them all that important. In my experience, the core logic essentially works, and works quite well.

When Heptapod will be more mature (so that students could learn version control with Heptapod instances of their university), it would be much better in terms of user experience to be able to also use topics for Git branches. They can learn with hg (and topics) and use the same commands/notions to interact with the Git world.

 

Anyway, I hope there's someone out there willing to discuss this 🙂

[1] https://foss.heptapod.net/mercurial/hg-git/-/merge_requests/42

--

Dan Villiom Podlaski Christiansen
dan...@gmail.com — +45 2728 9771

--
You received this message because you are subscribed to the Google Groups "hg-git" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hg-git+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hg-git/5FA09A61-8F66-4F55-8AF7-65389D6B475F%40gmail.com.