Read Only Subrepos?

50 views
Skip to first unread message

Ryan Wilcox

unread,
Aug 7, 2010, 9:57:52 AM8/7/10
to merc...@selenic.com
Hello everyone,

I have set up a number of subrepos in my repository. These subrepos
point to (open source) repositories where I have read only access.

I made some changes in my code, and I did an hg push:

$ hg push
pushing to ssh://h...@bitbucket.org/rwilcox/dotfiles/
pushing subrepo vim/bundles/fugitive
...
abort: the remote end hung up unexpectedly


Is there way to configure Mercurial to not try to push those subrepos
by default?

Thanks!
_Ryan Wilcox

--
_____________________________________________________________________
Ryan Wilcox             PGP: 0x2F4E9C31
http://flavors.me/rwilcox
_______________________________________________
Mercurial mailing list
Merc...@selenic.com
http://selenic.com/mailman/listinfo/mercurial

Matt Mackall

unread,
Aug 9, 2010, 1:35:53 PM8/9/10
to Ryan Wilcox, merc...@selenic.com
On Sat, 2010-08-07 at 09:57 -0400, Ryan Wilcox wrote:
> Hello everyone,
>
> I have set up a number of subrepos in my repository. These subrepos
> point to (open source) repositories where I have read only access.
>
> I made some changes in my code, and I did an hg push:
>
> $ hg push
> pushing to ssh://h...@bitbucket.org/rwilcox/dotfiles/
> pushing subrepo vim/bundles/fugitive
> ...
> abort: the remote end hung up unexpectedly
>
>
> Is there way to configure Mercurial to not try to push those subrepos
> by default?

Nope.

When you commit at the top-level, you're taking a snapshot of a coherent
project state. That includes the state of your subrepos. If you push the
top level and not the corresponding subrepos, you'll get something
incoherent on the other side - in fact something hg will refuse to check
out.

--
Mathematics is the supreme nostalgia of our time.

Steve Losh

unread,
Aug 9, 2010, 1:54:09 PM8/9/10
to Matt Mackall, Ryan Wilcox, merc...@selenic.com
On Aug 9, 2010, at 1:35 PM, Matt Mackall wrote:

> On Sat, 2010-08-07 at 09:57 -0400, Ryan Wilcox wrote:
>> Hello everyone,
>>
>> I have set up a number of subrepos in my repository. These subrepos
>> point to (open source) repositories where I have read only access.
>>
>> I made some changes in my code, and I did an hg push:
>>
>> $ hg push
>> pushing to ssh://h...@bitbucket.org/rwilcox/dotfiles/
>> pushing subrepo vim/bundles/fugitive
>> ...
>> abort: the remote end hung up unexpectedly
>>
>>
>> Is there way to configure Mercurial to not try to push those subrepos
>> by default?
>
> Nope.
>
> When you commit at the top-level, you're taking a snapshot of a coherent
> project state. That includes the state of your subrepos. If you push the
> top level and not the corresponding subrepos, you'll get something
> incoherent on the other side - in fact something hg will refuse to check
> out.

Unless I'm reading Ryan's email wrong I think the issue is that the subrepos haven't been changed, so no changes actually need to be pushed, but Mercurial is trying anyway, failing, and then refusing to push the main repo.

A workaround (since these are all open source repos) is to fork them and change .hgsub to point at them. Unfortunately that makes updating them take a few more steps.

Ryan Wilcox

unread,
Aug 9, 2010, 2:21:21 PM8/9/10
to Steve Losh, merc...@selenic.com
On Mon, Aug 9, 2010 at 1:54 PM, Steve Losh <st...@stevelosh.com> wrote:
> On Aug 9, 2010, at 1:35 PM, Matt Mackall wrote:
>
>> On Sat, 2010-08-07 at 09:57 -0400, Ryan Wilcox wrote:
>>> Hello everyone,
>>>
>>> I have set up a number of subrepos in my repository. These subrepos
>>> point to (open source) repositories where I have read only access.
[snip]

>>>
>>> Is there  way to configure Mercurial to not try to push those subrepos
>>> by default?
>>
>> Nope.
>>
>> When you commit at the top-level, you're taking a snapshot of a coherent
>> project state. That includes the state of your subrepos....

> Unless I'm reading Ryan's email wrong I think the issue is that the subrepos haven't been changed, so no changes
> actually need to be pushed, but Mercurial is trying anyway, failing, and then refusing to push the main repo.

Steve is correct: these are repositories where no changes have been
made (and in fact, I'll never make changes in them).

I absolutely understand that normally you'd want subrepos to be pushed
(coherent project state). In this case, I know I'll never make changes
to those subrepos, so I'm looking for a way to tell hg "this is a read
only subrepo, don't try pushing".

Does that help clarify my question?
_Ryan Wilcox

--
_____________________________________________________________________
Ryan Wilcox             PGP: 0x2F4E9C31
http://flavors.me/rwilcox

Dirkjan Ochtman

unread,
Aug 10, 2010, 3:28:25 AM8/10/10
to Ryan Wilcox, merc...@selenic.com
On Sat, Aug 7, 2010 at 15:57, Ryan Wilcox <ryanw...@gmail.com> wrote:
> Is there  way to configure Mercurial to not try to push those subrepos
> by default?

We've talked about supporting vendor/externals (invariant: subrepos
are considered read+pull-only by the main repo) better as subrepos
using a setting in .hgsub, but it hasn't been done yet. Supposedly
that would help this use case, as well.

Cheers,

Dirkjan

Ryan Wilcox

unread,
Aug 11, 2010, 11:12:37 AM8/11/10
to Dirkjan Ochtman, merc...@selenic.com
On Tue, Aug 10, 2010 at 3:28 AM, Dirkjan Ochtman <dir...@ochtman.nl> wrote:
> On Sat, Aug 7, 2010 at 15:57, Ryan Wilcox <ryanw...@gmail.com> wrote:
>> Is there  way to configure Mercurial to not try to push those subrepos
>> by default?
>
> We've talked about supporting vendor/externals (invariant: subrepos
> are considered read+pull-only by the main repo) better as subrepos
> using a setting in .hgsub, but it hasn't been done yet. Supposedly
> that would help this use case, as well.
>

Would patches implementing read only subrepos be accepted, or is this
(say) not the direction you want to take?

Let me know,
_Ryan Wilcox

--
_____________________________________________________________________
Ryan Wilcox             PGP: 0x2F4E9C31
http://flavors.me/rwilcox

Steve Borho

unread,
Aug 11, 2010, 12:03:10 PM8/11/10
to Ryan Wilcox, merc...@selenic.com
On Wed, Aug 11, 2010 at 10:12 AM, Ryan Wilcox <ryanw...@gmail.com> wrote:
> On Tue, Aug 10, 2010 at 3:28 AM, Dirkjan Ochtman <dir...@ochtman.nl> wrote:
>> On Sat, Aug 7, 2010 at 15:57, Ryan Wilcox <ryanw...@gmail.com> wrote:
>>> Is there  way to configure Mercurial to not try to push those subrepos
>>> by default?
>>
>> We've talked about supporting vendor/externals (invariant: subrepos
>> are considered read+pull-only by the main repo) better as subrepos
>> using a setting in .hgsub, but it hasn't been done yet. Supposedly
>> that would help this use case, as well.
>>
>
> Would patches implementing read only subrepos be accepted, or is this
> (say) not the direction you want to take?

The final decision would be on the merits of the patch, but this is a
wanted feature.

You probably want to discuss the details with developers on the
mercurial-devel mailing list or irc before diving into the code, to
avoid implementing it more than once.

--
Steve Borho

Matt Mackall

unread,
Aug 11, 2010, 8:35:57 PM8/11/10
to Ryan Wilcox, merc...@selenic.com
On Wed, 2010-08-11 at 11:12 -0400, Ryan Wilcox wrote:
> On Tue, Aug 10, 2010 at 3:28 AM, Dirkjan Ochtman <dir...@ochtman.nl> wrote:
> > On Sat, Aug 7, 2010 at 15:57, Ryan Wilcox <ryanw...@gmail.com> wrote:
> >> Is there way to configure Mercurial to not try to push those subrepos
> >> by default?
> >
> > We've talked about supporting vendor/externals (invariant: subrepos
> > are considered read+pull-only by the main repo) better as subrepos
> > using a setting in .hgsub, but it hasn't been done yet. Supposedly
> > that would help this use case, as well.
> >
>
> Would patches implementing read only subrepos be accepted, or is this
> (say) not the direction you want to take?

It's reasonable, but it's the wrong answer for this particular issue.

An empty push to a pullable repo should always work, regardless of
whether you have permission to push or not. The first part of the push,
discovery, is allowed for both push and pull, and the second part
(actually sending a bundle) is skipped if there's nothing to send.

So unmodified subrepos shouldn't be causing a problem.

--
Mathematics is the supreme nostalgia of our time.

Matt Mackall

unread,
Aug 12, 2010, 1:56:10 AM8/12/10
to Steve Borho, merc...@selenic.com
[cc:ed back to list]

On Wed, 2010-08-11 at 22:45 -0500, Steve Borho wrote:


> On Wed, Aug 11, 2010 at 7:35 PM, Matt Mackall <m...@selenic.com> wrote:
> > On Wed, 2010-08-11 at 11:12 -0400, Ryan Wilcox wrote:
> >> On Tue, Aug 10, 2010 at 3:28 AM, Dirkjan Ochtman <dir...@ochtman.nl> wrote:
> >> > On Sat, Aug 7, 2010 at 15:57, Ryan Wilcox <ryanw...@gmail.com> wrote:
> >> >> Is there way to configure Mercurial to not try to push those subrepos
> >> >> by default?
> >> >
> >> > We've talked about supporting vendor/externals (invariant: subrepos
> >> > are considered read+pull-only by the main repo) better as subrepos
> >> > using a setting in .hgsub, but it hasn't been done yet. Supposedly
> >> > that would help this use case, as well.
> >> >
> >>
> >> Would patches implementing read only subrepos be accepted, or is this
> >> (say) not the direction you want to take?
> >
> > It's reasonable, but it's the wrong answer for this particular issue.
> >
> > An empty push to a pullable repo should always work, regardless of
> > whether you have permission to push or not. The first part of the push,
> > discovery, is allowed for both push and pull, and the second part
> > (actually sending a bundle) is skipped if there's nothing to send.
> >
> > So unmodified subrepos shouldn't be causing a problem.
>

> I can attest this is still very much an issue. I quit using subrepos
> for my Windows installer build setup after just a week because every
> push took minutes even though I was only ever making commits to the
> root repository. And having just one remote URL being unresponsive
> made the entire push fail, even though you have no outgoing changes
> for that URL.

If we add read-only repos, we'll still have done nothing to address the
as-yet -undiagnosed- bug that started this thread. Let's focus on that
diagnosis first.

Lester Caine

unread,
Aug 12, 2010, 2:11:45 AM8/12/10
to mercurial

Well surely that is fairly obvious? Is there actually a mechanism that say 'you
do not have permission'? Personally I am stuck at this point in conversion over
to Hg since I need a method of working with 'subrepo' that is more compatible
with the CVS methods of creating them. There some modules I have full r/w access
and others are read only. The read only ones simply tell me when I try and
commit a change ... which is usually accidental bundled in with changes to other
modules that I do have access to.

What *I* have been thinking is that it would be very useful to be able to pick
up third party libraries direct from their code base rather than maintaining
local copies. but that may need CVS/SVN or xxx read/pull access to their code.
I'm not getting on too bad with my subrepo links being to git repos for each but
I am not convinced that even this is stable fully yet?

--
Lester Caine - G8HFL
-----------------------------
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk//
Firebird - http://www.firebirdsql.org/index.php

Matt Mackall

unread,
Aug 18, 2010, 6:11:49 PM8/18/10
to Lester Caine, mercurial

No. You claimed you didn't change it and yet push failed, I claimed an
unchanged repo shouldn't fail a push. So either there's a bug, or you've
changed it. Which is it?

> Is there actually a mechanism that say 'you
> do not have permission'?

That's way too nebulous a question. So you'll get a nebulous answer:
there's no central authority for what you can do with a clone. But if
you want to bind your own hands, that's easy enough: simply set a
precommit hook that returns false.

--
Mathematics is the supreme nostalgia of our time.

Reply all
Reply to author
Forward
0 new messages