github, again

[Chris Withers]

Hi,

I know GitHub ended up being a no-go because of the issue tracker, but
how about getting git mirrors up of sqlalchemy and alembic?

http://hgtip.com/tips/advanced/2009-11-09-create-a-git-mirror/

...appears to describe this and follows through getting changes back in
through the git mirror as well as sending them out that way.

Mike, how do you feel about this?

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk

[Michael Bayer]

I was playing around with having mirrors on github using hg-git, which I've also contributed towards as a result, including a patch for "author map translation" as well as a way to translate hg bookmark names:

https://bitbucket.org/durin42/hg-git

Though when I was experimenting with moving between git and hg, I was still encountering scenarios where things didn't work as expected, while I didn't look deeply into it I removed the git mirrors for the time being.

If you want to experiment with a github clone of your own for a bit using hg-git, maybe you could help us work out the kinks.

> --
> You received this message because you are subscribed to the Google Groups "sqlalchemy-devel" group.
> To post to this group, send email to sqlalche...@googlegroups.com.
> To unsubscribe from this group, send email to sqlalchemy-dev...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/sqlalchemy-devel?hl=en.
>

[Chris Withers]

On 06/03/2012 16:00, Michael Bayer wrote:
>
> If you want to experiment with a github clone of your own for a bit using hg-git, maybe you could help us work out the kinks.

Well, it's been a while, but what with wanting to do the pg ranges work,
I've set up a git clone:

https://github.com/cjw296/sqlalchemy

$ git push origin master
Counting objects: 72128, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (65882/65882), done.
Writing objects: 100% (72128/72128), 299.39 MiB | 32 KiB/s, done.
Total 72128 (delta 0), reused 72128 (delta 0)
To g...@github.com:cjw296/sqlalchemy.git
* [new branch] master -> master

Really? 300MiB of revisions? Does that sound right? If so, Mike, you've
been working too hard :-P

(if not, any idea what's going on here?)

Looks like that's just the tip, no other branches.

Mike, if I get the pg_ranges stuff in a branch on github, can you pull
it from there or do I need to get it back to my bitbucket account in hg
format?

cheers,

[Michael Bayer]

I just did a transfer using my fancy new tool https://bitbucket.org/zzzeek/hggitonce , and with all the branches it's 635M. It's eight years of intensive code so yeah. I will be removing some of those branches but I'm not sure that will change things much.

Not sure if you've noticed that Alembic is now on git. The next project will likely be Mako as I need to work out a new flow with my server as well as script up a new migration for trac links. As it seems to be a significant PITA to set up incoming git http and SSH is better anyway, I will probably have Bitbucket serve as where public commits and such go, seeing how that's where the pull requests are anyway, and then I'll just mirror a read-only git on my server so that trac can display it.

[Chris Withers]

On 16/05/2013 18:27, Michael Bayer wrote:
> I just did a transfer using my fancy new tool https://bitbucket.org/zzzeek/hggitonce , and with all the branches it's 635M.

Okay, 300M for tip sounds reasonable then :-)

> It's eight years of intensive code so yeah. I will be removing some of those branches but I'm not sure that will change things much.

I wonder about squashing down commits older than x years? Might make the
repo easier to work with. Of course, keep a full repo around somewhere
for forensics if required...

> Not sure if you've noticed that Alembic is now on git.

Yep, very happy to see that, will be on it once the project I'm working
on needs migrations ;-)

> As it seems to be a significant PITA to set up incoming git http and SSH is better anyway,

I think I follow: you mean git push to a server via ssh is trivial (it's
just ssh) but http is a PITA? Yeah, that's what I do for all my
non-public stuff.

> I will probably have Bitbucket serve as where public commits and such go,

Serve git or hg?

Still wish we could use GitHub's issue trackers, their pull request
handling is amazing; made the xlrd-to-python-3 work possible, and that
was a tough pull request involving many changesets, a lot of comments on
those changes along the way, and a protracted timeframe involving 3 or 4
disparate developers...

Anyway, just to confirm: you still want pull requests coming from a hg
repo on bitbucket, right?

cheers,

Chris

>
>
>
>
> On May 16, 2013, at 9:36 AM, Chris Withers<ch...@simplistix.co.uk> wrote:
>
>> On 06/03/2012 16:00, Michael Bayer wrote:
>>>
>>> If you want to experiment with a github clone of your own for a bit using hg-git, maybe you could help us work out the kinks.
>>
>> Well, it's been a while, but what with wanting to do the pg ranges work, I've set up a git clone:
>>
>> https://github.com/cjw296/sqlalchemy
>>
>> $ git push origin master
>> Counting objects: 72128, done.
>> Delta compression using up to 4 threads.
>> Compressing objects: 100% (65882/65882), done.
>> Writing objects: 100% (72128/72128), 299.39 MiB | 32 KiB/s, done.
>> Total 72128 (delta 0), reused 72128 (delta 0)
>> To g...@github.com:cjw296/sqlalchemy.git
>> * [new branch] master -> master
>>
>> Really? 300MiB of revisions? Does that sound right? If so, Mike, you've been working too hard :-P
>>
>> (if not, any idea what's going on here?)
>>
>> Looks like that's just the tip, no other branches.
>>
>> Mike, if I get the pg_ranges stuff in a branch on github, can you pull it from there or do I need to get it back to my bitbucket account in hg format?
>>
>> cheers,
>>
>> Chris
>>
>> --

>> Simplistix - Content Management, Batch Processing& Python Consulting

[Michael Bayer]

On May 16, 2013, at 3:09 PM, Chris Withers <ch...@simplistix.co.uk> wrote:

>
>> I will probably have Bitbucket serve as where public commits and such go,
>
> Serve git or hg?

it would be all git

>
> Still wish we could use GitHub's issue trackers,

GH's issue trackers are not nearly as fine grained as the one I have with Trac, and last I looked github doesn't really allow realistic imports of issues (no usernames, no dates, pointless). The SQLA issue base is kind of cherished enough that I really prefer to own it in it's full form on my own servers. I don't really trust Github (or even Bitbucket, though at least I have direct lines to Bitbucket developers) with this particular set of data. Repos are not a problem since we have full copies of them everywhere.

> their pull request handling is amazing;

I've had no problems with bitbuckets pullreq handling and once we are on git we can take pullreqs in either location anyway.

> made the xlrd-to-python-3 work possible, and that was a tough pull request involving many changesets, a lot of comments on those changes along the way, and a protracted timeframe involving 3 or 4 disparate developers...

I know you hate mercurial but four developers and a lot of changesets are really not a challenge for any DVCS, not to mention just any old VCS...

>
> Anyway, just to confirm: you still want pull requests coming from a hg repo on bitbucket, right?

that's what I have for the moment, though if I accelerate my transition to git SQLA could be there in a few weeks maybe, maybe not. no guarantees yet.

[Gabriel de Perthuis]

On Thu, 16 May 2013 13:27:56 -0400, Michael Bayer wrote:
> I just did a transfer using my fancy new tool
> https://bitbucket.org/zzzeek/hggitonce , and with all the branches
> it's 635M. It's eight years of intensive code so yeah. I will be
> removing some of those branches but I'm not sure that will change
> things much.

I just tried it (had to symlink mercurial's pure modules to make it work
in PyPy), after git gc --aggressive (with higher-than-default
pack.window and depth settings) it compacts to a very reasonable 17M,
the largest files being changelogs.

[Chris Withers]

Is the resulting git repo available anywhere?

[Gabriel de Perthuis]

On Fri, 17 May 2013 11:38:55 +0100, Chris Withers wrote:
> On 17/05/2013 07:52, Gabriel de Perthuis wrote:
>> I just tried it (had to symlink mercurial's pure modules to make it work
>> in PyPy), after git gc --aggressive (with higher-than-default
>> pack.window and depth settings) it compacts to a very reasonable 17M,
>> the largest files being changelogs.
>
> Is the resulting git repo available anywhere?

Here: https://github.com/g2p/sqlalchemy.converted

I didn't do any fine-tuning (like username conversions), and I just noticed
my hg repo wasn't quite current. I'll drop this when the canonical repo is up.

[Gabriel de Perthuis]

[Michael Bayer]

running straight up git gc --aggressive without changing any settings takes my 600M with all the branches down to 26M. The mercurial is 101M. I think that's good enough.

> --
> You received this message because you are subscribed to the Google Groups "sqlalchemy-devel" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-dev...@googlegroups.com.

> To post to this group, send email to sqlalche...@googlegroups.com.

> Visit this group at http://groups.google.com/group/sqlalchemy-devel?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>