[fossil-users] Fossil and Git joint projects?

[Nolan Darilek]

Hey folks.

So far, most of my Fossil use has been on projects I own almost
exclusively. However, I'm about to start working on projects with a
community, and while I'm going to push Fossil adoption hard, I may get
some pushback from Git users. Is it possible to use Fossil in a workflow
with people who would rather use Git/Github? IOW, can I make Fossil my
official project source but periodically push updates to Github, then
import the results of people's contributions into the official repo?

I see an incremental option for import. I'm guessing this helps with
that? Can I just incrementally import the latest changes from Git
repositories? I'm guessing this just adds in changesets from Git that
aren't present in Fossil?

I don't see a symmetric option in the export command. Do I just have to
keep exporting the full repository each time? I'm guessing that the
exports should be identical such that when I go to push the new export
to Github, that it won't look like an entirely new repo.

Hoping that I can push Fossil adoption for these projects, but I'm
worried that I'll have to use the lowest common denominators in these
communities, and in this case that will be Git. In that case, I'd really
rather build the project/site in Fossil, then win people over when they
realize that their Fossil clone of the project includes the wiki,
ticketing system, etc. and their Git pulls don't. :)
_______________________________________________
fossil-users mailing list
fossil...@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[Stephan Beal]

On Fri, Nov 4, 2011 at 5:43 PM, Nolan Darilek <no...@thewordnerd.info> wrote:

some pushback from Git users. Is it possible to use Fossil in a workflow with people who would rather use Git/Github?

Richard wrote a nice summary of that on Oct 16th which i'll paste in here:

---------------

Fossil does not currently support a hierarchical development model very well.  It wants everybody to be a peer.  It wants all developers to see everything all the time.  Fossil strives to avoid a "peeking order" in which some developers are hidden from view behind "lieutenants".  This is a more egalitarian model, but also one that does not scale as well.

To better support a hierarchy, Fossil would need the ability to sync individual branches in addition to its current behavior of always syncing everything on every sync request. (Recall that I asked for volunteers to implement such a thing a while back.)  But adding that feature quickly gets complicated when you then try to figure out how to deal with auto-sync.  You could, I suppose, put your local Fossil into a mode where it only syncs the branch you are currently working on or switching to.  But what about Wiki and Tickets and Events?  Do they get synced or not?  Once you leave the comfort of Fossils original model of "everybody sees all the code all the time" then various operational questions of this kind start to come up.

---------------

--
----- stephan beal
http://wanderinghorse.net/home/stephan/

[Nolan Darilek]

Thanks, but that's not really what I asked.

I totally get Fossil's development model, have used it for over a year and think that it'd be a great fit for this particular community. I also read this message when it was originally posted. But I may be working with people who would rather submit a quick change via Github rather than download and install a new piece of software. Yes, I get that it's easy, I'm just thinking that it might be a barrier here. So let me make the question more explicit:

1. Can I export my project to a Git repository, push that to Github, make a few commits, export the changes to Git and push the repository again? If so, will it look identical to the repository after the first step with a few extra commits such that someone who pulls doesn't get told that the repository after the changes is different?

2. If my canonical Fossil repository advances and someone makes changes on Github, can I do an incremental import of the Git repository and only get their changes without creating an entirely new Fossil repository?

3. Has anyone else done this, and how does it work? I'd really rather use Fossil, but am worried about losing contributors who don't want to learn a new and simpler system. Since we're developing applications to meet immediate needs (software for the various occupations), the response I may get from people might be to use the tools that everyone knows to maximize the community's ability/willingness to chip in. If that response comes, hopefully I can say that Fossil interacts with Git in the manner I've described here and can meet the need.

Thanks.

[Alek Paunov]

Hi Nolan, Stephan,

I think that this is doable for a restricted subclass of usage scenarios
(these conforming workflow style cited by Stephan above + bunch of
other restrictions).

We can benefit the fantastic opportunity of sqlite as fossil artifacts
container - I think that it is possible to be implemented git protocol
enabled python server with dulwich [1] on top of the fossil db, once we
are able to manage fossil/git commit hashes dictionary in additional db
table. Hg-git [2] also uses similar approach (but client side, It seems
to me that server side is the easier one).

As Fossil API, for the proof of the concept, the server can use the JSON
API, Stephan ?

Kind Regards,
Alek

[1] http://git.samba.org/?p=jelmer/dulwich.git
[2] http://hg-git.github.com/

P.S. github != git, maybe tickets and wikis can be synchronized too (to
some level) using the gihub API.

[Stephan Beal]

On Fri, Nov 4, 2011 at 7:48 PM, Alek Paunov <al...@declera.com> wrote:

As Fossil API, for the proof of the concept, the server can use the JSON API, Stephan ?

The biggest catch there is binary data, since JSON has no native way of handing it. However, my current thinking on binary data is this...

from JSON we serve URL paths which, which called by an HTTP client, will return the raw content (binary or not) for the given artifact. We could handle POSTed data the same way, _theoretically_. There is a bit of proof-of-concept for that in the current timeline output:

...

"files":[{

            "name":"src/json_user.c",

            "uuid":"d860ffd974654d397cf3be9a39ec1eb99ea472b2",

            "prevUuid":"4b9322d63bad1b18bb3fd24e435277fc852db7d6",

            "state":"modified",

            "downloadPath":"/raw/src/json_user.c?name=d860ffd974654d397cf3be9a39ec1eb99ea472b2"

          }]

 

[Alek Paunov]

On 04.11.2011 21:01, Stephan Beal wrote:
> The biggest catch there is binary data, since JSON has no native way of
> handing it. However, my current thinking on binary data is this...
>
> from JSON we serve URL paths which, which called by an HTTP client, will
> return the raw content (binary or not) for the given artifact. We could
> handle POSTed data the same way, _theoretically_. There is a bit of
> proof-of-concept for that in the current timeline output:

This approach (binaries via URL "pointers" on get/direct POST payloads
on send) seems OK to me ...

[Michael Barrow]

Apologies in advance if this makes no sense. I've only done a tiny bit with git and that was some time ago.

What if you have a single directory that both version control systems use? You pull from git, commit to Fossil, do your changes and commit to Fossil, and then push your changes to git. When you do the new pull from git, you just update Fossil and start the cycle again.

--

Michael L. Barrow

[Andreas Kupries]

On 11/4/2011 11:48 AM, Alek Paunov wrote:
> Hi Nolan, Stephan,
>
> I think that this is doable for a restricted subclass of usage scenarios (these
> conforming workflow style cited by Stephan above + bunch of other restrictions).
>
> We can benefit the fantastic opportunity of sqlite as fossil artifacts
> container - I think that it is possible to be implemented git protocol enabled
> python server with dulwich [1] on top of the fossil db, once we are able to
> manage fossil/git commit hashes dictionary in additional db table. Hg-git [2]
> also uses similar approach (but client side, It seems to me that server side is
> the easier one).

Does hg have something git's rebase and other history rewriting operations ?

If not, how do hg and the synchronizer (dulwich?) cope when the git repository
they talk to undergoes such a rewrite ?

This claim of 'lossless conversion of changesets' is certainly something of
interest.

Hm ... given that "This plugin was originally developed by the folks at
GitHub", maybe we can interest them in doing similar for fossil ?

> As Fossil API, for the proof of the concept, the server can use the JSON API,
> Stephan ?
>
> Kind Regards,
> Alek
>
> [1] http://git.samba.org/?p=jelmer/dulwich.git
> [2] http://hg-git.github.com/
>
> P.S. github != git, maybe tickets and wikis can be synchronized too (to some
> level) using the gihub API.
>

--
Andreas Kupries
Senior Tcl Developer
ActiveState, The Dynamic Language Experts

P: 778.786.1122
F: 778.786.1133
andr...@activestate.com
http://www.activestate.com
Get insights on Open Source and Dynamic Languages at www.activestate.com/blog

[Alek Paunov]

On 04.11.2011 22:32, Andreas Kupries wrote:
> Does hg have something git's rebase and other history rewriting
> operations ?

Yes - hg supports all of them and more (via various bundled [1] and
endless count of community extensions), but as you know, history
rewriting kills any repo/repo synchronization in any DVCS. Thus,
thinking of the partial Fossil/Git bridge, let's focus on ideal case -
no rebases, full DAG synchronization.

>
> If not, how do hg and the synchronizer (dulwich?) cope when the git
> repository they talk to undergoes such a rewrite ?

Dulwich is not a merge-application - dulwich is just a well designed Git
API (protocol and container format).

In the case of Mercurial/Git bridge, the actual synchronizer is the
hg-git itself (mercurial extension), which was started as common effort
with github.com, but these days is actually maintained by hg-subversion
team (with Augie Fackler [2] as lead)

[1] http://mercurial.selenic.com/wiki/RebaseExtension
[2] https://bitbucket.org/durin42/hg-git/changesets

[Nolan Darilek]

Just a heads-up:

I did a quick test, and this didn't seem to work:

* Created a new Fossil repository
* Opened it into ./fossil, added README.txt with contents "foo".
* Changed to parent directory, created Git repository in ./git
* Imported Fossil repo and switched to trunk branch
* Modified README.txt to contain "foo\nbar" and committed to Git
* Ran "git fast-export --all |fossil import --incremental ../test.fossil"
* Then ran "cd ../fossil; fossil up"

The result is that my initial commit of README.txt into Fossil appears in the timeline twice. The Git commit appears, but "fossil up" won't update to it, at least not without being explicitly asked. If you'd like to see the resulting repository, it's here:

http://dl.dropbox.com/u/147071/test.fossil

Sorry I didn't test in the first place, I thought that the answer would be something like "yes of course that works, that's what --incremental does, what a silly question." :) I guess I'll just use Fossil until I get pushback, then try promoting it and, if that fails, just export to Git.

Thanks.

[Steve Bennett]

On 10/11/2011, at 2:30 PM, Steve Bennett wrote:

I tried to do something similar, using a git tag to keep track of where I was up to

for the incremental import.

It mostly works, except the timeline shows each import as disconnected from

the previous import. See attached.

For the import at 21:40, I tried to import one previous rev hoping that they would be

merged, but this didn't happen. Is this "disconnect" fixable?

I guess the problem is that the first commit of an incremental import has no

ancestor identified. Would it be possible to identify the ancestor somehow, either

on the command line, or by finding the tip of the corresponding branch?

--

µWeb: Embedded Web Framework - http://uweb.workware.net.au/

WorkWare Systems Pty Ltd

W: www.workware.net.au      P: +61 434 921 300

E: ste...@workware.net.au   F: +61 7 3391 6002

<fossil-git-incr-import.png>

--

µWeb: Embedded Web Framework - http://uweb.workware.net.au/

WorkWare Systems Pty Ltd

W: www.workware.net.au      P: +61 434 921 300

E: ste...@workware.net.au   F: +61 7 3391 6002

[Steve Bennett]

On 10/11/2011, at 2:50 PM, Steve Bennett wrote:

On 10/11/2011, at 2:30 PM, Steve Bennett wrote:

I tried to do something similar, using a git tag to keep track of where I was up to

for the incremental import.

It mostly works, except the timeline shows each import as disconnected from

the previous import. See attached.

For the import at 21:40, I tried to import one previous rev hoping that they would be

merged, but this didn't happen. Is this "disconnect" fixable?

I guess the problem is that the first commit of an incremental import has no

ancestor identified. Would it be possible to identify the ancestor somehow, either

on the command line, or by finding the tip of the corresponding branch?

The attached patch works for my simple case of always importing to the tip of trunk,

but I'm sure it is highly dubious in other situations.

[Steve Bennett]

I tried to do something similar, using a git tag to keep track of where I was up to

for the incremental import.

It mostly works, except the timeline shows each import as disconnected from

the previous import. See attached.

For the import at 21:40, I tried to import one previous rev hoping that they would be

merged, but this didn't happen. Is this "disconnect" fixable?

Cheers,

Steve

On 05/11/2011, at 7:28 AM, Nolan Darilek wrote:

--

µWeb: Embedded Web Framework - http://uweb.workware.net.au/

WorkWare Systems Pty Ltd

W: www.workware.net.au      P: +61 434 921 300

E: ste...@workware.net.au   F: +61 7 3391 6002

[Steve Bennett]

No comments on this?

It works for me. Shall I commit this fix, or are there potential issues with it?

On 10/11/2011, at 3:17 PM, Steve Bennett wrote:

On 10/11/2011, at 2:50 PM, Steve Bennett wrote:

On 10/11/2011, at 2:30 PM, Steve Bennett wrote:

I tried to do something similar, using a git tag to keep track of where I was up to

for the incremental import.

It mostly works, except the timeline shows each import as disconnected from

the previous import. See attached.

For the import at 21:40, I tried to import one previous rev hoping that they would be

merged, but this didn't happen. Is this "disconnect" fixable?

I guess the problem is that the first commit of an incremental import has no

ancestor identified. Would it be possible to identify the ancestor somehow, either

on the command line, or by finding the tip of the corresponding branch?

The attached patch works for my simple case of always importing to the tip of trunk,

but I'm sure it is highly dubious in other situations.

[Richard Hipp]

On Sun, Nov 13, 2011 at 6:48 PM, Steve Bennett <ste...@workware.net.au> wrote:

No comments on this?

It works for me. Shall I commit this fix, or are there potential issues with it?

I think go ahead and commit.
 

_______________________________________________
fossil-users mailing list
fossil...@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

--
D. Richard Hipp
d...@sqlite.org

[Steve Bennett]

On 14/11/2011, at 9:55 AM, Richard Hipp wrote:

On Sun, Nov 13, 2011 at 6:48 PM, Steve Bennett <ste...@workware.net.au> wrote:

No comments on this?

It works for me. Shall I commit this fix, or are there potential issues with it?

I think go ahead and commit.

FYI, what I committed this doesn't work perfectly in the case where a branch is closed.

In this case, the closed branch is still used as the ancestor.

I hacked the code a bit to detect this case and instead branch from trunk.

It was made more complicated by the fact that there was no obvious way to check if a branch is closed.

name_to_uuid() does most of what I need, but then it's not easy to determine if the leaf is closed.

Cheers,

Steve