GitFS: Report #3

[Manzur]

Hi All!

For the past week I've spent most of the time realizing how styx works
and how to use styx modules.

Besides it I developed some functionality of the file server.
Namely:
- added functionality to ctl file like init, add, remove;
- added interface for logging. At the moment you can get log of the
repo by reading log file;
- added mapping for config and exclude files;
- mapped objects from directory to mountpoint. You can simply cat any
object in object store, and get the unpacked version of the file;
- all files that're located in source directory are mapped to "files/"
directory. But with the current representation it's inconvenient to
add files to the repository(imaginary situtation):
echo add dir1/dir2/text1 > ../../ctl

But I'm thinking about making it more convenient;

- changed other modules so that they could be used from fileserver;

This week I'm going to finish the file server implementation.

It'll be good if anybody could share his best practices for fileserver
implementation, especially debugging it.

The end.

[Roman Shaposhnik]

On Mon, Jun 22, 2009 at 3:00 PM, Manzur<manzu...@gmail.com> wrote:
>
> Hi All!
>
> For the past week I've spent most of the time realizing how styx works
> and how to use styx modules.
>
> Besides it I developed some functionality of the file server.
> Namely:
> - added functionality to ctl file like init, add, remove;
> - added interface for logging. At the moment you can get log of the
> repo by reading log file;
> - added mapping for config and exclude files;
> - mapped objects from directory to mountpoint. You can simply cat any
> object in object store, and get the unpacked version of the file;
> - all files that're located in source directory are mapped to "files/"
> directory. But with the current representation it's inconvenient to
> add files to the repository(imaginary situtation):
>          echo add dir1/dir2/text1 > ../../ctl
>
> But I'm thinking about making it more convenient;
>
> - changed other modules so that they could be used from fileserver;
>
> This week I'm going to finish the file server implementation.

Great. Please let me know when I can take it for a spin. In the great
tradition of bootstrapping I'd expect it to be able to access its
own Git repo ;-)

> It'll be good if anybody could share his best practices for fileserver
> implementation, especially debugging it.

Do you have any particular questions in mind?

Thanks,
Roman.

[ron minnich]

question: I assume that at top level, all branches are visible at the
same time? One of the major failings of git in my view is that I can't
see branches without checking them out.

ron

[Manzur]

To Roman:

I want to know yours and others' opinion about organizing files.
As I said above currently adding file1 to the repository is done by
(we're located in dir1/dir2 directory):

echo dir1/dir2/file1 > ../../ctl

This isn't good, especially without tab completion. The problem is
with the location of ctl file: user should specify either absolute or
relative path.

I've two solutions to that problem:

1) Create file server's special files in each subdirectory of the
<files> directory. So in the result we will have ctl, data in each
directory.

2) Make files under <files> directory read-only. Interpret data
written to the files under <files> dir as git commands. For example,
we could simply say

echo add > file1

to add file1 to the repository. The same could be done with other file-
specific git commands. Commands that aren't file-specific can be
executed via ctl and data files.


To Ron:

I agree with you that viewing branches would make git more convenient.
But decision was made towards saving on extra space needed for each
extracted branch.
But in my case it's possible to do it through extraction branch on the
fly.
I'll add that feature soon.


Thanks for your interest.
Manzur

[Federico G. Benavento]

I have a particular question, in the end are you going to be able to use
rm, cp, etc?

why not just "touch file" instead "echo add" ?

--
Federico G. Benavento

[Eric Van Hensbergen]

Sent from my iPhone

On Jun 23, 2009, at 3:26 PM, "Federico G. Benavento" <bena...@gmail.com

> wrote:

>
> I have a particular question, in the end are you going to be able to
> use
> rm, cp, etc?
>
> why not just "touch file" instead "echo add" ?

Agreed, you really should be able to interact with the synthetic more
naturally for these sorts of commands.

[ron minnich]

No disrespect intended here at all, but gitfs is sounding more like a
webfs than what I thought it was. I was hoping that at top level we
could create trees of the different branches and revert to standard
tools like diff and so on instead of git-*.

I think I misunderstood the goals.

Thanks

ron

[Manzur]

On 24 июн, 00:26, "Federico G. Benavento" <benave...@gmail.com> wrote:
> I have a particular question, in the end are you going to be able to use
> rm, cp, etc?
>
> why not just "touch file" instead "echo add" ?
>

For what purpose do you want to use cp and rm?

Let me show some use cases for gitfs.

Suppose you mount existing git repo:
mount {gitfs repo} /mnt/git

Files are located in two directories:

- files/. This directory contains all files under repo/ directory.

In case of first solution(see above) this dir is read/write. And
all operations are done as you do them with repo/ files. For example,
if you remove files/file1 the same file under repo/ dir is removed.

In case of second solution this dir is "read-only". All data you
write to any file is interpreted as file-specific git command. You
asked why not to use just touch command. The answer is: because git
has more than one file-specific command. Of course, we could use
other commands such as chmod with cryptic semantic, but it'll confuse
you more than pointing long path to ctl file.

- objects/. This directory contains object store(blobs,tree and
commits in zlib format), which is by default located at repo/.git/
objects.
This is read/write directory. You can use any operation with this
directory. And all operations are done as you do them with the file
under repo/.git/objects.

We(Mechiel, Yiyus and me) discussed some variants of directory layout.
Summary line:

- Mechiel suggested to develop shell commands that'll be used instead
of echoed commands:
echo add dir1/dir2/file1 > ../../ctl
will be changed to
git/add file1

- Yiyus suggested to represent index file as directory. So that you
can use touch, cp, rm commands instead of git add, git rm:
echo add dir1/dir2/file1 > ../../ctl
will be changed to
cp dir1/dir2/file1 index/

As you can see there's space to roll. Feel free to comment. Comments
help me to deduce pros and cons.

Thanks,
Manzur

[Devon H. O'Dell]

2009/6/25 Manzur <manzu...@gmail.com>:

>
> On 24 июн, 00:26, "Federico G. Benavento" <benave...@gmail.com> wrote:
>> I have a particular question, in the end are you going to be able to use
>> rm, cp, etc?
>>
>> why not just "touch file" instead "echo add" ?
>>
>
> For what purpose do you want to use cp and rm?

I think the idea is, `why can't I just cp /foo/bar /mnt/git' and have
that function as `git/add bar'; rm as git/remove or whatever. The idea
of having it in a fileserver is novel only when you are able to
interact with the fileserver using standard utilities. I think we're
not understanding what git/add is supposed to do that a cp wouldn't
accomplish.

--dho

[Eric Van Hensbergen]

2009/6/25 Devon H. O'Dell <devon...@gmail.com>:

A potential point of confusion is repository actions versus sandbox
actions. GIT (and other source control systems) force explicit
actions to add and remove stuff to the repository from the sandbox so
that transient files (like object files or editor temporary files)
don't flow into the repo.

Conceptually, dynamic binding and private namespaces could be used to
address these, but how to do this isn't completely clear (to me). You
could explicitly mark the "source" files with chmod (or by default
mark all files in the sandbox as temporary, and explictly chmod out
the temporary flag via an explicit command or script). Another option
would be to have a copy-on-change synthetic file system sandbox bound
over the gitfs, but then you'll need to explicitly identify which
files you changed which are source files.

In my mind, figuring out how to do this naturally is the true crux of
this project and probably should have been worked out during the
proposal phase. That being said, the current approach being taken by
the student should suffice as a low-level file system, with a follow
up project being the implementation of a higher-level abstraction
(which is in part defined by the development workflow) which actually
gives the natural file system semantics most of the reponders
(including myself) seem to desire.

-eric

[Roman Shaposhnik]

2009/6/25 Eric Van Hensbergen <eri...@gmail.com>:

To me this is more of a question of how much internal Git machinery
does one want to expose/see. The power of Git on a conventional
UNIX system clearly stems from the fact that it uses a three-staged
approach:
[working tree] <---> [index] <---> [history]

It is clear (to me at least) that history is a sort of singleton object.
It has a single instance and that's it. There could be different ways
of manipulating it, but still your are always manipulating the same
thing all the time.

Now, working tree on the other hand, has to be "polymorphic".
IOW, I would like to see as many working trees as I have SHA ids
in my history. Ands I also want these trees to be addressed
by tags, refs, etc.

Finally, the index is how you get from a working tree to history.
Perhaps, it is a worthy things to be exposed as well. So that
regardless of what you do, your trees are all read only, and
you'd have to always do
$ bind -ac /path/to/index/ /path/to/a/working/tree
to read and write inside a working tree to make it writable.
At that point, the question of adding to history becomes
more or a less a question of commiting the state of that
extra tree representing index.

But, as you said, this might be something to consider for
the next project.

As Ron pointed out, at this point it is more of a webfs,
than what we all want.

Thanks,
Roman.

[Roman Shaposhnik]

On Thu, Jun 25, 2009 at 11:30 AM, Manzur<manzu...@gmail.com> wrote:
> We(Mechiel, Yiyus and me) discussed some variants of directory layout.
> Summary line:
>
> - Mechiel suggested to develop shell commands that'll be used instead
> of echoed commands:
>       echo add dir1/dir2/file1 > ../../ctl
> will be changed to
>       git/add file1
>
> - Yiyus suggested to represent index file as directory. So that you
> can use touch, cp, rm commands instead of git add, git rm:
>      echo add dir1/dir2/file1 > ../../ctl
> will be changed to
>      cp dir1/dir2/file1 index/

Now, since there's a work-in-progress man page, I'd like it to have an
Example/Usage
section outlining typical git uses and how they translate into what gitfs has to
offer. Here's what I would expect to see there:
1. Simple commits (IOW: checkout, [edit, build]*, commit)
2. Merging
3. Interactive add
4. Git stash
5. Amending commits

Thanks,
Roman,

[Federico G. Benavento]

ok, I know little about git, what I do know is that is a versioning system,
and I don't get how that can't be mapped to a fs that looks like an actual
filesystem.

I get that git can have lot's of modes and quirks, but couldn't mount specs
be used to solve this

mount /srv/git.myapp /n/myapp-ro read-only

or whatever that makes sense in git world?

to me a fs like webfs it's more like a library, were once you have it, then
you need a client, hence a lot of work as well

brainstorming on possible layouts for the filesystems could be a good
start.

/mnt/git
clone
ctl
0/ # n per commit
tree/
file.c
...
special-files
1/

or

mount /srv/git /n/git this-commit read-only

/n/git
file.c
...


sorry if this is too naive, but in the end we want a filesystem that resembles a
filesystem, not one that serves the purpose of a library, that'll have to be
addapted eveytime the protocol changes and this probably involves changing
all the clients.

--
Federico G. Benavento

[ron minnich]

git is a file system. It contains many different roots, which they
call branches. Branches have 'scores'. I'm not the first person to
notice a passing similarity to venti in that way.

In venti, we can see every version of a file system, and even mount
it, given a score. That's why things like yesterday are nice.

To see a tree from git, I have to checkout a branch. I only have one
branch checked out at a time. To see difference to a branch, I have to
do this:
git-diff

I thought the intent of this project was to allow my to say, e.g.:
mount /srv/git /tree
cd /tree
diff brancha branchb

In this way git could be used in a way similar to venti.

If this is going to end up being like webfs, and the git file system
tree is not visible as a file system, I have a hard time seeing any
improvement over the git tools.

ron

[yy]

I don't think I'm understanding well the situation here.
If I got it right, all the operations ron wants are -or will be-
possible in gitfs. The echo add operation is only to add files to the
index [1], that tells git what files in the working dir are part of
the repository (it serves the same purpose than .hgignore files in
mercurial). If you just don't care about index files (ie. you clean up
the working directory before commiting) just do echo add * before
every commit.
I wish Manzur gave a better explanation. I didn't look at his code,
and I'm not really into this git thing... (and yes, a clear layout of
the file system as fgb suggested would be of much help).

[1] http://www.kernel.org/pub/software/scm/git/docs/user-manual.html#the-index

--
- yiyus || JGL .

[Steve Simon]

I wrote a CVS filesystem for plan9 some years ago.
I chose to create a filesystem which mimics plan9's dump
filesystem. It also provides access by CVS tag, and
automagically creates a ChangeLog text file which
lists all log messages and the files they effect.

However I could not find a clean way to implement
checkins so it reains a readonly view of the repository
and due to limitations of the CVS protocol it has to
scan all the files in the module at connect time which
can be slow.

if its of interest see:
http://lsub.org/magic/man2html/4/cvsfs
contrib/install steve/cvsfs

Also I came across this:

http://www.hipc.org/hipc2006/posters/repofs.pdf

-Steve

[Roman Shaposhnik]

On Thu, Jun 25, 2009 at 11:43 PM, ron minnich<rmin...@gmail.com> wrote:
>
> git is a file system. It contains many different roots, which they
> call branches. Branches have 'scores'. I'm not the first person to
> notice a passing similarity to venti in that way.
>
> In venti, we can see every version of a file system, and even mount
> it, given a score. That's why things like yesterday are nice.
>
> To see a tree from git, I have to checkout a branch. I only have one
> branch checked out at a time. To see difference to a branch, I have to
> do this:
> git-diff
>
> I thought the intent of this project was to allow my to say, e.g.:
> mount /srv/git /tree
> cd /tree
> diff brancha branchb

Agreed. But you should realise that so far, you have only
been talking about a read-only usage of Git history. The
interesting part is to devise a way of adding to it. I don't
think I know good FS metaphors for all the things I need
there.

Thanks,
Roman.

P.S. Not that the r-o shouldn't be implemented...

[Roman Shaposhnik]

On Fri, Jun 26, 2009 at 12:28 AM, yy<yiyu...@gmail.com> wrote:
>
> I don't think I'm understanding well the situation here.
> If I got it right, all the operations ron wants are -or will be-
> possible in gitfs. The echo add operation is only to add files to the
> index [1], that tells git what files in the working dir are part of
> the repository (it serves the same purpose than .hgignore files in
> mercurial). If you just don't care about index files (ie. you clean up
> the working directory before commiting) just do echo add * before
> every commit.
> I wish Manzur gave a better explanation.

Speaking of better explanations: I encourage Manzur to write a
man page (so that he has a bit of a spec to code against). You
can take a look at it here:
http://github.com/manzur/gitfs/raw/f505ff91eec342b34604853e87a6d217db9f6112/gitfs.man

Feel free to comment on things which are still not 100% clear there.

Thanks,
Roman.

P.S. But that is not to say that we should just go ahead and redefine the
scope of the project...

[Francisco J. Ballesteros]

>
> Agreed. But you should realise that so far, you have only
> been talking about a read-only usage of Git history. The
> interesting part is to devise a way of adding to it. I don't
> think I know good FS metaphors for all the things I need
> there.
>

Why don't just let the user write to the tree?
If you write to a version that is a leaf it could add to that
branch. If you write to a version that is not a leaf it could
create a branch. When you want to make something that
is not the default you could perhaps do by renaming the
root dir (naming the version?) so that the new name describes the new version.

[Roman Shaposhnik]

On Fri, Jun 26, 2009 at 10:15 AM, Francisco J.
Ballesteros<nemo...@gmail.com> wrote:
>
>>
>> Agreed. But you should realise that so far, you have only
>> been talking about a read-only usage of Git history. The
>> interesting part is to devise a way of adding to it. I don't
>> think I know good FS metaphors for all the things I need
>> there.
>>
>
> Why don't just let the user write to the tree?

At the moment, I simply don't see how well would that
fit into the use cases that I personally have around Git.

> If you write to a version that is a leaf it could add to that
> branch. If you write to a version that is not a leaf it could
> create a branch.

That is not a problem with Git. You can have sprouts from
every point in the history. Branches are only "tags" they
have no special meaning.

> When you want to make something that
> is not the default you could perhaps do by renaming the
> root dir (naming the version?) so that the new name describes the new version.

Ok, let me describe what issues I see: the 99% Git use case
on a UNIX system is the following:
$ git checkout <sha-1>
# [edit, compile, debug]*
$ [git status, git diff]*
$ git add file1 file2 file3
$ git commit -m"Change #1"
$ git add file4 file5
$ git commit -m"Change #2"

What can we offer with git-fs? Lets see...
% gitfs /remote-repo /mnt
% cd /mnt/trees/<sha-1>
So far so good, but once we start editing and
(especially!) compiling it seems unreasonable
for gitfs to be a backing store for *.o files
and other junk that has no place in the repo.

But suppose we even bite the bullet and
have all the junk managed by gitfs. What's
the FS action to correspond to:
$ git add file1 file2 file3
$ git commit -m"Change #1"
?

Does my tree get switched after the commint?
(remember I'm still in /mnt/trees/<sha-1>,
while commit has advanced everything to
/mnt/trees/<sha-2>).

Thanks,
Roman.

[Francisco J. Ballesteros]

>
> But suppose we even bite the bullet and
> have all the junk managed by gitfs. What's
> the FS action to correspond to:
>      $ git add file1 file2 file3
>      $ git commit -m"Change #1"
> ?
>

You would touch file1, file2, file3, rm file4

and compile and all you like. (Using ram storage for temporary files, or
a local temporary tree if you like).

Then, after mk clean if you don't want binaries,
you'd go to /mnt/trees/ and say
mv <sha1> clone

(clone would commit and then rename sh1 to the right name).

Regarding the comment for commit, you can have a fake
/mnt/trees/<sha1.README> or whatever
so that you could edit it, write whatever you have to say, and then
commit would use that as
the comment.

In fact I'd not use shaN as the names. I mean, you could cd into that if that's
waht you know but I think I'd make the name a name. Not too git, but
more friendly.
It's easy to keep an index so the user could
echo sha myname >>index

An alternative I'm thinking about is this:

cd /mnt/trees
sha1 there is considered read only
Then you decide to work on sha1 and do a
mv sha1 your-symbolic-name
you add, remove, edit files.

you could handle commit as I said above.

In any case, these are just ideas. I hope it might help to make it more
close to an actual FS than it would be having to issue multiple ctls.

In particular, I see more natural editing a COMMENTS or README file
to tag commited trees than to use a ctl, and being able to use the tree as seem
instead of having to remember which files to add, remove etc.

If you want to git status, you could have a
/mnt/tree/name/STATUS
file that you could cat.

And so on.

[Roman Shaposhnik]

On Fri, Jun 26, 2009 at 10:54 AM, Francisco J.
Ballesteros<nemo...@gmail.com> wrote:
>
>>
>> But suppose we even bite the bullet and
>> have all the junk managed by gitfs. What's
>> the FS action to correspond to:
>>      $ git add file1 file2 file3
>>      $ git commit -m"Change #1"
>> ?
>>
> You would touch file1, file2, file3, rm file4

But that's not how "git add" works. You never
add ahead of time. What you do is -- you edit
existing files, and you compile, etc. Then you
tell git which files are expected to be part
of the next commit by doing git add.
So, as far as a the FS-view is concerned I still
want all of my files to there. That's why your
suggestion of "rm file4" (and I suppose you
forgot "rm file5") is not acceptable. Those files
have to remain visible in the FS-view.

> Then, after mk clean if you don't want binaries,

Wait! I don't want to do mk clean. That's not helpful
at all. I'm in the middle of project development and
all I want is to commit a few changes to a permanent
history and you ask me to do mk clean everytime
I want that?

> you'd go to /mnt/trees/ and say
> mv <sha1> clone
>
> (clone would commit and then rename sh1 to the right name).

That might be a reasonable first step of doing things, but
it is also too much "cd'ing here and there". In fact, it is no
better than writing custom messages to custom files.

Oh, and that also doesn't cover the case of merges (IOW,
a single commit having two [or more] parents).

> Regarding the comment for commit, you can have a fake
> /mnt/trees/<sha1.README> or whatever
> so that you could edit it, write whatever you have to say, and then
> commit would use that as
> the comment.

Sure I can. But than it wouldn't we Git, would it?

> In fact I'd not use shaN as the names.

When I said shaN, what I really meant was treeish, if you're familiar
with Git terminology. Could be a name, could be anything else:
http://book.git-scm.com/4_git_treeishes.html

> I mean, you could cd into that if that's
> waht you know but I think I'd make the name a name. Not too git, but
> more friendly.
> It's easy to keep an index so the user could
> echo sha myname >>index

I don't think I understand that.

> An alternative I'm thinking about is this:
>
> cd /mnt/trees
> sha1 there is considered read only
> Then you decide to work on sha1 and do a
> mv sha1 your-symbolic-name
> you add, remove, edit files.

I like this part, yes. More on that later.

> you could handle commit as I said above.

Well, it could be just me, but I don't think it is convenient. Or,
at least, it is no more convenient than writing custom messages
to custom files.

> In any case, these are just ideas. I hope it might help to make it more
> close to an actual FS than it would be having to issue multiple ctls.
>
> In particular, I see more natural editing a COMMENTS or README file
> to tag commited trees than to use a ctl, and being able to use the tree as seem
> instead of having to remember which files to add, remove etc.

I believe that it'll be much easier to expose commits and treeish objects
as first class citizens. Treeish is just a node in your DAG. Commits are
links in that same DAG. Now, may be, the trouble we are having is
partially due to the fact that DAGs don't fit all that well in FSese.

So, lets suspend the R/W ideas for the moment and answer a very
basic R/O question: what's the best way to represent commits topoly
in a filesystem? The best I can come up with right now is something
like is an alternative root called commits with the following structure:
% cd /mnt/commits/<treeish>
% ls
tree parents children message.txt author.txt committer.txt

This is straighforward, but still doesn't answer a basic question
of how to build a commit topology in cases where there are
multiple parents for a single commit. I don't know of anything,
but a ctl file accepting messages that could help here.

Thanks,
Roman.

[Francisco J. Ballesteros]

On Fri, Jun 26, 2009 at 8:32 PM, Roman Shaposhnik<shapo...@gmail.com> wrote:
>
> On Fri, Jun 26, 2009 at 10:54 AM, Francisco J.
> Ballesteros<nemo...@gmail.com> wrote:
>>
>>>
>>> But suppose we even bite the bullet and
>>> have all the junk managed by gitfs. What's
>>> the FS action to correspond to:
>>> $ git add file1 file2 file3
>>> $ git commit -m"Change #1"
>>> ?
>>>
>> You would touch file1, file2, file3, rm file4
>
> But that's not how "git add" works. You never
> add ahead of time. What you do is -- you edit

I know

> existing files, and you compile, etc. Then you
> tell git which files are expected to be part
> of the next commit by doing git add.

You implement the Fs, therefore you could know which files
are written/added/removed. Although perhaps you are thinking
on a more simple program for gsoc.

> So, as far as a the FS-view is concerned I still
> want all of my files to there. That's why your
> suggestion of "rm file4" (and I suppose you
> forgot "rm file5") is not acceptable. Those files
> have to remain visible in the FS-view.

I may commit a new version after removing file 5
(that´s what I meant). I really removed it from my local tree.

>> Then, after mk clean if you don't want binaries,
>
> Wait! I don't want to do mk clean. That's not helpful
> at all. I'm in the middle of project development and
> all I want is to commit a few changes to a permanent
> history and you ask me to do mk clean everytime
> I want that?

Well, let the binaries get it then :)

> So, lets suspend the R/W ideas for the moment and answer a very
> basic R/O question: what's the best way to represent commits topoly
> in a filesystem? The best I can come up with right now is something
> like is an alternative root called commits with the following structure:
> % cd /mnt/commits/<treeish>
> % ls
> tree parents children message.txt author.txt committer.txt
>
> This is straighforward, but still doesn't answer a basic question
> of how to build a commit topology in cases where there are
> multiple parents for a single commit. I don't know of anything,
> but a ctl file accepting messages that could help here.

I think I´d keep a tree plus a DERIVEDFROM or something
file that I could read to see if there´re more than one parent. In fact,
perhaps just a STATUS file reporting everything you want (including,
say, a line like
derived from: /git/versiona /git/versionb
)
So that I could cat the status and then paste the path, e.g., to cd into it.

I agree that the mv thing is perhaps as obscure as a ctl can be. But
I was just trying to throw away ideas about this (perhaps noise).

I though it could be fun to make gitfs maintain also the local copy
(perhaps supplying it as a path when you start it) so that you could forget
about which files you added, which ones, removed, etc. That was what
bothered me most the last time I used a version control system, having to
keep in mind that it was not a bunch a files, but a version control system.

Thus, if for the typical checkout - work - commit sequence you can forget
everything and just use the file tree, it would be much easier to use. For
complex commands it´s likely you are going to need ctls in any case, but
how many times do you require them?).

In any case, ignore this if it doesn´t makes sense for you, just use whatever
I might have said that you find useful. It´s just that I have played in the
past with the idea although I never did anything about it. (rant?)

[Anthony Sorace]

for deciding which files to include in archives: fossil has already
solved this problem in the 9p/file server world. why not re-use the
existing solution? decide whether you want the default for new file
creation to be auto-added or not (git and other VCSs tend to not;
plan9 file servers tend to), then flip DMTMP where you want the other
behavior.

commit is trickier, but mostly because of the desire to get commit
messages. otherwise there'd be plenty of good fs-based methods for
doing so (cdfs, for example, commits on removal of the directory
you've been working in, an operation otherwise non-sensical for WORM
things like CDs and VCS systems). i'd suggest focusing on just the
unsolved part (commit messages); the quickist hacky method is a
reserved file name you must write to before the rm will work.

[ron minnich]

I think the decision has been made that Plan 9 file systems are not an SVN.

Hence this git project, and port of mecurial.

Let's not distract -- the decision to use git has been made, and to
write gitfs. The only real question now is how gitfs works.

ron

[Roman Shaposhnik]

On Sun, Jun 28, 2009 at 5:53 AM, Anthony Sorace<ano...@gmail.com> wrote:
>
> for deciding which files to include in archives: fossil has already
> solved this problem in the 9p/file server world. why not re-use the
> existing solution?

Git commit is very much akin to fossil's snap. And what is snap?
A control message. That's exactly what Manzur's first design is.

> commit is trickier, but mostly because of the desire to get commit
> messages.

Not only that, but also because we would like to record the
lineage of commits. Sometimes even with multiple parents.

> i'd suggest focusing on just the
> unsolved part (commit messages); the quickist hacky method is a
> reserved file name you must write to before the rm will work.

I don't think you cant get Git semantics that way. Trying to convince
the world that you actually don't need Git semantics is really
quite counterproductive at this point.

Thanks,
Roman.

P.S. If I'm mistaken and once CAN get Git semantics via FS
primitives (and them alone, without resourting to ctl files)
I'd be very curious to know how.

[Anthony Sorace]

ron, i think you misunderstood my intent. i entirely agree that Plan 9
file systems are not VCSs; my point was only that some of the design
points overlap. fossil (and kenfs before it, and things like cdfs in
other ways) has answered a lot of questions about archival storage;
when we see what looks to be a new question in the design of gitfs,
it's worth looking at what fossil did, if anything, when faced with a
similar problem.

put another way: i wasn't saying "fossil solved this, just use fossil"
but rather "fossil solved a similar problem in another context; the
solution may apply in this context, as well". i think it's a
reasonable line of inquiry, and for the two particular questions posed
here (adding files and explicit commit actions), seems to be
reasonably informative on at least one of them (adds; Roman's later
message may disqualify the commit method).

[Dave Eckhardt]

I think having gitfs export writable working
directories is worth pursuing. Here are some
suggestions:

1. gitfs provides in a "status" file its idea of
"the expected future" of a working directory, kind
of like /proc/*/ns:

add foo.c
add foo.8
modify mkfile # no details, just the fact

2. The user can provide guidance/settings, e.g.,

% echo ignore foo.8 > ctl

means that "status" will no longer contain the
"add foo.8" line, but will now include "ignore foo.8".
These "settings" are as sticky as they should be,
whatever that means.

3. At any point you can "cp status ctl" if you like
what you see.

4. One command is "log _filename_" which says where to
find the next commit message. Like "ignore", it doesn't
"do anything" except stick around as a setting.

5. When you want to commit, "echo commit > ctl". This
could be set to fail if the log file hasn't been declared
or doesn't exist. Also, maybe it should fail if the
"status" file contains pending "add" commands (this
protects you against checking in a broken build, where
somebody expects foo.c to exist but you forgot to add
it).

If it's possible for "ignore" commands to be implemented
in terms of whatever git's ignore settings are, then
presumably they will be sticky across commits. If the
"status" file is sorted a little (e.g., add then modify
then ignore), most of the time all you need to do as a
user is glance at the top of the file, maybe issue an
ignore command or two, edit the file containing the
commit message (probably the same file as last time),
and do the commit.

As a slight modulation, any time the declared commit
file is deleted one could be synthesized based on a
diff of the working directory against its parent.

I don't know how naming of directories should work,
but I assume once a directory has been committed
a read-only version named by a hash will turn up.
Anyway, when possible, "parent/" should refer to
the previous commit, so "diff parent/foo.c foo.c"
and "diff parent/parent/foo.c foo.c" should work.

Dave Eckhardt

[maht.lawless]

Sticking my ignorant oar in

> Wait! I don't want to do mk clean.

how about
% mk commit
Comments : Fixed bug 123123
^D
%

[Roman Shaposhnik]

On Mon, Jun 29, 2009 at 3:58 AM, Dave Eckhardt<davide...@cs.cmu.edu> wrote:
>
> I think having gitfs export writable working
> directories is worth pursuing.  Here are some
> suggestions:

Agreed. But not, necessarily as part of *this* GSOC project.

> 1. gitfs provides in a "status" file its idea of
> "the expected future" of a working directory, kind
> of like /proc/*/ns:
>
> add foo.c
> add foo.8
> modify mkfile # no details, just the fact

This status *file* starts to look awfully like a subdirectory
to me. Any reason that you prefer it to be a file instead
of working-dir/.git/* ? (IOW, you do all FS operations
on the content of a working-dir and you have them mapped
to files visible under working-dir/.git/*).

> 2. The user can provide guidance/settings, e.g.,
>
> % echo ignore foo.8 > ctl
>
> means that "status" will no longer contain the
> "add foo.8" line, but will now include "ignore foo.8".
> These "settings" are as sticky as they should be,
> whatever that means.

Any reason not to adopt the git way of honoring
.gitignore files?

> 3. At any point you can "cp status ctl" if you like
> what you see.

If a status is really a subdirectory called index,
you can commit it the same way, can't you?

> 4. One command is "log _filename_" which says where to
> find the next commit message.  Like "ignore", it doesn't
> "do anything" except stick around as a setting.
>
> 5. When you want to commit, "echo commit > ctl".  This
> could be set to fail if the log file hasn't been declared
> or doesn't exist.  Also, maybe it should fail if the
> "status" file contains pending "add" commands (this
> protects you against checking in a broken build, where
> somebody expects foo.c to exist but you forgot to add
> it).

Now this starts to sound awfully complicated :-(

> As a slight modulation, any time the declared commit
> file is deleted one could be synthesized based on a
> diff of the working directory against its parent.
>
> I don't know how naming of directories should work,
> but I assume once a directory has been committed
> a read-only version named by a hash will turn up.
> Anyway, when possible, "parent/" should refer to
> the previous commit, so "diff parent/foo.c foo.c"
> and "diff parent/parent/foo.c foo.c" should work.

So... is the answer to my DAG question: don't worry
about anything but trees?

Thanks,
Roman.

[Roman Shaposhnik]

Sure, if there is NO side effect of "mk clean" I'd love
to have that.

Thanks,
Roman.

[Federico G. Benavento]

I'd like to hear what Manzur has to say about all this,
right now it's looking like there are too many food
critics in the kitchen (me included), but I'd like to know
what the cooker has to say.

thanks

--
Federico G. Benavento