[fossil-users] Why you should not shun

180 views
Skip to first unread message

Richard Hipp

unread,
Oct 4, 2011, 2:06:54 PM10/4/11
to fossil-users
Here is a true story, from just a few minutes ago, about how a very nearly lost 5 hours of intense work due to an ill-advised shun, and was only saved by the "fossil undo" command....

I keep all of my OpenOffice slide presentations in a private Fossil repository, so that I can develop presentations on my Ubuntu desktop, then sync to my macbook to take on the road.  I spent this morning working on one such presentation.  Part of my changes this morning involved a image that I copy/pasted from wikipedia.  I didn't realize it at the time, but the image was a 24MB PNG.  Everything worked fine.  I saved my work and checked it in.  But after checking in, Fossil then tried to push the changes to my master repository on a server at Hurricane Electric.  24MB of uncompressible image takes time to go over ADSL.  So I realized there was a problem and cancelled the sync.

How to fix this?  I down-sampled the image to a lower resolution JPEG, reducing its size from 24MB to about 400KB.  But I still had the old version stored in my local repository.  "No problem", I thought.  I'll just shun it.  Which I did.  Then I did "fossil update previous" so that I could commit the new version.  But (oops) that deleted all of my changes.  Everything was lost.  Despair and gnashing of teeth followed.

Fortunately, I later realized that I could do "fossil undo" to reverse the update and get my changes back.

Three lessons:

(1) Know what you are checking in before you do the check-in.  If I had simply paid attention to the size of the *.odp file that I was committing, I would have realized that it would be a slow syncer.

(2) Don't use shun.  Even if you think you know what you are doing, you are very likely to get into trouble.

(3) If you ignore the previous two warnings and find yourself in a bad situation, don't panic.  Think about where you are and plan your recovery carefully.  If I had panicked, I might have entered other Fossil commands which would have purged my undo stack.  As it was, I walked away from the keyboard and later realized that a "fossil undo" would get my old files back.

Learn from my mistake:  Don't shun on a whim.  The only reason to shun is to remove truly harmful content, such as spam.  I should have just let the sync proceed even if it was slower than I would like.  If you must shun, plan your shuns very, very carefully.  After shunning, test your repository before syncing.  Don't do something you will later regret.

--
D. Richard Hipp
d...@sqlite.org

Stephan Beal

unread,
Oct 4, 2011, 2:13:08 PM10/4/11
to fossil...@lists.fossil-scm.org
On Tue, Oct 4, 2011 at 8:06 PM, Richard Hipp <d...@sqlite.org> wrote:
Learn from my mistake:  Don't shun on a whim.  The only reason to shun is to remove truly harmful content, such as spam.  I should have just let the sync proceed even if it was slower than I would like.  If you must shun, plan your shuns very, very carefully.  After shunning, test your repository before syncing.  Don't do something you will later regret.

Some people would have run directly to the mailing list and aimed some hate mail at the author, but in your case i guess that wouldn't have worked too well ;).

--
----- stephan beal
http://wanderinghorse.net/home/stephan/

Jos Groot Lipman

unread,
Oct 4, 2011, 2:19:43 PM10/4/11
to fossil...@lists.fossil-scm.org
Is -without shunning- the 24MB-version still present in your local repository? Won't is be synced sooner or later anyhow now?
 
--
Jos Groot Lipman


From: fossil-use...@lists.fossil-scm.org [mailto:fossil-use...@lists.fossil-scm.org] On Behalf Of Richard Hipp
Sent: dinsdag 4 oktober 2011 20:07
To: fossil-users
Subject: [fossil-users] Why you should not shun

Konstantin Khomoutov

unread,
Oct 4, 2011, 2:26:36 PM10/4/11
to fossil...@lists.fossil-scm.org
On Tue, 4 Oct 2011 20:19:43 +0200
"Jos Groot Lipman" <dont...@home.nl> wrote:

> Is -without shunning- the 24MB-version still present in your local
> repository? Won't is be synced sooner or later anyhow now?

It will be present of course. To quote the original message:


"I should have just let the sync proceed even if it was slower than I
would like."

_______________________________________________
fossil-users mailing list
fossil...@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Remigiusz Modrzejewski

unread,
Oct 4, 2011, 2:37:19 PM10/4/11
to fossil...@lists.fossil-scm.org

On Oct 4, 2011, at 8:06 PM, Richard Hipp wrote:

> (3) If you ignore the previous two warnings and find yourself in a bad
> situation, don't panic. Think about where you are and plan your recovery
> carefully. If I had panicked, I might have entered other Fossil commands
> which would have purged my undo stack. As it was, I walked away from the
> keyboard and later realized that a "fossil undo" would get my old files
> back.

And why exactly do we have a single-level stack?


Kind regards,
Remigiusz Modrzejewski

v...@lavabit.com

unread,
Oct 4, 2011, 2:39:28 PM10/4/11
to fossil...@lists.fossil-scm.org
> (2) Don't use shun. Even if you think you know what you are doing, you
> are
> very likely to get into trouble.

> Learn from my mistake: Don't shun on a whim. The only reason to shun is


> to
> remove truly harmful content, such as spam. I should have just let the
> sync
> proceed even if it was slower than I would like. If you must shun, plan
> your shuns very, very carefully. After shunning, test your repository
> before syncing. Don't do something you will later regret.

These are very good points. When fossil is used for version control (since
that's its main purpose). But fossil can be used for other purposes as
well, because it is such a great multifunctional swiss army knife - and
version control is just a nice free bonus then. There are some corner use
cases which I'd like to mention, now that best shunning (or rather not
shunning!) practices are being discussed.

Fossil makes a great mini CMS for hosting a small site. Built in wiki
makes it sweet. I think it is also possible to have something like a
forum, with wiki pages used as categories and tickets + comments to them
used as topics. In these scenarios, if anonymous posting to wikis and
tickets is allowed - there's a higher risk to get junk or malicious
content in the repo which may require forceful purge. A couple of places
where fossil could make it easier for a repo admin to clean up would be:

- in wiki page - shun the wiki page completely and also all changes to it
- in ticket page - shun a ticket completely, also all comments
- in ticket page - option to check certain ticket comments and nuke them
all in one shot

Konstantin Khomoutov

unread,
Oct 4, 2011, 2:41:42 PM10/4/11
to fossil...@lists.fossil-scm.org
On Tue, 4 Oct 2011 14:06:54 -0400
Richard Hipp <d...@sqlite.org> wrote:

[...]


> (1) Know what you are checking in before you do the check-in. If I
> had simply paid attention to the size of the *.odp file that I was
> committing, I would have realized that it would be a slow syncer.

Just a thought: what if `fossil status` showed the size of the affected
files? Something akin to what `vdir` or `ls -l` do but using the
metadata which is to be stored within the fossil repository upon commit.
That would be overkill for `fossil changes` but `fossil status`
already shows extended information so it seems that making it "more
extended" might turn out to be useful.

[...]


> (3) If you ignore the previous two warnings and find yourself in a bad
> situation, don't panic. Think about where you are and plan your
> recovery carefully. If I had panicked, I might have entered other
> Fossil commands which would have purged my undo stack. As it was, I
> walked away from the keyboard and later realized that a "fossil undo"
> would get my old files back.

I think this has been discussed already, but to me it appears something
resembling Git's "reflog" functionality could be of help in the cases
like this one. The basic idea is to implement a storage (kept in the
checkout database) for things `fossil undo` operates on, but with a
time-based expiration instead of being a stack (Git uses 30 days by
default IIRC). Conceptually Git's reflog looks just like a single line
of history ordered by time of events forming it.

Stephan Beal

unread,
Oct 4, 2011, 2:50:47 PM10/4/11
to fossil...@lists.fossil-scm.org
On Tue, Oct 4, 2011 at 8:39 PM, <v...@lavabit.com> wrote:
Fossil makes a great mini CMS for hosting a small site. Built in wiki
makes it sweet. I think it is also possible to have something like a
forum, with wiki pages used as categories and tickets + comments to them
used as topics.

Side note: the JSON API makes it possible to use wikis (and soon tickets) from within your own custom HTML interface, meaning that it can be used as a wiki back-end independently of the version control features (but automatically inheriting those features, should you need to back-rev a wiki page or whatever). Additionally, it allows clients to use their own wiki formats if they like (because it can serve the content in raw or fossil-parsed forms) provided that the client have a client-side renderer for the content. Creating a demo of such an app is on my todo list. i have a wiki management app (implemented in HTML+JS) which uses a different (but conceptually similar) back-end and it should not be difficult to port to a fossil back-end. The JSON API has all the features needed for this but i would first like to make more progress on the artifact fetching APIs so that it will be possible (client-side) to look through historical versions of pages.

A _very_ rough prototype of this idea can be seen here:


but you cannot save from that demo page unless you crack my password ;).

Erlis Vidal

unread,
Oct 4, 2011, 2:53:51 PM10/4/11
to fossil...@lists.fossil-scm.org
Before I ask

I would like to see the shun documentation, I'm not able to find it here http://www.sqlite.org/debug1/help

nor the command fossil help shun helps.

Thanks
Erlis

Stephan Beal

unread,
Oct 4, 2011, 2:55:42 PM10/4/11
to fossil...@lists.fossil-scm.org
n Tue, Oct 4, 2011 at 8:53 PM, Erlis Vidal <er...@erlisvidal.com> wrote:
I would like to see the shun documentation, I'm not able to find it here http://www.sqlite.org/debug1/help

Why would it be on the sqlite site? It's here:


Erlis Vidal

unread,
Oct 4, 2011, 2:57:15 PM10/4/11
to fossil...@lists.fossil-scm.org
I have fossil documentations bookmarked everywhere, I'll be fixing those bookmarks, thanks. Maybe I have to blame google  :)

Richard Hipp

unread,
Oct 4, 2011, 3:00:12 PM10/4/11
to fossil...@lists.fossil-scm.org
On Tue, Oct 4, 2011 at 2:37 PM, Remigiusz Modrzejewski <lr...@maxnet.org.pl> wrote:

On Oct 4, 2011, at 8:06 PM, Richard Hipp wrote:

> (3) If you ignore the previous two warnings and find yourself in a bad
> situation, don't panic.  Think about where you are and plan your recovery
> carefully.  If I had panicked, I might have entered other Fossil commands
> which would have purged my undo stack.  As it was, I walked away from the
> keyboard and later realized that a "fossil undo" would get my old files
> back.

And why exactly do we have a single-level stack?

I was concerned about using too much disk space if I let the stack grow without bound.  The undo/redo content is stored in the _FOSSIL_ database.  When you are dealing with 25MB files, it doesn't take too many operations to results in a really big undo/redo stack.
 


Kind regards,
Remigiusz Modrzejewski



_______________________________________________
fossil-users mailing list
fossil...@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Erlis Vidal

unread,
Oct 4, 2011, 3:55:32 PM10/4/11
to fossil...@lists.fossil-scm.org
Hi Richard,

I would like to understand how you lost your changes, but probably because I'm ignorant of the internals I'm not able to understand what just happened here.

I've read the Shun documentation and the Update documentation, and I cannot see how what you said here could occur.

On Tue, Oct 4, 2011 at 2:06 PM, Richard Hipp <d...@sqlite.org> wrote:

"No problem", I thought.  I'll just shun it.  Which I did.  Then I did "fossil update previous" so that I could commit the new version.  But (oops) that deleted all of my changes.  Everything was lost.  Despair and gnashing of teeth followed.


It looks like the udpate command was the one that actually create the problem, but following your entire story, it seems that the situation was created by the Shun you did.

Please (if is not much to ask) can you explain what creates the issue?

Sincerely,
Erlis

Richard Hipp

unread,
Oct 4, 2011, 3:59:14 PM10/4/11
to fossil...@lists.fossil-scm.org
On Tue, Oct 4, 2011 at 3:55 PM, Erlis Vidal <er...@erlisvidal.com> wrote:
Hi Richard,

I would like to understand how you lost your changes, but probably because I'm ignorant of the internals I'm not able to understand what just happened here.

I've read the Shun documentation and the Update documentation, and I cannot see how what you said here could occur.

On Tue, Oct 4, 2011 at 2:06 PM, Richard Hipp <d...@sqlite.org> wrote:

"No problem", I thought.  I'll just shun it.  Which I did.  Then I did "fossil update previous" so that I could commit the new version.  But (oops) that deleted all of my changes.  Everything was lost.  Despair and gnashing of teeth followed.


It looks like the udpate command was the one that actually create the problem, but following your entire story, it seems that the situation was created by the Shun you did.

Right.  I "shunned" my most recent check-in, removing the content from the repository.  Then I did an "update" to the previous check-in so that I could check in the changes again, but this time with the new smaller JPEG instead of the massive GIF.  But the "update" overwrote *all* of the changes I had made over the course of the morning.

So the "shun" removed the content from the repository.  And the "update" removed the content from the disk.

Fortunately, there was still one other copy in the undo stack....

 

Please (if is not much to ask) can you explain what creates the issue?

Sincerely,
Erlis

_______________________________________________
fossil-users mailing list
fossil...@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Remigiusz Modrzejewski

unread,
Oct 4, 2011, 4:09:39 PM10/4/11
to fossil...@lists.fossil-scm.org

On Oct 4, 2011, at 9:00 PM, Richard Hipp wrote:

> On Tue, Oct 4, 2011 at 2:37 PM, Remigiusz Modrzejewski
> <lr...@maxnet.org.pl>wrote:
>>

>> And why exactly do we have a single-level stack?
>>
>
> I was concerned about using too much disk space if I let the stack grow
> without bound. The undo/redo content is stored in the _FOSSIL_ database.
> When you are dealing with 25MB files, it doesn't take too many operations to
> results in a really big undo/redo stack.

But for most practical cases this is not an issue. I guess I'd be happier if it was time-based with maybe a hard limit somewhere around 1GB. I've never done a stupid thing with Fossil... Yet.

Erlis Vidal

unread,
Oct 4, 2011, 4:09:41 PM10/4/11
to fossil...@lists.fossil-scm.org
And that's what I don't understand, how could the Update "overwrote" your changes.

This is from the update help command

Change the version of the current checkout to VERSION.  Any uncommitted
changes are retained and applied to the new checkout.

If you do the same steps, without the shun, just modify some files, update previous. Would that remove my change in the current checkout?

Thanks,
Erlis

Richard Hipp

unread,
Oct 4, 2011, 4:15:10 PM10/4/11
to fossil...@lists.fossil-scm.org
On Tue, Oct 4, 2011 at 4:09 PM, Erlis Vidal <er...@erlisvidal.com> wrote:
And that's what I don't understand, how could the Update "overwrote" your changes.

This is from the update help command

Change the version of the current checkout to VERSION.  Any uncommitted
changes are retained and applied to the new checkout.

Because of the shun, Fossil was confused.  It thought that the changes had been committed, since they had been committed immediately prior to the shun.  So the files on disk were marked as clean.  Hence, Fossil overwrote them.

 

Bill Burdick

unread,
Oct 4, 2011, 4:33:22 PM10/4/11
to fossil...@lists.fossil-scm.org
You could handle that case -- if you shun the current commit, then check the disk against the current repo contents and mark files that are different as uncommitted.


Bill

Richard Hipp

unread,
Oct 4, 2011, 4:43:15 PM10/4/11
to fossil...@lists.fossil-scm.org
On Tue, Oct 4, 2011 at 4:33 PM, Bill Burdick <bill.b...@gmail.com> wrote:
You could handle that case -- if you shun the current commit, then check the disk against the current repo contents and mark files that are different as uncommitted.

But to do so would be hard.  Shunning is done in the repository database file:  "*.fossil".  But the information about which files are uncommitted is held in the check-out database file: "_FOSSIL_".  And while the check-out database knows about the existance of the repository database, the repository does not know about individual checkouts.  So if you run the "fossil rebuild" following the shun from the web interface (which does not know about individual check-outs) the rebuild operation has no way of finding the check-out databases and patching them up.

So what you'd probably have to do is modify the "update" command (as well as other similar commands) to check to see if any shunning has taken place and modify their behavior accordingly.  It would be tricky to implement and get right.  And it would be a lot of work.  All for something that is officially discouraged.

 

Lluís Batlle i Rossell

unread,
Oct 4, 2011, 4:49:56 PM10/4/11
to fossil...@lists.fossil-scm.org
On Tue, Oct 04, 2011 at 04:15:10PM -0400, Richard Hipp wrote:
> On Tue, Oct 4, 2011 at 4:09 PM, Erlis Vidal <er...@erlisvidal.com> wrote:
>
> > And that's what I don't understand, how could the Update "overwrote" your
> > changes.
> >
> > This is from the update help command
> >
> > *Change the version of the current checkout to VERSION. Any uncommitted

> > changes are retained and applied to the new checkout.
> > *

>
>
> Because of the shun, Fossil was confused. It thought that the changes had
> been committed, since they had been committed immediately prior to the
> shun. So the files on disk were marked as clean. Hence, Fossil overwrote
> them.

Ah, I thought you had shunned the artifact of the PNG file, not that of the
checkin. What would have happened, if you had removed only the artifact of the
PNG?

In any case, what do you think of a (at least) temporary 'fail-fast' code that
will make fossil die in that unfortunate situation (the update removing files)?
I'd prefer a fast abort() than the behaviour you saw.

Thank you,
Lluís.

Erlis Vidal

unread,
Oct 4, 2011, 4:50:32 PM10/4/11
to fossil...@lists.fossil-scm.org
You shun a commit or a file in a commit? Is in fossil the shun generating a different commit?

you can delete with git files that has history with

git filter-branch --index-filter 'git rm -r --cached --ignore-unmatch file_name' HEAD

and that will generate a new commit without the file you just deleted (yes I know, that git command is really cryptic that's why I'm liking fossil so much :)

but I don't see how this will affect modified files. If the files are commited before, then you will never will lose those changes... unless you have shun those files, but I don't think that's the case, the file that was shun was a big image file, or I'm wrong?

I don't understand, but I feel that maybe I have to learn more fossil before continue, because I have the feeling this was more a bug than a misuse of a feature.

Why the shun confused the update? That's the question I'll try to answer...

Thanks everyone...
Erlis


On Tue, Oct 4, 2011 at 4:33 PM, Bill Burdick <bill.b...@gmail.com> wrote:

Richard Hipp

unread,
Oct 4, 2011, 4:53:48 PM10/4/11
to fossil...@lists.fossil-scm.org
On Tue, Oct 4, 2011 at 4:50 PM, Erlis Vidal <er...@erlisvidal.com> wrote:
You shun a commit or a file in a commit? Is in fossil the shun generating a different commit?

I shunned both the commit artifact and the file that was changed, the openoffice *.odp file.
 

you can delete with git files that has history with

git filter-branch --index-filter 'git rm -r --cached --ignore-unmatch file_name' HEAD

and that will generate a new commit without the file you just deleted (yes I know, that git command is really cryptic that's why I'm liking fossil so much :)

but I don't see how this will affect modified files. If the files are commited before, then you will never will lose those changes... unless you have shun those files, but I don't think that's the case, the file that was shun was a big image file, or I'm wrong?

Both the commit and the file were shunned.
 

I don't understand, but I feel that maybe I have to learn more fossil before continue, because I have the feeling this was more a bug than a misuse of a feature.

The problem was operator error, not a bug in fossil.
 

Erlis Vidal

unread,
Oct 4, 2011, 4:56:59 PM10/4/11
to fossil...@lists.fossil-scm.org
Ok I got it, following your story it was not clear that you shun the commit, thinking that you just shun the big file was what lead me to the big confusion...

Regards,
Erlis

Mike Meyer

unread,
Oct 4, 2011, 5:34:06 PM10/4/11
to fossil...@lists.fossil-scm.org
On Tue, Oct 4, 2011 at 1:50 PM, Erlis Vidal <er...@erlisvidal.com> wrote:
You shun a commit or a file in a commit? Is in fossil the shun generating a different commit?

you can delete with git files that has history with

git filter-branch --index-filter 'git rm -r --cached --ignore-unmatch file_name' HEAD

and that will generate a new commit without the file you just deleted (yes I know, that git command is really cryptic that's why I'm liking fossil so much :)

but I don't see how this will affect modified files. If the files are commited before, then you will never will lose those changes... unless you have shun those files, but I don't think that's the case, the file that was shun was a big image file, or I'm wrong?

I don't understand, but I feel that maybe I have to learn more fossil before continue, because I have the feeling this was more a bug than a misuse of a feature.

You've just hit on a philosophical difference between git and fossil. In the fossil community - and hence in fossil itself - development history is pretty much sacrosanct. The very name "fossil" was to chosen to reflect the unchanging nature of things in that history.

In git (or rather, the git community), the development history is part of the published aspect of the project, so it provides tools for rearranging that history so you can present what you "should" have done rather than what you actually did.

As with any other tool, if you don't agree with the philosophy behind fossil, you're going find it really frustrating to use. That's why I don't use git. To many operations that should be simple wound up being "just use rebase to ...", and I'd rather avoid anything that mucks with history.

   <mike

Bill Burdick

unread,
Oct 4, 2011, 6:05:52 PM10/4/11
to fossil...@lists.fossil-scm.org
On Tue, Oct 4, 2011 at 3:43 PM, Richard Hipp <d...@sqlite.org> wrote:
On Tue, Oct 4, 2011 at 4:33 PM, Bill Burdick <bill.b...@gmail.com> wrote:
You could handle that case -- if you shun the current commit, then check the disk against the current repo contents and mark files that are different as uncommitted.

But to do so would be hard.  Shunning is done in the repository database file:  "*.fossil".  But the information about which files are uncommitted is held in the check-out database file: "_FOSSIL_".  And while the check-out database knows about the existance of the repository database, the repository does not know about individual checkouts.  So if you run the "fossil rebuild" following the shun from the web interface (which does not know about individual check-outs) the rebuild operation has no way of finding the check-out databases and patching them up.

So what you'd probably have to do is modify the "update" command (as well as other similar commands) to check to see if any shunning has taken place and modify their behavior accordingly.  It would be tricky to implement and get right.  And it would be a lot of work.  All for something that is officially discouraged.

Oh, I thought re-marking recently committed files would use the same information that the undo command uses.


Bill

Gilles

unread,
Oct 4, 2011, 7:53:29 PM10/4/11
to fossil...@lists.fossil-scm.org
On Tue, 4 Oct 2011 14:06:54 -0400, Richard Hipp
<d...@sqlite.org> wrote:
>Learn from my mistake: Don't shun on a whim. The only reason to shun is to
>remove truly harmful content, such as spam. I should have just let the sync
>proceed even if it was slower than I would like.

Thanks for sharing this experience. I think it'd be useful to update
the documentation so that users know what to do when they eg. commited
big files by mistake, and how to cancel this commit entirely so that
the repo doesn't grow needlessly

www.sqlite.org/debug1/doc/trunk/www/shunning.wiki

Lluís Batlle i Rossell

unread,
Oct 5, 2011, 3:09:43 AM10/5/11
to fossil...@lists.fossil-scm.org

I use to make an analogy to open-software/closed-software, talking about
open-development/closed-development.

git looks to me as a tool for publishing development steps, not necessarily very
related to the development history. The 'git version graph' is determined by the
users the same way as they choose filenames or directories, and many git groups
even consider keeping the git graph close to development history as a 'basic and
inexpert approach' to git.

And as determined by the creator, Linus, git is more like a tool meant for
sharing code preserving contributions authorship, other than storing development
history. I think Linus did not want for the kernel a storage for development
history; he only wanted to manage the contributions.

Regards,
Lluís.

Michal Suchanek

unread,
Oct 5, 2011, 12:06:07 PM10/5/11
to fossil...@lists.fossil-scm.org
2011/10/5 Lluís Batlle i Rossell <viri...@gmail.com>:

It is because it is not useful to have every *actual file edit*
recorded as commit. You often edit multiple files to perform a change,
or do some work on the project, save it, go for lunch, and then do
some more work on it.

From a totally purist point of view should the history be recorded
every time you save a file? Every time you write or delete a
character? No, it is saved in logical pieces - commits.

And when you find an issue with a commit that is some way back in your
personal branch it is more logical and easier to review your branch if
you append the fix to the commit where it belongs logically or if you
append it at the top of the history interspersed with some possibly
unrelated changes?

You can always create a new branch and put the commits in a different
order on that branch, merging the original commit with the fixup,
start applying your local changes at a later time in the upstream
project history. Git provides more tools to make this history
rewriting easier so that the presented development steps are easier to
understand to a reviewer.

You have to find some balance between shunning all history and
recording every character typed.

I don't mind fossil keeping these dead branches. In fact, git does so
too until you manually prune the repository of unreferenced objects.
You can also select to show only live branches in fossil so the
difference is mostly cosmetical, as are all the differences between
distributed VCSes.

In one you get a command shipped as part of core, in other you can
find a dozen contributed scripts that do the same or roll your own.
It's just a matter of taste or the core tools and available scripts
being more fitting for a particular task at hand.

Thanks

Michal

Lluís Batlle i Rossell

unread,
Oct 5, 2011, 12:24:30 PM10/5/11
to fossil...@lists.fossil-scm.org
On Wed, Oct 05, 2011 at 06:06:07PM +0200, Michal Suchanek wrote:
> 2011/10/5 Lluís Batlle i Rossell <viri...@gmail.com>:
> > git looks to me as a tool for publishing development steps, not necessarily very
> > related to the development history. The 'git version graph' is determined by the
> > users the same way as they choose filenames or directories, and many git groups
> > even consider keeping the git graph close to development history as a 'basic and
> > inexpert approach' to git.
>
> It is because it is not useful to have every *actual file edit*
> recorded as commit. You often edit multiple files to perform a change,
> or do some work on the project, save it, go for lunch, and then do
> some more work on it.
>
> From a totally purist point of view should the history be recorded
> every time you save a file? Every time you write or delete a
> character? No, it is saved in logical pieces - commits.

I consider commits as relevant milestones in the development, and I use branches
to keep differentg granularity of commits.

> And when you find an issue with a commit that is some way back in your
> personal branch it is more logical and easier to review your branch if
> you append the fix to the commit where it belongs logically or if you
> append it at the top of the history interspersed with some possibly
> unrelated changes?

Exactly, these are usual arguments among git lovers.

Nevertheless, git manuals say "every commit represents a state of the tree".
Then you may find logical to expect that every commit log talks about a state of
the tree. BUT all this goes away, when you start using 'rebase'. The commit logs
once written do not match anymore the 'state of the tree' at the time of commit
log writing. Assertions in commit logs like "this part works" may easily not
fit what was meant at the commit time, for example.

git even comes with 'quick automatic rebase' facilities, that allow not
rechecking any commit log (a task hard to do, in any case).

Making a review 'easier' by manipulating history can also end up being some
kind of manipulation to a reviewer. A reviewer may judge better by understanding
the development.

> I don't mind fossil keeping these dead branches. In fact, git does so
> too until you manually prune the repository of unreferenced objects.
> You can also select to show only live branches in fossil so the
> difference is mostly cosmetical, as are all the differences between
> distributed VCSes.

git even cannot store branch histories, beyond the big resulting graph of
unnamed commits. For example, a branch name can only designate a commit (leaf),
not more than one. And branch names are not part of history.
Some git deployments even forbid or limit pushing new branches to a public
repository.

On the other hand, fossil will always respect the checkin order, and the states
related to every checkin log. As many other VCS, but not with git.

Regards,
Lluís

Ron Wilson

unread,
Oct 5, 2011, 12:54:33 PM10/5/11
to fossil...@lists.fossil-scm.org
On Tue, Oct 4, 2011 at 3:59 PM, Richard Hipp <d...@sqlite.org> wrote:
> Fortunately, there was still one other copy in the undo stack....

I would add a fourth lesson: Always backup your work.

Konstantin Khomoutov

unread,
Oct 5, 2011, 1:56:14 PM10/5/11
to fossil...@lists.fossil-scm.org
On Wed, 5 Oct 2011 18:24:30 +0200
Lluís Batlle i Rossell <viri...@gmail.com> wrote:

[...]


> > And when you find an issue with a commit that is some way back in
> > your personal branch it is more logical and easier to review your
> > branch if you append the fix to the commit where it belongs
> > logically or if you append it at the top of the history
> > interspersed with some possibly unrelated changes?
>
> Exactly, these are usual arguments among git lovers.
>
> Nevertheless, git manuals say "every commit represents a state of the
> tree". Then you may find logical to expect that every commit log
> talks about a state of the tree. BUT all this goes away, when you
> start using 'rebase'. The commit logs once written do not match
> anymore the 'state of the tree' at the time of commit log writing.
> Assertions in commit logs like "this part works" may easily not fit
> what was meant at the commit time, for example.
>
> git even comes with 'quick automatic rebase' facilities, that allow
> not rechecking any commit log (a task hard to do, in any case).
>
> Making a review 'easier' by manipulating history can also end up
> being some kind of manipulation to a reviewer. A reviewer may judge
> better by understanding the development.

That sort of "we don't need it, we don't need it" mantra is a typical
case of the famous "Blub paradox".
I mean, if we have two DVCS tools one of which makes you able to
rewrite history and another one which doesn't, the first one is more
powerful _in this particular respect_. It's as simple as that.
By supporting a feature, the tool does not force you to employ that
feature in your workflow.

Mike Meyer

unread,
Oct 5, 2011, 2:12:31 PM10/5/11
to fossil...@lists.fossil-scm.org
On Wed, Oct 5, 2011 at 10:56 AM, Konstantin Khomoutov <flat...@users.sourceforge.net> wrote:
That sort of "we don't need it, we don't need it" mantra is a typical
case of the famous "Blub paradox".
I mean, if we have two DVCS tools one of which makes you able to
rewrite history and another one which doesn't, the first one is more
powerful _in this particular respect_.  It's as simple as that.
By supporting a feature, the tool does not force you to employ that
feature in your workflow.

First,  note that there is a difference between "rewriting history", which is what git supports, and "deleting unwanted items", which is the request that started this.

Second, that a feature doesn't affect you if you "just don't use it" is a fallacy. Sure, I think history should be sacrosanct, so I won't use rebase even if it's available. That doesn't stop others on the project (who  don't agree with me) from using it . The only way to make sure that doesn't happen is to not have the feature available *at all*.

Finally, having a feature that powerful available tends to cause the API to *not* include safe versions of common tasks that that dangerous feature handles. To see what I mean, take a look at mercurial, which shares the fossil philosophy, but provides a (disabled by default) rebase command. It has a number of commands (*not* disabled by default) for tasks that are handled in git using rebase. Unlike rebase, those commands are safe, in that mistakes with them can't wreck your repo the way a mistake with rebase can. It may not make a difference if you never make mistakes, but in that case you don't need rebase.

Bottom line: while "more features" may imply "more powerful", it doesn't imply "better".

     <mike

Michal Suchanek

unread,
Oct 5, 2011, 2:38:30 PM10/5/11
to fossil...@lists.fossil-scm.org
On 5 October 2011 20:12, Mike Meyer <m...@mired.org> wrote:
> On Wed, Oct 5, 2011 at 10:56 AM, Konstantin Khomoutov
> <flat...@users.sourceforge.net> wrote:
>>
>> That sort of "we don't need it, we don't need it" mantra is a typical
>> case of the famous "Blub paradox".
>> I mean, if we have two DVCS tools one of which makes you able to
>> rewrite history and another one which doesn't, the first one is more
>> powerful _in this particular respect_.  It's as simple as that.
>> By supporting a feature, the tool does not force you to employ that
>> feature in your workflow.
>
> First,  note that there is a difference between "rewriting history", which
> is what git supports, and "deleting unwanted items", which is the request
> that started this.
> Second, that a feature doesn't affect you if you "just don't use it" is a
> fallacy. Sure, I think history should be sacrosanct, so I won't use rebase
> even if it's available. That doesn't stop others on the project (who  don't
> agree with me) from using it . The only way to make sure that doesn't happen
> is to not have the feature available *at all*.

No, that's not how things work.

Rebase is nothing more than taking commits comprising a branch from
its branching point and applying them somewhere else. Not that
complicated, really.

As applying patches is doable, even patches stored in fossil history
already, this is doable with a bit of scripting around fossil as it is
doable with git. So not having the feature is not perfect defence, it
only defends against people who don't care about the feature. People
who find it useful for their workflow and want to use fossil for
compatibility with upstream of for other features it provides can do
so.

As an alternative you could use export/rebase/import which is just
different way to script this on a higher level.

>Finally, having a feature that powerful available tends to cause the API to *not* include safe versions of common tasks that >that dangerous feature handles. To see what I mean, take a look at mercurial, which shares the fossil philosophy, but >provides a (disabled by default) rebase command. It has a number of commands (*not* disabled by default) for tasks that >are handled in git using rebase. Unlike rebase, those commands are safe, in that mistakes with them can't wreck your repo >the way a mistake with rebase can. It may not make a difference if you never make mistakes, but in that case you don't >need rebase.

You can shoot yourself in the foot with any tool. Using tools that
suit your workflow makes that less likely, being dangerous or not.

I personally make a copy of any branch before rebasing it because in
the case rebasing fails you can still revert it but comparing with the
old branch makes it much clearer where you ended up.

If you never delete anything it makes your work safer but your
repository more hairy.

Thanks

Michal

Konstantin Khomoutov

unread,
Oct 5, 2011, 2:40:48 PM10/5/11
to fossil...@lists.fossil-scm.org
On Wed, 5 Oct 2011 11:12:31 -0700
Mike Meyer <m...@mired.org> wrote:

> > That sort of "we don't need it, we don't need it" mantra is a
> > typical case of the famous "Blub paradox".
> > I mean, if we have two DVCS tools one of which makes you able to
> > rewrite history and another one which doesn't, the first one is more
> > powerful _in this particular respect_. It's as simple as that.
> > By supporting a feature, the tool does not force you to employ that
> > feature in your workflow.
> First, note that there is a difference between "rewriting history",
> which is what git supports, and "deleting unwanted items", which is
> the request that started this.

Correct.

> Second, that a feature doesn't affect you if you "just don't use it"
> is a fallacy. Sure, I think history should be sacrosanct, so I won't
> use rebase even if it's available. That doesn't stop others on the
> project (who don't agree with me) from using it . The only way to
> make sure that doesn't happen is to not have the feature available
> *at all*.

I'm not entirely convinced.
Look at the workflow used by the Git team: they maintain the set of
well known branches, of which the only one, named "pu" ("proposed
updates"), is allowed to be rebased and that's actually what happens
with it each time the new release is made--it's cut from the master
afresh. I mean, while every committer has `git rebase` at their
disposal and knows how it works this does not mean they go off and break
the repository with rebases. So your point is only really valid when
the project is run by a bunch of idiots or complete newbies.

> Finally, having a feature that powerful available tends to cause the
> API to *not* include safe versions of common tasks that that
> dangerous feature handles. To see what I mean, take a look at
> mercurial, which shares the fossil philosophy, but provides a
> (disabled by default) rebase command. It has a number of commands
> (*not* disabled by default) for tasks that are handled in git using
> rebase. Unlike rebase, those commands are safe, in that mistakes with
> them can't wreck your repo the way a mistake with rebase can. It may
> not make a difference if you never make mistakes, but in that case
> you don't need rebase.

Git handles it from the opposite direction by having "the reflog".
But I find this point to be valid, yes, safety nets are a must when it
comes to handling precious data. BTW I'm a fan of `fossil undo` for
that matter.

> Bottom line: while "more features" may imply "more powerful", it
> doesn't imply "better".

Moot point.
I really miss Git's "index" and `git add --patch` here.
Is Fossil better than Git in this respect by not having those "more
features"? Surely it completely is in the eye of the beholder.

Mike Meyer

unread,
Oct 5, 2011, 2:49:08 PM10/5/11
to fossil...@lists.fossil-scm.org
On Wed, Oct 5, 2011 at 11:38 AM, Michal Suchanek <hram...@centrum.cz> wrote:
On 5 October 2011 20:12, Mike Meyer <m...@mired.org> wrote:
> On Wed, Oct 5, 2011 at 10:56 AM, Konstantin Khomoutov
> <flat...@users.sourceforge.net> wrote:
>>
>> That sort of "we don't need it, we don't need it" mantra is a typical
>> case of the famous "Blub paradox".
>> I mean, if we have two DVCS tools one of which makes you able to
>> rewrite history and another one which doesn't, the first one is more
>> powerful _in this particular respect_.  It's as simple as that.
>> By supporting a feature, the tool does not force you to employ that
>> feature in your workflow.
>
> First,  note that there is a difference between "rewriting history", which
> is what git supports, and "deleting unwanted items", which is the request
> that started this.
> Second, that a feature doesn't affect you if you "just don't use it" is a
> fallacy. Sure, I think history should be sacrosanct, so I won't use rebase
> even if it's available. That doesn't stop others on the project (who  don't
> agree with me) from using it . The only way to make sure that doesn't happen
> is to not have the feature available *at all*.

No, that's not how things work.

Rebase is nothing more than taking commits comprising a branch from
its branching point and applying them somewhere else. Not that
complicated, really.

No, that's not how things work. Either that, or the git rebase documentation is really badly broken. It sure looks to me like rebase moves the patches from one point to another in the repo.
 
As applying patches is doable, even patches stored in fossil history
already, this is doable with a bit of scripting around fossil as it is
doable with git. So not having the feature is not perfect defence, it
only defends against people who don't care about the feature. People
who find it useful for their workflow and want to use fossil for
compatibility with upstream of for other features it provides can do
so.

If rebase moves the patches, then the two operations are different. This is basically a merge - except you're doing it outside the SCM, so you get two copies of the patches. But whether you do it inside or outside the SCM, you wind up with a history of the patches having been applied twice. Rebase changes that.

    <mike

Erlis Vidal

unread,
Oct 5, 2011, 2:51:23 PM10/5/11
to fossil...@lists.fossil-scm.org
I get the two points of view and I'm not saying one is right or wrong. Modifying the history versus keeping everything as indeed happens (the history after all)

Yesterday I was confused because I though the shun was done in the big file, but indeed the shun was done in the commit also... will that modify the history? I got the feeling that shunning a commit will change the history... not sure about it, you tell me.

If I'm working under the premisses that the history cannot be changed, is shunning a commit (in case it change the history) a valid operation? Maybe fossil shouldn't allow to shun a  commit, just individual files, if you really want to shun all files in that commit, then fossil could allow that, (shun --all) but that shouldn't touch ever the commit artifact..

Regards,
Erlis

Gé Weijers

unread,
Oct 5, 2011, 8:48:09 PM10/5/11
to fossil...@lists.fossil-scm.org

On Wed, 5 Oct 2011, Michal Suchanek wrote:

> And when you find an issue with a commit that is some way back in your
> personal branch it is more logical and easier to review your branch if
> you append the fix to the commit where it belongs logically or if you
> append it at the top of the history interspersed with some possibly
> unrelated changes?

One of the downsides of rebasing is that the following workflow does
present problems:

- develop & commit
- sync with upstream, rebase/commit
- test
- sync with upstream, rebase/commit and push

The version you tested now no longer exists in the repository. This may
not be a big issue in some environments, but it may be a showstopper
elsewhere (where I work it is).

If you have to use a Git repository in such an environment you end up
using hooks to log all updates, and block all forced updates (updates that
edit history). Otherwise one clueless developer can do serious damage.

Ge'

Michal Suchanek

unread,
Oct 6, 2011, 4:17:53 AM10/6/11
to fossil...@lists.fossil-scm.org
On 6 October 2011 02:48, Gé Weijers <g...@weijers.org> wrote:
>
>
> On Wed, 5 Oct 2011, Michal Suchanek wrote:
>
>> And when you find an issue with a commit that is some way back in your
>> personal branch it is more logical and easier to review your branch if
>> you append the fix to the commit where it belongs logically or if you
>> append it at the top of the history interspersed with some possibly
>> unrelated changes?
>
> One of the downsides of rebasing is that the following workflow does present
> problems:
>
> - develop & commit
> - sync with upstream, rebase/commit
> - test
> - sync with upstream, rebase/commit and push
>
> The version you tested now no longer exists in the repository. This may not
> be a big issue in some environments, but it may be a showstopper elsewhere
> (where I work it is).
>
> If you have to use a Git repository in such an environment you end up using
> hooks to log all updates, and block all forced updates (updates that edit
> history). Otherwise one clueless developer can do serious damage.

Most sane git workflows require that sync with upstream does not
require forced updates. Some project have experimental branch that may
but all else should update cleanly. You can always make a copy of your
branch before you rebase so that you do have a copy.

Thanks

Michal

Reply all
Reply to author
Forward
0 new messages