Proposal: Migrate TiddlyWiki core dev to GitHub

[chris...@gmail.com]

As many are no doubt aware, open source projects have, in the last few
years, been moving in droves from centralized code repositories, such
as subversion, to distributed systems such as git, mercurial and
others.

TiddlyWeb and TiddlySpace code has been kept on http://github.com/
This has proven to be a success. The increased visibility,
accessibility and sociability has led to a broad spectrum of
interested and contributing parties.

Git's more flexible and easier branch management has also made it
possible to experiment with alternate functionality with less fear.

Therefore, barring substantial objections from the community, we'd
like to move the management of the core TiddlyWiki code from
http://svn.tiddlywiki.org/ to a to be determined location on GitHub.

To make that happen we'll need to resolve some details here. I've
started a list of questions below, please add your own as well as your
response, as you think of them. Where possible I provide a proposal as
a starting point.

The proposal is based on the idea that the web works. All stuff
doesn't need to be in the same place, because if things have URIs then
they can be anywhere on the web.

There's no timeline for this yet. It depends on how much or little
feedback there is. The feedback will ensure that this process a)
happens, b) happens correctly.

! The Questions

1. The TiddlyWiki svn repo is full of all kinds of stuff. What portion
is to be moved? What happens to those parts that are not moved? Of
those pieces that move, will they all go to the same place? Will
history be moved?

Proposal:

The existing svn repo will be frozen in a read only state. Individuals
with code there are welcome to migrate their own stuff wherever they
like. TiddlyWiki core components will move to one user on github, with
potentially several repositories, but at last one which is what would
be considered _the_ core. History for those components will not be
migrated.

The top of the svn repo will be annotated with information such that
visitors will be able to determine that a migration has happened.

The large number of contributors in
http://svn.tiddlywiki.org/Trunk/contributors/ will be responsible for
migrating (if they desire) their own stuff. Given that it is now easy
to find free hosting for code in multiple places on the internet this
is not considered onerous. If you are one of those contributors and
you are concerned, please speak out.

2. What about trac and tickets?

Proposal:

trac and the tickets therein will also be frozen in a read only state.
tickets will _not_ be migrated. The goal here is to get rid of stale
tickets and the most straightforward way to do that is to let
important tickets be recreated, in the github issues system[1].

3. What about www.tiddlywiki.org (the wiki)?

This will be handled in a separate message to the TiddlyWiki group.
The gist is to drop mediawiki in favor of using TiddlySpace to host
the content in a TiddlyWiki.

[1] This is still very open for debate as there's some concern that
github issues system is not up to the task. This has not been my
experience, in fact I rather like it. Actually, just to be clear,
everything is very open for debate.

--
Chris Dent http://burningchrome.com/
[...]

[FND]

> we'd like to move the management of the core TiddlyWiki code from
> http://svn.tiddlywiki.org/ to a to be determined location on GitHub

That'd be great. In fact, it almost seems like a necessary (if not
sufficient) step in order to bring about some reinvigoration.

FWIW, I registered https://github.com/TiddlyWiki a few years ago (I
guess I was hoping for this move to occur sooner... ) - and would of
course be happy to hand over the credentials.

> The proposal is based on the idea that the web works. All stuff
> doesn't need to be in the same place, because if things have URIs
> then they can be anywhere on the web.

+1

> What portion is to be moved?

Off the top of my head, I can think of these basic/official components:
* TiddlyWiki core (Trunk/core/, along with tests and such)
* Cook (Trunk/tools/cooker/)
* jQuery plugins (Trunk/core/jquery/plugins/)

Each of those could, and probably should, be a separate repository.
Ideally, their respective directory structures should be
reviewed/reorganized.

> What happens to those parts that are not moved?

Just leave them in SVN, from where they can be migrated individually as
necessary (e.g. contributors' and some of the association stuff would
probably be migrated eventually by their respective maintainers).

> Of those pieces that move, will they all go to the same place?

I don't think so - see above.

> Will history be moved?

For almost every component, history has proven very important over the
years (<obligatory note about commit messages>) - so we should try our
best to retain that. I'm pretty sure git-svn supports partial cloning,
so it shouldn't be much of a problem.

> If you are one of those contributors and you are concerned, please
> speak out.

I'm not actually concerned, but it's worth mentioning that most
verticals' recipes will probably stop working - unless, perhaps, the
existing TiddlyWiki core recipe(s) remain in place, essentially just
redirecting to the new URIs.

> trac and the tickets therein will also be frozen in a read only
> state. tickets will _not_ be migrated. The goal here is to get rid of
> stale tickets and the most straightforward way to do that is to let
> important tickets be recreated

I approve.

> in the github issues system

> [...]

> This is still very open for debate as there's some concern that
> github issues system is not up to the task. This has not been my
> experience, in fact I rather like it.

I'm not a huge fan of GitHub Issues - but then, all issue trackers suck.
From our TiddlySpace experience, GHI is good enough, probably even
better than Trac (if only because it's less bloaty, though it also has a
nice API). So sure, I approve.
(Note that a GitHub account is required to raise tickets there - but
that's not necessarily a bad thing; there's always the mailing lists.)

-- F.

[Eric Shulman]

> History for those components will not be migrated.

Is there some reason why you would discard this information? Is it
not correct? Even if it's very old, the modification history can be
*vital* to determining what changed and why.. and, VERY old versions
of TiddlyWiki are still relevant. (I just responded to a question
about someone using TW1.2.12, from 2006). Please don't throw away the
past in an attempt to move towards a better future.

> trac and the tickets therein will also be frozen in a read only state.
> tickets will _not_ be migrated. The goal here is to get rid of stale
> tickets and the most straightforward way to do that is to let
> important tickets be recreated, in the github issues system[1].

I *strongly* object to this. There are HUNDREDS of open tickets that
have simply been deferred over and over and over again, due to "short
term priorities" at Osmosoft. These tickets may be old, but they are
NOT irrelevant. In many cases, there are extensive discussions and
detailed information in these tickets, not only about the outstanding
problems, but also, in many cases, solutions that can be readily
implemented *if* there is the will to do so!

Forcing the re-entry of all of this information is NOT the way to
review the issues. Certainly, some of these open tickets are not
important any more, but that cannot be said for MOST of them. Perhaps
there is a way to migrate the tickets but declare them as 'migrated'
and subject to review before being marked as 'active'.

-e

[chris...@gmail.com]

Before I respond below, I'd just like to repeat that what I suggested
is a proposal for the sake of discussion. Some of the more "radical"
parts designed to draw out some of the real issues. My response below
should be taken as presenting my position and feelings on the issues
in the discussion.

On Thu, 20 Jan 2011, Eric Shulman wrote:

>> History for those components will not be migrated.
>
> Is there some reason why you would discard this information? Is it
> not correct? Even if it's very old, the modification history can be
> *vital* to determining what changed and why.. and, VERY old versions
> of TiddlyWiki are still relevant. (I just responded to a question
> about someone using TW1.2.12, from 2006). Please don't throw away the
> past in an attempt to move towards a better future.

I didn't say "discard". The svn repo would remain in place, with full
history. Reviewing that history would require an extra step, but still
be possible.

In my experience deep knowledge of why certain structures are in place
is a good way of making sure they stay in place. I think it can be
argued that such thinking hampers TiddlyWiki across the board.

> I *strongly* object to this. There are HUNDREDS of open tickets that
> have simply been deferred over and over and over again, due to "short
> term priorities" at Osmosoft. These tickets may be old, but they are
> NOT irrelevant. In many cases, there are extensive discussions and
> detailed information in these tickets, not only about the outstanding
> problems, but also, in many cases, solutions that can be readily
> implemented *if* there is the will to do so!

A driving force behind moving to github is to remove both the
perception and reality of any Osmosoft priority over priorities. If
you keep your own fork of TiddlyWiki on github, and manage it in a
shareable way, then it becomes easy for your changes and fixes to be
merged into an official core, or even for your version to be become
preferred.

In any case, again, the tickets would not be destroyed, just left in
their current state. That is to say: idle.

> Forcing the re-entry of all of this information is NOT the way to
> review the issues. Certainly, some of these open tickets are not
> important any more, but that cannot be said for MOST of them. Perhaps
> there is a way to migrate the tickets but declare them as 'migrated'
> and subject to review before being marked as 'active'.

Or they can just be left there and people who have the will to do them
will migrate _only_ the ones that are actually important.

I'm not saying this to be cantankerous nor contrary. I'm just as
annoyed as anyone else by the seeming policy of deferment in TiddlyWiki
development. I've agitated for this github stuff exactly to shake things
up enough so that the morbidity in the development of the TiddlyWiki
core can be blown out the doors, washed down the streets and become an
interesting memory.

If we get stuck deep in process discussion about how to save tickets,
how to migrate tickets, or how to pick and choose then things will
stay stuck. Stuff that matters will move to a new system because it
matters. Stuff that doesn't move either doesn't matter, or the
problems (and solutions) will rise up again because they do.

There are 350 tickets in trac.tiddlywiki.org described by one of the
reports as "active". The oldest is dated May of 2006. It can't be that
active if it is 5 years old and still exists. Amusingly this
particular bug (http://trac.tiddlywiki.org/ticket/17) is one that
keeps coming back up but somehow just keeps getting its milestone
changed, and that's it. What does that say about the development
process and about the ticket handling process? Nothing good. Further,
the bug in question is one of those bugs that's pretty obvious and
we'll get it again if for some reason all TiddlyWiki tickets are
destroyed. The gist is: Search in TiddlyWikis sucks, especially when
there are lots of results.

We'll continue to know that if ticket 17 dropped off the face of the
earth.

I'm sure there are plenty of counter examples of tickets with full and
luscious histories that include multiple proposed solutions so this
doesn't need to be an invitation to show that #17 is anomalous. That's
not the point. The point is that ticket handling and management, the
social process surrounding tickets, is _broken_.

I think it will be easier to fix that social process from something
closer to scratch.

Thanks for providing me the opportunity to rant. Sorry for being
strident.

[Eric Shulman]

> The point is that ticket handling and management, the
> social process surrounding tickets, is _broken_.
>
> I think it will be easier to fix that social process from something
> closer to scratch.
>
> Thanks for providing me the opportunity to rant. Sorry for being
> strident.

In general, I concur with your thinking on this. The process *does*
need to be 'shaken up'. My concern is not with the process, but with
the potential loss of important accumulated knowledge that may result.

For example, it seems all to easy for ticket details to be mis-
interpreted during manual migration and, once the new ticket exists,
how diligent will people be to continue checking the old ticket system
for relevant information?

I suspect that most people, after perhaps a brief period of "dual-
system use" will simply ignore the old ticket system and forget to
check it. Given that people *today* aren't really using the Trac
system propery (or at least not effectively), it seems likely that
some tickets (especially older ones) will simply be forgotten or even
outright ignored if they are only stored in a frozen, obsolete Trac
ticketing system.

Again, my suggestion is to automatically migrate the old tickets, but
classify them in some manner as 'migration', without actually making
them *active* tickets. When people search on the new system, it could
still report tickets with previously noted relevant content,
potentially including details of proposed, but *unimplemented*
solutions to the problem. Then, they can create a fresh ticket with
content either pasted from the older ticket, or by including a link in
the new ticket content that points back at the old ticket.

your thoughts?

thanks,
-e

[chris...@gmail.com]

On Thu, 20 Jan 2011, Eric Shulman wrote:

> Again, my suggestion is to automatically migrate the old tickets, but
> classify them in some manner as 'migration', without actually making
> them *active* tickets. When people search on the new system, it could
> still report tickets with previously noted relevant content,
> potentially including details of proposed, but *unimplemented*
> solutions to the problem. Then, they can create a fresh ticket with
> content either pasted from the older ticket, or by including a link in
> the new ticket content that points back at the old ticket.

Doing an automatic migration is possible, it seems there are tools
out there that will do it. Here's an example:

https://github.com/adamcik/github-trac-ticket-import

(Thanks to imexil for pointing that out)

I'm not opposed to _someone_ doing that. I just don't want it to be
me :)

[Paul Downey]

> I didn't say "discard". The svn repo would remain in place, with full
> history. Reviewing that history would require an extra step, but still
> be possible.

I can live with freezing trac.tiddlywiki.org and svn.tiddlywiki.org
and starting with an empty list in github with only actionable tickets
open which get fixed quickly. Chris has demonstrated this approach
works well with TiddlySpace.

> A driving force behind moving to github is to remove both the
> perception and reality of any Osmosoft priority over priorities. If
> you keep your own fork of TiddlyWiki on github, and manage it in a
> shareable way, then it becomes easy for your changes and fixes to be
> merged into an official core, or even for your version to be become
> preferred.

Perception is the key word here. I can fork TiddlyWiki as of now, but
it wouldn't be The TiddlyWiki that everyone else uses; the one
anointed by Jeremy Ruston.

What has hampered TiddlyWiki development is a need to remain backwards
compatible with a myriad of adaptors, plugins and tweaks which hijack,
eval and monkey-patch the core in unpredictable ways.

I think most developers quickly find git preferable to svn, if only
because of being able to work offline, and stage changes, and provide
and manage patches in an almost entertaining way via github.

Backwards compatibility and consensus are hard issues to tackle and
are orthogonal to github (v) svn/trac, though new tools with a clean
slate can only help progress.

--
Paul (psd)
http://blog.whatfettle.com

[Ton van Rooijen]

My humble 2 cents:
I am not a TiddlyWiki developer nor am I familiar with Git.
My only contributions to TW so far are the Dutch and Spanish
translations, or localizations if you whish. The language-files as
well as ready-to-use translated empty TWs are stored in SVN too, up
until now (also for other languages of course). They are in:
http://svn.tiddlywiki.org/Trunk/association/locales/core/...
(see also:
http://trac.tiddlywiki.org/wiki/Translations )
Since I don't know Git I cannot compare it with SVN, and as such I
don't have an opinion on which one is better for TW.
The only thing I would like to ask is to not forget the translators,
and the consequences for the way they can deliver their contributions.
Thanks.

Ton van Rooijen.

[Jeremy Ruston]

> .

> The only thing I would like to ask is to not forget the translators,
> and the consequences for the way they can deliver their contributions.
> Thanks.

I'm afraid that the current arrangements for translators are far from perfect:

- translators have to deal with JavaScript's string quoting and
encoding rules, which are not particularly user friendly
- translators have to use subversion

To deal with the second issue, I think it makes sense to keep
translations at the upcoming, TiddlySpace-powered tiddlywiki.org,
removing the need for translators to learn git or subversion (there
are already quite a few translations at tiddlyspace.com). Dealing with
the JavaScript issue could be done by moving to a tiddler slice and
section syntax, admittedly a fairly significant core change.

Cheers

Jeremy

> --
> You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
> To post to this group, send email to tiddly...@googlegroups.com.
> To unsubscribe from this group, send email to tiddlywikide...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/tiddlywikidev?hl=en.
>

[Måns]

Hi Jeremy

> Dealing with the JavaScript issue could be done by moving to a tiddler slice and
> section syntax, admittedly a fairly significant core change.

Could we have a Space for translation (@translator space), which works
sth like privateer, and is only meant for temporary inclusion...
You include the @translator space and open a tiddler i.e [[localize]]
and fill in a form which consists of a table with English terms and a
corresponding inputfield.
When you are finished - you click "submit" and a new tiddler is
created with all slices and sections (working in the background of the
form) interpreted and put into the needed JavaScript quotes of a
standard .js systemConfig tiddler.
You exclude the @translator space and have the translation in your
space. There should be some fallback-procedure, if you decide not to
make a full translation, - then untouched fields should use the
English terms suggested by the form... - no core change needed...

Doable?

Cheers Måns Mårtensson

[Måns]

--- addendum

> Dealing with the JavaScript issue could be done by moving to a tiddler slice and
> section syntax, admittedly a fairly significant core change.

Could we have a Space for translation (@translator space), which works
sth like privateer, and is only meant for temporary inclusion...
You include the @translator space and open a tiddler i.e [[localize]]
and fill in a form which consists of a table with English terms and a
corresponding inputfield.

I believe translations for Spaces actually would need a second template which deals with everything specific for TiddlySpace i.e TiddlySpace backstage items and TiddlySpace theme (css wrapped items) like sideBarTabs etc... Or have I overlooked/missed an alternative translation template made especially for TiddlySpace??

 Cheers Måns Mårtensson

 

[Jeremy Ruston]

Yes, I think an arrangement like that could work. The key first step
would be to construct a mechanism for processing translations in
slice/section format and automatically loading them up into the
variables defined by Lingo.js.

I'm not recommending that we do this right now, of course - migrating
tiddlywiki.org and subversion is the first step,

Best wishes

Jeremy

> --
> You received this message because you are subscribed to the Google Groups
> "TiddlyWikiDev" group.
> To post to this group, send email to tiddly...@googlegroups.com.
> To unsubscribe from this group, send email to
> tiddlywikide...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/tiddlywikidev?hl=en.
>

--
Jeremy Ruston
mailto:jer...@osmosoft.com
http://www.tiddlywiki.com

[Måns]

Hi again

Yes, I think an arrangement like that could work. The key first step
would be to construct a mechanism for processing translations in
slice/section format and automatically loading them up into the
variables defined by Lingo.js.

 

Great :-) I'm looking forward to something like that! 

I'm not recommending that we do this right now, of course - migrating
tiddlywiki.org and subversion is the first step,

Understandable, I will be very patient...

Cheers Måns Mårtensson

[PMario]

Hi,
Having all the stuff at github would be nice.

According to translations, it would be nice, if a translation wouldn't
add file size to the TW.
I'd like to have something like to include a "system-de" instead of
"system-en"

-m

[Jeremy Ruston]

> According to translations, it would be nice, if a translation wouldn't
> add file size to the TW.
> I'd like to have something like to include a "system-de" instead of
> "system-en"

That's it. It might be separate from the system bag, for instance "lingo-en".

Cheers

Jeremy

[rakugo]

I think this is a great move.
I personally think it would be best to start afresh in a TiddlyWiki
version of git - ie. not migrate anything.

Some of the issues from 5 years ago to me are not issues as they have
been alive for 5 years without being solved - they haven't received
the conversation they require. If anything I think starting from
scratch would lead to rejuvenating the really important issues. For
instance the search is something I care about but I haven't been
following that ticket due to the fact it is "lost" amongst the trac
graveyard. It would be a good opportunity for the community to see
what people perceive as the most important issues and fix them.

As long as http://svn.tiddlywiki.org and http://trac.tiddlywiki.org
(and the internet! :)) remains alive read only, the important ones
will not be forgotten and available including patches that have been
suggested previously. I can imagine us creating issues on git to the
important ones and referencing urls on trac.

Jon

[Martin Budden]

I'm also in favour of a move to github. To reply to some of Chris's proposals:

1) The TiddlyWiki svn repo is full of all kinds of stuff. What portion
is to be moved...

In addition to the components mentioned by FND (TiddlyWiki core
(Trunk/core/, along with tests and such), Cook (Trunk/tools/cooker/)
and jQuery plugins (Trunk/core/jquery/plugins/)) I think the following
need to be migrated:

From association:
a) locales (translations).
I think it is important to have the translations in a repository so
that people can build a non-English version of TiddlyWiki without
manual intervention. The translators themselves do not need to learn
github - they can use a tiddlyspace based system as suggested by
Jeremy. What is required is that someone moves these translations into
a repository so that they can be used by build scripts. This is
similar to the current TiddlyWiki situation - most translators don't
use subversion directly - the translation is normally in a TiddlyWiki
which I then upload to subversion.
I'd like to see the locale directory be a subdirectory off the core directory.

b) Adaptors. Some are in the association, some are scattered in
contributors directories - these would perhaps benefit from being in a
common github repository.

c) Possibly also the themes directory.

From verticals:
a) the tiddlywiki.com vertical. Perhaps also the beta vertical
(although git probably removes the need for this, since it could just
be a branch).

I favour several smaller repositories over one large repository and
agree with Chris that there should be one repository that is "the
core".

I agree with the proposal to freeze the subversion repository and only
move over the tip revision. The subversion repository and the trac
source code browser should be left available so that the history is
available for bug fixing and other purposes.

2. What about trac and tickets? -trac and the tickets therein will
also be frozen in a read only state. tickets will _not_ be migrated...

I'm only in partial agreement. I don't think that tickets should be
frozen (I agree that they should not be mandated as a matter of
course). I'd rather see some parallel running of the old and new
ticketing systems. So, in actioning a ticket, a developer could either
move it to the new issues system, and then make the fix, or just make
the fix and then mark the ticket as fixed in Trac.

Martin

[chris...@gmail.com]

On Mon, 24 Jan 2011, Martin Budden wrote:

> b) Adaptors. Some are in the association, some are scattered in
> contributors directories - these would perhaps benefit from being in a
> common github repository.

No. I've found that aggregating by type just creates a mess and
works against robust testing, packing, documentation for the
individual thing and limits discoverability and linkability.

I started out on this path when I first made the tiddlyweb-plugins
repository. I now regret that and as time allows move the individual
plugins that are in there into their own repos.

Compartmentalizing things at a more granular level also helps to
encourage decoupling and reuse.

[tiddlygrp]

Hi,

I would like to see tiddlywiki moved to github. However I have two
(well three) major concers:

-- What happens to the contributor code? Chris indicated that
contributors can move their code as they wish. For me that means some
will move, some will stay, and some may move elsewhere.
I think this contributes to the fragmentation of tiddlywiki even
more. I often have difficulty locating plugins, but if also the
contributors archive from trac/svn diffuse, how am I able to find
components back? If on github there are a lot of tiddlywiki
repositories, how do I know about them? And how do I know which
dependencies I need to build a specific vertical? I am really not
keen on fragmentation of the repository!

-- Lost history. I think just moving to github without history is a
tremendous waste of knowledge and effort. I can see two relatively
easy solutions: 1) Import history with svn2git and just continue with
development (my preference). 2) Ditch history, keep trac/svn online
AND start on github with a new MAJOR version of tiddlywiki. Then it
is very clear for humans that a break in the source happened.

-- trac tickets etc. Just loosing them would be a waste of work.
Unfortunately I don't know about a good, distributed ticket tracker
for git. I think minimally the trac infomation needs to be mirrored
read-only into a git repository. then it can be searched, and old
tickets can be referenced from a new ticket system. Just keeping the
old trac system running is inferior I think because once it will stop
and the tickets are lost. An option may be http://fossil-scm.org/ .
This has a distributed ticket tracker, and could even replace git.

yours,
Vlak

[chris...@gmail.com]

On Thu, 27 Jan 2011, tiddlygrp wrote:

> -- What happens to the contributor code? Chris indicated that
> contributors can move their code as they wish. For me that means some
> will move, some will stay, and some may move elsewhere.
> I think this contributes to the fragmentation of tiddlywiki even
> more.

That's interesting. I agree that finding stuff is and has been a big
concern but I've felt that the contributors setup on
svn.tiddlywiki.org has actually worked against discoverability of
TiddlyWiki plugins. I think this is due in part to the special
nature of those plugins: they are code that can run on the web but only
in a TiddlyWiki. Therefore it makes sense that the point of
distribution for such things ought to be a TiddlyWiki, on the web.
Mind, the point of distribution and point of code repository does
not have to be the same thing.

So, my thinking is that where someone chooses to store their plugin
code isn't that relevant to discoverability.

Also, the fact is that the current setup, wherein all the
contributors stuff is in the same repo mean that the repo (in part
because it is svn, which is slow in modern terms) is extremely time
consuming to update if you have the whole thing checked out. And
then once you have thing there's so much there that has little to
nothing to do with whatever your current purpose might be.

You make a valid point about not knowing which dependencies you need
to build a vertical, but I think that's already true, and already
fixable with cook recipes. Those recipes can use content from all
over the web, not just local disk. Effectively used, that can make
the repositories associated with a repo tight and focused, with
minimal duplication.

> -- Lost history. I think just moving to github without history is a
> tremendous waste of knowledge and effort. I can see two relatively
> easy solutions: 1) Import history with svn2git and just continue with
> development (my preference). 2) Ditch history, keep trac/svn online
> AND start on github with a new MAJOR version of tiddlywiki. Then it
> is very clear for humans that a break in the source happened.

I think maintaining the history of the core will be fine. I'm less
motivated about ditching code history than I am ditching stale
tickets.

[tiddlygrp]

Hi Chris,

thanks for your thoughts. I have a few remarks:

On Jan 27, 5:06 pm, chris.d...@gmail.com wrote:

> Also, the fact is that the current setup, wherein all the
> contributors stuff is in the same repo mean that the repo (in part
> because it is svn, which is slow in modern terms) is extremely time
> consuming to update if you have the whole thing checked out. And
> then once you have thing there's so much there that has little to
> nothing to do with whatever your current purpose might be.

I think with git it could be different. But also svn allows partial
checkouts!

>
> You make a valid point about not knowing which dependencies you need
> to build a vertical, but I think that's already true, and already
> fixable with cook recipes. Those recipes can use content from all
> over the web, not just local disk. Effectively used, that can make
> the repositories associated with a repo tight and focused, with
> minimal duplication.

This is in principle true. In practice this implies that versioned
repositories of the plugins exist, which can be accessed by cook.
Just pointing to a tw with the plugin is not sufficient, as you may
need a specific version. I find cook not so strong with regard to
versioning as it does not parse (i.e. check versions) of things it
puts together. A possibility is that someone (who???) mirrors on
github the repositories of the plugin makers such that plugins and
contributor code is available close to the core. However that
requires plugin writers to make their own repo, which may be too much.

> > -- Lost history.  I think just moving to github without history is a
> > tremendous waste of knowledge and effort.  I can see two relatively
> > easy solutions:  1) Import history with svn2git and just continue with
> > development (my preference).  2) Ditch history, keep trac/svn online
> > AND start on github with a new MAJOR version of tiddlywiki.  Then it
> > is very clear for humans that a break in the source happened.
>
> I think maintaining the history of the core will be fine. I'm less
> motivated about ditching code history than I am ditching stale
> tickets.
>

Maybe, even if you ditch stale tickets, etc, the whole trac wiki can
be imported into git read only in a subdirectory called old (or
something like this). Then it is searchable in github. I think the
important thing is easy access to old knowledge, not necessarily
keeping the old ticket system running.

[chris...@gmail.com]

On Thu, 27 Jan 2011, tiddlygrp wrote:

> I think with git it could be different. But also svn allows partial
> checkouts!

Yes, of course, but because the system is organized and operated in a
way that says, "it's all here" you thus have to operate as if it is
all there, so doing a partial checkout is often not particularly useful.

It will be better when the expectation is "it's all spread around"
because then people will create their tools for that mode of work,
which in the long run is better: more flexible for code owners and
code users.

[recipe and content from all over the web]

> This is in principle true. In practice this implies that versioned
> repositories of the plugins exist, which can be accessed by cook.
> Just pointing to a tw with the plugin is not sufficient, as you may
> need a specific version.

Why, in any common case, would you want a specific version which isn't
otherwise represented by a tag or branch (both of which are more
visible and accessible (over the web) in git than in svn)?

Being dependent on specific version ought to be a very rare case[1] in
a suitably healthy ecosystem.

> However that
> requires plugin writers to make their own repo, which may be too much.

I reckon it is far easier for a plugin writer to make a repo on github
than it is for them to sign up for, get and then use the svn.tiddlywiki
repo?

> Maybe, even if you ditch stale tickets, etc, the whole trac wiki can
> be imported into git read only in a subdirectory called old (or
> something like this). Then it is searchable in github. I think the
> important thing is easy access to old knowledge, not necessarily
> keeping the old ticket system running.

I'm hoping that the people who have expressed a big commitment to the
older tickets will devise a strategy for managing them that they are
all happy with. Whether that means ditching them, doing what you are
suggesting, or doing some kind of import (as Eric seems to want?) I'm
happy to help orchestrate.

--don't read past here if you'd like to stay on topic--

[1] That said, I think TiddlyWiki would gain a lot by allowing plugins
to declare a specific version of tiddlywiki beyond which they don't
work. I think this will make it possible for TiddlyWiki to move
forward without the crippling onus of being backwards compatible with
all plugins out there. Because of the unique style of plugging that
TiddlyWiki plugis use (monkey patching, hooking, overriding, etc) it
is very challenging to refactor the APIs (which do not reflect modern
javascript style preferences).

[Martin Budden]

Following on from your comments:

"What happens to the contributor code?" - there are very few
contributors who keep their code in the contributors directory in
subversion. A few years back we tried actively to get contributors to
use subversion, but there was little appetite for it. Contributors can
already move their code as they wish, so I don't personally think that
we will, in practice, get any more fragmentation of plugins.

"Lost history". There are disadvantages and advantages of importing
the history into git. History is mainly of interest to core developers
of TiddlyWiki, and they will be slightly inconvenienced by having to
look in subversion for history. But a clean break means a smaller and
more easily usable repository for those not interested in the history.
So a question - how often do you look at the history?

"Tickets". I agree that this is a difficult issue, and I don't think
there is an entirely satisfactory solution.

Martin

[rakugo]

> "Lost history". There are disadvantages and advantages of importing
> the history into git. History is mainly of interest to core developers
> of TiddlyWiki, and they will be slightly inconvenienced by having to
> look in subversion for history. But a clean break means a smaller and
> more easily usable repository for those not interested in the history.
> So a question - how often do you look at the history?

Clean break please. I hardly ever look at the history. If we keep svn
up as read only, we still allow access to history if it is important.
I don't really see the gain in moving it into git.

> "Tickets". I agree that this is a difficult issue, and I don't think
> there is an entirely satisfactory solution.

Personally again, I would prefer that any important issues are
manually moved over... even if that is just referencing a ticket on
trac. I think this would be valuable to the TiddlyWiki project as a
way of determining which issues are really really important and
actionable.

[Eric Shulman]

> > "Tickets". I agree that this is a difficult issue, and I don't think
> > there is an entirely satisfactory solution.
>
> Personally again, I would prefer that any important issues are
> manually moved over... even if that is just referencing a ticket on
> trac. I think this would be valuable to the TiddlyWiki project as a
> way of determining which issues are really really important and
> actionable.

There are tickets in the Trac system that have been repeatedly re-
scheduled to "sometime in a future release", usually as a result of
some short-term priorities at Osmosoft. This doesn't mean they aren't
important or 'actionable', just that there were other items that were
deemed, at the time, to be more immediately in need of attention.

I *do* agree that new tickets should be created for all actionable
issues. However, I am greatly concerned that creating a new ticket
that merely *references* a ticket on Trac doesn't ensure that people
actually examine the information in the old ticket and it also assumes
that *only* the referenced tickets are relevant to the current issue.

My concern is that, given how poorly people *currently* use the
information in Trac, I think they are unlikely to suddenly become more
rigorous in checking that information when it is not integrated into
the new ticket system. There is *always* an "out of sight, out of
mind" effect when information is split across systems, especially if
one of those systems is 'mothballed' for reference-use only and is not
integrated into a single search process that examines ALL ticket
information (both old and new) for related issues.

I also get a sense that there is a desire by some people to simply
ignore and *discard* old tickets and move on without actually
resolving the issues addressed by those tickets, unless they are
*personally* interesting. It's simply too easy for someone to leave
out the references to old Trac tickets and pretend that those problems
never existed.

As I previously posted, I feel *very* strongly that the existing
tickets should be migrated into the new system (as *read-only*
information) so that the old ticket information is easily searchable
without having to search on two separate systems. Please note, I am
*not* suggesting that these tickets be made *active* in the new
system. What I am advocating is that ALL ticket information be
retained and migrated, so that there is no chance that it is simply
"lost" or ignored as a result of the transition.

-e

[Martin Budden]

"I also get a sense that there is a desire by some people to simply
ignore and *discard* old tickets and move on without actually
resolving the issues addressed by those tickets, unless they are
*personally* interesting. It's simply too easy for someone to leave
out the references to old Trac tickets and pretend that those problems
never existed."

As one of the primary users of the ticket system, I'll address this comment.

There are certainly tickets that I am reluctant to address, but this
reluctance has nothing to do with whether I find the tickets
"interesting". My reluctance is normally caused by one of four
factors:

i) The view that fixing the problem may break backwards compatibility
and stop existing plugins and macros from working. This is especially
true if the problem is relatively minor and there are established
work-arounds. A good example is Ticket #472, "Invalid tiddler IDs (due
to spaces)" - I've avoided fixing this ticket because I know it will
break quite a few of Eric's plugins. (In the case of this Ticket the
issue is being forced by jQuery - TiddlyWiki will not work with the
next version of jQuery unless this is fixed.)

I take a fairly conservative view here, and I know that there are
those who feel that we should be less concerned about backwards
compatibility.

ii) The view that the feature is does not really belong in the core.

Part of this is influenced by my view that "users who don't use a
feature, shouldn't pay for that feature". Every feature added to the
core increases its size and so its load time. Although the effect for
a single feature is small, unless a conservative approach is taken the
core, over time, will become bloated.

A good example of this is the famous Ticket 17, "Improve searching
user interface". The current search is adequate for many users, and
users who want better search can use one of the many plugins.

iii) The view that the problem is a theoretical one, rather than a
practical one. There are a class of bugs that are what might be called
"theoretical" - that is in principle they might cause problems, but in
practice they do not, because the circumstances in which the bugs
manifest themselves are extremely rare or even non-existant.

iv) The view that the problem is difficult to fix and that the
benefits are small or even unnoticeable. A good example is Ticket 34,
"TiddlyWiki should generate proper <P> paragraph tags". This is a
difficult (and interesting) problem, but one I just don't feel is
worth fixing. I imagine that the vast majority of TW users don't care
about the underlying HTML format of their tiddlers, they just care if
they look OK. Actually Ticket 34 falls into several categories ( i:
fixing it may break backwards compatibility, in that people's
TiddlyWikis may display differently, ii: it's mainly a theoretical
problem (there is the occasional HTML-purist who complains, but the
average user doesn't notice), and the problem is difficult to fix).

(And for the record I am one of those HTML-purists: it pains me that
TW does not do proper paragraph formatting).

One of the problems is my own reluctance to close down tickets
completely. The migration to a new ticketing system is perhaps an
opportunity to do this - if people believe a closed down ticket is
important, then they can re-open it.

Martin

[Paul Downey]

Nicely said, Martin ..

> i) The view that fixing the problem may break backwards compatibility
> and stop existing plugins and macros from working.

This is the biggest issue for TiddlyWiki in general, how to make
changes without disturbing monkey patches, many of which a fixer might
not be aware even exist.

To be fair, I think as a community we've managed this issue pretty
well so far, but this is where the time and effort goes in improving
the core.

> I take a fairly conservative view here, and I know that there are
> those who feel that we should be less concerned about backwards
> compatibility.

I think using TiddlySpace as a platform for experimenting with the
tiddler data model, building new interfaces and handling multiple
representations is exactly the right thing, now.

> ii) The view that the feature is does not really belong in the core.

Agreed. My gut reaction on first seeing TiddlyWiki was it would
benefit from radical surgery, being dissolved into a micro-kernel with
everything being an optional plugin, even wikitext formatting. I
became convinced this wasn't possible because of i) but that would be
my approach if starting from a clean slate.

> iii) The view that the problem is a theoretical one, rather than a
> practical one.

Tests should help this. We don't have a culture of writing tests in
TiddlyWiki, apart from some work for TiddlySpace. This compounds i)
greatly.

> iv) The view that the problem is difficult to fix and that the
> benefits are small or even unnoticeable.

Not having a patch which doesn't impact i) or ii) means the ticket
isn't tractable and will languish forever.

> One of the problems is my own reluctance to close down tickets
> completely. The migration to a new ticketing system is perhaps an
> opportunity to do this - if people believe a closed down ticket is
> important, then they can re-open it.

That's where I'm at: not zero-history, but a manageable task list.

[Michael Mahemoff]

+1 for GitHub from someone who'd like to keep watching and staying involved with the project on a casual basis; the social and notification features of GitHub make it suitable for that. Furthermore, the ability to meaningfully fork and contribute without asking permission are good for any distributed open source project of this nature and especially in line with the philosophy of TiddlyWiki.

On Thu, Jan 20, 2011 at 1:01 PM, <chris...@gmail.com> wrote:

As many are no doubt aware, open source projects have, in the last few
years, been moving in droves from centralized code repositories, such
as subversion, to distributed systems such as git, mercurial and
others.

TiddlyWeb and TiddlySpace code has been kept on http://github.com/ This has proven to be a success. The increased visibility,
accessibility and sociability has led to a broad spectrum of
interested and contributing parties.

Git's more flexible and easier branch management has also made it
possible to experiment with alternate functionality with less fear.

Therefore, barring substantial objections from the community, we'd
like to move the management of the core TiddlyWiki code from
http://svn.tiddlywiki.org/ to a to be determined location on GitHub.

To make that happen we'll need to resolve some details here. I've
started a list of questions below, please add your own as well as your
response, as you think of them. Where possible I provide a proposal as
a starting point.

The proposal is based on the idea that the web works. All stuff
doesn't need to be in the same place, because if things have URIs then
they can be anywhere on the web.

There's no timeline for this yet. It depends on how much or little
feedback there is. The feedback will ensure that this process a)
happens, b) happens correctly.

! The Questions

1. The TiddlyWiki svn repo is full of all kinds of stuff. What portion
is to be moved? What happens to those parts that are not moved? Of
those pieces that move, will they all go to the same place? Will
history be moved?

Proposal:

The existing svn repo will be frozen in a read only state. Individuals
with code there are welcome to migrate their own stuff wherever they
like. TiddlyWiki core components will move to one user on github, with
potentially several repositories, but at last one which is what would
be considered _the_ core. History for those components will not be
migrated.

The top of the svn repo will be annotated with information such that
visitors will be able to determine that a migration has happened.

The large number of contributors in
http://svn.tiddlywiki.org/Trunk/contributors/ will be responsible for
migrating (if they desire) their own stuff. Given that it is now easy
to find free hosting for code in multiple places on the internet this
is not considered onerous. If you are one of those contributors and
you are concerned, please speak out.

2. What about trac and tickets?

Proposal:

trac and the tickets therein will also be frozen in a read only state.

tickets will _not_ be migrated. The goal here is to get rid of stale
tickets and the most straightforward way to do that is to let
important tickets be recreated, in the github issues system[1].

3. What about www.tiddlywiki.org (the wiki)?

This will be handled in a separate message to the TiddlyWiki group.
The gist is to drop mediawiki in favor of using TiddlySpace to host
the content in a TiddlyWiki.

[1] This is still very open for debate as there's some concern that
github issues system is not up to the task. This has not been my
experience, in fact I rather like it. Actually, just to be clear,
everything is very open for debate.

--
Chris Dent                                   http://burningchrome.com/
                               [...]

[chris...@gmail.com]

On Mon, 31 Jan 2011, Eric Shulman wrote:

> There are tickets in the Trac system that have been repeatedly re-
> scheduled to "sometime in a future release", usually as a result of
> some short-term priorities at Osmosoft. This doesn't mean they aren't
> important or 'actionable', just that there were other items that were
> deemed, at the time, to be more immediately in need of attention.

I've seen this phrase "short-term priorities at Osmosoft" bounced
around this thread a few times. The implication is that Osmosoft is
making all the core decisions and core commits.

Since that is considered a problem let me restate one of the reasons
for a move to github:

Make it possible for the at large community surrounding TiddlyWiki
to make patches and other contributions to the core.

Once it happens Eric gets to choose which core code he contributes and
which pull requests he makes.

> My concern is that, given how poorly people *currently* use the
> information in Trac, I think they are unlikely to suddenly become more
> rigorous in checking that information when it is not integrated into
> the new ticket system.

I agree. I don't think saving the data will change that in any way.
Thus my desire for a reboot. There are bad habits and biases ingrained
in the current systems.

> As I previously posted, I feel *very* strongly that the existing
> tickets should be migrated into the new system (as *read-only*
> information) so that the old ticket information is easily searchable
> without having to search on two separate systems.

Have you done any research to see what tools are available to make
this possible?

[FND]

> I've seen this phrase "short-term priorities at Osmosoft" bounced
> around this thread a few times. The implication is that Osmosoft is
> making all the core decisions and core commits.
> Since that is considered a problem let me restate one of the reasons
> for a move to github:
> Make it possible for the at large community surrounding TiddlyWiki
> to make patches and other contributions to the core.

+1

If I care about something and feel strongly that it should be in the
core, why expect the core developers (let alone Osmosoft) to implement
this instead of myself? This perception and lack of diverse _code_
contributions has been a _major_ flaw of this community in the last few
years. (Obviously, we all - myself included - contributed to this. No
pun intended.) Anyway, I'm hopeful that the GitHub move will improve
this situation as it lowers the barrier to contributing and reduces
gatekeepers' significance.

> As I previously posted, I feel *very* strongly that the existing
> tickets should be migrated into the new system

I have equally strong beliefs that this would do (much) more harm than
good, for all the reasons discussed so far. Simply dumping that data in
the ticket system will do nothing to improve triage and resolution. Once
an issue is being tackled, reviewing relevant notes is just part of the
due diligence, no matter the URI.

-- F.

[Paul Downey]

> Anyway, I'm hopeful that the GitHub move will improve
> this situation as it lowers the barrier to contributing and reduces
> gatekeepers' significance.

Well, to be fair, there's nothing to stop anyone forking the core as
of now. All git/GitHub does is introduce a neater process for making a
fork and providing patches. You still need a gatekeeper to accept your
patch before it's a part of The project called "TiddlyWiki".

[Tobias Beer]

Somewhere along this thread I read that the discussion is about some
+-300 active issues. I would suggest that someone (preferably
@Osmosoft) took a few days to browse through them... prioritize the
issues (by a beforehand agreed upon evaluation scheme) - see Martin's
points - and move whatever is deemed important enough to the new
system ...not waiting for anyone else - perhaps new to tw-and-its-
shiny-new-git-thingy - to be searching through abandoned (trac)
archives for some unresolved backlogs.

As for issues like the (in)famous paragraph (and list) structure, I
would certainly think that those should be found when searched via the
new issue tracker. I would also think that the kind of tabula rasa -
as favoured by a felt majority in this thread - eventually means a
whole lot more work than just copying over at least the bigger part of
what simply does not deserve to be in an archived state.

In any case, if things should initially be moved out of trac, they
should definetely be REMOVED from trac and not redundantly stay in a
frozen archived state as well... which is another reason why I would
certainly favour an initial clean up. It's housekeeping and as such
somewhat essential.

Just putting things out of view in a big cellar will NOT help
anyone... at least try and get the new frontdoor as well as the cellar
organized at the beginning ...otherwise the cellar is nothing short
from a landfill site making it hard to find what in fact was not meant
to be dumped into some abandoned place like that.

That's my 5 cts,


Cheers, Tobias.

[rakugo]

On Feb 1, 10:08 pm, Tobias Beer <beertob...@googlemail.com> wrote:
> Somewhere along this thread I read that the discussion is about some
> +-300 active issues. I would suggest that someone (preferably
> @Osmosoft) took a few days to browse through them... prioritize the
> issues (by a beforehand agreed upon evaluation scheme) - see Martin's
> points - and move whatever is deemed important enough to the new
> system ...not waiting for anyone else - perhaps new to tw-and-its-
> shiny-new-git-thingy - to be searching through abandoned (trac)
> archives for some unresolved backlogs.

I agree with this apart from the "Osmosoft" doing this. We are an open
source project and should act like it. The issues list is quite simply
too big. As Martin points out some of those issues have changes that
will break backwards compatibility, these are not actionable and
should be recorded somewhere other than an issues list. A developer
wiki or some other system.

What might be a good idea is to collaboratively as a community review
these trac tickets. We could imagine setting up a TiddlySpace which
has imported all the trac tickets where any registered member (or any
interested TiddlyWiki community member) can review the tickets CREATE/
AMEND but not DELETE. We could imagine using conventions such as
adding a tag "discard", "needswork" or "keep" to each of these
tickets. After this process any tagged discard, we delete, any that
have been tagged needswork are improved, any tagged keep are migrated
to github.

I think a transparent review system like the above will help here.

Thoughts?

[Jeremy Ruston]

The primary practical point under discussion is whether the existing
tickets should be transferred from Trac into GitHub's issue system. I
share Chris's concern that this is a potentially big job to automate,
and I would find it hard to justify undertaking it.

I would favour freezing Trac and Subversion, without making any
attempt at an automated transfer of information, and encourage
individuals to raise new tickets for the issues that are fixable,
given Martin's criteria above.

Cheers

Jeremy

> --
> You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
> To post to this group, send email to tiddly...@googlegroups.com.
> To unsubscribe from this group, send email to tiddlywikide...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/tiddlywikidev?hl=en.
>
>

--

[Tobias Beer]

@rakugo,

I quite like your trains of thoughts here and would sure welcome the
effort... a simple TiddlySpace of that sort might just start out with
a list of all (open?) tickets. People who cared enough would then
somewhat 'grab' them, evaluate them and transfer them if so desired -
keeping Martin's suggestions in mind - while also being able to get a
feel for which issue has already been dealt with (in terms of
migration to git) and which hasn't.

Although (if not "because") this might put the opensource idea quite
to the test, I think your suggestions make a lot of sense. However, I
would still favour that anything which during the process indeed were
to be transferred to git should be removed from the trac
archives ...unless only some part of the ticket that has been deemed
"essential" by the person reviewing it was to be transferred.
Eventually, reduncancies hardly ever are anything but cumbersome when
it comes to tracking issues.

So, if there is a communal effort to sort out remaining tickets...
there should be someone with administrative access to the archives who
will remove anything from trac that got transferred during the
process. Also it might turn out beneficial to point from a trac ticket
to a new git ticket once the latter is being created and - for
reference purposes - vice versa.

Cheers, Tobias.

[Martin Budden]

Taking to heart your suggestion of "cleaning up the cellar" before we
move house, I've created a new milestone in Trac "2011.cleanup". I,
and other members of the Osmosoft team, will start moving tickets we
think shouldn't be fixed to there.

Having a milestone with all these tickets makes the review process
public and easily visible: everyone is encouraged to look at this
milestone and raise any disagreements, that is tickets they believe
should be fixed. Disagreement can be raised either by re-opening the
ticket, or making a comment in this group.

Note that marking a ticket as "won't fix" and moving it to the
2011.cleanup milestone is not a firm decision not to fix the ticket,
rather it is a statement of intent - "This ticket won't get fixed
unless someone objects". So, I repeat, people are encouraged to object
if they disagree with a decision about a ticket.

Hopefully this will get the number of outstanding tickets down to a
reasonable number, so that the migration to the new ticketing system
will be easier.

Martin

[Jeremy Ruston]

The direct URL for seeing the tickets assigned to the milestone is:

http://trac.tiddlywiki.org/milestone/2011.cleanup

Best wishes

Jeremy

--

[chris...@gmail.com]

On Jan 20, 5:34 pm, FND <F...@gmx.net> wrote:
> Off the top of my head, I can think of these basic/official components:
> * TiddlyWiki core (Trunk/core/, along with tests and such)
> * Cook (Trunk/tools/cooker/)
> * jQuery plugins (Trunk/core/jquery/plugins/)

To get things started I went ahead and today created a cooker repo and
imported the existing cooker "stuff" into it.

https://github.com/TiddlyWiki/cooker

I used svn2git[1] with this command line:

svn2git http://svn.tiddlywiki.org/Trunk/tools/cooker --rootistrunk

I tried a few different incarnations of svn2git and git-svn and the
above was the one that worked. I was not inclined to investigate why
the other techniques did not work. Presumably something to do with the
combination of the non-standard repo layout of svn.tiddlywiki.org and
the fact that a subdir was being imported.

If this looks okay, I'll start the process for some of the other
parts.

I added a very basic README to the cooker repo, and made the
test_suite run (not that it asserts anything).

Presumably a next step with cook and ginsu would be to package them in
a proper Ruby way[2].

[1] https://github.com/nirvdrum/svn2git

[2] The reason for doing so is that it encourages encapsulation and
separation of concerns. These tools should live and breathe
independently of other things; if you want to use them, just install
them...

[Martin Budden]

I notice that, by svn2git, you've also imported the history. Cook is
small enough that it does not really matter if you import the history
or not.

However for the main body of TiddlyWiki, I thought we'd decided *not*
to import the history. That's certainly my preference. I'm willing to
be persuaded otherwise, if people feel it is important that we import
the history, but I'd like that to be made as a conscious decision
rather than a side effect of how we migrate the codebase.

Martin

[FND]

> To get things started I went ahead and today created a cooker repo
> and imported the existing cooker "stuff" into it.

Looks good - thanks Chris!

> Presumably a next step with cook and ginsu would be to package them
> in a proper Ruby way

+1

> I thought we'd decided *not* to import the history.

I don't think we did.
As explained before, I believe retaining *code* history is important.

> [discarding history is] certainly my preference. I'm willing to be
> persuaded otherwise

I don't see any significant benefit in discarding code history (in
contrast to discarding stale tickets, which we agreed was useful).

For TiddlyWiki in particular, the source code and its evolution are
often the only clues as to why things are the way they are - so
discarding that would be actively harmful. (Due to things like
git-blame, it's not as simple as looking it up at another URI - again,
unlike with tickets.)

-- F.

[chris...@gmail.com]

On Mon, 7 Feb 2011, Martin Budden wrote:

> However for the main body of TiddlyWiki, I thought we'd decided *not*
> to import the history. That's certainly my preference. I'm willing to
> be persuaded otherwise, if people feel it is important that we import
> the history, but I'd like that to be made as a conscious decision
> rather than a side effect of how we migrate the codebase.

I didn't get that impression from the thread. What I saw was that I
and you were not in favor of including history, while FND and Eric
were, and their cases were strong.

Given that the tools make it easy, and the esteem with which we should
hold FND and Eric's opinions, I figure may as well make FND and
Eric happy.

Especially if it encourages them to be more flexible about tickets :)

[Martin Budden]

As I understood it, Eric did not want the history to be discarded, and
that criterion is satisfied by keeping the frozen svn repository.

One of the reasons I'm in favour of not importing the history into git
is that a clean repository is in some ways attractive to new users.
I'm in particular thinking of a friend of Paul's who was interested in
doing some work on TiddlyWiki, but didn't bother in the end because
the repository took so long to download. Now admittedly this probably
won't be a big problem with a new core repository, but I still think
there is an advantage to making a clean start.

What I am wary of here is policy being influenced by people who are
not affected by the policy. In other words do Fred and Eric want the
history in git because:

a) moving the history to git is "a good thing"
or
b) they actually look at the history quite often, and not moving the
history to git would be an inconvenience

I'm happily swayed by argument (b), but if the argument is (a) then
those who actively use the repository should have more sway.

So before we move the history to git based on Fred and Eric's
opinions, I'd like them to confirm that they actually do look at the
history often enough to be inconvenienced by not moving it.

Martin

[FND]

> One of the reasons I'm in favour of not importing the history into git
> is that a clean repository is in some ways attractive to new users.
> I'm in particular thinking of a friend of Paul's who was interested in
> doing some work on TiddlyWiki, but didn't bother in the end because
> the repository took so long to download.

As you've already stated, that won't be an issue anymore once there's a
separate repository for the core. Indeed, that new repo will be tiny
compared to most serious projects. In other words, My Hair is a Bird.

> What I am wary of here is policy being influenced by people who are
> not affected by the policy.

That sounds like you do not expect other developers to analyze or even
contribute to the core. As stated earlier*, this notion is a massive
problem of and in this community.

> In other words do Fred and Eric want the history in git because:
> a) moving the history to git is "a good thing"
> or
> b) they actually look at the history quite often, and not moving the
> history to git would be an inconvenience

Both, but primarily (b) - so I *do* expect being directly affected, both
as a (potential) core and third-party developer.

Just for the record, IMO even (a) would be sufficient, as the cost of
retaining history is negligible.

-- F.

* http://groups.google.com/group/tiddlywikidev/msg/603776133060a464

[Martin Budden]

"That sounds like you do not expect other developers to analyze or even
contribute to the core"

Actually my expectation is not that people don't look at the core, but
that people don't look at the history.

From a personal point of view I practically never look at source code
history, other than immediate history (that is the bit of history I've
created while I'm working on a particular feature or bug fix). I'm not
talking just about TiddlyWiki, but about every software project I have
ever worked on. This is also true of many/most of the developers I
have worked with.

I personally find that there is no point in looking at history to
understand a bit of code, whether it is to fix a bug or implement a
new feature.

In my view source code history is a bit like credit card slips - you
look at this month's slips when you reconcile your credit card bill,
but never look at them again.

To me the desire to keep history is a bit like the desire some people
have to keep their old credit card slips: "Oh no, we can't throw them
away."

So my desire for a clean start in git is no more than my desire to
throw away that pile of credit card slips sitting in the corner. But
as I said if there really are people who want to look at them, I happy
to keep them. But I'm not happy to keep them if there are just a few
people saying "those credit card slips might come in useful sometime
in the future."

Martin

[Eric Shulman]

> From a personal point of view I practically never look at source code
> history, other than immediate history (that is the bit of history I've

> I personally find that there is no point in looking at history to
> understand a bit of code, whether it is to fix a bug or implement a
> new feature.

> In my view source code history is a bit like credit card slips - you
> look at this month's slips when you reconcile your credit card bill,
> but never look at them again.

> To me the desire to keep history is a bit like the desire some people
> have to keep their old credit card slips: "Oh no, we can't throw them
> away."

While *some* users actively update all their TiddlyWiki documents each
time a new TW core is released, there is an enormous collection of
active documents whose core code is not regularly updated, and
continue to use much older versions of the TW core, sometimes for
*years* beyond the update.

Of course, use of these older documents tends to fade as their
codebase 'ages out'. When people encounter problems with these older
documents, the usual recommendation is to first upgrade to the current
TW core code to eliminate any bugs that have already been fixed by
previous releases, and then see if the problem persists.

Unfortunately, when people attempt to upgrade these documents, they
often run into conflicts with their installed plugins, due to the
extent of core changes that have occurred over time. Frequently,
these conflicts have already been noted and resolved (usually by an
update to the plugin code to work around the problems), and people can
just update their plugins to match the current TW core and get back to
work.

However, if the problems *do* persist after updating all code, it then
is necessary to dig deep into the core *history* to reconstruct the
full picture of what has changed.

Tracking changes to the core from release to release is often a very
difficult process if you are not on the *inside*. I generally start
by making a diff of old vs new core. Most of the time, when comparing
the current release to the most recent previous release, the changes
shown in the diff output are simple and obvious.

However, while the diff does identify all the changed bits of code, it
does not make it clear WHY things have changed, especially in code
that has several bug fixes, made over several releases. It can
sometimes be very complicated to figure out which changes are relevant
to which fixes, particularly if it is just a line or two, buried deep
in some other changes.

SO... I go to the code history, and start working backwards. Unlike a
simple diff, the history reveals the *connections* between code
changes, not just the *existence* of those changes... particularly
when the affected code is distributed across several different core
functions.

For example, if a core function get a new parameter, the *callers* of
that function are updated to include that parameter, which can require
even more code be added in the calling function in order to calculate
a suitable value for the new parameter. I look at each and every
changed function carefully, to consider whether it has impact on any
of my existing code (i.e., my plugins). While this process is
tedious, it is the only way that I have found to really *understand*
the details of the changes.

As far as *migrating* the history rather than simply freezing Trac...
it is my opinion that using two different systems for reviewing code
history will, despite people's best intentions to be diligent,
inevitably result in information in the old Trac system being
overlooked, forgotten, or simply ignored.

To prevent this, all historical information should be included in the
new GIT-based system so that there is one tool for reviewing all code
changes, both from past releases and moving forward.

-e

[Martin Budden]

Thanks for the detailed explanation, Eric. I'm now convinced.

Martin

[tiddlygrp]

Hi,

Please keep the code history. Once the tw core becomes a separate
repository on github, it will be a lot smaller than the current svn
repository for tw ,plugins, cook etc.
Keeping the history allows (in principle) to test and bisect the
repository when a new major tw version is coming.
I also think with svn2git keeping the history is not really a burden,
but a great feature.