[Trac-dev] User management, Generic trac, Object model, and Multi-project
Itamar O
So brace yourselves ;-) (or abandon now)
The root for this discussion is my need for richer user management in Trac (http://trac.edgewall.org/ticket/2456#comment:49),
and Christian's reply that this should go into core, but employ GenericTrac (http://trac.edgewall.org/wiki/GenericTrac) instead of yet another model.
I have spent a good part of the time since then looking into tons of wiki pages and tickets related to the topics mentioned in the subject (about 60 open tabs... good thing I have tab-kit for Firefox), that seem to be very inter-related to one another, and ended up with lots of insights and ideas, but also with much confusion as to how to proceed.
From the GenericTrac page it is not clear whether the concept is still being designed and actively discussed, or completed the design phase, and if implementation has begun somewhere (trunk / experimental branch / ..).
Bottom line - what should I do in order to jump in and make it happen??
Furthermore, the details on the GenericTrac page are pretty focused on DB schema design, while I was also expecting details on a generic object model.
This generic object model is mentioned in http://trac.edgewall.org/wiki/TracDev/Proposals/DataModel (which refers the reader back to GenericTrac), and also in http://trac.edgewall.org/wiki/TracObjectModelProposal (suggestions for a term for a "facet" of a TracObject: aspect, view, trait, descriptor).
It's a little spaghetti-docs, so it's hard to tell which is the authoritative source on the subject, and the same questions as raised concerning GenericTrac apply here as well (state of design / discussion, state of implementation, pointer if I want to join).
Considerations of multi-project support are intertwined throughout the previous 3 major topics, and I couldn't agree more that introducing GenericTrac and a unified generic ObjectModel is an excellent chance to apply multi-project considerations. Unfortunately - this makes the task even scarier in size than it's already is, which contribute to feelings of confusion and disorientation I experience (I'm certain that if it wasn't a holiday here, and I wasn't looking for excuses to avoid working on my thesis-research, I would never make it this far in digging into the docs and code).
I think what should help in moving this forward is:
- A coherent description of the state of things today (my questions above)
- Some organization in the docs to avoid the spaghetti-sensation I experienced
- An operative plan to implement the generic DB-schema and ObjectModel, that is clear enough to supply pointers to contributors who are interested in working on this (for example, I'm interested in working on user management, and am willing to do extra in order to make the generic stuff happen).
If such a plan doesn't already exist in some way or another, I guess this is an opportunity to form that plan and discuss it.
I think that, ultimately, there should be the following resources on t.e.o to track this endeavor:
- Proposal-wiki-pages describing the *chosen* design of: Generic DB schema, Generic object model, Multi-project support (with historical ideas, discussions and brainstorming conveniently hidden in sub-wiki's under these proposals (or better yet, if it existed - another "facet", like discussions, or brainstorming)).
- A dedicated milestone ("Generic Trac") with the operative plan as its description.
- Assigning existing relevant tickets to this milestone, and creating new relevant tickets under this milestone.
there are really lots of tickets that are relevant, so it might be useful to select a handful as leading use-cases and include them in the above operative plan.
I'll finish with a little off-topic remark on the method of discussing proposals:
if I read the wiki correctly, it appears that discussions took place directly within the wiki, by editing the content.
this seems somewhat uncomfortable as compared to mailing list threads or ticket comments,
since I don't get notifications for wiki-updates (no cc on wiki's), and it's not as trivial to track who changed what and why.
don't read this as a complaint, but as a use-case for the generic object model, combined with the tabbed views concept (e.g. wiki's should also have cc-fields, and a discussion view (facet? :-) ), that can exist "behind" the "current content").
so, thanks for reading up to here,
and I hope this will help make stuff happen!
(there are numerous multi-year-old defects and enhancements that can be solved if we do this right!)
Cheers,
- Itamar O.
doki_pen
On Oct 1, 10:08 am, Itamar O <itamar...@gmail.com> wrote:
> don't read this as a complaint, but as a use-case for the generic object
> model, combined with the tabbed views concept (e.g. wiki's should also have
> cc-fields, and a discussion view (facet? :-) ), that can exist "behind" the
> "current content").
in Trac proper. It includes a wiki "watcher" that allows users to
simply click a "Watch This" link to receive change notifications on
wiki pages. This, in my opinion, is a much better experience then
adding a CC field, although that could be useful too. This doesn't
help users who haven't gone to their preference page and setup an
email address though. Perhaps we could (eventually) introduce a
workflow where, if you don't have an email address and click "Watch
This", you are prompted to enter one, then sent back to the referrer.
The GenericTrac does affect my work, and I have added some comments in
certain places. It would certainly reduce the amount of code required
for announcer, but it wouldn't be a drastic reduction. Only a small
amount of code is concerned with event listeners.
Shameless plug: Anyway, I'd love help/feedback with announcer if
anyone is interested. If you don't code, even documentation would be
helpful. Feedback would be especially helpful.
Christian Boos
Thanks for your interest on the topic. Let me shed some light on this.
First, some apologies for the "spaghetti" aspect, it's really because
this is all about notes which have been taken sporadically and ideas
which have evolved for now about 5 years, all starting in the
TracObjectModelProposal and consolidated during my practice of
maintaining Trac.
I did some coding experiments (an old trac-object branch, the xref
branch) but it never went far. But I kept going and thinking about this,
and now 5 years later, it looks like most aspects are crystallizing, so
we might finally see something emerge from this vaporware ;-)
I fully agree that a kind of operational summary of these ideas is now
needed, but the usual disclaimer applies: time is scarse, and pressing
bugs, new cool features and patches keep flowing in the project
meanwhile...
With this in mind, let me give you some (beginning of) answers.
On 10/1/2010 4:08 PM, Itamar O wrote:
> As the subject suggests, I am going to try to discuss several gigantic
> topics in Trac development.
> So brace yourselves ;-) (or abandon now)
>
> The root for this discussion is my need for richer user management in
> Trac (http://trac.edgewall.org/ticket/2456#comment:49),
> and Christian's reply that this should go into core, but employ
> GenericTrac (http://trac.edgewall.org/wiki/GenericTrac) instead of yet
> another model.
>
> I have spent a good part of the time since then looking into tons of
> wiki pages and tickets related to the topics mentioned in the subject
> (about 60 open tabs... good thing I have tab-kit for Firefox), that
> seem to be very inter-related to one another, and ended up with lots
> of insights and ideas, but also with much confusion as to how to proceed.
It seems the current focus is on the ticket subsystem, from several
leads. That would be one starting point, as this is also the most
complex situation, so if handled successfully, all the rest could follow
easily. But that's only one possible start, as hinted in the GenericTrac
page.
>
> From the GenericTrac page it is not clear whether the concept is still
> being designed and actively discussed, or completed the design phase,
> and if implementation has begun somewhere (trunk / experimental branch
> / ..).
Still designing, but now I'm *really* close to get started, and the fact
that you seem interested is just another "trigger" for me.
> Bottom line - what should I do in order to jump in and make it happen??
Say you just did ;-)
>
> Furthermore, the details on the GenericTrac page are pretty focused on
> DB schema design, while I was also expecting details on a generic
> object model.
> This generic object model is mentioned in
> http://trac.edgewall.org/wiki/TracDev/Proposals/DataModel (which
> refers the reader back to GenericTrac), and also in
> http://trac.edgewall.org/wiki/TracObjectModelProposal (suggestions for
> a term for a "facet" of a TracObject: aspect, view, trait, descriptor).
> It's a little spaghetti-docs, so it's hard to tell which is the
> authoritative source on the subject, and the same questions as raised
> concerning GenericTrac apply here as well (state of design /
> discussion, state of implementation, pointer if I want to join).
The main ideas about the API is even not yet written down. In short, I
think about representing the data for the resources using an immutable
View object behaving like a read-only dict, which can also be used for
caching. Then we'd have a model object which would wrap this immutable
view (think about the ._old dict) and store changes in a mutable dict. A
store delegate would take care of dumping the changes to the database
(or elsewhere -> #1132).
> Considerations of multi-project support are intertwined throughout the
> previous 3 major topics, and I couldn't agree more that introducing
> GenericTrac and a unified generic ObjectModel is an excellent chance
> to apply multi-project considerations. Unfortunately - this makes the
> task even scarier in size than it's already is, which contribute to
> feelings of confusion and disorientation I experience (I'm certain
> that if it wasn't a holiday here, and I wasn't looking for excuses to
> avoid working on my thesis-research, I would never make it this far in
> digging into the docs and code).
I've started to experiment about "object" relations when working on the
#31 topic (ticket-links branch on t.e.o), and as I currently see it, the
relations would be better kept as properties on the dependent object
(i.e. a ticket would have a "project" property linking to the id of the
project resource).
> I think what should help in moving this forward is:
> - A coherent description of the state of things today (my questions above)
> - Some organization in the docs to avoid the spaghetti-sensation I
> experienced
> - An operative plan to implement the generic DB-schema and
> ObjectModel, that is clear enough to supply pointers to contributors
> who are interested in working on this (for example, I'm interested in
> working on user management, and am willing to do extra in order to
> make the generic stuff happen).
> If such a plan doesn't already exist in some way or another, I guess
> this is an opportunity to form that plan and discuss it.
It's all open. We could develop that here on Trac-Dev, or on a Wiki
page. But what is really needed is to get started coding, as only this
can validate or invalidate the assumptions I made in those various notes.
> I think that, ultimately, there should be the following resources on
> t.e.o to track this endeavor:
> - Proposal-wiki-pages describing the *chosen* design of: Generic DB
> schema, Generic object model, Multi-project support (with historical
> ideas, discussions and brainstorming conveniently hidden in sub-wiki's
> under these proposals (or better yet, if it existed - another "facet",
> like discussions, or brainstorming)).
> - A dedicated milestone ("Generic Trac") with the operative plan as
> its description.
> - Assigning existing relevant tickets to this milestone, and creating
> new relevant tickets under this milestone.
> there are really lots of tickets that are relevant, so it might be
> useful to select a handful as leading use-cases and include them in
> the above operative plan.
Well, yes, there are lots of tickets about this. I don't think moving
those to a new milestone is a good idea. Better use keywords. Most of
the relevant tickets already have the tracobject keyword, we could use
the generic keyword to tag those which can be used to bootstrap the effort.
The GenericTrac page can be used for the chosen design, but for now
there's nothing "chosen", we need to experiment first.
And maybe what is needed to get a good overview and summary is just
*another point of view* (hint, hint).
> I'll finish with a little off-topic remark on the method of discussing
> proposals:
> if I read the wiki correctly, it appears that discussions took place
> directly within the wiki, by editing the content.
> this seems somewhat uncomfortable as compared to mailing list threads
> or ticket comments,
> since I don't get notifications for wiki-updates (no cc on wiki's),
> and it's not as trivial to track who changed what and why.
For the edits on the wiki, It's easy: they were mainly written by me ;-)
The discussion on Trac-Dev could never happen so far, as it was
"trolled" many times by comments like "-1", "this is a bad idea" and the
like. I think this won't happen again, as the people behind those
comments have not contributed anything to the project in years and it
would be very bold from them to just jump in now to try to derail this
effort. You never know, but if this happens, just ignore it. What we
need is a rationale development discussion, just as it often happen on
tickets, not hand waving.
> don't read this as a complaint, but as a use-case for the generic
> object model, combined with the tabbed views concept (e.g. wiki's
> should also have cc-fields, and a discussion view (facet? :-) ), that
> can exist "behind" the "current content").
>
> so, thanks for reading up to here,
> and I hope this will help make stuff happen!
That would be a pleasure!
> (there are numerous multi-year-old defects and enhancements that can
> be solved if we do this right!)
>
I'm glad you realize that, as this is the whole point of the exercise.
-- Christian
osimons
> Hello Itamar,
>
> Thanks for your interest on the topic. Let me shed some light on this.
> First, some apologies for the "spaghetti" aspect, it's really because
> this is all about notes which have been taken sporadically and ideas
> which have evolved for now about 5 years, all starting in the
> TracObjectModelProposal and consolidated during my practice of
> maintaining Trac.
>
> For the edits on the wiki, It's easy: they were mainly written by me ;-)
> The discussion on Trac-Dev could never happen so far, as it was
> "trolled" many times by comments like "-1", "this is a bad idea" and the
> like. I think this won't happen again, as the people behind those
> comments have not contributed anything to the project in years and it
> would be very bold from them to just jump in now to try to derail this
> effort. You never know, but if this happens, just ignore it. What we
> need is a rationale development discussion, just as it often happen on
> tickets, not hand waving.
do prefer the loosely coupled and explicit nature of objects calling
each other with simple arguments to get work done. I, like many
others, came into Trac because the simplicity of the Trac internals. I
made plugins because they were easy, powerful and flexible.
GenericTrac is a new "Framework" to be grafted into this structure,
and it is not like a strong case has been made so far for it being a
better way of doing things?
Still, it is a philosophy thing, and about as useful to argue about as
"red" being a better colour than "blue". GenericTrac can no doubt be
made to work in code as well. I have no reason to believe that you
will not make it work, it will be well designed, and it will look
deceptively simple. Anything can be made to work, without the idea
being either good or bad, or right or wrong. It would just be
different.
This "different" is fine until you need to really understand it. I
contribute to Trac, but what I mainly do is mainly plugins. You and
Remy are almost completely dedicated to core code (which is great of
course), but I feel this "different" apect in ways you don't.
1. Complex classes. Formatter, Context, GenericObject (or whatever)
contains an inner life and structure that I'd need to understand.
Complex, nested in several layers. They have non-descript names and
what they do and how they do it is not obvious by looking at them.
"Flat is better than nested."
2. Complex classes. As a plugin developer I appreciate the fact that
the internals are managed by me. The components that I write do what
they do, and they don't support all kinds of facets and features that
I don't want or need. My classes don't suddenly adapt and change
behaviour just because and admin installed the latest Trac from trunk,
which then morphed my features into something it is not made to
support, or broke security by changing some internal assumptions.
"Simple is better than complex."
2. Complex classes. Output from one call depends on the settings and
input at some other facet of the object. Perhaps set at a different
time, and in some other completely unrelated coding context for this
object being reused or passed around. Say I'm writing a request filter
or somethings else that detects and manipulates the structure, I'd
need to fully understand all the ways this object can manifest itself.
I'd need to understand what effect any change I make may have later
when for instance the template calls MyObject.render_yourself(). Side-
effects are hard to understand and debug for all but core developers.
"Explicit is better than implicit."
3. Optional complex classes. So, we are to spread our already thin
layer of effort into supporting two alternative usage "philosophies"
indefinately? I don't think so, and that certainly does not make
sense. If this is the way we go, then the rest of Trac would need to
change to support this new and better model of how things are done. We
can't do both - as illustrated by the ongoing db context refactoring.
"There should be one-- and preferably only one --obvious way to do
it."
Five years as you say, and noone really understands what it means or
involves to get it done or how exactly it will make things better...
Christian, I really appreciate all the hard work you put in. However,
as a word of advice, one of the problems I have with GenericTrac is
how it is over-abused in tagging tickets and ideas with it. It is made
out to be something that can solve all complex issues of aspected
behaviour or relationships. Like this discussion about user management
(which is a very positive thing!), gets "GenericTrac" tangled into it
for no apparent reason other than that the project has given the
impression that for all new structures, a generic way of doing things
is the way to go. In discussions like this one, we need to decouple
features from implementation to a much larger extent.
"Sparse is better than dense."
I've stayed with the project for a long time, and I'm planning to stay
on. I'll respect the direction the majority want to take, and as you
also said it is really only fair that when you write the code you have
much more important say in what should be done and how. I have no
problem with that. I do however want to use my voice to express my
concerns about this direction.
:::simon
https://www.coderesort.com
http://www.ohloh.net/accounts/osimons
Christian Boos
Your whole mail is on the premise that Trac is "simple" and that this
horribly "complex framework" is going to end this. Well, reality is a
bit different. Over the years, the Trac code base grew into something
quite complex (not overly complex, otherwise I wouldn't hang around),
and from time to time you need to take action to make things simple
*again*. The fact that from the start each submodule had a different API
and table layout to do approximately the same thing certainly
contributed to spread that complexity (and we're talking about
gratuitous differences here, not differences because of carefully made
design choices).
On the advice of Itamar, I've started to rewrite the GenericTrac
documentation into some kind of sensible plan, backed with code. This
won't happen overnight but I'm working on it. The hope is that the code
will speak for itself, it will be on the same level of complexity as the
Ticket is today, minus the quirks (like, how we grafted much
"non-relational" information in the ticket_change table).
> This "different" is fine until you need to really understand it. I
> contribute to Trac, but what I mainly do is mainly plugins. You and
> Remy are almost completely dedicated to core code (which is great of
> course), but I feel this "different" apect in ways you don't.
>
> 1. Complex classes. [...]
> "Flat is better than nested."
> 2. Complex classes. As a plugin developer I appreciate the fact that
> the internals are managed by me.
It depends on the plugin, I imagine. A lot of plugins simply try to get
around the limitations of the core system. And a lot of the complexity
of the plugins comes from the fact that the Trac internal data model is
overly complex (ticket_change anyone?). Some plugins add their own
subsystems, and there maybe it's better to stay in control, maybe it's
better to benefit from more infrastructure support. I guess it all
depends on the convenience level of the infrastructure and the degree of
specificity of the subsystem.
> "Simple is better than complex."
Amen.
> 2. Complex classes. [snip]
>
> "Explicit is better than implicit."
>
> 3. Optional complex classes. [re-snip]
>
> "There should be one-- and preferably only one --obvious way to do
> it."
> Five years as you say, and noone really understands what it means or
> involves to get it done or how exactly it will make things better...
Well, five years of me maintaining Trac, seeing all the features people
want out of it, the bugs and the quirks in the code, I can tell you I
have an idea or two how we could *improve* things. I also sometimes look
at what exists outside of Trac (really, this happens sometimes!), and I
see that we missed quite a few opportunities of not starting this effort
earlier. But well, it's not too late, 0.13, 0.14, 0.15 and 0.16 in 2
years scope, I can assure you that people will come back ;-)
I did hope that the original page (TracObjectModelProposal) would have
spoken for itself and that people would simply understand what it would
take to achieve this vision and join me to make this happen, solving the
technical details one by one. This indeed didn't really happen.
> "Sparse is better than dense."
>
> I've stayed with the project for a long time, and I'm planning to stay
> on. I'll respect the direction the majority want to take, and as you
> also said it is really only fair that when you write the code you have
> much more important say in what should be done and how. I have no
> problem with that. I do however want to use my voice to express my
> concerns about this direction.
>
See, *my* concern is that instead of discussing in a constructive way
about the practical problems the Trac code base has, and its possible
solutions, you come now with very high-level software engineering
advices, which feels a bit out of place. Thanks, but I do also read
occasionally, and in general, yes I prefer code that's easy to
understand and maintain. Especially given the fact that *I* maintain
that damn code base (not alone of course, but a good deal of it), and I
feel the pressing need for this simplification.
So I really think the kind of philosophical discussion you started leads
nowhere. We need experiments, testing, people pointing shortcomings,
suggesting better APIs, improved datamodels, smarter ways of caching,
this sort of things. And at *that* level, feel free to exercise your
software engineering knowledge, there it will be useful and you know I'm
always open to suggestions, advices and *better* ideas than mine.
As R�my said, maybe it's now time for some code...
-- Christian
Itamar O
On 10/1/2010 9:47 PM, osimons wrote:
On Oct 1, 5:16 pm, Christian Boos<cb...@neuf.fr> wrote:
[uber-snip]
See, *my* concern is that instead of discussing in a constructive way about the practical problems the Trac code base has, and its possible solutions, you come now with very high-level software engineering advices, which feels a bit out of place. Thanks, but I do also read occasionally, and in general, yes I prefer code that's easy to understand and maintain. Especially given the fact that *I* maintain that damn code base (not alone of course, but a good deal of it), and I feel the pressing need for this simplification.
I've stayed with the project for a long time, and I'm planning to stay
on. I'll respect the direction the majority want to take, and as you
also said it is really only fair that when you write the code you have
much more important say in what should be done and how. I have no
problem with that. I do however want to use my voice to express my
concerns about this direction.
So I really think the kind of philosophical discussion you started leads nowhere. We need experiments, testing, people pointing shortcomings, suggesting better APIs, improved datamodels, smarter ways of caching, this sort of things. And at *that* level, feel free to exercise your software engineering knowledge, there it will be useful and you know I'm always open to suggestions, advices and *better* ideas than mine.
As Rémy said, maybe it's now time for some code...
-- Christian
yayks.
I didn't know this was such a hot potato (or pandora box, or can of worms, or a better phrase in Hebrew that I cannot share or translate).
At least it triggered some discussion, and maybe some code to follow! :-)
I do agree a lot with a comment by Remy on the IRC -
"Try something, if it looks good, improve and extend. If it doesn't, drop it and try something else. All of this on a small part of the code."
And I think that the direction that Christian is taking (experiment with the Ticket module) is exactly that (try something on a small part of the code), as long as we remember to stop and evaluate between the ticket module and spreading it to other modules.
From the discussion (and IRC), I think we can actually leverage the "opposing camp" with the "strong opinion" by asking Simon (and those who agree strongly with his view) to formulate criteria for evaluating the benefit of the ticket module experiment.
Anyway, I will try to pick up Christian's hints later on, maybe with some of my ideas, although I think I have a simplistic and naive comprehension of the existing model(s), as a result of not having years of experience with the code base (or even plugin development).
--
You received this message because you are subscribed to the Google Groups "Trac Development" group.
To post to this group, send email to trac...@googlegroups.com.
To unsubscribe from this group, send email to trac-dev+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/trac-dev?hl=en.
Christian Boos
> contains an inner life and structure that I'd need to understand.
> Complex, nested in several layers. They have non-descript names and
> what they do and how they do it is not obvious by looking at them.
To improve the situation a bit, I'd like to rename Context to
RenderingContext, on trunk.
The old name will be kept for compatibility, Context = RenderingContext.
In the WikiParser branch, the new formatter classes could be renamed
WikiFormatter.
For Resource, I think it might be clearer to have an explicit `copy()`
method when creating a resource from another one, rather than the
shortcut `__call__` form (as a proof, before looking again at the code,
I thought `__call__` would do the same as `child()` ;-) ).
Same clarification needed for RenderingContext, were a `nested()` method
would be better than `__call__()`.
-- Christian
osimons
used in our code. I don't mind it also supporting the short-form via
__call__ -> copy(), but we should make our own code as readable as we
can.
As for nested (parent-child) contexts (and resources), could we not
just use the familiar .append()?
:::simon
Christian Boos
Ok.
> As for nested (parent-child) contexts (and resources), could we not
> just use the familiar .append()?
For resources, we have already the notion of .parent, so for creating a
sub-resources, using .child() seemed preferable to, say, ".sub()". And
talking about a child resource makes sense, I think, to convey the
notion of dependence upon its parent.
For rendering contexts, the situation is a bit different, as it's really
about nesting representation of one resource into others (e.g.
information about a ticket rendered into the description of another
ticket, rendered in a [[TicketQuery]] within a Wiki page). While talking
about the parent context, when going from the most nested to the
wrapping context makes sense, "nested" seems more appropriate than
child() for the other direction (as there's no real dependence upon the
wrapping context).
But in neither of the two above cases, .append() seems descriptive and
really appropriate to me?
-- Christian
osimons
> See, *my* concern is that instead of discussing in a constructive way
> about the practical problems the Trac code base has, and its possible
> solutions, you come now with very high-level software engineering
> advices, which feels a bit out of place. Thanks, but I do also read
> occasionally, and in general, yes I prefer code that's easy to
> understand and maintain. Especially given the fact that *I* maintain
> that damn code base (not alone of course, but a good deal of it), and I
> feel the pressing need for this simplification.
>
> So I really think the kind of philosophical discussion you started leads
> nowhere. We need experiments, testing, people pointing shortcomings,
> suggesting better APIs, improved datamodels, smarter ways of caching,
> this sort of things. And at *that* level, feel free to exercise your
> software engineering knowledge, there it will be useful and you know I'm
> always open to suggestions, advices and *better* ideas than mine.
>
> As R my said, maybe it's now time for some code...
my reasons for not wanting all code to build on the same object model
with all kinds of features poking out. It is not (surprise, surprise)
to make it easier for you to maintain. Nor is it because some
architecture is better or worse than others. As I said in the
beginning of my first post, it is not about good or bad architecture.
It is just about the differences and the changes and what it means for
THIS project (in my opinion of course).
My reason is to make it easier for me and everyone else to contribute
and help out. It is to make it easier to move new features into Trac
in a contained manner. It is to make it easy for new developers to
come in and take responsibility for individual modules within Trac
without needing to know all there is to know about all parts. The
larger picture is saved for a few people like you and Remy that can
with confidence make changes in all parts of Trac.
My fear is that we change the Trac code base into a large, shared,
generic code where you need to fully understand ALL consequences for
ALL modules in oder to confidently change ANYTHING. If someone at some
stage needs a tweaked handling for wiki module, such changes cannot be
made directly without making a case before the "committe of shared
code".
An all-shared future brings with it a whole new class of problems. If
your perspective is only how it makes it easier for YOU to maintain
the code, I fear that what you then build will stay largely a one-man
show. As I said before, heaps of credit to both you and Remy for all
the effort you put in. This project is not a straight democracy, and
the merit of your efforts now puts you in the driver seat for the
future of the code base.
What I'm trying to do (probably not all that well) is to influence how
you think about the code, and what considerations you weigh when
making your decisions.
:::simon
Leho Kraav
both approaches, but since GenericTrac has not been implemented, it
very much tickles me to at least get somewhat of a branch going to try
it out on a couple of widespread and some custom plugins. For me, just
getting a better understanding with some hands-on work with
GenericTrac would be very worthwhile.
Remy Blank
> My reason is to make it easier for me and everyone else to contribute
> and help out. It is to make it easier to move new features into Trac
> in a contained manner. It is to make it easy for new developers to
> come in and take responsibility for individual modules within Trac
> without needing to know all there is to know about all parts. The
> larger picture is saved for a few people like you and Remy that can
> with confidence make changes in all parts of Trac.
Thanks for the explanation, I finally understood your motivations. Yes,
it took me a long time, but I did get there in the end :) The two
motivations are partly conflicting, but probably not as much as it seems.
While I agree that an all-shared structure will require a longer
learning curve for contributors to *make changes* to the infrastructure,
it should make it easier to *use* the infrastructure, and to use the
gained knowledge in a different context. For example, if I have learned
how to use the wiki model object, I would then know how to use (at least
the common parts of) ticket model object as well.
Trac's component architecture is a good example of a complicated
infrastructure that is easy to use. It took me several months until I
fully understood how it worked and was able to make changes to it. But
it must have taken me less than an hour to understand how to use it, and
I didn't need to know more for a long time.
I was in the "new contributor" situation two years ago, and one thing
that gave me difficulties was the heterogeneity in core when doing the
same thing in different modules. For example, why does Ticket raise an
exception if a ticket doesn't exist, but WikiPage doesn't (and I have to
check .exists)?
Now, consistency can certainly be achieved without an all-shared
structure, but I think the latter makes it easier to achieve.
I'm happy to finally understand both views, and I hope we can find a
balance between low entry barrier and maintainability.
-- Remy
Christian Boos
>
> My reason is to make it easier for me and everyone else to contribute
> and help out. It is to make it easier to move new features into Trac
> in a contained manner. It is to make it easy for new developers to
> come in and take responsibility for individual modules within Trac
> without needing to know all there is to know about all parts.
I'm not sure how easy the current architecture makes it for new
developers when they realize they can't reuse what they learned with e.g
the wiki subsystem when dealing with the ticket subsystem... That common
part will by definition be common and that means corresponding code is
already present in the different modules, in some way or another...
> The
> larger picture is saved for a few people like you and Remy that can
> with confidence make changes in all parts of Trac.
Well, that's also partly because the larger picture is under documented,
partly because it's more complex than it should be.
> My fear is that we change the Trac code base into a large, shared,
> generic code where you need to fully understand ALL consequences for
> ALL modules in oder to confidently change ANYTHING. If someone at some
> stage needs a tweaked handling for wiki module, such changes cannot be
> made directly without making a case before the "committe of shared
> code".
That's certainly a risk, I agree that we should take this concern
seriously. But we already have lots of such "infrastructure" code, and
so far it's manageable.
Take for example the CacheManager
(http://trac.edgewall.org/wiki/TracDev/CacheManager). Sure it looks
intimidating at first, and if in your plugin you just want to cache a
few things and avoid repeated lookups in the database for example, the
first reflex is certainly to think: "Oh, I don't need this, I'll just
use a flag that I'll set after populating the cache on first use and
clear it at appropriate times to invalidate it.". And this works.
However, later you may notice that sometimes you don't see up-to-date
information, like if your cache hasn't been invalidated when it should
have. You may even trace that correctly to the fact that the
invalidation actually happened, but in a different server process... If
you try to fix this, good luck, it's not easy and it took us a few
iterations to have a workable and reusable solution. Don't you think
it's much simpler now to have a quick read at the documentation of the
CacheManager and add the @cache decorator? But are people forced to use
this? No. Does it add to the overall complexity? Definitely, but less
than if we would have to replicate the fix across the different modules.
And IIRC, we even *reduced* the amount of code needed to handle the
various (single-process) caches we had. So I hope this illustrates the
process, take a few quirks away by replacing them with a solid common
base, all of this, only when it makes sense, and possibly with taste, of
course (and I must say that in this particular example, credits for the
nicely designed API goes to Remy, which came with a much better proposal
than mine, see
http://trac.edgewall.org/wiki/TracDev/Proposals/CacheInvalidation ;-) ).
> An all-shared future brings with it a whole new class of problems. If
> your perspective is only how it makes it easier for YOU to maintain
> the code,
Well, me or anybody wanting to help. The goal remains to reduce the
overall complexity, so at least we can agree on that.
> I fear that what you then build will stay largely a one-man
> show. As I said before, heaps of credit to both you and Remy for all
> the effort you put in. This project is not a straight democracy, and
> the merit of your efforts now puts you in the driver seat for the
> future of the code base.
>
> What I'm trying to do (probably not all that well) is to influence how
> you think about the code, and what considerations you weigh when
> making your decisions.
Please keep doing so, I'm sure the message gets through somehow ;-)
-- Christian
osimons
provide core services for modules to use and reuse. Much in the spirit
of how we have provided various other services (and APIs) - request
and args parsing, session code, rendering, i18n +++. As developer and
plugin author, I can choose other solutions if I really, really wanted
to. Or wanted to try out alternative approaches for custom needs. But
that usually makes no sense as the core services offered by Trac keeps
such high quality. The essence however is that the services are to a
large extent 'opt-in' (like the CacheManager), and sometimes there
isn't a 'one-size fits all'.
The 'opt-in' approach is very different from some future where caching
gets built into the 'object framework', and where generic-based
objects get caching by inheritance. It is easy to say it will be
optional to build on these objects, but we all know that it won't be
rational to maintain *both* coding practices over time - supporting
two best ways is almost double the effort, and much of the gains will
be lost. Over time many standalone opt-in services will become
unsupported and fade away.
:::simon
Steffen Hoffmann
Hash: SHA1
Christian Boos wrote:
> On 10/8/2010 10:56 AM, osimons wrote:
>> On Oct 1, 11:25 pm, Christian Boos<cb...@neuf.fr> wrote:
>> ...
>>
>> My reason is to make it easier for me and everyone else to contribute
>> and help out. It is to make it easier to move new features into Trac
>> in a contained manner. It is to make it easy for new developers to
>> come in and take responsibility for individual modules within Trac
>> without needing to know all there is to know about all parts.
>
> I'm not sure how easy the current architecture makes it for new
> developers when they realize they can't reuse what they learned with e.g
> the wiki subsystem when dealing with the ticket subsystem... That common
> part will by definition be common and that means corresponding code is
> already present in the different modules, in some way or another...
>
>> The
>> larger picture is saved for a few people like you and Remy that can
>> with confidence make changes in all parts of Trac.
>
> Well, that's also partly because the larger picture is under documented,
> partly because it's more complex than it should be.
I think, I'm one of these people trying to figure out some inner
workings of Trac recently in order to contribute (to ticket system).
"Documentation" was just what I wanted to raise as well while reading
here for a while, long before "overall complexity". A "too much object
orientated/nested code base" just for the sake of itself would certainly
be to avoid, but there are already good examples mentioned like
CacheManager or the i18n helper method domain_functions() from
trac.util.translation, that actually do reduce complexity.
API documentation is what commonly is taking hours if not days away from
my spare time for coding, so effectively disabling further/better
contributions, as I experienced so far.
OTOH an infrastructure (DVCS) for easier contributions is gradually
building up recently, that looks promising to me. If it doesn't stop
there at the "add another recognized VCS" but goes further into the
"required steps for acceptable contribution" section this would be even
more effectively encouraging more contributions to Trac code.
Even if the bar is high for patches to finally go into Trac trunk, as
detailed i.e. in
http://trac.edgewall.org/wiki/TracDev/SubmittingPatches, people not
familiar with this procedure are usually not frown away. More at the
contrary I've got attention for largely unfinished code and I still feel
happy and grateful to receive this attention, even if it takes me a lot
to understand the moves of core development and keep up with it's pace
(certainly not high, but I'm still not much experienced with Python an
usually have to learn a lot before I understand the arguments raised and
decisions made here).
Even if it's not possible to build (all) that documentation, I emphasize
on another criteria for enabling/attracting contributors: friendliness.
We hardly can have a "too much" here, and I have to thank Odd Simon
(osimons), Carsten (rts), Bob (doki_pen) and others at #trac IRC channel
for sharing there experience and encouraging me to not give up much too
early many times. This has been an effective support when facing all the
avoidable and unavoidable difficulties I encountered so far.
(For being a non-native English speaker and tending to go OT many times
before I hope all this makes sense. I apologize, if not.)
Steffen Hoffmann
(hasienda)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkyvmlYACgkQ31DJeiZFuHdFSwCfXnGShuaR+zeIiDMkWVLobSCC
MOkAn2iKLGCtTdDne8NXmMfgs+iZYqTG
=RUGQ
-----END PGP SIGNATURE-----
Carsten Klein
> The 'opt-in' approach is very different from some future where caching
> gets built into the 'object framework', and where generic-based
> objects get caching by inheritance. It is easy to say it will be
Good point.
So, you, or we, the community, will need to figure out the actual
requirements for the new architecture in order to make it work with
existing (yours and others) extensions so far.
I would say: nah - just burn bridges along the way and force extension
developers to hop in on the wagon and adjust their code.
But then again, we could also provide for a more seamless solution, at
both the API level and the DBMS/DDL level.
Basically this would mean, keep the old data models and APIs intact, but
also introduce the new generic data model and APIs that go along with it.
How can this be accomplished during the transition phase?
First, the transition phase should be no longer than approx. a year. After
that, all of trac's data will be transmogrified to the new generic model,
including also its APIs, incl. also the data of the actively maintained
extensions.
During the transition phase, and at the api level we would then implement
facades that would provide the same basic functionality, but under the
hood we would map the old data model to the new generic data model during
the transition phase.
At the dbms level, we would keep the old ddl intact, so that existing
extensions would still be able to access the existing data, except for the
new data that would then be introduced by the generic data model, for
example content relation and so on.
So what does this mean in practice?
We will
- keep all of the currently existing DDL intact
- introduce the new tables for the generic trac model and map existing
data unto it but do not duplicate, except perhaps for the change history
- incrementally re-implement existing code so that it makes use of the
new generic data model
- introduce facades in exchange for the old apis that will hide away the
fact that we actually use both, the old and the new generic data model
via the new apis
- introduce the new apis which will make use of both the old data model
and the new generic one
In the long run:
- we will move all trac data over to the new generic model
- by that time, all extensions should have been rewritten to use the new
apis, if not, who cares since those extensions are no longer maintained,
which would actually be also a good opportunity to sort out unmaintained
projects from t.h.o and dump them to an archive page
- existing data from non maintained extensions will be kept intact
- users who dearly need the data from the unmaintained extensions will be
forced to migrate the extension to the new generic model by themselves or
hire others to do it
- win-win situation for everyone
Dear me, it is getting late (3:25am here), I have some films to watch and
I am also getting tired ;) -- will continue on this, and leave you with
the above for further discussion.
Regards
-- Carsten