Developer Documentation Project Discussion
Chris Davenport
I'd like to kick some ideas around about where the Dev Doc effort should
be going in the short, medium and long terms. To get everyone started
these are some of my current thoughts.
Medium to long term
(medium: 3 to 12 months, long: 12 months to several years, say)
My thinking at the moment is that, despite its drawbacks, phpDocumentor
is still the best tool available for handling the bulk of our
documentation. Given that it is now possible to maintain the phpdoc
tags ourselves without needing Core Dev time, we could, if we choose to
do so, make the phpDoc sources primary. By this I mean that our primary
documents are fed into phpDoc which produces the various output formats
that we need. Something like this:
Joomla! source code including phpdoc tags
+ example code fragments (can we/should we also include test code?)
+ tutorial text in DocBook XML (what tools do we use to originate and
maintain?)
is fed into phpDocumentor to produce:
* HTML on api.joomla.org + zip and tarball for users to download
* CHM (without source code?)
* PDF (probably without the source code)
* DocBook XML (it might be better/easier to generate the PDF from this)
* Wiki (do we want/need to do this?)
* Joomla! content (file containing SQL statements?) (dunno about this
one)
* Anything else? Have I missed something we should be thinking about?
Latex? Postscript? RTF? OpenOffice Writer (ODF?), Heaven forbid, M$
Word?
Note that this is a shift away from producing the docs on the wiki. The
wiki becomes just another way to view the generated docs. Assuming we
continue with the wiki at all since direct public contributions are not
permitted (due to spam).
Would this scenario cater for all our documentation needs?
Would this satisfy the needs of our translation partners?
Is there a better way to present tutorial/how-to style information than
buried in the phpDocumentor output? (I'm thinking yes, by using a
DocBook stylesheet we can render the XML before it gets processed by
phpDocumentor) How do we do that in the context of dev.joomla.org?
How do we encourage additional community contributions?
What kind of process should we set up for maintaining the phpdoc tags
for the long term? I'm thinking that we should have 1 (or more?) people
tasked with monitoring the SVN codebase (and the changelog) on a daily
basis for changes that affect the documentation. Having detected a
change they create a tracker artefact for it.
Short term (0 to 3 months, say)
- continue work on the wiki API reference.
- move Dev Doc content from help.joomla.org to dev.joomla.org.
- continue maintenance on the phpdoc tags.
- develop a phpdoc template for api.joomla.org that matches
dev.joomla.org.
- do any experimentation needed to thrash out a long-term strategy for
the dev docs.
Is the quality assurance process for API doc pages still appropriate (it
was originally devised in Mambo days when everything was on the help
site)? I'm thinking that the public review phase may be superfluous
given that all pages are publicly viewable anyway.
Is the Forge task manager the most appropriate means of managing the
documentation effort? Any better ideas? Could the process be
simplified?
Okay, that should get you started. :-)
Regards,
Chris.
--
Chris Davenport
ch...@dcsnet.demon.co.uk
Ian MacLennan
> Hi All,
>
> I'd like to kick some ideas around about where the Dev Doc effort should
> be going in the short, medium and long terms. To get everyone started
> these are some of my current thoughts.
>
> Medium to long term
> (medium: 3 to 12 months, long: 12 months to several years, say)
>
> My thinking at the moment is that, despite its drawbacks, phpDocumentor
> is still the best tool available for handling the bulk of our
> documentation. Given that it is now possible to maintain the phpdoc
> tags ourselves without needing Core Dev time, we could, if we choose to
> do so, make the phpDoc sources primary. By this I mean that our primary
> documents are fed into phpDoc which produces the various output formats
> that we need. Something like this:
>
> Joomla! source code including phpdoc tags
> + example code fragments (can we/should we also include test code?)
> + tutorial text in DocBook XML (what tools do we use to originate and
> maintain?)
>
>
like? Are we talking about dumping all of our docs into the Joomla!
source code? (examples and all?) I know that before we had tried to
get it going so that the examples were in a separate page. Have we
figured out all the ins and outs of this? Or is this why it is long term?
> is fed into phpDocumentor to produce:
> * HTML on api.joomla.org + zip and tarball for users to download
> * CHM (without source code?)
> * PDF (probably without the source code)
> * DocBook XML (it might be better/easier to generate the PDF from this)
> * Wiki (do we want/need to do this?)
> * Joomla! content (file containing SQL statements?) (dunno about this
> one)
> * Anything else? Have I missed something we should be thinking about?
> Latex? Postscript? RTF? OpenOffice Writer (ODF?), Heaven forbid, M$
> Word?
>
>
If we get phpDocumentor output going I wouldn't worry about the wiki...
I'll be working on as well one additional format - Eclipse help format.
But this would be kinda separate, I think. Just some tweaking to the
template though I think.
> Note that this is a shift away from producing the docs on the wiki. The
> wiki becomes just another way to view the generated docs. Assuming we
> continue with the wiki at all since direct public contributions are not
> permitted (due to spam).
>
>
I'm fine with this... the wiki docs do allow developer feedback though,
so long term we could have a larger registered developer base and give
more people the ability to add comments?
> Would this scenario cater for all our documentation needs?
>
What is the scope of our team now? Are we still just API? Is there
another group covering tutorials, etc. for extensions development and such?
> Would this satisfy the needs of our translation partners?
>
> Is there a better way to present tutorial/how-to style information than
> buried in the phpDocumentor output? (I'm thinking yes, by using a
> DocBook stylesheet we can render the XML before it gets processed by
> phpDocumentor) How do we do that in the context of dev.joomla.org?
>
I'm not following this completely? I'm not fully sure how docbook fits
into everything? Can you detail the process a little bit?
> How do we encourage additional community contributions?
>
>
I think we need to lower the bar for contributing first of all. Make it
easy for people to contribute (i.e. low commitment). What community
contributions do we want? If we want to produce short how-tos for
example, perhaps we could create a list of what we want, and when we see
somebody in the forums discussing/working with something similar and
having success, ask them to do a short one-off howto and how they did it?
> What kind of process should we set up for maintaining the phpdoc tags
> for the long term? I'm thinking that we should have 1 (or more?) people
> tasked with monitoring the SVN codebase (and the changelog) on a daily
> basis for changes that affect the documentation. Having detected a
> change they create a tracker artefact for it.
>
>
I think it is good to have somebody monitoring the SVN codebase. I
think the devs need to play an important role in this, so that whenever
they change the API they need to make a note of it in the CHANGELOG.
This happens to an extent now, and it is harder when there is a lot of
flux in the API happening, but this is the best way to do it, IMO. We
can also monitor the SVN and possibly even the tracker to note bugs that
have been found in the framework in the docs and to note what releases
fixed the bug.
> Short term (0 to 3 months, say)
>
> - continue work on the wiki API reference.
> - move Dev Doc content from help.joomla.org to dev.joomla.org.
> - continue maintenance on the phpdoc tags.
> - develop a phpdoc template for api.joomla.org that matches
> dev.joomla.org.
> - do any experimentation needed to thrash out a long-term strategy for
> the dev docs.
>
>
I agree with this... what level of maintenance do we do on the phpdoc
tags though? Can you expand this?
Also, I made a note to you about another thread discussing a redesign
for joomla.org. I think if we did things right we could get the design
team to develop the template for us, and then we might just have to
tweak some stuff. Unless someone here is really aching to develop the
template. I've no interest.
Can we discuss CirTap's suggestion that we maintain the wiki content in
the forge? I think there is a lot of merit in this. It all depends I
guess on getting marko to copy the stuff from the forge to the wiki.
Should we consult him on this? It may be a lot easier for testing and
development to do it locally (on local dokuwiki installations) and then
should commit to the forge. If this is a go, I will consult Marko on
this. Maybe we can automate a nightly or so process to copy from the
forge to the wiki?
There is a definite problem that we have to grapple with and that is
wiki performance. Its performance has slowed to a crawl. I've asked
Marko to look into it a couple of times, but nothing has come of it. He
set up his own server to host them and we found it was faster. We're
not sure why joomla.org is slow for this - it seems fast with everything
else.
> Is the quality assurance process for API doc pages still appropriate (it
> was originally devised in Mambo days when everything was on the help
> site)? I'm thinking that the public review phase may be superfluous
> given that all pages are publicly viewable anyway.
>
>
I hadn't though of this... it does seem a little superfluous, but then
maybe we need to announce the public review so that we can get people to
look at it? I dunno.
> Is the Forge task manager the most appropriate means of managing the
> documentation effort? Any better ideas? Could the process be
> simplified?
>
>
I think it has worked fairly well so far. I don't know if there is a
better way to do it. If we make use of it, I think it is handy. We
should maybe reset stuff that has been assigned to non-active people though.
> Okay, that should get you started. :-)
>
> Regards,
> Chris.
>
>
Ian
m.schmuck
i have cleaned a little the wiki so it is a little faster now, as you
mentioned on my site it is really faster, i talk with Brad and he also
don't know why it is slower on joomla.org and faster on my site.
Will make due to this more tests.
If you need help as stated i'm sure willing helping you ;)
Marko
Ian MacLennan schrieb:
--
Marko Schmuck
Joomla! Core Member
Ian MacLennan
> Hi Ian,
>
> i have cleaned a little the wiki so it is a little faster now, as you
> mentioned on my site it is really faster, i talk with Brad and he also
> don't know why it is slower on joomla.org and faster on my site.
>
> Will make due to this more tests.
>
> If you need help as stated i'm sure willing helping you ;)
>
> Marko
>
>
It does seem a bit faster. That's great.
We need to discuss it further as the documentation working group, but
would it be possible to feed the wiki from the forge? i.e. you would
send us a copy of the wiki initially, we would commit it to our forge,
and then the stuff from the forge would be copied to the wiki periodically?
This would allow us to version control everything and to make changes
much easier.
Ian
CirTap
Chris Davenport schrieb:
> Joomla! source code including phpdoc tags
> + example code fragments (can we/should we also include test code?)
> + tutorial text in DocBook XML (what tools do we use to originate and
> maintain?)
although it seems handy, I'd not like to have handmade
<code>fo ba </code>
stuff inside the core code comments, but use
@example /class/method/example-X
@tutorials /class/method/tutorial-X
which I assume you're refering to.
> * Wiki (do we want/need to do this?)
> * Joomla! content (file containing SQL statements?) (dunno about this
> one)
For output (read-only) DokuWiki's .txt files would allow to instantly
change a bulk of files, even drop or replace entire folders. SQL files
(dumps) could be harder to maintain, i.e. if the Joomla! content should
move or merge into another database. The same is true in case a DB based
Wiki would be used in the future. Record IDs cannot be relied upon in
case of a merge and I doubt J!'s "keyword" parameter of com_content
would be very helpful.
For editing, we should move to something that allows sematic, be it
XHTML or DocBook. Either could serve as the source for further
transformations wheras HTML would be easier to create for most people
and it provides many (barely used) semantic elements that are easy to
use. The files could run thru XSLT if needed and become DocBook
compliant XML files.
I'd however not want to edit this type of documents as J! content
(online) for none of the existing RTE-editors (XStandard?) is capable to
provide all the nice and necessary HTML elements this type of
documentation would require, and editing API specs in those tiny limited
editors would be a major pita and not much better than wiki syntax.
Ergo: go (or start) with regular XHTML files that "somehow" make into
any of the destination formats and storage systems.
> Latex? Postscript?
probably only useful for people who want to create print versions and
want/need to change the layout -- PDF would not allows doing this.
> Would this scenario cater for all our documentation needs?
> Would this satisfy the needs of our translation partners?
In case of DocBook + XSLT: yes and maybe.
If you have ever read a PHP manual in one of the other languages you'll
find several frequently-used terms are being automagically translated.
The formatting is of course identical as well.
The crux of DocBook is that there are no real editing application
available, by means of WYSIWYG -- or close to that. OOo only has a
limited support.
Speaking og OpenOffice: presuming there'd be a document template, would
be make sense to use ODT files for parts of the documentation? They're
XML actually and on the long run, I'm sure there'll be support in this
app from a Redmond company for ODT as well. Just thinking ...
> How do we do that in the context of dev.joomla.org?
very good question :)
Too bad it's still J! 1.0. With 1.5, Ian could write an extension in no
time using the new framework and submit it to ESD ;-)
> How do we encourage additional community contributions?
I suggest to join forces with Michelle in that direction since she also
has the needs for people to comment to the user manuals. I'll talk to
her tonight anyway, and will ask her if there's been any progress.
There were also references to a system similar to user ocmments in the
PHP manual.
Whatever solution pops up, it shuld no not force people to create
yet-another-account to login and comment on "documentation areas" if
they already have an account on either the extension site or in the
forums. The latter is a large source to find people who are willing to
contribute, and they'd be able to do so by using their know account.
Whenever you force people to register in a place the "think" they have
already registered for, you also block smaller contributions.
Ian MacLennan schrieb:
> What kind of process should we set up for maintaining the phpdoc tags
> for the long term? I'm thinking that we should have 1 (or more?) people
> tasked with monitoring the SVN codebase (and the changelog) on a daily
> basis for changes that affect the documentation. Having detected a
> change they create a tracker artefact for it.
I dunno if "daily" is required, but frequently, e.g. 2-3 time a week
should be fairly enough. I suspect there aren't that many changes in 1.5
If 1.6 or 2.0 hit the alpha-shelves, this is however likely to change.
This also speaks for using some documentation format that supports easy
parsing, analysis, and references: XML + XSLT. Several API docs (incl.
examples and tutorials) will xross-reference each other more or less and
a change in the API in one class f.i. may also require tweaks in other
documents.
Ian MacLennan schrieb:
> I think the devs need to play an important role in this, so that
> whenever they change the API they need to make a note of it in the
> CHANGELOG
I fully second and third that. THEY are the first to know :)
Side note: is there chance to have their UnitTests available someplace
in SVN? They could make up perfect examples or at least "inspire" in
writing some.
Ian MacLennan schrieb:
> [Wiki] performance has slowed to a crawl.
I would blame it to the incredible amount of file I/O it performs. I
somehow doubt it's core (DokuWiki) is optimized for high traffic sites.
On a heavly load server such as *.joomla.org this could certainly be a
killer, and explain why it runs nice on others and of course locally.
In addition, none of the Wiki pages sets a Cache or Expire header, hence
the browser always reloads the whole thing, all js scripts, all the nice
images from api.joomla.org ...
Have fun,
CirTap
Chris Davenport
MacLennan <joo...@ianmaclennan.org> writes
>
>Chris Davenport wrote:
>>
>> Joomla! source code including phpdoc tags
>> + example code fragments (can we/should we also include test code?)
>> + tutorial text in DocBook XML (what tools do we use to originate and
>> maintain?)
>>
>I don't know too much about this... what does the file structure look
>like? Are we talking about dumping all of our docs into the Joomla!
>source code? (examples and all?) I know that before we had tried to
>get it going so that the examples were in a separate page. Have we
>figured out all the ins and outs of this? Or is this why it is long term?
We just add @tutorial and @example tags to the source code. These
reference files that are held in the Doc Project SVN and simply merged
by phpDocumentor. Tutorials (which could be any kind of text really,
not just tutorials) need to be in DocBook, the examples can be any PHP
code enclosed in <?php and ?> tags (so the syntax highlighting works).
So only a modest increase the size of the Joomla! download.
There is an example of an @example tag on api.joomla.org. Open the
Application subpackage and click on JApplication. Scroll down to
getCfg. Oh, actually it's in the next update which is being uploaded as
I type. It's a 70Mb upload so you might have to wait a few minutes.
>> is fed into phpDocumentor to produce:
>> * HTML on api.joomla.org + zip and tarball for users to download
>> * CHM (without source code?)
>> * PDF (probably without the source code)
>> * DocBook XML (it might be better/easier to generate the PDF from this)
>> * Wiki (do we want/need to do this?)
>> * Joomla! content (file containing SQL statements?) (dunno about this
>> one)
>> * Anything else? Have I missed something we should be thinking about?
>> Latex? Postscript? RTF? OpenOffice Writer (ODF?), Heaven forbid, M$
>> Word?
>>
>>
>If we get phpDocumentor output going I wouldn't worry about the wiki...
>
>I'll be working on as well one additional format - Eclipse help format.
>But this would be kinda separate, I think. Just some tweaking to the
>template though I think.
Sounds good.
>> Note that this is a shift away from producing the docs on the wiki. The
>> wiki becomes just another way to view the generated docs. Assuming we
>> continue with the wiki at all since direct public contributions are not
>> permitted (due to spam).
>>
>>
>I'm fine with this... the wiki docs do allow developer feedback though,
>so long term we could have a larger registered developer base and give
>more people the ability to add comments?
I don't have a problem with allowing more people access to commenting,
only with full public access. AIUI the comments are kept in a separate
directory from the wiki files themselves so automatically updating the
files from phpDoc wouldn't cause the loss of the comments.
But do we want to try to merge the comments into official docs, or just
leave them as extras that you need to access the wiki to read?
>> Would this scenario cater for all our documentation needs?
>>
>What is the scope of our team now? Are we still just API? Is there
>another group covering tutorials, etc. for extensions development and such?
At the moment our original API/DB Mini-Team scope is just the API. The
Core Devs have written most of the tutorial-style material. But this is
a short/medium term strategy because documenting the API is/was a
priority. I hope that medium/long term we'll be spending more time
writing tutorial-style stuff and less documenting the API. Documenting
the API should be merely an ongoing maintenance task rather than the
mountainous challenge it is at present. The scope of the Dev Doc Team
covers ALL developer docs.
>> Is there a better way to present tutorial/how-to style information than
>> buried in the phpDocumentor output? (I'm thinking yes, by using a
>> DocBook stylesheet we can render the XML before it gets processed by
>> phpDocumentor) How do we do that in the context of dev.joomla.org?
>>
>I'm not following this completely? I'm not fully sure how docbook fits
>into everything? Can you detail the process a little bit?
Well, let's say we take the current tutorial stuff from the wiki, write
it up in DocBook (which is required for phpDoc to process it) then merge
it into the phpDoc output. Then if we do nothing else the only way to
access those tutorials is by navigating the phpDoc tree on
api.joomla.org. I don't think that's a very friendly way to present
this important information.
But, I'm pretty sure that the same source files (in DocBook format)
could be rendered on dev.joomla.org if we apply an appropriate
stylesheet. In fact, back in Mambo days, this was the way that all the
help files were presented.
So, in other words, a single tutorial file written in DocBook can be
presented directly on dev.joomla.org using a stylesheet AND merged into
the phpDoc run to be included in all the various output formats. Needs
some experimentation, but that's the theory.
>
>> How do we encourage additional community contributions?
>>
>>
>I think we need to lower the bar for contributing first of all. Make it
>easy for people to contribute (i.e. low commitment). What community
>contributions do we want? If we want to produce short how-tos for
>example, perhaps we could create a list of what we want, and when we see
>somebody in the forums discussing/working with something similar and
>having success, ask them to do a short one-off howto and how they did it?
This is one of the downsides of shifting to DocBook and using phpDoc for
(just about) everything: it actually makes it harder for newcomers to
contribute. At present the only real means of allowing contributions is
through the Dev Doc Forum. Is there a better way?
>> What kind of process should we set up for maintaining the phpdoc tags
>> for the long term? I'm thinking that we should have 1 (or more?) people
>> tasked with monitoring the SVN codebase (and the changelog) on a daily
>> basis for changes that affect the documentation. Having detected a
>> change they create a tracker artefact for it.
>>
>>
>I think it is good to have somebody monitoring the SVN codebase. I
>think the devs need to play an important role in this, so that whenever
>they change the API they need to make a note of it in the CHANGELOG.
>This happens to an extent now, and it is harder when there is a lot of
>flux in the API happening, but this is the best way to do it, IMO. We
>can also monitor the SVN and possibly even the tracker to note bugs that
>have been found in the framework in the docs and to note what releases
>fixed the bug.
>
>> Short term (0 to 3 months, say)
>>
>> - continue work on the wiki API reference.
>> - move Dev Doc content from help.joomla.org to dev.joomla.org.
>> - continue maintenance on the phpdoc tags.
>> - develop a phpdoc template for api.joomla.org that matches
>> dev.joomla.org.
>> - do any experimentation needed to thrash out a long-term strategy for
>> the dev docs.
>>
>>
>I agree with this... what level of maintenance do we do on the phpdoc
>tags though? Can you expand this?
I think the real issue is how much information can we reasonably include
in the docblocks before we get complaints that the download is getting
bloated. :-). My feeling at the moment is that all the extra verbiage
that we've added in the wiki could be backported comfortably, but some
of the longer passages might be better in @tutorial's. What I'm
concerned about is that once we've completed the big push to get
everything up-to-date, that we already have the processes in place to
keep it up-to-date efficiently.
>Also, I made a note to you about another thread discussing a redesign
>for joomla.org. I think if we did things right we could get the design
>team to develop the template for us, and then we might just have to
>tweak some stuff. Unless someone here is really aching to develop the
>template. I've no interest.
Yep, thanks for the heads-up. I've nearly finished a dev.joomla.org
lookalike template for phpDoc, but I've ended up having to create a new
converter (HTMLframelessConverter) to do it. I'm happy for the design
team to take over if they want to, but they may need to get grips with
the Smarty tags. For now I'm happy to finish what I started.
>Can we discuss CirTap's suggestion that we maintain the wiki content in
>the forge? I think there is a lot of merit in this. It all depends I
>guess on getting marko to copy the stuff from the forge to the wiki.
>Should we consult him on this? It may be a lot easier for testing and
>development to do it locally (on local dokuwiki installations) and then
>should commit to the forge. If this is a go, I will consult Marko on
>this. Maybe we can automate a nightly or so process to copy from the
>forge to the wiki?
I don't have a problem with that approach, providing Marko doesn't mind.
Letting SVN do the versioning would certainly solve the problem of the
occasional major updates that we've needed.
>There is a definite problem that we have to grapple with and that is
>wiki performance. Its performance has slowed to a crawl. I've asked
>Marko to look into it a couple of times, but nothing has come of it. He
>set up his own server to host them and we found it was faster. We're
>not sure why joomla.org is slow for this - it seems fast with everything
>else.
Yes, this is an ongoing puzzle.
>> Is the quality assurance process for API doc pages still appropriate (it
>> was originally devised in Mambo days when everything was on the help
>> site)? I'm thinking that the public review phase may be superfluous
>> given that all pages are publicly viewable anyway.
>>
>>
>I hadn't though of this... it does seem a little superfluous, but then
>maybe we need to announce the public review so that we can get people to
>look at it? I dunno.
Public reviews are announced on the Dev Doc Forum. Prior to moving to
the wiki this was also the first chance that people had of viewing the
new pages. Now, everything is public as soon as it's written.
It's just occurred to me that if we move to having the wiki generated
from phpDoc then there is no way to get the page status in the header
blocks (ie. The last reviewed date and doc status fields). People will
not know the page status from looking at it. Is this a problem? Do you
think anyone bothers to look anyway? They're a pain to maintain so
perhaps we should simply drop those fields and rely on the Forge task
manager.
>> Is the Forge task manager the most appropriate means of managing the
>> documentation effort? Any better ideas? Could the process be
>> simplified?
>>
>>
>I think it has worked fairly well so far. I don't know if there is a
>better way to do it. If we make use of it, I think it is handy. We
>should maybe reset stuff that has been assigned to non-active people though.
Sometimes it is not easy to define "non-active". :-(
Thanks for your comments Ian. Most helpful.
Ian MacLennan
> We just add @tutorial and @example tags to the source code. These
> reference files that are held in the Doc Project SVN and simply merged
> by phpDocumentor. Tutorials (which could be any kind of text really,
> not just tutorials) need to be in DocBook, the examples can be any PHP
> code enclosed in <?php and ?> tags (so the syntax highlighting works).
> So only a modest increase the size of the Joomla! download.
>
> There is an example of an @example tag on api.joomla.org. Open the
> Application subpackage and click on JApplication. Scroll down to
> getCfg. Oh, actually it's in the next update which is being uploaded as
> I type. It's a 70Mb upload so you might have to wait a few minutes.
>
>
think this is a good idea down the road.
>
> I don't have a problem with allowing more people access to commenting,
> only with full public access. AIUI the comments are kept in a separate
> directory from the wiki files themselves so automatically updating the
> files from phpDoc wouldn't cause the loss of the comments.
>
> But do we want to try to merge the comments into official docs, or just
> leave them as extras that you need to access the wiki to read?
>
>
The comments are kept in a separate directory, so this is correct.
I don't think we worry about merging the comments at this point. I
think we wait until there are helpful comments being left (and I'm not
sure there have been too many at this point). Comments that should be
included as part of the document itself should be used a basis for
edits, other comments can perhaps down the road be included in official
docs, but I wouldn't worry about it at this point.
> At the moment our original API/DB Mini-Team scope is just the API. The
> Core Devs have written most of the tutorial-style material. But this is
> a short/medium term strategy because documenting the API is/was a
> priority. I hope that medium/long term we'll be spending more time
> writing tutorial-style stuff and less documenting the API. Documenting
> the API should be merely an ongoing maintenance task rather than the
> mountainous challenge it is at present. The scope of the Dev Doc Team
> covers ALL developer docs.
>
Fair enough. I agree with prioritization.
> Well, let's say we take the current tutorial stuff from the wiki, write
> it up in DocBook (which is required for phpDoc to process it) then merge
> it into the phpDoc output. Then if we do nothing else the only way to
> access those tutorials is by navigating the phpDoc tree on
> api.joomla.org. I don't think that's a very friendly way to present
> this important information.
>
> But, I'm pretty sure that the same source files (in DocBook format)
> could be rendered on dev.joomla.org if we apply an appropriate
> stylesheet. In fact, back in Mambo days, this was the way that all the
> help files were presented.
>
> So, in other words, a single tutorial file written in DocBook can be
> presented directly on dev.joomla.org using a stylesheet AND merged into
> the phpDoc run to be included in all the various output formats. Needs
> some experimentation, but that's the theory.
>
>
So docbook is a format? What are you referring to with tutorial stuff?
I know before we were writing up the api reference as 'tutorials' - is
this what you mean? Or be tutorials do you mean extension tutorials?
Okay, I think it is good to experiment with this at some point.
>>> How do we encourage additional community contributions?
>>>
>>>
>>>
>> I think we need to lower the bar for contributing first of all. Make it
>> easy for people to contribute (i.e. low commitment). What community
>> contributions do we want? If we want to produce short how-tos for
>> example, perhaps we could create a list of what we want, and when we see
>> somebody in the forums discussing/working with something similar and
>> having success, ask them to do a short one-off howto and how they did it?
>>
>
> This is one of the downsides of shifting to DocBook and using phpDoc for
> (just about) everything: it actually makes it harder for newcomers to
> contribute. At present the only real means of allowing contributions is
> through the Dev Doc Forum. Is there a better way?
>
>
I think the formatting is the easy part... I think we need to define a
rough standard for what things should look like (i.e. describe what
we're doing), and then target people and ask them to write short
howtos. There are enough developers in the community that we could end
up with a good quantity of good stuff. I think the key is breaking it
down into easily manageable pieces and asking people directly to take on
a task, with the free understanding that they can say no. If I asked
somebody to write a howto for the helpsite, this is a bit overwhelming,
but if I asked somebody who was in the process of implementing parameter
xml files in their component to write a quick howto on it, this would
form the basis of something we could edit and publish, and it might seem
more manageable. I could be wrong though.
> I think the real issue is how much information can we reasonably include
> in the docblocks before we get complaints that the download is getting
> bloated. :-). My feeling at the moment is that all the extra verbiage
> that we've added in the wiki could be backported comfortably, but some
> of the longer passages might be better in @tutorial's. What I'm
> concerned about is that once we've completed the big push to get
> everything up-to-date, that we already have the processes in place to
> keep it up-to-date efficiently.
>
>
I don't think we should change our process right now. I think for right
now we stick with the wiki and follow through with it. When we are at a
good place with this, we work on a process to move it into docblocks.
If I can have one more week to work on my parsing script, I'll do a
Proof of Concept.
At present, my thoughts are this for the docblocks:
For each class, use the description that is on the class page from the wiki.
For each method in the class, in the docblocks we put the description
from the wiki class pages. These descriptions are shorter than the
descriptions that are on the method pages. I think this is what we want
in the docblocks (or do we want the longer descriptions?)
We only have one description for parameters. These would have to come
from the method pages, but in general I don't think these are too large.
The examples would go in @example files (these are separate, right?)
>> Also, I made a note to you about another thread discussing a redesign
>> for joomla.org. I think if we did things right we could get the design
>> team to develop the template for us, and then we might just have to
>> tweak some stuff. Unless someone here is really aching to develop the
>> template. I've no interest.
>>
>
> Yep, thanks for the heads-up. I've nearly finished a dev.joomla.org
> lookalike template for phpDoc, but I've ended up having to create a new
> converter (HTMLframelessConverter) to do it. I'm happy for the design
> team to take over if they want to, but they may need to get grips with
> the Smarty tags. For now I'm happy to finish what I started.
>
>
only thing is that it looks like the joomla.org template is going to
change...
>> Can we discuss CirTap's suggestion that we maintain the wiki content in
>> the forge? I think there is a lot of merit in this. It all depends I
>> guess on getting marko to copy the stuff from the forge to the wiki.
>> Should we consult him on this? It may be a lot easier for testing and
>> development to do it locally (on local dokuwiki installations) and then
>> should commit to the forge. If this is a go, I will consult Marko on
>> this. Maybe we can automate a nightly or so process to copy from the
>> forge to the wiki?
>>
>
> I don't have a problem with that approach, providing Marko doesn't mind.
> Letting SVN do the versioning would certainly solve the problem of the
> occasional major updates that we've needed.
>
>
I mentioned this to marko, let's see what he says. This would only be
for our folder, of course (I think everything under references is ours).
>> There is a definite problem that we have to grapple with and that is
>> wiki performance. Its performance has slowed to a crawl. I've asked
>> Marko to look into it a couple of times, but nothing has come of it. He
>> set up his own server to host them and we found it was faster. We're
>> not sure why joomla.org is slow for this - it seems fast with everything
>> else.
>>
>
> Yes, this is an ongoing puzzle.
>
>
>>> Is the quality assurance process for API doc pages still appropriate (it
>>> was originally devised in Mambo days when everything was on the help
>>> site)? I'm thinking that the public review phase may be superfluous
>>> given that all pages are publicly viewable anyway.
>>>
>>>
>>>
>> I hadn't though of this... it does seem a little superfluous, but then
>> maybe we need to announce the public review so that we can get people to
>> look at it? I dunno.
>>
>
> Public reviews are announced on the Dev Doc Forum. Prior to moving to
> the wiki this was also the first chance that people had of viewing the
> new pages. Now, everything is public as soon as it's written.
>
> It's just occurred to me that if we move to having the wiki generated
> from phpDoc then there is no way to get the page status in the header
> blocks (ie. The last reviewed date and doc status fields). People will
> not know the page status from looking at it. Is this a problem? Do you
> think anyone bothers to look anyway? They're a pain to maintain so
> perhaps we should simply drop those fields and rely on the Forge task
> manager.
>
>
I don't think these are too big of a deal... but perhaps we should go
through the docs and get rid of stuff that is only empty shells (from
the phpdoc generated wiki pages that I uploaded).
>>> Is the Forge task manager the most appropriate means of managing the
>>> documentation effort? Any better ideas? Could the process be
>>> simplified?
>>>
>>>
>>>
>> I think it has worked fairly well so far. I don't know if there is a
>> better way to do it. If we make use of it, I think it is handy. We
>> should maybe reset stuff that has been assigned to non-active people though.
>>
>
> Sometimes it is not easy to define "non-active". :-(
>
>
fair enough... I don't think this is a widespread problem anyway, though.
> Thanks for your comments Ian. Most helpful.
>
> Regards,
> Chris.
>
>
Thanks for your work in coordinating everything - it is a big job!
Ian
Chris Davenport
<cirtap...@webmechanic.biz> writes
>although it seems handy, I'd not like to have handmade
> <code>fo ba </code>
>stuff inside the core code comments, but use
> @example /class/method/example-X
> @tutorials /class/method/tutorial-X
>which I assume you're refering to.
Yes, that's what I was thinking.
>> * Wiki (do we want/need to do this?)
>> * Joomla! content (file containing SQL statements?) (dunno about this
>> one)
>For output (read-only) DokuWiki's .txt files would allow to instantly
>change a bulk of files, even drop or replace entire folders. SQL files
>(dumps) could be harder to maintain, i.e. if the Joomla! content should
>move or merge into another database. The same is true in case a DB based
>Wiki would be used in the future. Record IDs cannot be relied upon in
>case of a merge and I doubt J!'s "keyword" parameter of com_content
>would be very helpful.
Agreed. The half-baked idea I had in mind was that phpDoc would
generate SQL statements that could be imported into a fresh Joomla!
install. Hey presto, (nearly) instant documentation running on your own
local webserver. I think we'll forget this one for now though.
>For editing, we should move to something that allows sematic, be it
>XHTML or DocBook. Either could serve as the source for further
>transformations wheras HTML would be easier to create for most people
>and it provides many (barely used) semantic elements that are easy to
>use. The files could run thru XSLT if needed and become DocBook
>compliant XML files.
If we stick with phpDoc (and I think we should), then we'll need to
think DocBook rather than XHTML. I take your point about running XHTML
through XSLT to generate DocBook, but wouldn't it be easier/better to
write in DocBook in the first place?
>I'd however not want to edit this type of documents as J! content
>(online) for none of the existing RTE-editors (XStandard?) is capable to
>provide all the nice and necessary HTML elements this type of
>documentation would require, and editing API specs in those tiny limited
>editors would be a major pita and not much better than wiki syntax.
>Ergo: go (or start) with regular XHTML files that "somehow" make into
>any of the destination formats and storage systems.
Agreed. I think we definitely need to be using offline editors. But
what are our choices? Do you know any really good DocBook editors that
will produce output that phpDoc is happy with? I tried OOo about 6
months ago (maybe it's changed since) but it insisted on saving the text
as a complete book rather than the separate snippets that phpDoc wants.
Maybe there's a way around this; I couldn't find one at the time.
>> Latex? Postscript?
>probably only useful for people who want to create print versions and
>want/need to change the layout -- PDF would not allows doing this.
Okay.
>> Would this scenario cater for all our documentation needs?
>> Would this satisfy the needs of our translation partners?
>In case of DocBook + XSLT: yes and maybe.
>If you have ever read a PHP manual in one of the other languages you'll
>find several frequently-used terms are being automagically translated.
>The formatting is of course identical as well.
So are you saying that there are auto-translators for DocBook? I'm
ashamed to admit that my knowledge of non-English idioms is woefully
inadequate.
>The crux of DocBook is that there are no real editing application
>available, by means of WYSIWYG -- or close to that. OOo only has a
>limited support.
WYSIWYG would be nice, but given the likely qualifications required to
join the Dev Doc Team, is WYSIWYG really necessary? Could we use one
(or more) of the available XML editors?
>Speaking og OpenOffice: presuming there'd be a document template, would
>be make sense to use ODT files for parts of the documentation? They're
>XML actually and on the long run, I'm sure there'll be support in this
>app from a Redmond company for ODT as well. Just thinking ...
I believe phpDoc requires docs to be in DocBook format, so other formats
could not be included. We could still host such docs on dev.joomla.org
of course. Is that what you meant? In theory I suppose phpDoc could be
made to output in ODT too. I've no idea how difficult that would be
though.
>> How do we do that in the context of dev.joomla.org?
>
>very good question :)
Okay, but what's the answer ;-)
>Too bad it's still J! 1.0. With 1.5, Ian could write an extension in no
>time using the new framework and submit it to ESD ;-)
You mean assuming he hasn't resigned for lack of work before then ;-)
>> How do we encourage additional community contributions?
>I suggest to join forces with Michelle in that direction since she also
>has the needs for people to comment to the user manuals. I'll talk to
>her tonight anyway, and will ask her if there's been any progress.
>There were also references to a system similar to user ocmments in the
>PHP manual.
Yes, this is a general problem for all us documentors.
>Whatever solution pops up, it shuld no not force people to create
>yet-another-account to login and comment on "documentation areas" if
>they already have an account on either the extension site or in the
>forums. The latter is a large source to find people who are willing to
>contribute, and they'd be able to do so by using their know account.
>Whenever you force people to register in a place the "think" they have
>already registered for, you also block smaller contributions.
Agreed.
>Ian MacLennan schrieb:
>> What kind of process should we set up for maintaining the phpdoc tags
>> for the long term? I'm thinking that we should have 1 (or more?) people
>> tasked with monitoring the SVN codebase (and the changelog) on a daily
>> basis for changes that affect the documentation. Having detected a
>> change they create a tracker artefact for it.
>I dunno if "daily" is required, but frequently, e.g. 2-3 time a week
>should be fairly enough. I suspect there aren't that many changes in 1.5
>If 1.6 or 2.0 hit the alpha-shelves, this is however likely to change.
Yes, the frequency is likely to vary depending on the flux of changes to
the code.
>This also speaks for using some documentation format that supports easy
>parsing, analysis, and references: XML + XSLT. Several API docs (incl.
>examples and tutorials) will xross-reference each other more or less and
>a change in the API in one class f.i. may also require tweaks in other
>documents.
Not sure I follow you. Whoever is monitoring code changes will need to
be familiar with all the various bits of documentation that might be
affected by any given change. But phpDoc is our friend and it handles
almost all of the cross-referencing.
>Ian MacLennan schrieb:
> > I think the devs need to play an important role in this, so that
> > whenever they change the API they need to make a note of it in the
> > CHANGELOG
>
>I fully second and third that. THEY are the first to know :)
They're the first to know about code changes, but in general they are
less likely to be familiar with the docs. Yes, it would be nice if the
devs inform us of changes that affect the docs, but we also need a
safety net to catch changes that they forget about.
>Side note: is there chance to have their UnitTests available someplace
>in SVN? They could make up perfect examples or at least "inspire" in
>writing some.
I seem to remember seeing something in the Joomla! SVN about test
scripts. Don't know what the status is on that though.
Thanks for your comments, CirTap. Very helpful.
Chris Davenport
>> leave them as extras that you need to access the wiki to read?
>>
>think we wait until there are helpful comments being left (and I'm not
>sure there have been too many at this point). Comments that should be
>included as part of the document itself should be used a basis for
>edits, other comments can perhaps down the road be included in official
>docs, but I wouldn't worry about it at this point.
Okay, I'll go along with that.
>> Well, let's say we take the current tutorial stuff from the wiki, write
>> it up in DocBook (which is required for phpDoc to process it) then merge
>> it into the phpDoc output. Then if we do nothing else the only way to
>> access those tutorials is by navigating the phpDoc tree on
>> api.joomla.org. I don't think that's a very friendly way to present
>> this important information.
>>
>> But, I'm pretty sure that the same source files (in DocBook format)
>> could be rendered on dev.joomla.org if we apply an appropriate
>> stylesheet. In fact, back in Mambo days, this was the way that all the
>> help files were presented.
>>
>> So, in other words, a single tutorial file written in DocBook can be
>> presented directly on dev.joomla.org using a stylesheet AND merged into
>> the phpDoc run to be included in all the various output formats. Needs
>> some experimentation, but that's the theory.
>>
>>
>So docbook is a format? What are you referring to with tutorial stuff?
>I know before we were writing up the api reference as 'tutorials' - is
>this what you mean? Or be tutorials do you mean extension tutorials?
>
>Okay, I think it is good to experiment with this at some point.
DocBook is XML but with a specialised tag set. I'm using the term
"tutorial" because that's what phpDoc calls them. They could be any
chunk of text that is linked to a particular phpDoc object (like a
package, file or class).
>
>>>> How do we encourage additional community contributions?
>>>>
>>>>
>>>>
>>> I think we need to lower the bar for contributing first of all. Make it
>>> easy for people to contribute (i.e. low commitment). What community
>>> contributions do we want? If we want to produce short how-tos for
>>> example, perhaps we could create a list of what we want, and when we see
>>> somebody in the forums discussing/working with something similar and
>>> having success, ask them to do a short one-off howto and how they did it?
>>>
>>
>> This is one of the downsides of shifting to DocBook and using phpDoc for
>> (just about) everything: it actually makes it harder for newcomers to
>> contribute. At present the only real means of allowing contributions is
>> through the Dev Doc Forum. Is there a better way?
>>
>>
>I think the formatting is the easy part... I think we need to define a
>rough standard for what things should look like (i.e. describe what
>we're doing), and then target people and ask them to write short
>howtos. There are enough developers in the community that we could end
>up with a good quantity of good stuff. I think the key is breaking it
>down into easily manageable pieces and asking people directly to take on
>a task, with the free understanding that they can say no. If I asked
>somebody to write a howto for the helpsite, this is a bit overwhelming,
>but if I asked somebody who was in the process of implementing parameter
>xml files in their component to write a quick howto on it, this would
>form the basis of something we could edit and publish, and it might seem
>more manageable. I could be wrong though.
Sounds like a really good idea to me. I think we should try it. Let me
mull that over tonight.
>
>> I think the real issue is how much information can we reasonably include
>> in the docblocks before we get complaints that the download is getting
>> bloated. :-). My feeling at the moment is that all the extra verbiage
>> that we've added in the wiki could be backported comfortably, but some
>> of the longer passages might be better in @tutorial's. What I'm
>> concerned about is that once we've completed the big push to get
>> everything up-to-date, that we already have the processes in place to
>> keep it up-to-date efficiently.
>>
>>
>I don't think we should change our process right now. I think for right
>now we stick with the wiki and follow through with it. When we are at a
>good place with this, we work on a process to move it into docblocks.
>If I can have one more week to work on my parsing script, I'll do a
>Proof of Concept.
Yes, we're talking long term here. We need to continue with our focus
on the wiki docs for the moment. Look forward to your POC.
>
>At present, my thoughts are this for the docblocks:
>For each class, use the description that is on the class page from the wiki.
>For each method in the class, in the docblocks we put the description
>from the wiki class pages. These descriptions are shorter than the
>descriptions that are on the method pages. I think this is what we want
>in the docblocks (or do we want the longer descriptions?)
>We only have one description for parameters. These would have to come
>from the method pages, but in general I don't think these are too large.
>The examples would go in @example files (these are separate, right?)
We need to be aware that anything that goes into docblocks bloats the
code, so if a long description can be broken out into a linked @tutorial
then it should be. Yes, the @example files are separate so we can make
them as big as we want!
>> Yep, thanks for the heads-up. I've nearly finished a dev.joomla.org
>> lookalike template for phpDoc, but I've ended up having to create a new
>> converter (HTMLframelessConverter) to do it. I'm happy for the design
>> team to take over if they want to, but they may need to get grips with
>> the Smarty tags. For now I'm happy to finish what I started.
>>
>>
>only thing is that it looks like the joomla.org template is going to
>change...
So I gather :-( The template as such is already done (and looks really
good, though I say so myself). What I need to finish is getting the
collapsing menus to display the right information at the right time,
which is a Smarty/phpDoc converter issue. I've had to create a new
converter which is a hybrid of HTMLframesConverter and
HTMLSmartyConverter in order to get the tags for the menus into a
frameless template. Changing the template should not, in theory, be a
major task right now. When it's done I'll put a test release somewhere
public so you can take a look.
>>> Can we discuss CirTap's suggestion that we maintain the wiki content in
>>> the forge? I think there is a lot of merit in this. It all depends I
>>> guess on getting marko to copy the stuff from the forge to the wiki.
>>> Should we consult him on this? It may be a lot easier for testing and
>>> development to do it locally (on local dokuwiki installations) and then
>>> should commit to the forge. If this is a go, I will consult Marko on
>>> this. Maybe we can automate a nightly or so process to copy from the
>>> forge to the wiki?
>>>
>>
>> I don't have a problem with that approach, providing Marko doesn't mind.
>> Letting SVN do the versioning would certainly solve the problem of the
>> occasional major updates that we've needed.
>>
>>
>I mentioned this to marko, let's see what he says. This would only be
>for our folder, of course (I think everything under references is ours).
Yep, let's go for it. Can I leave you to organise the details and
liaise with Marko, Ian?
>> It's just occurred to me that if we move to having the wiki generated
>> from phpDoc then there is no way to get the page status in the header
>> blocks (ie. The last reviewed date and doc status fields). People will
>> not know the page status from looking at it. Is this a problem? Do you
>> think anyone bothers to look anyway? They're a pain to maintain so
>> perhaps we should simply drop those fields and rely on the Forge task
>> manager.
>>
>>
>I don't think these are too big of a deal... but perhaps we should go
>through the docs and get rid of stuff that is only empty shells (from
>the phpdoc generated wiki pages that I uploaded).
Er, not sure what you mean. Example?
>Thanks for your work in coordinating everything - it is a big job!
It sure is, but great fun!
CirTap
> DocBook is XML but with a specialised tag set. I'm using the term
> "tutorial" because that's what phpDoc calls them. They could be any
> chunk of text that is linked to a particular phpDoc object (like a
> package, file or class).
http://www.oasis-open.org/committees/tc_cat.php?cat=doccent
here you'll find both DocBook and the Open Document Formats used by
OpenOffice 2 and Sun.
Head of to "Documents" in each section to get a bunch of formats for
the specs.
At http://www.docbook.org/tdg/ you find the "Definitive Guide".
I was using the CHM verson of it, which is guiet handy to look up things.
What I don't grasp is the "package format" used by phpDocumentor. I
actually never managed to make them work or parse properly :)
That's however another story ...
Have fun,
CirTap
Ian MacLennan
this stuff. The Joomla! documentation team is my first foray into any
of this stuff (and it has been an enjoyable one, I must say).
then? From what I gather:
phpDoc tags for each class and method (built into the code)
Example pages that are linked from the phpDoc tags
a tutorial page for each class and method?
I think the challenge will be maintaining everything without having too
much duplication. My ideal would be to be able to make all changes in
one place and have that central repository update everything else...
I'm not sure if this is reasonable though.
>>> Yep, thanks for the heads-up. I've nearly finished a dev.joomla.org
>>> lookalike template for phpDoc, but I've ended up having to create a new
>>> converter (HTMLframelessConverter) to do it. I'm happy for the design
>>> team to take over if they want to, but they may need to get grips with
>>> the Smarty tags. For now I'm happy to finish what I started.
>>>
>>>
>>>
>> only thing is that it looks like the joomla.org template is going to
>> change...
>>
>
> So I gather :-( The template as such is already done (and looks really
> good, though I say so myself). What I need to finish is getting the
> collapsing menus to display the right information at the right time,
> which is a Smarty/phpDoc converter issue. I've had to create a new
> converter which is a hybrid of HTMLframesConverter and
> HTMLSmartyConverter in order to get the tags for the menus into a
> frameless template. Changing the template should not, in theory, be a
> major task right now. When it's done I'll put a test release somewhere
> public so you can take a look.
>
>
Can't wait... a new converter was probably inevitable in all this...
to properly generate the wiki docs a new converter would also have been
needed, but I ended up leaving everything in one file.
>>>> Can we discuss CirTap's suggestion that we maintain the wiki content in
>>>> the forge? I think there is a lot of merit in this. It all depends I
>>>> guess on getting marko to copy the stuff from the forge to the wiki.
>>>> Should we consult him on this? It may be a lot easier for testing and
>>>> development to do it locally (on local dokuwiki installations) and then
>>>> should commit to the forge. If this is a go, I will consult Marko on
>>>> this. Maybe we can automate a nightly or so process to copy from the
>>>> forge to the wiki?
>>>>
>>>>
>>> I don't have a problem with that approach, providing Marko doesn't mind.
>>> Letting SVN do the versioning would certainly solve the problem of the
>>> occasional major updates that we've needed.
>>>
>>>
>>>
>> I mentioned this to marko, let's see what he says. This would only be
>> for our folder, of course (I think everything under references is ours).
>>
>
> Yep, let's go for it. Can I leave you to organise the details and
> liaise with Marko, Ian?
>
>
Yep, I will follow up with Marko.
>>> It's just occurred to me that if we move to having the wiki generated
>>> from phpDoc then there is no way to get the page status in the header
>>> blocks (ie. The last reviewed date and doc status fields). People will
>>> not know the page status from looking at it. Is this a problem? Do you
>>> think anyone bothers to look anyway? They're a pain to maintain so
>>> perhaps we should simply drop those fields and rely on the Forge task
>>> manager.
>>>
>>>
>>>
>> I don't think these are too big of a deal... but perhaps we should go
>> through the docs and get rid of stuff that is only empty shells (from
>> the phpdoc generated wiki pages that I uploaded).
>>
>
> Er, not sure what you mean. Example?
>
>
http://dev.joomla.org/component/option,com_jd-wiki/Itemid,31/id,references:joomla.framework:application:jmodulehelper/
http://dev.joomla.org/component/option,com_jd-wiki/Itemid,31/id,references:joomla.framework:application:jpluginhelper-getplugin/
http://dev.joomla.org/component/option,com_jd-wiki/Itemid,31/id,references:joomla.framework:application:jplugin/
i.e. these classes at first glance look like they are done, but there is
really nothing there beyond what was in the docblock tags months ago
when they were created. That was after I had originally created the
phpdoc template for the wiki. If we have documentation that we have
finished and is waiting for review, I have no problem putting that up.
But we really should get rid of this stuff that will only waste people's
time (IMO).
I think there are fewer of these than I imagined, though... I guess we
are making progress...
Cheers,
Ian
CirTap
Chris Davenport schrieb:
> If we stick with phpDoc (and I think we should), then we'll need to
> think DocBook rather than XHTML. I take your point about running XHTML
> through XSLT to generate DocBook, but wouldn't it be easier/better to
> write in DocBook in the first place?
if you can convince any author to learn the necessary DocBook taga and
they have the time to learn them, sure.
I was suggesting (X)HTML since it basically provides all the required
"sematic" elements to write such documents. Given that more people have
the knowledge to write HTML (syntax wise) even using apps like
Dreamweaver, GoLive or <cough> Frontpage for editing, an XSLT could
process these files to transform them into DocBook XML, do the lookups
of keys etc.
> So are you saying that there are auto-translators for DocBook? I'm
> ashamed to admit that my knowledge of non-English idioms is woefully
> inadequate.
In case of the PHP manual, the stylesheet utilize a feature of XSLT to
lookup keys in external XML documents. They contain the "fixed" phrases
or fragments that are inserted in the main document. Nice feature :)
> WYSIWYG would be nice, but given the likely qualifications required to
> join the Dev Doc Team, is WYSIWYG really necessary? Could we use one
> (or more) of the available XML editors?
you're actually right.
I'd just be the amount of new tagnames and nesting that ppl. need to
learn that could stop them or at least slow down writing.
Maybe a nice XML editor with the ability to provide a DTD or WSL for
element nesting and names would be nice. I never tried the editor in
Eclipse to work with DocBook, but it did work well with less complex DTDs.
> I believe phpDoc requires docs to be in DocBook format, so other formats
> could not be included.
I though of using the well-documented, standrd ODT XML file and use that
to generate the DocBook XML. Again with XSLT. It'd be worth looking at
Oasis if such stylesheets exists.
>> Too bad it's still J! 1.0. With 1.5, Ian could write an extension in no
>> time using the new framework and submit it to ESD ;-)
>
> You mean assuming he hasn't resigned for lack of work before then ;-)
exactly <lol>
>> This also speaks for using some documentation format that supports easy
>> parsing, analysis, and references: XML + XSLT. Several API docs (incl.
>> examples and tutorials) will xross-reference each other more or less and
>> a change in the API in one class f.i. may also require tweaks in other
>> documents.
>
> Not sure I follow you. Whoever is monitoring code changes will need to
> be familiar with all the various bits of documentation that might be
> affected by any given change. But phpDoc is our friend and it handles
> almost all of the cross-referencing.
I doubt there'll ever be one person to know all the docs by heart and
where which parameter and method is mentioned.
phpDocumentor can only maintain the information it's aware of, the
status quo. Remove a method from the code, and links inside tutorials or
API just disappear, but no one would probably notice that this
particular method or even it's arguments might be further discussed in
some other text or context (tutorial) -- until someone reads that
"outdated" information, posts a comment and urge someone from the
docteam have it fixed :)
You may even "loose" a link to a set of documents that suddenly become
abandoned for no real reason.
> They're the first to know about code changes, but in general they are
> less likely to be familiar with the docs.
well, presuming that the complete API will be documented (some day), any
rename of a class, method or argument is a candidate. I admit I have not
read the CHANGELOG as intense as I did recently. Entries like the ones
of 15-Nov-2006 to 21-Nov-2006 are fairly good enough IMHO to ring an
"alarm".
> I seem to remember seeing something in the Joomla! SVN about test
> scripts. Don't know what the status is on that though.
guess we best add one of the develpers. I'm pretty sure there's a nice
set of testcases available someplace.
Have fun,
CirTap
Ian MacLennan
the forge. He will now be working on getting it set up to copy over to
the wiki on a periodic basis.
Hope all are well!
Ian
CirTap
> Okay, I have gotten the wiki content from Marko and have committed it to
> the forge. He will now be working on getting it set up to copy over to
> the wiki on a periodic basis.
excellent! I was just in the progress in editing my local files.
Thanks, Ian and Marko!!
> Hope all are well!
much better now :)
Have fun,
CirTap
Chris Davenport
MacLennan <joo...@ianmaclennan.org> writes
>
>>> At present, my thoughts are this for the docblocks:
>>> For each class, use the description that is on the class page from the wiki.
>>> For each method in the class, in the docblocks we put the description
>>>
>> >from the wiki class pages. These descriptions are shorter than the
>>
>>> descriptions that are on the method pages. I think this is what we want
>>> in the docblocks (or do we want the longer descriptions?)
>>> We only have one description for parameters. These would have to come
>>>
>> >from the method pages, but in general I don't think these are too large.
>>
>>> The examples would go in @example files (these are separate, right?)
>>>
>>
>> We need to be aware that anything that goes into docblocks bloats the
>> code, so if a long description can be broken out into a linked @tutorial
>> then it should be. Yes, the @example files are separate so we can make
>> them as big as we want!
>>
>>
>So for a given class, what documentation would we end up having on it
>then? From what I gather:
>phpDoc tags for each class and method (built into the code)
>Example pages that are linked from the phpDoc tags
>a tutorial page for each class and method?
Yes, that's what I'm thinking at the moment.
>I think the challenge will be maintaining everything without having too
>much duplication. My ideal would be to be able to make all changes in
>one place and have that central repository update everything else...
>I'm not sure if this is reasonable though.
I can see we're on the same wavelength. :-)
>> So I gather :-( The template as such is already done (and looks really
>> good, though I say so myself). What I need to finish is getting the
>> collapsing menus to display the right information at the right time,
>> which is a Smarty/phpDoc converter issue. I've had to create a new
>> converter which is a hybrid of HTMLframesConverter and
>> HTMLSmartyConverter in order to get the tags for the menus into a
>> frameless template. Changing the template should not, in theory, be a
>> major task right now. When it's done I'll put a test release somewhere
>> public so you can take a look.
>>
>>
>Can't wait... a new converter was probably inevitable in all this...
>to properly generate the wiki docs a new converter would also have been
>needed, but I ended up leaving everything in one file.
If the feeling is that we need to keep the wiki, for commenting and the
like, then at some point we will need to write a proper converter for
the wiki format. One for the to-do list, I think.
Okay, I see what you mean. By all means delete pages if they are
misleading or not fit for purpose.
Yes, it's a big mountain to climb, but the view is already impressive
and the view from the top will make all the effort seem insignificant!
Ian MacLennan
>>>
>> So for a given class, what documentation would we end up having on it
>> then? From what I gather:
>> phpDoc tags for each class and method (built into the code)
>> Example pages that are linked from the phpDoc tags
>> a tutorial page for each class and method?
>>
>
> Yes, that's what I'm thinking at the moment.
>
>
>> I think the challenge will be maintaining everything without having too
>> much duplication. My ideal would be to be able to make all changes in
>> one place and have that central repository update everything else...
>> I'm not sure if this is reasonable though.
>>
>
> I can see we're on the same wavelength. :-)
>
>
>>> So I gather :-( The template as such is already done (and looks really
>>> good, though I say so myself). What I need to finish is getting the
>>> collapsing menus to display the right information at the right time,
>>> which is a Smarty/phpDoc converter issue. I've had to create a new
>>> converter which is a hybrid of HTMLframesConverter and
>>> HTMLSmartyConverter in order to get the tags for the menus into a
>>> frameless template. Changing the template should not, in theory, be a
>>> major task right now. When it's done I'll put a test release somewhere
>>> public so you can take a look.
>>>
>>>
>>>
>> Can't wait... a new converter was probably inevitable in all this...
>> to properly generate the wiki docs a new converter would also have been
>> needed, but I ended up leaving everything in one file.
>>
>
> If the feeling is that we need to keep the wiki, for commenting and the
> like, then at some point we will need to write a proper converter for
> the wiki format. One for the to-do list, I think.
>
>
I wouldn't bother with a wiki format (at least for a while). This might
be better done with a Joomla! exporter and a Joomla! comments
component. I think we've seen that the wiki is just too slow (at least
in its current state) to meet our long term goals. I was merely
commenting on the fact that you had to make a new converter.
will do this when I get a chance.
I don't know if you picked this up or not, but there is a thread:
http://forum.joomla.org/index.php?topic=115765.new;topicseen#new you
might want to check out and keep tabs on. I'll work with the group as I
am able to help them out. Also, I noticed there seems to be a response
to the 'Help Wanted' posting.
> Yes, it's a big mountain to climb, but the view is already impressive
> and the view from the top will make all the effort seem insignificant!
>
> Regards,
> Chris.
>
>
Looking forward to the view!
Ian
Chris Davenport
MacLennan <joo...@ianmaclennan.org> writes
>I don't know if you picked this up or not, but there is a thread:
>http://forum.joomla.org/index.php?topic=115765.new;topicseen#new you
>might want to check out and keep tabs on. I'll work with the group as I
>am able to help them out. Also, I noticed there seems to be a response
>to the 'Help Wanted' posting.
Yes, this might be a fruitful source of some useful tutorial or how-to
style documentation. As you suggested yesterday I think that if we
proactively encourage individuals or small groups to write stuff up as
they are in the process of doing something, then we could end up with
more useful documentation and quicker.
I think if the team members can be encouraged to blog about what they
are doing and what they have learned and the documentation can evolve in
a wiki, then we could end up with something really valuable.
Re: 'Help Wanted'. Already in hand.
Thanks for the heads up.
Ian MacLennan
Just thought I'd poke my head in here again to see if there are more
things we need to resolve? Should we summarize our short medium and
long term tasks somewhere so that we know where we are going?
Ian
Chris Davenport
You must be telepathic, I was just thinking the same thing myself! I
decided to re-read everything over the weekend and work up some sort of
strategy document we can all refer to.
If there are any unresolved topics then please feel free to continue the
discussion.
Thanks to everyone for a lively and interesting exchange. Some useful
ideas were raised and will be followed up. I feel that this was a
fruitful exercise and one that we should repeat from time to time.
Regards,
Chris.
In message <BAYC1-PASMTP0770974...@CEZ.ICE>, Ian
MacLennan <joo...@ianmaclennan.org> writes
>
--
Chris Davenport
ch...@dcsnet.demon.co.uk
Ian MacLennan
> Yes, we're talking long term here. We need to continue with our focus
> on the wiki docs for the moment. Look forward to your POC.
>
> Chris.
>
>
Okay... here it is... my proof of concept for converting the wiki
docs. Some of it is pretty rough code, but it works (more or less).
A few notes:
you have to configure the location of the wiki docs (it will tell you
when you select the component). this path should be relative to your
Joomla! install (mine is pages, and I have the files in
/htdocs/joomla15/pages. Joomla! won't allow a component to look outside
its directory.
After you have configured the component (i.e. clicked on the
configuration icon and the popup comes up), you have to refresh and then
the link will be displayed to import.
It currently imports only the method files (not the class files).
It will list the methods that were imported. To see all the data, you
have to look in the database. It creates six tables - packages,
classes, methods, properties, examples and parameters.
I think that should be it... interested in feedback.
Ian
CirTap
nice work!
So that's the component that demonstrates that it's far too easy to
write extensions for J! 1.5? :)
> I think that should be it... interested in feedback.
no problem. It's a long one.
(Maybe we should move a further discussion to the doc-forum?)
"technical" issues:
- I had to enable the legacy layer for the installation (a J! bug?)
- configuring the path didn't really work, the component insisted to
import from "/my-joomla-root/pages/" only
- it didn't add "JObject" into the classes table 'cos there were no
"method files" available for this class, but "jobject.txt" did exist.
That's probably a known "limitation" according to this comment:
// here is where we parse a class file
<g>
The /pages/ folder contained a backup from my very first local wiki
tests with only a few files.
Now I'm actually wondering why the "method" parser is allowed to create
a new "class" record? I assume it's a temporary workaround.
DB Schema:
A couple of date fields would be nice then: created, updated and such,
to aid tracking/filtering of revisions.
How do you plan do go from here?
Suggestions:
- Description
The description field could be devided in two: summary + description,
you name it.
phpDoc (like JavaDoc) provides different meanings for the first
paragraph of a comment and the remaining text. Child classes can
@inherit the first line/paragrah of a comment (summary) from parent
classes or from doc-groups
/**@#+
* This comment will apply to any subsequent item.
*/
...
/**#- end */
The text that's supposed to go back into the core code (as planned) can
be limited to the content of such a "summary" collumn w/o parsing it
again. Both summary + description would however appear in the published
documentation.
- Field names
This one's probably a matter of style and habits but anyway: I would
have named the foreign key fields "class_id", "method_id" etc. rather
than just "class" or "method". Several DB tools are capable to rebuild
(reverse engineer) the relationship of tables if they see "tablename_id":
bar.id => foo.bar_id
stuff.id => foo.stuff_id
- Example code (part 1)
This could be due to the POC status, but IMHO the "example" column
should not contain the Wiki markup <code php|Example></code> but the
plain code only.
Doing so allows reusing the code snippets w/o extra parsing and for the
variety of possible output formats and wrap'em up in whatever suits the
target media or style, e.g:
- HTML
<pre class="code php">
... content of example column ...
</pre>
- DocBook
<InformalExample Conformance="1.5" UserLevel="moderate"
XRefLabel="JFoo.methodBar.example123">
<para>preceeding blurb</para>
<ProgramListing ID="123">
... content of example column ...
<ProgramListing>
<para>trailing blurb</para>
</InformalExample>
- phpDoc's @example file
<?php
... content of example column ...
?>
Eclipse help or "Cheat sheets":
I have no idea :)
- Examples (part 2)
The content of the "example" column accidently reports a flaw in the
current formatting + structure in the Wiki pages that no parser would be
able to catch, so I don't "blame" your code :)
There's no clear separation between each individual example. Descriptive
text may appear before and after the actual code. An example may as well
consist of several code fragments.
If you look at the rather minimalistic DocBook snippet above you can see
how it 'could' be structured.
Impementation:
excellent usage of the eazy J! Framework :)
Code "errors":
with E_ALL reporting
in view.html.php, line 16, throws a notice:
$this->assignRef('configured', $state->get('configured'));
The get() method does not return a reference. It depends on PHP.INI
allow_call_time_pass_reference = On/Off whether this code will work or not.
To be safe, replace the one-liner with:
$configured = $state->get('configured');
$this->assignRef('configured', $configured);
or use
$this->assign('configured', $state->get('configured') );
"Reinventing the wheel"
I'd suggest that instead of writing a new "Wiki parser", include
/com_openwiki/inc/parser/lexer.php + parser.php + handler.php
as a "3PD library" and use them for parsing the files.
You could probably even mimic DokuWiki's concept of "plugins" to *store*
the data instead of formatting it for output.
The original code and workflow of the DokuWiki Lexer/Parser might need
some (minor) tweaks to ommit Wiki specific procedures, but it should
save you a lot of time.
see: http://wiki.splitbrain.org/wiki:parser
Have fun,
CirTap
Ian MacLennan schrieb:
Ian MacLennan
> Hi Ian,
>
> nice work!
> So that's the component that demonstrates that it's far too easy to
> write extensions for J! 1.5? :)
>
> > I think that should be it... interested in feedback.
> no problem. It's a long one.
> (Maybe we should move a further discussion to the doc-forum?)
>
> "technical" issues:
> - I had to enable the legacy layer for the installation (a J! bug?)
>
other purposes, so I wasn't aware that I was using anything legacy.
> - configuring the path didn't really work, the component insisted to
> import from "/my-joomla-root/pages/" only
>
whoops... had it hard coded at first and then added the ability to
configure it. Guess I didn't change the part where it actually reads
the directory.
> - it didn't add "JObject" into the classes table 'cos there were no
> "method files" available for this class, but "jobject.txt" did exist.
> That's probably a known "limitation" according to this comment:
> // here is where we parse a class file
> <g>
>
yep... I made a note about that in my original email. This was only
proof of concept.
> The /pages/ folder contained a backup from my very first local wiki
> tests with only a few files.
> Now I'm actually wondering why the "method" parser is allowed to create
> a new "class" record? I assume it's a temporary workaround.
>
>
The method files need a class record to link to. So they will check if
the class exists. If the class exists, then it will just grab its id
and use it. If the class doesn't exist, it will create a record for it
with the skeleton information that it has. The method parser parses
method files, which contain some information about the class (i.e. the
name and the package). The method parser can also create two package
records (Joomla.Framework and say Application, which is linked to
Joomla.Framework through the parent field). This should also work (I
think) for JFactory and JVersion (though I may need to add a few ifs
into the parser).
> DB Schema:
> A couple of date fields would be nice then: created, updated and such,
> to aid tracking/filtering of revisions.
>
> How do you plan do go from here?
>
>
My original intention was to write something simple just to parse the
method files and put them into the database. It started off as a
regular script, then I decided for easy of distribution to package it
into a component, and then since I've been playing around with MVC, I
made it into MVC (although I'm not a software engineer, this is more of
a hobby, so there are probably things that are a little out of place).
I may also use this in the interim to create the doc files for the
J!Code project (Eclipse).
I also thought about adding a version field, so that it would do basic
version control, but I think this would cause more hassle than it is
worth, especially since it also will eventually collate info from
different places.
> Suggestions:
> - Description
> The description field could be devided in two: summary + description,
> you name it.
>
yeah, I will probably add a short description field. My intention was
to get this information from the class file, since it contains shorter
descriptions. If you have other thoughts, I'm open to suggestions...
> phpDoc (like JavaDoc) provides different meanings for the first
> paragraph of a comment and the remaining text. Child classes can
> @inherit the first line/paragrah of a comment (summary) from parent
> classes or from doc-groups
> /**@#+
> * This comment will apply to any subsequent item.
> */
> ...
> /**#- end */
> The text that's supposed to go back into the core code (as planned) can
> be limited to the content of such a "summary" collumn w/o parsing it
> again. Both summary + description would however appear in the published
> documentation.
>
> - Field names
> This one's probably a matter of style and habits but anyway: I would
> have named the foreign key fields "class_id", "method_id" etc. rather
> than just "class" or "method". Several DB tools are capable to rebuild
> (reverse engineer) the relationship of tables if they see "tablename_id":
> bar.id => foo.bar_id
> stuff.id => foo.stuff_id
>
okay... good to know... again, not a software engineer, just a
hobbyist... I'll change it appropriately.
> - Example code (part 1)
> This could be due to the POC status, but IMHO the "example" column
> should not contain the Wiki markup <code php|Example></code> but the
> plain code only.
> Doing so allows reusing the code snippets w/o extra parsing and for the
> variety of possible output formats and wrap'em up in whatever suits the
> target media or style, e.g:
> - HTML
> <pre class="code php">
> ... content of example column ...
> </pre>
>
>
So I just need to translate <code php|Example> into <?php and </code>
into ?> then? I hadn't done too much with the example code. Sounds
like a pretty simple regular expression.
> - DocBook
> <InformalExample Conformance="1.5" UserLevel="moderate"
> XRefLabel="JFoo.methodBar.example123">
> <para>preceeding blurb</para>
> <ProgramListing ID="123">
> ... content of example column ...
> <ProgramListing>
> <para>trailing blurb</para>
> </InformalExample>
>
> - phpDoc's @example file
> <?php
> ... content of example column ...
> ?>
>
> Eclipse help or "Cheat sheets":
> I have no idea :)
>
> - Examples (part 2)
> The content of the "example" column accidently reports a flaw in the
> current formatting + structure in the Wiki pages that no parser would be
> able to catch, so I don't "blame" your code :)
>
yes... haven't done much with the examples and didn't think about this.
I'll look into that... thanks. The actual parser part was pretty easy
though... the file format is pretty standard. The wiki parser isn't
really enough either, because it doesn't know what the positioning of
things means semantically. It knows that ===== stuff ===== is a title,
but in our files the order and position of those things means something
more. I may be wrong on this though (i.e. about what the wiki parser
can do).
Thanks for your feedback and comments.
Hope you are well,
Ian
Ian MacLennan
Here is an update with the following fixes:
No longer requires legacy mode (turned out it was just my xml install
file - the version was 1.1.0 instead of 1.5.0. I'll check the docs for
this because I think I pulled the shell off the wiki.
Actually reads the configured path now.
Ian
Chris Davenport
As promised, though very late, the following is a brief summary of the
main points of this discussion thread, so you don't have to read through
everything.
1. phpDoc outputs to support (at some point)
HTML, CHM, PDF, DocBook XML, wiki, Eclipse help (Ian)
2. We should consider increasing the number of people registered and
able to add comments to the wiki.
3. Idea: Create a list of tutorials/how-to's that we'd like to see
written and when we see somebody in the forums discussing/working with
something similar and having success, ask them to do a short one-off on
how they did it.
4. Once phpDoc tags are more stable we could task someone with
monitoring the SVN codebase, changelog, etc., looking for changes that
affect the docs and creating (doc) tracker artefacts for them. Devs
should be encouraged to always note changes that affect docs in the
changelog and also to assist in keeping phpDoc tags up-to-date as
changes are made.
5. It was suggested that the wiki content should be maintained in the
doc project SVN repository. This makes it easier to edit the wiki as it
can be done offline and SVN has better version control. This was agreed
and Ian was asked to coordinate this.
6. Source documents should, if possible, use semantic tagging. Either
XHTML or DocBook XML. Online editors are not really practical for this
kind of work.
7. Whatever commenting solution we use should not require people to
register with yet another website as this will discourage contributions.
8. Wiki comments should, for the time being at least, be kept separate.
Unless we start to get good quality contributions here there is no need
to consider rolling them into the documentation proper.
9. We need to experiment with writing "tutorial" text in DocBook and
then rendering directly in Joomla! using a stylesheet as well as merging
into the phpDoc output.
10. At some point in the future we will want to backport the work we
have done in the wiki into the DocBlocks. Ian has been working on a
proof of concept script that could be used to, at least partially,
automate the backport.
11. It would be useful to research possible offline DocBook editors, not
necessarily WYSIWYG. Need to consider XHTML + XSLT -> DocBook too. Also
ODT + XSLT -> DocBook?
Thanks again to everyone for a useful discussion. No doubt we will
repeat the exercise again at some point.
Regards,
Chris.
--
Chris Davenport
chris.d...@joomla.org