CherryPy Documentation Improvement Planning

[Derek Litz]

In a previous thread CP development came up and documentation quickly became a hot topic.

https://groups.google.com/forum/#!topic/cherrypy-users/ZwQvlbwxIEI

Some high level goals were stated there, reasons why, and some debate about CP3 vs CP4 efforts, so it is recommended reading.

This threads goal is to test the waters and see if the community can come up with a plan to make a leap with the current CP3 documentation.

Skeleton:

PlanName (so we can refer to it, use a unique studly case name) 1 (version so we can iterate on it)

           Goals (In order of priority)

           Phases (In order of execution)

Avoid details that will depend more on the people who participate in each phase that should be planned at the outset.  Focus more on the merit of the plan.

----

BlankSlate 1

This plan stemmed from an off the cuff idea of starting from a blank slate.  This avoids several complexities of trying to improve existing documentation in place.  The downside is we may end up with documentation that no one is familiar with (which is OK if it is usable).

Goals

1. Collect and reorganize content created over the years in a more accessible manner for jogging one's memory and for learning the basics for new users.
2. Remove barriers to finding the correct documentation.  Is it on the current wiki, older wiki, the current docs, older docs, stack overflow, or the source code?  Maybe its been moved or isn't even easy to find on the internet anymore...
3. Fill in missing content, rewrite hard to understand content, and add additional more targetetted content areas (tutorials for specific stacks, etc.).  Basically just address community demand at this point.

Phases

1. Design, Organization, and Layout.

2. Content collection and reorganization

3. Content evaluation and rewrite

4. Institutionalize an iterative documentation content improvement plan (much like django has managed to do) or make high quality documentation an important part of the core development process.  Some combination of both maybe, since there are disadvantages and advantages to both approaches.

----

Thread Etiquette and Focus:

• Focus on high level goals and phases that will allow us to improve the current situation effectively (completely different goals and phases are welcome)
  
• If we are iterating on the basics of another plan please just version it
• Versions do not indicate a better more improved plan, just a similar plan idea
• Everyone should feel free to iterate on another persons plan

Let's focus on detailed discussion that will help us decide whether or not a plan is feasible and the things that differentiate plans from each other significantly.  Use the plans name and version number when talking about plans.

Ok, let's see if this gets us somewhere :)

[Robert Brewer]

Derek Litz wrote:
> 1. Design, Organization, and Layout.

Since I suck at all the process-y parts of that plan, I'll just chime in on the design bits:

I've found it very helpful over the last few years at work to separate docs into four separate products based on the different target audiences:

* Architecture: why CherryPy is designed the way it is
* Developer's Guide: how to contribute to CherryPy, and under what design constraints
* User's Guide: how an application should use CherryPy, including tutorials, API references, etc.
* Administrator's Guide: how to deploy and maintain an application that uses CherryPy

It certainly feels like more work in the beginning, but the structure pays off quickly. How to integrate with nginx does not then bleed over into how to resolve URL's. For example, our docs on Tools run the gamut from architect to CherryPy developer to CherryPy extension developer to application developer. Three different docs would allow the beginning or expert user to simply use a Tool without having to know how to build a Tool. It could then link to how to make a new Tool in a different doc, and that could then link to why Tools are built the way they are.

One reality of the Python web world is that so many people are the developer and administrator (and sometimes even user) rolled into one, and that has made CherryPy's docs suffer over the years. Separating these functions allows each one to be much more focused and easy to grasp.


Robert Brewer
fuma...@aminus.org

[Sylvain Hellegouarch]

Hi Robert,

Back when I was writing the book, Rémi had made the same suggestion. Indeed, how to deploy CherryPy is one of the most common questions we have on IRC so it's it's important our doc cover it well. I believe the current doc tries to follow that structure but, even I get lost in it. So I like your idea but we'll have to be careful in how we organise the content indeed.

--
- Sylvain
http://www.defuze.org
http://twitter.com/lawouach

[Sylvain Hellegouarch]

Derek, All,

Venues

I will second Bob's feeling towards process... With that said, thank you for getting the ball rolling. Mostly, I with the idea of reviewing what we have at hand.

I don't know how much the documentation is used but here are a few other regular venues:

* IRC. Our channel is quiet most of the time but it has had a steady number of people in it for years (by the way the channel will be 10 years old in November): between 30 to 40. Everytime Robert made a session at pycon, we had a few new visitors.

* This mailing-list. We used to have a developers mailing-list but I think it died sometime ago. This is by far the best place to ask for the feedback from the community. I can tell many people are used to CherryPy and happy to help out. Problem is, a mailing-list is poor place to document stuff on its own (unless it has all the right keywords so that search engines can point to it).

* Stack Overflow [1]: I try to help there, others do way better than me. I have the feeling, questions there are usually from people who want a snippet to copy/paste from. They are mostly consumers in my mind. Still, lots of interesting questions and having a good presence over there will give the feeling the framework is actually alive and kicking. Nonetheless, I wish responses could actually be leading to the documentation.

Two sources are used for examples:

* The source code and its test suite

* I believe the few recipes I have written [2] are used a little too.

Twitter isn't a place where people seem to look for help. But it's a rather sweet place to learn about projects using CherryPy. As reminder, we have an account: @cherrypy. 

I don't know any other common places. Anyone else?

So overall, the best path, I believe, would be that everytime we answer a question, we can point to the documentation. So that we'd push people going there.

Structure and what's in it? Or rephrased, who are we talking to?

This is where I believe the current structure (which Robert's comment echoes) is not that stupid. Indeed, I tend to think that we can't solely talk to CherryPy users. We need to tackle how to deploy it, how to log, how to integrate it with other frameworks, how to tune it, how to test it, how to contribute and obviously explain its core design concepts. Don't also forget the documentation is the best marketing tool we could have.

I really believe Flask [3] is a good starting point. However, I don't like the fact that the page is sooooo long... it makes my head spin. I like the topics covered but I would need them to be organised differently if we were to follow it.

Maybe we could point at a few other doc projects to discuss around them? 

For instance:

http://tangelo.readthedocs.org/en/latest/

http://developer.android.com/training/index.html

Food for thoughts...

[1] http://stackoverflow.com/search?q=cherrypy

[2] https://bitbucket.org/Lawouach/cherrypy-recipes/src  and  https://bitbucket.org/Lawouach/twiseless/src

[3] http://flask.pocoo.org/docs/
-- 

[Derek Litz]

> I will second Bob's feeling towards process...

And what feelings are those? ;) I know what you mean.  Benefits of process are hard to grok, and when they are hard to grok, no one wants to do it. So its pointless... and it just takes the fun out of leaping without thinking.

But, if that's process, I may be inclined to agree with Dijkstra that Software Engineering is "The Doomed Discipline".

Those are just manageable units so that leadership doesn't have to worry about an exorbitant amount of complexity (i.e. impossible to manage).  It has lot more do with project management than process. The "process" of completing each needs to be tailored to the actual people that are going to do the work for each phase.  The golden rule is always "Planning is everything, the plan is nothing.", anyways.  In other words, if you can't think on your feet your doomed and if you can't plan ahead your doomed, so you need to do both.

> I really believe Flask [3] is a good starting point. However, I don't like the fact that the page is sooooo long... it makes my head spin. I like the topics covered but I would need them to be organised differently if we were to follow it.

IMO the long pages that flask has are great design. It makes stuff very findable with text searches in web browsers. Honestly, all they did was copy the Python docs (I try to copy almost everything the Python core team does, they are really good at what they do).  Sure the outline on flask is a bit too long for beginners, I'll give you that, so they'd benefit from a better design.  The real benefit of their design is they don't need to be good designers to have something that is usable.

Your post speaks abounds about why CherryPy is easy it learn and in wide use, but people still complain about the docs.  The first time I was learning CherryPy it took a bit to comprehend everything.  Now I just want to find and skim the documentation I read before and I can't easily do so (it's literally all over the place, and changing in strange ways). I don't have that problem with something as complex as SQLAlchemy.  Their docs are at least one step up from the simplicity of flask's.

> I've found it very helpful over the last few years at work to separate docs into four separate products based on the different target audiences.

I agree that organizing things based on users perspectives could be a good idea for CherryPy, but what I really want is succinct reference that is indexed, outlined, and organized in one place for a web developer.  i.e. So when the developer goes, "Oh, shit, how should I do that?" Then they can go to a page and say, "Oh. Here are my options."  That being said all the perspectives have merit, but one is way more important than the others IMHO.

Why not just start over with something simple like Flask and just go from there?

[Sylvain Hellegouarch]

Why not just start over with something simple like Flask and just go from there?

That suits me just well really :)

[Milos Gradjanin]

Hi all, I'm going through the official tutorial and as a user I can give you a suggestion about what I'd like to see in a documentation, but before that I'd like to report a typo in the "Creating RESTful API chapter":

In the PUT section, example, I think there should be

song = songs[id]
instead of
song = songs['id']

Also, in the part
"Try sending some PUT HTTP requests to 127.0.0.1:8080/api/songs/3 via cURL, 
and check the result by requesting 127.0.0.1:8080/api/songs/4 in your browser:"
it should be "check the result by requesting 127.0.0.1:8080/api/songs/3 in your browser:", right?

I'm a still a newbie to web development and there's one thing I like to see in tutorials, documentations... it's the examples.
That's why I like the "Creating a RESTful API" chapter. It guides me in the direction of creating what I actually want and that would be a simple website with a database. So, that's another example (with the database), maybe it's already in the tutorial, I haven't read through it all, yet. So pretty much examples to everyday stuff are what make it easier for me, I learn through it.
Explanations as to which function serves which purpose in the framework and the HTTP are very useful, too. This tutorial is good in teaching these down to earth stuff.

Milos

[Sylvain Hellegouarch]

Thanks.

I'll try to fix the typos quickly. I've started a bunch of docs on a fork, I will commit this week a first draft.

--
You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cherrypy-user...@googlegroups.com.
To post to this group, send email to cherryp...@googlegroups.com.
Visit this group at http://groups.google.com/group/cherrypy-users.
For more options, visit https://groups.google.com/groups/opt_out.

[Milos Gradjanin]

Somebody probably mentioned this already, there are links on the web (search results, stackoverflow) towards something in the docs, but the links lead to nowhere, as if the documentation has moved (I guess that's the case). If possible, a simple redirection to the new docs home page is good enough.
The docs pages are pretty cool, by the way.

[Sylvain Hellegouarch]

Hi guys,

I've started working on a rework of the docs and I pushed it to RTD: http://cherrypydocrework.readthedocs.org/

It's far from being complete obviously and there is so much work to be done but I hope it'll provide a basis to discuss.

Cheers,

On 21 February 2014 10:47, Milos Gradjanin <milo...@gmail.com> wrote:

Somebody probably mentioned this already, there are links on the web (search results, stackoverflow) towards something in the docs, but the links lead to nowhere, as if the documentation has moved (I guess that's the case). If possible, a simple redirection to the new docs home page is good enough.
The docs pages are pretty cool, by the way.

--

You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cherrypy-user...@googlegroups.com.
To post to this group, send email to cherryp...@googlegroups.com.
Visit this group at http://groups.google.com/group/cherrypy-users.
For more options, visit https://groups.google.com/groups/opt_out.

[Milos Gradjanin]

This looks good, Sylvian.

One thing I find in the current documentation really useful are references, links, to some HTTP security subjects. Like a link to wikipedia, i.e.

Looking forward to the tutorials and examples.

Milos

[Sylvain Hellegouarch]

Hi there,

Thanks Milos :)

I've added a couple more tutorials. I'm almost done with them. I will then move on to more advanced features.

Please ave al ook again guys (make sure to hard refresh): http://cherrypydocrework.readthedocs.org/

Thanks,

--

You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cherrypy-user...@googlegroups.com.
To post to this group, send email to cherryp...@googlegroups.com.
Visit this group at http://groups.google.com/group/cherrypy-users.
For more options, visit https://groups.google.com/groups/opt_out.

[Milos Gradjanin]

I like the explanation about sqlite and the threads issue. I know how to manipulate an sqlite database with python, but I didn't know about this issue. Are there examples with other databases like postgresql, which work well for this purpose? I am searching the web in the meantime...

[Michiel Overtoom]

On Mar 3, 2014, at 11:33, Milos Gradjanin wrote:

> I like the explanation about sqlite and the threads issue. I know how to manipulate an sqlite database with python, but I didn't know about this issue. Are there examples with other databases like postgresql, which work well for this purpose?

I use MySQL in a CherryPy app, with the 'oursql' database interface (DBAPI2.0 compliant). However, oursql relies on libmysql, and libmysql is not threadsafe. I solve that issue with DBUtils (DBUtils.PersistentDB to be more precise). Works like a charm, for years already.

Greetings,

--
Veel spullen zijn tegenwoordig waardeloos, maar toch hecht men er veel waarde aan. Mensen hebben vaak teveel spullen. Hoe komt dat en wat doe je ertegen? - een essay van Paul Graham, op http://www.michielovertoom.com/articles/paul-graham-spullen

"Een prima telefoon met draaischijf weggooien? Die zou ik nog wel eens nodig kunnen hebben!"

[Sylvain Hellegouarch]

These kind of "recipes" will be part of a specific section in the doc I've started writing. I'd be pleased if, at some point, you can suggest some code on the subject :)

Something along those other recipes : https://bitbucket.org/Lawouach/cherrypy-recipes/src

Thanks,

--
You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cherrypy-user...@googlegroups.com.
To post to this group, send email to cherryp...@googlegroups.com.
Visit this group at http://groups.google.com/group/cherrypy-users.
For more options, visit https://groups.google.com/groups/opt_out.

[Derek Litz]

I'm going to focus on critiques :)

1. The left nav is duplicating the contents area.  This seems to be a common anti-pattern on other pages too (duplicate outline in the content area). If we want an outline in the content area it should be just an outline that links to sub pages with more dedicated content.
2. The left nav scrolls past the top of the page and becomes blank making it not so useful
3. The left nav could be better utilized to add additional navigation features as well.

For the above:

• The Python Language Reference is a good example of left nav usage, content, and usability.
  
• A good complicated sub page is Data Model
  

3. It seems to be a goal to show off CherryPy, but we also need a page that is a good entry point for more learned users that is more focused on quickly providing the big picture of existing documentation. What you have is a great start for the former and I see no reason not to go with what you have started.  Based on what you have I think the dedicated areas are going to end up being:

• Quick examples of features and tutorials. In this regard the landing page is pretty close already, it just needs trimming of things that are too complicated for quick example and are glossed over as a result, and perhaps removal of some of the things that seem redundant in light of other content on the page. Some reorganization so related areas are close together seems like a good idea too. Perhaps a "there's more!" section for things are a bit too advanced for a beginner just getting something running.
  
• A more in depth documentation outline page like the The Python Language Reference where we can start building up more advanced documentation.
• All the great architecture diagrams should be shown off somewhere as well. Both dedicated architecture pages and incorporating them with advanced documentation seems like good idea.
• I'm sure there's more that could be added (FAQ, Contributing to CherryPy, etc.).
  

4. There is extensive Help documentation contained in the doc strings of most objects which contains very useful information and is probably some of the best documentation. I remember there being some documentation on the bitbucket too! A couple goals should be to:

Reduce and simplify the help documentation so it acts more like a quick reference for those with experience, and move the more extensive documentation (it's a good stop gap as-is so I don't recommend removing it without awesome docs in place). The help doc strings can certainly serve as a basis for much of the advanced documentation the fixing the doc strings to be a quick reference should be a secondary concern.

Consolidate the more advanced documentation from other places as well.

A great example is the BIFs from python docs. Comparing what help outputs to what's in the documentation gives some good ideas of the kind of content to keep in help and the kind to take out and put in the documentation.

And then as

Thanks much for taking this on Sylvain! (I hope to do more then criticize at some point ;).

[Milos Gradjanin]

"A more in depth documentation outline page like the The Python Language Reference where we can start building up more advanced documentation."

I'd like this, too. At the moment I am finding the Trac pages a useful resource.

[Sylvain Hellegouarch]

Hi Derek,

Regarding the left menu, it's indeed not what I was aiming for. I focused on the main content and wrote everything into a single file. Sphinx ain't that clever, and since I haven't declared a hidden toc, it uses the whole damn content structure. Eventually, I will create a much tighter more useful menu bar.

The advanced section is on my todo list but I lacked time recently.

Cheers for the feedback,

For more options, visit https://groups.google.com/d/optout.

[Sylvain Hellegouarch]

Hi guys,

It's been a while but I reworked the left menu of the doc I started writing:

http://cherrypydocrework.readthedocs.org/

If you guys could give me your thoughts, it'd be much appreciated.

By the way, I reused some sections from the existing doc instead of reiventing the wheel.

Cheers,

[Derek Litz]

I like the behavior in the case I can view the entire left nav content.  It sticks there and I can see all the navigable parts of the section.

However, if I cannot see the entire left nav it scrolls with the content and quickly disappears.  You will want to copy the python docs on this one (I believe the link is above)

1. When you scroll down the main content the left nav scrolls to the bottom with it and sticks once there's no more content the the nav.  After this the main content scrolls normally.

2. When you scroll up main content the left nav would scroll back up to the top and sticks once at the top of the left nav.

You can reproduce this easily with the larger tutorials section by reducing window height.

Otherwise it looks good and works. The previous and next buttons at the bottom content seem like a good enough convenience solution, especially in light of a usable left nav. 🍺🍺

[Sylvain Hellegouarch]

Hi Derek,

Thanks for the feedback. It seems the RTD theme I'm using ought to support sticky sidebar but something is not working right. I'll have a look later on.

I need to add more content as well.

[Ralph Heinkel]

Hi Sylvain,

I've briefly looked over it - I think the 'basics' section is really
well done - many things now get much clearer especially for beginners!

Thanks for putting so much effort into it!

ciao ciao

Ralph

On 03.04.14 22:36, Sylvain Hellegouarch wrote:
> Hi guys,
>

[Sylvain Hellegouarch]

Hi Ralph,

Thank you for the encouraging words. I'm trying to make time to finish it so that we can decide if we move to that doc instead of the current one.

--
You received this message because you are subscribed to the Google Groups "cherrypy-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cherrypy-users+unsubscribe@googlegroups.com.
To post to this group, send email to cherrypy-users@googlegroups.com.

Visit this group at http://groups.google.com/group/cherrypy-users.
For more options, visit https://groups.google.com/d/optout.

[Sylvain Hellegouarch]

Hi guys,

Again, it's been a while but I added a section on plugins and tools.

http://cherrypydocrework.readthedocs.org/extend.html

Hope you guys like it.

[Michiel Overtoom]

On Apr 27, 2014, at 17:09, Sylvain Hellegouarch wrote:

> Again, it's been a while but I added a section on plugins and tools.
> http://cherrypydocrework.readthedocs.org/extend.html

A very nice read, indeed!

I noticed a few misspelled words: "subscirbers", "passas", "application’s lifecycle", "subscirbing".

(The TimingTool you describe, also inspires me to make some kind of tool which records the number of SQL queries an exposed method makes, and how long they take).

Greetings,

--
The decline of interest in sciences is directly paralleled by the average person abandoning responsibilities in order to seek "rights", and disdaining real knowledge and study in preference to entertainment, pleasure and ignorance.

[Sylvain Hellegouarch]

On 27 April 2014 22:42, Michiel Overtoom <mot...@xs4all.nl> wrote:

On Apr 27, 2014, at 17:09, Sylvain Hellegouarch wrote:

> Again, it's been a while but I added a section on plugins and tools.
> http://cherrypydocrework.readthedocs.org/extend.html

A very nice read, indeed!

I noticed a few misspelled words: "subscirbers", "passas", "application's lifecycle", "subscirbing".

(The TimingTool you describe, also inspires me to make some kind of tool which records the number of SQL queries an exposed method makes, and how long they take).

Greetings,

Thanks for the feedback. I hate those words indeed :p 

I would like to add a page on Contributing to the project as well. By the way, how do people feel about moving to that doc instead of the current existing one?

[Sylvain Hellegouarch]

Hi all,

I've submitted the doc as a PR as I considered featured enough now.

https://bitbucket.org/cherrypy/cherrypy/pull-request/72/documentation-reworking

It contains typos and probably is missing a good lot but hopefully it'll be decent enough for now.

I'm a bit washed out writing it so I hope others will chime in ;)

Cheers,

[Sylvain Hellegouarch]

Hi all

I've submitted my doc modifications to the project as a PR but I would like a little more votes for it before merging it.

Although there is still more work to do on it, I believe it's quite rich already and is ready to replace the existing doc.

Could you guys go through it again and then vote for it (if you accept it) on bitbucket?

http://cherrypydocrework.readthedocs.org

https://bitbucket.org/cherrypy/cherrypy/pull-request/72/documentation-reworking

Thanks folks,

[Michiel Overtoom]

Hi Sylvain,

> Could you guys go through it again

Sure.

I found the latest version of your docs an excellent read, thanks a lot! Nice work. I learnt some new things from it, even after using cherrypy for years.

I proofreaded your new documentation from the "Installation" through "Configure" chapters. I have a few minor remarks, and a few typos:



> This is what ultimately uses to try and locate an appropriate exposed function or method to respond.

This caused a ParseError in my brain ;-) Maybe the sentence better starts as follows?

This is what is used to try and locate ...



> Without going into its details, one strategy is to follow the REST principles edicted by Roy T. Fielding in his thesis.

This links to Fielding's thesis. However, I find this thesis rather academic. IBM's explanation of REST is much more approachable and practical for software developers: http://www.ibm.com/developerworks/library/ws-restful/index.html Maybe include that link instead of the thesis? (BTW, the thesis is also refererenced from the IBM page).



> The last two lines are simply starting application server.

maybe better: "The last two lines are simply starting the application server."



> Your application may aobviously already use the logging module

typo in "aobviously".



> The cherrypy.config object is a dictionary and the update method merge the passed dictionary into it.

"merge" -> "merges"



> cherrypy.config.update() is not mean to be used to configure the application.

"mean" -> "meant"



> To configure your application, pass a dictionary or a file when you associate ther application to the server.


"ther" -> "their". And maybe "pass in" instead of "pass"?



> Sessions is one of the most common mechanism used by developers

"is" -> "are"



> Using sessions in your applications is done as follow:

"as follow" -> "as follows"



> If the choice is not valid, you can simply set more media-types as follow:

"as follows:"



> You can serve a single file as follow:

"as follows:"



> A WSGI application can be obtained from your application as follow:

"as follows:" ;-)



> Notabley you will learn about sessions expiricy.

"Notabley" -> "Notably", "expiricy" -> "expiration"



> Using a filesystem is a simple not to lose your sessions between reboots.

"a simple way to not lose"



> "If you have several static section to configure"

"static sections"



> "CherryPy has a built-in support for JSON encoding and decoding"

"CherryPy has built-in support for ..."



> CherryPy provides support for two very simple authentications mechanism,

"CherryPy provides support for two very simple authentication mechanisms,"



> This makes the matching of segments with page handler signatures easier and help CherryPy understand

"and helps CherryPy"



> From that point on, the CherryPy ramework mostly steps out of the way

"ramework" -> "framework"



> You will find here a recipe on how integrating them

"a recipe on how to integrate them"



> Web application, like any other kind of code, must be tested.

"Web applications, ..."


Greetings,

--
"If you don't know, the thing to do is not to get scared, but to learn." - Ayn Rand

[Sylvain Hellegouarch]

Hi folks,

I wanted to thank you guys for the feedback. I've pushed the new doc into the tip branch and recreated it on RTD. It may still need a little more love but hopefully this will be useful as it is!

Thanks again,

--
You received this message because you are subscribed to the Google Groups "cherrypy-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cherrypy-user...@googlegroups.com.
To post to this group, send email to cherryp...@googlegroups.com.

Visit this group at http://groups.google.com/group/cherrypy-users.
For more options, visit https://groups.google.com/d/optout.