Can't get TG2, Sprox and Dojo to work using the tutorial

[Kees van den Broek]

Hi,

Tried to setup Sprox (0.6.4) with tw.dojo (0.9.4) to get pagination.
Following the TG2 crud controller tutorial at:
http://www.turbogears.org/2.0/docs/main/Extensions/Crud/index.html?highlight=sprox#using-dojo
I setup __url__, but the table data is still fetched from some other
(autogenerated it seems) URL that doesn't exist:
http://127.0.0.1:6060/categorys.json?start=0&count=20

This is the code: http://pastebin.com/m60837e9d

Did I make an error somewhere?

Thanks,
Kees

[Jamie Bullock]

Hi Kees,

I think a lot of people have come unstuck trying to follow that
tutorial, and in general I think TG2 documentation is very poor. I
have everything working here, but I still don't fully understand
aspects of it like what __url__ actually does.

Anyhow, ranting aside, I found the most useful approach to learning
was to unpick the working examples found in svn here:

svn co http://pythontutorials.googlecode.com/svn/tutorials/moviedemo/trunk
moviedemo

Good luck!

Jamie

[Kees van den Broek]

> I think a lot of people have come unstuck trying to follow that  
> tutorial, and in general I think TG2 documentation is very poor. I  
> have everything working here, but I still don't fully understand  
> aspects of it like what __url__ actually does.

Maybe you could you provide an example?

> Anyhow, ranting aside, I found the most useful approach to learning  
> was to unpick the working examples found in svn here:

Tried to install the examples, hit an error. Reported as a bug...

Thanks,
Kees

[Jamie Bullock]

On 26 Aug 2009, at 18:07, Kees van den Broek wrote:

>
>> I think a lot of people have come unstuck trying to follow that
>> tutorial, and in general I think TG2 documentation is very poor. I
>> have everything working here, but I still don't fully understand
>> aspects of it like what __url__ actually does.
>
> Maybe you could you provide an example?
>

What I mean is that when I installed from svn, the
SproxMovieController 'just worked' I didn't have to do anything special.

Whereas I seem to remember that following the code on the site, or
installing from the tarball didn't work for me, but that was a couple
of months ago, so maybe it's been updated now.

>> Anyhow, ranting aside, I found the most useful approach to learning
>> was to unpick the working examples found in svn here:
>
> Tried to install the examples, hit an error. Reported as a bug...

What was the error?

Jamie

[Kees van den Broek]

> > Tried to install the examples, hit an error. Reported as a bug...
>
> What was the error?

http://code.google.com/p/pythontutorials/issues/detail?id=2

Anyway, I figured out my earlier issue.
I got the CrudRestController mounted as:

categories = CategoryController()

Inside the CategoryController, I used this definition a couple of
times:
__model__ = Category

The Crud controller constructs a JSON data request that is (seemingly)
generated from the lowercase name of the model plus an extra 's'.
So the JSON request becomes categorys.json.
I tried to guide the CrudRestController (as it works with the
RestController) by specifying __url__, but that parameter was ignored
by de CrudRestController.
What 'fixed' it was defining the CrudRestController as:

categorys = CategoryController()

Of course, in the Movie example, this is not an issue, class Movie
becomes movies.

Cheers,
Kees

[Jamie Bullock]

On 26 Aug 2009, at 21:31, Kees van den Broek wrote:

>
>
> The Crud controller constructs a JSON data request that is (seemingly)
> generated from the lowercase name of the model plus an extra 's'.
> So the JSON request becomes categorys.json.
> I tried to guide the CrudRestController (as it works with the
> RestController) by specifying __url__, but that parameter was ignored
> by de CrudRestController.
> What 'fixed' it was defining the CrudRestController as:
>
> categorys = CategoryController()

Ah, that makes sense. It's a real shame that all this 'magic' isn't
documented properly somewhere. It would save people banging their
heads to figure it out.

best,

Jamie

[Kees van den Broek]

> Ah, that makes sense. It's a real shame that all this 'magic' isn't  
> documented properly somewhere. It would save people banging their  
> heads to figure it out.

I've been thinking about this for a while now.

Although the devs are really putting a lot of effort into the
documentation, there's a fundamental problem. Let me try to describe
it.

The reason I've been developing with TG1, and now TG2 is because of
the flexibility it offers. I started with SQLObject. Didn't like it,
switched to SQLAlchemy. Great documentation there. Then I discovered
Elixir, used it for a while in simpler projects. Now I switched back
to SA for more advanced stuff. Learned a lot along the way, gained
some insights, because it showed me different approaches for using the
object model with a SQL backend.

Because of the wide range of interchangeable pieces in TG2. It all
needs to be covered in documentation, and it is, most of the time.
But the problem I often find is the lack of _depth_ in the
documentation. The examples look elegant and easy, they work, those
were the code snippets that made me enthusiastic about TG.
But just when you want to build a reald world application, you need
something a bit more powerful and all you're left with is the TG
source. That's the feeling I get sometimes. I've got no idea if
anybody feels the same way.

What to do about it?
I'd love to see source code of real-world TG deployments. With all its
imperfect workarounds.
I guess it only takes a TG wikipage with these 2 columns:
"project repository", "heavily used components".

I'll provide the first:
http://github.com/kvdb/6pp/tree/master, TGWebServices

Cheers,
Kees

[percious]

> The Crud controller constructs a JSON data request that is (seemingly)
> generated from the lowercase name of the model plus an extra 's'.
> So the JSON request becomes categorys.json.
> I tried to guide the CrudRestController (as it works with the
> RestController) by specifying __url__, but that parameter was ignored
> by de CrudRestController.
> What 'fixed' it was defining the CrudRestController as:
>
> categorys = CategoryController()

CRC doesn't have a __url__ attribute. Sprox's DojoTableBase does. To
make your Category controller have sane URLs you might do something
like:

class CategoryController(CrudRestController):

class table_type(DojoTableBase):
__model__ = Category
__url__ = './categories.json'

I have opened a new ticket to address this issue:
http://trac.turbogears.org/ticket/2353

I've been doing some work on tgext.crud lately while adding mongodb
support, so I will see if I can address this in the next release.

You are invited and also encouraged to open your own tickets when an
issue like this comes up.

cheers.
-chris

[percious]

On Aug 26, 4:06 am, Jamie Bullock <jamie.b.bull...@googlemail.com>
wrote:

> Hi Kees,
>
> I think a lot of people have come unstuck trying to follow that  
> tutorial, and in general I think TG2 documentation is very poor. I  
> have everything working here, but I still don't fully understand  
> aspects of it like what __url__ actually does.

What do you mean "unstuck"? Fyi, that tutorial took about 8 hours to
write. I know it's not super in-depth, but to get any more in-depth
you have to read the code. There really is not a ton of code in CRC,
most of it is in Sprox. Sprox has in-depth documentation, here are
the docs on TableBase that you may have not seen:
http://www.sprox.org/modules/sprox.tablebase.html#sprox.tablebase.TableBase
.

Your post made me realize that I had not expressed the __url__
modifier in the latest Sprox Documentation on DojoTableBase:
http://www.sprox.org/modules/sprox.dojo.tablebase.html

That problem has since been rectified.

cheers.
-chris

[percious]

> Although the devs are really putting a lot of effort into the
> documentation, there's a fundamental problem. Let me try to describe
> it.
>

> ...

>
> Because of the wide range of interchangeable pieces in TG2. It all
> needs to be covered in documentation, and it is, most of the time.
> But the problem I often find is the lack of _depth_ in the
> documentation. The examples look elegant and easy, they work, those
> were the code snippets that made me enthusiastic about TG.
> But just when you want to build a reald world application, you need
> something a bit more powerful and all you're left with is the TG
> source. That's the feeling I get sometimes. I've got no idea if
> anybody feels the same way.

I think everyone on the TG team would like to get more feedback as to
_where_ the depth is needed. We currently have 2 people who have
really dedicated themselves to documenting TG2 (thanks to them!) and
they are very open to suggestions. We are also planning a doc sprint
as soon as the docs get to a place where they are organized enough for
such an activity. I personally think the problem is not depth, but
organization of the docs, because I frequently am told there is
nothing on a subject, only to provide a person a link to more in-depth
information.

For instance, in this case you needed more information about Sprox,
not CRC, and there is a more in-depth tutorial at:
http://www.sprox.org/tutorials/table.html Sprox also has module-level
documentation, so there should be enough information there to get what
you need out of it.

The challenge with tutorials is that to make a good tutorial you need
to focus on one particular topic, and gloss over others. I have been
doing my best to write tutorials for all of the code I write and use
every day, but most tutorials take between 8 and 12 hours to write,
and this work is done in addition to my regular day job(s). In short,
TG needs help in this area. Many of the tickets here:
http://trac.turbogears.org/query?status=new&status=assigned&status=reopened&milestone=2.1
are documentation related.

Also, you might drop into the trunk dev list and check out posts like
this one:
http://groups.google.com/group/turbogears-trunk/browse_thread/thread/4b40c38088761123

Finally, I provide a reasonable amount of _FREE_ live tech support on
the irc channel (#turbogears @ irc.freenode.net) So feel free to drop
in and ask some questions. If I can't answer them, there are usually
a few people in there who can either tell you the answer, or find
someone for you that can.

cheers.
-chris

[Kees van den Broek]

As discussed on IRC, I started collecting issues.

Is this useful?
http://trac.turbogears.org/ticket/2354

Bye,
Kees

[Michael Pedersen]

On Fri, Aug 28, 2009 at 6:18 AM, Kees van den Broek <kee...@gmail.com> wrote:

I've been thinking about this for a while now.

I've been working on this for a while now, actually. Time for my every so often post to the list :)
 

Although the devs are really putting a lot of effort into the
documentation, there's a fundamental problem. Let me try to describe
it.

You're partially right in your description. If you check out http://web.icelus.tzo.com/~marvin/tg2/ and compare that with http://turbogears.org/2.0/docs/index.html you will see major differences appearing right away in the experience.

For reference, when I talk about the docs below, I'll be comparing these two URLs:

• Old Docs: http://turbogears.org/2.0/docs/index.html
• New Docs: http://web.icelus.tzo.com/~marvin/tg2/

The New Docs url is from the repository I've been working with, and will eventually be merged back into the mainline. Basically, new docs will become current when 2.1 comes out. I'm hoping to have most of the problems below identified and eliminated by then.

The docs have, really, a few major problems. In order of importance:

1. Organization (or lack thereof). The current organization makes it very difficult to find something. You start up, read the docs, and then a month later find you need something that you read a month ago. Finding it again is nearly impossible, and this is due to the way the docs are organized. The new version of the docs will have significantly better organization, as you can already see.
2. Unknown holes in the docs. When looking at the current docs, it's easy to find several items labeled as "Write this up" or "XXX: Get this done" or "TODO: Write me" type things. Sphinx, the tool that produces the HTML docs, provides support for a todo list. During my reorganization, I'm going over every single file looking for these holes, and making actual todo list entries. If you go to http://web.icelus.tzo.com/~marvin/tg2/todo.html you will see the current full todo list.
   
   Once we have all the holes identified, we'll be organizing a doc sprint to plug those holes. Hopefully, with only one or two of those (plus some concerted effort by yours truly), we can get the todo list cleared out entirely within a month or so.
3. Bad code in the docs. I've seen it in there already, and fixing all of it right now is not a quick process. The reorganization is helping me to spot the issues, but we actually have places where shell commands are listed as code blocks, as are HTML fragments, and the like. All of these have to be cleaned up.
   
   Basically, once 1 and 2 are done, this is the next task. The goal here will be to turn all code fragments into items that pass automated testing, and then keep them there.
4. Finally, inconsistent templates. Some pages list a status, or an intended audience, and others dive right into the topic at hand. Once 1, 2, and 3 are done, I'll be introducing actual templates, deployed using Paster, to bring the docs into a consistent look and feel regardless of page.

As you can see, there's a lot of work to be done on the templates. I'm hoping to get all of the above done by sometime in January. Earlier if possible, but I wouldn't bet on it. We have about 130 files in the docs tree, and each of them has to be vetted at each of those steps.

Mind you, I will gladly take any/all help from folks on this. Check out http://web.icelus.tzo.com/~marvin/tg2/building_docs.html for a complete step by step to checking out and updating the docs tree, and getting your updates to me for inclusion in the final result.

All of that said, I'll now address your specific issue:

Because of the wide range of interchangeable pieces in TG2. It all
needs to be covered in documentation, and it is, most of the time.
But the problem I often find is the lack of _depth_ in the
documentation. The examples look elegant and easy, they work, those
were the code snippets that made me enthusiastic about TG.
But just when you want to build a reald world application, you need
something a bit more powerful and all you're left with is the TG
source. That's the feeling I get sometimes. I've got no idea if
anybody feels the same way.

Actually, I have a very specific place for exactly what you're describing: http://web.icelus.tzo.com/~marvin/tg2/testimonials.html

It's something I had already identified as lacking, and would help people out. That area is meant for people to provide in depth pieces on something they've done.

I also have another area to look at: http://web.icelus.tzo.com/~marvin/tg2/recipesandfaq.html

That area is for shorter pieces, specifically targeted at more basic questions like "How do I connect to multiple databases?"

Between the two of them, that covers much of what you were thinking of doing. If you're up for it, I'd love to have you contribute your ideas and docs to whichever spot you feel is more appropriate.

--

Michael J. Pedersen
My IM IDs: Jabber/pede...@icelus.tzo.com, ICQ/103345809, AIM/pedermj022171
         Yahoo/pedermj2002, MSN/pederm...@hotmail.com