new ownership and license

[Matthew Scott]

Some posts I made tonight that are of interest to those following
Schevo:

http://goldenspud.com/rotr/index.php/2009/04/21/new-ownership-and-license-for-schevo/
http://11craft.com/blog/2009/04/schevo-project-now-owned-managed-by-elevencraft/
http://11craft.com/blog/2009/04/schevo-31-to-be-relicensed-under-the-mit-license/

[AnimusMontus]

Does this mean we get some updated documentaion for the use of Schevo?

[Matthew Scott]

You bet. I won't be making a 3.1 release until there's much clearer
information about how to get started with it. In addition, making
Schevo more accessible to other developers is a need that I have with
my current professional work.

I'd like to ask, since you replied so quickly :) -- what in your
opinion is the one thing (or top three things) as far as documentation
that you wish were there, but found lacking?

On Apr 22, 7:46 am, AnimusMontus <animusmon...@bellsouth.net> wrote:
> Does this mean we get some updated documentaion for the use of Schevo?
>
>
>
> Matthew Scott wrote:
> > Some posts I made tonight that are of interest to those following
> > Schevo:
>

> >http://goldenspud.com/rotr/index.php/2009/04/21/new-ownership-and-lic...
> >http://11craft.com/blog/2009/04/schevo-project-now-owned-managed-by-e...
> >http://11craft.com/blog/2009/04/schevo-31-to-be-relicensed-under-the-...

[AnimusMontus]

Well the first seems to be lacking is how do you use the schema after
you've created it. The documentation gives a pretty good information
about creating the schema, but it doesn't (or I have not seen it) seem
to explain how to use it. I actually had to download someone else's
code and take a look to see how its done.

Second, maybe some documentation on how the backends work (or
implementing one - I don't remember seeing anything about this).

I can't think of a third right now. But I will eventually.

[Matthew Scott]

On Wed, Apr 22, 2009 at 08:21, AnimusMontus <animus...@bellsouth.net> wrote:

Well the first seems to be lacking is how do you use the schema after
you've created it. The documentation gives a pretty good information
about creating the schema, but it doesn't (or I have not seen it) seem
to explain how to use it. I actually had to download someone else's
code and take a look to see how its done.

I'm glad to hear the documentation is reasonable as far as creating a schema.  :)

As far as "what to do next", there are two paths that you can take so far.

The first is to create a single-user desktop app using the SchevoGtk package along with PyGTK 2.x.

Perhaps a more useful approach for most folks nowadays will be to create a web-based app.  There aren't any toolkits that make this easy for Schevo yet, but I'm actually working on one now for a commercial project.  It isn't at the point where it can be unleashed as open source yet, but it is being built in such a way as to make it easy to do so.  I'll post details about it here if/when that happens.

If you were to choose between having a tutorial for building an app using the existing SchevoGtk, or to having an open source web-UI toolkit and tutorial, which would you choose?  (I ask because I want to make sure I prioritize the "free time" I can devote to Schevo to best suit those who are interested in using it)

 

Second, maybe some documentation on how the backends work (or
implementing one - I don't remember seeing anything about this).

The backends are currently very simple and low-level, designed to support a ZODB/Durus style protocol since they are so similar.  Originally Schevo just had a fork of Durus built in (and it still does, as "schevo.store"), but there was interest by a user to support ZODB as well, as well as interest in using the latest non-forked Durus.  So we created the backend architecture and made SchevoDurus the default backend, so that the default is to use the latest Durus.

 

I can't think of a third right now. But I will eventually.

No problem.  :)   Thanks again for your feedback!

 

--
Matthew R. Scott

[AnimusMontus]

How do I open or create a file with the schema once it is designed? I
am not using SchevoGTK and don't plan on using it at all. I will be
using wxPython for the gui. I need to know how to use the schema to
open or create the database.


SchevoZodb uses ZODB to store things on the local machine it is
running on. I would like to convert it to using ZEO instead and would
like to know how the backends are structured and what is needed to do
this. I also would like to see in creating a backend for possibly
another local persistent system.


Also I would like to know more about f.entity_list and f.entity_set.
Do these fields required the objects stored in it to reference the
parent? Does it required a special helper table (i.e. - a table that
points to the parent and child objects)?

[Matthew Scott]

On Wed, May 6, 2009 at 22:00, AnimusMontus <animus...@bellsouth.net> wrote:

How do I open or create a file with the schema once it is designed? I
am not using SchevoGTK and don't plan on using it at all. I will be
using wxPython for the gui. I need to know how to use the schema to
open or create the database.

schevo.database.create:

http://github.com/gldnspud/schevo/blob/feba3e006695c4bf182e7a97e57a035278123fa6/schevo/database.py#L193

schevo.database.open:

http://github.com/gldnspud/schevo/blob/feba3e006695c4bf182e7a97e57a035278123fa6/schevo/database.py#L357

The implementations for the command-line tools "schevo db create" and "schevo shell" might be of assistance.

Example usage of schevo.database.create:

http://github.com/gldnspud/schevo/blob/feba3e006695c4bf182e7a97e57a035278123fa6/schevo/script/db_create.py#L144

Example usage of schevo.database.open:

http://github.com/gldnspud/schevo/blob/feba3e006695c4bf182e7a97e57a035278123fa6/schevo/script/shell.py#L43

Keep in mind that Schevo is not thread-safe by itself, so you need to make sure you use locks:

http://github.com/gldnspud/schevo/blob/feba3e006695c4bf182e7a97e57a035278123fa6/schevo/mt/plugin.py#L16

The usage instructions there could use some updating.  You can do this too, instead of the try/finally that is in there:

    from __future__ import with_statement       # not required in python 2.6

    with db.write_lock():

        tx = db.Foo.t.create(...)

        db.execute(tx)

        # (other things that might modify the database)

 

MROW (multiple-reader, one-writer) locking is hard to get right, and I have doubts as to the 100% effectiveness of the implementation, so right now when I need to do multithreaded stuff I tend to wrap all code blocks that either read from or write to the database in a write lock.

So, it's not perfect.  :)

Perhaps a better solution for multithreading, or even multiprocess, is to have one thread in one process that manages the database, interact with it using message passing, and offer a nice client API that makes it comfortable to use.  But as the main uses of Schevo have so far been either single-threaded GUI apps or low-traffic web apps, I haven't put any effort into such a beast.

 

SchevoZodb uses ZODB to store things on the local machine it is
running on. I would like to convert it to using ZEO instead and would
like to know how the backends are structured and what is needed to do
this. I also would like to see in creating a backend for possibly
another local persistent system.

The backends are a bit of a hack, since all of the storages that Schevo uses follow this pattern, introduced by ZODB and more or less duplicated by Durus:

 - Pickle-based arbitrary object graphs

 - Storage-aware implementations of list, dict, and b-tree

 - All-or-nothing commit and rollback mechanism

To implement a backend for something that did not follow that pattern, such as for sqlite or something filesystem-based, you'd need to re-implement the API exposed by schevo.database2.Database, which contains the "guts" of Schevo:

http://github.com/gldnspud/schevo/blob/feba3e006695c4bf182e7a97e57a035278123fa6/schevo/database2.py#L35

As far as using ZEO with the SchevoZodb backend, it's worth noting that while the SchevoZodb backend, and the storage backend system in general, was created with that eventuality in mind, the person who was originally interested in getting that implemented has not been active with Schevo.  The core developers of Schevo have not actually used the SchevoZodb backend in any production system.

If you would like to try to get it working with ZEO, I'd love to see what your results are and to review any patches.  :)  But I have no plans myself to implement that.

 

Also I would like to know more about f.entity_list and f.entity_set.

Those are awesome.  They were created to simplify one-to-many relationships, and IMHO are some of the more interesting bits of Schevo that set it apart from SQL-oriented modeling.

 

Do these fields required the objects stored in it to reference the
parent?

No, entities placed in entity_list or entity_set fields do not require explicit references to the entity that contains that field.

It is worth noting though, that when you reference one entity from another, Schevo automatically keeps a back-reference.

For example, given this schema...

    class TodoList(E.Entity):

        name = f.string()

        items = f.entity_list('TodoItem', default=[])

    class TodoItem(E.Entity):

        description = f.string()

        done = f.boolean(default=False)

And this data...

    >>> apples = db.execute(db.TodoItem.t.create(description='Apples'))

    >>> broccoli = db.execute(db.TodoItem.t.create(description='Broccoli'))

    >>> grocery_list = db.execute(db.TodoList.t.create(name='Groceries', items=[apples, broccoli]))

Then you can not only get the items from the todolist:

    >>> grocery_list.items == [apples, broccoli]

    True

But you can also get the TodoList entities that refer to each TodoItem:

    >>> apples.m.todo_lists() == [grocery_list]

    True

    >>> broccoli.m.todo_lists() == [grocery_list]

    True

 

Does it required a special helper table (i.e. - a table that
points to the parent and child objects)?

Such constructs used to require an intermediate extent, and you can still use one if you require additional metadata about each link in the relationship, but if it's a typical parent/child relationship then you can just use entity_list or entity_set.

 

--
Matthew R. Scott