removal of database format 1 (in favor of format 2)

[Matthew Scott]

There are currently two database formats supported by Schevo 3.1.

Format 1 is used by databases created by Schevo 3.0.

Format 2 is used by default when creating new databases in Schevo 3.1.  It adds the ability to have fields containing more than one entity reference in their value, and by doing so has cleaned up and generalized the code used under the hood to store field values.

As part of adding advanced expression support for find(), I'm removing support for format 1 for the "pros" reasons listed below.

Pros:

- No more maintenance of database format 1 when introducing new features.

- Less complex code base.

- Faster test runs, as the current tests run against both format 1 and format 2 databases.

Cons:

- Schevo 3.1 would no longer be able to open format 1 databases.

- Anyone with existing format 1 databases would need to use a special snapshot of Schevo 3.1 to access them.

Questions for the group:

- Do any of you have format 1 databases in production?  You can check by looking at the value of "db.format".

- If so, is there anything preventing you from upgrading to format 2?

Please note that it is possible to use a specific snapshot of Schevo 3.1 if it becomes necessary to work with format 1.  I'll provide assistance to anyone who finds themselves in this situation.

Also, the removal has not yet occurred; I'll post another message when the 'master' branch no longer supports format 1.

--
Matthew R. Scott

[Matthew Scott]

Clarification:  Schevo 3.1 will still be able to convert from format 1 to format 2, but will no longer be able to open a format 1 database.

On Tue, Sep 22, 2009 at 17:43, Matthew Scott <ma...@11craft.com> wrote:

Cons:

- Anyone with existing format 1 databases would need to use a special snapshot of Schevo 3.1 to access them.

Please note that it is possible to use a specific snapshot of Schevo 3.1 if it becomes necessary to work with format 1.  I'll provide assistance to anyone who finds themselves in this situation.

--
Matthew R. Scott

[Matthew Scott]

'master' branch no longer supports opening databases in format 1.

As noted in the previous message, you can still use "schevo db convert URL" to convert a database from format 1 to format 2.

On Tue, Sep 22, 2009 at 17:43, Matthew Scott <ma...@11craft.com> wrote:

I'll post another message when the 'master' branch no longer supports format 1.

--
Matthew R. Scott

[Etienne Robillard]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Matthew et al --

Some questions about this database format change...

1.) I can't seem to find Schevo "3.1a1" package anywhere, and that
version is being required when I updated SchevoDurus to the "dev" tag.
Is this related to the newer database format change ?

2.) Also, I'm just starting up playing with the new Schevo 3.0 branch.
Would it then be possible to specify either to use "epoll" or "sendfile"
on Linux, somewhere, so that it can be explicitely configured to use one
or another extension ?

I noticed that upon install of latest SchevoDurus branch, py-sendfile is
being required but py-epoll isn't. I therefore had to install that
package manually.

Your help is greatly appreciated! Thanks for your continuing efforts! :)

Regards,

Etienne

erob@localhost:~/ncvs/schevo-3.0$ /opt/bin/git log | head
commit 9220ae68bc94203743e20a0a3fa6dbf96dd28431
Author: Matthew R. Scott <gldn...@gmail.com>
Date: Tue Sep 22 22:11:41 2009 -0700

threadlocal database opener

erob@localhost:~/ncvs/schevo-duruses$ /opt/bin/git log | head
commit 6d57ddf0a6335c20d1c4718da8a7a11c977ec26a
Author: Matthew R. Scott <gldn...@gmail.com>
Date: Tue Sep 22 20:33:24 2009 -0700

No longer supporting format 1 databases

- --
Etienne Robillard <robillar...@gmail.com>
Green Tea Hackers Club <http://gthc.org/>
Blog: <http://gthc.org/blog/>
PGP Fingerprint: 178A BF04 23F0 2BF5 535D 4A57 FD53 FD31 98DC 4E57
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkrGDJgACgkQ/VP9MZjcTlc2twCgwVF3vjYfgi52pLN4Vl3PPLMR
T98AnRjsbGhyQRSO8VPexVMDJIaPCAQN
=MW73
-----END PGP SIGNATURE-----

[Etienne Robillard]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> 1.) I can't seem to find Schevo "3.1a1" package anywhere, and that
> version is being required when I updated SchevoDurus to the "dev" tag.
> Is this related to the newer database format change ?

Ah, well, you can forget that one, as it turns out a lot of mess
was accumulating in the site-packages directory. Removing the old egg
directories and reinstalled Schevo solved this. Sorry for the noise.. :)

Cheers!

Etienne

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkrGD24ACgkQ/VP9MZjcTlfAdwCgmEaVOEQDm5C2ZWGYBSGxLpfj
wYcAoMVwWDZON4TbmaAjK0qFhfdIBaFZ
=+wTy
-----END PGP SIGNATURE-----

[Matthew Scott]

On Fri, Oct 2, 2009 at 07:22, Etienne Robillard <robillar...@gmail.com> wrote:

2.) Also, I'm just starting up playing with the new Schevo 3.0 branch.
Would it then be possible to specify either to use "epoll" or "sendfile"
on Linux, somewhere, so that it can be explicitely configured to use one
or another extension ?

I noticed that upon install of latest SchevoDurus branch, py-sendfile is
being required but py-epoll isn't. I therefore had to install that
package manually.

I have not yet played around much with the details on Linux.

The use of epoll/sendfile is based on the use of cogen, which is the server framework (currently) used by the Durus extended server.

cogen is not necessarily the be-all, end-all solution either.  The server is simple enough that I may even rewrite it using Twisted since it has a longer history of improvements and tested corner cases.  I'm also open to other ideas.  cogen had an API that was comfortable enough, and seemed nice and lightweight so it ended up as the choice.  :)

 

--
Matthew R. Scott