KT> Oh, gosh, no, theory is way out of my league.

Theory is a crutch for too-smart folks who cannot hear the problem
describing its own solution. The noise of their own intellects deafens
them.

Just kidding, I wish I was smart enough to follow the deeper comp sci
stuff, but I am not. The point of what I wrote is that I think it says
something pretty deep about theory that I am too dumb to understand it
but do great without it.

My theory is that theory is cool, but better answers come from listening
to the problem.

--

re pure ODB: We are using AStore atop ObjectStore, having a ball with
it.

Versant is another ODB vendor offering stuff for Java.

--

  The theory is the result of listening to the problem.  When the theory
  acquires a life of its own because some people like it more than the real
  world, all kinds of uninspiring, uninteresting things happen, so the key
  is _both_ to listen to the problem and to study the theory.  But always
  remember that just as much theory is bunk as there are buggy solutions.
  There is nothing more wrong with "theory" than "solutions" -- both their
  quality and their applicability are orthogonal to their existence.

///
--
  In a fight against something, the fight has value, victory has none.
  In a fight for something, the fight is a loss, victory merely relief.

Immediately upon choosing "OODB," that means that you have chosen to
use a particular language, and have to _start_ the process by defining
the base data-storing C++/Java classes.

If that's not a "lockstep" process, I don't know what is.

SQL/RDBMS at least has the merit of being separate enough from any
specific programming language that code and data don't merge
_completely_ invisibly.  

It's ugly in different ways.

CORBA is another useful thing to look at, despite not being a data
_storage_ system; it's a characteristic scheme for _communications_
that immediately points to the same sorts of issues.  

Another endpoint there would be to consider that the original
implementation of M$ COM was essentially a light serialization of M$
Visual C++ objects; it was somewhat convenient for communicating
between "components," assuming that all code would be written using
Visual C++.

CORBA is pretty ugly because you're always having to do work to push
arguments into the CORBA marshalling scheme; that's ugly more or less
the same way that SQL is.  COM was "nicer," assuming that you're happy
with living with One True Language, and in _starting_ by tying down
how data gets moved around to however VC++ "marshalls" it given its
druthers.

More recent COM releases have moved towards being more like CORBA;
other schemes like SOAP may use XML underneath but are otherwise much
the same, requiring doing a bunch of work to transform "local" data
into some "marshalled" form.

Stepping back to Lisp...

On the one hand, Lisp ought to be a somewhat nicer language to do
CORBA-like stuff with, what with having a bit more reflectivity than
C/C++/Java.  That bodes well for "playing well" with SQL RDBMSes.
You're still suffering from the need to do a lot of "marshalling;" no
real way around that problem.

On the other hand, you could build a nice "OODB" system atop Lisp,
too; it would be equally likely suffer from roughly the same problems
as the other OODB systems, unfortunately.

ok fixed, felt penned qwerty letters on top of keys and changed back to
qwerty layout for now. seems my mind is fubarred for now...

But one thing is certain: RDB cannot directly store OO-based data,
meaning an OO app has to transform the data to read or write it. That
alone proves RDB is Deeply Wrong. It had its day, but it just does not
scale semantically.

--

http://linux.rice.edu/~rahul/hbaker/letters/CACM-RelationalDatabases....

covered similar ground 11 years ago.

Native XML is of stunningly little use.  The 'theory' behind it is back in
1960's hierachical databases, although after a few more years you might
get to the point of '70s network databases, but I doubt it.

Of course, 'XML' is now an umbrella term for a dizzying array of
ever-bifurcating standards and proposals, so saying a database is 'native
XML' doesn't tell me anything except that I'm in trouble.  Concurrent to
this, XML is changing fast enough to doom any projects I write with a
'native XML' DB to being rewritten every couple years.

: Nah, ODB is a superset of RDB.

Prove it.  I'm entirely open to the idea.

:> designing your datastore in lockstep with
:> your application,

: to a degree, yes. but if one likes OO, ODB only makes sense. And RDB
: looks like a dinosaur.

If you are designing a single-user backend to a single application, then
yes, OODBs are an excellent solution to persisting data in the same format
your single application will use it in.

If, on the other hand, you are planning on running a large datasource to
several programs written in different languages, with many users using in
simultaneously, then a normalized relational database with a common
language-independent query form like SQL may be more of what you need to
allow interoperation and avoid corruption.

: it supports variable structures (a weak I consider of
: the relational model).

That's not a failure of the relational model, it's the failure of SQL
databases as currently implemented.  Postgres is working out some of the
issues with 'domains', as objects are called in RDBMS-land; but most
vendors haven't exposed enough functionality to allow users to define new
object types and derive all the comparison functions you need to use them
conditionally in queries.

: it uses a prototyping object system (actually my
: version of a prototyping object system).

What do you mean by that?

My guess: the person meant a "prototype" based object system, a la Self,
Newtonscript, or Javascript. To wit, there are no classes (rather, an
object isn't primarily an instance of a class). The primary mechanism for
creating objects is clone rather than new (and instantiation).

There's a lot of neat things you can do with prototype object systems, and
for a lot of tasks they're *very* nice to use.

Really all you need to write is a B+-tree implementation where the
OODBMS guarantees that the nodes are memory page (or disk block)
sized. Actually, some OODBMSes come with indicies already. The hardest
part would probably be writing the language interface for the
relational operators.

: You can write a relational engine using an object system. QED.

: Really all you need to write is a B+-tree implementation where the
: OODBMS guarantees that the nodes are memory page (or disk block)
: sized. Actually, some OODBMSes come with indicies already. The hardest
: part would probably be writing the language interface for the
: relational operators.

I would just call it a prototyping object system except that some details I
consider my own (as initially I had thought it up and discovered later that
it allready existed). my design involves "specific"/"inspecific" objects,
which have slightly different behaviors (though it could probably get along
without the difference, but I figure the difference will make things a
little easier to optimize...).

I will describe my general idea for my objects as applied to a db:
inspecific objects can not be modified;
objects (specific) may be placed in sets;
the structure of objects in a set is restriced by compatibility objects
(inspecific) or predicates;
inversly the compatibility objects/predicates are to hold for all objects
in a set;
objects in a set are referred to by their fields or by keys (not written:
or on compatibility with a query predicate);
soft references will consist of a set reference and a query predicate;
sets may be placed in other sets however the parent set may restrict if it
may contain sets (and the structure of the sets);
sets will be considered objects;
hard references may be disallowed for interobject references.

worth noting: objects are considered as residing in sets, but not part of
the sets; inversly, fields in objects are considered parts of the objects.

from what little I have read I think my design will be a little different
than self (I have not looked at newtonscript or javascript).

at some point I will have to write down how I imagine the object system.

if anyone knows more about what I am talking about than I do then tips
might be helpfull...

based on what I have heard, RDb is an excuse for two ignorances:

(1) I do not know what language to use. (answer: duhhhhhhh, lisp!!!)

(2) I do not know what index I will want tomorrow at 3PM (answer: don't
quit your day job.)

--

FWIW, I don't recall *ever* seeing "prototyping object system" before, and
I've read the Self papers, programming Newtons, and studied Javascript (to
my dismay). So, even if "correct", "prototyping" is nonstandard, or,
at least, likely to provoke some confusion.

Plus, you lose symatry with "class based object system". And there's a
problem that a class based object system can be a "prototyping" object
system for a fairly natural reading of the phrase (i.e., Smalltalk may be
a prototyping object system (ick) because it supports, like Lisp, rapid,
flexible prototyping of applications).

I like ODB precisely for its tight linking to the language; it takes the
whole DB thang and says, look, we're just making instances persistent.
To a degree the DB is transparent. That makes development easier, and
nothing is more important. Hardware is cheap, performance can be dealt
with later (as a rule--not agreeing ODBs are slower).

--

: Chya. I guess that is why Oracle is in such a panic to declare they do
: native XML. (Don't tell me I have to go find the link to ORCL for you.)

You can find all the links you want.  I guess that would really impress
me, since I've been using Oracle's XML Java stored procs to pull XML out
of Oracle since 1999.
It's also fairly absurd to try to use "this big company is doing
something" as a definitive data point in favor of 'something'.  Big
Companies have lots and lots of engineers; they can run lots and lots of
projects concurrently.  This lets them use projects for marketing
purposes, but you are a fool if you think that everything else they are
doing just got shelved in favor of the one thing they sold you on.

XML is a structed application interchange format.  It is not an
intra-application storage format -- apps will always specialize their
backends.

:> Of course, 'XML' is now an umbrella term for a dizzying array of
:> ever-bifurcating standards and proposals

: oh shit, don't tell me there is a new technology which might be going
: thru some polishing!!! run way! run away!!!...the difference between an
: adventure and a disaster is the attitude you take into it

And yet, that isn't a unmitigated win.  So it's hard to argue with fervor
in favor of a tool used where it is inappropriate just on the grounds of
"maybe it will get better in another decade".

: your projects--if they cannot be ported from RDB or XML to ISAM or GKW
: in a week--get thrown out anyway every couple of years.

I can't figure out if this is a silly ad hominum or just poor grammar.
Maybe it's the viewpoint of someone used to very small projects.

:> Problem 3 is that 'get next record' in a relational DB means 'add the
:> offset to this pointer and read from the memmap'ed file';

: chya. so you have RDB code at great expense creating the /illusion/ of
: "add the offset". This is a terribly intellectually dishonest assertion.

If you leave fantasyland and learn to think in disk blocks, operations,
and memory allocation, you'll change your mind.

:> with 'native
:> XML' that means 'break out the parser, rip through a few megs of text, and
:> build a tree to walk through'.

: ok, thx for clarifying that you know nothing about native XML. (DMBEI)

Once again, you brush aside a valid point with an ad hominum attack.  Will
you ever bring content to the table?

:>  Good news for hardware manufacturers, not
:> so good news for me.

: uh, aside from the fact that you are wrong, the hardware folks are
: upping their performance so fast that not even Moore's law can keep up.
: How /you/ doin'?

My hardware is fine, thank you.  But I still like to support as many users
per processer as I can.  If you support bloat, more power to you.  By the
way, which platform are you on?  I need to buy some stock.

: based on what I have heard, RDb is an excuse for two ignorances:

: (1) I do not know what language to use. (answer: duhhhhhhh, lisp!!!)

: (2) I do not know what index I will want tomorrow at 3PM (answer: don't
: quit your day job.)

  Would you mind trying to act a bit more intelligent?

| chya. so you have RDB code at great expense creating the /illusion/ of
| "add the offset". This is a terribly intellectually dishonest assertion.
| apologize to c.l.l. imediately. (doing my best erik imitation.)

  Try imitating something within your reach, first, like a dog.

| uh, aside from the fact that you are wrong, the hardware folks are upping
| their performance so fast that not even Moore's law can keep up.

  Are you sure you have figured out what Moore's law says?

| How /you/ doin'?

  Less slang, more intelligibility, please.

| based on what I have heard, RDb is an excuse for two ignorances:
|
| (1) I do not know what language to use. (answer: duhhhhhhh, lisp!!!)
|
| (2) I do not know what index I will want tomorrow at 3PM (answer: don't
| quit your day job.)

  So you _have_ no clue.  I hate it when people confirm my suspicions.

  People who think object-orientation is so great, have generally failed to
  grasp the value of data-driven designs despite the serious attempt at
  making such design easier to model, and think solely in terms of code-
  driven designs where their class hierarchies are poor adaptations to
  their incompetent coding styles.  This is extremely depressing, as the
  interminable "software crisis" is a result of code-driven design.  SGML
  and XML were attempts at promoting data-driven design that would produce
  data that was _supposedly_ indepedent of any application.  The result is
  that people who have so little clue they should have attracted one simply
  by the sucking power of vacuum do code-driven designs in XML, which is
  _really_ retarded, and then they need to store their moronically designed
  data in databases, which is, of course, too hard given their braindamaged
  designs, so the relational model does not "work" for them.

///
--
  In a fight against something, the fight has value, victory has none.
  In a fight for something, the fight is a loss, victory merely relief.

What about the new XMLType in 9i? I saw this InfoWorld story:

http://www.infoworld.com/articles/hn/xml/01/11/30/011130hnoraclexdb.xml

<start>
"Oracle to boost XML, Web services support

By Mark Leon and Tom Sullivan
November 30, 2001 6:18 am PT

ORACLE SAYS IT will raise the bar in XML database and Web services
support next week at the Oracle OpenWorld conference in San Francisco.

The buzz surrounds Oracle's XDB project that the company reports will
utilize the object database functions it built in the mid-90s that allow
users to store XML as an object and index the tags, then stored it in a
compressed table.

...

As for the advantages of XDB, Burton explained that XML documents are
stored "natively" but will not require special treatment. "You can use
SQL, and don't need to master the new XML query languages like XQuery,"
Burton said.

"The object technology that makes this possible was a hammer looking for
a nail," Burton said, "and XML is suddenly the biggest nail on the
planet. If XML had not come along, you could argue that a lot of our
object development efforts were wasted effort."

"With XDB," Burton continued, "the overhead of managing XML, parsing for
example, largely disappears."
<end>

Agreed, Oracle is not the oracle on anything, but I think it's a fun
data point. The article actually goes on quite a bit more, mentioning
IBM, Tamino and XML Global (native XML). No mention of Excelon, tho.

Did your group look at XIS at all? The 9i XMLType? Of the latter, at
http://oracle.oreilly.com/news/oracleessential_0701.html they say:
"XMLType eliminates the need to parse the documents coming into and out
of the database."

--

The sad thing about XML is that it's a pretty trivial uglification of lisp
sexps.  Sure, they approached the problem space from a different direction,
but, as usual, they've reinvented-the-wheel, poorly.

"Oooh no, lisp has too many parentheses, let's represent our
pretty-bog-standard tree structures with incredibly verbose syntax
featuring redundant* close tags and angle brackets everywhere... I think
that's much better than parens, don't you?"

"No."

*XML's nesting rules mean that any well-formed xml fragment does not
overlap tags (no <x>hello <y>there</x>, world</y>). Thus, using XML gains
you essentially nothing over just using sexps for the same data (unless
you're stuck talking to M$'s or other sundry XML-weenies'  systems).  While
it may sometimes look/feel like you're marking up a stream of text when you
use XML, you're not, really  XML is just building humdrum tree structures.

(if XML were a proper markup language, you'd be able to overlap tags and
all sorts of things - markup should notionally be a layer of "highlighter
pen" on top of the text stream - the XMLers started forgetting that, and
thinking their tags were more important than the text, and manage, through
the woeful DOM, to thoroughly mix up their "markup" (ha!) and text...).

So, XML is  an annoying way of writing down data structures that lispers
have been dealing with (usually much more sensibly) since time immemorial.  
  That's why the average lisp programmer, confronted with XML,
tends to think "this seems a bit silly", even if he can't put his finger on
why.

In may 10 or 20 years time, the Java and XML crowd will probably have
reinvented most of Lisp, badly...

broken-record-mode:
See http://ssax.sourceforge.net for a nice  XML-to-scheme specification.

<SETQ> X <LAMBDA> <ARG>A B</ARG> <PLUS>A B</PLUS> </LAMBDA> </SETQ>

Relational DBMSes do lookup by _search_, not by reference. The engine
inside the RDBMS does lookup by reference to implement the lookup by
search. An object-oriented system does lookup by reference as well.