Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

How to create a DBMS from scratch?

19 views
Skip to first unread message

Alan Po

unread,
Oct 14, 2002, 11:54:42 PM10/14/02
to
Dear All

When I am a Bachelor student in University, I have learned the structure
inside a DBMS. However, do not know anything about the internal structure,
e.g. the program code of indexing, query and file structure of a database.
Unfortunately, I am now studying Master course in University and my
assignment is to build an index file on XML data and design a query language
on it. But how?

Please help me. Would you give me any information on the web which help me
to learn more about the programme code on the DBMS?

Thanks a lot.

Alan Po

al...@alanpro.com


mountain man

unread,
Oct 15, 2002, 12:40:39 AM10/15/02
to
"Alan Po" <ala...@ust.hk> wrote in message news:aog3gt$k1b$1...@news.ust.hk...


Google "SQL"

--
Farmer Brown
Falls Creek, Australia
http://www.mountainman.com.au/software


Michael Gast

unread,
Oct 15, 2002, 3:46:04 AM10/15/02
to
Hi Alan,

look on firebird.sourceforge.net. Firebird is a free RDBMS and you can
study all sources. In addition, i suggest you to have a look on the book
'Fundamentals of Database Systems' from Ramez Elmasri and Shamkant
Navathe, published by Addison Wesley.

Alan Po schrieb:

--
All emails sent to this address are never read and never will be
answered. Sorry, but until someone cleans up the spam mess, that's the
way it has to be.

E-Mails, die direkt an diese Adresse geschickt werden, lese und
beantworte ich nicht. Ich bedauere diesen Umstand sehr, kenne derzeit
aber keine bessere Möglichkeit, um die Spam-Flut abzustellen.

Mit freundlichen Grüßen / Best Regards
Michael Gast
SEPP MED GmbH

Jan.Hidders

unread,
Oct 15, 2002, 5:02:50 AM10/15/02
to
In article <aog3gt$k1b$1...@news.ust.hk>, Alan Po <ala...@ust.hk> wrote:
>
>When I am a Bachelor student in University, I have learned the structure
>inside a DBMS. However, do not know anything about the internal structure,
>e.g. the program code of indexing, query and file structure of a database.

H. Garcia-Molina, J. D. Ullman, and J. Widom. Database System
Implementation. Prentice Hall, 2000.

>Unfortunately, I am now studying Master course in University and my
>assignment is to build an index file on XML data and design a query
>language on it. But how?

Indexing XML is not just a matter of applying well-known RDBMS techniques
and there is a whole body of literature on the specifics of indexing XML.
(On its own its already a subject for a PhD) For starters I'd look at:

- DataGuides: Enabling Query Formulation and Optimization in Semistructured
Databases (1997), Roy Goldman, Jennifer Widom
http://citeseer.nj.nec.com/126680.html

- Index Structures for Path Expressions (1997), Tova Milo, Dan Suciu
http://citeseer.nj.nec.com/milo97index.html

- Indexing XML Data with ToXin (2001), Flavio Rizzolo, Alberto Mendelzon
http://citeseer.nj.nec.com/rizzolo01indexing.html

The part about "designing" a query language looks a bit strange. There are
already standard query languages out there such as XPath and XQuery so it
would probably be a good idea to take an easy subset of them.

Good luck,

-- Jan Hidders

manuel

unread,
Oct 15, 2002, 5:19:06 AM10/15/02
to

"Michael Gast" <mig...@web.de> wrote in message
news:3DABC7BC...@web.de...

> Hi Alan,
>
> look on firebird.sourceforge.net. Firebird is a free RDBMS and you can
> study all sources. In addition, i suggest you to have a look on the book
> 'Fundamentals of Database Systems' from Ramez Elmasri and Shamkant
> Navathe, published by Addison Wesley.
>

I would also recommend reading 'Transaction processing: concepts and
Techniques'Jim Gray and Andreas Reuter. it describes the basic concepts of
tp monitors but has quite a complete chapter on indexing and locking.

Robert Sundström

unread,
Oct 17, 2002, 8:25:17 AM10/17/02
to
In article <aogmib$mdf57$1...@ID-158542.news.dfncis.de>, "manuel" <m...@cemag.de> wrote:
>I would also recommend reading 'Transaction processing: concepts and
>Techniques'Jim Gray and Andreas Reuter. it describes the basic concepts of
>tp monitors but has quite a complete chapter on indexing and locking.

Not that I believe concurrency control was within the scope of the original
posters assignment but one problem with that book is that is talks a lot about
locking protocols and treats other sorts of concurrency protocols (such as
optimistic concurrency control) as mere 'exotics'. The only reason locking has
catched on in the database world is that it is a simple concept to understand.

In nearly all aspects locking is inferior, mainly because it restricts access
_other_ users access to data. A poorly written application will degrade
server performance uniformly, which open up for a bag of administration
hassles on large systems with many applications.


-
Robert Sundström, Mimer SQL Development
Upright Database Technology AB, http://www.mimer.com
Validate your SQL statements at http://developer.mimer.com/parser

Art Pollard

unread,
Oct 31, 2002, 12:35:08 AM10/31/02
to

"Michael Gast" <mig...@web.de> wrote in message
news:3DABC7BC...@web.de...
> Hi Alan,
>
> look on firebird.sourceforge.net. Firebird is a free RDBMS and you can
> study all sources. In addition, i suggest you to have a look on the book
> 'Fundamentals of Database Systems' from Ramez Elmasri and Shamkant
> Navathe, published by Addison Wesley.

I agree with this recommendation.

This is an excellent book and perhaps one of the best books in the field.
Transaction Processing (which someone else recommended) is also
quite good but in many respects, Fundamentals ... is probably a bit
better.

I find it interesting that there really aren't that many books on actual
database
implementation. There are plenty of books on _using_ a database system
but very few on the workings of database systems themselves.

-Art

--
Art Pollard
http://www.lextek.com/
Suppliers of High Performance Text Retrieval Engines.


Tony Ms

unread,
Nov 18, 2002, 5:09:55 PM11/18/02
to
Visit http://www.xmldb.org/

Cheers

Tony


Anthony Youngman

unread,
Nov 19, 2002, 7:04:13 AM11/19/02
to
"Tony Ms" <to...@montgomery-smith.org> wrote in message news:<arbofj$ssl$1...@knossos.btinternet.com>...
> Visit http://www.xmldb.org/
>
Or www.maverick-dbms.org.

You commented particularly about XML ... Note that putting XML into a
relational database is nasty because most DBMSs can't handle nested
data that well. Certainly I've heard that at a recent PostGreSQL
conferences the people working on that DB have just thrown their hands
in the air and walked away from the problem.

But MaVerick is a multi-value database, one of the Pick family (look
also for the IBM databases UniVerse and Unidata, known collectively as
U2, for jBASE, D3, mvBASE, Mentor Pro ...) and we keep having deja-vu
every time we look at this sort of thing. Pick has thought in an XML
paradigm since it was first designed, as the system known as GIRLS, in
the early-to-mid 60s.

So if you want to get into an environment where the XML way of
thinking is second nature, look at the Pick-MultiValue stuff. We've
been doing it for nearly forty years (not me personally, I'm not old
enough :-)

Cheers,
Wol

Emmett

unread,
Nov 19, 2002, 6:24:19 PM11/19/02
to
Anthony....@ECA-International.com (Anthony Youngman) wrote in message news:<9a993dee.02111...@posting.google.com>...

> "Tony Ms" <to...@montgomery-smith.org> wrote in message news:<arbofj$ssl$1...@knossos.btinternet.com>...
> > Visit http://www.xmldb.org/
> >
> Or www.maverick-dbms.org.
>
> You commented particularly about XML ... Note that putting XML into a
> relational database is nasty because most DBMSs can't handle nested
> data that well. Certainly I've heard that at a recent PostGreSQL
> conferences the people working on that DB have just thrown their hands
> in the air and walked away from the problem.

Can't say I agree with that. XML is just data.

I've been storing XML into Oracle and haven't had any problems.

However, I researched the items users will need to search on, and
for those items I store those items in separate columns.

I'm storing the XML into varchar fields. It's just data. Ascii text.

However, Oracle 9 offers an alternative. It has an xml type which
works like a varchar except that it provides for XPATH queries.
I haven't used it because the DBA's haven't installed 9i client
on our development machine.

I have experimented with the Oracle XDK. That's great for interacting
with a DB and Oracle.

I figure the problems being referred to in this post have to do with making
XPath queries available on data - not in storing XML.

>
> But MaVerick is a multi-value database, one of the Pick family (look
> also for the IBM databases UniVerse and Unidata, known collectively as
> U2, for jBASE, D3, mvBASE, Mentor Pro ...) and we keep having deja-vu
> every time we look at this sort of thing. Pick has thought in an XML
> paradigm since it was first designed, as the system known as GIRLS, in
> the early-to-mid 60s.
>
> So if you want to get into an environment where the XML way of
> thinking is second nature, look at the Pick-MultiValue stuff. We've
> been doing it for nearly forty years (not me personally, I'm not old
> enough :-)

This sounds like good advise.

In my stuff I store XML and fetch everything from the database with
utility methods returning JDOM elements. The utility methods use metadata
so the column names in the database become element names in the JDOM
Element.

The XML is wonderful at getting you to reuse code (different queries)
in different contexts. And if you HTML pages have 20 or 30 controls on
a page, it's nice not to have to put each control in a database column.
Another factor, around here deploying jsp's is a drawn out process,
and by putting the XSLs into the database it saves a lot of redeploying.

As to the original question ...

making a DB from scratch ... there are already many alternatives.

The question is strange.

What language Perl, C, C++, JAVA? Why from scratch?


>
> Cheers,
> Wol

Christopher Browne

unread,
Nov 19, 2002, 11:13:07 PM11/19/02
to
The world rejoiced as emme...@onebox.com (Emmett) wrote:
> Anthony....@ECA-International.com (Anthony Youngman) wrote in message news:<9a993dee.02111...@posting.google.com>...
> I figure the problems being referred to in this post have to do with
> making XPath queries available on data - not in storing XML.

That's probably about the size of it.

There's precisely nothing preventing people from storing XML data in
an RDBMS VARCHAR field.

The challenge comes in treating the set of data as a hierarchy and
invoking hierarchy-related queries in an intelligent way.

There are two fundamental issues vis-a-vis SQL:
a) SQL queries return sets that are not inherently ordered, which
means that there's not a totally natural mapping of ordered
hierarchical data onto the DBMS;

b) A set is not a tree, which is kind of the converse of a). SQL
queries return sets, so that you can't get a tree out of a query.

You can't put in a tree /as a tree/, and can't get it out /as a
tree/... That makes using XML a bit troublesome.
--
(reverse (concatenate 'string "moc.enworbbc@" "enworbbc"))
http://cbbrowne.com/info/advocacy.html
"We believe Windows 95 is a walking antitrust violation"
-- Bryan Sparks

Anthony Youngman

unread,
Nov 20, 2002, 11:02:20 AM11/20/02
to
Christopher Browne <cbbr...@acm.org> wrote in message news:<arf24j$hosci$1...@ID-125932.news.dfncis.de>...

> The world rejoiced as emme...@onebox.com (Emmett) wrote:
> > Anthony....@ECA-International.com (Anthony Youngman) wrote in message news:<9a993dee.02111...@posting.google.com>...
> > I figure the problems being referred to in this post have to do with
> > making XPath queries available on data - not in storing XML.
>
> That's probably about the size of it.
>
> There's precisely nothing preventing people from storing XML data in
> an RDBMS VARCHAR field.
>
> The challenge comes in treating the set of data as a hierarchy and
> invoking hierarchy-related queries in an intelligent way.
>
> There are two fundamental issues vis-a-vis SQL:
> a) SQL queries return sets that are not inherently ordered, which
> means that there's not a totally natural mapping of ordered
> hierarchical data onto the DBMS;
>
> b) A set is not a tree, which is kind of the converse of a). SQL
> queries return sets, so that you can't get a tree out of a query.
>
> You can't put in a tree /as a tree/, and can't get it out /as a
> tree/... That makes using XML a bit troublesome.

Precisely ... which is why I said "Look at Pick". It is ABSOLUTELY
TRIVIAL, given a XML record, to declare a Pick "table", import the
record set, and then declare indices, search, whatever.

Apart from the terminology, the Pick mindset and the XML mindset are
very similar, almost to the point of a simple 1-to-1 topological
equivalence.

Okay, I'd need to study XML, but if I was wanting to manipulate XML my
first instinct would be to pull it into a Pick database, manipulate it
with the database tools, and then chuck it out as XML again. Now who
in their right mind would even contemplate such an approach, unless
the conversion between paradigms was a no-brainer? And I'm pretty
confident that, even including the conversion costs, I could run an
*extremely* tight race against "native XML" tools, maybe (indeed,
probably) even beating them for speed on large or complex datasets.

Cheers,
Wol

Andy Dingley

unread,
Nov 20, 2002, 8:49:34 PM11/20/02
to
On 20 Nov 2002 08:02:20 -0800, Anthony....@ECA-International.com
(Anthony Youngman) wrote:

>Apart from the terminology, the Pick mindset and the XML mindset are
>very similar, almost to the point of a simple 1-to-1 topological
>equivalence.

This is one of XML's biggest failings (and although this XML=Pick
analogy isn't true, it's close enough)

Oracle 8 & 9 have a powerful XML storage mechanism, but this is itself
pointlessly limited. Put simply, storing XML with a known XML Schema
is a no-brainer in almost any DB technology, but it's just not an
interesting problem, nor hardly a useful problem to solve.

XML is easy, but most of the data we receive (rather than create)
isn't this well behaved. XML is structured, text is unstructured, and
the real-world is by and large semi-structured, somewhere in the
middle. We can represent this structure (RDF) and we can describe this
representation (DAML) or even explain it (DAML+OIL). What we don't get
though is prior notice when we build our application of how the data
it receives in the future wil lbe structured. Our consumer app has to
"make it up as it goes along" and be enormously fluid, not only about
what it stores, but what it also permits queries upon.

As it happens, I work on just this problem and have built some big and
_efficient_ RDF triple stores on top of commercial SQL's.

Emmett

unread,
Nov 21, 2002, 1:03:22 PM11/21/02
to
Andy Dingley <din...@codesmiths.com> wrote in message

Andy, you might tell us more about all those fancy buzzwords.
RDF, DAML, OIL (not to be confused with the snake version I gather).

> XML is easy, but most of the data we receive (rather than create)
> isn't this well behaved. XML is structured, text is unstructured, and
> the real-world is by and large semi-structured, somewhere in the
> middle. We can represent this structure (RDF) and we can describe this
> representation (DAML) or even explain it (DAML+OIL). What we don't get
> though is prior notice when we build our application of how the data
> it receives in the future wil lbe structured. Our consumer app has to
> "make it up as it goes along" and be enormously fluid, not only about
> what it stores, but what it also permits queries upon.
>
> As it happens, I work on just this problem and have built some big and
> _efficient_ RDF triple stores on top of commercial SQL's.

I don't know about the real world. I just go with the small subset
of the applications I'm fortunate enough to write. And there creating
data into XML - even when it's not well structured isn't a hard task.
Infact, the fact that the data doesn't have to be structured is the
reason for using XML in the first place.

OK. So I'm writing in Java. And I'm using JDOM. I convert request
variables
to JDOM (see example below). All data stores occur in a relational DB
with a column for XML, and all data fetches return XML (JDOM
Elements). Instead of JSPs each page is an XSL rendering XHTML.

...

This is what I use to convert requests into XML (JDOM) :


public abstract class Command
{
protected HttpServletRequest request = null;

Command() {}

public Element Create( HttpServletRequest request ) {
this.request = request;
Element command = new Element("xml_data");
Enumeration enum = request.getParameterNames();
while ( enum.hasMoreElements()) {
String sKey = (String) enum.nextElement();
String [] temp = (String[])
request.getParameterValues(sKey);
int iLen = temp.length;
for ( int j=0 ; j < iLen ; j++ ){
String sVal = temp[j];
String sName = sKey;
if ( sName.length() > 5 &&
"cdata".equalsIgnoreCase((sName.substring(0,5)))) {
command.addContent(new
Element(sName).addContent(new CDATA(sVal)));
} else {
command.addContent(new
Element(sName).addContent(sVal));
}
}
}
return command;
}

public Element Execute(Element elm) throws ChainedException {
return elm;
}

}

And this is a typical fetch (uses my DB code but you should
get the idea) ....

protected Element DoQuery( String sql ) throws ChainedException {
DBConnection dbc = null;
Element xmlData = null;
try {
dbc = new DBConnection();
dbc.ProcessSQL(sql);
xmlData = new Element("forms");
while ( dbc.getResultSet().next() ) { // right indent to
make easy to read
int ccnt = dbc.getResultSet().getMetaData().getColumnCount();
Element form = new Element("form");
for( int i = 1 ; i <= ccnt ; i ++ )
{
String colName =
dbc.getResultSet().getMetaData().getColumnName(i);
String content = dbc.SQLHelper(dbc.getResultSet().getString(i));
form.addContent(new
Element(colName.toLowerCase()).addContent(content));
}
xmlData.addContent(form);
}
} catch ( Exception ex ) {
throw ( new ChainedException( ex, "DoQuery failed : " +
sql ) );
}
finally
{
try {
dbc.closeResultSet();
dbc.closeStatement();
} catch ( Exception ex ) {
throw ( new ChainedException( ex, "SQL Failed : " +
sql ) );
}
}
JDOM.write(xmlData);
return xmlData;
}

Rob Tweed

unread,
Nov 25, 2002, 7:45:29 AM11/25/02
to
Well having highlighted the natural suitability of Pick for XML
strorage, the same is true of MUMPS (the best known implementation
being Cache)

You may want to have a look at our eXtc product which provides a
native XML DOM implementation straight into Cache. I won't waffle on
here about it - have a look at our web site for details -
www.mgateway.com

Its interesting the way XML has brought back new life into the old
hierarchical databases, while the projection of XML into relational
databases is messy to say the least (although of course they all have
XML capabilities)

Rob

On Thu, 21 Nov 2002 16:37:57 +0000, Mark Preston
<use...@mpreston.demon.co.uk> wrote:

>On 20 Nov 2002 08:02:20 -0800, Anthony....@ECA-International.com
>(Anthony Youngman) wrote:
>

>>Precisely ... which is why I said "Look at Pick". It is ABSOLUTELY
>>TRIVIAL, given a XML record, to declare a Pick "table", import the
>>record set, and then declare indices, search, whatever.
>>

>Anthony, as a Pick developer of far too many years standing to gloat
>about it, might I point out that while the development of Pick (and
>associated mvBase systems) databases is indeed simple and they are
>indeed well-suited to XML data _records_ the one thing that they are
>not well-suited for is XML document TREES.
>
>In simple terms - if you can extract the data from the document tree,
>it is easy to use Pick and I would agree with you. However, Pick is
>NOT well-suited to getting that data in the first place. For that, you
>would be much better off IMHO with an object-based parser.

---
Rob Tweed
M/Gateway Developments Ltd

Global DOMination with eXtc : http://www.mgateway.tzo.com
---

Andy Dingley

unread,
Nov 27, 2002, 6:08:23 PM11/27/02
to
On Mon, 25 Nov 2002 12:45:29 +0000, Rob Tweed
<rtw...@imtcons.demon.co.uk> wrote:

>Its interesting the way XML has brought back new life into the old
>hierarchical databases,

Interesting, but not a good thing.

This commonality between XML and hierarchical DB's isn't a validation
of the DB, more a condemnation of XML's rigidly tree-structured data
model.

Christopher Browne

unread,
Nov 27, 2002, 10:01:15 PM11/27/02
to
In the last exciting episode, Andy Dingley <din...@codesmiths.com> wrote::

The fact that certain varieties of hierarchical queries are not
particularly convenient in SQL is a bit of condemnation thereof.

You tend to have two choices:

a) You iterate through the tree, an SQL query for each node, or
b) Go with some cleverness ala the "Celko nested sets" approach.

A query-per-node gets inefficient fast. And while "nested sets" are
very clever, that requires that anyone that manipulates such queries
has to be *very clever*, and in the real world, we have to live with
the fact that most people /aren't/ *very clever*.

That doesn't prevent XML-oriented approaches from having their own
daunting concomittent problems...
--
(concatenate 'string "cbbrowne" "@acm.org")
http://cbbrowne.com/info/spiritual.html
"People need to quit pretending they can invent THE interface and walk
away from it, like some Deist fantasy." -- Michael Peck

oblomov

unread,
Nov 27, 2002, 10:30:26 PM11/27/02
to
On 28 Nov 2002 03:01:15 GMT
Christopher Browne <cbbr...@acm.org> wrote:

> In the last exciting episode, Andy Dingley <din...@codesmiths.com> wrote::
> > On Mon, 25 Nov 2002 12:45:29 +0000, Rob Tweed
> > <rtw...@imtcons.demon.co.uk> wrote:
> >
> >>Its interesting the way XML has brought back new life into the old
> >>hierarchical databases,
> >
> > Interesting, but not a good thing.
> >
> > This commonality between XML and hierarchical DB's isn't a validation
> > of the DB, more a condemnation of XML's rigidly tree-structured data
> > model.
>
> The fact that certain varieties of hierarchical queries are not
> particularly convenient in SQL is a bit of condemnation thereof.
>
> You tend to have two choices:
>
> a) You iterate through the tree, an SQL query for each node, or
> b) Go with some cleverness ala the "Celko nested sets" approach.

Why not just run an LDAP and an RDB at the same time? It's not impossible in any way. Keep heirchial design in heirachial persistency and cross-referenced relations in relation-style persistency.

Or if you ask me, get an OODB and skip the mapping-layers so it's up to you and your design.

karl

Undercover Elephant

unread,
Nov 29, 2002, 2:58:10 AM11/29/02
to
"oblomov" <sp...@oblomov.org> wrote in message
news:20021128043026...@oblomov.org...

Yes use LDAP for XML. This book chapter does a nice comparison of several
approaches (RDB, OODB, Native XML, LDAP):

C. Schmauch and T. Fellhauer (2003) A comparison of database approaches for
storing XML documents. In: XML Data Management: Native XML and XML-Enabled
Database Systems, A.B. Chaudhri, A. Rashid and R. Zicari (eds.),
Addison-Wesley, Reading, Massachusetts, 2003. To appear.
http://www.aw.com/catalog/academic/product/1,4096,0201844524,00.html

Whilst there are a number of documented approaches for storing XML in an RDB
(e.g. see the book by Graves), Celko's nested sets has been used
successfully by my colleague, Dr. Richard Edwards. The code can be
downloaded from sourceforge:

http://minx.sourceforge.net/

Richard also has a nice chapter in the above book mentioned :-)

vr

--
Akmal B. Chaudhri
Zone Editor, Special Projects (Eclipse, Grid, Web Services)
IBM developerWorks - http://www.ibm.com/developerWorks/
WebDatabases ------- http://www.btinternet.com/~webdatabases/


Andy Dingley

unread,
Nov 29, 2002, 9:25:01 AM11/29/02
to
On 28 Nov 2002 03:01:15 GMT, Christopher Browne <cbbr...@acm.org>
wrote:

>The fact that certain varieties of hierarchical queries are not
>particularly convenient in SQL is a bit of condemnation thereof.

Indeed. I don't think this will happen (legacies being what they are)
but I can see a potential future where commerical DB's split into OLAP
(this being what most business clock cycles are now spent on) and
graph-based triple stores.

I'm resistant to object stores and OODBMS in general - this tree
structuring is both restrictive, and pervasive through most of them.
Oracle 9i is a case in point - my current project is a flexible rich
media / metadata store, and the Oracle implementation that has been
suggested is horribly limiting - it's still stuck with this "XSLT
transforms can fix everything" mentality.

Anthony W. Youngman

unread,
Dec 3, 2002, 4:47:32 PM12/3/02
to
In article <nureuuoiu6quuc8g8...@4ax.com>, Andy Dingley
<din...@codesmiths.com> writes

>On 28 Nov 2002 03:01:15 GMT, Christopher Browne <cbbr...@acm.org>
>wrote:
>
>>The fact that certain varieties of hierarchical queries are not
>>particularly convenient in SQL is a bit of condemnation thereof.
>
>Indeed. I don't think this will happen (legacies being what they are)
>but I can see a potential future where commerical DB's split into OLAP
>(this being what most business clock cycles are now spent on) and
>graph-based triple stores.

Maybe there's a reason for legacies being there ... as for business
clock cycles being spent there - are they being spent wisely, or being
spent on the latest buzz-words ...


>
>I'm resistant to object stores and OODBMS in general - this tree
>structuring is both restrictive, and pervasive through most of them.
>Oracle 9i is a case in point - my current project is a flexible rich
>media / metadata store, and the Oracle implementation that has been
>suggested is horribly limiting - it's still stuck with this "XSLT
>transforms can fix everything" mentality.

The trouble (or one of them) with SQL is that its proponents believe
that it is the 42 of databases. The fact is, the real world does not fit
neatly into pigeonholes. And okay, I'm very much a Pick fanatic. But
seeing as I work with Pick and SQL, every time I hit SQL I'm frustrated
by all the limitations it imposes on me which Pick doesn't.

One of the reasons Pick survives is it does what it does very well. And
what it does is model the REAL world, not what narrow-minded people
believe the world should be. It's been commented that Pick programmers
tend to be far more business-aware than their SQL counterparts. Not
surprising, actually, many Pick people slipped into it from business,
which is both a strength and weakness. I've commented that Pick is very
close to XML, others have implied they think of Pick as an "outdated
hierarchical" database, it's often referred to as a "post-relational"
database ... and I tend to think mostly relational when programming
Pick.

At the end of the day, Pick is flexible. It can pretend to be anything.
BECAUSE it FITS the real world. There are a lot of legacy Pick systems.
And one of the reasons is that those companies modern enough to replace
them usually find that they rapidly succumb to the competition (who
haven't replaced their Pick systems ...). There's very few figures
available, but the only study I know of concluded that Pick-based
businesses typically spent half the amount of money on their databases
than companies using other DBs ...

I'm not sure whether Pick would suit you, but it's always nice to hear
people who don't believe in being lemmings, and don't believe that
favourite is perfect. That said, if you're storing *data* (and not
objects like photos, recordings, etc), I've yet to meet anything that
beats Pick ... :-)

Cheers,
Wol
--
Anthony W. Youngman - wol at thewolery dot demon dot co dot uk
Witches are curious by definition and inquisitive by nature. She moved in. "Let
me through. I'm a nosey person.", she said, employing both elbows.
Maskerade : (c) 1995 Terry Pratchett

Emmett

unread,
Dec 4, 2002, 8:07:30 PM12/4/02
to
Sometime in 1996 or 1997 I recall reading in the Wall Street Journal
where Oxford .. Oxford something ... some gigantic health insurer
based in Chicago (I think) migrated away from Pick to a
relational database (I think Oracle).

It was a total disaster precisely because so much of what was
done in Pick couldn't be done at all using a relational DB.

Since Oxford was growing outrageously quickly they weren't
able to bill all their patients. It was common for doctors to wait
six to eight months for payment.

Oxford wasn't able to bill many of it's patients because
their database system was hosed.

The entired deal parciptated some big event - like Oxford going
out of business completely - or getting in trouble for having
merged with some other insurer and messing things up.

Great story. I wish I could remember it better.

Jan Hidders

unread,
Dec 5, 2002, 1:34:40 AM12/5/02
to

Me too, because the details matter here. It is trivial to find a mapping
from the Pick data model to a relational model, but the question is what
your requirements are. So it all hinges upon the question how good your
system analysts, who have to define this mapping, are. If these people don't
know their job you can even get into the same problems when moving from an
Excel-based application to a database-based application. That doesn't prove
that Excel is a better database.

-- Jan Hidders

Christopher Browne

unread,
Dec 5, 2002, 7:47:22 AM12/5/02
to

More than likely the /real/ problems were business problems. They
almost always are.

If a big consulting firm was involved (often the case!), the disaster
could have readily still occurred even if they stayed with Pick, as
the firm would certainly not have centered the work around "how does
Pick work?" (any more than it is likely that they centered the
/actual/ work around "how does Oracle work?"), but would rather have
the scope of work based on the question: "What is Our Methodology?"

Throw together:
a) A Methodology that doesn't fit the application,
b) Account executives that are happy so long as they are billing,
c) An attitude that technical problems aren't all that important,
and you can get a /huge/ disaster that cannot legitimately be tied to
any particular piece of technology.

<http://www.itworld.com/Man/2680/CWD001030ITSnafus/> points at some of
the notable disasters. All of them are essentially /project/
failures, not /technology/ failures.


--
(concatenate 'string "cbbrowne" "@acm.org")

http://www.ntlug.org/~cbbrowne/
"Let me get this straight: A company that dominates the desktop, and
can afford to hire an army of the world's best programmers, markets
what is arguably the world's LEAST reliable operating system?
What's wrong with this picture?" -- <fr...@cc.UManitoba.CA>

Eric Junkermann

unread,
Dec 5, 2002, 8:38:34 AM12/5/02
to
emme...@yahoo.com (Emmett) wrote in message news:<92d3639d.0212...@posting.google.com>...

>
> It was a total disaster precisely because so much of what was
> done in Pick couldn't be done at all using a relational DB.
>

... by the people who were doing it! They had to design and test the
new application and design and test the data conversion - someone got
some or all of it wrong. There are enough failed software
developments in the world that you can probably find one to apparently
back up any x vs y argument.

I'm afraid that if you want this one to carry any weight you'll have
to dig up a genuine statement of something that was done in Pick that
can't be done in a relational DBMS (or maybe just couldn't be done in
Oracle at the time, since that is _not_ the same thing).

Eric Junkermann
-- beware of hammer syndrome!

Elspeth Ferguson

unread,
Dec 5, 2002, 9:58:07 AM12/5/02
to
On 5 Dec 2002 07:34:40 +0100, hid...@REMOVE.THIS.uia.ua.ac.be (Jan
Hidders) wrote:

>Emmett wrote:
>>Sometime in 1996 or 1997 I recall reading in the Wall Street Journal
>>where Oxford .. Oxford something ... some gigantic health insurer
>>based in Chicago (I think) migrated away from Pick to a
>>relational database (I think Oracle).

<snip>

>>Great story. I wish I could remember it better.
>
>Me too, because the details matter here.

Oxford Health Plan is one of the hoariest war stories in the MV world,
I think. A quick Google reveals:

URL:http://groups.google.com/groups?q=Oxford+Health+Plan+group:comp.databases.pick&hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=97.dec.10.oxhp%40nytimes.com&rnum=1

or how Oxford lost a shedload of money very quickly because they
couldn't process their invoicing, and

URL:http://groups.google.com/groups?q=Oxford+Health+Plan+group:comp.databases.pick&hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=3pa6nssl4v9rhblmq42gv9legsu0rrkkr8%404ax.com&rnum=7&filter=0

which is from a very similar conversation here on c.d.p about two and
a half years ago. But as Eric and Christopher have pointed out
further down the thread, "MV rocks - Oracle sucks" is far from the
whole story in this case, however true we wish it to be ;o)

Cheers,
--
Elspeth Ferguson www.ukpua.org
MultiValue News edi...@ukpua.org

hRodstein

unread,
Dec 5, 2002, 4:37:14 PM12/5/02
to
How about an n-level commission structure calculation? (where n is in excess
of 20 recursions) Something that falls apart in SQL very quickly. Recursive
data relationship problems are "pert-near impossible.
Remember, "relational" data bases are clumsy ways to associate flat data
record images. SQL was developed so data collected by mainframes using
COBOL could more easily be accessed with a 4 gl language. The basic model
is STILL Holorith cards (developed in the late 19th century). The Pick/MV
model came out of a TRW think-shop in the late 60's. Even Codd in his later
papers praised the MV model and agreed that hist original "rules" for
relational data bases was sorely inadaquate for real-world business
applications. But, so much "main stream" money was pumped into the data
bases based on flat tables and Codd's original rukes, he was promptly
ignored.
In my experience, Pick shops that have moved from the mv model to
"relational" data bases have less than 60% of the functionality of their mv
System- 5-6 years after the fact However, the IT director can command a
300K salary. Great for IT, sucks for the business.


"Eric Junkermann" <er...@deptj.demon.co.uk> wrote in message
news:70568f73.02120...@posting.google.com...

Anthony W. Youngman

unread,
Dec 5, 2002, 6:55:41 PM12/5/02
to
In article <70568f73.02120...@posting.google.com>, Eric
Junkermann <er...@deptj.demon.co.uk> writes
>emme...@yahoo.com (Emmett) wrote in message news:<92d3639d.0212041707.f1b767@po

>sting.google.com>...
>>
>> It was a total disaster precisely because so much of what was
>> done in Pick couldn't be done at all using a relational DB.
>>
>
>... by the people who were doing it! They had to design and test the
>new application and design and test the data conversion - someone got
>some or all of it wrong. There are enough failed software
>developments in the world that you can probably find one to apparently
>back up any x vs y argument.
>
>I'm afraid that if you want this one to carry any weight you'll have
>to dig up a genuine statement of something that was done in Pick that
>can't be done in a relational DBMS (or maybe just couldn't be done in
>Oracle at the time, since that is _not_ the same thing).
>
Provide a decent response time ...

That was always a strength of Pick - if your app is large relative to
the computer it's on, Pick would smoke any other database for speed.

I'm sure others would provide more details about the Oxford debacle, but
basically, the management were sold on Oracle, tried to migrate from
Pick, and the costs ballooned while the new software was unable to cope.
Oh - that's another strength of Pick - you can usually do the same work
at half the cost ...

David Cressey

unread,
Dec 6, 2002, 9:55:22 PM12/6/02
to
Especially since Oracle has the "CONNECTED BY" feature which enables tree
traversal to be done
fairly trivially. Yeah, it's non standard SQL, but if you just want to
manipulate trees easily, it's handy.

--
Regards,
David Cressey
www.dcressey.com


0 new messages