Sensible and NonsenSQL Aspects of the NoSQL Hoopla

[Jan Hidders]

One of the best and most insightful reads in the SQL vs noSQL debate
I've seen lately:

http://www.edbt.org/Proceedings/2013-Genova/papers/edbt/a2-mohan.pdf

-- Jan Hidders

[.NET Sales Support]

"Jan Hidders" <hid...@gmail.com> wrote in message <news:514c2466$0$6325$e4fe...@dreader35.news.xs4all.nl>...

> One of the best and most insightful reads in the SQL vs noSQL debate I've seen lately:
>
> http://www.edbt.org/Proceedings/2013-Genova/papers/edbt/a2-mohan.pdf
>
> -- Jan Hidders

It should be possible to throw out the SQL bathwater without throwing the relational baby out along with it.

http://www.c2.com/cgi/wiki?TqlRoadmap

[karl.s...@o2online.de]

It's a nice paper, but due to sloppy terminology it misses vital points:

- With database cursors and indizes there are only "Not only SQL" systems.

- Transactions are no part of the relational model

- The relational modell is never "correct" implemented. Without "holographic"
storage media I see no way to implement "tables" where "row- and column order
are insignificant". Defining relations as a "interface" views of storage
objects leads to practical, but "pseudo" relational systems.

- The real problem of NoSQL systems is the missing of a standard for "system
catalogs" as a repository for metadata. Mixing metadata and data as a
solution for large "sparse" tables is nice, but not a full substitute for
a metadata repository.

The "SQL vs noSQL debate" is fruitless, unless Codd's theory will finally
be consolidated. Looking into Codd's paper of 1970 You will find:

- Normalization and 1NF is a "workaround" to avoid "too complicated
datastructures". A sound idea 1970 with Fortran, Assembler and PreAnsiC
it was never revised

- Defining relations and tuples only via it's contents limits the "physical
data independence" to "record oriented storages". "Column oriented storages"
needs additional data for mapping (rowid)

- Relation and tuples seems to e a blurred vision of object oriented storage
management. Accepting the object nature of relations and tuples, many
problems (missing data, identity, table inheritance ... ) simply disappear

m.f.G.

Karl Scheurer

[Eric]

On 2013-08-21, karl.s...@o2online.de <karl.s...@o2online.de> wrote:

> Am Freitag, 22. M?rz 2013 10:29:10 UTC+1 schrieb Jan Hidders:
>> One of the best and most insightful reads in the SQL vs noSQL debate
>>
>> I've seen lately:
>>
>>
>>
>> http://www.edbt.org/Proceedings/2013-Genova/papers/edbt/a2-mohan.pdf
>>
>>
>>
>> -- Jan Hidders
>
> It's a nice paper, but due to sloppy terminology it misses vital points:
>
> - With database cursors and indizes there are only "Not only SQL" systems.

That's just confusing.

> - Transactions are no part of the relational model

No, they are orthogonal to it. Codd's original paper contains the word
only once, in relation to checking for inconsistencies in data, and it
seems that he is viewing them as part of the known universe.

> - The relational modell is never "correct" implemented. Without "holographic"
> storage media I see no way to implement "tables" where "row- and column
> order are insignificant". Defining relations as a "interface" views of
> storage objects leads to practical, but "pseudo" relational systems.

You do not appear to be aware of the difference between logical and
physical models.

> - The real problem of NoSQL systems is the missing of a standard for "system
> catalogs" as a repository for metadata. Mixing metadata and data as a
> solution for large "sparse" tables is nice, but not a full substitute for
> a metadata repository.

As far as I can see NoSQL systems don't have "system catalogs" because
the data in them is insufficiently organised to allow one to be
designed. However, whether or not this is a real problem for them has
got nothing to do with any other argument about their merits relative
to SQL or the relational model or anything else.

> The "SQL vs noSQL debate" is fruitless, unless Codd's theory will finally
> be consolidated. Looking into Codd's paper of 1970 You will find:
>
> - Normalization and 1NF is a "workaround" to avoid "too complicated
> datastructures". A sound idea 1970 with Fortran, Assembler and PreAnsiC
> it was never revised

Codd does not use the word "workaround". He says that "... more
complicated data structure is necessary for a relation with one or more
nonsimple domains" and so "the possibility of eliminating nonsimple
domains appears worth investigating!" and that "there is, in fact, a very
simple elimination procedure, which we shall call normalization." Not
a workaround, and nothing to do with how old-fashioned the languages
being used are.

> - Defining relations and tuples only via it's contents limits the "physical
> data independence" to "record oriented storages". "Column oriented storages"
> needs additional data for mapping (rowid)

Once again, you do not appear to be aware of the difference between
logical and physical models. A relational implementation could quite easily
use column-wise storage instead of or as well as any other storage
structure without imposing any limitations on its faithfulness to the
relational model. It's just an implementation layer, not part of the
relational model!

> - Relation and tuples seems to e a blurred vision of object oriented storage
> management. Accepting the object nature of relations and tuples, many
> problems (missing data, identity, table inheritance ... ) simply disappear

Relations and tuples do not have an object nature, they are not objects.
Tables do not have inheritance, they are relations which are not
objects. Missing data and identity are not problems anyway.

The whole object thing is a methodology which can be very helpful to
programmers, if done correctly, but it is harder than most programmers
think to do correctly. It has nothing whatever to do with "Large Shared
Data Banks". The key word in that phrase is "Shared", a fact that most
proponents of "objects" do not seem to get at all.

Eric
--
ms fnd in a lbry

[karl.s...@o2online.de]

Am Mittwoch, 21. August 2013 19:17:09 UTC+2 schrieb Eric:
> On 2013-08-21, karl.s...@o2online.de <karl.s...@o2online.de> wrote:
>
>
> > - With database cursors and indizes there are only "Not only SQL" systems.
>
>
>
> That's just confusing.
>

Why? with database cursor You can traverse a table in a very old fashioned
navigational way (first, next, last, previous). Indizes are logical data
structures when efficient subsets are essential (conditional indizes)

>

>
> > - The relational modell is never "correct" implemented. Without "holographic"
>
> > storage media I see no way to implement "tables" where "row- and column
>
> > order are insignificant". Defining relations as a "interface" views of
>
> > storage objects leads to practical, but "pseudo" relational systems.
>
>
>
> You do not appear to be aware of the difference between logical and
>
> physical models.
>

As a programmer I don't see the advantage of a "partial" logical view to
real life structures. A "mathematical" relation can be implemented as a
incomplete view on a more complex structure. A "computational" relation
has relation and tuple identities and a life before and after filled with
data. As long as I see no alternative working implementation I see no use
for such a limited "logical model"

>
>
> As far as I can see NoSQL systems don't have "system catalogs" because
>
> the data in them is insufficiently organised to allow one to be
>
> designed. However, whether or not this is a real problem for them has
>
> got nothing to do with any other argument about their merits relative
>
> to SQL or the relational model or anything else.
>
>

Wrong!
It's no problem to create Your own "system catalogs". We have done this for
a application when we integrate our regulation database into a workflow
system. The real problem is the missing standard.

>
>
> Codd does not use the word "workaround". He says that "... more
>
> complicated data structure is necessary for a relation with one or more
>
> nonsimple domains" and so "the possibility of eliminating nonsimple
>
> domains appears worth investigating!" and that "there is, in fact, a very
>
> simple elimination procedure, which we shall call normalization." Not
>
> a workaround, and nothing to do with how old-fashioned the languages
>
> being used are.
>

Maybe I gave Codd to much credit for his reasoning. Based on my experiences
with these languages it seems to be reasonable to shift complexity in a
easier manageable area (tuple fractioning). "Since more complicated data structures" became part of standard libraries, I see no reason to cope with
exploding numbers of tables when simpled solutions exists.

>
>
> > - Defining relations and tuples only via it's contents limits the "physical
>
> > data independence" to "record oriented storages". "Column oriented storages"
>
> > needs additional data for mapping (rowid)
>
>
>
> Once again, you do not appear to be aware of the difference between
>
> logical and physical models. A relational implementation could quite easily
>
> use column-wise storage instead of or as well as any other storage
>
> structure without imposing any limitations on its faithfulness to the
>
> relational model. It's just an implementation layer, not part of the
>
> relational model!
>

Wrong!
- Try column-wise storage on a relation with composite primary key
- Try export a Access database table to a Excel worksheet and reimport
the data and compare the tables.

>
>
> Relations and tuples do not have an object nature, they are not objects.
>
> Tables do not have inheritance, they are relations which are not
>
> objects. Missing data and identity are not problems anyway.
>

"It's not a bug, it's a feature". Tables can have inheritance (at least in Postgres).

"Missing data and identity are not problems anyway" is a very bold statement without limit it to Your field of work. We develop applications for inspection and maintainance of powerplants. Missing data is a problem for us without a
system created identity.

>
>
> The whole object thing is a methodology which can be very helpful to
>
> programmers, if done correctly, but it is harder than most programmers
>
> think to do correctly. It has nothing whatever to do with "Large Shared
>
> Data Banks". The key word in that phrase is "Shared", a fact that most
>
> proponents of "objects" do not seem to get at all.
>

You are partially right. Storing application objects in "objectrepositories"
is indeed ridiculous. Separating applications and databases is no argument
against a object view to databases. Though no Microsoft fan I believe the
ADO framework is a step in the right direction. ADO's "Datasets" are "relation
objects".

m.f.G.
Karl Scheurer

[Eric]

On 2013-08-22, karl.s...@o2online.de <karl.s...@o2online.de> wrote:
> Am Mittwoch, 21. August 2013 19:17:09 UTC+2 schrieb Eric:
>> On 2013-08-21, karl.s...@o2online.de <karl.s...@o2online.de> wrote:
>>
>>> - With database cursors and indizes there are only "Not only SQL" systems.
>>
>> That's just confusing.
>>
> Why? with database cursor You can traverse a table in a very old fashioned
> navigational way (first, next, last, previous). Indizes are logical data
> structures when efficient subsets are essential (conditional indizes)

I know what cursors and indexes are, but it is "Not only SQL" that is
confusing. They are not part of relational theory, but they are part of
SQL.

>>> - The relational modell is never "correct" implemented. Without
>>> "holographic" storage media I see no way to implement "tables" where
>>> "row- and column order are insignificant". Defining relations as a
>>> "interface" views of storage objects leads to practical, but "pseudo"
>>> relational systems.
>>
>> You do not appear to be aware of the difference between logical and
>> physical models.
>>
> As a programmer I don't see the advantage of a "partial" logical view to
> real life structures. A "mathematical" relation can be implemented as a
> incomplete view on a more complex structure. A "computational" relation
> has relation and tuple identities and a life before and after filled with
> data. As long as I see no alternative working implementation I see no use
> for such a limited "logical model"

The logical model is the set of relations and constraints that you
define to represent as much of a set of real life structures as you wish
to store and manipulate. The physical model is how your database
software organises the data for efficient storage and retrieval. As a
programmer, you deal with the logical model, it is the tables you write
SQL to manipulate. As a programmer, you don't care about the physical
model unless you think it needs to be modified for performance reasons.
The fact that present database products have a physical model which can
be, and therefore often is, an exact representation of the logical model
leads too many people to make unfortunate modifications to their logical
model when they should be thinking only about the physical model and how
it can map to the (unchanged) logical model.

It seems to me that you may be talking about something else here.

>> As far as I can see NoSQL systems don't have "system catalogs" because
>> the data in them is insufficiently organised to allow one to be
>> designed. However, whether or not this is a real problem for them has
>> got nothing to do with any other argument about their merits relative
>> to SQL or the relational model or anything else.
>>
> Wrong!
> It's no problem to create Your own "system catalogs". We have done this for
> a application when we integrate our regulation database into a workflow
> system. The real problem is the missing standard.

I believe you, For any specific case, yes, but it is the general case
that is the problem, which is why is is at best difficult to create a
standard.

>>> The "SQL vs noSQL debate" is fruitless, unless Codd's theory will
>>> finally
>>> be consolidated. Looking into Codd's paper of 1970 You will find:
>>>
>>> - Normalization and 1NF is a "workaround" to avoid "too complicated
>>> datastructures". A sound idea 1970 with Fortran, Assembler and
>>> PreAnsiC
>>> it was never revised
>>

>> Codd does not use the word "workaround". He says that "... more
>> complicated data structure is necessary for a relation with one or more
>> nonsimple domains" and so "the possibility of eliminating nonsimple
>> domains appears worth investigating!" and that "there is, in fact, a very
>> simple elimination procedure, which we shall call normalization." Not
>> a workaround, and nothing to do with how old-fashioned the languages
>> being used are.
>>
> Maybe I gave Codd to much credit for his reasoning. Based on my
> experiences with these languages it seems to be reasonable to shift
> complexity in a easier manageable area (tuple fractioning). Since
> "more complicated data structures" became part of standard libraries,
> I see no reason to cope with exploding numbers of tables when simpled
> solutions exists.

I don't think it has anything to do with the languages or libraries,
but merely with the attempt to define easily usable and easily manageable
data structures. Having a normalised relational model of your data means
that every element of data is easily accessible both individually and
in context, easily updated (subject to constraints), easily compared
both with its peers and with suitable other elements, easily filtered
and easily aggregated or summarised (and even easily ignored).

If you have a type of complex structure for which you do not need to
deal with its internal complexity until after you have identified and
retrieved it (or when you are constructing it prior to storage), then
you can treat it as a "simple domain" in relational terms. Incidentally,
this makes it an element of a tuple, it will not be a table, or a row
of a table (though the physical representation in the back-end may look
like anything at all).

>>> - Defining relations and tuples only via it's contents limits the
>>> "physical data independence" to "record oriented storages". "Column
>>> oriented storages" needs additional data for mapping (rowid)
>>
>> Once again, you do not appear to be aware of the difference between
>> logical and physical models. A relational implementation could quite easily
>> use column-wise storage instead of or as well as any other storage
>> structure without imposing any limitations on its faithfulness to the
>> relational model. It's just an implementation layer, not part of the
>> relational model!
>>
> Wrong!
> - Try column-wise storage on a relation with composite primary key

For efficiency reasons you might need key-wise storage (in effect a key
index) as well as or instead of pure column-wise storage, but that is
about designing the physical storage layer.

> - Try export a Access database table to a Excel worksheet and reimport
> the data and compare the tables.

What data property does this not preserve? But it doesn't really matter,
they are different tools for different purposes and it is no surprise
that a double translation would lose something unless the mechanisms had
been specifically designed not to.

>>
>> Relations and tuples do not have an object nature, they are not objects.
>> Tables do not have inheritance, they are relations which are not
>> objects. Missing data and identity are not problems anyway.
>
> "It's not a bug, it's a feature". Tables can have inheritance (at least
> in Postgres).

"Feature under construction" judging by the documentation. It is a new
way to deal with sub-types (in the Entity-Relationship modelling sense)
and might turn out to be a good idea (or not) but I am sure that people
will find ways to misuse it because it says "inheritance" and they will
assume that it is the same as object inheritance.

> "Missing data and identity are not problems anyway" is a very bold
> statement without limit it to Your field of work. We develop applications
> for inspection and maintainance of powerplants. Missing data is a problem
> for us without a system created identity.

Missing data is of two kinds:
- It is not currently applicable to the real-world thing we are dealing
with (this kind is never a problem)
- Is is applicable and intended to be in the database, but for some
reason we currently do not have it for this particular real-world
thing. In this case we have to either disregard (and discard?) the
real-world thing, or we have to provide for storing the fact that
it is missing (and/or the reason for this). Problems arise here only
because database designers do not pay attention to the alternatives.

A real-world thing is either identifiable in the real world, or it isn't.
In the latter case we either don't store anything about it, or we consider
it as one of a counted collection of a type that is identifiable. Another
possibility is that we assign it a system-created identifier and mark
the real-world thing with the identifier (which may be impossible and/or
undesirable).

If is is identifiable we have to have the complete set of identifying
features in the database. This is a natural key, but it may be awkward to
use, so we might need to use a system-created identifier which is mapped,
in one relation, to the real-world identifying features. Again it might
sometimes be appropriate to mark the real-world thing with the identifier.

>> The whole object thing is a methodology which can be very helpful to
>> programmers, if done correctly, but it is harder than most programmers
>> think to do correctly. It has nothing whatever to do with "Large Shared
>> Data Banks". The key word in that phrase is "Shared", a fact that most
>> proponents of "objects" do not seem to get at all.
>>
> You are partially right. Storing application objects in "object
> repositories" is indeed ridiculous.

Thankyou.

> Separating applications and databases is no argument against a object
> view to databases.

Agreed, though many of the attempts at the object view are (deliberately
or not) still trying to be object repositories.

> Though no Microsoft fan I believe the ADO framework is a step in the
> right direction. ADO's "Datasets" are "relation objects".

I have insufficient knowledge to judge this, but it at least sounds
plausible.

[karl.s...@o2online.de]

Am Sonntag, 25. August 2013 15:38:37 UTC+2 schrieb Eric:

>
> I know what cursors and indexes are, but it is "Not only SQL" that is
>
> confusing. They are not part of relational theory, but they are part of
>
> SQL.
>

I use SQL as a "relational query language". "Nonrelational" Querying
(special case handling, short circuit boolean evaluation) is easy
scanning cursors from begin to end.

>
> The logical model is the set of relations and constraints that you
>
> define to represent as much of a set of real life structures as you wish
>
> to store and manipulate. The physical model is how your database
>
> software organises the data for efficient storage and retrieval. As a
>
> programmer, you deal with the logical model, it is the tables you write
>
> SQL to manipulate. As a programmer, you don't care about the physical
>
> model unless you think it needs to be modified for performance reasons.
>
> The fact that present database products have a physical model which can
>
> be, and therefore often is, an exact representation of the logical model
>
> leads too many people to make unfortunate modifications to their logical
>
> model when they should be thinking only about the physical model and how
>
> it can map to the (unchanged) logical model.
>
>
>
> It seems to me that you may be talking about something else here.
>

Right!
Tracking "businesss transactions" a "logical model" might be sufficient.
Documenting inspection and maintainance we need a "conceptual" model.
Interpreting relations as logical statements (propositions) we have to
deal with propositions like "valve of type x produced by y has a cast
iron case" and "valve of type x produced by y has a forged case" and
both statements are true and legal. Try normalization without prior
knowledge of technical development and producer policy.

Try detail planning (inspection, tests...) when you only know that "50
valves of type x" are ordered.

>
>
> I don't think it has anything to do with the languages or libraries,
>
> but merely with the attempt to define easily usable and easily manageable
>
> data structures. Having a normalised relational model of your data means
>
> that every element of data is easily accessible both individually and
>
> in context, easily updated (subject to constraints), easily compared
>
> both with its peers and with suitable other elements, easily filtered
>
> and easily aggregated or summarised (and even easily ignored).
>

Based on personal experiences must I disagree. Arrays without "index out
of bound protection" had to be very simple. Since data structures became
part of the standard library´of modern programming languages I prefer
a more sophisticated approach. Normalization is useful for simple or for
data with medium complexity. Realistic systems like SAP/R3 yields a number
of tables beyound comprehension (>100000).

>
>
> If you have a type of complex structure for which you do not need to
>
> deal with its internal complexity until after you have identified and
>
> retrieved it (or when you are constructing it prior to storage), then
>
> you can treat it as a "simple domain" in relational terms. Incidentally,
>
> this makes it an element of a tuple, it will not be a table, or a row
>
> of a table (though the physical representation in the back-end may look
>
> like anything at all).
>

Your assumption is not valid. As in the valve examples have to deal with
the internal structure when querying (special case handling).

>
> For efficiency reasons you might need key-wise storage (in effect a key
>
> index) as well as or instead of pure column-wise storage, but that is
>
> about designing the physical storage layer.
>
>
>
> > - Try export a Access database table to a Excel worksheet and reimport
>
> > the data and compare the tables.
>
>
>
> What data property does this not preserve? But it doesn't really matter,
>
> they are different tools for different purposes and it is no surprise
>
> that a double translation would lose something unless the mechanisms had
>
> been specifically designed not to.
>

What is the benefit of "logical model" that cannot be mapped lossless to
a physical storage? Sorry, but relations are abstractions of "record
oriented" storages. Primary key is the y-coordinate and column-names are
the correspondent x coordinates. Column oriented storages need a alternative
y-address.

>
> "Feature under construction" judging by the documentation. It is a new
>
> way to deal with sub-types (in the Entity-Relationship modelling sense)
>
> and might turn out to be a good idea (or not) but I am sure that people
>
> will find ways to misuse it because it says "inheritance" and they will
>
> assume that it is the same as object inheritance.
>

What's wrong with that assumption?

>
> A real-world thing is either identifiable in the real world, or it isn't.
>

True! But what about ordered real-world things ordered and not currently
present? Planing activities on these things is bread and butter for us.
After finishing our activities every real-world thing is identifiable in
your sense. Our solution is to create a internal system created identity.
This identity is used to make the entity unique and for referential integrity.

> In the latter case we either don't store anything about it, or we consider
>
> it as one of a counted collection of a type that is identifiable. Another
>
> possibility is that we assign it a system-created identifier and mark
>
> the real-world thing with the identifier (which may be impossible and/or
>
> undesirable).
>

Why? What is your solution to our planning example?
>
m.f.G.

Karl Scheurer

[Eric]

On 2013-08-26, karl.s...@o2online.de <karl.s...@o2online.de> wrote:
> Am Sonntag, 25. August 2013 15:38:37 UTC+2 schrieb Eric:
>

Plenty of point-by-point answers possible if I had time, but since I
may not, I have picked out the one statement (and broken it up a bit)
that had most impact on me:

> What is the benefit of "logical model" that cannot be mapped lossless to
> a physical storage?

That it is logical? In any case I am not claiming that it cannot be
mapped to physical storage, just that it may be different.

> Sorry, but relations are abstractions of "record
> oriented" storages.

No, relations are an adaptation of a mathematical concept to which you
can map your real-world data so that there is a clearly defined set of
logical (and logically correct) operations for manipulating that data. If
you try to show a relation on paper it looks like a table, which in turn
looks like "record-oriented" storage (though you could draw the table
the other way, in which case it looks like "column-oriented" storage). In
every case "looks like" is an analogy, not an exact equivalence.

> Primary key is the y-coordinate and column-names are
> the correspondent x coordinates.

You can think of like that if you wish, there is no overriding reason to
store it like that.

> Column oriented storages need a alternative
> y-address.

And for efficient use, record-oriented storage requires indexes, whereas
carefully designed column-wise storage removes the need for many (or
even most) of them.

Physical storage of a relation that is exactly record-oriented storage
is only justifiable for efficiency if you are retrieving all "records"
that correspond to a single relation. Add in any conditions or joins and
you need "complications" of the storage to allow efficient retrieval.


The relations (tables if you must) that you use are given to you by an
abstraction layer that is the business of the RDBMS you use. It may
offer multiple storage methods, accompanied by advice on when each is
appropriate. A good, experienced DBA for that RDBMS will be able to
improve on that advice.

To be honest I think your database is a tool you are using without
sufficient understanding, particularly of the theory behind it. I don't
doubt that your systems do what is expected of them efficiently enough,
but how easy is it to keep them doing that in the face of changing
operational requirements and the appearance of unanticipated reporting
requirements?

[James K. Lowden]

On Mon, 26 Aug 2013 03:20:44 -0700 (PDT)
karl.s...@o2online.de wrote:

> Since data structures became part of the standard library of modern
> programming languages I prefer a more sophisticated approach.

Many programming languages had data structures when the relational
model was invented. Not all, of course, not Forth for example. But
Cobol, Fortran, Algol, and PL/1 did.

What we have not seen the decades since is anything like the
sophistication of set theory in programming languages.

--jkl

[karl.s...@o2online.de]

Am Mittwoch, 28. August 2013 05:00:46 UTC+2 schrieb James K. Lowden:
> On Mon, 26 Aug 2013 03:20:44 -0700 (PDT)
>
> karl.s...@o2online.de wrote:
>
> Many programming languages had data structures when the relational
>
> model was invented. Not all, of course, not Forth for example. But
>
> Cobol, Fortran, Algol, and PL/1 did.
>

Sorry, data structures like records and arrays are data types in our
terminology. We call linked lists, tree, graphs, stacks, queues ...
(dynamic) data structures. These structures are not part of standard
libraries before C++. The next step after data structues are objects.
(data structures with special behaviours). We started objects with
homegrown matrix- and complex- number objects for engineering calculations.
Relations can be implemented as objects as I know at least one system
complete done with objects.

I challenge a non object implementation for relations with satisfying
Codd' constraints (row-, column order are insignificant).

m.f.G.
Karl Scheurer

[karl.s...@o2online.de]

[Eric]

That's because you can't get hold of the idea that physical storage is
irrelevant (as long as it works and is efficient enough). What the
constraints mean in practice is that when you retrieve a set of rows
you can make _no_ assumptions about the order they will be in unless you
explicitly specify an order. Column order is irrelevant because you have
to specify which columns you want and then (usually) get them in the
order specified. If you use SELECT * you are back to the order being
unpredictable (though in practice usually predictable in any single
RDBMS, but this is an artifact of the implementation of the RDBMS, not a
requirement of relational theory).

[com...@hotmail.com]

On Monday, August 26, 2013 3:20:44 AM UTC-7, karl.s...@o2online.de wrote:
> What is the benefit of "logical model" that cannot be mapped lossless to
> a physical storage? Sorry, but relations are abstractions of "record
> oriented" storages.

You have a misconception of the relational model. A relational schema is intended to express what you call a "conceptual model". A relation variable or query has an associated predicate and its tuples are those that make true propositions/statements.

The notion of distinct "conceptual" and "logical" models is a perversion that arose from non-relational thinking. Even ORM2/FCO-IM, which has always made an effort to map directly to a relational schema, is an unnecessary layer in the sense that only the relational schema, its "logical" model, ends up being the form of the model one actually uses. (With its conceptual a relational view of its logical.) You seem to be misinformed in a typical ER culture way, where a relational schema is wrongly taken to be a sort of implementation data structure. (And the role of constraints is not understood.)

The point of the relational model is that when one's model is a relational schema, which as I say includes predicates about the world (from which constraints follow), one can query directly about that world generically in terms of those predicates (either via predicate logic formulas or equivalent relation expressions) with an automatic optimized execution.

philip

[com...@hotmail.com]

ps

On Monday, August 26, 2013 3:20:44 AM UTC-7, karl.s...@o2online.de wrote:

> Primary key is the y-coordinate and column-names are
> the correspondent x coordinates. Column oriented storages need a alternative
> y-address.

A tuple/row of a relation/table is a point in a multidimensional space. The attribute/column names are the dimension names. They are the points in the space that satisfy the relation's predicate.

Every FD in a relation corresponds to a function from a point in the space of the determining attributes to a point in the space of the dependent attributes. Other functions and relations can be derived from this.

To think of a candidate key as an ordinate is, again, a non-relational way of thinking and reduces a relation to a mere data structure instead of the extension of the predicate of the variable or query that it is the value of. (Also primary keys have no theoretical basis.)

philip

[karl.s...@o2online.de]

Am Donnerstag, 29. August 2013 00:54:31 UTC+2 schrieb com...@hotmail.com:
> On Monday, August 26, 2013 3:20:44 AM UTC-7, karl.s...@o2online.de wrote:
>
> > What is the benefit of "logical model" that cannot be mapped lossless to
>
> > a physical storage? Sorry, but relations are abstractions of "record
>
> > oriented" storages.
>
>
>
> You have a misconception of the relational model. A relational schema is >intended to express what you call a "conceptual model". A relation variable or >query has an associated predicate and its tuples are those that make true >propositions/statements.

Sorry! I did not phrase my critics in a clear and comprehensible way. Based
on a background in algorithms I see no way to implement the relational model
in a complete and lossless way. Every "real" relational system is like a car
with 5 gears using only 3. Implementing relations as a kind of objects is no
difficult task and been done several times. Accepting the object nature of relations can be the difference between success and failure. My problem is
not the usability of the relational model, but the irrational hold on as a
complete answer for all database problems. Codd's model emerged out of the
technology of the seventies and needs urgently a revision.

m.f.G
Karl Scheurer

[karl.s...@o2online.de]

Am Donnerstag, 29. August 2013 06:30:58 UTC+2 schrieb com...@hotmail.com:

>
> A tuple/row of a relation/table is a point in a multidimensional space. The attribute/column names are the dimension names. They are the points in the space that satisfy the relation's predicate.
>

...

>
>
> To think of a candidate key as an ordinate is, again, a non-relational way of >thinking and reduces a relation to a mere data structure instead of the >extension of the predicate of the variable or query that it is the value of.

No you got it! To access or to distinguish points in a multidimensional space
you need additional information not provided with the relational model. Only
the tacit assumption of grouping information into a tuples/rows... the missing
information can calculated from the column data. Intepreting this grouping as
records relations or statements is pretty irrelevant.

m.f.G.
Karl Scheurer

[James K. Lowden]

On Sat, 31 Aug 2013 08:22:41 -0700 (PDT)
karl.s...@o2online.de wrote:

> Codd's model emerged out of the technology of the seventies and needs
> urgently a revision.

That's a novel observation. I'm sure others would be interested to
know of any aspect of the relational model rooted in 1970s technology.

Do tell.

--jkl

[com...@hotmail.com]

On Saturday, August 31, 2013 8:22:41 AM UTC-7, karl.s...@o2online.de wrote:
> I see no way to implement the relational model in a complete and lossless way.

I am interested to know what that is trying to say. Especially since:

On Wednesday, August 28, 2013 3:54:31 PM UTC-7, com...@hotmail.com wrote:
> The point of the relational model is that when one's model is a relational schema, which as I say includes predicates about the world (from which constraints follow), one can query directly about that world generically in terms of those predicates (either via predicate logic formulas or equivalent relation expressions) with an automatic optimized execution.

Still (from all your comments):

> You have a misconception of the relational model.

philip

[karl.s...@o2online.de]

My observation is based on Codd's paper of 1970 "A Relational Model of Data for Large Shared Data Banks"

He addresses the following problems
1.2.1. Ordering Dependence.
"Let us consider those existing systems which either require or permit data
elements to be stored in at least one total ordering which is closely associated with the hardware-determined ordering of addresses.
"
Without bypassing all operating systems this is impossible nowadays. Before
UNIX and other operating systems it was common practice to file layout directly
in storages.

1.2.2. Indexing Dependence.
"...destroy indices from time to time will probably be necessary. The question then arises: Can application programs and terminal activities remain invariant as indices come and go?..."

In the seventies "bigdata" had to be stored on sequential data storages
(tapes, cards). Querying data from sequential media cannot use indices
("indices go").

1.2.3. Access Path Dependence.
"
One solution to this is to adopt the policy that once a
user access path is defined it will not be made obsolete until
all application programs using that path have become
obsolete. Such a policy is not practical, because the number
of access paths in the total model for the community of
users of a data bank would eventually become excessively
large."

That statement is based on the hardware of the seventies.

First normal form and normalization
"
So far, we have discussed examples of relations which are
defined on simple domains-domains whose elements are
atomic (nondecomposable) values. Nonatomic values can
be discussed within the relational framework. Thus, some
domains may have relations as elements. These relations
may, in turn, be defined on nonsimple domains, and so on.
"
It is clear, Codd started 1970 with a design like the
"document storages" in NOSQL or the N1F systems of the past.

For reasons not comprehensible any more (Codd's reference is
out of print and not online available), he restricted his model

"1.4. NORMAL FORM
A relation whose domains are all simple can be represented
in storage by a two-dimensional column-homogeneous
array of the kind discussed above. Some more

complicated data structure is necessary for a relation with

one or more nonsimple domains. For this reason (and others
to be cited below) the possibility of eliminating nonsimple
domains appears worth investigating! There is, in fact, a

very simple elimination procedure, which we shall call
normalization.
"

Reading more than enough horror stories about program failures
based on "index out of bound" it was reasonable to keep the
design simple and avoid complex dynamic data structures. Meanwhile
are complex dynamic data structures (trees, graphs, lists... )
part of standard liraries for common mainstream programing
languages.

Last but not least
"Future users of large data banks must be protected from
having to know how the data is organized in the machine (the
internal representation)"
For me as a programmer this sounds like a textbook example for
object design. If objects are the "best" way to implement
relations, then do it the relational way is like driving a car
with 5 gears and only using 3 gears.

m.f.G.
Karl Scheurer

[karl.s...@o2online.de]

Am Mittwoch, 28. August 2013 21:53:02 UTC+2 schrieb Eric:
>
> That's because you can't get hold of the idea that physical storage is
>
> irrelevant (as long as it works and is efficient enough).

Physical storage can become irrelevant when the logical model is
sufficiently complete. The Relational Model is not, proved by the
fact of SQL extensions already starting with DB2. Codd's requirement
of row order insignificance excludes grouping and grouped aggregats
from his model. Grouping is pretty easy to comprehend, when comparisons
between adjacent tuples can be formulated. SQL 1999 with generalized
grouping is the next step, but not relational anymore.

To repeat it once more: every theoretical model based on cursors is a
valid solution. Cursors are lists of database objects with some sort
order and comply with some criteria. Cursors with uniform objects can
be viewed as "relations". Directly working with cursors is at least
a last resort option when everthing else fail.

m.f.G.
Karl Scheurer

[Eric]

On 2013-08-31, karl.s...@o2online.de <karl.s...@o2online.de> wrote:
> Am Donnerstag, 29. August 2013 06:30:58 UTC+2 schrieb com...@hotmail.com:
>
>>
>> A tuple/row of a relation/table is a point in a multidimensional
>> space. The attribute/column names are the dimension names. They are the
>> points in the space that satisfy the relation's predicate.
>>
> ...
>>
>> To think of a candidate key as an ordinate is, again, a non-relational
>> way of >thinking and reduces a relation to a mere data structure instead
>> of the >extension of the predicate of the variable or query that it is
>> the value of.
>
> No you got it! To access or to distinguish points in a multidimensional space

> you need ...

... to know all the coordinates of any point, i.e. a set of values with
some way of knowing which axis each refers to. Sounds like a tuple in
which each element belongs to a specific domain. So for all the points
you know about, you have a set of tuples. Sounds like a relation.

> (need) additional information not provided with the relational model.

So what am I missing here? What information do you need that is not
provided in the relational model?

> Only the tacit assumption of grouping information into a
> tuples/rows... the missing information can calculated from the column
> data.

Again, what missing information?

> Intepreting this grouping as records relations or statements is pretty
> irrelevant.

On the assumption that there is missing information.

[James K. Lowden]

On Sun, 1 Sep 2013 03:47:35 -0700 (PDT)
karl.s...@o2online.de wrote:

> Am Samstag, 31. August 2013 19:59:52 UTC+2 schrieb James K. Lowden:
> > On Sat, 31 Aug 2013 08:22:41 -0700 (PDT)
> > karl.s...@o2online.de wrote:
> > > Codd's model emerged out of the technology of the seventies and
> > > needs urgently a revision.
> >
> > That's a novel observation. I'm sure others would be interested to
> > know of any aspect of the relational model rooted in 1970s
> > technology.
> >

> My observation is based on Codd's paper of 1970 "A Relational Model
> of Data for Large Shared Data Banks"

I appreciate the effort you took to defend your idea, and I think I now
understand what you mean. But you are confusing the incidental
reference to contemporary technology with its influence on the
relational model.

Yes, you can find references to 1970's technology in a 1970s paper and,
yes, Codd was a product of his day just as anyone is. His work was
motivated by commercial needs in a commercial firm, and he was applying
his mathematical expertise to real-world problems. His paper relates
his theoretical work to those problems in terms of the technology then
extant.

What you will not find is any aspect of the theory that is *tied* to any
technology, 1970s or other. The theory is all about relations and
operations and constraints. Naturally both Codd and his contemporaries
were interested in how that theory might be applied to real computers
of the day; then as now many insisted that his pointy-headed math could
never be implemented efficiently. That quite different from saying the
theory is somehow blinkered by the limitations of those computers.

> He addresses the following problems
> 1.2.1. Ordering Dependence.
> "Let us consider those existing systems which either require or
> permit data elements to be stored in at least one total ordering
> which is closely associated with the hardware-determined ordering of
> addresses. "

If you remove "hardware-determined" from the sentence, it's exactly as
true now as then.

> 1.2.2. Indexing Dependence.
> "...destroy indices from time to time will probably be necessary. The
> question then arises: Can application programs and terminal
> activities remain invariant as indices come and go?..."
>
> In the seventies "bigdata" had to be stored on sequential data
> storages (tapes, cards). Querying data from sequential media cannot
> use indices ("indices go").

Hmm, no, I'm pretty sure VSAM and IMS were available in the 70s.
Cullinet was selling IDMS.

Codd's "competition" wasn't 1890s hollerith cards. It was (what were
later called) hierarchical and network DBMSs that imposed great
constraints and complexity on application programmers.

Funny how little has changed, right? You move to Hadoop City and build
an entire application around a "known" application domain on a
nonstandard filesystem. Then comes the day you'd like summaries by zip
code instead by customer account, and you have to write an application
instead of a query. Santayana rides again!

> 1.2.3. Access Path Dependence.
> "
> One solution to this is to adopt the policy that once a
> user access path is defined it will not be made obsolete until
> all application programs using that path have become
> obsolete. Such a policy is not practical, because the number
> of access paths in the total model for the community of
> users of a data bank would eventually become excessively
> large."
>
> That statement is based on the hardware of the seventies.

On the web I believe it's called "404".

> First normal form and normalization
> "
> So far, we have discussed examples of relations which are
> defined on simple domains-domains whose elements are
> atomic (nondecomposable) values. Nonatomic values can
> be discussed within the relational framework. Thus, some
> domains may have relations as elements. These relations
> may, in turn, be defined on nonsimple domains, and so on.
> "
> It is clear, Codd started 1970 with a design like the
> "document storages" in NOSQL or the N1F systems of the past.

Yes.

> For reasons not comprehensible any more (Codd's reference is
> out of print and not online available), he restricted his model

No mystery. Books in a library are hardly lost texts of Babylon. And
he states his motivation plainly: "the possibility of eliminating

nonsimple domains appears worth investigating!"

> "1.4. NORMAL FORM
> A relation whose domains are all simple can be represented
> in storage by a two-dimensional column-homogeneous
> array of the kind discussed above. Some more
> complicated data structure is necessary for a relation with
> one or more nonsimple domains. For this reason (and others
> to be cited below) the possibility of eliminating nonsimple
> domains appears worth investigating! There is, in fact, a
> very simple elimination procedure, which we shall call
> normalization.
> "

The model is not "restricted". It is *simplified*, a feature, not a
bug. By showing -- more, *proving* -- that logical inferences could be
drawn from data manipulated with a small number of operators closed
over a domain, Codd released programmers from low-level complexity and
man-centuries of work.

> Meanwhile are complex dynamic data structures (trees, graphs,
> lists... ) part of standard liraries for common mainstream programing
> languages.

If you're programming a computer, graphs are a your natural ally
because they can be mapped directly onto the computer's memory. They're
of no use, though, when you want to manage data logically. How, for
example, do you define a subset of a cyclic graph?

You're right to say that graphs are more complex than relations. It's a
mistake, though, to conclude therefore that they are more powerful.
It's been proved mathematically that graphs and relations are
interchangeable in the sense that they can represent the same
information. The difference is that relational theory is much
simpler. That's its advantage, not a handicap.

> "Future users of large data banks must be protected from
> having to know how the data is organized in the machine (the
> internal representation)"
>
> For me as a programmer this sounds like a textbook example for
> object design.

OK, but it's not.

Consider the UNIX filesystem, for instance, which you refered to
earlier. Upon a time, when my mother wrote disk access routines for
Univac, the programmer had to know all the particulars of the device,
and read/write data in terms of the device's design. Unix
revolutionized the field by abstracting all disk access into today's
familar stream of bytes. No addresses, no heads or sectors or tracks.
A catalog to facilitate sharing that anyone (potentially) can update,
not just the system programmers. Works pretty good for nonrotating
media, too, and over the network. And not an object in sight.

On the other hand, you are in some sense right, if the DBMS is
the object. What OO calls "data hiding" is analogous to what RM calls
"data independence". In both cases, the goal is to isolate the
application from details it doesn't need and that might change, to
permit the application programmer to operate at a higher level. Both
also have a notion of "consistent state". I have long thought that
stored procedures are to databases what methods are to objects, and
subscribe to the idea that applications should access the data only
through views and procedures.

Part of your critique is actually of DBMSs that we have, not of RM.
SQL DBMSs largely support only a few primitive types that the user may
then further constrain or write functions for. One cannot, for
example, define an aggregate type as a set of columns, and use that
name in, say, FK declarations. Nor can we usually define types of blobs
and comparison functions for them (although I'm unconvinced that's a
good idea).

And of course SQL itself -- not RM! -- is deeply rooted in IBM's 1970s
notion of an end-user query language, to let users write their own
reports. You could talk all day to IT departments about math and
logic, but you could close the deal convincing them that their reports
would write themselves. So we're saddled now with a language no one
likes, and that no one thinks expresses relational algebra or calculus
well. I wonder if we're going to have live through the rediscovery of
the purpose and benefits of the relational model before we see a
re-implementation of it that provides a relational language in which to
express our queries.

--jkl

P.S. Since you've read this far and we're debating the 70s, I hope
you've seen http://www.masswerk.at/google60/.

[Norbert_Paul]

The paper does not mention the spatial databasers. What are there
historical part of the NoSQL movement?

For example http://www.spatial.maine.edu/~max/RJ6.html not only argues
against actual shortcomings of SQL1 in the standard of 1898
http://www2.yk.psu.edu/~lxn/IST_210/sql1_versus_sql2.html
but also against the relational model. However, I simply don't get
some of the arguments therein:

----------------------------------------------------------------------
* The tabular representation "is not the most natural form for
modelling spatial data and leads to unnecessarily complex queries."
[p. 2]

What does that mean? Is "most natural" equal to "representable in
one single table"?

----------------------------------------------------------------------
* High level spatial concepts can only be "simulated" in "terms of the
few, predefied non-spatial concepts". [p. 2]

What is the difference beween "simulating" concepts on a computer
and "not simulating" them. Does an algorithm only "simulate its
functionality in terms of elementary operations"?

----------------------------------------------------------------------
* The relationships of pure SQL, such as 'less' or 'greater', are too
low level to address all the spatial concepts sufficiently. [p. 4]

This is simply a completely unsubstantiated claim. And wrong, too.

----------------------------------------------------------------------
* "overloading of SQL's standard predicates [...] is insufficient for
spatial relationships because the set of standard predicates in SQL
is too small to cover all spatial relationships." [p. 5]

In contrast, all the "spatial relationships" (aka nine-intersections)
can be modelled and queries with SQL1. However this is not an
elementary task.

----------------------------------------------------------------------
* "The description of graphical display of the query result has drawn
only little attention." [p. 5]
"Relations are only displayed in tabular form." [p. 9 "Decoupled
retrieval and display"].

At that time IRIX was already available. It had the "dual" shortcoming
of lacking DBMS functionality. OpenGL today is still lacking DBMS
functionality ;).

----------------------------------------------------------------------
* "no interaction with the currently displayed result is possible."
[p. 6]

This shortcoming of SQL1 that has also been critisized by others and
has been removed in 1992. These "others" did not merely toss away SQL.

----------------------------------------------------------------------
* "[...] SQL [...] does not support such fundamental as metadata
queries and higher order queries." [pp. 6-7]
"As SQL lets users query only the values of the tuples, it does not
support metadata queries." [p. 8]

This is called "catalog" and was also required by Codd himself (Codd
RMV2 1990). This shows that the authors was not familiar with the
relational model with which they confounded SQL. Implementation of
missing catalog functionality by the data modeller does NOT introduce
redundancies. Also, why are catalog queries more important for spatial
data than for other application domains?

----------------------------------------------------------------------
* "... the retrieval part of SQL is inherently value based, i.e. all
operations are performed on attributes, and tuples can only be
compared for equal values, but not for identity." [p. 7]

Codd: "We shall see, the unique identifier is the name of the R-table,
together with its primary key value." (RMV2 1990) So the tuple name
(primary key value) IS the tuple identity. Note that the author again
showed non-familiarity with the relational model.

----------------------------------------------------------------------
* "For example, although it is possible to ask for 'all roads in
Penobscot county that are wider than the road from Bangor to Bar
Harbor' it is impossible to formulate an SQL query for 'the
relation between the widths of Interstate I95 and Route 1A', with
an expected answer such as, 'I95 is wider than Route 1A'" [p. 8]

"SELECT r1.name
FROM r1 road, r2 road
WHERE r2.name = "1A" and
r1.width > 2.width" [p. 9]

First, the quoted query is syntactically wrong. It should read

SELECT r1.name
FROM road r1, road r2 -- tuple variables go after table names
WHERE r2.name = '1A' and -- string constants are 'single quoted'
r1.width > 2.width;

The SQL1-queries

CREATE TABLE OPS(op char not null primary key);

INSERT INTO OPS(op)values('<'),('='),('>');

SELECT r1.name as road1, OPS.op AS relation, r2.name as road2
FROM road r1, road r2, OPS
WHERE r1.name='I95' AND r2.name = '1A' AND
((r1.width > r2.width AND OPS.op = '>') OR
(r1.width < r2.width AND OPS.op = '<') OR
(r1.width = r2.width AND OPS.op = '='));

show that "impossible" equals "impossible for the author". Maybe this
query demonstrates why "SQL per se is already difficult to use."
[p. 11]. It may be necessary to write non-elementary queries.
Non-familiarity with SQL, too .

----------------------------------------------------------------------
* "SQL per se is already difficult to use." [p. 11]

Yes. Correct syntax is hard to learn and non-elementary queries
are difficult to write.

----------------------------------------------------------------------

The problem is, that this old claim that bare SQL be unsuitable for
spatial data still seems to be believed in. At least I have the
impression that there still exist people who believe in such
publications and that they put firm resistance against any attempt
towards using the classical relational model for spatial data. This
means that people which have such poor knowledge in the relational
model that they produce or believe in such gibberish are still
confounded with experts.


Norbert Paul

[Eric]

On 2013-09-01, karl.s...@o2online.de <karl.s...@o2online.de> wrote:
> Am Samstag, 31. August 2013 19:59:52 UTC+2 schrieb James K. Lowden:
>> On Sat, 31 Aug 2013 08:22:41 -0700 (PDT)
>> karl.s...@o2online.de wrote:
>>
>> > Codd's model emerged out of the technology of the seventies and needs
>> > urgently a revision.
>>
>> That's a novel observation. I'm sure others would be interested to
>> know of any aspect of the relational model rooted in 1970s technology.
>>
>> Do tell.
> My observation is based on Codd's paper of 1970 "A Relational Model of
> Data for Large Shared Data Banks"
>
> He addresses the following problems
> 1.2.1. Ordering Dependence.
> "Let us consider those existing systems which either require or permit
> data elements to be stored in at least one total ordering which is
> closely associated with the hardware-determined ordering of addresses."
>
> Without bypassing all operating systems this is impossible nowadays.
> Before UNIX and other operating systems it was common practice to file
> layout directly in storages.

Nowadays you can not, in general, get that close to the hardware (there
are exceptions). However, people still keep stuff in sequential files,
thus perpetuating the problem.

> 1.2.2. Indexing Dependence.
> "...destroy indices from time to time will probably be necessary. The
> question then arises: Can application programs and terminal activities
> remain invariant as indices come and go?..."
>
> In the seventies "bigdata" had to be stored on sequential data storages
> (tapes, cards). Querying data from sequential media cannot use indices
> ("indices go").

But Codd also said that indexing "tends to improve response to queries
and updates and, at the same time, slow down response to insertions and
deletions", so he wasn't thinking about sequential media. Random-access
external storage existed before 1970 (though usually expensive and heavy).

> 1.2.3. Access Path Dependence.
> "One solution to this is to adopt the policy that once a user access path
> is defined it will not be made obsolete until all application programs
> using that path have become obsolete. Such a policy is not practical,
> because the number of access paths in the total model for the community
> of users of a data bank would eventually become excessively large."
>
> That statement is based on the hardware of the seventies.

This one definitely has nothing to do with hardware, but with
tree-structured and network-structured storage layouts.

> First normal form and normalization

(this is towards the end of 1.3)

> "So far, we have discussed examples of relations which are defined on
> simple domains-domains whose elements are atomic (nondecomposable)
> values. Nonatomic values can be discussed within the relational
> framework. Thus, some domains may have relations as elements. These
> relations may, in turn, be defined on nonsimple domains, and so on."
>
> It is clear, Codd started 1970 with a design like the
> "document storages" in NOSQL or the N1F systems of the past.

I don't think that it is clear at all. You are looking at it with the
"benefit" of hindsight starting from a particular point of view. You
may be in danger of presenting a circular argument.

> For reasons not comprehensible any more (Codd's reference is
> out of print and not online available), he restricted his model

Which reference are you talking about here?

> 1.4. NORMAL FORM
> "A relation whose domains are all simple can be represented in storage
> by a two-dimensional column-homogeneous array of the kind discussed
> above. Some more complicated data structure is necessary for a relation
> with one or more nonsimple domains. For this reason (and others to be
> cited below) the possibility of eliminating nonsimple domains appears
> worth investigating! There is, in fact, a very simple elimination
> procedure, which we shall call normalization."
>
> Reading more than enough horror stories about program failures
> based on "index out of bound" it was reasonable to keep the
> design simple and avoid complex dynamic data structures. Meanwhile
> are complex dynamic data structures (trees, graphs, lists... )
> part of standard liraries for common mainstream programing
> languages.

"Index out of bounds" has nothing to do with it. This is not only about
simplicity of representation and storage or transfer of data between
systems, but about avoiding storage structures that leave "read
everything" as the only way to get to some piece of data, and, in fact,
about ensuring that all data is created equal without making you look
at it all at once.

> Last but not least
> "Future users of large data banks must be protected from having to know
> how the data is organized in the machine (the internal representation)"
>
> For me as a programmer this sounds like a textbook example for
> object design. If objects are the "best" way to implement
> relations, then do it the relational way is like driving a car
> with 5 gears and only using 3 gears.

Two different things that both hide information. The relational model
hides the implementation of storage, an object system hides processing
and data, at as many levels as you like. Hiding data is only useful
because the hidden processing uses it, and any higher level doesn't
need to. As soon as the data is required outside, particularly if it is
needed as part of retrieval criteria, the fact that it is hidden is bad.
The trouble is that you can't predict when some new use will be found
(_Shared_ Data Banks, remember?).

On the other hand the only time you need to unhide the storage
implementation is when it needs to be changed for performance reasons
and then, because of the relational model, you do not have to change
any of your programs at all.

[Eric]

On 2013-09-01, karl.s...@o2online.de <karl.s...@o2online.de> wrote:

> Am Mittwoch, 28. August 2013 21:53:02 UTC+2 schrieb Eric:
>> That's because you can't get hold of the idea that physical storage is
>> irrelevant (as long as it works and is efficient enough).
>
> Physical storage can become irrelevant when the logical model is
> sufficiently complete. The Relational Model is not, proved by the
> fact of SQL extensions already starting with DB2.

In what way is the relational model insufficiently complete?

SQL is not the relational model, merely an imperfect attempt at
implementing it. With suitable precautions it can be used as part of
producing an application that views its data relationally. The fact that
SQL has been (and is still being) extended is irrelevant. Some of the
extensions take it further away from being relational.

> Codd's requirement of row order insignificance excludes grouping and
> grouped aggregats from his model. Grouping is pretty easy to comprehend,
> when comparisons between adjacent tuples can be formulated. SQL 1999
> with generalized grouping is the next step, but not relational anymore.

Row order insignificance is required because you may want to order by
and/or group by any part of the data, and merely means that the system
will not impose an order for you. Ordering and grouping are not part of
the data definition, they are operations.

> To repeat it once more: every theoretical model based on cursors is a
> valid solution. Cursors are lists of database objects with some sort
> order and comply with some criteria. Cursors with uniform objects can
> be viewed as "relations". Directly working with cursors is at least
> a last resort option when everthing else fail.

That paragraph seems a little muddled, but in any case it does not seem
to be any sort of argument about the relational model or its adequacy.

[Norbert_Paul]

The paper does not mention the spatial databasers. What are their

historical part of the NoSQL movement?

For example http://www.spatial.maine.edu/~max/RJ6.html not only argues
against actual shortcomings of SQL1 in the standard of 1898
http://www2.yk.psu.edu/~lxn/IST_210/sql1_versus_sql2.html
but also against the relational model. However, I simply don't get
some of the arguments therein:

----------------------------------------------------------------------
* The tabular representation "is not the most natural form for
modelling spatial data and leads to unnecessarily complex queries."
[p. 2]

What does that mean? Is "most natural" equal to "representable in
one single table"?

----------------------------------------------------------------------
* High level spatial concepts can only be "simulated" in "terms of the
few, predefied non-spatial concepts". [p. 2]

What is the difference beween "simulating" concepts on a computer
and "not simulating" them. Does an algorithm only "simulate its
functionality in terms of elementary operations"?

----------------------------------------------------------------------
* The relationships of pure SQL, such as 'less' or 'greater', are too
low level to address all the spatial concepts sufficiently. [p. 4]

This is simply a completely unsubstantiated claim, and wrong, too.

RMV2 1990). This shows that the author was not familiar with the
relational model with which he confounds SQL. Implementation of

This shows non-familiarity with SQL, too .

----------------------------------------------------------------------
* "SQL per se is already difficult to use." [p. 11]

Yes. Correct syntax is hard to learn and non-elementary queries
are difficult to write.

----------------------------------------------------------------------

The problem is, that this old claim that bare SQL be unsuitable for
spatial data still seems to be believed in. At least I have the
impression that there still exist people who believe in such
publications and that they put firm resistance against any attempt
towards using the classical relational model for spatial data. This
means that people which have such poor knowledge in the relational
model that they produce or believe in such gibberish are still
confounded with experts.


Norbert Paul

Jan Hidders wrote:

[karl.s...@o2online.de]

Am Sonntag, 1. September 2013 19:49:48 UTC+2 schrieb Eric:

>
> In what way is the relational model insufficiently complete?
>

In my line of work the most frustating limit of the relational model
is the omission of the order of tuples. Sorting part of SQL and not
part of Codd's relational model. Even if sorting is entered through
the back door it remains a tool to beautify the output and cannot be
exploited while querying, since relations are unordered sets. Don't
know if grouping and querying groups can be comprehend without reference
to sorting order, since viewing and analyzing groups as a sorted list
is easy to communicate and to implement. Hiding these algorithms behind
"macro" functions like done in SQL is not the problem.

> Ordering and grouping are not part of the data definition, they are
> operations.

Grouping are part of the data definition:

employes are a set of (teachers, assistents, secretaries ...)

Every set with potential subsets can be defined with groups

m.f.G.

Karl Scheurer

[karl.s...@o2online.de]

Am Sonntag, 1. September 2013 18:53:22 UTC+2 schrieb James K. Lowden:

>
> If you remove "hardware-determined" from the sentence, it's exactly as
>
> true now as then.
>

if you remove hardware-determined from the sentence, then it's no difference
between pointers and foreign keys.

>
>
> > 1.2.2. Indexing Dependence.
>
> > "...destroy indices from time to time will probably be necessary. The
>
> > question then arises: Can application programs and terminal
>
> > activities remain invariant as indices come and go?..."
>
> >
>
> > In the seventies "bigdata" had to be stored on sequential data
>
> > storages (tapes, cards). Querying data from sequential media cannot
>
> > use indices ("indices go").
>
>
>
> Hmm, no, I'm pretty sure VSAM and IMS were available in the 70s.
>
> Cullinet was selling IDMS.
>

Maybe! In the 70s storage was very expensive and indices are additional
costs. Using a model without the need for indices sounds attractive

>
> > 1.2.3. Access Path Dependence.
>
> > "
>
> > One solution to this is to adopt the policy that once a
>
> > user access path is defined it will not be made obsolete until
>
> > all application programs using that path have become
>
> > obsolete. Such a policy is not practical, because the number
>
> > of access paths in the total model for the community of
>
> > users of a data bank would eventually become excessively
>
> > large."
>
> >
>
> > That statement is based on the hardware of the seventies.
>
>
>
> On the web I believe it's called "404".
>

No! with cheap storage it can be reasonable to keep all user access path,
at least the policy has to be reconsidered.

>
> > For reasons not comprehensible any more (Codd's reference is
>
> > out of print and not online available), he restricted his model
>
>
>
> No mystery. Books in a library are hardly lost texts of Babylon. And
>
> he states his motivation plainly: "the possibility of eliminating
>
> nonsimple domains appears worth investigating!"
>

Without any objective reasons this seems to be a personal opinion to
me. I don't object for all cases, but using a unchecked rule without
considering potential alternatives is not a good thing.

>
> The model is not "restricted". It is *simplified*, a feature, not a
>
> bug. By showing -- more, *proving* -- that logical inferences could be
>
> drawn from data manipulated with a small number of operators closed
>
> over a domain, Codd released programmers from low-level complexity and
>
> man-centuries of work.
>

Codd shifted the complexity from one area to another. When dealing with
n entities is replaced with n*x relations is a considerable increase in
complexity. One reference told me that a SAP R3 system contains more than
100000 tables (one hundred thousand!). What about complexity?

>
>
> If you're programming a computer, graphs are a your natural ally
>
> because they can be mapped directly onto the computer's memory. They're
>
> of no use, though, when you want to manage data logically. How, for
>
> example, do you define a subset of a cyclic graph?
>

Depends on the subset definition. In our field of work we use a ordered
list representation of graphs. A subset is a range of items satisfying
same criterias.

>
>
> You're right to say that graphs are more complex than relations. It's a
>
> mistake, though, to conclude therefore that they are more powerful.
>
> It's been proved mathematically that graphs and relations are
>
> interchangeable in the sense that they can represent the same
>
> information. The difference is that relational theory is much
>
> simpler. That's its advantage, not a handicap.
>

It really a handicap. Try to express grouping and grouped aggregates
in a frame with only "unordered sets". SQL deviates from the relational
theory with implementing groups and grouped aggegats.

>

>
> OK, but it's not.
>
>
> Consider the UNIX filesystem, for instance, which you refered to
>
> earlier. Upon a time, when my mother wrote disk access routines for
>
> Univac, the programmer had to know all the particulars of the device,
>
> and read/write data in terms of the device's design. Unix
>
> revolutionized the field by abstracting all disk access into today's
>
> familar stream of bytes. No addresses, no heads or sectors or tracks.
>
> A catalog to facilitate sharing that anyone (potentially) can update,
>
> not just the system programmers. Works pretty good for nonrotating
>
> media, too, and over the network. And not an object in sight.
>

That depends on your object definition. For me a object is a data type
with addition internal procedures, identity and means to communicate
(signals, messages...) and can be implemented in various manors. Unix
files fit in this description.


>
> ... I have long thought that

>
> stored procedures are to databases what methods are to objects, and
>
> subscribe to the idea that applications should access the data only
>
> through views and procedures.
>

Ok!

>
>
> Part of your critique is actually of DBMSs that we have, not of RM.
>
> SQL DBMSs largely support only a few primitive types that the user may
>
> then further constrain or write functions for. One cannot, for
>
> example, define an aggregate type as a set of columns, and use that
>
> name in, say, FK declarations. Nor can we usually define types of blobs
>
> and comparison functions for them (although I'm unconvinced that's a
>
> good idea).
>

On the contrary! My critic is on RM and not on the real SQL databases. My
only critic on SQL is to pretend to be relational. My critic on RM can be
summarized in the statement "relations are unordered sets". Any model
based on a definition like "relations are sets with any (exploitable) sort order" is much better.

m.f.G.

Karl Scheurer

[paul c]

On 01/09/2013 11:38 PM, karl.s...@o2online.de wrote:
> Am Sonntag, 1. September 2013 19:49:48 UTC+2 schrieb Eric:
>
>> >
>> > In what way is the relational model insufficiently complete?
>> >
> In my line of work the most frustating limit of the relational model
> is the omission of the order of tuples. Sorting part of SQL and not
> part of Codd's relational model. Even if sorting is entered through
> the back door it remains a tool to beautify the output and cannot be
> exploited while querying, since relations are unordered sets.

Ordering depends on domain-specific operators, eg., the various complete
orderings of integers depend on the integer arithmetic operators.

Omission is the essential technique of abstraction. You are suggesting
that ordering is somehow essential. Well, you, anybody and everybody who
think that are free to try to devise an abstraction that is
'ordering-complete'. Codd abstracted out domains which makes his
abstraction more universal. How would you order the domain of the
colours of the rainbow or would you just exclude rainbow colours from
the things that Codd's relations can manipulate?

Beyond that, introducing ordering even for only a limited number of
domains would need to show that the proof mechanisms of FOL would remain
as sound or complete as they are without such introduction, otherwise
the usual optimization techniques couldn't be proved. Eg., would two
relations with the same tuples but different ordering be logically
equivalent? If they were not equivalent, just how would their values
indicate that? What new operators would be needed and what would
replace the sixteen potential operators of a two-valued truth system?
Or, if they are equivalent/equal, what purpose would be served by
introducing ordering operators to the algebra?

Beyond that, why should SQL with ordering be compared to the RM when SQL
without ordering isn't even comparable?

(Many criticisms of the RM seem to have a motive of less abstraction and
therefore less generality. What does an HTML frame have to do with
relations? The various resulting practices, OO dbms'es, XML syntax, etc
etc, all introduce additional mechanisms which usually verge into the
physical, eg., physical representations of values rather than the values
themselves. It seems that the majority of people can't always separate
one from the other, and never will. The majority crowd can always make
more noise. Codd himself strayed away from the simplest abstraction
when he suggested nulls and therefore a relation's name that can stand
for different predicates at the same time. I wonder if that was because
he couldn't hear himself think for all the noise.)

[paul c]

Forgot to ask: Can anybody point me to the paper "Extending the
Relational Model to Capture More Understanding"? (author unknown)

[paul c]

On 02/09/2013 7:42 AM, paul c wrote:
> Forgot to ask: Can anybody point me to the paper "Extending the
> Relational Model to Capture More Understanding"? (author unknown)

Or the title might have been "Extending Relational Understanding to
Capture More Relations Than There Are".

[paul c]

On 02/09/2013 1:07 AM, karl.s...@o2online.de wrote:
> Codd shifted the complexity from one area to another. When dealing with
> n entities is replaced with n*x relations is a considerable increase in
> complexity. One reference told me that a SAP R3 system contains more than
> 100000 tables (one hundred thousand!). What about complexity?

No. Codd shifted what was common in programs (programs of the 1960's,
not the 1970's by the way) to the dbms. Obviously no one person nor any
manageable group of people can decide how redundant SAP is, so SAP must
be regarded as a failure as far as Codd's idea is concerned. If your
statistic is accurage, SAP must be regarded as just as much a failure as
the systems it seeks to replace.

James Lowden is right that the obvious programming leverage of the RM is
in removing data manipulation from program text, eg. through declarative
views. The DB world has generally failed at this too. The answer is
obviously not more education of application programmers, not more data
manipulation in app program text. Eg., most application programmers
have been so cultured that they cannot imagine how to function without,
for example, assignment. The general mistake is failing to recognize
that there is more demand for programming talent than there is
programming talent. That's true in every field but it's especially so
in IT which is a much younger field than most.

If part of your meaning is that some things today would be much
different had we had today's hardware in the 1960's, I agree with that.
For one thing, there was no commercial online database then that
couldn't be recorded in today's RAM.

[paul c]

On 01/09/2013 11:38 PM, karl.s...@o2online.de wrote:

> Am Sonntag, 1. September 2013 19:49:48 UTC+2 schrieb Eric:

...

>> Ordering and grouping are not part of the data definition, they are
>> operations.
>
> Grouping are part of the data definition:
>
> employes are a set of (teachers, assistents, secretaries ...)
>
> Every set with potential subsets can be defined with groups
>

This pretends that a predicate that is about individuals is the same as
a predicate about sets of individuals. Tuple elements that are sets
aren't the same thing as tuples that are sets.

Not every group can be ungrouped, which should be enough to show that
the grouped relation can't possibly have the same predicate as the
ungrouped one. If the predicates must be different then the two data
definitions aren't defining the same thing, so you are talking apples
and oranges.

As far as I can tell, the RM doesn't prevent its own application in an
implementation that chooses to equate grouped relations with ungrouped
ones, eg., where the empty set as a value of an attribute is considered
meaningless so that every such attribute-set-value can be
expressed/recorded as a set of singleton sets.

[Eric]

On 2013-09-02, karl.s...@o2online.de <karl.s...@o2online.de> wrote:
> Am Sonntag, 1. September 2013 19:49:48 UTC+2 schrieb Eric:
>
>>
>> In what way is the relational model insufficiently complete?
>>
>
> In my line of work the most frustating limit of the relational model
> is the omission of the order of tuples. Sorting part of SQL and not
> part of Codd's relational model.

As paul c says in another post in this thread, ordering depends on
domain-specific operators, and adds a lot of extra complexity if the
model is to remain logical and consistent, to the point where that may
not even be possible.

I think you are somehow missing two things here. Firstly, relations are
unordered sets because that is what the mathematical definition of a
relation is, and also because Codd was trying to eliminate order
dependencies which existed in many systems of the time, i.e. he was, in
effect, banning any implementation which forced applications to process
some data in an order determined by the storage method.

Secondly, ordering is conceptually a post-retrieval operation (and yes,
outside the relational model), but there is nothing to forbid being able
to ask the DBMS to return data in a order you specify, which it may do,
either by actual sorting or by (if it can in any particular case)
exploiting its internal storage structures to produce the requested
order more efficiently. There is also nothing to forbid being able to
tell the DBMS to use storage methods that make retrieving certain orders
efficient as long as that does not make any possible order prohibitively
inefficient. That ability is nothing to do with the DBMS's correctness
in following relational theory, it is merely twiddling dials on the
implementation.

> Even if sorting is entered through the back door it remains a tool to
> beautify the output

or, presumably, to specify the processing order for non-relational
operations on a result-relation after you have retrieved it,

> and cannot be exploited while querying, since relations are unordered
> sets.

How do you want to exploit order while querying? "Top n" and analogous
results come from what I have already said ...

> Don't know if grouping and querying groups can be comprehend without
> reference to sorting order,

... and a group is simply a subset of the tuples in a relation for which
some element(s) of each tuple have a particular (set of) values.

> since viewing and analyzing groups as a sorted list
> is easy to communicate and to implement.

I think you are thinking only of algorithms, that is, of process more
than results. A bit like those people who ask "how do I do x" and then
reveal in their answer to "why do you want to do x?" that they should go
back a couple of steps and then do p and q instead, because then they
will get the result they want.

> Hiding these algorithms behind "macro" functions like done in SQL is
> not the problem.

Of course not, algorithms are meant to be hidden in functions. It is not
the problem, but it is not the point either.

>> Ordering and grouping are not part of the data definition, they are
>> operations.
>
> Grouping are part of the data definition:
>
> employes are a set of (teachers, assistents, secretaries ...)
>
> Every set with potential subsets can be defined with groups

Indeed, but I have already said that grouping does not require ordering.
The fact that ordering may be a suitable implementation algorithm is
not relevant because it is happening behind the scenes.

[Eric]

On 2013-09-02, paul c <toledob...@oohay.ac> wrote:
> Forgot to ask: Can anybody point me to the paper "Extending the
> Relational Model to Capture More Understanding"? (author unknown)

Are you perhaps thinking of Codd's "Extending the database relational
model to capture more meaning" (available, for a price, from
http://dl.acm.org/citation.cfm?id=320109, though I think there is a free
copy somewhere)?

[Bob Badour]

On 9/2/2013 8:13 AM, paul c wrote:
> On 02/09/2013 1:07 AM, karl.s...@o2online.de wrote:
>> Codd shifted the complexity from one area to another. When dealing with
>> n entities is replaced with n*x relations is a considerable increase in
>> complexity. One reference told me that a SAP R3 system contains more than
>> 100000 tables (one hundred thousand!). What about complexity?
>
> No. Codd shifted what was common in programs (programs of the 1960's,
> not the 1970's by the way) to the dbms. Obviously no one person nor any
> manageable group of people can decide how redundant SAP is, so SAP must
> be regarded as a failure as far as Codd's idea is concerned. If your
> statistic is accurage, SAP must be regarded as just as much a failure as
> the systems it seeks to replace.

SAP can hardly be held up as a relational example. Those idiots made the
conscious choice to avoid any declarative integrity and codify any
integrity in ABAP applications. The product was originally constructed
to do things entirely 1 record at a time, and it was only after it
became a mature product before they added anything even remotely
set-at-a-time.

On the bright side, the 4-letter table names put an upper bound on the
number of tables.

[Eric]

On 2013-09-02, karl.s...@o2online.de <karl.s...@o2online.de> wrote:

> Am Sonntag, 1. September 2013 18:53:22 UTC+2 schrieb James K. Lowden:

>> On Sun, 1 Sep 2013 03:47:35 -0700 (PDT)
>> karl.s...@o2online.de wrote:
>>> He addresses the following problems
>>> 1.2.1. Ordering Dependence.
>>> "Let us consider those existing systems which either require or
>>> permit data elements to be stored in at least one total ordering
>>> which is closely associated with the hardware-determined ordering of
>>> addresses. "

>> If you remove "hardware-determined" from the sentence, it's exactly as
>> true now as then.

> if you remove hardware-determined from the sentence, then it's no difference
> between pointers and foreign keys.

Impossible. A pointer is a pointer (address, offset, ordinal position),
a foreign key is a value which matches the value of the primary key of
another (or the same) relation.

>>> 1.2.2. Indexing Dependence.
>>> "...destroy indices from time to time will probably be necessary. The
>>> question then arises: Can application programs and terminal
>>> activities remain invariant as indices come and go?..."
>>>
>>> In the seventies "bigdata" had to be stored on sequential data
>>> storages (tapes, cards). Querying data from sequential media cannot
>>> use indices ("indices go").
>>
>> Hmm, no, I'm pretty sure VSAM and IMS were available in the 70s.
>> Cullinet was selling IDMS.
>>
> Maybe! In the 70s storage was very expensive and indices are additional
> costs. Using a model without the need for indices sounds attractive

Indices are not part of the model, they are a possible part of the
implementation.

>>> 1.2.3. Access Path Dependence.
>>> "
>>> One solution to this is to adopt the policy that once a
>>> user access path is defined it will not be made obsolete until
>>> all application programs using that path have become
>>> obsolete. Such a policy is not practical, because the number
>>> of access paths in the total model for the community of
>>> users of a data bank would eventually become excessively
>>> large."
>>>
>>> That statement is based on the hardware of the seventies.
>>
>> On the web I believe it's called "404".
>>
> No! with cheap storage it can be reasonable to keep all user access path,
> at least the policy has to be reconsidered.

It's not just about storage. You have to maintain every path whenever
date is added, deleted, or updated. This in a situation where there is
no upper bound on the number of paths.

>> > For reasons not comprehensible any more (Codd's reference is
>> > out of print and not online available), he restricted his model
>>
>> No mystery. Books in a library are hardly lost texts of Babylon. And
>> he states his motivation plainly: "the possibility of eliminating
>> nonsimple domains appears worth investigating!"
>>
> Without any objective reasons this seems to be a personal opinion to
> me. I don't object for all cases, but using a unchecked rule without
> considering potential alternatives is not a good thing.

If you were meaning footnote 4 after "worth investigating", it refers
to a person. If you then look at the end of the paper, that person is
thanked for "helpful discussions", there is no reference. If not, what
reference did you mean? Footnote 4 does nothing for the argument either
way. Codd investigates what is worth investigating in this paper, it has
been thoroughly discussed since, and there were no remaining arguments
(other than how far can you go with normalization) until the arrival of
object proponents who didn't understand the relational model properly.

>> The model is not "restricted". It is *simplified*, a feature, not a
>> bug. By showing -- more, *proving* -- that logical inferences could be
>> drawn from data manipulated with a small number of operators closed
>> over a domain, Codd released programmers from low-level complexity and
>> man-centuries of work.
>>
> Codd shifted the complexity from one area to another. When dealing with
> n entities is replaced with n*x relations is a considerable increase in
> complexity. One reference told me that a SAP R3 system contains more than
> 100000 tables (one hundred thousand!). What about complexity?

SAP does what it says on the tin. It says a lot of things on the tin.
However there is no particular reason to hold SAP up as a good example
of relational design.

>> If you're programming a computer, graphs are a your natural ally
>> because they can be mapped directly onto the computer's memory. They're
>> of no use, though, when you want to manage data logically. How, for
>> example, do you define a subset of a cyclic graph?
>>
> Depends on the subset definition. In our field of work we use a ordered
> list representation of graphs. A subset is a range of items satisfying
> same criterias.

Confusing implementation and concept again.

>> You're right to say that graphs are more complex than relations. It's a
>> mistake, though, to conclude therefore that they are more powerful.
>> It's been proved mathematically that graphs and relations are
>> interchangeable in the sense that they can represent the same
>> information. The difference is that relational theory is much
>> simpler. That's its advantage, not a handicap.
>>
> It really a handicap. Try to express grouping and grouped aggregates
> in a frame with only "unordered sets". SQL deviates from the relational
> theory with implementing groups and grouped aggegats.

Did this one in another post.

>> OK, but it's not.
>> Consider the UNIX filesystem, for instance, which you refered to
>> earlier. Upon a time, when my mother wrote disk access routines for
>> Univac, the programmer had to know all the particulars of the device,
>> and read/write data in terms of the device's design. Unix
>> revolutionized the field by abstracting all disk access into today's
>> familar stream of bytes. No addresses, no heads or sectors or tracks.
>> A catalog to facilitate sharing that anyone (potentially) can update,
>> not just the system programmers. Works pretty good for nonrotating
>> media, too, and over the network. And not an object in sight.
>>
> That depends on your object definition. For me a object is a data type
> with addition internal procedures, identity and means to communicate
> (signals, messages...) and can be implemented in various manors. Unix
> files fit in this description.

No. A file doesn't behave like an object. You might see object
parallels in the implementation, but that's different, it's not visible
to the user or application.

>> ... I have long thought that
>> stored procedures are to databases what methods are to objects, and
>> subscribe to the idea that applications should access the data only
>> through views and procedures.
>>
> Ok!

Not altogether (but that would be a separate discussion).

>> Part of your critique is actually of DBMSs that we have, not of RM.
>> SQL DBMSs largely support only a few primitive types that the user may
>> then further constrain or write functions for. One cannot, for
>> example, define an aggregate type as a set of columns, and use that
>> name in, say, FK declarations. Nor can we usually define types of blobs
>> and comparison functions for them (although I'm unconvinced that's a
>> good idea).
>
> On the contrary! My critic is on RM and not on the real SQL databases. My
> only critic on SQL is to pretend to be relational. My critic on RM can
> be summarized in the statement "relations are unordered sets". Any model
> based on a definition like "relations are sets with any (exploitable)
> sort order" is much better.

Yet again, the RM is _not_ about ordering, it does not ban ordering you
can use, merely ordering that you are forced to use. Please don't quote
Codd's paper again, it is not an absolute truth or even an attempt at one,
it describes a theory which, used in an appropriate manner, provides a
useful and logically correct way to manipulate and present data to an
application. If it needs to be extended, fine, but if you extend it by
breaking it then it is no longer the RM, and it is likely to no longer
be logically correct.

[com...@hotmail.com]

On Monday, September 2, 2013 2:25:43 PM UTC-7, Eric wrote:
> On 2013-09-02, paul c

> > Forgot to ask: Can anybody point me to the paper "Extending the

> I think there is a free copy somewhere)?

google "codd extending the relational model pdf"
qv http://en.wikipedia.org/wiki/Relational_Model/Tasmania

philip

[James K. Lowden]

On Mon, 2 Sep 2013 21:45:34 +0100
Eric <er...@deptj.eu> wrote:

> Codd was trying to eliminate order
> dependencies which existed in many systems of the time, i.e. he was,
> in effect, banning any implementation which forced applications to
> process some data in an order determined by the storage method.

Just to expand on that well placed point, the issue isn't so much
"process" as "specify". You might well want to process something in
some particular order. But how do you want to specify it to the DBMS?

Codd sought to relieve us of the need to know something about the data
not intrinsic to the data.

In days of yore -- which sadly we seemed destined to return to -- it
was common to have the data stored "in order"

1 A
2 B
3 C
4 D

Here, the second column is data, and the first column is the navigation
number supplied by the system on insertion, and used by the application
on retrieval. The program specified not "B" but instead "record 2".
This was handy because no one had invented auto-incrementing columns
yet! But there was a problem with deletion

1 A
2
3 C
4 D

To preserve C's location -- which we know is 3, right? -- it was
necessary to preserve B's location, even though B had been evicted.
Then, when retrieving the whole [1,4] set, it was necessary to mention
somehow that B's place was there, but not B. That's the not 2 B Hamlet
was worried about.

OTOH there were advantages some still seek today. You could easily
have

1 B
2 B
3 B
4 B

and not get tangled up with pesky primary keys and the like.

Now arises the question, though: which B? Codd recognized that the
only thing intrinsic about the data were they data themselves. All
those B's are the same, so

1 B

will do, or, better

B

If you really care there were four B's, count them

B 4

Here "4" is a quantity, not an instrument of navigation. You can find
all the things whose quantity is 4, and you can remove a B by
decrementing the quantity.

By eliminating the order, Codd removed extraneous nondata from the
system, and permitted the programmer to access the data strictly on
their terms: by value. And lo, it works with the first example just as
well

A
B
C
D

and, happily, deletion works pretty well, too

A
C
D

Michaelangelo produced David by starting with a block of marble and
removing the parts that weren't David. That's what Codd did with
database theory.

Lots of things are missing from RM. That's the point.

--jkl

[vldm10]

Dana utorak, 3. rujna 2013. 06:02:38 UTC+2, korisnik James K. Lowden napisao je:
> On Mon, 2 Sep 2013 21:45:34 +0100
>
>
>
>
>

Here "4" is not a quantity, "4" is similar to "3". "3" is an identifier. Maybe the following example will shed more light on what this is all about.

https://www.youtube.com/watch?v=MUcYSR-Xl_E&feature=related

>
> all the things whose quantity is 4, and you can remove a B by
>
> decrementing the quantity.
>
>
>
> By eliminating the order, Codd removed extraneous nondata from the
>
> system, and permitted the programmer to access the data strictly on
>
> their terms: by value. And lo, it works with the first example just as
>
> well
>
>
>
> A
>
> B
>
> C
>
> D
>
>
>
> and, happily, deletion works pretty well, too
>
>
>
> A
>
> C
>
> D
>
>
>
> Michaelangelo produced David by starting with a block of marble and
>
> removing the parts that weren't David. That's what Codd did with
>
> database theory.

What Codd did not do that is that he did not say that he copied Gottlob Frege. The Relational Model is copy of Frege's theory. And this is a bad copy, because Codd did not understand completely Frege's theory. Note that I am speaking here only about Relational Model, I'm not talking about Relational Algebra and Functional Dependencies.

Michaelangelo is an authentic genius as it was Gottlob Frege. Code is far from these two genius. Frege did these things 120 years ago, when there were no computers. In fact Frege's theory is much more general than the RM.

>
>
>
> Lots of things are missing from RM. That's the point.
>
>
>
> --jkl

Vladimir Odrljin

[Jan Hidders]

On 2013-09-01 18:37:17 +0000, Norbert_Paul said:

> The paper does not mention the spatial databasers. What are their
> historical part of the NoSQL movement?

I don't think has played a big part in practice, although the NoSQL
advocates nodoubt will be happy to use it as ammunition when arguing
that there are inadequacies to be found in the relational model and/or
SQL.

> For example http://www.spatial.maine.edu/~max/RJ6.html not only argues
> against actual shortcomings of SQL1 in the standard of 1898
> http://www2.yk.psu.edu/~lxn/IST_210/sql1_versus_sql2.html
> but also against the relational model. However, I simply don't get
> some of the arguments therein:

[.. SNIP ..]

Neither do I, and my compliments for the analyis. Am I right in reading
some irritation between the lines that goes beyond a natural distaste
for shoddy reasoning?

What to say about these things? Yes, it is unfortunate that in this
type of papers, where certain languages or models are bashed and other
are promoted, for some reason all basic scientific principles seem to
go out of the window and all reasoning becomes more or less based on
intuitive gut-feeling and uncritically accepting communis opinio from
the community that the writer is publishing in. Science should be about
critically and skeptically investigating claims or providing good
evidence for them that convinces the informed intelligent but skeptical
reader. But there are practical reasons that I'm all too familiar with
for why for real-world database researcher that bar is often a bit too
high.

For the record, as you know I'm also very critical of the relational
camp around Date, Fabian etc., and in fact of the opinion that they
also commit offences against proper scientific etiquette that are at
least as bad.

PS. Nice to see that there seems to be a little bit of life coming back
to comp.databases.theory. :-)

-- Jan Hidders

[Eric]

On 2013-09-03, vldm10 <vld...@yahoo.com> wrote:
> Dana utorak, 3. rujna 2013. 06:02:38 UTC+2, korisnik James K. Lowden napisao je:

...

>> If you really care there were four B's, count them
>>
>> B 4
>>
>> Here "4" is a quantity, not an instrument of navigation. You can find
>
> Here "4" is not a quantity, "4" is similar to "3". "3" is an
> identifier. Maybe the following example will shed more light on what
> this is all about.
>
> https://www.youtube.com/watch?v=MUcYSR-Xl_E&feature=related

Smart dogs notwithstanding, 4 is a quantity. We understand quantities,
they are what we get when we count things, and that is the context of
the statement you are arguing with. For the dog "3" is an identifier,
rather that the ordinal we might have expected, but that's in a
different context, and therefore irrelevant.

...

> What Codd did not do that is that he did not say that he copied Gottlob
> Frege. The Relational Model is copy of Frege's theory. And this is a bad
> copy, because Codd did not understand completely Frege's theory.

If it's a bad copy, maybe it's not a copy at all. Are you saying that
Codd deliberately copied the theory and deliberately chose to not
acknowledge it?

...

> In fact Frege's theory is much more general than the RM.

And therefore not necessarily appropriate to data management. In fact,

>> Lots of things are missing from RM. That's the point.

[Norbert_Paul]

Jan Hidders wrote:
> On 2013-09-01 18:37:17 +0000, Norbert_Paul said:
>
>> The paper does not mention the spatial databasers. What are their
>> historical part of the NoSQL movement?
>
> I don't think has played a big part in practice, although the NoSQL
> advocates nodoubt will be happy to use it as ammunition when arguing
> that there are inadequacies to be found in the relational model and/or SQL.

The author is one of (maybe /the/)the most influential protagonists in
spatial data and Geoinformation. Whereas the paper may not have been
important the author surely is.

>> For example http://www.spatial.maine.edu/~max/RJ6.html not only argues
>> against actual shortcomings of SQL1 in the standard of 1898
>> http://www2.yk.psu.edu/~lxn/IST_210/sql1_versus_sql2.html
>> but also against the relational model. However, I simply don't get
>> some of the arguments therein:
>
> [.. SNIP ..]
>
> Neither do I, and my compliments for the analyis. Am I right in reading
> some irritation between the lines that goes beyond a natural distaste
> for shoddy reasoning?

Yes. The irriatation is based on bad personal experience within the
last three years. For example a (deliberate?) publication delay of
more than three years. There seems to be an extremely influential group
of researchers around the author where this kind of shoddy (thanks for
the new vocabulary) work is the only one accepted. Have a look at
"alternating hierarchical decomposition" (AHD). It just doesn't work
in 3D. Or consider "topological relations" and "nine intersections".
Each paper has its own definition of "region". Still all claim that
only eight (or nine) such intersections can occur on "regions" but
most papers have no valid proof.

> What to say about these things? Yes, it is unfortunate that in this type
> of papers, where certain languages or models are bashed and other are
> promoted, for some reason all basic scientific principles seem to go out
> of the window and all reasoning becomes more or less based on intuitive
> gut-feeling and uncritically accepting communis opinio from the
> community that the writer is publishing in. Science should be about
> critically and skeptically investigating claims or providing good
> evidence for them that convinces the informed intelligent but skeptical
> reader. But there are practical reasons that I'm all too familiar with
> for why for real-world database researcher that bar is often a bit too
> high.

The problem is, when evidence is given, reviewers say that this is "too
abstract and technical" or they have an article wait forever to become
published. On the other hand, when a scientist's position is secured he
can make /any/ claim without any evidence. I thought this could be an
alternative channel here to post something critical. There is no
reviewing on comp.databases.theory.

> For the record, as you know I'm also very critical of the relational
> camp around Date, Fabian etc., and in fact of the opinion that they also
> commit offences against proper scientific etiquette that are at least as
> bad.

I will look up Date and Fabian. Are they influential?

> PS. Nice to see that there seems to be a little bit of life coming back
> to comp.databases.theory. :-)

OK. Let's start the party.


Norbert Paul

[Jan Hidders]

Like this?:

http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5293499&tag=1

So the claim in the abstract about "scalable to any dimension" is wrong?

>
>> What to say about these things? Yes, it is unfortunate that in this type
>> of papers, where certain languages or models are bashed and other are
>> promoted, for some reason all basic scientific principles seem to go out
>> of the window and all reasoning becomes more or less based on intuitive
>> gut-feeling and uncritically accepting communis opinio from the
>> community that the writer is publishing in. Science should be about
>> critically and skeptically investigating claims or providing good
>> evidence for them that convinces the informed intelligent but skeptical
>> reader. But there are practical reasons that I'm all too familiar with
>> for why for real-world database researcher that bar is often a bit too
>> high.
>
> The problem is, when evidence is given, reviewers say that this is "too
> abstract and technical" or they have an article wait forever to become
> published. On the other hand, when a scientist's position is secured he
> can make /any/ claim without any evidence. I thought this could be an
> alternative channel here to post something critical. There is no
> reviewing on comp.databases.theory.

Indeed. But also not much of an audience. But who knows, it might come,
and it's easier and less demanding (usually :-)) then a blog.

>> For the record, as you know I'm also very critical of the relational
>> camp around Date, Fabian etc., and in fact of the opinion that they also
>> commit offences against proper scientific etiquette that are at least as
>> bad.
>
> I will look up Date and Fabian. Are they influential?

My mistake. I meant Chris Date and Fabian Pascal. In the database
research community they have no discernable influence as far as I can
tell.

-- Jan Hidders

[Jan Hidders]

On 2013-09-03 19:46:22 +0000, Eric said:

> On 2013-09-03, vldm10 <vld...@yahoo.com> wrote:
>> Dana utorak, 3. rujna 2013. 06:02:38 UTC+2, korisnik James K. Lowden
>> napisao je:
> ...
>

>> What Codd did not do that is that he did not say that he copied Gottlob
>> Frege. The Relational Model is copy of Frege's theory. And this is a bad
>> copy, because Codd did not understand completely Frege's theory.
>
> If it's a bad copy, maybe it's not a copy at all. Are you saying that
> Codd deliberately copied the theory and deliberately chose to not
> acknowledge it?

Codd explicitly referred to first-order logic in his work, so the link
with Frege was clearly made. A reference to Tarski would have been
nice, though. :-) The clear link notwithstanding, the relational model
cannot be meaningfully described or thought of as a copy of FOL, either
bad or good, since it has a very different purpose and makes entirely
different claims about its purpose and utility. To confuse the two is
misunderstanding entirely what database theory and database models are
about.

-- Jan Hidders

[Eric]

Well put. Thankyou.

[paul c]

On 04/09/2013 9:35 AM, Jan Hidders wrote:
> Codd explicitly referred to first-order logic in his work, so the link
> with Frege was clearly made. A reference to Tarski would have been nice,
> though. :-) The clear link notwithstanding, the relational model cannot
> be meaningfully described or thought of as a copy of FOL, either bad or
> good, since it has a very different purpose and makes entirely different
> claims about its purpose and utility. To confuse the two is
> misunderstanding entirely what database theory and database models are
> about.

To add to that, I like this quote from his 1979 paper which I think
isn't controversial (unlike some other parts of that paper) and which I
think makes it pretty clear that his model (or models if you like) is in
part an application of FOL. If one agrees that it is indeed an
application, that helps underline that it's not an alternative to FOL as
well as not being some other things such as an implementation. I've
re-paragraphed it to emphasize the last sentence of the first paragraph
which I think is one of his most basic breakthroughs:

<quote ...
Suppose we think of a database initially as a set of formulas in
first-order predicate logic. Further, each formula has no free variables
and is in as atomic a form as possible (e.g, A & B would be replaced by
the component formulas A, B). Now suppose that most of the formulas are
simple assertions of the form Pub . - . z (where P is a predicate and a,
b, . . . , z are constants), and that the number of distinct predicates
in the database is few compared with the number of simple assertions.
Such a database is usually called formatted, because
the major part of it lends itself to rather regular structuring. One
obvious way is to factor out the predicate common to a set of simple
assertions and then treat the set as an instance of an n-ary relation
and the predicate as the name of the relation.

A database so structured will then consist of two parts: a regular part
consisting of a collection of time-varying relations of assorted degree
(this is sometimes called the extension) and an irregular part
consisting of predicate logic formulas that are relatively stable over
time (this is sometimes called the intension, although it may not be
what the logicians Russell and Whitehead originally intended by this
word). One may also view the intension as a set of integrity constraints
(i.e., conditions that define all of the allowable extensions) and thus
decouple these notions from variability with time.
...
Whether the open or closed interpretation is adopted, the relational
model is closely related to predicate logic. It is this closeness which
accounts for the plethora of relational data sublanguages that are based
on predicate logic.
... end quote>

[Norbert_Paul]

Jan Hidders wrote:
> On 2013-09-04 05:28:01 +0000, Norbert_Paul said:
>>> Neither do I, and my compliments for the analyis. Am I right in reading
>>> some irritation between the lines that goes beyond a natural distaste
>>> for shoddy reasoning?
>>
>> Yes. The irriatation is based on bad personal experience within the
>> last three years. For example a (deliberate?) publication delay of
>> more than three years. There seems to be an extremely influential group
>> of researchers around the author where this kind of shoddy (thanks for
>> the new vocabulary) work is the only one accepted. Have a look at
>> "alternating hierarchical decomposition" (AHD). It just doesn't work
>> in 3D.
>
> Like this?:
>
> http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5293499&tag=1
>
> So the claim in the abstract about "scalable to any dimension" is wrong?

Yes, exactly. Bulbul makes all proofs and examples in 2D. There, indeed,
all seems to work. But Take a 3D cube, say [-1,1]^3, and its center point,
say (0,0). Partiton that cube into the six pyramids with each cube face as
its bottom and the center point as its apex (convec closure of a face and
the center). Then remove three pyramids making sure that two of them
are opposite. Apply the AHD and learn that it recurses endlessly.
Reason: The convex closure of the shape is equal to the convex closure
of its "differece shape" (aka difference to convex closure).

An Ascii garphics illustration:

*-------------------*
/ /|
/ / |
/ / |
+-------------------+ |
| . . | |
| . . | |
| * | |
| / \ | +
| / \ | /
| / \ | /
|/ \|/
+ +


Example could not be published in 3D GeoInfo 2012.
Reviewer: "good example, but explain why does it not terminate. What
is the problem with their algorithm?"
OK: The algorithm is so ill specified that you cannot establish a
strict counter example for it. So I refrained from claimig that it will
not work. Therefore I have just written

``However, with the shape
$$\begin{picture}(75,90)
\setlength{\unitlength}{1.5\unitlength}
\put(0,10){\line(2,1){20}}
\put(0,10){\line(5,4){25}}
\put(0,10){\line(0,1){40}}

\put(0,50){\line(5,-4){25}}
\put(0,50){\line(3,-1){30}}
\put(0,50){\line(2,1){20}}

\put(20,20){\line(1,2){5}}

\put(20,60){\line(3,-1){30}}

\put(25,30){\line(1,-6){5}}
\put(25,30){\line(1,2){5}}

\put(30,0){\line(2,1){20}}
\put(30,0){\line(0,1){40}}

\put(30,40){\line(2,1){20}}

\put(50,10){\line(0,1){40}}
\end{picture}$$
that method may not terminate.''

to avoid backdoor arguments that sloppy specifications always leave open
in the sense of "But I meant it that way ... ".

>> The problem is, when evidence is given, reviewers say that this is "too
>> abstract and technical" or they have an article wait forever to become
>> published. On the other hand, when a scientist's position is secured he
>> can make /any/ claim without any evidence. I thought this could be an
>> alternative channel here to post something critical. There is no
>> reviewing on comp.databases.theory.
>
> Indeed. But also not much of an audience. But who knows, it might come,
> and it's easier and less demanding (usually :-)) then a blog.

Google archives Usenet. Maybe one day ...

>> I will look up Date and Fabian. Are they influential?
>
> My mistake. I meant Chris Date and Fabian Pascal. In the database
> research community they have no discernable influence as far as I can tell.

Then it is not interesting to critizise them. My problem is not that
someone is wrong. Then just live and let live. The problem is when
seems as if (Note, "as if" doesn't say this indeed happens) a bunch of
ignorants managed to establish kind of a trust.

Norbert

[Jan Hidders]

Impressive.

Is the description in the following not precise enough? (Only looked
briefly myself, busy day tomorrow, sorry.)

http://www.geocomputation.org/2009/PDF/Bulbul_et_al.pdf

Btw. it might be nice to see if there is no easy fix for this, like for
example cutting the difference shape in two. Intutively my first
inclination would be to base the proof of termination on the number of
vertices being reduced, which indeed in your conterexample does not
happen. This raises the question of how to do this optimally, for
certain definiitons of optimal (least number of nodes in the tree?).

>>> The problem is, when evidence is given, reviewers say that this is "too
>>> abstract and technical" or they have an article wait forever to become
>>> published. On the other hand, when a scientist's position is secured he
>>> can make /any/ claim without any evidence. I thought this could be an
>>> alternative channel here to post something critical. There is no
>>> reviewing on comp.databases.theory.
>>
>> Indeed. But also not much of an audience. But who knows, it might come,
>> and it's easier and less demanding (usually :-)) then a blog.
>
> Google archives Usenet. Maybe one day ...

Let's hope so. So let the record show you have also an arxiv paper on
this: http://arxiv.org/abs/1205.5691

Btw. it has a few typos, you might want to quickly fix those now. ;-)
Like "however, can expensive in dimension higher than 2"

-- Jan Hidders

[James K. Lowden]

On Wed, 04 Sep 2013 07:28:01 +0200
Norbert_Paul <norbertpau...@yahoo.com> wrote:

> There seems to be an extremely influential group of researchers
> around the author where this kind of shoddy (thanks for the new
> vocabulary) work is the only one accepted.

"Shoddy" is an interesting word, too. It gained its current meaning
during the American Civil War, when uniforms and blankets made of
shoddy were roundly despised.

http://www.historyhouse.co.uk/articles/shoddy.html
http://www.maine.gov/tools/whatsnew/index.php?topic=arcsesq&id=125216&v=article

--jkl

[Norbert_Paul]

Jan Hidders wrote:
> On 2013-09-04 21:00:22 +0000, Norbert_Paul said:
>> ``However, with the shape
>> $$\begin{picture}(75,90)

...

>> \end{picture}$$
>> that method may not terminate.''
>>
>> to avoid backdoor arguments that sloppy specifications always leave open
>> in the sense of "But I meant it that way ... ".
>
> Impressive.
>
> Is the description in the following not precise enough? (Only looked
> briefly myself, busy day tomorrow, sorry.)
>
> http://www.geocomputation.org/2009/PDF/Bulbul_et_al.pdf

Is apply DecompositionTree equal to apply AHD? I had another publication
at hand. Actually I tried to understand it in a way that noe doubt whatsoever
on every algorithm's detail is left. As not all was perfectly clear to me,
as an engineer, I simply decided to err at the safe side in avoiding the
explicit claim that the algorithm does not always terminate. Hence I wrote
"may" not terminate.

> Btw. it might be nice to see if there is no easy fix for this, like for
> example cutting the difference shape in two. Intutively my first
> inclination would be to base the proof of termination on the number of
> vertices being reduced, which indeed in your conterexample does not
> happen. This raises the question of how to do this optimally, for
> certain definiitons of optimal (least number of nodes in the tree?).

Cutting shapes is not a good idea in 3D and needs much care.
Google for "Chazelle convex partitions of polyhedra".

>>> Indeed. But also not much of an audience. But who knows, it might come,
>>> and it's easier and less demanding (usually :-)) then a blog.
>>
>> Google archives Usenet. Maybe one day ...
>
> Let's hope so. So let the record show you have also an arxiv paper on
> this: http://arxiv.org/abs/1205.5691
>
> Btw. it has a few typos, you might want to quickly fix those now. ;-)
> Like "however, can expensive in dimension higher than 2"

Thanks for the "be". I'll do so later.

Yes that was my alternative approach:

When I developped it, I was unaware of Bulbul's work. Later on searching
related work so as to please a reviewer I stumbled over it and found its
weakness.

My paper was rejected on two conferences on GIS and spatial data for being
"extremely theoretical and ... not ... appealing ... to a large audience",
"very difficult", and, interestingly, not GIS-related enough whereas AHD
got published on a GIS-conference. I then published the approach in a
conference on computation for building and construction. This is my
home-community. Civil engineers reject mathematical statements iff (no typo)
they are wrong.


Norbert

[vldm10]

Dana srijeda, 4. rujna 2013. 18:35:36 UTC+2, korisnik Jan Hidders napisao je:

Hi Jan,

> Codd explicitly referred to first-order logic in his work, so the link
>
> with Frege was clearly made. A reference to Tarski would have been
>
> nice, though. :-) The clear link notwithstanding, the relational model
>
> cannot be meaningfully described or thought of as a copy of FOL, either
>
> bad or good, since it has a very different purpose and makes entirely
>
> different claims about its purpose and utility. To confuse the two is
>
> misunderstanding entirely what database theory and database models are
>
> about.
>
>
>
> -- Jan Hidders

I think it is necessary to clearly specify the model and exactly what people are doing when they use the model. Given that this is a very important and broad topic, I'll simplify this, in order to get a better explanation what the model is. Unfortunately on this topic has no literature.
When a man does some complex projects in the real world he uses the model. Models are used extensively in architecture, engineering, databases etc.. Schematically, this can be represented as the following triplet:

(*) Man Model RealWorldProject

Here, the model could be anything. But the most important and most powerful type models are mathematical models. For example, the model may be a partial differential equation that describes some of the technical-physical project. The model can be Boolean algebra that models the design of electronic circuits. Obviously this type of model is pure mathematics, these models are not a technical or special models flavored with mathematics. Many database theorists for many parts of db theory claim that it is math. I also think it is math, in fact I think that's part of the new math that has a great future.
So I think that the Relational Model is pure mathematics, which includes substantial mathematical theories: Propositional Logic, Predicate Logic, Relations, Sets, Formal theory of spoken languages, Semantics …
In the last 50-60 years, the mathematical theory of “semantic models” was developed dramatically. In this area, the most important is the Model Theory. Many famous mathematicians claim that this is the highest achievement in mathematics. The father of semantics is G. Frege and the father of the Model Theory is A. Tarski.
The relational model is a specific semantic model that fully describes how works the following triplet: Man-Model-RealWorldProject.
============================
In this thread I claim: everything that exists in the relational model exists in the works of G. Frege. If you think there is something that belongs to the Relational model and does not belong to the Frege's theory, then you should put it on this topic. Of course, in this case, my statement is not true. Otherwise, my statement is correct.
I've never seen that Codd mentions Frege, that's why I wrote "What Codd did not do that is that he did not say that he copied Gottlob Frege."
=================================
However, I think that Codd is very important for theory of database, primarily because he has used Frege's theory as Relational Model, and thus he raised the theory of database to the highest level, that is, on the level of mathematical theory. Codd also has significant results in the field of FDs. I also think his results in Relational Algebra are important, but I've learned on this user group that Tarski has done a lot in this area. I guess you mentioned Tarski in this regard.

Vladimir Odrljin

[Jan Hidders]

Also "dimension higher than 2" does not sound correct to me. It's the
number of dimensions that is higher, not a single dimension. I would say
"can be expensive for more than 2 dimensions".

-- Jan Hidders

[Jan Hidders]

On 2013-09-06 11:22:21 +0000, vldm10 said:

> Dana srijeda, 4. rujna 2013. 18:35:36 UTC+2, korisnik Jan Hidders napisao je:
>
> Hi Jan,
>> Codd explicitly referred to first-order logic in his work, so the
>> link>> with Frege was clearly made. A reference to Tarski would have
>> been>> nice, though. :-) The clear link notwithstanding, the relational
>> model>> cannot be meaningfully described or thought of as a copy of
>> FOL, either>> bad or good, since it has a very different purpose and
>> makes entirely>> different claims about its purpose and utility. To
>> confuse the two is>> misunderstanding entirely what database theory and
>> database models are>> about.
>

> I think it is necessary to clearly specify the model and exactly what
> people are doing when they use the model. Given that this is a very
> important and broad topic, I'll simplify this, in order to get a better
> explanation what the model is. Unfortunately on this topic has no
> literature.
> When a man does some complex projects in the real world he uses the
> model. Models are used extensively in architecture, engineering,
> databases etc.. Schematically, this can be represented as the following
> triplet:
>
> (*) Man Model RealWorldProject
> Here, the model could be anything. But the most important and most
> powerful type models are mathematical models. For example, the model
> may be a partial differential equation that describes some of the
> technical-physical project. The model can be Boolean algebra that
> models the design of electronic circuits. Obviously this type of model
> is pure mathematics, these models are not a technical or special models
> flavored with mathematics. Many database theorists for many parts of db
> theory claim that it is math. I also think it is math, in fact I think
> that's part of the new math that has a great future.

If you define the Relational Model as just the datamodel, then you
would be right that there was not much new there. But Codd was not just
proposing a data model, but rather a set of engineering principles for
building effective generic DBMSs. The data model is only a part of
that, and the new insight and/or claim is here that this is in
practice an effective model for the intended purpose. This is not a
mathematical claim but an engineering claim. So even though it has
strong links with certain mathematical theories, to think of the thing
itself as a mathematical theory is to entirely miss the point.

-- Jan Hidders

[Norbert_Paul]

Jan Hidders wrote:
>> Thanks for the "be". I'll do so later.
>
> Also "dimension higher than 2" does not sound correct to me. It's the
> number of dimensions that is higher, not a single dimension. I would say
> "can be expensive for more than 2 dimensions".

No, singular is correct. In topology (as in linear algebra) you spek of
"the dimension" (singular) of a space:
http://en.wikipedia.org/wiki/Dimension

But, actually, in topology there are more than one dimensions. For example:
- small inductive dimension "ind"
- large inductive dimension "Ind"
- Krull dimension "dim"
- covering dimension "dim"

But still, singular is correct, because for topological /data/ the Krull
dimension is the most adequate, and it is the dimension notion I use.
The Krull dimension is then sometimes called the "combinatorial dimension".
Alas, the English Wikipedia article (and also
http://nl.wikipedia.org/wiki/Krull-dimensie ) only mentions its /algebraic/
definition. Its topological counterpart is presented in the corresponding
German, and French Wikipedia articles.

Norbert

[Jan Hidders]

Norbert_Paul <norbertpau...@yahoo.com> wrote:
> Jan Hidders wrote:
>>> Thanks for the "be". I'll do so later.
>>
>> Also "dimension higher than 2" does not sound correct to me. It's the
>> number of dimensions that is higher, not a single dimension. I would say
>> "can be expensive for more than 2 dimensions".
>
> No, singular is correct. In topology (as in linear algebra) you spek of
> "the dimension" (singular) of a space:
> http://en.wikipedia.org/wiki/Dimension

RIght. I stand corrected. Thanks. Didn't know that.

> But, actually, in topology there are more than one dimensions. For example:
> - small inductive dimension "ind"
> - large inductive dimension "Ind"
> - Krull dimension "dim"
> - covering dimension "dim"
>
> But still, singular is correct, because for topological /data/ the Krull
> dimension is the most adequate, and it is the dimension notion I use.
> The Krull dimension is then sometimes called the "combinatorial dimension".
> Alas, the English Wikipedia article (and also
> http://nl.wikipedia.org/wiki/Krull-dimensie ) only mentions its /algebraic/
> definition. Its topological counterpart is presented in the corresponding
> German, and French Wikipedia articles.

What's stopping you from writing that into the English version? :-)

-- Jan Hidders

[Norbert_Paul]

Jan Hidders wrote:
>
> What's stopping you from writing that into the English version? :-)

Maybe later, I'll do so.
I considered that, too.

[vldm10]

Dana srijeda, 4. rujna 2013. 18:35:36 UTC+2, korisnik Jan Hidders napisao je:

> On 2013-09-03 19:46:22 +0000, Eric said:
>
>
>

> Codd explicitly referred to first-order logic in his work, so the link
>
> with Frege was clearly made.
>

> -- Jan Hidders

Gottlob Frege is a scientist, who in the last ten years, emerges to the surface as a scientist of great importance. Lately, more and more scientists reveals its importance in mathematics, philosophy and other sciences. Some of the most famous scientists proclaim him as the greatest mathematician and philosopher, so far.
Here I will briefly mention his work, which determines the relational model as a semantic model:

1) The theory of language.
a) The formalization of spoken languages
b) Formal languages for Propositional Logic and FOL
c) Semantics. This part is actually completely new in the history of science and has strong influence on the work of A. Tarski (Model Theory) and R. Montague. (The formalization (syntax and semantic) of spoken languages.)
d) Philosophy of language - introduces a whole new realm of philosophy.

2) Complete construction of the Propositional logic
a) Formalization of Propositional Logic
b) Frege introduces the following algorithm: the truth-value of a proposition is
determined by its form and the truth-values of its atomic constituents.
Here the form is a complex proposition which joins its atomic propositions using the five logical connectives. The constituents are atomic propositions.

Note that Frege defined that the meaning of a sentence is its truth value. In this way he connected logic and semantic at the level of sentences.
Frege was the first who begin to develop a technique for the proof theory. He was representing propositions as trees. As far as I know, Wittgenstein introduced truth-tables. However, as I already mentioned Frege introduced the algorithm for the complex propositions; he was using trees. Note that there is no a proof for truth-tables.

3) Frege developed completely FOL. He introduced formal language for FOL and he developed philosophy for spoken languages. I will mention only the part which is related to Predicate Calculus. I will briefly described Frege’s ideas that are related to sentences, names and predicates.
I already mentioned that the meaning of a sentence is its truth value. The meaning of names was elaborated in Frege’s deep theory about names. This theory has changed many things in science. So if somebody says that E. Codd was the first who introduced the new representation of the relations by using names, then it is the false sentence. Even more, it seems to me that this claim implies that E. Codd in fact introduced relationships between names and predicates.

I will explain that this is a part of Frege’s theory. Frege introduces the following:

(i) Saturated expressions;
Here we have grammatical categories Names, (N is shortcuts for a name) and
Sentences (S is shortcuts for a sentence)

(ii) Unsaturated expressions
There are many types of unsaturated expressions, which have one or more “gaps”.
If the gaps were filled in with an expressions and if they produce expressions of type S then these unsaturated expressions we call predicates.

The notation that is suitable for a formal work with these gaps and with other Frege’s ideas can be represented with the following scheme:
===============================
(1) S/N, S/NN, S/NNN , … This kind of types Frege called relations.


(2) Second-level, predicates
S/(S/N), S/(S/N)(S/N), S/(S/N)(S/N)(S/N).….


(3) Third-level predicates
S/(S(S/N)),….
……etc
================================
Of course, in this representation, there are cross-combinations.
What is important here for the Relational model is the case (1). The types from case (1), Frege called relations, and case (1) completely defines the relational model as a semantic model.

Definition of predicate: A predicate, in a spoken language, is an incomplete phrase with specified gaps such that when gaps filled with names of entities the phrase becomes a proposition.
------------------------
In the relational model, we do not use the gaps, but the names (of attributes).
Now we can introduce atomic predicates by using atomic propositions. The conjunction of atomic propositions is commutative. From this fact follows that the relation can be represented by the names of the attributes and order of attributes (columns) is irrelevant.
-------------------------
Above written notation was done by Polish mathematician K. Ajdukiewicz. This is a formalization of Frege's theory of linguistic structures - the part that refers to the predicate, sentences and names and its logic and semantic laws.
This issue of the names and predicates is important also because it is part of a more general theory given by Frege. Note that philosophy is divided on two huge parts: the realm of the external world and the realm of the purely mental.
Today it is clear that Frege introduced the third realm, that is, the realm of semantics that connects two mentioned realms. Therefore, the claim that E. Codd first introduced relations, whose values refer to corresponding names of attributes, reaches far into the substance of the Frege's theory.
With regard to what is shown above in this post, it is clear that the claim that E. Codd first introduced representation of relations, using the name of the attributes is not correct. Note that the claim about "Codd's Relations" can be found (even) in some textbooks, which are at university level?!
----------------------
Mentioned notation was used by the Polish mathematician Ajdukiewicz for Categorical grammar (in 1935). Montague grammar (syntax) also used Categorical grammar.
I write about this in order to emphasize the importance, the power and influence of Frege's work.

Note that the semantic here(in RM) is realized just by using the language constructs. For example, in my data model the semantic is based on concepts.

There are also questions about (maybe) some limitations of predicate logic related to formalization of the corresponding propositions? For example for tensed propositions.

Vladimir Odrljin

[James K. Lowden]

On Mon, 23 Sep 2013 15:27:04 -0700 (PDT)
vldm10 <vld...@yahoo.com> wrote:

> Dana srijeda, 4. rujna 2013. 18:35:36 UTC+2, korisnik Jan Hidders
> napisao je:

> > Codd explicitly referred to first-order logic in his work, so the
> > link with Frege was clearly made.
>

> Gottlob Frege is a scientist, who in the last ten years, emerges to
> the surface as a scientist of great importance. Lately, more and more
> scientists reveals its importance in mathematics, philosophy and
> other sciences. Some of the most famous scientists proclaim him as
> the greatest mathematician and philosopher, so far.

...

> 3) Frege developed completely FOL. He introduced formal language for
> FOL and he developed philosophy for spoken languages. I will mention
> only the part which is related to Predicate Calculus. I will briefly

> described Frege?s ideas that are related to sentences, names and

> predicates. I already mentioned that the meaning of a sentence is its

> truth value. The meaning of names was elaborated in Frege?s deep

> theory about names. This theory has changed many things in science.
> So if somebody says that E. Codd was the first who introduced the new
> representation of the relations by using names, then it is the false
> sentence. Even more, it seems to me that this claim implies that E.
> Codd in fact introduced relationships between names and predicates.

Vladimir, I don't know what you're so exercised about.

However important Frege may be, predicate logic was only something
Codd used. I'm unware of any claim he made to extending it. The
idea that one could convert the proposition

the box is red

to the predicate

? is red

has been around a long time. No one disputes that.

Codd simply applied logic to database theory, which at the time didn't
exist. But simple it wasn't, and isn't yet still.

When nearly everyone else was treating database management as an
engineering problem -- as, by and large, everyone still does -- Codd
recognized that databases could be defined in terms of predicates, thus
vastly simplifying the query language, never mind query construction.

That was a great insight. He may not have been the only one to have
it, but he got credit for it. Someone must, just ask Leibniz. He also
carried the ball far enough to change the industry for the better,
something that doesn't happen very often. And he left us a theory that
has yet to be improved on in any fundamental way. Not bad for an
afternoon's work.

Lots of people work in a field for a lifetime without advancing it one
iota. For all intents and purposes, Codd invented his. Don't you
think that's worthy of recognition?

--jkl

[vldm10]

My bad English sometimes is not appropriate, especially when I'm writing in a hurry. Actually I wanted to say the following: Codd just applied Frege's theory to database theory.

The other thing I wanted to say is that Codd should quote Frege's work, because here come the whole theories as Propositional Logic, FOL, Semantics, etc. which are completely changed a spirit of mathematics. Frege's mathematics is applied to databases, meaning this is very complex application of mathematics. I mean, application of Frege's math are much more complex than applications of classic math.

In my opinion, people who want to have a good understanding of relational databases and who want to study it, they have to know where the source of these important theories is. That source is the theories which were built by G. Frege. Otherwise, if someone does not read G. Frege's theory, I do not believe that this person will understand well RM. Every person has the right to the original source.

In this thread I am talking exclusively about the data model. Of course, Codd is credited for the raising the theory of databases on the scientific level. It is clear that Codd did a lot of other things that have significant scientific value, for example FDs and Relation Algebra. Note that Codd, is perhaps the only person who was able to create the new theory of database and implement it in practice as totally dominant data model.

Finally, I would like to say that Frege's work, are not understood by his contemporaries. In Germany, the best mathematicians did not understand Frege's work.
Italian mathematician Giuseppe Peano fully had understood Frege's theory. In fact, Peano and Frege are the real founders of mathematical logic.
In my opinion, B. Russell sets back Frege's work. After Russell's letter to Frege about the paradox in Frege's theory, Frege has stopped working for a longer period.

Vladimir Odrljin

[James K. Lowden]

On Wed, 25 Sep 2013 15:05:06 -0700 (PDT)
vldm10 <vld...@yahoo.com> wrote:
> Dana utorak, 24. rujna 2013. 01:57:40 UTC+2, korisnik James K. Lowden
> napisao je:

> > However important Frege may be, predicate logic was only something
> > Codd used. I'm unware of any claim he made to extending it.

...

> > Lots of people work in a field for a lifetime without advancing it
> > one iota. For all intents and purposes, Codd invented his. Don't
> > you think that's worthy of recognition?
>

> My bad English sometimes is not appropriate, especially when I'm
> writing in a hurry. Actually I wanted to say the following: Codd just
> applied Frege's theory to database theory.
>
> The other thing I wanted to say is that Codd should quote Frege's
> work, because here come the whole theories as Propositional Logic,
> FOL, Semantics, etc. which are completely changed a spirit of
> mathematics. Frege's mathematics is applied to databases, meaning
> this is very complex application of mathematics. I mean, application
> of Frege's math are much more complex than applications of classic
> math.
>
> In my opinion, people who want to have a good understanding of
> relational databases and who want to study it, they have to know
> where the source of these important theories is. That source is the
> theories which were built by G. Frege. Otherwise, if someone does not
> read G. Frege's theory, I do not believe that this person will
> understand well RM. Every person has the right to the original source.
>
> In this thread I am talking exclusively about the data model. Of
> course, Codd is credited for the raising the theory of databases on
> the scientific level. It is clear that Codd did a lot of other things
> that have significant scientific value, for example FDs and Relation
> Algebra. Note that Codd, is perhaps the only person who was able to
> create the new theory of database and implement it in practice as
> totally dominant data model.

The above is very clear; thank you for taking the time.

Because of your fondness for Frege, and not knowing much about the
subject, I spent an hour tonight reading the first hit Google returned
for "first order logic history":

http://www.mcps.umn.edu/philosophy/11_4moore.pdf

From that article, I can only conclude that one can credit many
contributors to FOL. Whether or not Frege gets the credit he deserves
is a question over which reasonable people can disagree.

I was particularly struck by Hilbert's disagreement with Frege over the
truth of axioms,

"... if the arbitrarily given axioms do not contradict each
other with all their consequences, then they are true and the things
defined by the axioms exist. For me this is the criterion of truth and
existence."

because it reflects how databases are designed in practice!

Just as Hilbert justifies his axioms on their noncontradictory
consequences, so are database designs iteratively modified until
inferences drawn from them are deemed an accurate reflection of the
reality they're intended to model. In effect, the database designer
determines his axioms -- postulates about reality -- by examining their
consequences.

Of course, everyone reading c.d.t. has seen databases that weren't
designed that way, or even well, or even "designed" at all other than
as a dog's breakfast. But if you're trying to do a good job putting
your knowledge into practice, Hilbert is wind at your back.

--jkl

[vldm10]

>
>
>
> Note that Frege defined that the meaning of a sentence is its truth value. In >this way he connected logic and semantic at the level of sentences.

This is what I mistakenly wrote. In order to briefly write such a complex
part of the theory, this part is escaped my attention.

It should be the following text: The referent of a sentence of type S is its truth value.

However, Frege explained that the meaning of a sentence is something which
is relevant to determining its truth or falsity. Note that this is related to referents of saturated expressions. The unsaturated expressions, which are predicates, have referents of another kind. (Here the referents are not truth values.)

Vladimir Odrljin

[vldm10]

On Thursday, September 26, 2013 3:35:24 AM UTC+2, James K. Lowden wrote:
> On Wed, 25 Sep 2013 15:05:06 -0700 (PDT)
>

>

> Because of your fondness for Frege, and not knowing much about the
>
> subject,

I am focused on the work of a scientist rather than some idealized picture about
the scientist. I think that work of G. Frege belongs to the group of the most
significant results in mathematics. However I also consider important works of
others great mathematicians, such as Godel, Tarski, Hibert, Cantor etc.
In fact, we do not know these scientists, as people, we don't know them by their
personalities, ethic values, human values etc. Therefore I just write about
their scientific works.
As for Frege, I think that his work was degraded on one hand, and on the other
hand his work was used by some other scientists. Here, first of all I think on
B. Russell, and also those who have supported B. Russell

>I spent an hour tonight reading the first hit Google returned
>
> for "first order logic history":
>
>
>
> http://www.mcps.umn.edu/philosophy/11_4moore.pdf
>
>
>
> From that article, I can only conclude that one can credit many
>
> contributors to FOL. Whether or not Frege gets the credit he deserves
>
> is a question over which reasonable people can disagree.
>

To my knowledge, this article is inaccurate. To me, some important things and
events of that period are well known. This article begins with the Skolem in
1923rd However, Frege's first major work in logic was published in 1879 under
the title Begriffsschrift. Many important things were happened between 1879 and
1923, for example, set theory was created during this period, including
Zermelo’s axioms.

G. Frege's main idea was that the whole of mathematics can be ground on logic.
Therefore Frege had a grandiose idea, to do mathematics as an extension of
logic. The entire work of Frege, roughly speaking, comes from this idea. Of
course, in Frege's work was also included Set Theory. G. Frege had a great
written correspondence with G. Cantor.

In 1889, G. Peano published his work, Arithmetics principia, with his axioms of
natural numbers. Peano introduced the simple, computable procedure (mathematical
induction), which effectively works with infinite set of natural numbers. This
is the first compete mathematical theory that is based on logic.

Frege also did the theory of natural numbers based on logic, in that period.
Mathematicians today reveal a surprisingly good idea in Frege's work. Note that
only Peano and Frege at that time used different symbols for math and logic.
Most of today's symbols from logic and set theory come from G. Peano.
Often, in the books the reader can find the claim that B. Russell came up with
the idea that mathematics can be founded on logic. Note that B. Russell was 17
years old when Giuseppe Peano and Gottlob Frege published their books in which
the mathematical theories were built on logic.

In this article, which you mentioned, the author begins to write about the logic
from 1923: “In 1923, when a young Norwegian mathematician named Thoralf Skolem
argued that set theory should be based on first-order logic, it was a radical
and unprecedented proposal”.
Note that Frege introduced FOL because of his project. He devised “Set Theory"
that is based on his definition of the concept. This huge Frege's project was
stopped with Russell's paradox, which Russell presented with an example from Set
Theory. G. Frege introduced the so called law V, which is actually an axiom.
Russell's paradox refers to the law V.

In my work from 2008, I showed that Russell's paradox does not make sense and
that its conception is wrong. See my thread on this user group: Does the
phrase " Russell's paradox " should be replaced with another phrase? , posted on
December 11, 2012.
In my solution, Frege's law V is accepted as good. So, my work does not improve
the law V, because it's good. I've added a theory that is missing in Frege's
theory. It was presented on this user group. So, my work is public; it means
that my work is subject to criticism. Maybe someone can prove that I am wrong.
I'm pretty sure that my work is a good solution.

Recently (in 2013) it was shown that it is possible to prove Frege's Theorem,
without the use of Frege's law V. It was shown that it is enough to use the
"Hume's Principle". In this regard see work from the 2013 : “Frege's Theorem
and Foundations for Arithmetic”
at http://plato.stanford.edu/entries/frege-theorem/

On the other hand, if we accept the law V, then we change the "Set Theory" from
the ground up. For example, the "primitives" must be changed in this change.
Some axioms of set theory can be derived from Frege's definition of a concept.

Thus, with the use FOL, Frege's definition of Concept and with the addition of
my work, it is possible to form the new set theory and to derive directly,
certain axioms from the existing set theory.
Note that my solution has two procedures. One procedure determines whether an
object satisfies a certain concept. The second procedure performs the
identification of the object that we put in the first procedure. So in my work
is the emphasis on semantic procedures. Both of these procedures makes link
between mind and the real world (external world).

I write about this in order to show that Frege's work not only introduces FOL in
set theory, but it also changes the foundations of Set Theory. I also want to
point out that after 120 years, mathematicians starting to realize the work of
G.Frege in its entirety.

>
> I was particularly struck by Hilbert's disagreement with Frege over the
>
> truth of axioms,
>
>

I see this debate as a significant contribution to the creation of new
mathematical theories that have influenced the history of mathematics. These are
primarily the theory of axiomatic theories and Model Theory.
As for the axiomatic theories, Frege and Hilbert had opened the door of the room
into which entered K. Godel, 30 years later, and surprised the world of science
with his work.
So I think that these two great mathematicians, each in their own way made a
significant contribution to the development of mathematics in this discussion.
Note that this discussion lasted several years, and that some other people also
participated in the debate, in addition to Frege and Hilbert.


Vladimir Odrljin

[vldm10]

In the text above, we need some clarification. The term "FOL" I used
imprecisely in the first part of the text, to emphasize that Frege introduced
FOL. However, it is also known that Frege introduced a second-order logics and
generally speaking, he introduced high-order logics. Law V uses second order
logic, it uses quantifiers to predicates.
In a situation where Russell's paradox is dismissed as irrelevant, there is a
problem for the construction of Frege-inspired set theories. In that sense I
have left open the question of the use of FOL.


In this user group I showed that with the addition of appropriate theory,
Frege's law V is good. This means that Russell's Paradox does not make sense. It
also means that the law V is more general than "Hume's Principle".
Frege used "Hume's Principle" and he cited Hume. However, many feel that this
Frege's citation ought to have been a citation of Cantor.


Vladimir Odrljin