"Theoreticians” are Clueless about Relational Data Modelling, Teach Anti-Relational Muddling
189 views
Skip to first unread message
Derek Ignatius Asirvadem
Mar 29, 2021, 3:59:46 AM3/29/21
to
Introduction
When it came out in 1970, the Relational Model (Codd, not the pretenders) made a paradigm shift in all areas of perceiving data (examination; data modelling; storage and retrieval; programming). Codd had an uphill battle against the established academics because they could not understand it, and against a few of the DBMS vendors, because they were interested in protecting their established products. It took about ten years to overcome those obstacles, and then it took off. IBM had an internal product that progressed through the 1970’s, eventually becoming SQL, and the market begged them to release the definition, so that it could be made a Standard, such that other vendors could provide platforms.
For those who are not old enough to remember, we had perfectly good DBMS in those days, and fairly intense design methods, first for data in general and second for the implementation in our particular DBMS. It just was not a model, in the sense that the /RM/ is a genuine model, defined mathematically and based on First Order Predicate Calculus.
I worked for Cincom, one of the “big five” DBMS vendors, in addition to being expert in their Network DBMS, we were well schooled in the competitions capabilities, in order to compete seriously. We did not care much for academic papers because the science was already deteriorating, and the best scientists (including all progressions in the science, except for the /RM/ ) were invented by engineers who worked for the DBMS vendors.
In contrast to the ever-changing freeware/whatmewear/vapourware/nowhere filth (massive suite of programs, aka open sore) that is marketed these days as “SQL” and as a “platform”, those were the days of real SQL (compliant with the standard), and real platforms (server architecture). We already had ACID Transaction from 1960 IBM/CICS/TPC, thus all pre-Relational DBMS were heavily Transaction oriented and ACID compliant. Thus the first few SQL platforms were high-throughput, high-concurrency, ACID Transaction Processing engines.
There were only two: IBM and Sybase (Britton-Lee the Database Machine people, implemented an SQL Database Machine), and there are still only two, plus a bastard son of Sybase MS/SQL for the low end of the market.
The filth, including Orable, is neither SQL, nor ACID-compliant. Hint: with the schizophrenic notion of MVCC, they cannot handle even simple contention, let alone high-throughput, or high-concurrency, or ACID. (If you wish to comment on this point, there is a separate thread open, begging for progress.)
Methods
Re methods, it was a matter of moving from the established pre-Relational methods to the Relational methods, as opposed to inventing methods for the first time. We did not have, and certainly did not dry about not having, Modelling Tools: we drew diagram on paper, using a stencil. We never heard of, nor used, ERD. Apparently it made a big splash in the academic circles.
Note well, the /Relational Model/ is a single paper that defines a single method, like all academic papers, it defines one thing, it cannot be expected to define related things (such as database design or data modelling or how to blow ones nose without getting snot on ones hands). It is telling that the pig-poop eaters (Date; Darwen; Fagin; and all their followers) wring their hands and wail about “Codd did not define this or Codd did not define that ...). Practitioners are fully capable of continuing the data modelling and database design methods while conforming to the new /RM/, without the need to be fed from a baby bottle.
In 1983 Robert Brown extended his modelling method Logical Database Design Technique to cater for the Relational Paradigm. It became used by a few important customers such as DoD, and thus became known to all DBMS vendors (I personally wrote a database for DoD in Cincom’s TOTAL DBMS), and eventually all their customers. It became widely known by 1987, when the first product ERwin came out (first by LogicWorks, and later by Computer Associates). The IDEF people took it over, named it IDEF1X (following IDEF1 for informational modelling and X for Relational), and maintained it. By 1993 it was a NIST Standard.
Several Data Modelling tools followed, such as PowerDesigner and ER/Studio, but they are nowhere near ERwin in terms of true IDEF1X or platform-neutral implementation.
The point of this thread. In the real world (as distinct from the “world” as concocted by “theoreticians” who allege to serve this space), we have:
- the real /Relational Model/ 1970,
- the real Relational Data Modelling method IDEF1X since 1983 (1987 as a product), and
- real SQL platforms since 1981.
The question begs:
- why is that unknown to the “theoreticians” ?
- why do the pig poop eaters NOT teach Relational Data Modelling ?
--- why do they NOT teach IDEF1X ?
- why do they teach instead the defunct; obsolete; anti-Relational ERD ?
These sad people are operating FIFTY YEARS behind the leading edge AFA the /RM/ is concerned, FORTY YEARS behind the leading edge AFA Standards and SQL Platforms are concerned.
Great news. There is one single academic across the entire planet, who has started a journey from the anti-Relational-marketed-as-Relational land that is their studied home, across the great divide, to the Relational world. We touched on the subject in another thread, but it deserves its own thread, so I have brought it here.
> On Wednesday, 24 March 2021 at 23:27:48 UTC+11, Nicola wrote:
> > On 2021-03-24, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
> >
> > In case you missed them. Could you please respond to these questions.
> >
> >> On Thursday, 18 March 2021 at 14:38:13 UTC+11, Derek Ignatius Asirvadem wrote:
> >> > On Tuesday, 16 March 2021 at 23:23:28 UTC+11, Nicola wrote:
> >>
> >> > Note that the "circularity" in that diagram is only apparent (ERD are
> >> > misleading).
> >>
> >> So, why in heavens name, do professors teach ERD ?
>
> I cannot speak for anyone else...
First, let me thank you for your very interesting and considerate response.
> One reason is Chen's Entity-Relationship models are likely perceived as
> easier to grasp for novices than something like IDEF1X. Whether that is
> a correct perception, it can be debated. Another reason is probably just
> historical, due to the huge influence of Chen's seminal paper.
It may have been seminal for the academics, let me assure you, it was not seminal, or even great, for practitioners of the day. It does not have a semantic base, whereas the /RM/ and IDEF1X does.
For academics. I can grant, up to 1970, the paper was amazing. After 1970, it was ageing, badly. By 1983, it was dead; obsolete; defunct; superseded.
Ok, to rephrase the question, why was that pre-Relational method being taught after 1983, and why is it still being taught after the advent of the /RM/, after the /RM/was proved, and after the first genuine SQL platforms came out (1983), and after IDEF1X came out (1983 to practitioners, 1989 as a product, 1993 as an international standard) ?
Novices grasp anything you give them, so the responsible professor gives them the right thing, not cabbage patch dolls for Computer Science.
> In a general setting, ERD is fine for teaching the basics of modeling:
> entities, relationships, generalizations. For someone who has never done
> it, taking a description in a natural language and analyze it to distill
> such concepts and put them in diagrammatic form is not a trivial task
> (well, it's not trivial even for experts!).
> ERDs can be useful as
> a gentle introduction to such topics.
>
> ERD can be introduced without any background knowledge in databases
For introductory levels, show them IDEF1X/Entity level.
> (although that is true about IDEF1X to some extent, when you are
> teaching IDEF1X you are in fact teaching the Relational Model),
Yes !!! Instead of whatever mumbo jumbo that is NOT Relational. That is pretty much the central point. To think about data in terms of the /RM/, not in terms of the physical records that the freaks can contrive in their little crania, because they cannot grasp the /RM/.
> and even
> for purposes that are different from mapping into the Relational Model.
> For instance, one may use ERDs in the context of programming—say, to
> design a game.
Data, the /RM/, is not application dependent. It is data dependent. It is far superior to any other method or non-method or non-thinking.
> When it comes to using ERD vs IDEF1X for designing Relational databases,
> my experience is as follows:
>
> - when teaching ERDs, one also has to explain the rules to translate
> them into the RM:
One can’t. I don’t. I didn’t in the example. It is like translating the thinking that goes into rap “music”, into the thinking that goes into Beethoven’s Fifth. Don’t. Just think classical music. Don’t draw a diagram that has an inferior understanding of data (actually NO understanding of data), and then attempt to translate it into a higher form. No, think in the higher form. and then draw the higher form.
> this is a layer of accidental complexity that IDEF1X
> completely eliminates.
Yes. And it does so by design, not by accident.
> - On other hand, when teaching IDEF1X one has to introduce a richer
> language, so it doesn't take less time to teach than ERD; but I'd say
> that it doesn't take much longer either, as some people might might
> expect.
When you teach a principle (the RM), the whole lesson, even though it is rich, is actually faster, and more deeply understood. When you teach without the principle (ERD, no RM), you struggle to teach because there is no principle, just a collection of low-level axioms. Fragments of meat on the ground that only pigs and rats forage for, but hey, that is what the “theoreticians” promote.
> - Perhaps contrary to someone's opinion, IDEF1X models can be grasped by
> novices: I have successfully taught IDEF1X to people without any
> background in Computer Science and high-school level of acquaintance
> with logic for a few years now.
Absolutely. I have been doing it for forty years. Half the time, I don’t even tell them what it is, I just erect the data model during the discussion. If they ask, I give them my short IDEF1X Intro:
https://www.softwaregems.com.au/Documents/Documentary%20Examples/IDEF1X%20Introduction.pdf
> - ERDs make it difficult or impossible to reason about keys effectively:
> think split keys, for instance. Or common ancestor constraints. This
> is one of my biggest issues with ERDs.
Well, it is completely 100% brain-dead re the concept of Keys. Which is why I say that ERD is anti-Relational. The central concept in the /RM/ is logical Keys, that are composite. It is not possible to teach Relational using a method that cannot describe Relational.
On the other hand, if one wishes to cancel the DATA MODELLING exercise, and to go into the establishment of premature entities (which the entire OO/ORM world slavishly does), sure, ERD works fine: the “modelling” is not modelling abut a single iteration of drawing what Entities one can imagine (that is imagined, not real) for a first evaluation, based on the need (not on the data), and one knows that one will stamp as ID field on every file. In the other thread, OP has given us three perfect examples of precisely that, the total absence of modelling.
ERD cannot be used for (a) Relational modelling of any kind, (b) modelling data of any kind by a mind that is cognisant of the /RM (1970, fully established by 1981, available in SQL platforms 1983).
Since ERD does not handle Relational Keys, the bottom line is, ERD is anti-Relational.
The freak brigade (Date; Darwen; Fagin; and their followers) play the Straw Man that is their god, they use ERD to muddle instead of model, and then on the basis that ERD fails re Keys, they declare that the /RM/ doesn’t handle Keys very well. Lies upon lies upon lies.
> If you are using them to design
> a video game, that might not be a problem, but for mapping them into
> the RM, well, they effectively cripple your design process.
Yes. Again, one cannot use ERD for Data Modelling of any kind, let alone for Data Modelling under the Relational paradigm.
I reject the notion that a database that is required for any category whatsoever is better designed without reference to the /RM/. The notion is ridiculous if you understand FOPC. If you suggest there is, please open a new thread and provide an example.
> - There are some aspects of ERDs that are *far* from intuitive. If you
> have ever tried to explain ERD relationships of degree higher than two
> (and especially how to constrain the cardinalities of the participant
> entities), then IDEF1X seems a lot simpler!
Yes, on both points. Technically, there is no Cardinality at all, any Cardinality that is drawn by the designer, is beyond ERD, and extension. The reason they do that, is precisely because Cardinality is important to the modelling process. As evidenced in the source thread.
Criteria for considering redundant relationships in ERD
https://groups.google.com/g/comp.databases.theory/c/8rUC80JwTaE
> I have recommended my colleagues to switch to IDEF1X without success
> (but without insisting either, to be honest). I guess this is simply
> because habits are hard to change and most textbooks do not use it. New
> generations pick up what they have been taught by previous generations.
That is true, and it is the same as for cannibals that live in mud huts. The Creutzfeldt–Jakob Disease is not going to effect a change in their habits. Exactly the opposite of what is required of a scientist.
> Some may feel IDEF1X is "low-level" (too many details) compared to ERDs.
False: display only the level of detail necessary for any given exercise {Entity|Key|Attribute}.
> Or some may have criticisms along the lines of this analysis:
>
> https://www.essentialstrategies.com/publications/modeling/idef1x.htm
God help me. Academics just love to sabotage themselves, and then cry about the fact that they have been sabotaged.
That is too superficial and ignorant, I don’t understand why a guy like you (with some actual experience of IDEF1X) would resort to it. It has the odour of the academic, isolated from reality, writing about how yesterdays milk is sour, without ever having milked a cow.
It is very sad that that is all that is available to academia, re IDEF1X. Perhaps try searching.
I will not answer such superficial, ignorant nonsense. However, the relevant issues are noted by you, and answered by me, below.
> Another issue may be that IDEF1X is not so popular in the industry
> either, except for some sectors or in some geographical areas. E.g., it
> doesn't seem to be so popular in Europe.
Academics know absolutely nothing about the industry. They tell each other what the “industry” is, and use that to make sure they stay FORTY YEARS behind the industry. Think about the fact that there are many DM tools available, and each of them provide IDEF1X, and the fact that academics will not use them, in order to pretend that both the DM tools, and IDEF1X, does not exist. Pure ignorance of the industry that they purport to serve. Sure, they stick to the freeware, eg. the various anti-SQL, which is not SQL< and complain that “SQL is broken”. Likewise, there is no freeware DM tool, so they say it does not exist, and does not exist in their contrived “industry”. Meanwhile, back in the industry, we have had those tools since 1983, automated since 1989, and we have resolved those problems since 1983, automated since 1989.
> My main complaint with IDEF1X is the treatment of generalization
> hierarchies, in particular the lack of a correct way to model a total
> inclusive specialization (A must be one of B1, ..., Bn and it may be any
> of B1, ..., Bn); the approach I have seen around (e.g., see Thomas
> Bruce's book or ERwin's manual) does not cover such case in a sound way.
> And the standard never mentions inclusive specializations, AFAICT.
The problem there is, you do not understand Transactions (ACID, since 1960 in IBM Mainframes; since 1970 in pre-relational DBMS; since 1981 in RDBMS). It is a major blind spot for academics and theoreticians, that practitioners simply do not have. There are several threads in this forum that simply do not progress to closure due to the unwillingness of academics to engage, and to learn about ACID Transactions. Again, academics are stuck in their vacuum tube of what they themselves have written (filth from Stonebraker; Ellis; Date; Darwen; Fagin; etc), and are in reinforced pathological denial of the actual industry; the actual commercial products that serve the industry, and they remain so for FIFTY YEARS.
Any good NON-"theoretical" data modeling/Codd-relational database design courses/books/papers out there
https://groups.google.com/g/comp.databases.theory/c/T8sRg3I2vJU
[[Relational] Database] Open Architecture Standard
https://groups.google.com/g/comp.databases.theory/c/umEPHEi5FA8
MVCC, Advantages & Disadvantages
https://groups.google.com/g/comp.databases.theory/c/f474bCuvZ_A
Since you are the single academic who is venturing out from their safe space, I welcome such an engagement, please feel free to engage those threads, if and when you have the interest.
Yes, absolutely, out here in the real world, using real SQL RDBMS platforms since 1983, we have been implementing precisely both Exclusive and Non-Exclusive Subtypes perfectly, since 1983. That is:
- for a Non-Exclusive Basetype, we guarantee that there must be at least one Subtype
- for an Exclusive Basetype, we guarantee that there must be one Subtype only
This document may assist somewhat in explanation, noting that the full and formal explanation lies in ACID Transactions, which has not progressed in this forum, and indeed in academia, since 1970.
https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/Subtype.pdf
If there is anything in that doc that is not completely understood, please ask.
> Another thing: people who already know ERDs easily accept the
> distinction between independent and dependent entities (which reminds
> them of the distinction between ERD's strong and weak entities), but
> they may not see what additional information identifying and
> non-identifying relationships bring to the table
They can’t. Because it is not taught, not defined. Teaching the distinction between strong::weak, but not teaching how to implement that, in terms of the child PK, is criminal. Chen’s strong::weak is weak. RM/IDEF1X Independent vs Dependent; Identifying vs Non-Identifying, is strong (has not changed since 1970).
Since it is one concept, not two, in the RM/IDEF1X, you get the one concept with the one implementation method, and it can be perceived several ways, any way you like, at any level-of-detail you like.
> (“If you drew all the
> relationships without dashed lines, you would still be able to interpret
> your model in the same way"). I found it difficult sometimes to convince
> them that such a distinction is important.
Only in the early stages of data modelling. Once the Keys are exposed, and worked, which is the bulk of the modelling exercise, that distinction is exposed, and difference is plain.
When engaging the modelling exercise fully, meaning, in order, FOPC; the /RM; and IDEF1X as the rendition, and as you confirm “meaning is king”, you may realise:
- the solid lines define ownership (Extension, of the Matter of the parent)
- the dashed lines define classification (imposition of the Form of the parent)
Assuming you are familiar with Aristotelian concepts, otherwise please ask.
> Another criticism about IDEF1X is the lack of many-to-many relationships
> (except in "Level 1"/"ER" diagrams), which sometimes makes it
> awkward to read a relationship, because it is more natural to skip the
> intermediate associative entity.
You have the right idea, but not the precise method.
> For instance, one might want to model
> the following:
>
> - a customer BUYS zero or more products;
> - a product IS BOUGHT BY zero or more customers.
>
> In IDEF1X, you'd do it like this:
>
> +-----------+ +-----------+
> | | | |
> | Customer | | Product |
> | | | |
> +-----------+ +-----------+
> | |
> | |
> | +-----------+ |
> | | | |
> +-----O| Purchase |O-----+
> | |
> +-----------+
>
> How would you label the relationships? You might use:
>
> - a customer PERFORMS (?) zero or more purchases;
> - a product IS SOLD AS (?) zero or purchases.
>
> But the most natural reading is as above, with "buys"/"is bought by"—
> which does not mention purchases, though.
Well, in the previous model, Purchase does NOT exist, those verbs are relevant in that context only. It is natural to not-reference what is not-there.
In the model in which Purchase DOES exist, it is not natural to ignore it. The verbs must deal with the entities at the two ends of each relation, not some entity at the end of a chain.
>>>>
I believe you understand that the /RM/ is founded on First Order Predicate Calculus, and thus there is nothing in the universe that cannot be defined in FOPC, and therefore in a RDB. Thus you can’t mess with the Predicates, the are atoms, that you must treat as such. You don’t want to mis-read a Predicate in a single “link”, for focussing too much on what is at the end of a Predicate chain.
Separate point. You can’t take a summary at level 1, and apply it as detail in level 2. You have to either expand the summary at level 1 (full detail, which can then be applied at lower levels), or vacate the notion that a level 2 article can be described by a summary at level 1.
<<<<
At level 2, since Purchase exists,
Each customer BUYS zero or more products
Each product IS BOUGHT BY zero or more customers
is simply false. The truth is:
Each Customer buys 0-to-n Purchases
Each Product is sold as 0-to-n Purchases
Each Purchase is bought by 1 Customer
Each Purchase is a sale of 1 Product
Having said that, it is quite acceptable to READ OFF a Predicate chain, without violating the atomic Predicates:
- Each Customer buys 0-to-n Products via Purchase
- Each Product is sold to 0-to-n Customers via Purchase
In this example, p :
https://www.softwaregems.com.au/Documents/Documentary%20Examples/Order%20DM%20Advanced.pdf
after observing the model, it is quite acceptable to READ OFF:
- Each Party offers 0-to-n Objects via PartVendor
- Each Party uses 1-to-n Addresses via PartyAddress
Note that the above explanation regards the /Relational Model/, and FOPC, its foundation, not IDEF1X, which just happens to be the notation we are using for this data model, and which facilitates modelling data Relationally. It is not a “problem” in IDEF1X.
Put another way, if FOPC Predicates, not necessarily the calculus, but the full articulation of logic definition, is taught first, the pupils obtain a solid grounding in the definition of facts, using formal logic. That is the /semantics/ that we will later expect in a model. After that, teaching the /RM/ is easy. After that, issues in rendering a Relational Data Model in IDEF1X is easy. Issues such as these are minor and easily resolved in the context of the three layers. It is not a “problem” isolated in the third layer, devoid of context.
> IMO, this is a valid criticism. Some additional (an)notation to pick out
> "pure associative entities" (for a lack of a better term) might help.
First understand that the [pure] Associative entity consists of Keys only, no attributes. If it has attributes, it is not an Associative entity.
Then, for Associative entities, the VerbPhrase should apply to the entity at the end of the chain, not the proximate entity. That is an exception.
> >> Why, since we have had IDEF1X for Relational Data Modelling since
> >> 1983 (established and well-known international standard since 1993),
> >> do professors suppress that, and instead “teach”; push; shove;
> >> market; propagate the broken ERD, which is a pre-Relational artefact
> >> ?
> I think I have answered this above, at least by providing my own
> perspective.
Thanks.
As per the entire body of evidence, consistent, since 1970, they are actively suppressing the /RM/, and IDEF1X that goes with it. They are actively promoting pre-1970, pre-DBMS Record Filing Systems fraudulently, as “Relational”, along with the ancient; broken ERD that goes with it, and fraudulently calling it “Relational Data Modelling”.
Why ? Because their goal is the destruction of science, the suppression of truth, by causing confusion, by maintaining non-resolution, by pleading that falsity is “truth”, by denying reality, teh Four Laws of Thought.
> >> > In your model, each teacher must always be assigned to at least one
> >> > course. With
> >> >
> >> > (a) such an (unrealistic) constraint, *and*
> >> Why is that constraint “unrealistic” ? In the real world, it is
> >> a common; even pedestrian, requirement. We have been implementing
> >> such in Relational databases since 1983.
> I was thinking more in terms of "department employees", one of whose
> duties may be teaching (but they may not be teaching all the time). But
> sure, you may think in terms of employees who teach (a subset of all the
> employees): then they must teach something, of course.
Sorry, that is not what I meant.
Let me rephrase the question. Why do professors commonly labour and state that a 1::1-to-n relation is difficult, why do they pretend it is uncommon ? And how is such a relation (you correctly identify it as a Constraint) established in a Relational database (and in the data model erected in IDEF1X).
The core difference between pre-Relational theory; methods; and DBMS platforms vs the Relational Model; Relational Data Modelling; and RDBMS, is Relational Keys. The core of Relational Keys is Identity, the identity of the subject. To teach Data Modelling after say 1983 without teaching the relevance of Keys, without teaching precisely how to identify a subject, is criminal.
Here is a quick document that clarifies the issues you questioned, that I have answered. To be complete, I have added an item re Rolenames, because it is the third common “issue” that people raise:
https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/IDEF1X/Nicola%20IDEF1X%202.pdf
------------
-- Status --
------------
In essence, you have picked up IDEF1X, you have determined it is *THE* Standard for Relational Data Modelling, and you have started to use it. Excellent, the single academic who does. However, you are hindered by reading the nonsense that academics have to say about it (noting that they actively suppress it and instead promote ERD), which is as usual ignorant and incorrect, totally unscientific.
There are no problems or difficulties whatsoever in using IDEF1X in the entire exercise of genuine Relational Data Modelling. I have close to 100 such databases, some with over 500 tables (which is why and how I have developed Extensions). Instead, if you have any questions; any difficulty at all, please ask me. Hopefully you will progress much faster.
I would say, get more experience, and use data models that have some complexity (problems to solve).
(The only problem, if one could say such a thing exists, is that IDEF1X cannot enforce the /RM/, people can still do what ever they want. I have a remedy for that, but that is beyond the scope of this thread.)
----------------
-- Conclusion --
----------------
It is very sad that academics deny the real world; the industry; the science; the methods; the platforms, and instead indulge themselves in an isolated contrived “reality”, fixed in 1969. FIFTY YEARS OUT OF DATE.
We need to appreciate that as per the mountain of consistent evidence, the “theoreticians” teach the anti-Relational, pre-1970, pre-DBMS methods as “Relational”. FOR FIFTY YEARS. Now, categorically, as per the evidence, it is clear that the “theoreticians” do not teach the Standard Relational Data Modelling, but instead teach porcine excreta as “Relational Data Modelling”. FOR FORTY YEARS.
I don’t know how you don’t strangle your colleagues.
Cheers
Derek
When it came out in 1970, the Relational Model (Codd, not the pretenders) made a paradigm shift in all areas of perceiving data (examination; data modelling; storage and retrieval; programming). Codd had an uphill battle against the established academics because they could not understand it, and against a few of the DBMS vendors, because they were interested in protecting their established products. It took about ten years to overcome those obstacles, and then it took off. IBM had an internal product that progressed through the 1970’s, eventually becoming SQL, and the market begged them to release the definition, so that it could be made a Standard, such that other vendors could provide platforms.
For those who are not old enough to remember, we had perfectly good DBMS in those days, and fairly intense design methods, first for data in general and second for the implementation in our particular DBMS. It just was not a model, in the sense that the /RM/ is a genuine model, defined mathematically and based on First Order Predicate Calculus.
I worked for Cincom, one of the “big five” DBMS vendors, in addition to being expert in their Network DBMS, we were well schooled in the competitions capabilities, in order to compete seriously. We did not care much for academic papers because the science was already deteriorating, and the best scientists (including all progressions in the science, except for the /RM/ ) were invented by engineers who worked for the DBMS vendors.
In contrast to the ever-changing freeware/whatmewear/vapourware/nowhere filth (massive suite of programs, aka open sore) that is marketed these days as “SQL” and as a “platform”, those were the days of real SQL (compliant with the standard), and real platforms (server architecture). We already had ACID Transaction from 1960 IBM/CICS/TPC, thus all pre-Relational DBMS were heavily Transaction oriented and ACID compliant. Thus the first few SQL platforms were high-throughput, high-concurrency, ACID Transaction Processing engines.
There were only two: IBM and Sybase (Britton-Lee the Database Machine people, implemented an SQL Database Machine), and there are still only two, plus a bastard son of Sybase MS/SQL for the low end of the market.
The filth, including Orable, is neither SQL, nor ACID-compliant. Hint: with the schizophrenic notion of MVCC, they cannot handle even simple contention, let alone high-throughput, or high-concurrency, or ACID. (If you wish to comment on this point, there is a separate thread open, begging for progress.)
Methods
Re methods, it was a matter of moving from the established pre-Relational methods to the Relational methods, as opposed to inventing methods for the first time. We did not have, and certainly did not dry about not having, Modelling Tools: we drew diagram on paper, using a stencil. We never heard of, nor used, ERD. Apparently it made a big splash in the academic circles.
Note well, the /Relational Model/ is a single paper that defines a single method, like all academic papers, it defines one thing, it cannot be expected to define related things (such as database design or data modelling or how to blow ones nose without getting snot on ones hands). It is telling that the pig-poop eaters (Date; Darwen; Fagin; and all their followers) wring their hands and wail about “Codd did not define this or Codd did not define that ...). Practitioners are fully capable of continuing the data modelling and database design methods while conforming to the new /RM/, without the need to be fed from a baby bottle.
In 1983 Robert Brown extended his modelling method Logical Database Design Technique to cater for the Relational Paradigm. It became used by a few important customers such as DoD, and thus became known to all DBMS vendors (I personally wrote a database for DoD in Cincom’s TOTAL DBMS), and eventually all their customers. It became widely known by 1987, when the first product ERwin came out (first by LogicWorks, and later by Computer Associates). The IDEF people took it over, named it IDEF1X (following IDEF1 for informational modelling and X for Relational), and maintained it. By 1993 it was a NIST Standard.
Several Data Modelling tools followed, such as PowerDesigner and ER/Studio, but they are nowhere near ERwin in terms of true IDEF1X or platform-neutral implementation.
The point of this thread. In the real world (as distinct from the “world” as concocted by “theoreticians” who allege to serve this space), we have:
- the real /Relational Model/ 1970,
- the real Relational Data Modelling method IDEF1X since 1983 (1987 as a product), and
- real SQL platforms since 1981.
The question begs:
- why is that unknown to the “theoreticians” ?
- why do the pig poop eaters NOT teach Relational Data Modelling ?
--- why do they NOT teach IDEF1X ?
- why do they teach instead the defunct; obsolete; anti-Relational ERD ?
These sad people are operating FIFTY YEARS behind the leading edge AFA the /RM/ is concerned, FORTY YEARS behind the leading edge AFA Standards and SQL Platforms are concerned.
Great news. There is one single academic across the entire planet, who has started a journey from the anti-Relational-marketed-as-Relational land that is their studied home, across the great divide, to the Relational world. We touched on the subject in another thread, but it deserves its own thread, so I have brought it here.
> On Wednesday, 24 March 2021 at 23:27:48 UTC+11, Nicola wrote:
> > On 2021-03-24, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
> >
> > In case you missed them. Could you please respond to these questions.
> >
> >> On Thursday, 18 March 2021 at 14:38:13 UTC+11, Derek Ignatius Asirvadem wrote:
> >> > On Tuesday, 16 March 2021 at 23:23:28 UTC+11, Nicola wrote:
> >>
> >> > Note that the "circularity" in that diagram is only apparent (ERD are
> >> > misleading).
> >>
> >> So, why in heavens name, do professors teach ERD ?
>
> I cannot speak for anyone else...
First, let me thank you for your very interesting and considerate response.
> One reason is Chen's Entity-Relationship models are likely perceived as
> easier to grasp for novices than something like IDEF1X. Whether that is
> a correct perception, it can be debated. Another reason is probably just
> historical, due to the huge influence of Chen's seminal paper.
It may have been seminal for the academics, let me assure you, it was not seminal, or even great, for practitioners of the day. It does not have a semantic base, whereas the /RM/ and IDEF1X does.
For academics. I can grant, up to 1970, the paper was amazing. After 1970, it was ageing, badly. By 1983, it was dead; obsolete; defunct; superseded.
Ok, to rephrase the question, why was that pre-Relational method being taught after 1983, and why is it still being taught after the advent of the /RM/, after the /RM/was proved, and after the first genuine SQL platforms came out (1983), and after IDEF1X came out (1983 to practitioners, 1989 as a product, 1993 as an international standard) ?
Novices grasp anything you give them, so the responsible professor gives them the right thing, not cabbage patch dolls for Computer Science.
> In a general setting, ERD is fine for teaching the basics of modeling:
> entities, relationships, generalizations. For someone who has never done
> it, taking a description in a natural language and analyze it to distill
> such concepts and put them in diagrammatic form is not a trivial task
> (well, it's not trivial even for experts!).
> ERDs can be useful as
> a gentle introduction to such topics.
>
> ERD can be introduced without any background knowledge in databases
For introductory levels, show them IDEF1X/Entity level.
> (although that is true about IDEF1X to some extent, when you are
> teaching IDEF1X you are in fact teaching the Relational Model),
Yes !!! Instead of whatever mumbo jumbo that is NOT Relational. That is pretty much the central point. To think about data in terms of the /RM/, not in terms of the physical records that the freaks can contrive in their little crania, because they cannot grasp the /RM/.
> and even
> for purposes that are different from mapping into the Relational Model.
> For instance, one may use ERDs in the context of programming—say, to
> design a game.
Data, the /RM/, is not application dependent. It is data dependent. It is far superior to any other method or non-method or non-thinking.
> When it comes to using ERD vs IDEF1X for designing Relational databases,
> my experience is as follows:
>
> - when teaching ERDs, one also has to explain the rules to translate
> them into the RM:
One can’t. I don’t. I didn’t in the example. It is like translating the thinking that goes into rap “music”, into the thinking that goes into Beethoven’s Fifth. Don’t. Just think classical music. Don’t draw a diagram that has an inferior understanding of data (actually NO understanding of data), and then attempt to translate it into a higher form. No, think in the higher form. and then draw the higher form.
> this is a layer of accidental complexity that IDEF1X
> completely eliminates.
Yes. And it does so by design, not by accident.
> - On other hand, when teaching IDEF1X one has to introduce a richer
> language, so it doesn't take less time to teach than ERD; but I'd say
> that it doesn't take much longer either, as some people might might
> expect.
When you teach a principle (the RM), the whole lesson, even though it is rich, is actually faster, and more deeply understood. When you teach without the principle (ERD, no RM), you struggle to teach because there is no principle, just a collection of low-level axioms. Fragments of meat on the ground that only pigs and rats forage for, but hey, that is what the “theoreticians” promote.
> - Perhaps contrary to someone's opinion, IDEF1X models can be grasped by
> novices: I have successfully taught IDEF1X to people without any
> background in Computer Science and high-school level of acquaintance
> with logic for a few years now.
Absolutely. I have been doing it for forty years. Half the time, I don’t even tell them what it is, I just erect the data model during the discussion. If they ask, I give them my short IDEF1X Intro:
https://www.softwaregems.com.au/Documents/Documentary%20Examples/IDEF1X%20Introduction.pdf
> - ERDs make it difficult or impossible to reason about keys effectively:
> think split keys, for instance. Or common ancestor constraints. This
> is one of my biggest issues with ERDs.
Well, it is completely 100% brain-dead re the concept of Keys. Which is why I say that ERD is anti-Relational. The central concept in the /RM/ is logical Keys, that are composite. It is not possible to teach Relational using a method that cannot describe Relational.
On the other hand, if one wishes to cancel the DATA MODELLING exercise, and to go into the establishment of premature entities (which the entire OO/ORM world slavishly does), sure, ERD works fine: the “modelling” is not modelling abut a single iteration of drawing what Entities one can imagine (that is imagined, not real) for a first evaluation, based on the need (not on the data), and one knows that one will stamp as ID field on every file. In the other thread, OP has given us three perfect examples of precisely that, the total absence of modelling.
ERD cannot be used for (a) Relational modelling of any kind, (b) modelling data of any kind by a mind that is cognisant of the /RM (1970, fully established by 1981, available in SQL platforms 1983).
Since ERD does not handle Relational Keys, the bottom line is, ERD is anti-Relational.
The freak brigade (Date; Darwen; Fagin; and their followers) play the Straw Man that is their god, they use ERD to muddle instead of model, and then on the basis that ERD fails re Keys, they declare that the /RM/ doesn’t handle Keys very well. Lies upon lies upon lies.
> If you are using them to design
> a video game, that might not be a problem, but for mapping them into
> the RM, well, they effectively cripple your design process.
Yes. Again, one cannot use ERD for Data Modelling of any kind, let alone for Data Modelling under the Relational paradigm.
I reject the notion that a database that is required for any category whatsoever is better designed without reference to the /RM/. The notion is ridiculous if you understand FOPC. If you suggest there is, please open a new thread and provide an example.
> - There are some aspects of ERDs that are *far* from intuitive. If you
> have ever tried to explain ERD relationships of degree higher than two
> (and especially how to constrain the cardinalities of the participant
> entities), then IDEF1X seems a lot simpler!
Yes, on both points. Technically, there is no Cardinality at all, any Cardinality that is drawn by the designer, is beyond ERD, and extension. The reason they do that, is precisely because Cardinality is important to the modelling process. As evidenced in the source thread.
Criteria for considering redundant relationships in ERD
https://groups.google.com/g/comp.databases.theory/c/8rUC80JwTaE
> I have recommended my colleagues to switch to IDEF1X without success
> (but without insisting either, to be honest). I guess this is simply
> because habits are hard to change and most textbooks do not use it. New
> generations pick up what they have been taught by previous generations.
That is true, and it is the same as for cannibals that live in mud huts. The Creutzfeldt–Jakob Disease is not going to effect a change in their habits. Exactly the opposite of what is required of a scientist.
> Some may feel IDEF1X is "low-level" (too many details) compared to ERDs.
False: display only the level of detail necessary for any given exercise {Entity|Key|Attribute}.
> Or some may have criticisms along the lines of this analysis:
>
> https://www.essentialstrategies.com/publications/modeling/idef1x.htm
God help me. Academics just love to sabotage themselves, and then cry about the fact that they have been sabotaged.
That is too superficial and ignorant, I don’t understand why a guy like you (with some actual experience of IDEF1X) would resort to it. It has the odour of the academic, isolated from reality, writing about how yesterdays milk is sour, without ever having milked a cow.
It is very sad that that is all that is available to academia, re IDEF1X. Perhaps try searching.
I will not answer such superficial, ignorant nonsense. However, the relevant issues are noted by you, and answered by me, below.
> Another issue may be that IDEF1X is not so popular in the industry
> either, except for some sectors or in some geographical areas. E.g., it
> doesn't seem to be so popular in Europe.
Academics know absolutely nothing about the industry. They tell each other what the “industry” is, and use that to make sure they stay FORTY YEARS behind the industry. Think about the fact that there are many DM tools available, and each of them provide IDEF1X, and the fact that academics will not use them, in order to pretend that both the DM tools, and IDEF1X, does not exist. Pure ignorance of the industry that they purport to serve. Sure, they stick to the freeware, eg. the various anti-SQL, which is not SQL< and complain that “SQL is broken”. Likewise, there is no freeware DM tool, so they say it does not exist, and does not exist in their contrived “industry”. Meanwhile, back in the industry, we have had those tools since 1983, automated since 1989, and we have resolved those problems since 1983, automated since 1989.
> My main complaint with IDEF1X is the treatment of generalization
> hierarchies, in particular the lack of a correct way to model a total
> inclusive specialization (A must be one of B1, ..., Bn and it may be any
> of B1, ..., Bn); the approach I have seen around (e.g., see Thomas
> Bruce's book or ERwin's manual) does not cover such case in a sound way.
> And the standard never mentions inclusive specializations, AFAICT.
The problem there is, you do not understand Transactions (ACID, since 1960 in IBM Mainframes; since 1970 in pre-relational DBMS; since 1981 in RDBMS). It is a major blind spot for academics and theoreticians, that practitioners simply do not have. There are several threads in this forum that simply do not progress to closure due to the unwillingness of academics to engage, and to learn about ACID Transactions. Again, academics are stuck in their vacuum tube of what they themselves have written (filth from Stonebraker; Ellis; Date; Darwen; Fagin; etc), and are in reinforced pathological denial of the actual industry; the actual commercial products that serve the industry, and they remain so for FIFTY YEARS.
Any good NON-"theoretical" data modeling/Codd-relational database design courses/books/papers out there
https://groups.google.com/g/comp.databases.theory/c/T8sRg3I2vJU
[[Relational] Database] Open Architecture Standard
https://groups.google.com/g/comp.databases.theory/c/umEPHEi5FA8
MVCC, Advantages & Disadvantages
https://groups.google.com/g/comp.databases.theory/c/f474bCuvZ_A
Since you are the single academic who is venturing out from their safe space, I welcome such an engagement, please feel free to engage those threads, if and when you have the interest.
Yes, absolutely, out here in the real world, using real SQL RDBMS platforms since 1983, we have been implementing precisely both Exclusive and Non-Exclusive Subtypes perfectly, since 1983. That is:
- for a Non-Exclusive Basetype, we guarantee that there must be at least one Subtype
- for an Exclusive Basetype, we guarantee that there must be one Subtype only
This document may assist somewhat in explanation, noting that the full and formal explanation lies in ACID Transactions, which has not progressed in this forum, and indeed in academia, since 1970.
https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/Subtype.pdf
If there is anything in that doc that is not completely understood, please ask.
> Another thing: people who already know ERDs easily accept the
> distinction between independent and dependent entities (which reminds
> them of the distinction between ERD's strong and weak entities), but
> they may not see what additional information identifying and
> non-identifying relationships bring to the table
They can’t. Because it is not taught, not defined. Teaching the distinction between strong::weak, but not teaching how to implement that, in terms of the child PK, is criminal. Chen’s strong::weak is weak. RM/IDEF1X Independent vs Dependent; Identifying vs Non-Identifying, is strong (has not changed since 1970).
Since it is one concept, not two, in the RM/IDEF1X, you get the one concept with the one implementation method, and it can be perceived several ways, any way you like, at any level-of-detail you like.
> (“If you drew all the
> relationships without dashed lines, you would still be able to interpret
> your model in the same way"). I found it difficult sometimes to convince
> them that such a distinction is important.
Only in the early stages of data modelling. Once the Keys are exposed, and worked, which is the bulk of the modelling exercise, that distinction is exposed, and difference is plain.
When engaging the modelling exercise fully, meaning, in order, FOPC; the /RM; and IDEF1X as the rendition, and as you confirm “meaning is king”, you may realise:
- the solid lines define ownership (Extension, of the Matter of the parent)
- the dashed lines define classification (imposition of the Form of the parent)
Assuming you are familiar with Aristotelian concepts, otherwise please ask.
> Another criticism about IDEF1X is the lack of many-to-many relationships
> (except in "Level 1"/"ER" diagrams), which sometimes makes it
> awkward to read a relationship, because it is more natural to skip the
> intermediate associative entity.
You have the right idea, but not the precise method.
> For instance, one might want to model
> the following:
>
> - a customer BUYS zero or more products;
> - a product IS BOUGHT BY zero or more customers.
>
> In IDEF1X, you'd do it like this:
>
> +-----------+ +-----------+
> | | | |
> | Customer | | Product |
> | | | |
> +-----------+ +-----------+
> | |
> | |
> | +-----------+ |
> | | | |
> +-----O| Purchase |O-----+
> | |
> +-----------+
>
> How would you label the relationships? You might use:
>
> - a customer PERFORMS (?) zero or more purchases;
> - a product IS SOLD AS (?) zero or purchases.
>
> But the most natural reading is as above, with "buys"/"is bought by"—
> which does not mention purchases, though.
Well, in the previous model, Purchase does NOT exist, those verbs are relevant in that context only. It is natural to not-reference what is not-there.
In the model in which Purchase DOES exist, it is not natural to ignore it. The verbs must deal with the entities at the two ends of each relation, not some entity at the end of a chain.
>>>>
I believe you understand that the /RM/ is founded on First Order Predicate Calculus, and thus there is nothing in the universe that cannot be defined in FOPC, and therefore in a RDB. Thus you can’t mess with the Predicates, the are atoms, that you must treat as such. You don’t want to mis-read a Predicate in a single “link”, for focussing too much on what is at the end of a Predicate chain.
Separate point. You can’t take a summary at level 1, and apply it as detail in level 2. You have to either expand the summary at level 1 (full detail, which can then be applied at lower levels), or vacate the notion that a level 2 article can be described by a summary at level 1.
<<<<
At level 2, since Purchase exists,
Each customer BUYS zero or more products
Each product IS BOUGHT BY zero or more customers
is simply false. The truth is:
Each Customer buys 0-to-n Purchases
Each Product is sold as 0-to-n Purchases
Each Purchase is bought by 1 Customer
Each Purchase is a sale of 1 Product
Having said that, it is quite acceptable to READ OFF a Predicate chain, without violating the atomic Predicates:
- Each Customer buys 0-to-n Products via Purchase
- Each Product is sold to 0-to-n Customers via Purchase
In this example, p :
https://www.softwaregems.com.au/Documents/Documentary%20Examples/Order%20DM%20Advanced.pdf
after observing the model, it is quite acceptable to READ OFF:
- Each Party offers 0-to-n Objects via PartVendor
- Each Party uses 1-to-n Addresses via PartyAddress
Note that the above explanation regards the /Relational Model/, and FOPC, its foundation, not IDEF1X, which just happens to be the notation we are using for this data model, and which facilitates modelling data Relationally. It is not a “problem” in IDEF1X.
Put another way, if FOPC Predicates, not necessarily the calculus, but the full articulation of logic definition, is taught first, the pupils obtain a solid grounding in the definition of facts, using formal logic. That is the /semantics/ that we will later expect in a model. After that, teaching the /RM/ is easy. After that, issues in rendering a Relational Data Model in IDEF1X is easy. Issues such as these are minor and easily resolved in the context of the three layers. It is not a “problem” isolated in the third layer, devoid of context.
> IMO, this is a valid criticism. Some additional (an)notation to pick out
> "pure associative entities" (for a lack of a better term) might help.
First understand that the [pure] Associative entity consists of Keys only, no attributes. If it has attributes, it is not an Associative entity.
Then, for Associative entities, the VerbPhrase should apply to the entity at the end of the chain, not the proximate entity. That is an exception.
> >> Why, since we have had IDEF1X for Relational Data Modelling since
> >> 1983 (established and well-known international standard since 1993),
> >> do professors suppress that, and instead “teach”; push; shove;
> >> market; propagate the broken ERD, which is a pre-Relational artefact
> >> ?
> I think I have answered this above, at least by providing my own
> perspective.
Thanks.
As per the entire body of evidence, consistent, since 1970, they are actively suppressing the /RM/, and IDEF1X that goes with it. They are actively promoting pre-1970, pre-DBMS Record Filing Systems fraudulently, as “Relational”, along with the ancient; broken ERD that goes with it, and fraudulently calling it “Relational Data Modelling”.
Why ? Because their goal is the destruction of science, the suppression of truth, by causing confusion, by maintaining non-resolution, by pleading that falsity is “truth”, by denying reality, teh Four Laws of Thought.
> >> > In your model, each teacher must always be assigned to at least one
> >> > course. With
> >> >
> >> > (a) such an (unrealistic) constraint, *and*
> >> Why is that constraint “unrealistic” ? In the real world, it is
> >> a common; even pedestrian, requirement. We have been implementing
> >> such in Relational databases since 1983.
> I was thinking more in terms of "department employees", one of whose
> duties may be teaching (but they may not be teaching all the time). But
> sure, you may think in terms of employees who teach (a subset of all the
> employees): then they must teach something, of course.
Sorry, that is not what I meant.
Let me rephrase the question. Why do professors commonly labour and state that a 1::1-to-n relation is difficult, why do they pretend it is uncommon ? And how is such a relation (you correctly identify it as a Constraint) established in a Relational database (and in the data model erected in IDEF1X).
The core difference between pre-Relational theory; methods; and DBMS platforms vs the Relational Model; Relational Data Modelling; and RDBMS, is Relational Keys. The core of Relational Keys is Identity, the identity of the subject. To teach Data Modelling after say 1983 without teaching the relevance of Keys, without teaching precisely how to identify a subject, is criminal.
Here is a quick document that clarifies the issues you questioned, that I have answered. To be complete, I have added an item re Rolenames, because it is the third common “issue” that people raise:
https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/IDEF1X/Nicola%20IDEF1X%202.pdf
------------
-- Status --
------------
In essence, you have picked up IDEF1X, you have determined it is *THE* Standard for Relational Data Modelling, and you have started to use it. Excellent, the single academic who does. However, you are hindered by reading the nonsense that academics have to say about it (noting that they actively suppress it and instead promote ERD), which is as usual ignorant and incorrect, totally unscientific.
There are no problems or difficulties whatsoever in using IDEF1X in the entire exercise of genuine Relational Data Modelling. I have close to 100 such databases, some with over 500 tables (which is why and how I have developed Extensions). Instead, if you have any questions; any difficulty at all, please ask me. Hopefully you will progress much faster.
I would say, get more experience, and use data models that have some complexity (problems to solve).
(The only problem, if one could say such a thing exists, is that IDEF1X cannot enforce the /RM/, people can still do what ever they want. I have a remedy for that, but that is beyond the scope of this thread.)
----------------
-- Conclusion --
----------------
It is very sad that academics deny the real world; the industry; the science; the methods; the platforms, and instead indulge themselves in an isolated contrived “reality”, fixed in 1969. FIFTY YEARS OUT OF DATE.
We need to appreciate that as per the mountain of consistent evidence, the “theoreticians” teach the anti-Relational, pre-1970, pre-DBMS methods as “Relational”. FOR FIFTY YEARS. Now, categorically, as per the evidence, it is clear that the “theoreticians” do not teach the Standard Relational Data Modelling, but instead teach porcine excreta as “Relational Data Modelling”. FOR FORTY YEARS.
I don’t know how you don’t strangle your colleagues.
Cheers
Derek
Nicola
Mar 31, 2021, 7:58:03 AM3/31/21
to
On 2021-03-29, Derek Ignatius Asirvadem <derek.a...@gmail.com>
wrote:
academics" you mention? *Some* academics may have had issues with Codd's
proposal, but it was definitely understood, and supported, by many other
academics. And certainly at a greater degree than people in industry.
This is confirmed by those who were there:
https://archive.computerhistory.org/resources/access/text/2015/06/102702111-05-01-acc.pdf
E.g., quote from p. 8:
"So Codd's paper got some traction in the academic community where these
mathematical concepts were very familiar. It didn't, at first, get much
traction in the commercial database community. They had their
navigational systems. [...]"
Another first-hand historical perspective is provided by:
https://www.mcjones.org/system_r/sql_reunion_95/sqlr95.html
Quote from the "Prehistory" section:
"[...] Nevertheless, what kicked off this work [on project Gamma-0 and,
eventually, System R] was a key paper by Ted Codd - was it published in
1970 in CACM?
- Yes.
- A couple of us from the Systems Department had tried to read it -
couldn't make heads nor tails out of it. [laughter] At least back
then, it seemed like a very badly written paper: some industrial
motivation, and then right into the math. [laughter]"
Those are not academics talking.
Regarding the rest of your historical comments, I do not have anything
to add or object to. Except, perhaps, that the best DBMS in the world
apparently survives in legacy mode now.
> The point of this thread.
Let's come to the point.
>> On Wednesday, 24 March 2021 at 23:27:48 UTC+11, Nicola wrote:
>> One reason is Chen's Entity-Relationship models are likely perceived
>> as easier to grasp for novices than something like IDEF1X. Whether
>> that is a correct perception, it can be debated. Another reason is
>> probably just historical, due to the huge influence of Chen's seminal
>> paper.
>
> It may have been seminal for the academics, let me assure you, it was
> not seminal, or even great, for practitioners of the day. It does not
> have a semantic base, whereas the /RM/ and IDEF1X does.
>
> For academics. I can grant, up to 1970, the paper was amazing. After
> 1970, it was ageing, badly. By 1983, it was dead; obsolete; defunct;
> superseded.
Uh? Chen's ER paper is from 1976. That was a period of intense research
on so called "semantic modelling" (also spurred in part by Codd's work).
IDEF1X relies heavily upon Chen's shoulders. I'd say that the paper has
aged pretty well as we (let me simplify a bit) are still modelling in
terms of entities, relationships and attributes.
IMO, you are conflating the overall contributions of Chen (and others
working on semantic models), which are important, influential and still
relevant, with the specific details of the notation he invented, which
is not ideal for Relational database design.
> Ok, to rephrase the question, why was that pre-Relational method being
> taught after 1983, and why is it still being taught after the advent
> of the /RM/, after the /RM/was proved, and after the first genuine SQL
> platforms came out (1983), and after IDEF1X came out (1983 to
> practitioners, 1989 as a product, 1993 as an international standard) ?
I already told you my point of view.
Come out with a good database book, with a modern, sound, approach to
data modeling based on IDEF1X, and tell your publisher to send a copy
for evaluation to all academic database researchers. There is no
inherent reason why it should not get adopted because it dismisses ERDs
in favor of IDEF1X. Quite the opposite, actually. There is much need for
a really good reference on database design for the XXI century. I can't
recommend anything off the top of my head.
> after IDEF1X came out [in...] 1993 as an international standard)
The FIPS standard has been withdrawn by NIST in 2008. But IDEF1X has
also been an IEEE standard since 1998, and an ISO/IEC/IEEE standard
since 2012.
>> In a general setting, ERD is fine for teaching the basics of
>> modeling
>>
>> Some may feel IDEF1X is "low-level" (too many details) compared to
>> ERDs.
>
> False: display only the level of detail necessary for any given
> exercise {Entity|Key|Attribute}.
I agree. I did not mean to imply that I think otherwise; just that
a cursory look by someone who knows ERDs but not IDEF1X may lead to such
remarks.
>> Or some may have criticisms along the lines of this analysis:
>>
>> https://www.essentialstrategies.com/publications/modeling/idef1x.htm
>
> God help me. Academics just love to sabotage themselves, and then cry
> about the fact that they have been sabotaged.
Just brought it as an instance of what a superficial look at the
notation may lead to. I do not think that it is a good critical analysis
in any way. And not by an academic either.
>> My main complaint with IDEF1X is the treatment of generalization
>> hierarchies, in particular the lack of a correct way to model a total
>> inclusive specialization (A must be one of B1, ..., Bn and it may be
>> any of B1, ..., Bn); the approach I have seen around (e.g., see
>> Thomas Bruce's book or ERwin's manual) does not cover such case in
>> a sound way. And the standard never mentions inclusive
>> specializations, AFAICT.
>
> The problem there is, you do not understand Transactions
Look, as much as it may be disappointing to you: I know. Very well.
I know, in particular, how to deal with specialization hierarchies
transactionally.
> There are several threads in this forum that simply do not progress to
> closure due to the unwillingness of academics to engage, and to learn
> about ACID Transactions.
Yes, I did not feel those discussions were going towards fruitful
directions. But, time permitting, I might eventually return on the MVCC
vs 2PL matter.
> This document may assist somewhat in explanation, noting that the full
> and formal explanation lies in ACID Transactions, which has not
> progressed in this forum, and indeed in academia, since 1970.
>
> https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/Subtype.pdf
>
> If there is anything in that doc that is not completely understood,
> please ask.
That's perfectly clear.
>> (“If you drew all the relationships without dashed lines, you would
>> still be able to interpret your model in the same way"). I found it
>> difficult sometimes to convince them that such a distinction is
>> important.
>
> Only in the early stages of data modelling. Once the Keys are
> exposed, and worked, which is the bulk of the modelling exercise, that
> distinction is exposed, and difference is plain.
>
> When engaging the modelling exercise fully, meaning, in order, FOPC;
> the /RM; and IDEF1X as the rendition, and as you confirm “meaning is
> king”, you may realise: - the solid lines define ownership (Extension,
> of the Matter of the parent) - the dashed lines define classification
> (imposition of the Form of the parent)
That's clear, too.
>> Another criticism about IDEF1X is the lack of many-to-many
>> relationships (except in "Level 1"/"ER" diagrams), which sometimes
>> makes it awkward to read a relationship, because it is more natural
>> to skip the intermediate associative entity.
>
> You have the right idea, but not the precise method.
I'm good with your explanation, thanks.
>> >> > In your model, each teacher must always be assigned to at least
>> >> > one course. With
>> >> >
>> >> > (a) such an (unrealistic) constraint, *and*
>> >> Why is that constraint “unrealistic” ? In the real world, it is
>> >> a common; even pedestrian, requirement. We have been implementing
>> >> such in Relational databases since 1983.
>> I was thinking more in terms of "department employees", one of whose
>> duties may be teaching (but they may not be teaching all the time).
>> But sure, you may think in terms of employees who teach (a subset of
>> all the employees): then they must teach something, of course.
>
> Sorry, that is not what I meant.
>
> Let me rephrase the question. Why do professors commonly labour and
> state that a 1::1-to-n relation is difficult, why do they pretend it
> is uncommon ?
Ah, ok. Yes, on this point I agree that it depends on people not
thinking transactionally. I did *not* mean that 1::1-to-n constraints
are unrealistic!
> Here is a quick document that clarifies the issues you questioned,
> that I have answered. To be complete, I have added an item re
> Rolenames, because it is the third common “issue” that people raise:
> https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/IDEF1X/Nicola%20IDEF1X%202.pdf
A question: what is the use case for an association that does not imply
a foreign key (rightmost symbol in your "improved IE")? Why would you
require a parent entity to exist at the time of insertion, but not
later?
Regarding subtyping, my critique is specifically on the notation given
in ERwin Methods Guide at the crossing of "Complete" with "Inclusive
Subtype". Let me denote the specialization symbol with `-O||`, and
a categorization cluster with `P -O|| {C1, ..., Cn}`.
By definition, P -O|| {C1, ..., Cn} means:
"P must be exactly one of C1, ..., Cn".
If an entity P has two ore more clusters, each should be interpreted as
above. For instance:
P -O|| {A,B}
P -O|| {C,D}
reads as:
- P must be (exactly) one of A and B; and
- P must be (exactly) one of C and D.
The notation I'm criticizing corresponds to:
P -O|| {A}
P -O|| {B}
which, by the definition above, one should read as:
- P must be A; and
- P must be B.
Instead, the assumed interpretation is:
- P must be A or B; and
- P may be both A and B,
which is inconsistent with the general definition.
> I don’t know how you don’t strangle your colleagues.
It's for fear of law enforcement :D
Jokes apart, my database colleagues are among the best people in my
department!
Nicola
wrote:
> Introduction
>
> When it came out in 1970, the Relational Model (Codd, not the
> pretenders) made a paradigm shift in all areas of perceiving data
> (examination; data modelling; storage and retrieval; programming).
> Codd had an uphill battle against the established academics because
> they could not understand it,
I don't think that's a fair assessment. Who are those "established
>
> When it came out in 1970, the Relational Model (Codd, not the
> pretenders) made a paradigm shift in all areas of perceiving data
> (examination; data modelling; storage and retrieval; programming).
> Codd had an uphill battle against the established academics because
> they could not understand it,
academics" you mention? *Some* academics may have had issues with Codd's
proposal, but it was definitely understood, and supported, by many other
academics. And certainly at a greater degree than people in industry.
This is confirmed by those who were there:
https://archive.computerhistory.org/resources/access/text/2015/06/102702111-05-01-acc.pdf
E.g., quote from p. 8:
"So Codd's paper got some traction in the academic community where these
mathematical concepts were very familiar. It didn't, at first, get much
traction in the commercial database community. They had their
navigational systems. [...]"
Another first-hand historical perspective is provided by:
https://www.mcjones.org/system_r/sql_reunion_95/sqlr95.html
Quote from the "Prehistory" section:
"[...] Nevertheless, what kicked off this work [on project Gamma-0 and,
eventually, System R] was a key paper by Ted Codd - was it published in
1970 in CACM?
- Yes.
- A couple of us from the Systems Department had tried to read it -
couldn't make heads nor tails out of it. [laughter] At least back
then, it seemed like a very badly written paper: some industrial
motivation, and then right into the math. [laughter]"
Those are not academics talking.
Regarding the rest of your historical comments, I do not have anything
to add or object to. Except, perhaps, that the best DBMS in the world
apparently survives in legacy mode now.
> The point of this thread.
>> On Wednesday, 24 March 2021 at 23:27:48 UTC+11, Nicola wrote:
>> One reason is Chen's Entity-Relationship models are likely perceived
>> as easier to grasp for novices than something like IDEF1X. Whether
>> that is a correct perception, it can be debated. Another reason is
>> probably just historical, due to the huge influence of Chen's seminal
>> paper.
>
> It may have been seminal for the academics, let me assure you, it was
> not seminal, or even great, for practitioners of the day. It does not
> have a semantic base, whereas the /RM/ and IDEF1X does.
>
> For academics. I can grant, up to 1970, the paper was amazing. After
> 1970, it was ageing, badly. By 1983, it was dead; obsolete; defunct;
> superseded.
on so called "semantic modelling" (also spurred in part by Codd's work).
IDEF1X relies heavily upon Chen's shoulders. I'd say that the paper has
aged pretty well as we (let me simplify a bit) are still modelling in
terms of entities, relationships and attributes.
IMO, you are conflating the overall contributions of Chen (and others
working on semantic models), which are important, influential and still
relevant, with the specific details of the notation he invented, which
is not ideal for Relational database design.
> Ok, to rephrase the question, why was that pre-Relational method being
> taught after 1983, and why is it still being taught after the advent
> of the /RM/, after the /RM/was proved, and after the first genuine SQL
> platforms came out (1983), and after IDEF1X came out (1983 to
> practitioners, 1989 as a product, 1993 as an international standard) ?
Come out with a good database book, with a modern, sound, approach to
data modeling based on IDEF1X, and tell your publisher to send a copy
for evaluation to all academic database researchers. There is no
inherent reason why it should not get adopted because it dismisses ERDs
in favor of IDEF1X. Quite the opposite, actually. There is much need for
a really good reference on database design for the XXI century. I can't
recommend anything off the top of my head.
> after IDEF1X came out [in...] 1993 as an international standard)
The FIPS standard has been withdrawn by NIST in 2008. But IDEF1X has
also been an IEEE standard since 1998, and an ISO/IEC/IEEE standard
since 2012.
>> In a general setting, ERD is fine for teaching the basics of
>> modeling
>>
>> ERDs can be useful as a gentle introduction to such topics.
>>
>> ERD can be introduced without any background knowledge in databases
>
> For introductory levels, show them IDEF1X/Entity level.
Sure, I do that.
>>
>> ERD can be introduced without any background knowledge in databases
>
> For introductory levels, show them IDEF1X/Entity level.
>> Some may feel IDEF1X is "low-level" (too many details) compared to
>> ERDs.
>
> False: display only the level of detail necessary for any given
> exercise {Entity|Key|Attribute}.
a cursory look by someone who knows ERDs but not IDEF1X may lead to such
remarks.
>> Or some may have criticisms along the lines of this analysis:
>>
>> https://www.essentialstrategies.com/publications/modeling/idef1x.htm
>
> God help me. Academics just love to sabotage themselves, and then cry
> about the fact that they have been sabotaged.
notation may lead to. I do not think that it is a good critical analysis
in any way. And not by an academic either.
>> My main complaint with IDEF1X is the treatment of generalization
>> hierarchies, in particular the lack of a correct way to model a total
>> inclusive specialization (A must be one of B1, ..., Bn and it may be
>> any of B1, ..., Bn); the approach I have seen around (e.g., see
>> Thomas Bruce's book or ERwin's manual) does not cover such case in
>> a sound way. And the standard never mentions inclusive
>> specializations, AFAICT.
>
> The problem there is, you do not understand Transactions
I know, in particular, how to deal with specialization hierarchies
transactionally.
> There are several threads in this forum that simply do not progress to
> closure due to the unwillingness of academics to engage, and to learn
> about ACID Transactions.
directions. But, time permitting, I might eventually return on the MVCC
vs 2PL matter.
> This document may assist somewhat in explanation, noting that the full
> and formal explanation lies in ACID Transactions, which has not
> progressed in this forum, and indeed in academia, since 1970.
>
> https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/Subtype.pdf
>
> If there is anything in that doc that is not completely understood,
> please ask.
>> (“If you drew all the relationships without dashed lines, you would
>> still be able to interpret your model in the same way"). I found it
>> difficult sometimes to convince them that such a distinction is
>> important.
>
> Only in the early stages of data modelling. Once the Keys are
> exposed, and worked, which is the bulk of the modelling exercise, that
> distinction is exposed, and difference is plain.
>
> When engaging the modelling exercise fully, meaning, in order, FOPC;
> the /RM; and IDEF1X as the rendition, and as you confirm “meaning is
> king”, you may realise: - the solid lines define ownership (Extension,
> of the Matter of the parent) - the dashed lines define classification
> (imposition of the Form of the parent)
>> Another criticism about IDEF1X is the lack of many-to-many
>> relationships (except in "Level 1"/"ER" diagrams), which sometimes
>> makes it awkward to read a relationship, because it is more natural
>> to skip the intermediate associative entity.
>
> You have the right idea, but not the precise method.
>> >> > In your model, each teacher must always be assigned to at least
>> >> > one course. With
>> >> >
>> >> > (a) such an (unrealistic) constraint, *and*
>> >> Why is that constraint “unrealistic” ? In the real world, it is
>> >> a common; even pedestrian, requirement. We have been implementing
>> >> such in Relational databases since 1983.
>> I was thinking more in terms of "department employees", one of whose
>> duties may be teaching (but they may not be teaching all the time).
>> But sure, you may think in terms of employees who teach (a subset of
>> all the employees): then they must teach something, of course.
>
> Sorry, that is not what I meant.
>
> Let me rephrase the question. Why do professors commonly labour and
> state that a 1::1-to-n relation is difficult, why do they pretend it
> is uncommon ?
thinking transactionally. I did *not* mean that 1::1-to-n constraints
are unrealistic!
> Here is a quick document that clarifies the issues you questioned,
> that I have answered. To be complete, I have added an item re
> Rolenames, because it is the third common “issue” that people raise:
> https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/IDEF1X/Nicola%20IDEF1X%202.pdf
a foreign key (rightmost symbol in your "improved IE")? Why would you
require a parent entity to exist at the time of insertion, but not
later?
Regarding subtyping, my critique is specifically on the notation given
in ERwin Methods Guide at the crossing of "Complete" with "Inclusive
Subtype". Let me denote the specialization symbol with `-O||`, and
a categorization cluster with `P -O|| {C1, ..., Cn}`.
By definition, P -O|| {C1, ..., Cn} means:
"P must be exactly one of C1, ..., Cn".
If an entity P has two ore more clusters, each should be interpreted as
above. For instance:
P -O|| {A,B}
P -O|| {C,D}
reads as:
- P must be (exactly) one of A and B; and
- P must be (exactly) one of C and D.
The notation I'm criticizing corresponds to:
P -O|| {A}
P -O|| {B}
which, by the definition above, one should read as:
- P must be A; and
- P must be B.
Instead, the assumed interpretation is:
- P must be A or B; and
- P may be both A and B,
which is inconsistent with the general definition.
> I don’t know how you don’t strangle your colleagues.
Jokes apart, my database colleagues are among the best people in my
department!
Nicola
Nicola
Mar 31, 2021, 11:54:53 AM3/31/21
to
On 2021-03-31, Nicola <nic...@nohost.org> wrote:
> Regarding subtyping, my critique is specifically on the notation given
> in ERwin Methods Guide at the crossing of "Complete" with "Inclusive
> Subtype". Let me denote the specialization symbol with `-O||`, and
> a categorization cluster with `P -O|| {C1, ..., Cn}`.
> By definition, P -O|| {C1, ..., Cn} means:
>
> "P must be exactly one of C1, ..., Cn".
>
> If an entity P has two ore more clusters, each should be interpreted as
> above. For instance:
>
> P -O|| {A,B}
> P -O|| {C,D}
>
> reads as:
>
> - P must be (exactly) one of A and B; and
> - P must be (exactly) one of C and D.
>
> The notation I'm criticizing corresponds to:
>
> P -O|| {A}
> P -O|| {B}
>
> which, by the definition above, one should read as:
>
> - P must be A; and
> - P must be B.
>
> Instead, the assumed interpretation is:
>
> - P must be A or B; and
> - P may be both A and B,
>
> which is inconsistent with the general definition.
More precisely, it is inconsistent with IDEF1X's ISO (or FIPS) standard.
> Regarding subtyping, my critique is specifically on the notation given
> in ERwin Methods Guide at the crossing of "Complete" with "Inclusive
> Subtype". Let me denote the specialization symbol with `-O||`, and
> a categorization cluster with `P -O|| {C1, ..., Cn}`.
> By definition, P -O|| {C1, ..., Cn} means:
>
> "P must be exactly one of C1, ..., Cn".
>
> If an entity P has two ore more clusters, each should be interpreted as
> above. For instance:
>
> P -O|| {A,B}
> P -O|| {C,D}
>
> reads as:
>
> - P must be (exactly) one of A and B; and
> - P must be (exactly) one of C and D.
>
> The notation I'm criticizing corresponds to:
>
> P -O|| {A}
> P -O|| {B}
>
> which, by the definition above, one should read as:
>
> - P must be A; and
> - P must be B.
>
> Instead, the assumed interpretation is:
>
> - P must be A or B; and
> - P may be both A and B,
>
> which is inconsistent with the general definition.
In a previous post I wrote that ERwin's (i.e., Bruce's) interpretation
is not sound. That is not correct. The interpretation above is well
defined. It is not entirely satisfying, though, because it introduces an
exception in the interpretation of the complete categorization symbol:
the "completeness" (in the sense that each instance of the parent must
be an instance of one of the children) applies to the whole cluster
*set*, as opposed to each single cluster, when clusters have one child.
For instance, the figure from ERwin's Guide has two single-child
clusters:
+-----------+
| P |
+-----------+
| |
+--- ---+
| |
O O
----- -----
----- -----
| |
+---------+ +---------+
| A | | B |
+---------+ +---------+
and the assumed interpretation is that every instance of P must be an
instance A or of B (or both), as opposed to "every instance of P must be
A, and every instance of P must be B". The latter cannot be expressed
under ERwin's interpretation, but it not very important, because if
"every instance of P is always an instance of A" then A can be absorbed
into P.
IDEF1X's standard, on one hand, removes this exception, by stating
explicitly that, in each complete categorization cluster, each instance
of the parent must always be an instance of one of the children—even
when the cluster has only one child.
On the other hand, it makes it impossible to express the situation
above, i.e., that "P must be A, B, or both" (complete + inclusive).
I find it somewhat surprising that this aspect has not been clarified
and addressed in the standard, which was last revised in 2019.
Nicola
Derek Ignatius Asirvadem
Apr 2, 2021, 4:58:19 AM4/2/21
to
Nicola
Thanks for yours.
Before I get into it, let us understand something.
In academia, academics know nothing of the industry, they have a contrived notion of “industry”, namely anyone who uses PissGreNONsql.
Two exceptions. Codd, because he was a genuine academic who went beyond the academics of his time, and he had his head attached to reality (the commercial application). You, because you are trying to understand reality, and that sets you apart from the academics. The main block you have to get over is, the confidence that academics know reality, which is a total fiction.
If Industry means all who use a DBMS, up to ten years ago, basically it excluded freeware, and they used commercial DBMS, PissGreNONsql was unheard of. Where freeware is used, MyNONsql leads the field. It still does, it is 1,000 times better than PooGoo while remaining in the same NONsql category. But academia only know about the “industry” that uses their pet excreta producer.
Eg. in the Financial & Banking sector, Sybase has 95% share, with MSSQL the rest. Oracle cannot even compete. In high-performance markets (F&B being one category), it is Sybase and DB2, with MSSQL after a large gap. Oracle cannot compete. Freeware is unheard of, no one wants to execute at 10,000 times slower speed or rewrite their “sql” for every new version of the freeware. These are for databases that are permanent, not “refactored” every month or quarter, such as the ever-changing filth that OO/ORM crowd excretes, that academia caters for.
So the market that uses freeware is really the start-ups, the dumb dumb dumb dumb dummies, ignorant of what commercial DBMS provide, aware of only that fraction of DBMS capability that academia bleats about. Start-ups that will not exist in ten years. No one cares about them. Except academia. Because they too, use PooGrossNONsql. And they too have no clue about ACID Transactions or Server Architecture or the features in commercial DBMS that their pet freeware does not have, and will never have. Same as the academics.
In the commercial DBMS industry (ie. the DBMS suppliers), no one knows or cares about the academics, because they are, as evidenced, fifty years behind the industry. As evidenced, there is nothing that academics have come up with that has been implemented in commercial DBMS (except Codd, again). In Sybase (small company) I can assure you from decades of first-hand experience as a Partner, we have hundreds of PhD level scientists. It may even be 1,000, as the company has grown after the acquisition by SAP.
>>>>
Don’t include such things as the hysterical MVCC in that, because “MVCC” is not a real thing, it is a imaginary thing, collectively imagined and collectively assented to, by academia alone. All hail the imbecile Stonebraker and insist that he is a deity. As I posted in the “MVCC” thread, yes, due to the heavy marketing by academia, “MVCC” has become a check-box item, without being understood by those who request it. Thus Sybase have ADDED it on top of the [existing since 1960 in IBM; since 1977 in Britton-Lee; since 1981 in Sybase] Ordinary Locking (aka incorrectly called “Two Phased Locking” or “2PL” by the academics).
To be clear, if the boffin insists of imagining something that is not real, no problem, we add a set of resources, and we erect his fantasy for him. Meanwhile, regardless of what the boffin does, regardless of what multiverse he is fornicating with his beloved sow in, if he executes an SQL verb that does something to the database, it is done in the normal [forty years extant] “2PL” so that he contends with the other users (who are probably not slaves of academia and thus have an attachment to reality, the database content), and resolves that contention, and ADDITIONALLY, we erect his fantasy on the ADDITIONAL resources, and provide/deny his attempt against his fantasy that we have erected for him. It is the way a kind and gentle mental health nurse provides care for her mentally ill and locked up patients, locked up because they insist fantasy is “reality”, which is dangerous to the public.
<<<<
The above are all general points. If you wish to challenge any of them, do so as general points.
> On Wednesday, 31 March 2021 at 22:58:03 UTC+11, Nicola wrote:
> > On 2021-03-29, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
>
> > Introduction
> >
> > When it came out in 1970, the Relational Model (Codd, not the
> > pretenders) made a paradigm shift in all areas of perceiving data
> > (examination; data modelling; storage and retrieval; programming).
> > Codd had an uphill battle against the established academics because
> > they could not understand it,
> I don't think that's a fair assessment. Who are those "established
> academics" you mention?
The main fight was within IBM, with the likes of Date and Fagin, who pretended to be his supporters but as time has told, they are his subverters. There were others, more later.
There is absolutely zero articles in academia reinforcing or progressing the /RM/. That is evidence, for FIFTY YEARS, that academia does not understand it. All the nameless freaks, save one. Likewise there is not a single book that honestly promotes the /RM/ or progresses it. All of them that purport to articulate “Relational Database” promote instead 1960’s Record Filing Systems, based on RM/T, as confirmed in another thread. Even here, you are the only one looking into genuine Relational Data Modelling. The rest promote anti-Relational pig poop.
> *Some* academics may have had issues with Codd's
> proposal, but it was definitely understood, and supported, by many other
> academics.
Name one (prior to 1980, because that was the statement I made). After 1981, when Sybase came out, and the IBM product (many names) was established, yes, academia could no longer deny it.
Try naming one paper written any time after 1970.
> And certainly at a greater degree than people in industry.
> This is confirmed by those who were there:
>
> https://archive.computerhistory.org/resources/access/text/2015/06/102702111-05-01-acc.pdf
>
> E.g., quote from p. 8:
>
> "So Codd's paper got some traction in the academic community where these
> mathematical concepts were very familiar. It didn't, at first, get much
> traction in the commercial database community. They had their
> navigational systems. [...]"
That is not “industry”, that is one voice, and a self-serving one, an academic with no clue about the industry.
I, who worked for one of the big five DBMS vendors at the time, and previously a specialist on DEC (PDP; VAX) have already categorised it as false. Let me assure you that at Cincom, we had many scientists, many mathematicians, who had no problem understanding the /RM/. Yes, we saw the /RM/ as changing the entire landscape. We did not try to “relationalise” the product, thus we had no need to add anything to TOTAL, but the other DBMS vendors did. And that is why Codd had to write his famous Twelve Rules, to ensure that vendors were not fraudulently declaring that they were /RM/ compliant.
(The has progressed to the problem we have today, where even though SQL is THE established Standard, all the freeware and oracle do not comply, but fraudulently declare that they do.)
Likewise, let me assure you that DEC had hundreds of scientists, who understood and implemented the /RM/, their RDBMS called Rdb was brilliant. It still runs in legacy systems on VAX emulators, it still beats Orable by 10,000 times. I am not saying that emulators are good, but there is a market for it, patronised by corporations that do not fix what is NOT broken, which is why they still exist.
That (what you quote) is typical of the propaganda that academia produce and swallow. Waht you might get from guys like me (or any scientist working for a commercial DBMS provider)is, the actual reality. Obviously the thousands of scientists who work for IBM; Sybase; MS, do not even bother with fora such as this, because academics are irrelevant to the real world platforms, only I do.
Chamberlain contradicts himself all over the place, but you won’t catch it. He comes across as being a friend of Codd, but actually, if you examine it, he demeans Codd, sometimes quite subtely. Very dishonest. It is sad that academics view the Chamberlain memories as gospel. Identical to Date. Much like the Stonebraker stupidity, that has no commercial existence, but lives and thrives in academia and in freeware.
On p12 para 2:
“System R ... was not a product that was going to be released commercially. It was a feasibility demonstration. It was proof of the concept that Ted Codd's ideas actually made sense and could be implemented in a robust way. And there was a lot of skepticism about that.”
So the simple evidenced fact, from Chamberlains own mouth, his “memories”, confirms what I said, and he includes himself in that category:
>> Codd had an uphill battle against the established academics because
>> they could not understand it
On p8 para 1:
“Ted thought that we should use this concept [relations] to allow users to interact with stored data in a way that was independent of the physical representation of that data. He called this notion //data independence//”
And on p 8 para 3:
“Codd introduced the notion of relations as an abstraction for stored data, and he explained about //data independence// and why that was important”
The term in the /RM/ is Access Path Independence. Data Independence is quite a different thing.
It is amazing to me, that Chamberlain enumerates all those great mathematicians who were employed by his employer, specifically to go beyond their existing product, but still insists that DBMS providers did not have mathematicians, and were only interested in protecting their market share. Only people who are trained to accept contradiction without question can believe such utter hypocrisy. Pig poop rots the brain.
Here is what Chamberlain actually thought that the “industry” was; what the user base was, and his own admission that what he thought was absurd. Proving yet again that academics are clueless about the industry the purport to serve. p 17 para 5:
“What Ray and I thought we were doing was making access to databases available to a new class of people that hadn’t accessed databases before. We thought they were going to be engineers and architects and city planners and economists, and people like that who were not computer people, not programmers. We thought that these people shouldn’t have to turn themselves into computer experts or programmers in order to ask questions. They should ask questions in this English-keyword notation that we called SEQUEL. So with words like SELECT and FROM and WHERE and GROUP BY and ORDER BY and so on, maybe you could teach that kind of notation to people who didn’t have any computer training at all, and weren’t much interested in computers. They were interested in their domain of expertise. They wanted to study economics or something like that, and they needed to retrieve statistical information about economics, and they needed a language to do that, but they didn’t need to be programmers. That’s what SEQUEL was intended to do. I don’t think it was very successful at doing that. If you look back on it from a perspective of 30 years later, I think you don’t find a lot of architects and economists and city planners writing SEQUEL queries. SEQUEL queries are mostly written by experienced programmers who are trained and specialized in doing that, and a lot of them are generated by machines through some kind of interface that’s specialized to some domain.”
I am not suggesting that he, or any other academic at IBM Labs was stupid. Far from it. I am saying that as an academic, as evidenced, he has huge blind spots, eg. “industry”; “users”. And as is typical of academia, he is unaware that he has blind spots, and makes stupid declarations about them.
Overall, notice the difference, Chamberlain and a host of others (academic but impractical), against Codd (academic and practical). Codd was the bad guy because “he wasn’t a team player”. Thank God for that.
It is horrendous that he does not mention DSS queries in his discourse on that subject. In his /RM/, Codd provided both OLTP and DSS.
> Another first-hand historical perspective is provided by:
>
> https://www.mcjones.org/system_r/sql_reunion_95/sqlr95.html
404 - Not Found.
They probably realised that it was going to be inspected and removed the filth, because it would not withstand scrutiny.
MCJones is the same idiot (nicely propagandised, well prepared) who interviewed Chamberlain, and who dutifully spouted the same propaganda. I don’t need to read it to know that it will be in the same vein.
> Quote from the "Prehistory" section:
>
> "[...] Nevertheless, what kicked off this work [on project Gamma-0 and,
> eventually, System R] was a key paper by Ted Codd - was it published in
> 1970 in CACM?
>
> - Yes.
>
> - A couple of us from the Systems Department had tried to read it -
> couldn't make heads nor tails out of it. [laughter] At least back
> then, it seemed like a very badly written paper: some industrial
> motivation, and then right into the math. [laughter]"
>
> Those are not academics talking.
Agreed. Those are not capable practitioners either.
Again, at IBM; Cincom; Honeywell; DEC; Cullinane; etc, we had [academically qualified] scientists, rather than academics. And we laughed at academics and their isolation from reality even then. Nothing has changed, it is much worse now. Once a thing is in the toilet bowl, the only way forward, is down down down.
> Regarding the rest of your historical comments, I do not have anything
> to add or object to. Except, perhaps, that the best DBMS in the world
> apparently survives in legacy mode now.
Yes. Corps that exist beyond the start-up ten years don’t fix something that is not broken. After 9/11, Wall Street corps scoured the world for 2GB disk drives. (I don’t know if you will appreciate the relevance of that.)
> > The point of this thread.
>
> Let's come to the point.
>
> >> On Wednesday, 24 March 2021 at 23:27:48 UTC+11, Nicola wrote:
> >> One reason is Chen's Entity-Relationship models are likely perceived
> >> as easier to grasp for novices than something like IDEF1X. Whether
> >> that is a correct perception, it can be debated. Another reason is
> >> probably just historical, due to the huge influence of Chen's seminal
> >> paper.
> >
> > It may have been seminal for the academics, let me assure you, it was
> > not seminal, or even great, for practitioners of the day. It does not
> > have a semantic base, whereas the /RM/ and IDEF1X does.
> >
> > For academics. I can grant, up to 1970, the paper was amazing. After
> > 1970, it was ageing, badly. By 1983, it was dead; obsolete; defunct;
> > superseded.
>
> Uh? Chen's ER paper is from 1976. That was a period of intense research
> on so called "semantic modelling" (also spurred in part by Codd's work).
> IDEF1X relies heavily upon Chen's shoulders.
First let me clarify my statements above.
> > For academics. I can grant, [if it were published] up to 1970, the paper was amazing.
> > [Since it was published] After 1970, it was ageing, badly.
> > By 1983 [when RDBMS platforms were freely available, several providers, AND IDEF1X was freely available], it was dead; obsolete; defunct; superseded.
The fact that Chen published it in 1976, and that academics performed “intense research” is precisely what I was talking about. Hysterical absurdity, because the /RM/ was published and being accepted, because several vendors were coming out with genuine RDBMS Platforms. Anyone with half a brain (eg. the scientists at IBM; Cincom; DEC; and Britton-Lee) laughed at it, because we were implementing the /RM/, and because the /RM? was far superior to the idiotic ERD.
Do you understand, and appreciate:
- we had ISAM prior to 1960’s & 1970’s DBMS
--- that means we understood KEYS
- separate to being based on FOPC, and therefore a mathematical model, the /RM/ is based on (a) the extant HDBMS, and (b) the extant NDBMS, as detailed in this thread ?
--- Hierarchical Model and its Relevance in the Relational Model
--- https://groups.google.com/g/comp.databases.theory/c/5212JwYtip4
--- HDBMS TECHNOLOGY IS HEAVILY KEY-BASED (of course, the storage is pointer-based)
--- The /RM/ takes its notion of KEYS from HDBMS, and the notion of Independent/Dependent from NDBMS
ERD is devoid of KEYS, of any understanding of KEYS, and since it was 1976, is [yet again] anti-Relational.
That the academics salivated over it, and did their mutual circle-jerk with “intense research” is exactly what I am talking about: it is hysterical absurdity, that can only be done in reinforced isolation from reality, from the /RM/; from the actual [R]DBMS platforms available an becoming available.
> IDEF1X relies heavily upon Chen's shoulders.
Nonsense. Give one example, one article. Rob Brown already had a diagrammatic modelling method (as distinct from the various diagrams that data modellers created) that was widely accepted ... after consultation with Codd, he produced what became known as IDEF1X.
> I'd say that the paper has
> aged pretty well as we (let me simplify a bit) are still modelling in
> terms of entities, relationships and attributes.
I said the same thing in different words. As long as “we” means “we academics”, I agree. The idiotic broken method is being taught throughout academia, fraudulently, as “Relational Data Modelling”. In pathological denial of reality (FOPC; the /RM/; IDEF1X; actual Relational Data Modelling)
> IMO, you are conflating the overall contributions of Chen (and others
> working on semantic models), which are important, influential and still
> relevant, with the specific details of the notation he invented, which
> is not ideal for Relational database design.
I couldn’t have been “conflating” what you say I did, because you have introduced two new items that were not on the table at the time at which I made my comments. Yes, I rejected Chen; the ERD.
Now that you have added Semantic Models, a new response from me. Yes, I reject any and all academic endeavours re Data Modelling (Relational or other). Because the notion is hysterically absurd. For a scientist. But not for the academics, who are by definition [given your reports that they are/were working on “semantic models”] ignorant of the science. If you challenge this, please open a separate thread “Semantic Data Model”.
> > Ok, to rephrase the question, why was that pre-Relational method being
> > taught after 1983, and why is it still being taught after the advent
> > of the /RM/, after the /RM/was proved, and after the first genuine SQL
> > platforms came out (1983), and after IDEF1X came out (1983 to
> > practitioners, 1989 as a product, 1993 as an international standard) ?
> I already told you my point of view.
>
> Come out with a good database book, with a modern, sound, approach to
> data modeling based on IDEF1X, and tell your publisher to send a copy
> for evaluation to all academic database researchers. There is no
> inherent reason why it should not get adopted because it dismisses ERDs
> in favor of IDEF1X. Quite the opposite, actually. There is much need for
> a really good reference on database design for the XXI century. I can't
> recommend anything off the top of my head.
Gee, thanks. That was not what I was expecting, 8}.
But you didn’t answer the question. Other than implying that nothing exists in the entire collection of academic multiverses. So my accusation holds: academics do suppress the /RM/ for theory, while promoting the anti-Relational RFS as “Relational” (as proved in several threads), and now, academics do suppress IDEF1X for Data Modelling, while promoting instead the anti-Relational ERD.
Why would the academics promote my book ??? It destroys the academic mindset, it asserts science and reality, it gives Methods that have existed for FORTY YEARS that academics deny, it gives Progressions to the /RM/ and the Methods, both of which the academics deny. No. At best, I will find a few honest academics who will provide a peer review.
But the book is not expected to be received by the pig poop eaters, it is for practitioners who respect a theoretical foundation. But thanks again.
Data Modelling is not “based on IDEF1X” (or ERD for that matter). Data Modelling is a science. Yes, I have a definition, and a Method, rendered in IDEF0. IDEF1X and IDEF0 are merely Standards, not methods. It gives low-level rules, which provide a simple but incomplete form of rules [defined in the /RM/]. Which, is not THE method or A method for modelling data. It pertains to the finished data model (any particular version), rather than the exercise that is required to produce it. For the purpose of having agreed symbols; completeness; etc. Such that two parties can discuss a Data Model that is /rendered in/ a well-known Standard.
Sure, a person CAN perform Data Modelling “based on IDEF1X”, it would be a class above and beyond any data modelling based on ERD, but it would still be inferior to actual data modelling.
You saw what I did with Movie Title, and to a lesser degree what I did with the three ERDs in OP’s request, you should have more than a whiff. The Vehicle-Accident problem was a good one as well. All three sets of problems exposed two things:
- use of ERD made sure the modeller was stuck; stumped; and stupefied
- classic data modelling [with /RM/ in mind] solved the problem in a straight-forward manner
The simple and explicit crime, that I am trying to get across is, academia has failed in its duty to teach the science, the fact that they teach ERD as “Relational Data Modelling” is a serious fraud, it is subversion of the science, and easily identified as such by those who are not ignorant.
> > after IDEF1X came out [in...] 1993 as an international standard)
>
> The FIPS standard has been withdrawn by NIST in 2008. But IDEF1X has
> also been an IEEE standard since 1998, and an ISO/IEC/IEEE standard
> since 2012.
Yes. So the fact that the NIST Standard was withdrawn is irrelevant, by your own report. No one stopped using it. Even the deaf and blind OO/ORM crowd use a bastardised version of it.
> >> In a general setting, ERD is fine for teaching the basics of
> >> modeling
> >>
> >> ERDs can be useful as a gentle introduction to such topics.
> >>
> >> ERD can be introduced without any background knowledge in databases
> >
> > For introductory levels, show them IDEF1X/Entity level.
>
> Sure, I do that.
So then, you do know, IDEF1X with its full /RM/ articulation (Relational Keys; Independent/Dependent; Identifying/Non-Identifying; Subtypes; etc), its full IEEE Cardinality, even at the Entity level, provides far more richness and definition than ERD. So why are you suggesting ERD “is fine for ...” anything, post 1970 ???
> >> Some may feel IDEF1X is "low-level" (too many details) compared to
> >> ERDs.
> >
> > False: display only the level of detail necessary for any given
> > exercise {Entity|Key|Attribute}.
>
> I agree. I did not mean to imply that I think otherwise; just that
> a cursory look by someone who knows ERDs but not IDEF1X may lead to such
> remarks.
Who cares about what people, ignorant of subject X, say about subject X. It is like a pygmy insisting that a wattle-and-dung hut is the best house in the world. Or Date and Darwen insisting that the /RM/ or SQL is incomplete, and that they will produce something that soves all those problems (still failing, after THIRTY YEARS). By definition, it proves they are ignorant of the /RM/ and SQL, that they are talking about their Straw Man, RM/T plus RFS re-framed as the “Relational Model”. I would prefer if you keep that sort of remark out of our communications.
The only remarks that count; that are worthy of discussion, are from people who know IDEF1X, or both IDEF1X and ERD.
> >> Or some may have criticisms along the lines of this analysis:
> >>
> >> https://www.essentialstrategies.com/publications/modeling/idef1x.htm
> >
> > God help me. Academics just love to sabotage themselves, and then cry
> > about the fact that they have been sabotaged.
>
> Just brought it as an instance of what a superficial look at the
> notation may lead to. I do not think that it is a good critical analysis
> in any way. And not by an academic either.
Ditto.
> >> My main complaint with IDEF1X is the treatment of generalization
> >> hierarchies, in particular the lack of a correct way to model a total
> >> inclusive specialization (A must be one of B1, ..., Bn and it may be
> >> any of B1, ..., Bn); the approach I have seen around (e.g., see
> >> Thomas Bruce's book or ERwin's manual) does not cover such case in
> >> a sound way. And the standard never mentions inclusive
> >> specializations, AFAICT.
> >
> > The problem there is, you do not understand Transactions
>
> Look, as much as it may be disappointing to you: I know. Very well.
> I know, in particular, how to deal with specialization hierarchies
> transactionally.
Ok, fine. But your words contradict that, you can’t say that and also say “My main complaint with IDEF1X is the treatment of generalization hierarchies”.
So what, if anything, is your remaining complaint re “generalization hierarchies”.
Re Transactions. Again, there is contradiction. If you understand ACID, you would reject MVCC. If you accept MVCC even as a potential (it has never reached actual), it proves you don’t understand ACID. Sorry, I am not going by what I think, I am going by your words, and what they actually mean.
> > There are several threads in this forum that simply do not progress to
> > closure due to the unwillingness of academics to engage, and to learn
> > about ACID Transactions.
>
> Yes, I did not feel those discussions were going towards fruitful
> directions.
The very least in the category of “fruitful” is closure. Since you (as distinct from other academics) at least accept the notion of the Four Laws of Thought, which I am rigidly attached to, closure should be no problem. (Closure with other academics is not possible, simply because they are not scientifically honest: refer the multitude of threads, wherein the best they can do is ad hominem attacks; deflection; Straw Man; etc.)
> But, time permitting, I might eventually return on the MVCC
> vs 2PL matter.
Very important. Please and thank you.
> > This document may assist somewhat in explanation, noting that the full
> > and formal explanation lies in ACID Transactions, which has not
> > progressed in this forum, and indeed in academia, since 1970.
> >
> > https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/Subtype.pdf
> >
> > If there is anything in that doc that is not completely understood,
> > please ask.
> That's perfectly clear.
So you understand, Subtypes of any IEEE legal kind, can be implemented in SQL, using Constraints only (and ACID Transactions are a form of Constraint, the collection of which is the Database API).
> > Here is a quick document that clarifies the issues you questioned,
> > that I have answered. To be complete, I have added an item re
> > Rolenames, because it is the third common “issue” that people raise:
> > https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/IDEF1X/Nicola%20IDEF1X%202.pdf
>
> A question: what is the use case for an association that does not imply
> a foreign key (rightmost symbol in your "improved IE")? Why would you
> require a parent entity to exist at the time of insertion, but not
> later?
It is actually a very common requirement. Refer this Data Model:
____https://www.softwaregems.com.au/Documents/Student_Resolutions_2020/Giannis/RoomMonitor%20DM%20V0_51.pdf
The initial requirement is here, but he contacted me directly, and we progressed the Data Modelling exercise to completion off StackOverflow, in 25 iterations, which cannot be done at SO, which sucks dead bears. Read it only if interested, to get a feel, the detail in the *Q* is completely obsolete:
____https://stackoverflow.com/questions/61048436/modeling-optional-properties-of-an-entity-based-on-a-mutable-discriminator-in-a
The system distributes Room Monitors (Sensors: Temp; Humidity; DoorOpen; etc) to various hotels (Pensions) in Europe. The Monitors are produced; owned; and managed by the single company, but distributed widely to the locations. The Hotel owns the Network and actually receives the Sensor info (not the central corp). Over time, the specific hardware for a single Monitor gets removed from the Hotel and its Network; back to factory; then re-distributed to some other Hotel.
1. Guy needs the Monitor to be assigned properly in the first place (integrity with parent HotelNetwork), and it needs to be known as such for historical or audit purposes. But not maintained, eg. when it is removed. Therefore an FK Constraint will not work. The Constraint is implemented within an ACID Transaction:
____+Monitor CHECKS HotelNetwork[HotelChain, CountryCode, Town, Suburb) EXISTS.
2. Readings for a Room are stored permanently, for audit and legal reasons, which means Subordinate to Room, not Monitor. But of course, when a Reading is added, we need to ensure that it was produced by a particular Sensor, in a particular Monitor, that was located in that Room.
____Constraint ReadingMeasure_SensorIsValid_ck CHECK Sensor_Get_fn( Sensor.PK )
____which checks that the parent SensorSwitch EXISTS
which applies to the INSERT within the ACID Transaction only.
For understanding, I would say, there are three levels of maintaining history (Audit, etc), for designated tables, and it has some Temporal Database requirement.
1 At most, the requirement is to re-construct a row from the past. Separate table, and very tight KEYS plua DATETIME. Tight understanding of Temporal (not the Date and Darwen monstrosity marketed as “temporal”) is required, more for the coding than the data model.
3 At the least, it is what we have here in Room Monitor. Data is valid at INSERT, but not afterwards. It is not that a SELECT will fail, all SELECTS for those tables have to use OUTER JOIN, with ISNULL( “[Expired]” ) for the potentially non-existent parent. No understanding of Temporal is required.
2. In-between is the most common. When queried, the correct historic value must show up (but no need to re-construct). Reasonable understanding of Temporal (not the Date and Darwen monstrosity marketed as “temporal”) is required.
> Regarding subtyping, my critique is specifically on the notation given
> in ERwin Methods Guide at the crossing of "Complete" with "Inclusive
> Subtype".
What “crossing” ???
You can’t use both IDEF1X and IEEE notation in any single model, they cannot be mixed. They are exclusive. You can’t go back-and-forth either.
> Let me denote the specialization symbol with `-O||`, and
> a categorization cluster with `P -O|| {C1, ..., Cn}`.
> By definition, P -O|| {C1, ..., Cn} means:
>
> "P must be exactly one of C1, ..., Cn".
>
> If an entity P has two ore more clusters, each should be interpreted as
> above. For instance:
>
> P -O|| {A,B}
> P -O|| {C,D}
>
> reads as:
>
> - P must be (exactly) one of A and B; and
> - P must be (exactly) one of C and D.
Understood thus far. For two clusters, agreed thus far. In IEEE notation, it is perfectly clear. You might have to use something like “P--€{A,B}” for Exclusive, and “P--C{C,D}” for Non-Exclusive. You have two Exclusive clusters.
Since C is the classifier in your definition, I would prefer:
____P--€{A1,A2..An}
____P--C{X1,X2..Xm}
(“Inclusive” is incorrect, what we have is not the simple opposite of Exclusive. If it is not “Base is exactly one Subtype” which is Exclusive, then “Base is one or more Subtype” is simply Non-Exclusive. Nothing inclusive about that.)
> The notation I'm criticizing corresponds to:
>
> P -O|| {A}
> P -O|| {B}
>
> which, by the definition above, one should read as:
>
> - P must be A; and
> - P must be B.
>
> Instead, the assumed interpretation is:
>
> - P must be A or B; and
> - P may be both A and B,
Sorry. I don’t understand what you are trying to say.
If you mean the IDEF1X notation for Exclusive, no it does not exist, only {Complete|Incomplete} exists in IDEF1X notation. Don’t use the IDEF1X notation for Subtypes, we know it is inferior to the pre-existing IEEE notation, that is why we replaced the former with the latter.
Again, you can’t go back-and-forth: if you have interpreted the ERwin Methods Guide p 69 as allowing that, no, that is false. It is a diagram for comparison only. Use IDEF1X XOR IEEE Notation.
> which is inconsistent with the general definition.
>
> > I don’t know how you don’t strangle your colleagues.
>
> It's for fear of law enforcement :D
I love that !
> Jokes apart, my database colleagues are among the best people in my
> department!
Evidently, Udine is far better than US universities. But ... first school them in the Four Laws of Thought, the foundation of all science and of Justice. That will ensure determination of science vs anti-science. Then tell them to read Codd/RM, only Codd/RM, and nothing but Codd/RM. The RM/T; etc, has to be rejected because it contradicts the Codd/RM. Tell them NOT to read any other academic.
----
Since we are discussing IDEF1X, you might be interested in this Q&A:
____https://stackoverflow.com/q/4132044/484814
Cheers
Derek
Thanks for yours.
Before I get into it, let us understand something.
In academia, academics know nothing of the industry, they have a contrived notion of “industry”, namely anyone who uses PissGreNONsql.
Two exceptions. Codd, because he was a genuine academic who went beyond the academics of his time, and he had his head attached to reality (the commercial application). You, because you are trying to understand reality, and that sets you apart from the academics. The main block you have to get over is, the confidence that academics know reality, which is a total fiction.
If Industry means all who use a DBMS, up to ten years ago, basically it excluded freeware, and they used commercial DBMS, PissGreNONsql was unheard of. Where freeware is used, MyNONsql leads the field. It still does, it is 1,000 times better than PooGoo while remaining in the same NONsql category. But academia only know about the “industry” that uses their pet excreta producer.
Eg. in the Financial & Banking sector, Sybase has 95% share, with MSSQL the rest. Oracle cannot even compete. In high-performance markets (F&B being one category), it is Sybase and DB2, with MSSQL after a large gap. Oracle cannot compete. Freeware is unheard of, no one wants to execute at 10,000 times slower speed or rewrite their “sql” for every new version of the freeware. These are for databases that are permanent, not “refactored” every month or quarter, such as the ever-changing filth that OO/ORM crowd excretes, that academia caters for.
So the market that uses freeware is really the start-ups, the dumb dumb dumb dumb dummies, ignorant of what commercial DBMS provide, aware of only that fraction of DBMS capability that academia bleats about. Start-ups that will not exist in ten years. No one cares about them. Except academia. Because they too, use PooGrossNONsql. And they too have no clue about ACID Transactions or Server Architecture or the features in commercial DBMS that their pet freeware does not have, and will never have. Same as the academics.
In the commercial DBMS industry (ie. the DBMS suppliers), no one knows or cares about the academics, because they are, as evidenced, fifty years behind the industry. As evidenced, there is nothing that academics have come up with that has been implemented in commercial DBMS (except Codd, again). In Sybase (small company) I can assure you from decades of first-hand experience as a Partner, we have hundreds of PhD level scientists. It may even be 1,000, as the company has grown after the acquisition by SAP.
>>>>
Don’t include such things as the hysterical MVCC in that, because “MVCC” is not a real thing, it is a imaginary thing, collectively imagined and collectively assented to, by academia alone. All hail the imbecile Stonebraker and insist that he is a deity. As I posted in the “MVCC” thread, yes, due to the heavy marketing by academia, “MVCC” has become a check-box item, without being understood by those who request it. Thus Sybase have ADDED it on top of the [existing since 1960 in IBM; since 1977 in Britton-Lee; since 1981 in Sybase] Ordinary Locking (aka incorrectly called “Two Phased Locking” or “2PL” by the academics).
To be clear, if the boffin insists of imagining something that is not real, no problem, we add a set of resources, and we erect his fantasy for him. Meanwhile, regardless of what the boffin does, regardless of what multiverse he is fornicating with his beloved sow in, if he executes an SQL verb that does something to the database, it is done in the normal [forty years extant] “2PL” so that he contends with the other users (who are probably not slaves of academia and thus have an attachment to reality, the database content), and resolves that contention, and ADDITIONALLY, we erect his fantasy on the ADDITIONAL resources, and provide/deny his attempt against his fantasy that we have erected for him. It is the way a kind and gentle mental health nurse provides care for her mentally ill and locked up patients, locked up because they insist fantasy is “reality”, which is dangerous to the public.
<<<<
The above are all general points. If you wish to challenge any of them, do so as general points.
> On Wednesday, 31 March 2021 at 22:58:03 UTC+11, Nicola wrote:
> > On 2021-03-29, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
>
> > Introduction
> >
> > When it came out in 1970, the Relational Model (Codd, not the
> > pretenders) made a paradigm shift in all areas of perceiving data
> > (examination; data modelling; storage and retrieval; programming).
> > Codd had an uphill battle against the established academics because
> > they could not understand it,
> I don't think that's a fair assessment. Who are those "established
> academics" you mention?
There is absolutely zero articles in academia reinforcing or progressing the /RM/. That is evidence, for FIFTY YEARS, that academia does not understand it. All the nameless freaks, save one. Likewise there is not a single book that honestly promotes the /RM/ or progresses it. All of them that purport to articulate “Relational Database” promote instead 1960’s Record Filing Systems, based on RM/T, as confirmed in another thread. Even here, you are the only one looking into genuine Relational Data Modelling. The rest promote anti-Relational pig poop.
> *Some* academics may have had issues with Codd's
> proposal, but it was definitely understood, and supported, by many other
> academics.
Try naming one paper written any time after 1970.
> And certainly at a greater degree than people in industry.
> This is confirmed by those who were there:
>
> https://archive.computerhistory.org/resources/access/text/2015/06/102702111-05-01-acc.pdf
>
> E.g., quote from p. 8:
>
> "So Codd's paper got some traction in the academic community where these
> mathematical concepts were very familiar. It didn't, at first, get much
> traction in the commercial database community. They had their
> navigational systems. [...]"
I, who worked for one of the big five DBMS vendors at the time, and previously a specialist on DEC (PDP; VAX) have already categorised it as false. Let me assure you that at Cincom, we had many scientists, many mathematicians, who had no problem understanding the /RM/. Yes, we saw the /RM/ as changing the entire landscape. We did not try to “relationalise” the product, thus we had no need to add anything to TOTAL, but the other DBMS vendors did. And that is why Codd had to write his famous Twelve Rules, to ensure that vendors were not fraudulently declaring that they were /RM/ compliant.
(The has progressed to the problem we have today, where even though SQL is THE established Standard, all the freeware and oracle do not comply, but fraudulently declare that they do.)
Likewise, let me assure you that DEC had hundreds of scientists, who understood and implemented the /RM/, their RDBMS called Rdb was brilliant. It still runs in legacy systems on VAX emulators, it still beats Orable by 10,000 times. I am not saying that emulators are good, but there is a market for it, patronised by corporations that do not fix what is NOT broken, which is why they still exist.
That (what you quote) is typical of the propaganda that academia produce and swallow. Waht you might get from guys like me (or any scientist working for a commercial DBMS provider)is, the actual reality. Obviously the thousands of scientists who work for IBM; Sybase; MS, do not even bother with fora such as this, because academics are irrelevant to the real world platforms, only I do.
Chamberlain contradicts himself all over the place, but you won’t catch it. He comes across as being a friend of Codd, but actually, if you examine it, he demeans Codd, sometimes quite subtely. Very dishonest. It is sad that academics view the Chamberlain memories as gospel. Identical to Date. Much like the Stonebraker stupidity, that has no commercial existence, but lives and thrives in academia and in freeware.
On p12 para 2:
“System R ... was not a product that was going to be released commercially. It was a feasibility demonstration. It was proof of the concept that Ted Codd's ideas actually made sense and could be implemented in a robust way. And there was a lot of skepticism about that.”
So the simple evidenced fact, from Chamberlains own mouth, his “memories”, confirms what I said, and he includes himself in that category:
>> Codd had an uphill battle against the established academics because
>> they could not understand it
“Ted thought that we should use this concept [relations] to allow users to interact with stored data in a way that was independent of the physical representation of that data. He called this notion //data independence//”
And on p 8 para 3:
“Codd introduced the notion of relations as an abstraction for stored data, and he explained about //data independence// and why that was important”
The term in the /RM/ is Access Path Independence. Data Independence is quite a different thing.
It is amazing to me, that Chamberlain enumerates all those great mathematicians who were employed by his employer, specifically to go beyond their existing product, but still insists that DBMS providers did not have mathematicians, and were only interested in protecting their market share. Only people who are trained to accept contradiction without question can believe such utter hypocrisy. Pig poop rots the brain.
Here is what Chamberlain actually thought that the “industry” was; what the user base was, and his own admission that what he thought was absurd. Proving yet again that academics are clueless about the industry the purport to serve. p 17 para 5:
“What Ray and I thought we were doing was making access to databases available to a new class of people that hadn’t accessed databases before. We thought they were going to be engineers and architects and city planners and economists, and people like that who were not computer people, not programmers. We thought that these people shouldn’t have to turn themselves into computer experts or programmers in order to ask questions. They should ask questions in this English-keyword notation that we called SEQUEL. So with words like SELECT and FROM and WHERE and GROUP BY and ORDER BY and so on, maybe you could teach that kind of notation to people who didn’t have any computer training at all, and weren’t much interested in computers. They were interested in their domain of expertise. They wanted to study economics or something like that, and they needed to retrieve statistical information about economics, and they needed a language to do that, but they didn’t need to be programmers. That’s what SEQUEL was intended to do. I don’t think it was very successful at doing that. If you look back on it from a perspective of 30 years later, I think you don’t find a lot of architects and economists and city planners writing SEQUEL queries. SEQUEL queries are mostly written by experienced programmers who are trained and specialized in doing that, and a lot of them are generated by machines through some kind of interface that’s specialized to some domain.”
I am not suggesting that he, or any other academic at IBM Labs was stupid. Far from it. I am saying that as an academic, as evidenced, he has huge blind spots, eg. “industry”; “users”. And as is typical of academia, he is unaware that he has blind spots, and makes stupid declarations about them.
Overall, notice the difference, Chamberlain and a host of others (academic but impractical), against Codd (academic and practical). Codd was the bad guy because “he wasn’t a team player”. Thank God for that.
It is horrendous that he does not mention DSS queries in his discourse on that subject. In his /RM/, Codd provided both OLTP and DSS.
> Another first-hand historical perspective is provided by:
>
> https://www.mcjones.org/system_r/sql_reunion_95/sqlr95.html
They probably realised that it was going to be inspected and removed the filth, because it would not withstand scrutiny.
MCJones is the same idiot (nicely propagandised, well prepared) who interviewed Chamberlain, and who dutifully spouted the same propaganda. I don’t need to read it to know that it will be in the same vein.
> Quote from the "Prehistory" section:
>
> "[...] Nevertheless, what kicked off this work [on project Gamma-0 and,
> eventually, System R] was a key paper by Ted Codd - was it published in
> 1970 in CACM?
>
> - Yes.
>
> - A couple of us from the Systems Department had tried to read it -
> couldn't make heads nor tails out of it. [laughter] At least back
> then, it seemed like a very badly written paper: some industrial
> motivation, and then right into the math. [laughter]"
>
> Those are not academics talking.
Again, at IBM; Cincom; Honeywell; DEC; Cullinane; etc, we had [academically qualified] scientists, rather than academics. And we laughed at academics and their isolation from reality even then. Nothing has changed, it is much worse now. Once a thing is in the toilet bowl, the only way forward, is down down down.
> Regarding the rest of your historical comments, I do not have anything
> to add or object to. Except, perhaps, that the best DBMS in the world
> apparently survives in legacy mode now.
> > The point of this thread.
>
> Let's come to the point.
>
> >> On Wednesday, 24 March 2021 at 23:27:48 UTC+11, Nicola wrote:
> >> One reason is Chen's Entity-Relationship models are likely perceived
> >> as easier to grasp for novices than something like IDEF1X. Whether
> >> that is a correct perception, it can be debated. Another reason is
> >> probably just historical, due to the huge influence of Chen's seminal
> >> paper.
> >
> > It may have been seminal for the academics, let me assure you, it was
> > not seminal, or even great, for practitioners of the day. It does not
> > have a semantic base, whereas the /RM/ and IDEF1X does.
> >
> > For academics. I can grant, up to 1970, the paper was amazing. After
> > 1970, it was ageing, badly. By 1983, it was dead; obsolete; defunct;
> > superseded.
>
> Uh? Chen's ER paper is from 1976. That was a period of intense research
> on so called "semantic modelling" (also spurred in part by Codd's work).
> IDEF1X relies heavily upon Chen's shoulders.
> > For academics. I can grant, [if it were published] up to 1970, the paper was amazing.
> > [Since it was published] After 1970, it was ageing, badly.
> > By 1983 [when RDBMS platforms were freely available, several providers, AND IDEF1X was freely available], it was dead; obsolete; defunct; superseded.
The fact that Chen published it in 1976, and that academics performed “intense research” is precisely what I was talking about. Hysterical absurdity, because the /RM/ was published and being accepted, because several vendors were coming out with genuine RDBMS Platforms. Anyone with half a brain (eg. the scientists at IBM; Cincom; DEC; and Britton-Lee) laughed at it, because we were implementing the /RM/, and because the /RM? was far superior to the idiotic ERD.
Do you understand, and appreciate:
- we had ISAM prior to 1960’s & 1970’s DBMS
--- that means we understood KEYS
- separate to being based on FOPC, and therefore a mathematical model, the /RM/ is based on (a) the extant HDBMS, and (b) the extant NDBMS, as detailed in this thread ?
--- Hierarchical Model and its Relevance in the Relational Model
--- https://groups.google.com/g/comp.databases.theory/c/5212JwYtip4
--- HDBMS TECHNOLOGY IS HEAVILY KEY-BASED (of course, the storage is pointer-based)
--- The /RM/ takes its notion of KEYS from HDBMS, and the notion of Independent/Dependent from NDBMS
ERD is devoid of KEYS, of any understanding of KEYS, and since it was 1976, is [yet again] anti-Relational.
That the academics salivated over it, and did their mutual circle-jerk with “intense research” is exactly what I am talking about: it is hysterical absurdity, that can only be done in reinforced isolation from reality, from the /RM/; from the actual [R]DBMS platforms available an becoming available.
> IDEF1X relies heavily upon Chen's shoulders.
> I'd say that the paper has
> aged pretty well as we (let me simplify a bit) are still modelling in
> terms of entities, relationships and attributes.
> IMO, you are conflating the overall contributions of Chen (and others
> working on semantic models), which are important, influential and still
> relevant, with the specific details of the notation he invented, which
> is not ideal for Relational database design.
Now that you have added Semantic Models, a new response from me. Yes, I reject any and all academic endeavours re Data Modelling (Relational or other). Because the notion is hysterically absurd. For a scientist. But not for the academics, who are by definition [given your reports that they are/were working on “semantic models”] ignorant of the science. If you challenge this, please open a separate thread “Semantic Data Model”.
> > Ok, to rephrase the question, why was that pre-Relational method being
> > taught after 1983, and why is it still being taught after the advent
> > of the /RM/, after the /RM/was proved, and after the first genuine SQL
> > platforms came out (1983), and after IDEF1X came out (1983 to
> > practitioners, 1989 as a product, 1993 as an international standard) ?
> I already told you my point of view.
>
> Come out with a good database book, with a modern, sound, approach to
> data modeling based on IDEF1X, and tell your publisher to send a copy
> for evaluation to all academic database researchers. There is no
> inherent reason why it should not get adopted because it dismisses ERDs
> in favor of IDEF1X. Quite the opposite, actually. There is much need for
> a really good reference on database design for the XXI century. I can't
> recommend anything off the top of my head.
But you didn’t answer the question. Other than implying that nothing exists in the entire collection of academic multiverses. So my accusation holds: academics do suppress the /RM/ for theory, while promoting the anti-Relational RFS as “Relational” (as proved in several threads), and now, academics do suppress IDEF1X for Data Modelling, while promoting instead the anti-Relational ERD.
Why would the academics promote my book ??? It destroys the academic mindset, it asserts science and reality, it gives Methods that have existed for FORTY YEARS that academics deny, it gives Progressions to the /RM/ and the Methods, both of which the academics deny. No. At best, I will find a few honest academics who will provide a peer review.
But the book is not expected to be received by the pig poop eaters, it is for practitioners who respect a theoretical foundation. But thanks again.
Data Modelling is not “based on IDEF1X” (or ERD for that matter). Data Modelling is a science. Yes, I have a definition, and a Method, rendered in IDEF0. IDEF1X and IDEF0 are merely Standards, not methods. It gives low-level rules, which provide a simple but incomplete form of rules [defined in the /RM/]. Which, is not THE method or A method for modelling data. It pertains to the finished data model (any particular version), rather than the exercise that is required to produce it. For the purpose of having agreed symbols; completeness; etc. Such that two parties can discuss a Data Model that is /rendered in/ a well-known Standard.
Sure, a person CAN perform Data Modelling “based on IDEF1X”, it would be a class above and beyond any data modelling based on ERD, but it would still be inferior to actual data modelling.
You saw what I did with Movie Title, and to a lesser degree what I did with the three ERDs in OP’s request, you should have more than a whiff. The Vehicle-Accident problem was a good one as well. All three sets of problems exposed two things:
- use of ERD made sure the modeller was stuck; stumped; and stupefied
- classic data modelling [with /RM/ in mind] solved the problem in a straight-forward manner
The simple and explicit crime, that I am trying to get across is, academia has failed in its duty to teach the science, the fact that they teach ERD as “Relational Data Modelling” is a serious fraud, it is subversion of the science, and easily identified as such by those who are not ignorant.
> > after IDEF1X came out [in...] 1993 as an international standard)
>
> The FIPS standard has been withdrawn by NIST in 2008. But IDEF1X has
> also been an IEEE standard since 1998, and an ISO/IEC/IEEE standard
> since 2012.
> >> In a general setting, ERD is fine for teaching the basics of
> >> modeling
> >>
> >> ERDs can be useful as a gentle introduction to such topics.
> >>
> >> ERD can be introduced without any background knowledge in databases
> >
> > For introductory levels, show them IDEF1X/Entity level.
>
> Sure, I do that.
> >> Some may feel IDEF1X is "low-level" (too many details) compared to
> >> ERDs.
> >
> > False: display only the level of detail necessary for any given
> > exercise {Entity|Key|Attribute}.
>
> I agree. I did not mean to imply that I think otherwise; just that
> a cursory look by someone who knows ERDs but not IDEF1X may lead to such
> remarks.
The only remarks that count; that are worthy of discussion, are from people who know IDEF1X, or both IDEF1X and ERD.
> >> Or some may have criticisms along the lines of this analysis:
> >>
> >> https://www.essentialstrategies.com/publications/modeling/idef1x.htm
> >
> > God help me. Academics just love to sabotage themselves, and then cry
> > about the fact that they have been sabotaged.
>
> Just brought it as an instance of what a superficial look at the
> notation may lead to. I do not think that it is a good critical analysis
> in any way. And not by an academic either.
> >> My main complaint with IDEF1X is the treatment of generalization
> >> hierarchies, in particular the lack of a correct way to model a total
> >> inclusive specialization (A must be one of B1, ..., Bn and it may be
> >> any of B1, ..., Bn); the approach I have seen around (e.g., see
> >> Thomas Bruce's book or ERwin's manual) does not cover such case in
> >> a sound way. And the standard never mentions inclusive
> >> specializations, AFAICT.
> >
> > The problem there is, you do not understand Transactions
>
> Look, as much as it may be disappointing to you: I know. Very well.
> I know, in particular, how to deal with specialization hierarchies
> transactionally.
So what, if anything, is your remaining complaint re “generalization hierarchies”.
Re Transactions. Again, there is contradiction. If you understand ACID, you would reject MVCC. If you accept MVCC even as a potential (it has never reached actual), it proves you don’t understand ACID. Sorry, I am not going by what I think, I am going by your words, and what they actually mean.
> > There are several threads in this forum that simply do not progress to
> > closure due to the unwillingness of academics to engage, and to learn
> > about ACID Transactions.
>
> Yes, I did not feel those discussions were going towards fruitful
> directions.
> But, time permitting, I might eventually return on the MVCC
> vs 2PL matter.
> > This document may assist somewhat in explanation, noting that the full
> > and formal explanation lies in ACID Transactions, which has not
> > progressed in this forum, and indeed in academia, since 1970.
> >
> > https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/Subtype.pdf
> >
> > If there is anything in that doc that is not completely understood,
> > please ask.
> That's perfectly clear.
> > Here is a quick document that clarifies the issues you questioned,
> > that I have answered. To be complete, I have added an item re
> > Rolenames, because it is the third common “issue” that people raise:
> > https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/IDEF1X/Nicola%20IDEF1X%202.pdf
>
> A question: what is the use case for an association that does not imply
> a foreign key (rightmost symbol in your "improved IE")? Why would you
> require a parent entity to exist at the time of insertion, but not
> later?
____https://www.softwaregems.com.au/Documents/Student_Resolutions_2020/Giannis/RoomMonitor%20DM%20V0_51.pdf
The initial requirement is here, but he contacted me directly, and we progressed the Data Modelling exercise to completion off StackOverflow, in 25 iterations, which cannot be done at SO, which sucks dead bears. Read it only if interested, to get a feel, the detail in the *Q* is completely obsolete:
____https://stackoverflow.com/questions/61048436/modeling-optional-properties-of-an-entity-based-on-a-mutable-discriminator-in-a
The system distributes Room Monitors (Sensors: Temp; Humidity; DoorOpen; etc) to various hotels (Pensions) in Europe. The Monitors are produced; owned; and managed by the single company, but distributed widely to the locations. The Hotel owns the Network and actually receives the Sensor info (not the central corp). Over time, the specific hardware for a single Monitor gets removed from the Hotel and its Network; back to factory; then re-distributed to some other Hotel.
1. Guy needs the Monitor to be assigned properly in the first place (integrity with parent HotelNetwork), and it needs to be known as such for historical or audit purposes. But not maintained, eg. when it is removed. Therefore an FK Constraint will not work. The Constraint is implemented within an ACID Transaction:
____+Monitor CHECKS HotelNetwork[HotelChain, CountryCode, Town, Suburb) EXISTS.
2. Readings for a Room are stored permanently, for audit and legal reasons, which means Subordinate to Room, not Monitor. But of course, when a Reading is added, we need to ensure that it was produced by a particular Sensor, in a particular Monitor, that was located in that Room.
____Constraint ReadingMeasure_SensorIsValid_ck CHECK Sensor_Get_fn( Sensor.PK )
____which checks that the parent SensorSwitch EXISTS
which applies to the INSERT within the ACID Transaction only.
For understanding, I would say, there are three levels of maintaining history (Audit, etc), for designated tables, and it has some Temporal Database requirement.
1 At most, the requirement is to re-construct a row from the past. Separate table, and very tight KEYS plua DATETIME. Tight understanding of Temporal (not the Date and Darwen monstrosity marketed as “temporal”) is required, more for the coding than the data model.
3 At the least, it is what we have here in Room Monitor. Data is valid at INSERT, but not afterwards. It is not that a SELECT will fail, all SELECTS for those tables have to use OUTER JOIN, with ISNULL( “[Expired]” ) for the potentially non-existent parent. No understanding of Temporal is required.
2. In-between is the most common. When queried, the correct historic value must show up (but no need to re-construct). Reasonable understanding of Temporal (not the Date and Darwen monstrosity marketed as “temporal”) is required.
> Regarding subtyping, my critique is specifically on the notation given
> in ERwin Methods Guide at the crossing of "Complete" with "Inclusive
> Subtype".
You can’t use both IDEF1X and IEEE notation in any single model, they cannot be mixed. They are exclusive. You can’t go back-and-forth either.
> Let me denote the specialization symbol with `-O||`, and
> a categorization cluster with `P -O|| {C1, ..., Cn}`.
> By definition, P -O|| {C1, ..., Cn} means:
>
> "P must be exactly one of C1, ..., Cn".
>
> If an entity P has two ore more clusters, each should be interpreted as
> above. For instance:
>
> P -O|| {A,B}
> P -O|| {C,D}
>
> reads as:
>
> - P must be (exactly) one of A and B; and
> - P must be (exactly) one of C and D.
Since C is the classifier in your definition, I would prefer:
____P--€{A1,A2..An}
____P--C{X1,X2..Xm}
(“Inclusive” is incorrect, what we have is not the simple opposite of Exclusive. If it is not “Base is exactly one Subtype” which is Exclusive, then “Base is one or more Subtype” is simply Non-Exclusive. Nothing inclusive about that.)
> The notation I'm criticizing corresponds to:
>
> P -O|| {A}
> P -O|| {B}
>
> which, by the definition above, one should read as:
>
> - P must be A; and
> - P must be B.
>
> Instead, the assumed interpretation is:
>
> - P must be A or B; and
> - P may be both A and B,
If you mean the IDEF1X notation for Exclusive, no it does not exist, only {Complete|Incomplete} exists in IDEF1X notation. Don’t use the IDEF1X notation for Subtypes, we know it is inferior to the pre-existing IEEE notation, that is why we replaced the former with the latter.
Again, you can’t go back-and-forth: if you have interpreted the ERwin Methods Guide p 69 as allowing that, no, that is false. It is a diagram for comparison only. Use IDEF1X XOR IEEE Notation.
> which is inconsistent with the general definition.
>
> > I don’t know how you don’t strangle your colleagues.
>
> It's for fear of law enforcement :D
> Jokes apart, my database colleagues are among the best people in my
> department!
----
Since we are discussing IDEF1X, you might be interested in this Q&A:
____https://stackoverflow.com/q/4132044/484814
Cheers
Derek
Derek Ignatius Asirvadem
Apr 2, 2021, 5:36:28 AM4/2/21
to
Nicola
Thanks for yours.
> On Thursday, 1 April 2021 at 02:54:53 UTC+11, Nicola wrote:
Summary answer.
1. We had subtypes prior to RDBMS and IDEF1X. We had diagrams that were primitive compared to IDEF1X, but they worked perfectly well for the DBMS that we did have. We used IEEE notation.
2. So when IDEF1X was published, no one used {Complete|Incomplete}. [Technically, since one could always add a subtype in the future, every cluster might be Incomplete!].
3. The first (and only compliant) product ERwin provided both notations from day one.
4. ERwin Methods Guide was freely available, and even people who did not have ERwin used it as the proper definition for IDEF1X (rather than purchase).
I concur that the IDEF1X Notation for both Subtypes and Relations is inferior.
Therefore, forget about the thing that does not work, that is known to be inferior, that only academics use, and use the thing that does work, that practitioners have used since 1983, the IEEE notation. For both Subtypes and Relations. Every DM that I am aware of used IEEE (except yours, but you are earnestly trying to understand and use the Standard as is, rather than the modified Standard that practitioners use).
----
Nevertheless, I want to make sure that this particular item is resolved, because absolutely everything that occurs in reality can be modelled under FOPC; the /RM/; and IDEF1X. Refer to my Subtype doc for details and specific examples re what I state here.
First, absolutely and always, think of the Basetype+Subtype pair as a single LOGICAL row (I won’t use silly terms such as “tuple”). Rather than as parent-child (which it is in physical, SQL terms).
> On the other hand, it makes it impossible to express the situation
> above, i.e., that "P must be A, B, or both" (complete + inclusive).
If you give me an example (names, not A B C), I would be happy resolve it.
> ... each instance
If it is not optional, then it is as you say “absorbed” into the parent.
> I find it somewhat surprising that this aspect has not been clarified
> and addressed in the standard, which was last revised in 2019.
Yes. That is because the Date & Darwen freaks messed with it, and they pushed the totally obsolete IDEF1X notation over the IEEE notation. Same as you did before our discussion. Academia sucks dead sows. Even the notion of “Identity-Type” as an alternative to Keys, is hysterically backward, but hey, it now “validates” the OO/ORM approach.
Cheers
Derek
Thanks for yours.
> On Thursday, 1 April 2021 at 02:54:53 UTC+11, Nicola wrote:
Summary answer.
1. We had subtypes prior to RDBMS and IDEF1X. We had diagrams that were primitive compared to IDEF1X, but they worked perfectly well for the DBMS that we did have. We used IEEE notation.
2. So when IDEF1X was published, no one used {Complete|Incomplete}. [Technically, since one could always add a subtype in the future, every cluster might be Incomplete!].
3. The first (and only compliant) product ERwin provided both notations from day one.
4. ERwin Methods Guide was freely available, and even people who did not have ERwin used it as the proper definition for IDEF1X (rather than purchase).
I concur that the IDEF1X Notation for both Subtypes and Relations is inferior.
Therefore, forget about the thing that does not work, that is known to be inferior, that only academics use, and use the thing that does work, that practitioners have used since 1983, the IEEE notation. For both Subtypes and Relations. Every DM that I am aware of used IEEE (except yours, but you are earnestly trying to understand and use the Standard as is, rather than the modified Standard that practitioners use).
----
Nevertheless, I want to make sure that this particular item is resolved, because absolutely everything that occurs in reality can be modelled under FOPC; the /RM/; and IDEF1X. Refer to my Subtype doc for details and specific examples re what I state here.
First, absolutely and always, think of the Basetype+Subtype pair as a single LOGICAL row (I won’t use silly terms such as “tuple”). Rather than as parent-child (which it is in physical, SQL terms).
> On the other hand, it makes it impossible to express the situation
> above, i.e., that "P must be A, B, or both" (complete + inclusive).
> ... each instance
> of the parent must always be an instance of one of the children—even
> when the cluster has only one child.
A cluster that has only one child is a Normalisation error. It is not a Basetype-Subtype but an optional column and therefore an optional table.
> when the cluster has only one child.
If it is not optional, then it is as you say “absorbed” into the parent.
> I find it somewhat surprising that this aspect has not been clarified
> and addressed in the standard, which was last revised in 2019.
Cheers
Derek
Nicola
Apr 2, 2021, 3:39:42 PM4/2/21
to
>> Another first-hand historical perspective is provided by:
>>
>> https://www.mcjones.org/system_r/sql_reunion_95/sqlr95.html
>
> 404 - Not Found.
My fault (wrong case):
>>
>> https://www.mcjones.org/system_r/sql_reunion_95/sqlr95.html
>
> 404 - Not Found.
https://www.mcjones.org/System_R/SQL_Reunion_95/sqlr95.html
I'll come back to the rest at a later time.
Nicola
Derek Ignatius Asirvadem
Apr 2, 2021, 5:45:36 PM4/2/21
to
> On Saturday, 3 April 2021 at 06:39:42 UTC+11, Nicola wrote:
>
> >> https://www.mcjones.org/system_r/sql_reunion_95/sqlr95.html
> >
> > 404 - Not Found.
>
> My fault (wrong case):
>
> https://www.mcjones.org/System_R/SQL_Reunion_95/sqlr95.html
>
> >> https://www.mcjones.org/system_r/sql_reunion_95/sqlr95.html
> >
> > 404 - Not Found.
>
> My fault (wrong case):
>
> https://www.mcjones.org/System_R/SQL_Reunion_95/sqlr95.html
> On Wednesday, 31 March 2021 at 22:58:03 UTC+11, Nicola wrote:
>
>
> Another first-hand historical perspective is provided by:
>
>
> Quote from the "Prehistory" section:
>
> "[...] Nevertheless, what kicked off this work [on project Gamma-0 and,
> eventually, System R] was a key paper by Ted Codd - was it published in
> 1970 in CACM?
>
> - Yes.
>
> - A couple of us from the Systems Department had tried to read it -
> couldn't make heads nor tails out of it. [laughter] At least back
> then, it seemed like a very badly written paper: some industrial
> motivation, and then right into the math. [laughter]"
>
> Those are not academics talking.
No, no, no. Those ARE academics talking. Rather famous ones.
>
> "[...] Nevertheless, what kicked off this work [on project Gamma-0 and,
> eventually, System R] was a key paper by Ted Codd - was it published in
> 1970 in CACM?
>
> - Yes.
>
> - A couple of us from the Systems Department had tried to read it -
> couldn't make heads nor tails out of it. [laughter] At least back
> then, it seemed like a very badly written paper: some industrial
> motivation, and then right into the math. [laughter]"
>
> Those are not academics talking.
Those articles reinforce my declarations, not yours ! Each has a great understanding of a very narrow field (of research) and each remained clueless about the other fields, even related fields. And far removed from the “industry” and “users”.
At Cincom, let me assure you, we were very aware of that, which is typical in large corps such as IBM, and sought to avoid that sort of problem by having a small team made up of specialists, in one location. TOTAL for DEC/PDP was written by a team of four. I came to Australia to write the “next generation” TOTAL for DEC/VAX with full ACID, our team was five. We did it in 18 months, it was named ULTRA-DBMS. (Later, Cincom missed the Relational bus, and got side-lined.)
Cheers
Derek
Nicola
Apr 3, 2021, 6:27:43 PM4/3/21
to
say that it's not hard to find some that are supportive of the RM. Any
that have understood it? Hard to tell... Progressed it? Let's see:
I have a few papers on semantic modeling, which do not qualify; several
papers on data dependencies, normal forms and other purely theoretical
amenities such as closed/open worlds or the computational complexity of
joins, which do not qualify because they are not practical; and a few
papers on physical and systems design (such as the papers on System R),
which might qualify, but they are from the IBM Systems Journal or IBM
Research Labs.
I would consider Grant's 1977's critique on "Null values in a Relational
Data Base" a progress, as it pointed out the inconsistency of Codd's
preliminary proposal for the treatment of nulls. But you'd likely say
that Codd's treatment of nulls was a regression, so Grant would at most
move things back to the start.
I'd rather like to know why you think that, after 1981, papers with
the qualities that you cannot ascribe to '70s papers would exist, and
ask you to name one such paper, if you know any.
Anyway, I appreciate your own recount of those times, albeit somewhat
critical of the sources I have cited. I do not feel qualified to comment
on that any further, though.
>> IDEF1X relies heavily upon Chen's shoulders.
>
> Nonsense. Give one example, one article. Rob Brown already had
> a diagrammatic modelling method (as distinct from the various diagrams
> that data modellers created) that was widely accepted ... after
> consultation with Codd, he produced what became known as IDEF1X.
approach to IDEF information modeling] stemmed from the early work of
Dr. E. F. Codd on relational theory and Dr. P. P. S. Chen on the
entity-relationship model."
And two paragraphs later: "Application within industry had led to the
development in 1982 of a Logical Database Design Technique (LDDT) by R.
G. Brown of the Database Design Group. The technique was based on the
relational model of Dr. E. F. Codd, the entity-relationship model of Dr.
P. P. S. Chen, and the generalization concepts of J. M. Smith and D. C.
P. Smith."
> Ok, fine. But your words contradict that, you can’t say that and also
> say “My main complaint with IDEF1X is the treatment of generalization
> hierarchies”.
>
> So what, if anything, is your remaining complaint re “generalization
> hierarchies”.
a specific critique on the specific notation used by IDEF1X. That has
nothing to do with understanding transactions, or whatever. It is the
lack of a way (if you follow the standard) or a cumbersome way (if you
follow Bruce/ERwin) to represent what in your document is called
"Non-Exclusive Subtyping", using the "complete categorization" symbol.
>> Regarding subtyping, my critique is specifically on the notation given
>> in ERwin Methods Guide at the crossing of "Complete" with "Inclusive
>> Subtype".
>
> What “crossing” ???
at the intersection of the "Complete" column and "Inclusive Subtype"
row.
> You can’t use both IDEF1X and IEEE notation in any single model, they
> cannot be mixed. They are exclusive. You can’t go back-and-forth
> either.
> Sorry. I don’t understand what you are trying to say.
a nutshell, that non-exclusive subtyping is not straightforward as it
should be (as it is in IE).
>> > Here is a quick document that clarifies the issues you questioned,
>> > that I have answered. To be complete, I have added an item re
>> > Rolenames, because it is the third common “issue” that people raise:
>> > https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/IDEF1X/Nicola%20IDEF1X%202.pdf
>>
>> A question: what is the use case for an association that does not imply
>> a foreign key (rightmost symbol in your "improved IE")? Why would you
>> require a parent entity to exist at the time of insertion, but not
>> later?
>
> It is actually a very common requirement. Refer this Data Model:
> ____https://www.softwaregems.com.au/Documents/Student_Resolutions_2020/Giannis/RoomMonitor%20DM%20V0_51.pdf
model, but one thing that I have noticed is that there is no independent
Sensor entity. That strikes as something unusual. It's ok to associate
a Reading with a Room for auditing, but with an independent Sensor you
should also be able to maintain referential integrity even when
a sensor's location is changed as the MAC is immutable. But, again,
I need to re-read your requirements.
> Since we are discussing IDEF1X, you might be interested in this Q&A:
> ____https://stackoverflow.com/q/4132044/484814
Nicola
Nicola
Apr 3, 2021, 6:40:19 PM4/3/21
to
On 2021-04-02, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
>> On the other hand, it makes it impossible to express the situation
>> above, i.e., that "P must be A, B, or both" (complete + inclusive).
>
> If you give me an example (names, not A B C), I would be happy resolve it.
The example from your Subtype.pdf document is fine: a PastTime (P) must
>> On the other hand, it makes it impossible to express the situation
>> above, i.e., that "P must be A, B, or both" (complete + inclusive).
>
> If you give me an example (names, not A B C), I would be happy resolve it.
be a PasttimeHobby (A) or a PasttimeSport (B), or both.
Using standard IDEF1X symbology, and following the text of the standard
(FIPS 184 or ISO 31320:2, they don't differ on this), you cannot express
the assertion above, only approximate it, AFAICS.
In Bruce's book (and in ERwin) you'd use two clusters, each with one
child, but that requires an interpretation that does not find
an equivalent in the standards. At least, to the best of my
understanding.
Nicola
Derek Ignatius Asirvadem
Apr 4, 2021, 5:06:00 AM4/4/21
to
Nicola
> On Sunday, 4 April 2021 at 08:40:19 UTC+10, Nicola wrote:
Thanks for yours.
> > On 2021-04-02, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
>
> >> On the other hand, it makes it impossible to express the situation
> >> above, i.e., that "P must be A, B, or both" (complete + inclusive).
> >
> > If you give me an example (names, not A B C), I would be happy resolve it.
>
> The example from your Subtype.pdf document is fine: a Pastime (P) must
> be a PastimeHobby (A) or a PastimeSport (B), or both.
>
> Using standard IDEF1X symbology [excluding IEEE notation], and following the text of the standard
It appears this is boiling down to:
- you do not accept the IEEE notation for Relational Data Modelling
- you are in denial that IEEE notation exists
--- in fact it existed before you or I were born
- you do not accept correction of an incomplete Standard
--- even if the correction existed prior
--- or put otherwise, IDEF1X Subtype notation is a regression, and thus invalid
- therefore for you there is only the IDEF1X notation
Yes, the IDEF1X Subtype notation is broken. Yes, that is just one of the many things that one should be able to (a) model, and (b) model compliant with the /RM/, that cannot be rendered in IDEF1X/IDEF1X Subtype Notation.
The same problem exists with IDEF1X Cardinality notation.
So freaking what.
If you are a hardened academic, that is all there is in the modelling world, and you can’t accept a Data Modelling job of any kind, because the Standard is broken and you can’t model things that exist in nature correctly.
That is why academics cannot work in the industry that they theorise about. They never the leave university mindset.
Enter the practical man. Drum roll and dancing girls.
The Standard is broken ? No worries, I will fix it.
Please be advised, as detailed in my previous post, we had System Diagrams; File Definition Diagrams long before the /RM/, long before IDEF1X, and we used IEEE notation for Subtypes and Cardinality. Such that when IDEF1X came along, any one who was actually practicing (had diagrams and knowledge of IEEE notation) knew immediately that the IDEF1X Subtype & Cardinality notation was broken; that the well-known and pre-existing IEEE notation was NOT broken, and we used it instead. We viewed the IDEF1X Notation as an esoteric joke. As evidenced in the first product, ERwin, from the very first version (otherwise there is no purpose in it giving both notations).
The problem is, academics like you and the guy who wrote that superficial critique, are (a) hanging onto to the original IDE1X Standard definition, (b) refusing to accept that the IEEE notation existed prior, (c) refusing to accept that the IDEF1X notation is broken, and inferior to the IEEE, and (d) thus maintaining an unresolved space [the Non-Excluded Middle], whereas practitioners have a completely resolved space [the Middle, Excluded].
Whereas science was founded on the Four Laws of Thought, and progressed within those tight boundaries for 2,400 years, and thus such unresolved spaces were absolutely forbidden [Law of the Excluded Middle], with the advent of Modern “science”, the Four Laws; Causality; the Hierarchy, are all denied and suppressed, precisely in order to erect fantasy as “reality”. If you play the role of a medieval monk for a few moments, intending to obtain the reward of heaven (eg. no lies; no misrepresentations; no omissions):
- the Middle [d] is not acceptable, it must be Excluded, it would motivate you to obtain resolution
- the denial of reality in [b][c] has to be removed, reality must be admitted, otherwise you would be insane
- you will arrive at the determination that the published Standard is incomplete; unusable as is
- petition ISO to correct the published Standard, to use the pre-existing and not-broken IEEE notation
- use the pre-existing and not-broken IEEE notation until such time as ISO publishes the amendment
Modern academics alone, enjoy the luxury of wringing their hands because a Standard is incomplete, and can only do so in their ivory towers; their protected safe spaces, and in denial of reality. They get to argue; to debate; to discuss, things that are closed; resolved; unarguable to practitioners, and to do so for decades after resolution has been achieved in reality. These two items were closed in the 1980’s in manual diagrams, closed with the first version of ERwin in terms of Data Modelling Tools. But here we are today, forty years after the fact, still arguing about it, as if it has NOT been resolved.
> In Bruce's book (and in ERwin) you'd use two clusters, each with one
> child, but that requires an interpretation that does not find
> an equivalent in the standards. At least, to the best of my
> understanding.
Your understanding is correct.
That interpretation is false.
Bruce is an idiot (per reasons explained above, for noticing the problem and NOT resolving it).
Do not follow an idiot.
Further, the “two clusters with one child each” is a hideous, gross, Normalisation error.
Thus the model is invalid; incorrect, fix it.
The Standard (any Standard) is not a Method. At best, a Standard is a notation, that (a) prevents illegal arrangements by virtue of NOT having a notation for it, and (b) encourages correct definition by having mandatory notation. When a Standard has an error, yet another Standard, usually a higher Standard or a pre-existing one (ie. not your or my private fabrication) is used to correct it or to fill the gap. Issue closed, resolved for practitioners.
As a counter-point, think about UML. Even though it is heavily marketed as a “Standard”, it is not a Standard by any means, because it is (a) grossly incomplete: each modeller adds his own 5 to 10 notations to cover the missing bits because he has designed objects or interaction for which no UML notation exists. When a few hundred do that private dance and publish their models, those 400 or 500 notations become de facto UML, like it or not. Yes, you and I as purists would reject all that. But whereas I would model it in a genuine modelling notation, and thus resolve the issue, you would stand and argue that UML is broken, that the added notation is invalid.
Further, the main fault with UML, its self-destroying fault, is [b] the lack of integration in all areas, The many different diagrams simply do not come together as an integrated System Definition.
The most important missing capability is [c] that which is required for genuine Analysis, and genuine Design: composition/decomposition. Which of course the pre-existing Standards had, and still have. So the practical person uses IDEF1X Corrected for the Data Model; SSADM or IDEF0 for the Process Model; and dismisses UML. Which remains an endless study for the OO/ORM developers obsess about, to draw their precious dingalings in, and to keep replacing. Much like the insane study the toilet bowl. Eg. I could give my Transaction Definitions as UML Methods, which is limited, or as IDEF1X Extensions that include SQL pseudo-code (as in my Subtype doc). They are defined fully in my GUI Definition [interaction] docs.
The converse is, as you are fully aware, OO/ORM developers not-analysing the requirement, then not-designing the process, then obsessing about their Objects that “need persistence”, spending their entire life drawing and re-drawing said Objects, and last but not least, writing records to a ”persistent data store” that gets “refactored” every month or quarter. Viewing both the world, and the requirement through the myopic lens of oo/orm. Maslow’s Golden Hammer syndrome (If all you have in your toolbag is a hammer, every problem looks like a nail).
You have determined that UML is stupid for Data Modelling of any kind, great. You have accepted IDEF1X for Relational Data Modelling, great. You are at the brink of jettisoning ERD, but your blind loyalty to academia prevents you from calling it what it is, and your non-resolution of a simple problem in the IDEF1X Standard prevents you from embracing it.
Thus, I would prevail upon you, given your teaching position, the influence you have on bright eager young minds, to teach only resolutions, not ambiguity or non-resolution. As the industry has done, as they have built thousands of Relational Databases, for forty years.
I have updated this doc to reflect the above. In case there remains anything worth discussing on this issue, I have added a page with the Subtypes, and given the Predicates (which are an excellent method of verifying the model, a feedback loop).
____https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/IDEF1X/Nicola%20IDEF1X%202.pdf
Cheers
Derek
> On Sunday, 4 April 2021 at 08:40:19 UTC+10, Nicola wrote:
Thanks for yours.
> > On 2021-04-02, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
>
> >> On the other hand, it makes it impossible to express the situation
> >> above, i.e., that "P must be A, B, or both" (complete + inclusive).
> >
> > If you give me an example (names, not A B C), I would be happy resolve it.
>
> be a PastimeHobby (A) or a PastimeSport (B), or both.
>
> Using standard IDEF1X symbology [excluding IEEE notation], and following the text of the standard
> (FIPS 184 or ISO 31320:2, they don't differ on this), you cannot express
> the assertion above, only approximate it, AFAICS.
The "or both" is not a Predicate, it is a collection of instances.
> the assertion above, only approximate it, AFAICS.
It appears this is boiling down to:
- you do not accept the IEEE notation for Relational Data Modelling
- you are in denial that IEEE notation exists
--- in fact it existed before you or I were born
- you do not accept correction of an incomplete Standard
--- even if the correction existed prior
--- or put otherwise, IDEF1X Subtype notation is a regression, and thus invalid
- therefore for you there is only the IDEF1X notation
Yes, the IDEF1X Subtype notation is broken. Yes, that is just one of the many things that one should be able to (a) model, and (b) model compliant with the /RM/, that cannot be rendered in IDEF1X/IDEF1X Subtype Notation.
The same problem exists with IDEF1X Cardinality notation.
So freaking what.
If you are a hardened academic, that is all there is in the modelling world, and you can’t accept a Data Modelling job of any kind, because the Standard is broken and you can’t model things that exist in nature correctly.
That is why academics cannot work in the industry that they theorise about. They never the leave university mindset.
Enter the practical man. Drum roll and dancing girls.
The Standard is broken ? No worries, I will fix it.
Please be advised, as detailed in my previous post, we had System Diagrams; File Definition Diagrams long before the /RM/, long before IDEF1X, and we used IEEE notation for Subtypes and Cardinality. Such that when IDEF1X came along, any one who was actually practicing (had diagrams and knowledge of IEEE notation) knew immediately that the IDEF1X Subtype & Cardinality notation was broken; that the well-known and pre-existing IEEE notation was NOT broken, and we used it instead. We viewed the IDEF1X Notation as an esoteric joke. As evidenced in the first product, ERwin, from the very first version (otherwise there is no purpose in it giving both notations).
The problem is, academics like you and the guy who wrote that superficial critique, are (a) hanging onto to the original IDE1X Standard definition, (b) refusing to accept that the IEEE notation existed prior, (c) refusing to accept that the IDEF1X notation is broken, and inferior to the IEEE, and (d) thus maintaining an unresolved space [the Non-Excluded Middle], whereas practitioners have a completely resolved space [the Middle, Excluded].
Whereas science was founded on the Four Laws of Thought, and progressed within those tight boundaries for 2,400 years, and thus such unresolved spaces were absolutely forbidden [Law of the Excluded Middle], with the advent of Modern “science”, the Four Laws; Causality; the Hierarchy, are all denied and suppressed, precisely in order to erect fantasy as “reality”. If you play the role of a medieval monk for a few moments, intending to obtain the reward of heaven (eg. no lies; no misrepresentations; no omissions):
- the Middle [d] is not acceptable, it must be Excluded, it would motivate you to obtain resolution
- the denial of reality in [b][c] has to be removed, reality must be admitted, otherwise you would be insane
- you will arrive at the determination that the published Standard is incomplete; unusable as is
- petition ISO to correct the published Standard, to use the pre-existing and not-broken IEEE notation
- use the pre-existing and not-broken IEEE notation until such time as ISO publishes the amendment
Modern academics alone, enjoy the luxury of wringing their hands because a Standard is incomplete, and can only do so in their ivory towers; their protected safe spaces, and in denial of reality. They get to argue; to debate; to discuss, things that are closed; resolved; unarguable to practitioners, and to do so for decades after resolution has been achieved in reality. These two items were closed in the 1980’s in manual diagrams, closed with the first version of ERwin in terms of Data Modelling Tools. But here we are today, forty years after the fact, still arguing about it, as if it has NOT been resolved.
> In Bruce's book (and in ERwin) you'd use two clusters, each with one
> child, but that requires an interpretation that does not find
> an equivalent in the standards. At least, to the best of my
> understanding.
That interpretation is false.
Bruce is an idiot (per reasons explained above, for noticing the problem and NOT resolving it).
Do not follow an idiot.
Further, the “two clusters with one child each” is a hideous, gross, Normalisation error.
Thus the model is invalid; incorrect, fix it.
The Standard (any Standard) is not a Method. At best, a Standard is a notation, that (a) prevents illegal arrangements by virtue of NOT having a notation for it, and (b) encourages correct definition by having mandatory notation. When a Standard has an error, yet another Standard, usually a higher Standard or a pre-existing one (ie. not your or my private fabrication) is used to correct it or to fill the gap. Issue closed, resolved for practitioners.
As a counter-point, think about UML. Even though it is heavily marketed as a “Standard”, it is not a Standard by any means, because it is (a) grossly incomplete: each modeller adds his own 5 to 10 notations to cover the missing bits because he has designed objects or interaction for which no UML notation exists. When a few hundred do that private dance and publish their models, those 400 or 500 notations become de facto UML, like it or not. Yes, you and I as purists would reject all that. But whereas I would model it in a genuine modelling notation, and thus resolve the issue, you would stand and argue that UML is broken, that the added notation is invalid.
Further, the main fault with UML, its self-destroying fault, is [b] the lack of integration in all areas, The many different diagrams simply do not come together as an integrated System Definition.
The most important missing capability is [c] that which is required for genuine Analysis, and genuine Design: composition/decomposition. Which of course the pre-existing Standards had, and still have. So the practical person uses IDEF1X Corrected for the Data Model; SSADM or IDEF0 for the Process Model; and dismisses UML. Which remains an endless study for the OO/ORM developers obsess about, to draw their precious dingalings in, and to keep replacing. Much like the insane study the toilet bowl. Eg. I could give my Transaction Definitions as UML Methods, which is limited, or as IDEF1X Extensions that include SQL pseudo-code (as in my Subtype doc). They are defined fully in my GUI Definition [interaction] docs.
The converse is, as you are fully aware, OO/ORM developers not-analysing the requirement, then not-designing the process, then obsessing about their Objects that “need persistence”, spending their entire life drawing and re-drawing said Objects, and last but not least, writing records to a ”persistent data store” that gets “refactored” every month or quarter. Viewing both the world, and the requirement through the myopic lens of oo/orm. Maslow’s Golden Hammer syndrome (If all you have in your toolbag is a hammer, every problem looks like a nail).
You have determined that UML is stupid for Data Modelling of any kind, great. You have accepted IDEF1X for Relational Data Modelling, great. You are at the brink of jettisoning ERD, but your blind loyalty to academia prevents you from calling it what it is, and your non-resolution of a simple problem in the IDEF1X Standard prevents you from embracing it.
Thus, I would prevail upon you, given your teaching position, the influence you have on bright eager young minds, to teach only resolutions, not ambiguity or non-resolution. As the industry has done, as they have built thousands of Relational Databases, for forty years.
I have updated this doc to reflect the above. In case there remains anything worth discussing on this issue, I have added a page with the Subtypes, and given the Predicates (which are an excellent method of verifying the model, a feedback loop).
____https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/IDEF1X/Nicola%20IDEF1X%202.pdf
Cheers
Derek
Nicola
Apr 4, 2021, 6:50:43 AM4/4/21
to
On 2021-04-04, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
> The "or both" is not a Predicate, it is a collection of instances.
>
> It appears this is boiling down to:
> - you do not accept the IEEE notation for Relational Data Modelling
> - you are in denial that IEEE notation exists
> --- in fact it existed before you or I were born
> - you do not accept correction of an incomplete Standard
> --- even if the correction existed prior
> --- or put otherwise, IDEF1X Subtype notation is a regression, and thus invalid
> - therefore for you there is only the IDEF1X notation
I am not in denial of anything; I can read your data models without
> The "or both" is not a Predicate, it is a collection of instances.
>
> It appears this is boiling down to:
> - you do not accept the IEEE notation for Relational Data Modelling
> - you are in denial that IEEE notation exists
> --- in fact it existed before you or I were born
> - you do not accept correction of an incomplete Standard
> --- even if the correction existed prior
> --- or put otherwise, IDEF1X Subtype notation is a regression, and thus invalid
> - therefore for you there is only the IDEF1X notation
issues, for instance, and I am not asking you or anyone to use a "more
standard" notation for the sake of it.
That said, yes, I prefer to stick to IDEF1X notation; but I have no
problem diverging from the standard if I need to.
> Yes, the IDEF1X Subtype notation is broken. Yes, that is just one of
> the many things that one should be able to (a) model, and (b) model
> compliant with the /RM/, that cannot be rendered in IDEF1X/IDEF1X
> Subtype Notation.
1. are the subtype mutually exclusive or not?
2. is every instance of the parent entity an instance of some child
entity ("subtyping" in your terminology), or not?
So, four combinations. Following the terminology of your document:
| Exclusive?
Subtyping? | Yes No
-----------|----------------------------------------------
Yes | Exclusive Subtyping Non-exclusive subtyping
No | ? Optional attr.[Group], Not Subtype
In IDEF1X notation, "non-exclusive subtyping" is the problematic one.
What is marked with ? corresponds to an "incomplete categorization" in
IDEF1X terminology. Using IE notation, how would you model an
"incomplete categorization"? There is no example in your document.
Concrete example:
- An Employee may be a Consultant;
- An Employee may be a Full-Time Employee;
- An Employee may be neither a Consultant nor a Full-Time Employee;
- An Employee cannot be both a Consultant and a Full-Time Employee.
Note, I am not asking you how you would *implement* this situation; only
how a data model would look like.
> The same problem exists with IDEF1X Cardinality notation.
>> In Bruce's book (and in ERwin) you'd use two clusters, each with one
>> child, but that requires an interpretation that does not find
>> an equivalent in the standards. At least, to the best of my
>> understanding.
>
> Your understanding is correct.
> That interpretation is false.
> Bruce is an idiot (per reasons explained above, for noticing the problem and NOT resolving it).
> Do not follow an idiot.
> Further, the “two clusters with one child each” is a hideous, gross, Normalisation error.
> Thus the model is invalid; incorrect, fix it.
(with IE notation). I do not have an ERwin license, so I can't try it
myself.
> As a counter-point, think about UML. Even though it is heavily
> marketed as a “Standard”, it is not a Standard by any means, because
> it is (a) grossly incomplete: each modeller adds his own 5 to 10
> notations to cover the missing bits because he has designed objects or
> interaction for which no UML notation exists. When a few hundred do
> that private dance and publish their models, those 400 or 500
> notations become de facto UML, like it or not. Yes, you and I as
> purists would reject all that. But whereas I would model it in
> a genuine modelling notation, and thus resolve the issue, you would
> stand and argue that UML is broken, that the added notation is
> invalid.
glitch that can be easily fixed; "fixing UML" is something on another
level entirely: I'd not dare ask anything like that.
> Further, the main fault with UML, its self-destroying fault, is [b]
> the lack of integration in all areas, The many different diagrams
> simply do not come together as an integrated System Definition.
explanation purposes, I think I have little or nothing to reply to or
disagree with you about UML or ORM (by which I assume you mean
Object-Relational Mapping).
> I have updated this doc to reflect the above. In case there remains
> anything worth discussing on this issue, I have added a page with the
> Subtypes, and given the Predicates (which are an excellent method of
> verifying the model, a feedback loop).
>
> ____ https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/IDEF1X/Nicola%20IDEF1X%202.pdf
correctly. My only remaining question is the one above.
RE the added page, looking at the transactions, I think PersonPastime's
subtypes are not exclusive (the diagram has an X that should not be
there).
Happy Easter,
Nicola
Message has been deleted
Derek Ignatius Asirvadem
Apr 4, 2021, 10:32:52 PM4/4/21
to
1. Normal Forms
They are all a complete joke. Half are redundant, the other half are ignorant of the /RM/, and so they “invent” something that is already in the RM (I have written a couple of responses).
That does not count the various changes the Date; Darwen; Fagin gang keep making to the NFs, which are pig poop. None of the idiots have articulated the Normal Forms that are in the /RM/. Maybe if we wait another fifty years.
2. Complexity of Joins
Nothing compared to the papers already written by Sybase Engineers. We have the most intelligent, third generation, Parser and Query Optimiser. You will not believe the level of analysis and reduction of complexity, then determination of access paths based on Statistics. I have read some of those academic papers and cacked myself (I still have difficulty believing that academics are so isolated from the real world, that they read only what their colleagues write, and thus maintain a cocoon).
(In the last 20 years, we have stopped publishing papers, because we do not want to give the freeware crowd the Method, by publishing our Trade Secrets. I have yanked some of my Sybase doco, and there are a few more than I will yank soon. The Sybase Technical Forum has moved out of public view, into a closed SAP-Only stable, so the public don’t get the details even via technical chats.)
3. “Semantic Models”
It appears you do not get it, and this one is going to be hard nut to cut through, because the academic mindset is set and reinforced by many papers. Look, as an academic; as a mathematician; as a logician, why is it that every single diagram you draw is NOT semantic ???
You understand FOPC. FOPC is semantic. That is the foundation of the /RM/. Now whatever you draw, make sure it is a legal FOPC Predicate, independent, or dependent on other Predicates. Chose your notation, or write one yourself.
The problem is, the academics have concocted this notion of “semantic model”, and written 57 papers about it, with lots of self-serving citations, so they think it is “real”. It isn’t, it is a collectively assented fiction. Much like the problem of “universals”, the freaks deny that Universals exist; what causes their existence; what sustains their existence, and then they concoct a slew of possible ways that “universals” could exist, and they spend the rest of their lives debating the possibilities, never resolving anything. It was resolved in 350BC by Aristotle.
Same here. Academics deny the simple fundamental atomic existence of SEMANTICS, that it is in FOPC already, that conjuring up another “semantics” or forty three is a stupid but income-producing shell game that they can play with each other. Further by denying THE semantic root (FOPC), and then fabricating a “semantic” layer somewhere else, whatever semantics their “semantic” layer has is a mere fraction, and due to incorrect architectural deployment, both massive and hard to design and implement. In any case, it will be grossly incomplete, with no sense of integration.
I will give you a parallel example, which we have discussed to some degree (unfortunately not to closure), the Movie Title Thread. The Relational Data Modeller gave you the Relational Data Model for Movie Title, it has all the FOPC Predicates thought out and modelled. What you engaged in over 14 iterations. Strict FOPC and /RM/, therefore any SQL written against it will be straight-forward (no complexity, /Complexity of Joins/ papers may be irrelevant, yet another academic shell game). In fact any report requirement can be serviced via a single SELECT. But the starting position; the academic position was, “it can’t be done in the /RM/, we need an /ontology/ and a /description logics/“. Don’t mention a fat middleware server, and masses of s/w. Don’t mention that when the perception of data changes, all three layers must be changed, and the “database” has to be “refactored”..
When we insist of FOPC; the /RM/; and genuine Relational Modelling, the /hoontology/ and /descwiption non-logics/ are eliminated, they stand as vacuums. There is no need for an /onotology/ [what a filthy label] to tell the world what the structure of the data is, because the story is told in super integrated form, by the Data Model itself, because the DM is Semantic, because it engages FOPC Predicates.
Another parallel example. FOPC -> Relational Model -> SQL. So real SQL is strictly based on FOPC. But the Torrid Manifesto gulag, the Date; Darwen; Fagin religion totally suppress FOPC and the /RM/, and then assert that SQL is broken. Not only that, they will produce a “data sublanguage” that conforms to the “relational model” which is in fact the anti-Relational RM/T. Thirty years and still nothing.
Meanwhile, back at the farm, guys like me who are unsoiled with their pig poop find absolutely nothing wrong with SQL, there is nothing we cannot do in SQL over a genuine Relational Database.
Do you see what I am trying to say ? Academics are clueless about what SEMANTICS is, that it is already in FOPC, they are therefore clueless about FOPC, what it really and truly is, how the /RM/ isfounded on FOPC. But they write volumes of papers about what “semantics” could be.
> I would consider Grant's 1977's critique on "Null values in a Relational
> Data Base" a progress, as it pointed out the inconsistency of Codd's
> preliminary proposal for the treatment of nulls. But you'd likely say
> that Codd's treatment of nulls was a regression, so Grant would at most
> move things back to the start.
Three-valued logic is plain stupid. I have never used it. I have no Nulls in any database, not even from 1983.
It can be understood only in the context that Codd was bending over backwards to get some engagement from the academic community. For ten years. Same as RM/T. A massive regression to pre-1960’s filing systems. Not even 1970’s.
> I'd rather like to know why you think that, after 1981, papers with
> the qualities that you cannot ascribe to '70s papers would exist, and
> ask you to name one such paper, if you know any.
When I bracketed 1970 to 1980, it was about that critical period when Codd had no support, and was running around the country marketing the /RM/, and unfortunately bending over backwards to demonstrate how even Record-ID-based ISAM and HDBMS could benefit from the /RM/, which was RM/T, which was then fraudulently used by the pig poop Gulag to market as “the Relational Model”.
After 1980, when the /RM/ was accepted, and notably first by RDBMS providers, who actually produced RDBMSs, not by academics, the papers were commercial, internal.
Sure, there were total lunatics such as Stonebraker (great salesman, great cult leader, disgusting academic) that tried to implement parts of the /RM/, while being clueless about databases; contention; concurrency; and transactions (all of which were well and truly implemented and implemented correctly, for 30 years at that time). So in perfect academic fashion, he ignored all that reality, that the database is a shared resource that is in constant change, and concocted the schizophrenic [reality-contradicting] notion of the dumb user that has view of a fixed unchanging database to engage with.
It was called MVCC, but the label is false, there is no such thing in reality as “multiple versions” of a single database, that is the first layer of schizophrenia, like everyone going back to their private tiled cells in the asylum, each holding one row from the single ever-changing story in the library. And no problem, they take their time modifying “their” row, and they will submit in in the morning, after ablutions and a good breakfast. It breaks the first principle of a database: a single version of the truth, shared by all who access it. Maybe have a soy latte, and watch the little girls walk by.
The “concurrency control” part of the label is utterly Stonebraker-style false. It does absolutely nothing re concurrency (each user is left to his own devices, which is the pretence that it is a single-user database mounted between his big toes, to do as he pleases), and certainly nothing to control concurrency. Concurrency is denied, and resolution of the fantasised changes is deferred to the COMMIT.
The best that can be said for it, in honest technical terms that academics have no access to, is that it attempts to serialise the multiple [chronological] events that each user schizophrenically thinks he did (which he actually did not do), in denial of the other users’ events (some of which actually did happen), after the fact [of the fantasy that he did], after all the horses have bolted, when he sends COMMIT. And it fails miserably.
The mantra “readers never block writers, and writers never block readers" is exactly that, a mantra. Same as when a fat and ugly old man goes up to the mirror and repeats his mantra “I am a chick magnet”. It is very satisfying, thinking about events, that have never happened in the past, that will definitely positively absolutely happen in the future. Coz he said so. Check the resolute look in the mirror, tis “real”.
The simple fact is, MVCC does not work without locking, it is a gross lie that MVCC is a substitute for locking. PissGross has four LEVELS of locking (with minor levels within each), on top of MVCC, and it still can’t resolve the millions of multiple versions of records that it has to maintain.
____ https://stackoverflow.com/search?q=%5Bpostgresql%5D+lock+problem
Meanwhile, back at the commercial farm, since 1981 [for RDBMS], with Ordinary Locking (aka “2PL”), of the single version of the truth (ala the first Principle of a Database), we don’t have versions of data, let alone multiple versions, let alone deferred resolutions of multiple version.
Note that that description touches the fantasy of MVCC (which has hidden Locking) vs Locking, it does not mention the main limiting issue re concurrency: the Transaction Log, which also has to be serialised, and which is a higher order than the resolution of the multiple false versions of reality. It must be mentioned, there are two orders [not one] of issues that have to be resolved at COMMIT, which the guru of idiocy Stonebraker denies, and defers to the COMMIT.
The second and third layers of schizophrenia are the ways and means they have written, first in academic papers, and then in implementations, about how to not-have contention problems (by denying the existence of other users that cause contention); how to not-have consistency problems (by denying the existence of all other versions of the row except his “own” single row). Don’t worry, we will resolve everything when the drooling idiot COMMITS. Oopsy, it cannot be resolved, Sally’s row is 42 versions out-of-date, and Fred’s row that is blocking it is 36 versions out-of-date. No problem, let’s redefine “transactions”.
So they munchkins at Berkeley produced Ingres. Total failure. But heavily promoted by academia. So they produced PissGrossNONsql. Total failure. But heavily promoted by academia. “Don’t worry, we are clueless now, but at any moment now we won’t be, promise.” Same as the single cell from which all life forms “evolved”, “we will find it any moment now, even though we haven’t found it in 120 years”.
Larry Ellis saw a commercial opportunity, that he could do a better job of implementing the MVCC fiction than the Stonebraker ashram could (“hey, I don’t have to understand contention, I just deny its existence, and I can deny its existence better than Stonebraker”). It has no ACID, because MVCC cannot support ACID, it has contention problems that “2PL” systems simply do not have, but hey, everyone has a right to grab and hold a private row, while they wait for the coffee to get cold. It is institutionalised schizophrenia, a collective assent to a fiction, with masses of s/w and h/w resources demanded to make it “real”.
So the mantra “readers never block writers, and writers never block readers" may be true if we are charitable, but the great fraud is, they never mention that the method produces; tracks; sort-of-resolves, millions of multiple versions of the rows, which systems that utilise Locking do not do. And the second fraud is, the non-locking MVCC cannot resolve the multiple versions except by using Locking (and a disgusting mess of locks it is) on top of MVCC.
Hey, you get what you pay for. Freeware is free, no one can be held responsible.
> Anyway, I appreciate your own recount of those times, albeit somewhat
> critical of the sources I have cited. I do not feel qualified to comment
> on that any further, though.
> >> IDEF1X relies heavily upon Chen's shoulders.
> >
> > Nonsense. Give one example, one article. Rob Brown already had
> > a diagrammatic modelling method (as distinct from the various diagrams
> > that data modellers created) that was widely accepted ... after
> > consultation with Codd, he produced what became known as IDEF1X.
>
> FIPS 184, Background section: "The theoretical roots for [the initial
> approach to IDEF information modeling] stemmed from the early work of
> Dr. E. F. Codd on relational theory and Dr. P. P. S. Chen on the
> entity-relationship model."
So what, do you believe everything that is written ? They have written that homosexuals can have children, and that Chinese is a “language”.
When I said “article” I did not mean a report somewhere, I meant a concept, that is recognisable.
a. Name one thing, a method or rule or whatever, from the Chen ERD, that exists in IDEF1X.
b. Name one thing FROM the /RM/ that Chen has in his ERD.
> And two paragraphs later: "Application within industry had led to the
> development in 1982 of a Logical Database Design Technique (LDDT) by R.
> G. Brown of the Database Design Group. The technique was based on the
> relational model of Dr. E. F. Codd, the entity-relationship model of Dr.
> P. P. S. Chen,
> and the generalization concepts of J. M. Smith and D. C.
> P. Smith."
> > Ok, fine. But your words contradict that, you can’t say that and also
> > say “My main complaint with IDEF1X is the treatment of generalization
> > hierarchies”.
> >
> > So what, if anything, is your remaining complaint re “generalization
> > hierarchies”.
>
> My complaint is not about "generalization hierarchies" per se. It is
> a specific critique on the specific notation used by IDEF1X. That has
> nothing to do with understanding transactions, or whatever. It is the
> lack of a way (if you follow the standard) or a cumbersome way (if you
> follow Bruce/ERwin) to represent what in your document is called
> "Non-Exclusive Subtyping", using the "complete categorization" symbol.
If you take the {Complete|Incomplete} text in the Standard as gospel, and set it up as a rule for perceiving data [that you wish to model and understand], no, that will damage your perception of data badly. IDEF1X is not a DEFINITION of Data Modelling as an exercise, it is a notation. The Data Modelling exercise requires you to be open, to perceive the data as it actually exists in reality, not forced into a {Complete|Incomplete} mindset.
Separately, there is nothing that you cannot define [from the real universe, using legal Predicates], in FOPC; and thus the /RM/, which is why I asked, give me a real example, and I will resolve it for you, it has nothing to do with {Complete|Incomplete} vs {Exclusive|Non-Exclusive}, it has everything to do with accurate Data Modelling, without any fixed mindset.
> cumbersome
How exactly, is it “cumbersome” ? Predicates are Predicates, atomic. If your model does not resolve, due to not-having the right Predicates, and I resolve the model by inserting the missing Predicate, which of course is essential, and therefore minimal, how is that “cumbersome” ? It is only “cumbersome” to the person who holds the invalid perception of the unresolved model with the one less Predicate.
> >> Regarding subtyping, my critique is specifically on the notation given
> >> in ERwin Methods Guide at the crossing of "Complete" with "Inclusive
> >> Subtype".
> >
> > What “crossing” ???
>
> I did not express myself clearly. I meant the IDEF1X (not IE) notation
> at the intersection of the "Complete" column and "Inclusive Subtype"
> row.
> > Sorry. I don’t understand what you are trying to say.
>
> My fault, not the best example of clarity. But the point is, in
> a nutshell, that non-exclusive subtyping is not straightforward as it
> should be (as it is in IE).
Please inform me as to how Non-Exclusive Subypes is not straight-forward.
> >> > Here is a quick document that clarifies the issues you questioned,
> >> > that I have answered. To be complete, I have added an item re
> >> > Rolenames, because it is the third common “issue” that people raise:
> >> > https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/IDEF1X/Nicola%20IDEF1X%202.pdf
> >>
> >> A question: what is the use case for an association that does not imply
> >> a foreign key (rightmost symbol in your "improved IE")? Why would you
> >> require a parent entity to exist at the time of insertion, but not
> >> later?
> >
> > It is actually a very common requirement. Refer this Data Model:
> > ____https://www.softwaregems.com.au/Documents/Student_Resolutions_2020/Giannis/RoomMonitor%20DM%20V0_51.pdf
> I have had time only for a cursory look at the requirements and the
> model,
> but one thing that I have noticed is that there is no independent
> Sensor entity.
> That strikes as something unusual. It's ok to associate
> a Reading with a Room for auditing, but with an independent Sensor you
> should also be able to maintain referential integrity even when
> a sensor's location is changed as the MAC is immutable. But, again,
> I need to re-read your requirements.
When a Sensor is replaced, there is no other change to the Monitor, or to the location.
When a Monitor is replaced (repair; back to factory; etc), the extant Sensors stay with the Monitor. After repair and refurbishment, the Monitor gets shipped to a new location, the next waiting.
At the end of the day, it is Reading that is permanent, not Monitor; not Sensor.
Sure, it is possible to model/implement a permanent identifier for each little Sensor, that stays with the Room, rather than the Monitor, but I would have two problems with that: first, it is not reality (there is no permanent or unique Sensor identifier), so it would have to be fabricated, and thus not Relational. Second, the model would have to be re-worked raising Sensor above the Monitor that houses it, which is false.
Besides, that DM has been through 51 iteration slots by virtue of the version number, probably 34 actual iterations (whereas Movie Title was 14 iterations, not yet mature). It is pretty mature, and implemented with no problem whatsoever.
Cheers
Derek
Derek Ignatius Asirvadem
Apr 4, 2021, 11:59:09 PM4/4/21
to
> On Sunday, 4 April 2021 at 20:50:43 UTC+10, Nicola wrote:
> > On 2021-04-04, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
>
> > Yes, the IDEF1X Subtype notation is broken. Yes, that is just one of
> > the many things that one should be able to (a) model, and (b) model
> > compliant with the /RM/, that cannot be rendered in IDEF1X/IDEF1X
> > Subtype Notation.
Before I get into the response, as detailed in my previous post, you need to divest yourself of rules or “rules” that you take by implication of the IDEF1X/IDEF1X Subtype definition. And open up to the world of reality, of modelling data as it is, of course in conformation to FOPC and the /RM/ but not conforming to limitations of IDEF1X or anything else (such as physical implementation in limited freeware; etc).
> > On 2021-04-04, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
>
> > Yes, the IDEF1X Subtype notation is broken. Yes, that is just one of
> > the many things that one should be able to (a) model, and (b) model
> > compliant with the /RM/, that cannot be rendered in IDEF1X/IDEF1X
> > Subtype Notation.
> There are two orthogonal aspects in a generalization hierarchy:
Further, I accept wholeheartedly that the correct observation of genus vs species is an essential part of genuine Analysis, here Data Analysis.
> 1. are the subtype mutually exclusive or not?
____Exclusive
In Logic, that is an XOR Gate [Discriminator] on the Basetype that identifies the single Subtype. It is Semantic (examples in the progressing doc).
The remaining category, since it is not Exclusive, and certainly not inclusive, is called:
____Non-Exclusive
The Subtypes are not mutually exclusive. There is no Discriminator. Typically, the Basetype has more than one [but not many or all] Subtypes. Note that each Basetype-Subtype pair must be perceived as a logical row.
> 2. is every instance [row] of the parent entity an instance [row] of some child
> entity ("subtyping" in your terminology), or not?
For Exclusive: yes, exactly one Subtype row.
For Non-Exclusive: yes, one or more Subtype rows.
> So, four combinations.
Whoa. What four combinations ??? How can you combine Exclusive with Non-Exclusive ??? You will get “everything”, or scrambled eggs without salami. That is an error at the intellection level, the conceptualisation level. It is invalid.
> Following the terminology of your document:
>
> | Exclusive?
> Subtyping? | Yes No
> -----------|----------------------------------------------
> Yes | Exclusive Subtyping Non-exclusive subtyping
> No | ? Optional attr.[Group], Not Subtype
2. I don’t understand it, so much as I would like to respond, I can’t.
> In IDEF1X notation, "non-exclusive subtyping" is the problematic one.
Which I find seriously lacking, since {Exclusive|Non-Exclusive} Subtypes existed before the /RM/ and before IDEF1X. Further, I am not about to be restricted by artificial limitations that are part of some notation, so I dismiss it on a second count.
> What is marked with ? corresponds to an "incomplete categorization" in
> IDEF1X terminology.
> Using IE notation, how would you model an
> "incomplete categorization"?
Well you can’t. Because the two methods of classification cannot be mixed.
Nevertheless, because the IDEF1X notation is inferior, a lower-order concept, it can be progressed to a superior, higher-order concept. Just get rid of the notion of “incomplete/complete”.
> There is no example in your document.
>
> Concrete example:
>
> - An Employee may be a Consultant;
> - An Employee may be a Full-Time Employee;
> - An Employee may be neither a Consultant nor a Full-Time Employee;
> - An Employee cannot be both a Consultant and a Full-Time Employee.
Page 4 added:
____ https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/IDEF1X/Nicola%20IDEF1X%202.pdf
> Note, I am not asking you how you would *implement* this situation; only
> how a data model would look like.
Why on earth would I implement something other than what I know the data to be, which knowledge I obtain by modelling it ? Why would I waste time modelling data if I did not have an implementation intent or that is not-for-implementation ? It would be a pathetic relic made of cigar smoke. No thanks.
Ok, fine. It is different to you. Why ?
> > The same problem exists with IDEF1X Cardinality notation.
>
> Which problem do you have with IDEF1X cardinalities?
If the notation cannot be implemented, then it is something that only theoreticians can use (without confidence). Thus any data modelling notation must have enough definition to host an implementation. The academic notion that theory should not make reference to practice deems the academic unfit for the theory that is the foundation of practice. Which means, we have had no academic; no theoretician, since 1970 and Dr E F Codd.
> >> In Bruce's book (and in ERwin) you'd use two clusters, each with one
> >> child, but that requires an interpretation that does not find
> >> an equivalent in the standards. At least, to the best of my
> >> understanding.
> >
> > Your understanding is correct.
> > That interpretation is false.
> > Bruce is an idiot (per reasons explained above, for noticing the problem and NOT resolving it).
> > Do not follow an idiot.
> > Further, the “two clusters with one child each” is a hideous, gross, Normalisation error.
> > Thus the model is invalid; incorrect, fix it.
>
> You could also show us how you would solve the example above in ERwin
> (with IE notation). I do not have an ERwin license, so I can't try it
> myself.
> > As a counter-point, think about UML. Even though it is heavily
> > marketed as a “Standard”, it is not a Standard by any means, because
> > it is (a) grossly incomplete: each modeller adds his own 5 to 10
> > notations to cover the missing bits because he has designed objects or
> > interaction for which no UML notation exists. When a few hundred do
> > that private dance and publish their models, those 400 or 500
> > notations become de facto UML, like it or not. Yes, you and I as
> > purists would reject all that. But whereas I would model it in
> > a genuine modelling notation, and thus resolve the issue, you would
> > stand and argue that UML is broken, that the added notation is
> > invalid.
>
> Well, one thing is asking to fix something that it's just a notational
> glitch that can be easily fixed; "fixing UML" is something on another
> level entirely: I'd not dare ask anything like that.
> > I have updated this doc to reflect the above. In case there remains
> > anything worth discussing on this issue, I have added a page with the
> > Subtypes, and given the Predicates (which are an excellent method of
> > verifying the model, a feedback loop).
> >
> > ____ https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/IDEF1X/Nicola%20IDEF1X%202.pdf
>
> Yes, the updated chart looks better. IE notation covers subtyping
> correctly. My only remaining question is the one above.
> RE the added page, looking at the transactions, I think PersonPastime's
> subtypes are not exclusive (the diagram has an X that should not be
> there).
By the way, I have given you two ways of transacting a 1::1-to-n relation.
Happy Easter
Derek
Derek Ignatius Asirvadem
Apr 5, 2021, 1:53:22 AM4/5/21
to
> On Sunday, 4 April 2021 at 08:27:43 UTC+10, Nicola wrote:
> My complaint is not about "generalization hierarchies" per se. It is
> a specific critique on the specific notation used by IDEF1X.
> **That has nothing to do with understanding transactions, or whatever.**
> a specific critique on the specific notation used by IDEF1X.
Well it does. It is fair enough that the academics think in terms of fragments, isolated from the context in which it exists.
But that is not the real world, in which things are integrated, atoms are not fragmented. We do not want to entertain something in theory that is not possible in implementation. A database is a single recovery unit. As long as one is using a genuine SQL platform, one has ACID Transactions. The following Predicates for the Basetype::Subtype relation:
__ Basetype is one of {Subtype1|Subtype2|...} -- Exclusive
____ Cardinality 1::1
__ Basetype is any of {Subtype1|Subtype2|...} -- Non-Exclusive
____ Cardinality 1::1-to-n
are implemented by a Constraint (Declaration in SQL). Transactions are declared Constraints (Methods in UML-speak), that provide ACID (if the SQL Platform provides ACID). They form the Database API.
That [SQL platform] gives us confidence that the implied Database Integrity; the Cardinality of 1::1 or 1::1-to-n is possible, and quite ordinary.
Corollary: it is not possible to implement such Database Integrity on freeware and other mickey mouse program suites (they cannot be called “platforms”), that do not have ACID and fraudulently use “SQL” in the label.
Cheers
Derek
Nicola
Apr 5, 2021, 9:28:43 AM4/5/21
to
On 2021-04-04, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
>> On Sunday, 4 April 2021 at 20:50:43 UTC+10, Nicola wrote:
>>
>>
>> Concrete example:
>>
>> - An Employee may be a Consultant;
>> - An Employee may be a Full-Time Employee;
>> - An Employee may be neither a Consultant nor a Full-Time Employee;
>> - An Employee cannot be both a Consultant and a Full-Time Employee.
>
>>
>> - An Employee may be a Consultant;
>> - An Employee may be a Full-Time Employee;
>> - An Employee may be neither a Consultant nor a Full-Time Employee;
>> - An Employee cannot be both a Consultant and a Full-Time Employee.
>
> I fail to see the problem, I must be missing something. Sorry.
>
> Page 4
That captures the requirements correctly, but it introduces an entity
>
> Page 4
(Employee FT_or_C) that does not exist in reality, it is not in the
requirements and, in fact, is not inherently necessary. An IDEF1X model
would have just three entities => IDEF1X is better than your IE by
Occam's Razor. QED
Note that your model, in a larger context, might be a better choice in
some circumstances. But with your modeling language, it's the only one
you can build for the requirements above.
You may counter this, but know in advance that I will not respond,
because for me this matter is settled. The last word is yours!
>> Note, I am not asking you how you would *implement* this situation; only
>> how a data model would look like.
>
> There is a difference in your mind ?
The difference between logical and physical.
> By the way, I have given you two ways of transacting a 1::1-to-n
> relation.
Nicola
Nicola Vitacolonna
Apr 5, 2021, 9:50:56 AM4/5/21
to
>> So, four combinations.
>
> Whoa. What four combinations ??? How can you combine Exclusive with
> Non-Exclusive ???
doesn't matter, though, because the matter is settled for me, as per my
previous post.
>> In IDEF1X notation, "non-exclusive subtyping" is the problematic one.
>
> If you say so. But you are mixing things up: in IDEF1X Notation,
> there is no “Non-Exclusive” Subtyping, there is only
> {Complete|Incomplete}.
a concept. And IDEF1X has not straightforward notation for that.
>> What is marked with ? corresponds to an "incomplete categorization"
>> in IDEF1X terminology.
>
> Ok.
>
>> Using IE notation, how would you model an
>> "incomplete categorization"?
>
> Well, you can’t. Because it does not exist in IE. Because it does
> not exist in reality.
example is just that. Similarly to the above, "incomplete
categorization" is not a notation, it is a concept. And your IE has not
straightforward notation for that.
> Nevertheless, because the IDEF1X notation is inferior, a lower-order
> concept, it can be progressed to a superior, higher-order concept.
> Just get rid of the notion of “incomplete/complete”.
a generalization hierarchy with 1:1-0:1 relationships. While you can do
that, it may not be the most natural or more economical way of modeling
some situations, witness your Employee[Full-Time,Consultant] solution.
Nicola
Nicola
Apr 5, 2021, 10:32:29 AM4/5/21
to
On 2021-04-05, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
>> Anyway, I appreciate your own recount of those times, albeit somewhat
>> critical of the sources I have cited. I do not feel qualified to comment
>> on that any further, though.
>
> Hah. You will love the critique I have for the new list of academics,
> above.
Too bad the people you mention, and often insult, are not here to
>> Anyway, I appreciate your own recount of those times, albeit somewhat
>> critical of the sources I have cited. I do not feel qualified to comment
>> on that any further, though.
>
> Hah. You will love the critique I have for the new list of academics,
> above.
provide their point of view. You know, the discussion would be much more
interesting.
>> >> IDEF1X relies heavily upon Chen's shoulders.
>> >
>> > Nonsense. Give one example, one article.
>>
>
> Yes, it is written.
>
> So what, do you believe everything that is written ?
Never. Not by you, either :-)
> Yes, it is written.
>
> So what, do you believe everything that is written ?
Nicola
Nicola
Apr 5, 2021, 2:48:54 PM4/5/21
to
On 2021-04-04, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
>> On Sunday, 4 April 2021 at 20:50:43 UTC+10, Nicola wrote:
>>
>>
>> Concrete example:
>>
>> - An Employee may be a Consultant;
>> - An Employee may be a Full-Time Employee;
>> - An Employee may be neither a Consultant nor a Full-Time Employee;
>> - An Employee cannot be both a Consultant and a Full-Time Employee.
>
>>
>> - An Employee may be a Consultant;
>> - An Employee may be a Full-Time Employee;
>> - An Employee may be neither a Consultant nor a Full-Time Employee;
>> - An Employee cannot be both a Consultant and a Full-Time Employee.
>
> I fail to see the problem, I must be missing something. Sorry.
>
> Page 4
This is my comparative assessment of IE vs IDEF1X notation, re
>
> Page 4
generalization:
https://jirafeau.net/f.php?h=1mnluFoq
Feel free to add your comments, reuse as you see fit, or put on your
site.
Nicola
Derek Ignatius Asirvadem
Apr 7, 2021, 5:51:17 AM4/7/21
to
> On Monday, 5 April 2021 at 23:28:43 UTC+10, Nicola wrote:
> > On 2021-04-05, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
>
> >> Anyway, I appreciate your own recount of those times, albeit somewhat
> >> critical of the sources I have cited. I do not feel qualified to comment
> >> on that any further, though.
> >
> > Hah. You will love the critique I have for the new list of academics,
> > above.
>
> Too bad the people you mention, and often insult, are not here to
> provide their point of view. You know, the discussion would be much more
> interesting.
Not really. The slaves of the “people I mention”, as well as of the “sources you have cited” post in this forum, so there is no question that their fetid legacy lives on in spirit. They are not here in person, true, but they and their legacy are here, destroying science, and hammering anyone who does not toe the communist party line. It is a protected space for academics.
> > On 2021-04-05, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
>
> >> Anyway, I appreciate your own recount of those times, albeit somewhat
> >> critical of the sources I have cited. I do not feel qualified to comment
> >> on that any further, though.
> >
> > Hah. You will love the critique I have for the new list of academics,
> > above.
>
> Too bad the people you mention, and often insult, are not here to
> provide their point of view. You know, the discussion would be much more
> interesting.
If you notice the interaction in this forum over the last 15 years, I am the only one who rejects the insanity and takes them on, point by laboured point. I too, did not invent the science, I stand on the shoulders of Science (as defined 350BC to 1911, when the destruction commenced) and Codd, I too, am acting in that spirit, a slave of that system.
So what we have here is the the great war between sanity and insanity, being played out by second generation slaves. It used to be interesting when the golum army was active, but over the years, as O destroyed each of them, they have gone quiet, their only remaining weapon ad hominem, which advertises their position of total loser.
The discussion /was/ interesting. But in the sense of resolution, after the battle is over and the issue has been closed by me, there is not the academic Middle to keep debating, I have Excluded it.
It is with some gratitude that I welcome you, as leaving that crowd, and attempting to cross the great divide into reality (we will have to define that now). But as evidenced, it is slow, moving in fits and starts, often not resolving the issue discussed. And although you are trying hard to grow out of the academic mindset of non-reality, when cornered, you fall back into it.
> insult
Sure, the tone might sometimes be insulting, it is very hard to discuss an idiotic concept without calling it an idiotic concept. But I take pains to ensure the charge is clear. If you have noticed any insult that does not have an underlying charge, please let me know.
I do not retract my insults. You are free to protect the mentally ill; the deranged; the drooling idiots; the purposely evil saboteurs, who are established as academics, from normal humans. You have to decide between your loyalty to freaky academia and the truth.
One the one hand academics argue that an argument from authority is poor, but hysterically, hypocritically, they venerate published academics, regardless of how idiotic their papers are. I don’t hold imbeciles in positions of authority. Genuine authorities rule, and teach. None of you guys do the smallest ruling or tiniest teaching of the science, all of you guys make rulings on anti-Relational filth, and teach anti-Relational pig poop as “Relational”. You want nice treatment ? Go to a brothel. From the real world, you will get disrespect, insults.
What about the fact that for FIFTY years you have come up with nothing, while ranting and railing and caterwauling that you’ve producing the cutting edge of Jello. That is an insult to the undamaged intellect. You insult us, we will insult you back.
Respect is something that is earned, not something that is freely given. You want respect, sure, DO SOMETHING RESPECTABLE. Come up with one thing that articulates or progresses the /RM/, come up with one academic that attacks the fifty years worth of anti-Relational filth. Zero. We get only upset that we did not toe the academic line and genuflect to your imbeciles that never did anything, but tell great stories to interviewers.
Now you are the single exception. Please do not ask me to follow your lead and genuflect to imbecility. Come across the chasm and join me in the real world. Do not run back the safety, the reinforced bastion of insanity, at war with Reality.
> >> >> IDEF1X relies heavily upon Chen's shoulders.
> >> >
> >> > Nonsense. Give one example, one article.
> >>
> >> FIPS 184, Background section:
> >
> > Yes, it is written.
> >
> > So what, do you believe everything that is written ?
You, an established academic, capable of examining these things, did not answer this:
> >>
> >> FIPS 184, Background section:
> >
> > Yes, it is written.
> >
> > So what, do you believe everything that is written ?
> > When I said “article” I did not mean a report somewhere, I meant a concept, that is recognisable.
> > a. Name one thing, a method or rule or whatever, from the Chen ERD, that exists in IDEF1X.
> > b. Name one thing FROM the /RM/ that Chen has in his ERD.
What is written is lies.
[b] Therefore there is nothing in ERD that can be identified as Relational, not one skerrick.
c. Therefore given that (i) there is nothing Relational in ERD, (ii) the /RM/ has demands that ERD does not supply, (iii) that IDEF1X does supply, the use of ERD by all academics (except possibly you), is an anti-Relational act. Consistent with the freaks suppressing the /RM/, in order to promote their primitive concepts as the /RM/,
> > So what, do you believe everything that is written ?
>
> Never. Not by you, either :-)
The problem with academics is, they believe what the have written, they don’t wait for testing or peer reviews. “Here, MyLittlePony has rainbow mane, you can’t prove that that is false, therefore it is true. QED.” And then if anyone disagrees, they refuse to engage. “If you don’t believe that MyLittlePony has a rainbow mane, you are anti-semitic or homophobic or worse. I QED-ed you.”
It is a game for small children, devoid of science. But played all the time, by academics, presenting that filth as “science”.
> > On 2021-04-04, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
> >> On Sunday, 4 April 2021 at 20:50:43 UTC+10, Nicola wrote:
> >>
> >> Concrete example:
> >>
> >> - An Employee may be a Consultant;
> >> - An Employee may be a Full-Time Employee;
> >> - An Employee may be neither a Consultant nor a Full-Time Employee;
> >> - An Employee cannot be both a Consultant and a Full-Time Employee.
> >
> > I fail to see the problem, I must be missing something. Sorry.
> >
> > Page 4
>
> That captures the requirements correctly, but it introduces an entity
> (Employee FT_or_C) that does not exist in reality, it is not in the
> requirements and, in fact, is not inherently necessary. An IDEF1X model
> would have just three entities => IDEF1X is better than your IE by
> Occam's Razor. QED
God help me. By virtue of the evidence, you had an agenda from the outset, which was not declared. Then you played a stupid game, no doubt Masters Level, that I was not aware of (I thought you were engaged in honest technical discussion), in order to lead me into a “trap”, and once there, you gleefully declare that the “trap” worked.
> (Employee FT_or_C) that does not exist in reality, it is not in the
> requirements and, in fact, is not inherently necessary. An IDEF1X model
> would have just three entities => IDEF1X is better than your IE by
> Occam's Razor. QED
Ok, I grant, in the confines of your mind, you “won”. Ok, your academic programming is heavy and deep, and much as you tried to understand normal logic, you could not, you reverted to the mindless, logic-less, academic mindset. You “win” but you learned nothing.
Let’s try again, in the vain hope that honesty, your conscience motivates you to step outside your Masters level trick question.
> >> Concrete example:
Pffft. Nothing concrete about it. It is a concept level, simple Logic, question. Calling it “concrete” merely sets up the deceit for the “trap”. Called an ass a horse does not transform the ass into a horse, it only deems the person as dishonest at best, or a deranged evil “scientist” at worst.
Concept level question:
> >> - An Employee may be a Consultant;
> >> - An Employee may be a Full-Time Employee;
> >> - An Employee may be neither a Consultant nor a Full-Time Employee;
> >> - An Employee cannot be both a Consultant and a Full-Time Employee.
> >
> > I fail to see the problem, I must be missing something. Sorry.
> >
> > Page 4
> That captures the requirements correctly, but it introduces an entity
> (Employee FT_or_C) that does not exist in reality,
Your notion of “science” is in fact Materialism; Modernism, which artificially excludes science (Logic; the Four Laws; Causality; Hierarchy; Integration) that we have had from 350BC to 1911, such that you **van** pick out fragments of reality and treat them in isolation, divorced from their context and meaning (remember, you are learning about bringing meaning into the picture, very slowly, but you will not admit that which has excised meaning).
Thus the academic [your] sense of reality is perverted. It is (a) in denial of reality, and 9) a concoction, that becomes a collective agreement, widely cited, a “reality”. It is very narrow and self-serving, which is why people who are not crippled by the academic mindset, who model reality, come up with and use pedestrian methods for forty years, that you alone from academia are finding out about today.
Thus I, not being crippled; or ham-strung; or fed with pig poo, that is, relying only on undamaged science, which includes unperverted Logic, submit that Employee FT_or_C is a Logical Fact in Reality. Yes, it is not material at this silly conceptual level, so you being limited to Materialism, will be blind to the facts of reality.
> it is not in the requirements
1. The requirements for examples; class problems; and even Masters level tests, often leave something out (otherwise getting the answer would be a clerical, not technical, task). Check out the three ERDs that OP was labouring over. The solution is to *DETERMINE* the existence of FACTS that the entities in the ERD depend upon but were excluded (due to purposeful set up of a classroom problem, or criminal insanity [anyone giving those problems as solvable via ERD, as a part of Relational Data Modelling is criminally insane] ).
It is rather common in Data Modelling of reality, to determine the existence of such Facts which are not obvious to the novice. But academics evidently are blind to such: they concoct a “reality” and deny that reality exists. Therefore nothing that they produce can be used in reality. Eg. nothing since 1970, since Codd. If you challenge this, name one article.
2. If the example were a real one (exists in reality) as opposed to the classroom problem, then the producer of the test is an academic who is blind to reality, the Facts that exist but have no Matter (Materialism) to “sense” their existence.
3. But here we have a deceit, the typical academic Straw Man kind, the academic presents a silly classroom problem as “concrete”. And forces the End of Story. Precisely because he dare not go further, because going further will destroy his smugness, his glee, his “victory”. And after forcing a premature and anti-scientific termination to the study of the article he alleges to be studying, he prematurely sings out a “QED”. You asked a Q with an E and got a D.
> and, in fact, is not inherently necessary. An IDEF1X model
> would have just three entities => IDEF1X is better than your IE by
> Occam's Razor. QED
> Note that your model, in a larger context, might be a better choice in
> some circumstances.
Why is it that you attempt optimisation at the conceptual stage ? While making sure that the doctrine of **NOT** optimising at the early stages is taught ? Gee, an “extra” entity; an “extra” index; an “extra” INSERT. Hilarious. You do not have four wheels on the car yet, but you insist that wheel number three is “extra”.
Why can you not see, that I have done this a thousand times, in Reality, in real world models that are implemented, and after about the third time of optimising the four-table model into the three-table model (not resolved but let’s not get distracted) and then reverting to the four-table model, I the real world practitioner, have determined that it is better to implement the plain Facts; the plain Logic, that never need changes, rather than the optimised Facts that always need change.
Let’s see what happens if we progress the conceptual model falsely labelled “concrete” in the direction of concrete, just a little bit. Let’s see, what can we come up with is we contemplate the three concepts in Reality, with or without the establishment of the Fact of Employment. I hope you do not mind if I do not labour over each and every increment. I have added to page 4.
____ https://www.softwaregems.com.au/Documents/Article/Database/Relational%20Model/IDEF1X/Nicola%20IDEF1X%202.pdf
> But with your modeling language, it's the only one
> you can build for the requirements above.
> You may counter this, but know in advance that I will not respond,
> because for me this matter is settled.
> The last word is yours!
Please, define your problem in technical terms, as a definition (even Incomplete/“Partial” appears to be misunderstood, but I cannot confirm/deny that because you do not have a definition).
As detailed, do not mix mutually exclusive classes, such as IDEF1X **and** IEEE notation for Subtypes.
It appears to me, that you have taken up IDEF1X (hurrah, forty years late but hurrah), over ERD (another hurrah), but you are stuck in that academic mindset of defending academics; of promoting their false authority. Instead of taking up the real world practice of IDEF1X usage, you have taken up the original version (which is forty years out-of-date, and parts have been rejected by practitioners from day one), plus a superficial academic critique that you know to be false but you trumpet it anyway.
The solution might be, to trust a theoretically solid practitioner, instead of taking the academic route (IDEF1X notation for subtypes) which is known to fail by everyone except academics.
> >> Note, I am not asking you how you would *implement* this situation; only
> >> how a data model would look like.
> >
> > There is a difference in your mind ?
>
> The difference between logical and physical.
That difference is contrived. It is absurd. You turn it OFF and ON when convenient, as per the evidence. It is incoherent. Further, academics have no clue what LOGICAL means, but that should be a separate thread.
>
> The difference between logical and physical.
> your modeling language
> your IE
Pardon me. As evidenced in the ERwin Methods Guide, IE was established and known long before IDEF1X and the idiotic ERD came along. It belongs to IEEE, an international Standards body, it does not belong to me. I just use the Standard.
The language (semantics) is FOPC. As articulated by Dr E F Codd in his /RM/. And his RA. And articulated in IDEF1X.
IDEF1X lacks the definition of IEEE subtypes that we had prior. Get over it.
Last, again, please define your thus-far undefined and incoherent problem, so that we can resolve it. Or at least agree that there is a problem. Terminating the engagement is anti-science, anti-transparent. I lose nothing, you lose your credibility.
I will get to the other posts over the next few days.
Cheers
Derek
Derek Ignatius Asirvadem
Apr 7, 2021, 6:31:56 AM4/7/21
to
Nicola
> On Tuesday, 6 April 2021 at 04:48:54 UTC+10, Nicola wrote:
> > On 2021-04-04, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
> >> On Sunday, 4 April 2021 at 20:50:43 UTC+10, Nicola wrote:
> >>
> >> Concrete example:
> >>
> >> - An Employee may be a Consultant;
> >> - An Employee may be a Full-Time Employee;
> >> - An Employee may be neither a Consultant nor a Full-Time Employee;
> >> - An Employee cannot be both a Consultant and a Full-Time Employee.
> >
> > I fail to see the problem, I must be missing something. Sorry.
> >
> > Page 4
>
> This is my comparative assessment of IE vs IDEF1X notation, re
> generalization:
>
> https://jirafeau.net/f.php?h=1mnluFoq
There is a problem with that. Actually two. First, let me confirm that my docs are in the public domain, and you are free to use them. But a copyright notice generally means “use as is” and “if us
> On Tuesday, 6 April 2021 at 04:48:54 UTC+10, Nicola wrote:
> > On 2021-04-04, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
> >> On Sunday, 4 April 2021 at 20:50:43 UTC+10, Nicola wrote:
> >>
> >> Concrete example:
> >>
> >> - An Employee may be a Consultant;
> >> - An Employee may be a Full-Time Employee;
> >> - An Employee may be neither a Consultant nor a Full-Time Employee;
> >> - An Employee cannot be both a Consultant and a Full-Time Employee.
> >
> > I fail to see the problem, I must be missing something. Sorry.
> >
> > Page 4
>
> This is my comparative assessment of IE vs IDEF1X notation, re
> generalization:
>
> https://jirafeau.net/f.php?h=1mnluFoq