NoSql Databases

[chandru murthi]

I rarely read anything on computers, figuring working with them is bad enough ;).

Recently came across NoSql databases, and find, to my pleasant surprise, Pick versions listed as one of them (multivalue.)

So my question is: Does this reaction against relational databases have any legs? Does anyone in this world really think Pick is a member in good standing, so to speak, and...!!!...could the death of Pick has been put off again?

Interestingly, some of what I read talks about relational databases being "out of touch" and "not real-world," similar arguments made against Pick in thr lae '70's when Codd, bless his black heart, more-or-less singlehandedly destroyed the growth of Pick.

wiki: " simplicity of design, simpler "horizontal" scaling to clusters of machines (which is a problem for relational databases),^[2] and finer control over availability. The data structures used by NoSQL databases (e.g. key-value, wide column, graph, or document) are different from those used by default in relational databases, making some operations faster in NoSQL."

Chandru

[Wol's lists]

On 24/02/18 20:25, chandru murthi wrote:
>
> I rarely read anything on computers, figuring working with them is bad
> enough ;).
>
> Recently came across NoSql databases, and find, to my pleasant surprise,
> Pick versions listed as one of them (multivalue.)
>
> So my question is: Does this reaction against relational databases have
> any legs? Does anyone in this world really think Pick is a member in
> good standing, so to speak, and...!!!...could the death of Pick has been
> put off again?
>
> Interestingly, some of what I read talks about relational databases
> being "out of touch" and "not real-world," similar arguments made
> against Pick in thr lae '70's when Codd, bless his black heart,
> more-or-less singlehandedly destroyed the growth of Pick.
>
> wiki: " simplicity of design, simpler "horizontal" scaling

> <https://en.wikipedia.org/wiki/Horizontal_scaling#Horizontal_and_vertical_scaling>
> to clusters <https://en.wikipedia.org/wiki/Cluster_computing> of
> machines (which is a problem for relational databases),^[2]
> <https://en.wikipedia.org/wiki/NoSQL#cite_note-leavitt-2> and finer

> control over availability. The data structures used by NoSQL databases
> (e.g. key-value, wide column, graph, or document) are different from
> those used by default in relational databases, making some operations
> faster in NoSQL."
>

IMHO, Relational's 12 rules were defined not to make databases work in
the sense of being a decent data store, but were defined to make
databases work in the sense of the computer could actually cope. And
they got it wrong.

Rule 1: "Data comes in rows and columns". Does it heck. So right at the
very start you have to give a Data Analyst a sledge-hammer to try and
bash square pegs into round holes. And it goes downhill from there.

Pick is a proper superset of relational, so anything relational can do,
so can Pick.

Relational demands that everything be forced through the First Normal
Form gateway, so that if they discover faster ways of doing things, they
can keep the business logic separate from the database logic. The
problem is, the chokepoint that makes everything slow IS first normal
form, so most improvements can;'t be made without breaking the model.
Pick, on the other hand, is easy to prove that things *can't8 be made
any faster.

There's a lot that's good about the relational mathematical model, but
from the engineering viewpoint it's a disaster - one that can NOT be
fixed without junking the basic foundations! My favourite damnation of
it is that it is a model that consists solely of attributes and
relations. Attributes OF WHAT? Relations BETWEEN WHAT? Nuff said!

Cheers,
Wol

[Peter McMurray]

"Relational's 12 rules" . There is the first major error. There are in fact THIRTEEN. It is unfortunately obvious that Codd, who claimed to be a mathematician, did not understand the concept of Zero.

Zero is a point on the Number Line. Euclid defined a point 2,300 years ago as having position but neither width, length or breadth. It is where two elements meet, it does not represent an element in it's own right.

The more important one is forcing everything to First Normal Form. I remember Pick MV being defined as Third Normal Form when I attended the initial training session.

Far more important in my opinion is the ability to represent real world documents whilst maintaining strict definitions of an individual element.

Even in Pick it is reasonably easy to fall for simple traps when saving space.

I stored the debtor code in an invoice and used the relationship to pick up the name etc.. At the time we were required to maintain written records by the tax office.

However over the years disk capacity grew and the capacity to read stored tapes vanished. Unfortunately tax investigations can go back many years.

Now we store PDFs of the actual invoice so it does not matter if the customer name or address changes.

[Dawn Wolthuis]

I would love to give you the presentation I just did for Drake students last week. It was about PICK/NoSQL and SQL, pinpointing what relational theory did to the industry, why I do not like SQL, in general, and why it had a few decades of "winning." I also talked about what I do like about SQL features of 1st order predicate logic with the ability to both provide security and access to data without an application developer thinking twice about BI if they don't want to. I also show how what I did in the 70's with indexed sequential files, then the IMS DBMS in the 80's and then PICK/MV were a way of modeling data WITH LISTS that is back in style again with NoSQL DBMS.

By the way, it is a person from MV who has nosql.com .org and .net -- so that might tell you something.

My colleagues and I are working with Microsoft, because we like both their directions with SQL Server (you might notice they rarely mention SQL Server in any advertising these days) and with Cosmos. They are all about the Azure Cloud databases -- scalability, reliability, accessibility, etc. With MVON#, you can use what Microsoft offers. Microsoft, Oracle, and others have picked up on the NoSQL data models, which is very similar to MV (with infinite nesting -- I think that is too bad, really -- we could teach the JSON folks some data modeling best practices for NF2). I did a MongoDB POC and see no reason to use it in production -- both MVON# and OpenQM are far better "NoSQL" options. I'm told that with Couchbase and Cassandra, your mileage may vary, so I'm not seeing a big reason to switch from MV to a limited NoSQL product.

Many of us put in many hours over decades to get MV into the mainstream. It's there. We can use solid DBMS products with huge install bases for our MV applications now. Let me know if you want to see MV/NoSQL running in .NET with a Microsoft DBMS (we will show it with SQL Server as so far that is what everyone chooses, and it is a "NoSQL" data model projected for read/write SQL as well as NoSQL/MV).

--Dawn

Dawn M. Wolthuis

Take and give some delight today

--
You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com
To unsubscribe, email to: mvdbms+unsubscribe@googlegroups.com
For more options, visit http://groups.google.com/group/mvdbms

[chandru murthi]

Good points, wol.

Well, I have been reading more about NoSQL. It certainly has legs. Amazon & Google both have NoSQL databases, so I suspect sheer volume will prevail. The surprising thing to me is the number of NoSQL database styles, and my mind went numb after reading up what I counted to be over a hundred databases in about twenty categories.

I note that Multi-value databases most often do appear, as a separate category, but with little or no description or details. wiki editors of Pick, do your work!

[Patrick Payne]

Pick falls best into the Document Database group.  We are closest as a model to the jSON style document databases like Mongo and Cassandra.  A couple MV platforms have managed to get themselves listed in the wiki https://en.wikipedia.org/wiki/Document-oriented_database.

Last year I found this article published in 2010 that has a interesting opinion of the history of NoSQL.

http://blog.knuthaugen.no/2010/03/a-brief-history-of-nosql.html

[Dawn Wolthuis]

Chandru -- in case you have not read some earlier notes on this (earlier threads), there is this other odd history related to NoSQL -- it might be a coincidence, might not. 

I played the C'mon Get Happy song from the Partridge Family for students. It starts with "Hello World." The use of "Hello World" for trying out a new language or tool started during the time of the Partridge Family, after this song was on in many houses weekly. Coincidence? Could be.

Later I showed my blog entry from when Tom DeL acquired the nosql.com etc domain names -- scroll down to where you see the NoSQL image

http://www.tincat-group.com/mewsings/2007/01/otlt-metadata-piece-not-apartheid.html

This pre-dates the popularity of the NoSQL data model. In late 2006, when I was writing my Is Codd Dead blog entries, three of us (Ashley, Tom, and me) were in a skype chat and decided we would come up with an umbrella name for NF2 databases, so that MultiValue was just one type with various key-value pairs, XML, etc, as other types. By "No SQL" (we had a space in there), we meant that application developers did not need to use SQL to access data. You could access data with No SQL. We said at the time that we would toss it out there to the universe (so to speak) and see what might come of it. I put it in my blog, as you see here, and others likely did other things. Over the next year or so, my various blog entries were hit by more than 15,000 unique individuals (said google analytics). A couple of years later, "NoSQL" became popularized by others. Coincidence? Could be. In either case, MV made it to the nosql party BEFORE others were, and NoSQL is definitely a superset of MV.

When it first became popularized, I put it out there in a few forums, e.g. let Curt Monash and others outside of the MV community know that MV was a NoSQL data model.  I've been lax in recent years on that front, but now that we have joined up SQL and NoSQL data models so that all features are available for MV apps, and now that Microsoft is putting so much into this effort too (along with Google and others), I'm doing talks on SQL vs NoSQL again. I learned a lot from relational theorists as well as MV afficionados.

--Dawn

Dawn M. Wolthuis

Take and give some delight today

[Kevin Powick]

NoSQL databases are also "No Conformity among Vendors".  i.e. They're all highly proprietary.

Each vendor offering fulfills a specific need, such as horizontal scaling.  However, this may incur a "cost" in another area, such as data consistency.  Of course, there are often extensions and 3rd party options to help mitigate shortcomings.

One has to be very aware of the caveats of each NoSQL product before committing to a particular offering.  As is often said, choose the right tool for the job.

--

Kevin Powick

[Dawn Wolthuis]

Yes, precisely. This is why companies with database products that have broad feature-sets are adding NoSQL options.

Microsoft is doing something interesting. I'm hoping to get a few more clues next time I chat with them. They are focused on the cloud and not really promoting any DBMS products by their name right now (e.g. SQL Server) outside of talking about Azure.

They also rebranded their DocumentDB as Cosmos. Cosmos is the name of the company that wrote Revelation, the first PICK database on the Microsoft Windows platform. Coincidence? Hmmm, maybe, but there might also be some historians at Microsoft.

I'm hoping that over time this is their way of ditching "SQL" in the name of their DBMS product because they support multiple data models. 

--Dawn

Dawn M. Wolthuis

Take and give some delight today

[Will Johnson]

Document-oriented databases rely on metadata *within* the document to extract meaning.

Pick data does not have metadata within any particular document, but rather, you could say, within an entire file of documents

You could I suppose consider the entire file as being *a* document, but I really don't think the creators of json and other document databases would buy that

[Dawn Wolthuis]

From what I know, there are different approaches to the storage among NoSQL suppliers, some with and some without the metadata. The MV flavor of NoSQL obviously uses delimited data with descriptive, not prescriptive, metadata that is not intertwined. It can be presented as JSON, XML or anything else by blending the data and metadata.  --Dawn

Dawn M. Wolthuis

Take and give some delight today

--

[Glen Batchelor]

JSON and XML are methods of data description. Object notation and markup languages are not storage methods, just methods of describing relationships. Any database that stores ASCII can store XML or JSON in it, so that complicates the argument.

--

[Will Johnson]

Provide a specific example of any document-oriented database that has no metadata within the document.
I don't think you're speaking about the same thing I am.

NoSQL and "document-oriented" are not the same thing.
Document-oriented is *one type* of NoSQL

To unsubscribe, email to: mvdbms+un...@googlegroups.com

[Peter McMurray]

It seems that there is confusion as to what metadata is.

JSON and similar contain metadata about the structure of data that in a multivalue database is contained in the dictionary.

An MV database typically contains metadata that indicates where, with the use of the dictionary , further data may be obtained.

Just as a library index card contains metadata about what a book is about and where it is; an MV invoice will contain metadata such as product code that in conjunction with the dictionary can provide further information about the product.

I gave the example of where this can lead to faults down the track. A customer code in the invoice points to name and address data that may change, One can surmount this issue with a date control in the Name and address file or simply store the full detail in a PDF of the actual document. In the latter case the MV invoice file metadata tells one the name of the document, invoice number, and the dictionary where one can find it - typically stored outside the MV database.

[Glen Batchelor]

Peter, a product SKU recorded in a table as a reference is not metadata it is a foreign key constraint. It's a pointer. The SKU itself does not describe any data except for itself, it is a unique reference value. Yes, the dictionary can infer metadata if configured to additionally describe what the foreign key represents. The dictionary structure, not the result of the translation, is then considered metadata.

--

You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com

To unsubscribe, email to: mvdbms+unsubscribe@googlegroups.com

[Steve Trimble]

To me the most verbose metadata is an xml structure. it's begin/end tagging describes the data within that document. i believe it was modeled after html language with its beg/end tagging style. personally, data should be just that alone. mv's value of variable length feature still amazes me. how functional and quick. sometimes I wish a DICT for each attribute HAD to be described. If it's not there, here's hoping the programmer believes in documenting the input programs. (yet again, a variable length record)

you gotta love it :)

be well all,

Computerized Data Mgmt Inc
Steve Trimble
(501) 615-8674 office

(501) 772-3450 cell / text

cdm...@gmail.com

[Glen Batchelor]

 XML contains metadata and literal data in the same media stream which is why it's often used for data object modeling and data transferring. An XSD is typically used to define what's metadata and what's literal data inside the document structure.

[Patrick Payne]

I do not agree with the Metadata requirement.  Pick has no problem serving up json objects which do have metadata.  I don't believe storing the metadata at the storage level is required and is actually very wasteful from a storage standpoint.  There are plenty of discussions going on how to speed up internal json operations and constantly having to parse a verbose json document is costly.  I would believe some type of binary/stripped storage will eventually start being used which will then probably look closer to pick.  You could even do the same type of item over the wire to speed stuff up.  Json may be better than XML but they are both much heavier than a direct pick record.

btw I can name a few document databases that do not store metadata from https://en.wikipedia.org/wiki/Document-oriented_database

Cache

Mumps

Unidata

Universe

And possibly Lotus notes??

Jackrabbit (looks to be some type of tree layout)

On Thursday, March 1, 2018 at 12:57:43 PM UTC-8, GlenB wrote:

 XML contains metadata and literal data in the same media stream which is why it's often used for data object modeling and data transferring. An XSD is typically used to define what's metadata and what's literal data inside the document structure.

On Thu, Mar 1, 2018 at 3:47 PM, Steve Trimble <cdm...@gmail.com> wrote:

To me the most verbose metadata is an xml structure. it's begin/end tagging describes the data within that document. i believe it was modeled after html language with its beg/end tagging style. personally, data should be just that alone. mv's value of variable length feature still amazes me. how functional and quick. sometimes I wish a DICT for each attribute HAD to be described. If it's not there, here's hoping the programmer believes in documenting the input programs. (yet again, a variable length record)

you gotta love it :)

be well all,

Computerized Data Mgmt Inc
Steve Trimble
(501) 615-8674 office

(501) 772-3450 cell / text

cdm...@gmail.com

On Thu, Mar 1, 2018 at 2:11 PM, Glen Batchelor <batch...@gmail.com> wrote:

Peter, a product SKU recorded in a table as a reference is not metadata it is a foreign key constraint. It's a pointer. The SKU itself does not describe any data except for itself, it is a unique reference value. Yes, the dictionary can infer metadata if configured to additionally describe what the foreign key represents. The dictionary structure, not the result of the translation, is then considered metadata.

On Thu, Mar 1, 2018 at 3:04 PM, Peter McMurray <pgmcm...@gmail.com> wrote:

It seems that there is confusion as to what metadata is.

JSON and similar contain metadata about the structure of data that in a multivalue database is contained in the dictionary.

An MV database typically contains metadata that indicates where, with the use of the dictionary , further data may be obtained.

Just as a library index card contains metadata about what a book is about and where it is; an MV invoice will contain metadata such as product code that in conjunction with the dictionary can provide further information about the product.

I gave the example of where this can lead to faults down the track. A customer code in the invoice points to name and address data that may change, One can surmount this issue with a date control in the Name and address file or simply store the full detail in a PDF of the actual document. In the latter case the MV invoice file metadata tells one the name of the document, invoice number, and the dictionary where one can find it - typically stored outside the MV database.

--
You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com

To unsubscribe, email to: mvdbms+un...@googlegroups.com

For more options, visit http://groups.google.com/group/mvdbms

--
You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com

To unsubscribe, email to: mvdbms+un...@googlegroups.com

For more options, visit http://groups.google.com/group/mvdbms

--
You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com

To unsubscribe, email to: mvdbms+un...@googlegroups.com

[Wols Lists]

On 01/03/18 20:11, Glen Batchelor wrote:
> Peter, a product SKU recorded in a table as a reference is not metadata
> it is a foreign key constraint. It's a pointer. The SKU itself does not
> describe any data except for itself, it is a unique reference value.
> Yes, the dictionary can infer metadata if configured to additionally
> describe what the foreign key represents. The dictionary structure, not
> the result of the translation, is then considered metadata.

Are we describing meta-data from the computer's viewpoint, or the user's
viewpoint?

To me, the basic definition is that data is the information given to me
by the user, meta-data is information I can deduce from the data. That
does create a slight problem - is the description data or metadata? I
think we agree it's metadata, but my definition makes it sound like
data.And actually, I could argue it's neither ... :-)

Then, when we look at it from the computer's viewpoint, things are
different again. Pick draws no distinction between data and metadata,
It's down to the programmer to put the data in the DATA file, and the
metadata in the DICT file.

SQL seems a lot clearer - metadata goes in the table definition, and
data goes in the table. But hang on a sec!!! The whole job of the data
analyst is to hard-instantiate a lot of what I define as metadata, and
to store it in the data table! An unavoidable consequence of first
normal form. That imho is why fnf datasets are so complicated and
difficult to understand ...

Cheers,
Wol

[Peter McMurray]

Hi GlenB

I can see that you have a restricted view of the term metadata here is a better explanation

"Metadata is "data [information] that provides information about other data". Three distinct types of metadata exist: descriptive metadata, structural metadata, and administrative metadata"

I suggest that you reconsider the Customer Id example that I gave. The Name and address are required for the bill statement, the invoice and any later tax investigation yet they may change over time so the data referred to in the customer id is extremely important metadata. 

[Will Johnson]

Pick is not a "document-oriented database" for the exact reason that the metadata is not a required part of any structure.

That Pick (unidata, universe) are key-value stores, does not make them document-oriented.

References to them could be make in key-value store, but they should not be part of the article on document-oriented

[Dick Thiot]

Does MultiValue not requiring dictionaries mean that it isn’t a document-oriented database?  I don’t think so because it depends on how the database is implemented. If all data structures (files) have a well defined set of dictionaries then isn’t it a document database? One of the benefits of MultiValue is its flexibility meaning it can meet different requirements and different NoSQL database categories. 

--

[Kevin King]

It's all a refrigerator to me.  The only difference is the ones that have the built in shelves vs. the ones where you can arrange the shelves any way you want.

To unsubscribe, email to: mvdbms+unsubscribe@googlegroups.com

For more options, visit http://groups.google.com/group/mvdbms

--

You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com

To unsubscribe, email to: mvdbms+unsubscribe@googlegroups.com

For more options, visit http://groups.google.com/group/mvdbms

--

-K

[Patrick Payne]

It is like saying mongoDB is not a document database because it allows non-json items to be also stored.

Per the wiki document databases are a subset of key-value stores. The difference is we have some type of structure that allows more complex objects which per the specs allows the database engine to do further optimization.  Our Dictionaries and @am/@vm/@svm most definitely allow this (our query language, indexing, etc).  

In addition, jSON and xml are actually markup languages and not really meta data.  Focusing purely on the internal pick language and direct access to pick records is probably not fair.  A pick application with nice EQU includes does have metadata.  Exposing the data out via ODBC or json it does have meta data.  I have sites that expose the actual dictionaries out (or have a master dictionary) you have no exposed the meta data.  This has been used to build .net classes. 

There real point here is this design model is very closely coupled to the pick model, both in why they started it and why it is popular.  I see no reason for separating us from this definition.  It has been very helpful to describe to management and other IT professionals what our model is.  But at the same time we are not MongoDB (yet).

In addition you could always use nothing but arrays to create your json object and it would be every bit as cryptic as a pick dimensional array.

[Peter McMurray]

Pick can be a lot of things however its major advantage is that it is used as a document oriented database.

GlenB says that SKU is just a pointer to a stock item. No It isn't. Perhaps I should have put the whole file definition in but I was aiming for brevity.

The SKU in my case is used to retrieve the true Product and Pack code. SAP made the monumental error of assuming that an SKU defined the exact product. It does not. It defines a particular product and pack combination. A light bulb may be a 100 Watt  White Phillips but the customer could not care less if it is made in the Philippines or China. 20-50 grade motor oil can be in 20 Ltr packs with standard markings or special issue markings. Again the customer could not care less but the blessed SKU will say out of stock when there are hundreds of ANOther pack face.

The product, pack and invoice date are then used to establish the price in conjunction with the customer type gained from the Customer ID translation. The price and tax are then recorded along with the SKU in the invoice file - note that in Australia the GST is required to be calculated on the Price per Unit not the total invoice value. 

If we then extend this to !NF all this information would be in a transaction file.

[Dawn Wolthuis]

It's 1969, and what's happening at IBM?

-- The Army, TRW, and IBM have just rolled out the first production use of the MultiValue data model. With this model, from the perspective of application developers, metadata is separate and descriptive. This data model arose out of language and requirements for people who were not computing professionals to be able to ask questions of the database and get correct results returned. Spoken-language-oriented.

-- Codd wrote his paper at IBM introducing the Relational data model (it gets published in 1970). With this model, from the perspective of application developers, metadata is separate and prescriptive. This data model arose out of need to decouple the physical storing of data from the logical use of the data and in simplifying some of the mathematics -- for example, using first order predicate logic is possible with relations representing predicates that have no lists in the data (aka first normal form), else you need at least MSO logic. Set-based.

-- Goldfarb, Mosher & Lorie wrote the GML at IBM for document markup, later becoming the XML data model. With this data model, from the perspective of application developers, metadata is intertwined with data and descriptive unless associated with a prescriptive document (xsd). Document-language-oriented.

At Random in 1969, LANPAR was the first spreadsheet deployed, but not on an IBM (I think it was on a GE machine)

with the popular data exchange format of comma-quote (or comma-separated) following relational and spreadsheets with tidy single-valued fields (unless one adds new separators, of course)

The XML data model was so bulky it lead to the adoption of JSON, essentially Javascript associative arrays with a one-to-one correspondence between them, so if you can project to one of these, you can project to the other.

A "document data model" is descriptive of XML, JSON, and also of MV. The two key differences between XML and MV are in whether the metadata is intertwined with the data (from the perspective of the application developer) and how deep the data goes (values, multi-values, sub-values is typical for an MV data model where XML and JSON can be infinitely deep. A document data model may deal with either data intertwined with metadata or not. It is typically considered to permit "infinite" depth, although best practices would not indicate going beyond sub-values. For most business data processing purposes, just as spoken and document language does not typically go deeper than "values" (lists within predicates), nor should XML or JSON IMO.

Does MV employ "a document data model"? By my definition, yes. Others can define differently. It is marketing-speak.

Today I'm only interested in a DBMS for most business data processing purposes that can handle at least the data models that were around in 1969 and are in production applications today. I don't want to be restricted by the religion of the Relational model, of XML/JSON, nor of MV. I'm rather on the pragmatic side of things. In 2018 we can do at least all of  the 1969 data models now, right?

--Dawn

Dawn M. Wolthuis

Take and give some delight today

To unsubscribe, email to: mvdbms+unsubscribe@googlegroups.com

For more options, visit http://groups.google.com/group/mvdbms

--

You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com

To unsubscribe, email to: mvdbms+unsubscribe@googlegroups.com

[Will Johnson]

No Dick.  Document-oriented databases *require* the metadata to be *embedded* in the document.
Pick has neither of those

On Friday, March 2, 2018 at 2:52:25 PM UTC-8, Dick Thiot wrote:

[Will Johnson]

And while "document data model" might be marketing speak, and "document-oriented database" has a precisely defined computer science meaning

I have no idea why anyone wants to *force* Pick to be something it's not.  There is no need for that at all.

That Pick is a key-value store database is enough, we don't have to be everything to be what we actually are.

[Tony Gravagno]

The definition of a wiki is that it's a public resource.
EVERYONE is a wiki editor. If you care, modify some content. If you don't, then it will remain in accurate.
T

On Monday, February 26, 2018 at 12:08:04 PM UTC-8, chandru murthi wrote:

[Tony Gravagno]

According to the linked wiki entry, MV qualifies as both Relational and NoSQL.
- Relational, in that we do not store All document data in one place, we do refer to multiple tables with foreign keys.
- NoSQL / Document-oriented for this purpose, in that we can modify any record without regard for the structure of the table. As Dawn said, we're descriptive via an external dictionary, no prescriptive via schema.

There are many good cases for using MV files in document-oriented style, where there is no common structure among most records. BASIC programs is one example. The MD/VOC and dict files are another. Dict-type files are freeform but we define a recognized structure for some/most of the items in there, specifically those beginning with letters A, D, F, I, M, N, P, S, X, etc.

As to metadata, "data about data", it's optional, not a requirement, depending on the NoSQL platform. For an example with MV, in control records, inconsistent among other records in the same file, I typically structure something like this:

01 First value is a description for anyone who reads the item. Second value is the data
    SOME_DATA_HERE
02 There's no need for dict items to define attributes in a single item.
     12345
03 Code just uses REC<ATB,2>, no second value = null
04 Set v2 here to 1 to enable Foo, 0 to disable
     1

So v1 there is metadata, as valid as any JSON "SomeData": SOME_DATA_HERE or XML <SomeData>SOME_DATA_HERE</SomeData>.
But it's not required by this specific NoSQL model.

Since there are a ton of NoSQL options out there, each with their own proprietary nuances, I suggest we just be one of those, rather than trying to lock ourselves into some specific box, and alienating people who don't like that box.

T

[Peter McMurray]

The Pick System's strengths are at the same time its weaknesses. Designed as an operating system independent storage and retrieval system, Basic came later, it is extremely flexible. Therefore it is dependent on the designer to use it sensibly.

The operating system independence vanished with the onslaught of DOS and Unix.  Pick Basic stayed true to the data model but lost out in popularity to C and its many variants despite being far more suitable for commercial programming.

Now we have gone pretty well full circle the Multi Value data model fits wonderfully with NoSQl and the Basic goes well with the flexibility of JavaScript. The latter knocked up in 10 days twenty years ago and now a full blown major development tool for monster projects. TypeScript provides the structure that Dick dreamed of, universally used on all operating systems plus the strict structure that keeps the mid level programmer in check. It has the added benefit of flexibility.

Designed with care , forget subvalues that Dick hated, the data model fits the row, column design perfectly. The fact that each column in a row can be multi valued much like a stack of spreadsheets with identical row definitions has always been its greatest strength and now that is available in TypeScript.

The fact that the MV model is 8 bit based makes it a perfect fit for the universal coding  - Unicode. Hopefully the introduction of Python in to the programming mix by Rocket means that somebody will finally realise that UTF-8 fits like a glove with no database re-definition required, unlike the UTF-16 and 32 variants. UTF-8  is not only the universal encoding of the internet it is the default for Python.

So the Multi Value system can drop into the only operating system independent environment with the TypeScript interface and UTF-8 whilst losing nothing at the back end. Basic programming is still the best for database CRUD.