On Thu, 25 May 2017 17:09:15 -0700 (PDT), Gavino himself
<
jack...@gmail.com> wrote:
>TO what extent can you write large programs without resorting
>to a sql database like postgresql?
That's not a clear question ... many applications have no need
whatsoever for a database. And there are many forms of database.
Incidentally, SQL is a language, not a type of database. SQL is used
primarily with "relational" databases, but there also are
"non-relational" and "partly-relational" databases. There are also
"ACID" and "non-ACID", "distributed" and "non-distributed". There are
databases that don't use SQL at all, and some that use lobotomized SQL
because they can't handle everything SQL can do.
Databases have many degrees of freedom.
It is perfectly possible to create a full client-server DBMS using
Lisp. AFAIK it has never been done commercially, but there is no
reason it could not be. A non-ACID, key-store or ISAM system would be
relatively simple. A full ACID relational system is unbelievably
complex: commercial quality RDBMS have literally *thousands* of
man-years invested in their implementations.
You mentioned Postgresql. Postgresql evolved from an existing
database called "Postgres" (without the "ql"). Postgres itself
evolved from a prior system called "Ingres".
Ingres was THE VERY FIRST publicly available relational DBMS. It was
released in 1974, almost a year before the 'R' database from IBM Labs.
Moreover, Ingres was free and open-source, created by people at the
University of California. Ingres was the preferred DBMS on Berkeley
Unix and was widely used.
However, Ingres did not use SQL - it used a very different language
called QUEL. SQL, in fact, did not yet exist when Ingres was
released, and Ingres never *officially* supported SQL even though it
was an option in later versions.
Many people felt that QUEL was superior to SQL, but IBM's marketing
power won out.
The long-winded point I'm making here is that modern SQL based
Postgresql has been in development for decades. To recreate its
capabilities in Lisp would take years ... not nearly as long as the
original development, but still a long time ... and it would be a
waste of effort that should have gone into using the existing DBMS
rather than trying to recreate it.
>Could you ever do a lisp only program for say a e commerce website?
You could ... but it would be orders of magnitude more work than just
using an existing DBMS.
>How would you split the data up among say 10 or 30 boxes as the
>website scaled?
This is impossible to answer in the abstract - it depends on the data.
Better to ask questions like these in a database forum with examples
of the data.
Availability? Consistency? Tolerance for communication problems?
Pick any 2 ... you can't have all 3.
It is a trusim that all high availability distributed systems have to
sacrifice either data consistency across their nodes, or tolerance for
communication problems among their nodes.
TL;DR. I haven't looked specifically at Hypertable, but most HA
systems have chosen to sacrifice data consistency, meaning that in the
event of network problems, data that *should* be identical on
different nodes may be different, and queries against different nodes
in the incomplete system may return different results.
Many applications can tolerate temporary inconsistency for some
definitions of "temporary". For example, almost nobody gives a shit
whether there is a 24hr delay in a tweet being viewable on the other
side of the world. But most people would scream if they made a bank
deposit, then flew a few hours to another city and arrived to find the
deposited money wasn't available.
You can reduce the problem by, e.g., making sure every piece of data
is stored to at least N nodes, so that it is more likely that an
overall consistent view can be assembled from the set of nodes that
are still available. But it's impossible to completely eliminate the
problem.
Hypertable and systems like it have their place, but they aren't good
for everything and they aren't replacements for RDBMS.
George