Brontobyte databases

[Ted Stockwell]

I am looking for pointers to information sources regarding about very
large databases including such topics as:

managing non traditional data (images, image sequences (animations), audio data)
database engine internals
user interface issues (how to prevent information overload when there
are 2 billion records in the database you are "browsing")

books, personal experience, and examples of currently running systems are
all of interest.

Thanks!

--
Ted Stockwell, Aggregate Computing t...@aggregate.com
"Pragmatism... is that all you have to offer?!"

[Eugene N. Miya]

In article <TED.94Au...@janus.aggregate.com> t...@aggregate.com

(Ted Stockwell) writes:
>I am looking for pointers to information sources regarding about very
>large databases including such topics as:
>
> managing non traditional data (images, image sequences (animations),

> database engine internals
> user interface issues (how to prevent information overload when there
> are 2 billion records in the database you are "browsing")
>
>books, personal experience, and examples of currently running systems are
>all of interest.

Like what?
Many such databases are either secret or contain highly proprietary data.
The largest set I have had to deal with is a 200+ GB dataset of the
unsteady air flow around a descending VTOL delta wing. I need to rewrite the
current paper I have written. 90,000 time steps, 9,000+ messy file sets which
poorly data compressed (so we didnt'). Ls on something like this is a
joke. Moving it around was humorous (tragic). Directories? A
little too simplistic. One does not simply cp or rcp something this big.
You run into reliability problems of disks, tapes, and networks.
And if you restart things, you hope not to restart from the beginning.
Gives you a lot of time to read and post net news. 8^)

This is a very difficult open area of research akin to steering supertankers
(Bligh Reef? where?).
Too easily characterized by prefixes (tera, peta, eka, giga); the use of
each prefix should be proportionally harder, yeah that's the ticket.
Friday.

--eugene miya, NASA Ames Research Center, eug...@orville.nas.nasa.gov
Resident Cynic, Rock of Ages Home for Retired Hackers
{uunet,mailrus,other gateways}!ames!eugene
My 3rd favorite use of a flame thrower is "Fahrenheit 451."
A Ref: Uncommon Sense, Alan Cromer, Oxford Univ. Press, 1993.