Fwd: OBJECT STORAGE/TRACKING LIMITATIONS

[LTDPRM]

All,

I am wondering if this is a leading edge indicator for peril with object storage in the cloud.   We're promoting object stores with preservation objects.  How scalable is that strategy? What are the limits? Why are Atmos' error codes topping out at 100M as noted below?

How does this or does this affect any other object implementations or does it simply add another dimension we have to put on the list to check (audit)? 

Michael Peterson

Michael Peterson  |  IMERGE Consulting

Information Services and Digital Preservation Architect
SNIA Founder, past-President, and Chief Strategy Advocate
(805)201-3178       |  mpet...@ltdprm.org
www.ltdprm.org

Begin forwarded message:

EMC ATMOS

... I have spoken to a number of distributors and IT customers who have gone through both the paper analysis and actual proof-of-concept testing of Atmos. Given the choices available in the market today, nearly all of the IT customers chose to deploy other storage solutions. Why? Simple answer: They have a requirement to scale past 100 million objects. The customer that chose Atmos had no such object number requirement, but did have applications built to use Atmos' type of interface.

The distributors I have spoken with tell me that it is their opinion that "EMC will pull Atmos from the channel and move to isolated sales via their direct sales channel." I am told it is because of the error codes it spits out once the limited object store goes beyond 100 million objects. The distributors informed me that it was "becoming increasing more difficult to position the product within environments that need massive scale-out capability."

100 million objects may sound like a lot. In actuality it is not much given today's massive data storage requirements. This is especially true when you consider that a single Web-based company--even a single department of an entertainment company, for example--can have over 400 million JPEGs that need to be stored in the cloud. Given the recent acquisition of Isilon by EMC, it's my opinion that EMC will move Atmos' development into the Isilon team and integrate Atmos' object store within the Isilon Clustered File System.

http://www.networkcomputing.com/cloud-storage/229500401

[Henry Gladney]

A more thoughtful response than my current "knee-jerk" reaction is unlikely 5o differ substantially.  Nor is what follows likely to surprise you, given its source.

Scalability of a store is an orthogonal issue to LTDP, in my opinion.  If anything, it is less important for LTDP than for current usage, because it is unlikely that document owners will be interested in preserving more than a small fraction of digital holdings.

Vis-a-vis LTDP, cloud storage is unlikely to add any qualitative advantages or disadvantages to institutional digital libraries/archives/record repositories as these existed before anybody introduced the cloud hyperbole.

This is so because, from the viewpoint of record creators and eventual record users, both cloud stores and other archival stores are services provided remotely from any domains controlled by their users (either readers or writers), and are not substantially different w.r.t. the opportunities for misbehavior by employees of their controlling institutions.

As I have written 100 times (or is it 100,000 times?) what differentiates any "properly" preserved information bundle from a less valuable bundle is a complex of qualities inherent in the bundle in question--not a property of how it is managed, except that such management might degrade its pertinent aspects.

Best wishes, Henry

H.M. Gladney, Ph.D.   http://www.hgladney.com/  (408)867-3933