On-Premise vs. Cloud Storage

[Brian Rogers]

Hi - looking to speak w/anyone who has shifted from on-premise to cloud storage (Glacier, Nearline, Azure, et al) for archival/production master files. I hope to gain a sense of how workflows were impacted, as well as shifting from a "one-time" cost to monthly/annual recurring cost. I imagine I might have enough questions to warrant a phone call or off-group email. Thanks in advance. - Brian

[Michael Kjörling]

On 13 Jun 2016 09:45 -0700, from pqb...@mocs.utc.edu (Brian Rogers):

> shifting from a "one-time" cost to monthly/annual recurring cost.

Not directly related to your question, but I really have to point this
out: _if you view local storage as a one-time cost, even as a
simplification, then your model is flawed._ Storage is _never_ as
simple as an up-front one-time cost; particularly in a large storage
array, there's always going to be the disk that needs replacing, and
not uncommonly, that will happen when it's the least convenient. Think
two disks in a RAID 6 array starting to glitch in and out on the day
before Christmas Eve or a similar long holiday. Your array _might_ be
just fine, but will you trust that it will be?

You need to always ensure that any stakeholders understand that
storage devices, no matter the underlying technology (rotational
magnetic, optical, solid-state, tape, ...), will need to be replaced
on a regular rotating/ongoing schedule, and that some storage devices
are going to need to be replaced earlier than planned simply because
they failed (or are starting to throw sufficient errors in a RAID
array that they cannot be trusted).

Cloud storage does offload the menial work of actually replacing the
drives to someone else, and generally you pay a premium for that.

I'd encourage you to calculate the cost per stored gigabyte (with some
reasonable level of redundancy) _per month of drive warranty time_,
then add electrical power (for rotational HDDs, 5-12 watts apiece
while they are running; also keep in mind that rotational HDDs are
more prone to failure during spin-up than in continuous operation),
cooling, the casing and controllers, a fudge factor for the work of
replacing drives, etc., and compare the amount thus obtained to the
monthly cost of cloud storage for the same amount of data. That will
be a more realistic comparison than simply looking at the up-front
cost of drives and comparing that to the monthly cost of cloud
storage. For a more thorough list of things to include, for a start, I
suggest looking at <https://serverfault.com/q/263694/58408>.

--
Michael Kjörling • https://michael.kjorling.se • mic...@kjorling.se
“People who think they know everything really annoy
those of us who know we don’t.” (Bjarne Stroustrup)

[Brian Rogers]

Hi Michael - thanks for the response. I was playing loose with terms. I don't view on-premise as a "one-time" cost per se. However, putting forth a proposal that is several hundred per month for cloud vs. several thousand upfront for on-premise has the easy appearance of being "one-time" with the latter. Completely agree w/you about the stakeholder point. Thanks for the serverfault link. - Brian

[Michael Kjörling]

On 13 Jun 2016 13:40 -0700, from pqb...@mocs.utc.edu (Brian Rogers):

> putting forth a proposal that is several hundred per month for cloud
> vs. several thousand upfront for on-premise has the easy appearance
> of being "one-time" with the latter.

I agree that putting forth the proposal in that way does make it seem
very costly up front (and one can easily argue that it is). Which is
why I suggest taking some time to work the numbers and figure out,
with reasonable replacement rates for the hardware, what the monthly
cost for running on-premise storage will be compared to the monthly
cost for using cloud storage.

It's possible that cloud storage will turn out to be cheaper in your
case, or perhaps more likely that the added convenience will be
considered worth the extra cost; I don't know about that. But it does
present a much more comparable view of the costs involved.

Oh, and don't forget to account for cloud storage transfer fees, if
applicable with the provider you choose.

Just as an example, if the two "several" in your reply are of similar
magnitude, you will break even within a year or two, whereas
higher-end hardware can come with five-year warranty periods...

[John Scancella]

I would also add in administration costs for 

* monitoring to make sure that the hardware is working correctly 

* disaster recovery (including lost productivity during down time)

* training personal to keep up with latest best practices

* system upgrades to keep hardware secure

[Stian Soiland-Reyes]

All of these in a way also apply to cloud storage:

* Regularly verify (all?) of data is still accessible (and correct)
* Disaster Recovery (full reupload?)
* Training personnel to keep up with latest cloud practices (e.g. compare cloud storage now and just 4-5 years ago)
* software upgrades to keep compatible with cloud API changes and security enhancements

In addition:

* Regular reevaluation of choice of cloud provider and technology (as for hardware)
* Secure software further to avoid accidentally exposing cloud access keys if it has an compromised (this is particularly important when your software also run on the cloud and need write access to the storage)
* Monitoring of future changes in Terms & Conditions and quotas
* keep up to date with sensitivity issues and data protection regulations (cloud providers might move their data centres and change their legal home)
* exposure to currency fluctuations and potential future price/plan changes (which also needs monitoring)

Or for own hardware (and single data centre cloud storage)

* Electricity supply downtime and surges
* Environmental risks (flooding, earth quake)
* Terrorism, war and sabotage (might be more likely towards a cloud provider than your institution?)
* Industrial espionage and physical security
* Upgrades/changes that go wrong (cloud providers SHOULD plan this better - but are you sure?)
* .. I won't go on further :)

--
You received this message because you are subscribed to the Google Groups "Digital Curation" group.
To unsubscribe from this group and stop receiving emails from it, send an email to digital-curati...@googlegroups.com.
To post to this group, send email to digital-...@googlegroups.com.
Visit this group at https://groups.google.com/group/digital-curation.
For more options, visit https://groups.google.com/d/optout.