X-BeenThere: duracloud-dev@googlegroups.com
Received: by 10.142.7.26 with SMTP id 26ls1172550wfg.1.p; Sun, 31 Jan 2010 
	01:41:35 -0800 (PST)
Received: by 10.142.250.18 with SMTP id x18mr551533wfh.17.1264930895856;
        Sun, 31 Jan 2010 01:41:35 -0800 (PST)
Received: by 10.142.250.18 with SMTP id x18mr551532wfh.17.1264930895827;
        Sun, 31 Jan 2010 01:41:35 -0800 (PST)
Return-Path: <cwil...@gmail.com>
Received: from mail-iw0-f194.google.com (mail-iw0-f194.google.com [209.85.223.194])
        by gmr-mx.google.com with ESMTP id 24si1125600pzk.6.2010.01.31.01.41.34;
        Sun, 31 Jan 2010 01:41:34 -0800 (PST)
Received-SPF: pass (google.com: domain of cwil...@gmail.com designates 209.85.223.194 as permitted sender) client-ip=209.85.223.194;
Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of cwil...@gmail.com designates 209.85.223.194 as permitted sender) smtp.mail=cwil...@gmail.com; dkim=pass (test mode) header...@gmail.com
Received: by iwn34 with SMTP id 34so28367iwn.21
        for <duracloud-dev@googlegroups.com>; Sun, 31 Jan 2010 01:41:34 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=domainkey-signature:mime-version:sender:received:in-reply-to
         :references:date:x-google-sender-auth:message-id:subject:from:to
         :content-type;
        bh=2YIR718HngALnk/MBgtllDIe3cDg4vwqI78dh9T1DvI=;
        b=WY2Ll/h50HlIdMwsHbHASctrw/ymnmo0LVdzGGqTKgIafHZKljqFDPTq9j2HUkmnVR
         2UTNG7tusjbrTr9b5ghmPp1f4kJXlEbY2eYlvamFk3GLOHRFIXSe4NmZRoXLHBcZiB6v
         hx+fqB8H05GYVM0YeInHzFxII+jHBMLzaHq3E=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type;
        b=TzEywDa7k0Br7H3dPyxFW8QUumxOPESk7HWO7iChXdvfGHoGNgBQxi10hnGxDIpJ5Z
         IEjNdU1gar9PN5Dpg5ZufGdgIMxtzG3fy5tqKb7d2jHdxRPdP+ICtLomMRTrTugO/Jx+
         y6yMYEC5m3lbXeZea8dQftlbVkMGitYt6Ubjg=
MIME-Version: 1.0
Sender: cwil...@gmail.com
Received: by 10.231.169.71 with SMTP id x7mr4799018iby.18.1264930894267; Sun, 
	31 Jan 2010 01:41:34 -0800 (PST)
In-Reply-To: <33b5cdc51001301016n6ff5f7b5y5363bfc551d6b...@mail.gmail.com>
References: <9a85ff971001300914m6d2325c1g92b5eb57de23f...@mail.gmail.com>
	 <33b5cdc51001301016n6ff5f7b5y5363bfc551d6b...@mail.gmail.com>
Date: Sun, 31 Jan 2010 04:41:34 -0500
Message-ID: <9a85ff971001310141j12d7a4b3gb0e62c170da42...@mail.gmail.com>
Subject: Re: [duracloud-dev] DuraCloud-Akubra update and a couple questions
From: Chris Wilper <cwil...@duraspace.org>
To: duracloud-dev@googlegroups.com
Content-Type: text/plain; charset=UTF-8

Hi Andrew,

On Sat, Jan 30, 2010 at 1:16 PM, Andrew Woods <awo...@fedora-commons.org> wrote:
> Hello Chris,
> It is great that the Akubra/DuraCloud integration is advancing.
>
> In regards to question 1, yes ami-6c58ba05 is the best one to use
> right now. It brings up a good point, however, about having a
> consistent and visible strategy for keeping track of AMI's, what is on
> them, and how they should be used, etc.

So far I've just been keeping notes in a README.  As long as I can
ping you for the latest and greatest info, I'm fine.

> In regards to question 2, the upload of content is a synchronous call.
> Due to the underlying storage provider's load balancers, it sometimes
> happens that updates with regards to the addition/deletion of content
> on one slave server does not get propagated to another slave (which
> may be servicing a subsequent download request) before a subsequent
> query takes place. In theory, the update should be visible immediately
> after the DuraCloud call returns. In practice, this is not always the
> case.

Thanks for the clarification.  I didn't realize that, but indeed it's
considered normal behavior with S3.

http://docs.amazonwebservices.com/AmazonS3/latest/index.html?ConsistencyModel.html

There are good engineering reasons for a looser model of "eventual
consistency".  But so far, the assumption with Akubra (and certainly
Fedora) has been that stored content is immediately read-after-write
consistent.

Since DuraCloud is agnostic about this, I think anything written to
work against it in the general case must assume the looser guarantee.

And since Fedora currently depends on the stronger guarantee, it seems
like the logical place to "bridge the gap" is from within the
Akubra-DuraCloud plugin: Some sort of delay/poll logic for writes,
which can be turned off if you don't need it.

- Chris