here is a my chatlog with rodrigods about the media server architecture design.
please feel free to criticize :)
[12:07:23 AM] Status is now: Available
[12:14:52 AM] rodrigods: ping
[12:15:00 AM] dodo: pong
[12:15:04 AM] rodrigods: i did a sketch of how i visualize the media server
[12:15:05 AM] rodrigods: http://dl.dropbox.com/u/60827853/mediaserver.png [12:16:36 AM] dodo: what kind of storage controller do you have in mind?
[12:16:59 AM] rodrigods: if we use java, we can have a layer
[12:17:06 AM] rodrigods: that is independent of technology
[12:17:15 AM] rodrigods: but... for the reference implementation
[12:17:23 AM] rodrigods: i'm thinking about using mongodb
[12:17:28 AM] rodrigods: with gridfs
[12:19:04 AM] dodo: what do you think what kind of data will be stored
in the media server?
[12:19:26 AM] rodrigods: binary data
[12:19:26 AM] rodrigods: with their metadata
[12:20:45 AM] rodrigods: (and also previews, for the case of images and videos)
[12:20:53 AM] rodrigods: (also, binary data)
[12:21:14 AM] dodo: hm .. yes binary data .. isnt the best way to
store these to put them just in a filesystem?
[12:21:35 AM] rodrigods: gridfs is a file system
[12:21:46 AM] rodrigods: but we also have the option to store only the metadata
[12:21:52 AM] rodrigods: with the path
[12:21:54 AM] rodrigods: for a fs
[12:22:22 AM] dodo: ah .. ok.. good, that,, because i thougth you
wanted to store the files in mongodb too :)
[12:22:35 AM] dodo: s/that//
[12:22:49 AM] rodrigods: gridfs is like a fs the mongodb uses to store
binary files
[12:23:00 AM] rodrigods: it automatically
[12:23:06 AM] rodrigods: generates a hash of the file
[12:23:14 AM] rodrigods: and also have fields just only metadata
[12:23:27 AM] rodrigods: i thought that it would be easier to handle
[12:24:01 AM] dodo: the files can have metadata in gridfs?
[12:25:06 AM] rodrigods: yes
[12:25:31 AM] dodo: so you want to store metadata twice? :}
[12:25:43 AM] dodo: (in gridfs and in mongodb)
[12:26:30 AM] rodrigods: gridfs doesn't generate all metadatas that we need
[12:26:38 AM] rodrigods: and it has a field
[12:26:41 AM] rodrigods: just to add it
[12:28:10 AM] rodrigods:
http://www.mongodb.org/display/DOCS/GridFS+Specification [12:31:10 AM] dodo: what else does gridfs offer, that is neccessary
for the media server?
[12:31:48 AM] rodrigods: it scales very well
[12:31:58 AM] rodrigods: (people say, i haven't experience with it)
[12:32:42 AM] rodrigods: but... if the latency of adding the file to
it, becomes a problem
[12:32:47 AM] rodrigods: we can change the approach
[12:32:53 AM] dodo: but is it neccessary?
[12:32:54 AM] rodrigods: and stores directly on a data base
[12:33:49 AM] rodrigods: hmm
[12:34:00 AM] rodrigods: yes... in a first moment
[12:34:22 AM] rodrigods: i don't see why we would have a big
scalability requisite
[12:35:07 AM] rodrigods: but... what in my head
[12:35:16 AM] rodrigods: the media server should be able to be used
for a lot of users
[12:35:26 AM] rodrigods: of different bc deploys
[12:35:38 AM] rodrigods: we can have one media server
[12:35:51 AM] rodrigods: for more than one bc deploy
[12:36:48 AM] dodo: i think there will be two kind of people that will
install a buddycloud instance with a media server: sysadmins ad
average nerds
[12:37:31 AM] dodo: the average nerd has a small number of users ..
with low scalability problems (probably none)
[12:38:14 AM] dodo: the sysadmin is an "experienced" user that will
probably want to handle the scalability problem on his own
[12:39:19 AM] dodo: which more or less means for us: dont handle
scalability of the storage and use a simple default (simple in setup
_and_ maintainance)
[12:39:44 AM] dodo: do you agree with that? or do you think that im
total nuts? :D
[12:40:02 AM] rodrigods: heheh
[12:40:03 AM] rodrigods: i agree
[12:40:17 AM] dodo: ok, cool
[12:41:08 AM] dodo: so .. im my opinion a simple solution would be to
just use the filesystem available on the machine serving the media
server
[12:43:16 AM] rodrigods: yes... maybe i was too much concerned with
the wrong thing
[12:43:23 AM] dodo: this solves both cases quiet well .. if a sysadmin
wants to scale the storage he can do it with the tools that she is
already familiar with (the day to day fs tools) .. if a average nerd
wants to install a media server, all she needs to add is a root path
for the folder containing the data and the credentials for the
metadata database
[12:45:13 AM] rodrigods: so we have to implement a good way for
organizing the files
[12:45:28 AM] rodrigods: a clever way
[12:45:38 AM] dodo: and especially syncing the metadata with the storage
[12:45:47 AM] rodrigods: we must be aware about some particularities of fs
[12:45:52 AM] dodo: yes
[12:46:19 AM] dodo: but i think they boil down to just a couple of ground rules
[12:46:43 AM] rodrigods: yep
[12:46:49 AM] rodrigods: we do that later
[12:46:59 AM] dodo: agreed :)
[12:47:39 AM] rodrigods: so, you agree about building a java storage
controller that communicates with a java xmpp component?
[12:47:58 AM] rodrigods: (the lower layer of the drawing)
[12:50:56 AM] dodo: im still uncomfortable with the idea if using java :P
[12:51:14 AM] rodrigods: heheh
[12:51:20 AM] rodrigods: which are our options
[12:51:25 AM] rodrigods: for this layer
[12:51:25 AM] rodrigods: i mean
[12:55:06 AM] dodo: well .. since you're keen to use java, i asume you
know the language quiet well, which means this is still an option ..
the other options are: luvit, nodejs, erlang, python, ruby, haskell, perl, etc…
[12:55:49 AM] rodrigods: ok...
[12:55:58 AM] rodrigods: and about the http layer
[12:56:52 AM] dodo: i like software that is easy to hack to code ..
especially in software for federations, because that encorages people
to start add new and awesome features
[12:58:32 AM] dodo: about the http layer i think, that projects like
nginx and apache a quiet good in serving files .. all that we have to
figure out is how to set permissions for files .. but i think
something NIH (own http server) cant we that bad :P
[12:59:49 AM] dodo: i would like to prefer nginx :}
[01:00:50 AM] rodrigods: that's your area i think
[01:01:02 AM] rodrigods: this part i'm very (totally) naive
[01:02:06 AM] dodo: the permissions thingy is the main reason why we
need our own media server solution
[01:03:57 AM] rodrigods: in my head
[01:04:02 AM] rodrigods: the flow is like that:
[01:04:24 AM] rodrigods: client (web, etc) sends http reqs for the http layer
[01:04:33 AM] rodrigods: 1 - client (web, etc) sends http reqs for the
http layer
[01:05:25 AM] rodrigods: 2 - the http layer transmits the req details
for the storage controller (like user@server, and where to they want
to put the file)
[01:05:52 AM] rodrigods: 3 - the storage controller asks the xmpp
component if that user can do what he is intending to do
[01:06:16 AM] rodrigods: 4 - the xmpp server asks the bc server about
this user permissions and responds yes or no the storage controller
[01:07:16 AM] rodrigods: 5 - depending on the xmpp component response,
the storage controller stores or no the file and returns a preview for
the http layer
[01:07:35 AM] rodrigods: 6 - the http layer responds with the preview
for the client
[01:07:55 AM] rodrigods: for getting a file, a similar flow would work
[01:08:42 AM] dodo: so ,this is the flow for saving data on the media serveR?
[01:09:02 AM] rodrigods: that is what i imagine, at least
[01:10:58 AM] rodrigods: any thoughts?
[01:11:09 AM] rodrigods: i'm completely nuts?
[01:11:11 AM] rodrigods: heheh
[01:17:09 AM] dodo: im currently reading
https://buddycloud.org/wiki/Buddycloud_media_server [01:17:50 AM] dodo: is that what you used as draft?
[01:18:22 AM] rodrigods: i used this doc:
https://docs.google.com/document/d/14mU8eXNGpzSn4axR36-doOfr-cq726KXw... [01:19:00 AM] dodo: i mean your flow makes sense .. and suits that
description .. but i must say, its not very well thought trough
[01:19:25 AM] rodrigods: sure...
[01:19:32 AM] rodrigods: i'm the student here!
[01:19:42 AM] rodrigods: what do you think
[01:19:45 AM] rodrigods: that would be better?
[01:20:12 AM] dodo: sorry ..did mean your flow .. its the wiki page
that is not very useful :P
[01:20:22 AM] dodo: s/did/didnt/
[01:21:00 AM] rodrigods: i always was the beckend guy
[01:21:07 AM] rodrigods: implementing database
[01:21:12 AM] rodrigods: components
[01:21:17 AM] rodrigods: etc
[01:21:30 AM] rodrigods: sorry if sometimes i appear a little confused
[01:21:36 AM] rodrigods: with http stuff
[01:21:45 AM] rodrigods: maybe is better use nodejs for everything
[01:21:54 AM] rodrigods: and i'm not aware about it
[01:23:48 AM] dodo: no its alright .. its just that it doesnt make
sense to me to make first a http request to the mediaserver, that wait
until the media server asks back over xmpp and so on .. but thats what
the wiki pages says
[01:24:50 AM] rodrigods: i think that is because would use the
channels permissions
[01:24:52 AM] rodrigods: based on pubsub
[01:25:13 AM] rodrigods: with a http api for doing the outside requests
[01:28:07 AM] rodrigods: sorry, the http api would be provided for the
media server, so we can have an independent way to build clients
[01:28:26 AM] rodrigods: simon cited the flickr http api
[01:29:41 AM] rodrigods: as example
[02:01:21 AM] dodo: alright .. i think i got something more useful
[02:02:56 AM] dodo: the media server consists out of nginx (serving
public data), filestsystem (storage) and an http server with the xmpp
component running
[02:03:27 AM] dodo: well dont has to be neccessarily a http server
[02:03:42 AM] dodo: but at least the xmpp component has to be there
[02:04:36 AM] dodo: what a bc enabled client has to do is:
[02:04:56 AM] dodo: 1. discover the channels media server
[02:05:56 AM] dodo: 2. negotiage a backend ..
> here is a my chatlog with rodrigods about the media server architecture
> design.
> please feel free to criticize :)
> [12:07:23 AM] Status is now: Available
> [12:14:52 AM] rodrigods: ping
> [12:15:00 AM] dodo: pong
> [12:15:04 AM] rodrigods: i did a sketch of how i visualize the media server
> [12:15:05 AM] rodrigods: http://dl.dropbox.com/u/60827853/mediaserver.png > [12:16:36 AM] dodo: what kind of storage controller do you have in mind?
> [12:16:59 AM] rodrigods: if we use java, we can have a layer
> [12:17:06 AM] rodrigods: that is independent of technology
> [12:17:15 AM] rodrigods: but... for the reference implementation
> [12:17:23 AM] rodrigods: i'm thinking about using mongodb
> [12:17:28 AM] rodrigods: with gridfs
> [12:19:04 AM] dodo: what do you think what kind of data will be stored
> in the media server?
> [12:19:26 AM] rodrigods: binary data
> [12:19:26 AM] rodrigods: with their metadata
> [12:20:45 AM] rodrigods: (and also previews, for the case of images and
> videos)
> [12:20:53 AM] rodrigods: (also, binary data)
> [12:21:14 AM] dodo: hm .. yes binary data .. isnt the best way to
> store these to put them just in a filesystem?
> [12:21:35 AM] rodrigods: gridfs is a file system
> [12:21:46 AM] rodrigods: but we also have the option to store only the
> metadata
> [12:21:52 AM] rodrigods: with the path
> [12:21:54 AM] rodrigods: for a fs
> [12:22:22 AM] dodo: ah .. ok.. good, that,, because i thougth you
> wanted to store the files in mongodb too :)
> [12:22:35 AM] dodo: s/that//
> [12:22:49 AM] rodrigods: gridfs is like a fs the mongodb uses to store
> binary files
> [12:23:00 AM] rodrigods: it automatically
> [12:23:06 AM] rodrigods: generates a hash of the file
> [12:23:14 AM] rodrigods: and also have fields just only metadata
> [12:23:27 AM] rodrigods: i thought that it would be easier to handle
> [12:24:01 AM] dodo: the files can have metadata in gridfs?
> [12:25:06 AM] rodrigods: yes
> [12:25:31 AM] dodo: so you want to store metadata twice? :}
> [12:25:43 AM] dodo: (in gridfs and in mongodb)
> [12:26:30 AM] rodrigods: gridfs doesn't generate all metadatas that we need
> [12:26:38 AM] rodrigods: and it has a field
> [12:26:41 AM] rodrigods: just to add it
> [12:28:10 AM] rodrigods:
> http://www.mongodb.org/display/DOCS/GridFS+Specification > [12:31:10 AM] dodo: what else does gridfs offer, that is neccessary
> for the media server?
> [12:31:48 AM] rodrigods: it scales very well
> [12:31:58 AM] rodrigods: (people say, i haven't experience with it)
> [12:32:42 AM] rodrigods: but... if the latency of adding the file to
> it, becomes a problem
> [12:32:47 AM] rodrigods: we can change the approach
> [12:32:53 AM] dodo: but is it neccessary?
> [12:32:54 AM] rodrigods: and stores directly on a data base
> [12:33:49 AM] rodrigods: hmm
> [12:34:00 AM] rodrigods: yes... in a first moment
> [12:34:22 AM] rodrigods: i don't see why we would have a big
> scalability requisite
> [12:35:07 AM] rodrigods: but... what in my head
> [12:35:16 AM] rodrigods: the media server should be able to be used
> for a lot of users
> [12:35:26 AM] rodrigods: of different bc deploys
> [12:35:38 AM] rodrigods: we can have one media server
> [12:35:51 AM] rodrigods: for more than one bc deploy
> [12:36:48 AM] dodo: i think there will be two kind of people that will
> install a buddycloud instance with a media server: sysadmins ad
> average nerds
> [12:37:31 AM] dodo: the average nerd has a small number of users ..
> with low scalability problems (probably none)
> [12:38:14 AM] dodo: the sysadmin is an "experienced" user that will
> probably want to handle the scalability problem on his own
> [12:39:19 AM] dodo: which more or less means for us: dont handle
> scalability of the storage and use a simple default (simple in setup
> _and_ maintainance)
> [12:39:44 AM] dodo: do you agree with that? or do you think that im
> total nuts? :D
> [12:40:02 AM] rodrigods: heheh
> [12:40:03 AM] rodrigods: i agree
> [12:40:17 AM] dodo: ok, cool
> [12:41:08 AM] dodo: so .. im my opinion a simple solution would be to
> just use the filesystem available on the machine serving the media
> server
> [12:43:16 AM] rodrigods: yes... maybe i was too much concerned with
> the wrong thing
> [12:43:23 AM] dodo: this solves both cases quiet well .. if a sysadmin
> wants to scale the storage he can do it with the tools that she is
> already familiar with (the day to day fs tools) .. if a average nerd
> wants to install a media server, all she needs to add is a root path
> for the folder containing the data and the credentials for the
> metadata database
> [12:45:13 AM] rodrigods: so we have to implement a good way for
> organizing the files
> [12:45:28 AM] rodrigods: a clever way
> [12:45:38 AM] dodo: and especially syncing the metadata with the storage
> [12:45:47 AM] rodrigods: we must be aware about some particularities of fs
> [12:45:52 AM] dodo: yes
> [12:46:19 AM] dodo: but i think they boil down to just a couple of ground
> rules
> [12:46:43 AM] rodrigods: yep
> [12:46:49 AM] rodrigods: we do that later
> [12:46:59 AM] dodo: agreed :)
> [12:47:39 AM] rodrigods: so, you agree about building a java storage
> controller that communicates with a java xmpp component?
> [12:47:58 AM] rodrigods: (the lower layer of the drawing)
> [12:50:56 AM] dodo: im still uncomfortable with the idea if using java :P
> [12:51:14 AM] rodrigods: heheh
> [12:51:20 AM] rodrigods: which are our options
> [12:51:25 AM] rodrigods: for this layer
> [12:51:25 AM] rodrigods: i mean
> [12:55:06 AM] dodo: well .. since you're keen to use java, i asume you
> know the language quiet well, which means this is still an option ..
> the other options are: luvit, nodejs, erlang, python, ruby, haskell, perl,
> etc…
> [12:55:49 AM] rodrigods: ok...
> [12:55:58 AM] rodrigods: and about the http layer
> [12:56:52 AM] dodo: i like software that is easy to hack to code ..
> especially in software for federations, because that encorages people
> to start add new and awesome features
> [12:58:32 AM] dodo: about the http layer i think, that projects like
> nginx and apache a quiet good in serving files .. all that we have to
> figure out is how to set permissions for files .. but i think
> something NIH (own http server) cant we that bad :P
> [12:59:49 AM] dodo: i would like to prefer nginx :}
> [01:00:50 AM] rodrigods: that's your area i think
> [01:01:02 AM] rodrigods: this part i'm very (totally) naive
> [01:02:06 AM] dodo: the permissions thingy is the main reason why we
> need our own media server solution
> [01:03:57 AM] rodrigods: in my head
> [01:04:02 AM] rodrigods: the flow is like that:
> [01:04:24 AM] rodrigods: client (web, etc) sends http reqs for the http
> layer
> [01:04:33 AM] rodrigods: 1 - client (web, etc) sends http reqs for the
> http layer
> [01:05:25 AM] rodrigods: 2 - the http layer transmits the req details
> for the storage controller (like user@server, and where to they want
> to put the file)
> [01:05:52 AM] rodrigods: 3 - the storage controller asks the xmpp
> component if that user can do what he is intending to do
> [01:06:16 AM] rodrigods: 4 - the xmpp server asks the bc server about
> this user permissions and responds yes or no the storage controller
> [01:07:16 AM] rodrigods: 5 - depending on the xmpp component response,
> the storage controller stores or no the file and returns a preview for
> the http layer
> [01:07:35 AM] rodrigods: 6 - the http layer responds with the preview
> for the client
> [01:07:55 AM] rodrigods: for getting a file, a similar flow would work
> [01:08:42 AM] dodo: so ,this is the flow for saving data on the media
> serveR?
> [01:09:02 AM] rodrigods: that is what i imagine, at least
> [01:10:58 AM] rodrigods: any thoughts?
> [01:11:09 AM] rodrigods: i'm completely nuts?
> [01:11:11 AM] rodrigods: heheh
> [01:17:09 AM] dodo: im currently reading
> https://buddycloud.org/wiki/Buddycloud_media_server > [01:17:50 AM] dodo: is that what you used as draft?
> [01:18:22 AM] rodrigods: i used this doc:
> https://docs.google.com/document/d/14mU8eXNGpzSn4axR36-doOfr-cq726KXw... > [01:19:00 AM] dodo: i mean your flow makes sense .. and suits that
> description .. but i must say, its not very well thought trough
> [01:19:25 AM] rodrigods: sure...
> [01:19:32 AM] rodrigods: i'm the student here!
> [01:19:42 AM] rodrigods: what do you think
> [01:19:45 AM] rodrigods: that would be better?
> [01:20:12 AM] dodo: sorry ..did mean your flow .. its the wiki page
> that is not very useful :P
> [01:20:22 AM] dodo: s/did/didnt/
> [01:21:00 AM] rodrigods: i always was the beckend guy
> [01:21:07 AM] rodrigods: implementing database
> [01:21:12 AM] rodrigods: components
> [01:21:17 AM] rodrigods: etc
> [01:21:30 AM] rodrigods: sorry if sometimes i appear a little confused
> [01:21:36 AM] rodrigods: with http stuff
> [01:21:45 AM] rodrigods: maybe is better use nodejs for everything
> [01:21:54 AM] rodrigods: and i'm not aware about it
> [01:23:48 AM] dodo: no its alright .. its just that it doesnt make
> sense to me to make first a http request to the mediaserver, that wait
> until the media server asks back over xmpp and so on .. but thats what
> the wiki pages says
> [01:24:50 AM] rodrigods: i think that is because would use the
> channels permissions
> [01:24:52 AM] rodrigods: based on pubsub
> [01:25:13 AM] rodrigods: with a http api for doing the outside requests
> [01:28:07 AM] rodrigods: sorry, the http api would be provided for the
> media server, so we can have an independent way to build clients
> [01:28:26
On 30 April 2012 18:33, dodo <dodo.the.l...@googlemail.com> wrote:
> here is a my chatlog with rodrigods about the media server architecture
> design.
> please feel free to criticize :)
I have a few remarks regarding the storage aspect of the media server.
While I agree to keep a first implementation simple, don't cut back on
thinking about an massively scalable design. My main request is not to
start your design with the assumption that the media server will be a
single process, running on one computer and having a local file system for
the storage, whether it be MongoDB+GridFS or something else.
The media server will become a *very* important part of BuddyCloud. Look at
the amount of media that is shared daily on Twitter, Facebook, Google+, ...
So before going of and start coding, let's make sure that we start
designing from the base idea that the media server will be a clustered
architecture, fault tolerant, running on distributed storage. Implementing
the minimal single machine, single process media server (= shortest path to
get something working), will probably end up looking very different within
this setup than starting the other way around (= start implement the single
machine version, than evolve in a clustered architecture).
My request from the beginning is to use a storage abstraction layer from
the start. When using Java, have a look at JClouds (http://jclouds.org).
For Ruby, there is the fog library (http://fog.io) and for Python, there
exists the libcloud library (http://libcloud.apache.org). For a simple
setup, the storage layer can be configured to use the local file system
(also for developer testing), but for production deployment, I could as
easily configure it to use Amazon S3 (1) and the related CDN feature
CloudFront (2).
Hi All and cheers to Ringo to join the discussion,
On 30 April 2012 21:53, Ringo De Smet <ringo.des...@gmail.com> wrote:
> My request from the beginning is to use a storage abstraction layer from the
> start. When using Java, have a look at JClouds (http://jclouds.org). For
> Ruby, there is the fog library (http://fog.io) and for Python, there exists
> the libcloud library (http://libcloud.apache.org). For a simple setup, the
> storage layer can be configured to use the local file system (also for
> developer testing), but for production deployment, I could as easily
> configure it to use Amazon S3 (1) and the related CDN feature CloudFront
> (2).
- none of us know how the media server will be used. (We think it will be really useful). But we can eliminate this uncertantity by building v0.1-mostly-works and seeing how people use it. - get the minimum viable product out as quick as possible and if necessary rewrite based on what we learn. (this could look like file upload, associate to a channel, display quick and dirty). The next version could have auth and the rest.
On Thursday, 3 May 2012 11:35:38 UTC+2, tkoski wrote:
> Hi All and cheers to Ringo to join the discussion,
> On 30 April 2012 21:53, Ringo De Smet <ringo.des...@gmail.com> wrote: > > My request from the beginning is to use a storage abstraction layer from > the > > start. When using Java, have a look at JClouds (http://jclouds.org). > For > > Ruby, there is the fog library (http://fog.io) and for Python, there > exists > > the libcloud library (http://libcloud.apache.org). For a simple setup, > the > > storage layer can be configured to use the local file system (also for > > developer testing), but for production deployment, I could as easily > > configure it to use Amazon S3 (1) and the related CDN feature CloudFront > > (2).
> On 30 April 2012 18:33, dodo <dodo.the.l...@googlemail.com> wrote:
>> here is a my chatlog with rodrigods about the media server architecture
>> design.
>> please feel free to criticize :)
> I have a few remarks regarding the storage aspect of the media server. While
> I agree to keep a first implementation simple, don't cut back on thinking
> about an massively scalable design. My main request is not to start your
> design with the assumption that the media server will be a single process,
> running on one computer and having a local file system for the storage,
> whether it be MongoDB+GridFS or something else.
i agree.
> The media server will become a *very* important part of BuddyCloud. Look at
> the amount of media that is shared daily on Twitter, Facebook, Google+, ...
we don't run servers for more than half of all human beings.
buddycloud builds a distributed platform, which means it should be
more easily install-able than scale-able.
> So before going of and start coding, let's make sure that we start designing
> from the base idea that the media server will be a clustered architecture,
> fault tolerant, running on distributed storage. Implementing the minimal
> single machine, single process media server (= shortest path to get
> something working), will probably end up looking very different within this
> setup than starting the other way around (= start implement the single
> machine version, than evolve in a clustered architecture).
do you have experience with this?
i'm writing nodejs for some time now (and am therefor quiet used to
write low level clustered systems) and must admit, the difference is
only in the backend of the server if you're doing it right.
so if you write code for one machine, its not that hard to just change
everything that is not scaling to something different (at least in the
nodejs world, where everything is a small module and all it takes is
to find it and to make the interface work with the existing code).
> My request from the beginning is to use a storage abstraction layer from the
> start. When using Java, have a look at JClouds (http://jclouds.org). For
> Ruby, there is the fog library (http://fog.io) and for Python, there exists
> the libcloud library (http://libcloud.apache.org). For a simple setup, the
> storage layer can be configured to use the local file system (also for
> developer testing), but for production deployment, I could as easily
> configure it to use Amazon S3 (1) and the related CDN feature CloudFront
> (2).
> (1) http://aws.amazon.com/s3/ > (2) http://aws.amazon.com/cloudfront/
> Supporting GridFS is done by implementing a new storage provider for the
> chosen cloud library.
i agree, that clicking a AWS or whateva might be simple as hell as
well, but would you buy an external storage for your private use (like
sharing naked pictures with your GF/wife)?
we will keep cloud storage in mind, but i'm pretty sure it wont be
part of the first code
