serving assets directly from couchdb

[Matt Aimonetti]

Here is a great post explaining out to safely serve attachments directly from couchdb using nginx.

http://vermicel.li/blog/2009/06/16/efficiently-serving-couchdb-attachments.html?dsq=12023120

See my comment to see how to do the same using Merb's awesome router. (Same feature should be available in Rails3)
Just FYI, we benchmarked the Merb router and its as fast as serving static files using Apache.

So, in other words, if you need an advanced authentication/logging system for couchdb, it's really easy to do using nginx, merb (use merb-flat, the one file app version) and thin (should be better for this kind of stuff).

- Matt

[scottmotte]

Matt, why not just setup a proxy in nginx? I don't see the advantage
to the above solution. Can you clarify? Is it because it uses
send_data and so you can serve up blobs/binary too?

Here's what I've done:
1. secured couchdb by only making it available locally on server (I
think this is the default actually). This makes it impossible for the
user to type in http://website.com:5984/_utils etc.

2. created an nginx proxy to point anything with /assets to the built-
in couchdb http file serving.
i.e.
http://staging.twinstang.com/assets/twinstang_staging/58dddf4b58ecd5f9f9bded0a31fd3297/thumb.png
becomes
http://staging.twinstang.com:5984/twinstang_staging/58dddf4b58ecd5f9f9bded0a31fd3297/thumb.png

# this is a proxy for couchdb using nginx for files
# it routes /assets/ to the appropriate couchdb location
location /assets {
rewrite /assets/(.*) /$1 break;
proxy_pass http://127.0.0.1:5984;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

}




On Jul 1, 11:29 pm, Matt Aimonetti <mattaimone...@gmail.com> wrote:
> Here is a great post explaining out to safely serve attachments directly
> from couchdb using nginx.
>

> http://vermicel.li/blog/2009/06/16/efficiently-serving-couchdb-attach...

[Matt Aimonetti]

The solution I shared doesn't use send_data. Here is a quick review of what's being done in my merb app:

Router:

  # This is the code handling couchdb attachment

  # all urls must be prepended by /assets/

  # the router can run some auth/logging etc...

  # and instruct nginx to serve the file directly from couchdb

  #

  match("/assets/:db/:doc_id/:filename(.:ext)").defer_to do |request, params|

    filename  = CGI.escape "#{params[:filename]}#{params[:ext] ? ".#{params[:ext]}" : ''}"

    doc_uri   = "#{params[:db]}/#{params[:doc_id]}/#{filename}"

    case Merb.environment

    when /production|staging/

      # TODO protect access to some DBs + auth access (check session, HTTP query, auth etc..)

      header = {}

      header['Content-Type'] = 'application/octet-stream'  # We could also load the doc stub to get the real content-type

      header['Content-Disposition'] = "inline; filename=#{filename}"

      header['X-Accel-Redirect'] = "/couch/#{doc_uri}"

      [200, header, ""]

    else

      redirect "#{DB.uri}/#{doc_uri}"

    end

  end 

CouchDB is setup as follows:

        location /couch/ {

            internal;

            proxy_set_header  X-Real-IP  $remote_addr;

            proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;

            proxy_pass http://localhost:5984/;

        } 

As you see, everything is transparent in the app itself. In the case of your proxy, you are basically opening the entire DB to anyone. You could probably start by limiting to only GET access but you won't have any control over who can see what document. Our previous solution was limiting access using the same type of rewrite you did but with db limitation and file format restrictions. The problem is that you still can't easily limit or log accesses to the assets. You also have another major issue: such a setup won't work in a development/test mode if you don't use nginx.

Basically, using Merb's router + nginx you get full control over who can access what and how. Plus logging capabilities etc.. all that, almost for free since the router is super fast and you are not dispatching any requests.

I hope it helps,

- Matt

[scottmotte]

That does help - especially with your latest blog post (http://
merbist.com/2009/07/13/ruby-authentication-of-couchdb-requests/).
Thanks Matt.

>             proxy_passhttp://localhost:5984/;

> >http://staging.twinstang.com/assets/twinstang_staging/58dddf4b58ecd5f...
> >   becomes
>
> >http://staging.twinstang.com:5984/twinstang_staging/58dddf4b58ecd5f9f...

>
> >        # this is a proxy for couchdb using nginx for files
> >        # it routes /assets/ to the appropriate couchdb location
> >        location /assets {
> >                rewrite /assets/(.*) /$1 break;

> >                proxy_passhttp://127.0.0.1:5984;