Re: CouchRest needlessly updates view?
Rich Morin
At 17:50 -0800 2/2/09, Rich Morin wrote:
> The act of looking up a document in CouchRest is updating
> _rev on the view that it uses. This seems inefficient.
>
> * Is there a reason for this behavior?
>
> * If not, is there a way to prevent it?
>
> I have a test script (http://pastie.org/377884) which looks
> up a document by its "p_uuidc" field. Using Futon, I can
> confirm that _rev is changing in the _design document when
> I run the script.
>
> Alternatively, when I use a Rack-based forwarding/tracing
> script, I see a PUT for the _design document, despite the
> fact that I'm not asking for any such thing.
>
> Wazzup?
At 18:14 -0800 2/2/09, Chris Anderson wrote:
> On Mon, Feb 2, 2009 at 5:50 PM, Rich Morin <r...@cfcl.com> wrote:
>> * Is there a reason for this behavior?
>
> No. It's ready to be optimized away.
I'm delighted to hear that! I was afraid it was some sort
of low-level design issue.
> There's a mailing list for CouchRest here:
>
> http://groups.google.com/group/couchrest
Cool; I hadn't know about that. Hi, folks!
> I hope that the design doc is only being updated once per
> script load - eg running 2 view requests should only update
> the design doc once.
Using my tracing app, I can confirm that only a single PUT is
done per run of the test script.
> Design docs should not need to be PUT for each definition
> of the model class. It should be simple to make a fix for
> getting the design doc to see if it matches the ruby view
> definition.
>
> Do you feel up to making a patch? Running `rake` in the
> CouchRest source directory should give you a passing test
> suite, so patching it is just a matter of fixing the bug
> without breaking any of the other specs.
I'll pull down the code and take a look, but I don't have
much confidence in my ability to fix it. So, if anyone can
offer a hint (or simply wants to jump in), please do so...
-r
--
http://www.cfcl.com/rdm Rich Morin
http://www.cfcl.com/rdm/resume r...@cfcl.com
http://www.cfcl.com/rdm/weblog +1 650-873-7841
Technical editing and writing, programming, and web development
Rich Morin
> Design docs should not need to be PUT for each definition
> of the model class. It should be simple to make a fix for
> getting the design doc to see if it matches the ruby view
> definition.
I don't know if this would make a noticeable difference in
performance for most users (is a GET cheaper than a PUT?).
Philosophically, however, it seems wrong to update a _rev
when no writing has been requested by the client program.
In any case, does anyone have any pointers to offer for
where this fix should go in the code?
The suggested fix might also help in my (quite unusual)
situation. I'm running untrusted code which may try to do
Bad Things to the database. If I can simply turn off all
PUT requests (including ones that update views), that will
keep a lot of nastiness from happening.
The way it might play out is that the untrusted code would
try a search, specifying a named view. If the view code
differs from that in the database, CouchRest would try to
do a PUT. I'd kill that, and the untrusted code would die.
Is this anything like a reasonable approach?
Chris Anderson
>
> At 18:14 -0800 2/2/09, Chris Anderson wrote:
>> Design docs should not need to be PUT for each definition
>> of the model class. It should be simple to make a fix for
>> getting the design doc to see if it matches the ruby view
>> definition.
>
> I don't know if this would make a noticeable difference in
> performance for most users (is a GET cheaper than a PUT?).
> Philosophically, however, it seems wrong to update a _rev
> when no writing has been requested by the client program.
>
> In any case, does anyone have any pointers to offer for
> where this fix should go in the code?
>
You'd probably want to start by adding some puts traces to
CR::Model#refresh_design_doc and the code that calls it.
That's likely where you can avoid extra saves if you don't need them.
I think this problem is unrelated to untrusted views problem, as the
CouchRest views are generated from Ruby code. The changes you're
talking about making shouldn't have any effect other than optimizing
database chatter.
Unless there will be Ruby code in your system that you can't trust,
you'd probably do better to think of the trust boundary as outside of
your application server.
--
Chris Anderson
http://jchris.mfdz.com
rich_morin
> ...
> you'd probably do better to think of the trust boundary as outside of
> your application server.
Ruby
code to be run, access the database, etc. Basically, it's a wiki
whose
extension language is eRuby. So, when a page is shown, the Ruby is
run, generating content, etc. For more information, see:
http://cfcl.com/twiki/bin/view/Projects/Ontiki/WebHome
http://cfcl.com/twiki/bin/view/Projects/Ontiki/Examples
candlerb
> CouchRest views are generated from Ruby code.
using couchapp), and only query them from the application itself.
couchapp then becomes the equivalent of a "migration" in a Rails app.
I notice with couchrest, even if you use Foo.view :bar, it still tries
to refresh the design doc. So it looks like I have to use db.view 'foo/
bar' to query a view without touching the design doc. But you then
need to map { } the results to wrap them in your model object.
What I'd like is to be able to bind a model to a pre-existing view,
perhaps like this:
class Foo
use_view :bar, "foo/bar"
use_view :bar # defaults to class/view
end
...
Foo.bar :params=>....
This would be like db.view 'foo/bar', wrapping the results in a model
as fetch_view_with_docs does (but not making any attempt to refresh
the design doc as fetch_view does)
Does this make sense? Or is it going completely against the grain of
CouchRest::Model which assumes it should be managing the design docs
for you?
Regards,
Brian.
candlerb
class CouchRest::Model
class << self
def use_view(method, viewname=nil)
viewname ||= "#{self.name.snake_case}/#{method}"
class_eval <<EOS
def self.#{method}(params={}, &block)
if params.delete(:raw) || params[:reduce]
database.view(#{viewname.to_s.inspect}, params, &block)
else
view = database.view(#{viewname.to_s.inspect}, params.merge
(:include_docs=>true), &block)
view['rows'] && view['rows'].collect{|r|new r['doc']}
end
end
EOS
end
end
end
This lets you do
class Foo
use_view :all
end
to override the default 'all' behaviour. But to make use of existing
views created by CouchRest you need to know the 'slug' it has put in
the design doc name:
use_view :all, 'Foo-d5694b15c33fd69d4bf14f76fb15ef65/all'
or
use_view :all, Foo.instance_eval { "#{design_doc_slug}/all" }
On a completely separate point: I am playing with CouchRest::Model
because I found too many bugs in couch-potato. However both of them
have hard-coded what attribute name they use for the class name
("ruby_class" versus "couchrest-type"). Wouldn't it be nice if this
were a tunable option, to make it possible to swap easily from one to
the other?
Regards,
Brian.
Chris Anderson
I think CouchRest::Model has an option to define the type field.
the use_view method looks intriguing. is it available on github? with
some docs and a test case I'd be happy to add it.
candlerb
$ find lib -name '*.rb' | xargs grep couchrest-type | grep -v ': *#'
lib/couchrest/mixins/design_doc.rb: if (doc
['couchrest-type'] == '#{self.to_s}') {
lib/couchrest/mixins/extended_views.rb: opts[:guards].push
"(doc['couchrest-type'] == '#{self.to_s}')"
lib/couchrest/core/model.rb: self['couchrest-type'] =
self.class.to_s
lib/couchrest/core/model.rb: opts[:guards].push "(doc
['couchrest-type'] == '#{self.to_s}')"
lib/couchrest/core/model.rb: if (doc['couchrest-type']
== '#{self.to_s}') {
Chris Anderson
>
>> I think CouchRest::Model has an option to define the type field.
>
> At the moment it's hard-coded all over the place...
>
Darn, I think I lost a patch from someone back along the way. I
totally support making couchrest-type configurable.
candlerb
automatic building of design documents with slugs. Maybe this feature
would be useful if you didn't define *any* views within couchrest; but
as soon as you do
view_by :foo, ...
then couchrest will start creating its own design documents anyway.
Perhaps I should go with the couchrest flow and use
view :my_view, :map=>..., :reduce=>...
so that couchrest would incorporate my custom view for me. I just
don't want couchrest to keep touching the design document.
In fact, I'm not sure why it has to. As far as I can see, the slug is
created as an MD5 over all your map and reduce function bodies. So, as
soon as you change any view definition, a new slug will be used, and
hence a new design document will be created.
So at the time when the design_doc_slug is calculated, you can GET the
design doc and compare all the functions, and set it to be fresh if
they match. But probably only refresh_design_doc needs to be changed
then. I'll have a look at it (which means putting some branches in my
repo :-)
Mark....@gmail.com
> Darn, I think I lost a patch from someone back along the way. I
> totally support making couchrest-type configurable.
http://jchris.lighthouseapp.com/projects/17807/tickets/22-lets-use-couchrest_type#ticket-22-6
+1 for something similar. This would be very handy!
Cheers,
Mark