pushing views

[Matt Aimonetti]

Did anyone find a good solution to avoid pushing a new revision of the queue every time they start their server app?

- Matt

[Nicholas Orr]

What about this?

Default behaviour as it is right now.
Then able to set some sort of variable called CouchRest::Mixins::DesignDoc.auto_refresh(false)

Now refreshing design views is effectively turned off, and should get errors if design views have changed in code but not on the running couchdb instance server.

Now we need some sort of manual callable CouchRest::Mixins::DesignDoc.make_fresh - this would go in some sort of rake task or step as part of deploying.

However having typed the above - this sounds complicated and not very relaxing....

(having to remember to refresh views is not fun)

So ideally what does sound relaxing is for the view to be compared in code with that of the couchdb instance. Let ruby do an obj compare DesignView.map == Model.map ?

Maybe add in a callable method that looks for all the views and it can say weather all the views in the code match the couchdb instance?

This way one could abort starting a production application as pushing the update will modify the views thus having to wait for re-indexing.

At the end of the day aren't we trying to prevent re-indexing of huge chunks of data for no reason, however if there is a real need to reindex, have the opportunity to figure out how best to handle it.

Those are my thoughts,

Nick

2009/11/19 Matt Aimonetti <mattai...@gmail.com>

Did anyone find a good solution to avoid pushing a new revision of the queue every time they start their server app?

- Matt

--

You received this message because you are subscribed to the Google Groups "CouchRest" group.
To post to this group, send email to couc...@googlegroups.com.
To unsubscribe from this group, send email to couchrest+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/couchrest?hl=.

[Hedley Robertson]

I think this is a good solution, and a better approach to a rather hacky (but similar) approach that I took.  I am using a global constant that I set via then ENV hash... passing environment var on my merb command line (like I said, hacky).  A bit of monkey patching of the relevant methods that refresh the views makes couchrest look for this constant and skip the push if set true.

So calling a script that leverages this might look like:

DEV_ENV=production SKIP_DESIGN_DOCS=true /usr/local/bin/ruby services/tweet_puller.rb

I kinda like the ability to set via command line... but the rest of it is somewhat hacked together.  Having an API method such as CouchRest::Mixins::DesignDoc.auto_refresh(false) would be preferable.

> (having to remember to refresh views is not fun)

Yeah, but we are already used to migrations in production RDBMS, right?   I don't mind a manual refresh, things like this are part of a normal production prop anyway.

Hedley

[RicRoberts]

I'd be perfectly happy with having to do a 'manual' refresh for
production mode. As Headly says, it's how you deal with migrations
for MySQL or whatever.

Speaking of migrations... How do people deal with the situation when
you've changed your Ruby code to expect a different structure in your
couchdb documents? This doesn't happen very often, but when it does
it's a bit of a pain. i.e. how do you manage running of scripts
against the database which massage the data into the new format?

So far, I've just been creating a script, testing it locally, and then
running it on the server straight after deploying the new code (far
from ideal). Maybe a migration-like approach (i.e. a folder of
numbered scripts) might work well here?

Any thoughts welcome.
Ric


On Nov 19, 9:51 pm, Hedley Robertson <hedley.robert...@gmail.com>
wrote:

> I think this is a good solution, and a better approach to a rather
> hacky (but similar) approach that I took.  I am using a global constant that
> I set via then ENV hash... passing environment var on my merb command line
> (like I said, hacky).  A bit of monkey patching of the relevant methods
> that refresh the views makes couchrest look for this constant and skip the
> push if set true.
>
> So calling a script that leverages this might look like:
>
> DEV_ENV=production SKIP_DESIGN_DOCS=true /usr/local/bin/ruby
> services/tweet_puller.rb
>
> I kinda like the ability to set via command line... but the rest of it is
> somewhat hacked together.  Having an API method such as
> CouchRest::Mixins::DesignDoc.auto_refresh(false) would be preferable.
>
> > (having to remember to refresh views is not fun)
>
> Yeah, but we are already used to migrations in production RDBMS, right?   I
> don't mind a manual refresh, things like this are part of a normal
> production prop anyway.
>
> Hedley
>

> > 2009/11/19 Matt Aimonetti <mattaimone...@gmail.com>

>
> > Did anyone find a good solution to avoid pushing a new revision of the
> >> queue every time they start their server app?
>
> >> - Matt
>
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "CouchRest" group.
> >> To post to this group, send email to couc...@googlegroups.com.
> >> To unsubscribe from this group, send email to

> >> couchrest+...@googlegroups.com<couchrest%2Bunsu...@googlegroups.com>
> >> .

> >> For more options, visit this group at
> >>http://groups.google.com/group/couchrest?hl=.
>
> >  --
> > You received this message because you are subscribed to the Google Groups
> > "CouchRest" group.
> > To post to this group, send email to couc...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > couchrest+...@googlegroups.com<couchrest%2Bunsu...@googlegroups.com>
> > .

[will leinweber]

I've been playing around with the idea of managing those sorts of
migrations, but I don't have anything quite ready yet.

I have a view that determines a signature based on the keys of a
document. Then I can write scripts that transform dove with the old
signature to what I want. This has the advantage of not needing
numbers for the migrations, which even when rails switched to
datestamps, is crummy.

_will

On Nov 26, 2009, at 11:03 AM, RicRoberts <rici...@googlemail.com>
wrote:

> For more options, visit this group at http://groups.google.com/group/couchrest?hl=en
> .
>
>

[RicRoberts]

Thanks Will. That sounds like a good idea.

Have you got any code I could look at? It doesn't matter if it's not
production quality... just something to start from would be great. :)

On Nov 26, 6:28 pm, will leinweber <will.leinwe...@gmail.com> wrote:
> I've been playing around with the idea of managing those sorts of
> migrations, but I don't have anything quite ready yet.
>
> I have a view that determines a signature based on the keys of a
> document. Then I can write scripts that transform dove with the old
> signature to what I want. This has the advantage of not needing
> numbers for the migrations, which even when rails switched to
> datestamps, is crummy.
>
> _will
>

> On Nov 26, 2009, at 11:03 AM, RicRoberts <ricisb...@googlemail.com>

[will leinweber]

All of this could be crazy. I know big advertisement of couch is the lack of a schema and to relax, but I /do/ think that keeping tabs on some sort of document version is a real issue. I've been using couch for just over a year now, but maybe relational dbs are still to engrained.

Like I said, I haven't put this schema versioning in to production, so what I did wasn't in our git repo. I poked around on my laptop for a while, but I couldn't find what I had. I could have sworn that it'd be in ~/code/sandbox where I keep most of my experiments, but couldn't find anything.

I can describe what I had done though, and maybe we can work together and build something.

What I was doing is in a map, take all the keys of a document, stick them together and run that through md5, then emit [md5, doc.id] as the key.

Then you can run the view to get all the documents of a given md5 value, make the changes you need to each, and verify you got those changes right, since their md5s would have changed.

It ran into problems when the documents had arbitrarily nested parts, though. So I cut it off at the top level keys, but that's not very robust either.

I was thinking, maybe if instead of generating the "schema version" id just by the keys, if it was just put on each document as an extra field. You could have a yaml file that had all of your extended document classes and their current version md5/sha. Then have that generated when rails starts, and base it by reflecting at all the properties each class has. Actually, you wouldn't need to write it to the yaml file, just have that as an extra key/value on all documents.

Sorry for rambling on… just brainstorming a bit.

-will 

To unsubscribe from this group, send email to couchrest+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/couchrest?hl=en.

[RicRoberts]

I'm still pondering the 'migration' issue, but I thought I'd share my
current hack to prevent existing design docs from being updated in
production. (I'm not yet comfortable enough with the CouchRest code to
make this into a proper patch, but maybe this will be of use to
someone).

* In my web app, I set :env on the thread to a symbol representing the
current environment.
* I have an intermediate class between ExtendedDocument and my models,
in which I overwrite the update_design_doc class method with the code
below. (i.e. you end up with a hierarchy of MyModel <
IntermediateClass < CouchRest::ExtendedDocument).
* When I deploy changes to the models, as part of my capistrano tasks,
I run a rake task which updates the design docs.

# Writes out a design_doc to a given database, returning the
# updated design doc
def self.update_design_doc(design_doc, db = database)
saved = db.get(design_doc['_id']) rescue nil
if saved
# If the design doc is already there, don't update it in prod mode.
We'll do it manually.
unless Thread.current[:env] && Thread.current[:env] == :production
design_doc['views'].each do |name, view|
saved['views'][name] = view
end
db.save_doc(saved)
end
saved
else
design_doc.database = db
design_doc.save
design_doc
end
end

-Ric

> > > > .
> > > > For more options, visit this group athttp://
> > groups.google.com/group/couchrest?hl=en
> > > > .
>
> > --
>
> > You received this message because you are subscribed to the Google Groups
> > "CouchRest" group.
> > To post to this group, send email to couc...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > couchrest+...@googlegroups.com<couchrest%2Bunsu...@googlegroups.com>

> > .
> > For more options, visit this group at

> >http://groups.google.com/group/couchrest?hl=en.

[Matt Aimonetti]

it's not a bad solution, the real problem is that couchrest doesn't create the same design doc twice in a row and therefore the map function is run again. Maybe the easiest solution would be to fix that or to use a md5 hash to compare the local design doc and the remote one. (could be done in a separate thread)

- Matt

To unsubscribe from this group, send email to couchrest+...@googlegroups.com.