Re: [ts] Rolling deploys

[Pat Allan]

Hi Michael

Are you using ts:rebuild? You probably don't need to do that - indeed, treat the ts:rebuild task like db:migrate: it's only necessary when you add/remove indices or make changes to their structure. There's no need to restart Sphinx on every deploy.

Cheers

--
Pat

On 07/06/2012, at 9:39 PM, Michael Guterl wrote:

> Currently when I deploy, searchd is restarted which causes my application to raise errors if someone is trying to search or if a resque process is using searchd. I'm using the capistrano tasks included in thinking-sphinx and I'm wondering if there's any way to achieve a rolling (or zero-downtime) deploy when using these tasks?
>
> Am I doing something wrong?
>
> Thanks,
> Michael Guterl
>
> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.
> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/yXAAtZ0hRkQJ.
> To post to this group, send email to thinkin...@googlegroups.com.
> To unsubscribe from this group, send email to thinking-sphi...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.

[Michael Guterl]

Nope. I was just following the instructions detailed here:

http://freelancing-god.github.com/ts/en/deployment.html

I was thinking I only need to symlink, but I probably don't even need to do that seeing that the sphinx.yml points to the shared path. Do you agree?

Thanks,

Michael Guterl

On Thursday, June 7, 2012 3:55:23 PM UTC-4, Pat Allan wrote:

Hi Michael

Are you using ts:rebuild? You probably don't need to do that - indeed, treat the ts:rebuild task like db:migrate: it's only necessary when you add/remove indices or make changes to their structure. There's no need to restart Sphinx on every deploy.

Cheers

--
Pat

On 07/06/2012, at 9:39 PM, Michael Guterl wrote:

> Currently when I deploy, searchd is restarted which causes my application to raise errors if someone is trying to search or if a resque process is using searchd. I'm using the capistrano tasks included in thinking-sphinx and I'm wondering if there's any way to achieve a rolling (or zero-downtime) deploy when using these tasks?
>
> Am I doing something wrong?
>
> Thanks,
> Michael Guterl
>
> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.
> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/yXAAtZ0hRkQJ.

> To post to this group, send email to thinking-sphinx@googlegroups.com.
> To unsubscribe from this group, send email to thinking-sphinx+unsubscribe@googlegroups.com.

[Pat Allan]

Yeah, if you keep things to the shared paths, then you should be fine. Those docs certainly need some work (most of my deployments are to Heroku these days, so I'm a bit out of practice with Capistrano).

--
Pat

On 08/06/2012, at 5:50 PM, Michael Guterl wrote:

> Nope. I was just following the instructions detailed here:
>
> http://freelancing-god.github.com/ts/en/deployment.html
>
> I was thinking I only need to symlink, but I probably don't even need to do that seeing that the sphinx.yml points to the shared path. Do you agree?
>
> Thanks,
> Michael Guterl
>
> On Thursday, June 7, 2012 3:55:23 PM UTC-4, Pat Allan wrote:
> Hi Michael
> Are you using ts:rebuild? You probably don't need to do that - indeed, treat the ts:rebuild task like db:migrate: it's only necessary when you add/remove indices or make changes to their structure. There's no need to restart Sphinx on every deploy.
>
> Cheers
>
> --
> Pat
>
> On 07/06/2012, at 9:39 PM, Michael Guterl wrote:
>
> > Currently when I deploy, searchd is restarted which causes my application to raise errors if someone is trying to search or if a resque process is using searchd. I'm using the capistrano tasks included in thinking-sphinx and I'm wondering if there's any way to achieve a rolling (or zero-downtime) deploy when using these tasks?
> >
> > Am I doing something wrong?
> >
> > Thanks,
> > Michael Guterl
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.
> > To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/yXAAtZ0hRkQJ.

> > To post to this group, send email to thinkin...@googlegroups.com.
> > To unsubscribe from this group, send email to thinking-sphi...@googlegroups.com.

> > For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
>
>
>
>
>
>
>
>
>
>

> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/xlq7lE__VGcJ.
> To post to this group, send email to thinkin...@googlegroups.com.
> To unsubscribe from this group, send email to thinking-sphi...@googlegroups.com.

[Ngan]

Is there a reliable, programatic way of determining if your indexes have changed?  E.g. a SHA of the indexes?

On Friday, June 8, 2012 8:52:32 AM UTC-7, Pat Allan wrote:

Yeah, if you keep things to the shared paths, then you should be fine. Those docs certainly need some work (most of my deployments are to Heroku these days, so I'm a bit out of practice with Capistrano).

--
Pat

On 08/06/2012, at 5:50 PM, Michael Guterl wrote:

> Nope. I was just following the instructions detailed here:
>
> http://freelancing-god.github.com/ts/en/deployment.html
>
> I was thinking I only need to symlink, but I probably don't even need to do that seeing that the sphinx.yml points to the shared path. Do you agree?
>
> Thanks,
> Michael Guterl
>
> On Thursday, June 7, 2012 3:55:23 PM UTC-4, Pat Allan wrote:
> Hi Michael
> Are you using ts:rebuild? You probably don't need to do that - indeed, treat the ts:rebuild task like db:migrate: it's only necessary when you add/remove indices or make changes to their structure. There's no need to restart Sphinx on every deploy.
>
> Cheers
>
> --
> Pat
>
> On 07/06/2012, at 9:39 PM, Michael Guterl wrote:
>
> > Currently when I deploy, searchd is restarted which causes my application to raise errors if someone is trying to search or if a resque process is using searchd. I'm using the capistrano tasks included in thinking-sphinx and I'm wondering if there's any way to achieve a rolling (or zero-downtime) deploy when using these tasks?
> >
> > Am I doing something wrong?
> >
> > Thanks,
> > Michael Guterl
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.
> > To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/yXAAtZ0hRkQJ.

> > To post to this group, send email to thinking-sphinx@googlegroups.com.
> > To unsubscribe from this group, send email to thinking-sphinx+unsubscribe@googlegroups.com.

> > For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
>
>
>
>
>
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.
> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/xlq7lE__VGcJ.

> To post to this group, send email to thinking-sphinx@googlegroups.com.
> To unsubscribe from this group, send email to thinking-sphinx+unsubscribe@googlegroups.com.

[Pat Allan]

Well, you could SHA or MD5 the files to easily note whether they've changed - but that won't tell you whether it's just the data that's changed, or the structure of them - so it's not useful as an indicator of whether you need to run ts:rebuild as part of a deploy, if that's what you were thinking?

--
Pat

> > > To post to this group, send email to thinkin...@googlegroups.com.
> > > To unsubscribe from this group, send email to thinking-sphi...@googlegroups.com.

> > > For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.
> > To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/xlq7lE__VGcJ.

> > To post to this group, send email to thinkin...@googlegroups.com.
> > To unsubscribe from this group, send email to thinking-sphi...@googlegroups.com.

> > For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
>
>
>
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/50QMvBurEJcJ.
> To post to this group, send email to thinkin...@googlegroups.com.
> To unsubscribe from this group, send email to thinking-sphi...@googlegroups.com.

[Ngan Pham]

As you suggested that it's not necessary to restart TS every deploy except when making index changes.  I just wanted to automate the process so a developer would have to remember to restart TS or not.  Would be nice for the deploy process to see that the indexes have changed and restart TS automatically.

[Pat Allan]

How would you do that from a database migrations perspective?

--
Pat

[Ngan Pham]

I think we're misunderstanding each other…

Currently when we deploy our application, we always do a TS restart.  If we were to remove the TS restart step, a developer would have to know that a TS index was changed in the code and restart TS within the deployment procedure manually.  I would be great if the deployment process can figure out if TS needs to be restarted or not basic on whether or not indexes were changed since the last deployment.  I was thinking that a way of doing this would be to SHA the all the indexes that TS has and compare them for each deployment.  If the SHA has changed, that means the indexes have changed and a TS restart is required.

[Pat Allan]

But that won't work, because what you need to compare is the code in the define_index blocks. If those index definitions change (or you add or remove an index on a model), then you need to run ts:rebuild. But if there's been no change to the definitions, then you shouldn't need to restart or rebuild.

--
Pat

[Ngan Pham]

Right.  My thinking was that if TS is the one responsible for aggregating that code, it can generate a SHA of all those index definitions, no?

class User < AR

  define_index do

    indexes :username

    ….

  end

end

The define_index interpreted and built with TS right? So I would assume TS can loop through all its defined indexes and hash it somehow…

I'm probably way off so sorry if I'm just talking nonsense :-)

Just an idea…

[Pat Allan]

Ah… so then, you'd want to SHA of the configuration file - that should do the trick, right?

--
Pat

[Ngan Pham]

Ah yes. perfect.  thanks Pat!

You rock!