Umlaut sfx service fails with timeout

[Dale Poulter]

Good morning,

 We upgraded to 3.3 last week and everything was working great until this morning when all requests have started to fail.  When I look at the service_errors it looks like timeout errors but we have not made any changes in the past week after the upgrade.  Any suggestions? 

[Scot Thomas Dalton]

Hi Dale,

I'm seeing the same thing in our QA environment. No solution yet, just commiseration.

I'll keep you posted.

Thanks,

Scot

On Mon, Jul 21, 2014 at 9:28 AM, Dale Poulter <dale.p...@vanderbilt.edu> wrote:

Good morning,

 We upgraded to 3.3 last week and everything was working great until this morning when all requests have started to fail.  When I look at the service_errors it looks like timeout errors but we have not made any changes in the past week after the upgrade.  Any suggestions? 

--
You received this message because you are subscribed to the Google Groups "Umlaut" group.
To unsubscribe from this group and stop receiving emails from it, send an email to umlaut-softwa...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Scot Dalton
New York University

[Poulter, Dale]

Great to hear that it is not just a local issue.

[Poulter, Dale]

Scot,

 

Have you tried changing sfx_timeout?  Based on the sfx.rb the default timeout is set to 8 s which may be causing the issue.  I made this change earlier this morning and have not seen the error again – yet.  I increased the value to 60 s which seems high but will work for testing. 

 

From: Scot Thomas Dalton [mailto:st...@nyu.edu]
Sent: Monday, July 21, 2014 9:02 AM
To: Poulter, Dale
Cc: umlaut-...@googlegroups.com
Subject: [External] Re: [umlaut] Umlaut sfx service fails with timeout

 

Hi Dale,

[Scot Thomas Dalton]

Dale,

Thanks for the tip! We're actually seeing it for other services, not just SFX.  Still investigating.

-Scot

[Jonathan Rochkind]

Hi Dale, I'm afraid I was away on summer vacation, just returned. Are
you still having this problem? Did you downgrade to an earlier version
of Umlaut in the meantime?

Jonathan

> --
> You received this message because you are subscribed to the Google
> Groups "Umlaut" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to umlaut-softwa...@googlegroups.com

> <mailto:umlaut-softwa...@googlegroups.com>.

[Jonathan Rochkind]

Doh, Scot too! Sorry about this guys. Have you figured out what was up
yet? I wonder what mistake I made!

Jonathan

On 7/21/14 10:01 AM, Scot Thomas Dalton wrote:
> Hi Dale,
>
> I'm seeing the same thing in our QA environment. No solution yet, just
> commiseration.
>
> I'll keep you posted.
>
> Thanks,
> Scot
>
>
> On Mon, Jul 21, 2014 at 9:28 AM, Dale Poulter

> <dale.p...@vanderbilt.edu <mailto:dale.p...@vanderbilt.edu>> wrote:
>
> Good morning,____
>
> __ __We upgraded to 3.3 last week and everything was working great

> until this morning when all requests have started to fail. When I
> look at the service_errors it looks like timeout errors but we have
> not made any changes in the past week after the upgrade. Any
> suggestions?
>

> ____

>
> --
> You received this message because you are subscribed to the Google
> Groups "Umlaut" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to umlaut-softwa...@googlegroups.com

> <mailto:umlaut-softwa...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> --
> Scot Dalton
> New York University
>

> --
> You received this message because you are subscribed to the Google
> Groups "Umlaut" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to umlaut-softwa...@googlegroups.com

> <mailto:umlaut-softwa...@googlegroups.com>.

[Poulter, Dale]

We are still having this issue but we were able to at least mask the issue by increasing the sfx_timeout to 120 so everything is ok right now. We will probably move back to the delivered menu shortly with a few customizations.


-Dale

[Jonathan Rochkind]

So something doesn't seem right there, because an Umlaut upgrade should
not cause your SFX to respond any slower than it did before! And your
SFX really ought not to take 120 seconds to respond -- are you actually
having to wait up to 2 minutes for your Umlaut page with SFX results to
display?

In addition to being a pretty terrible experience for your users, I
think that's also going to result in a pretty unstable app, if it's
blocking that long before returning results.

Scot, did you figure out anything else about this?

Dale, can you let me know what version of Umlaut you were using
previously, and what version you upgraded to? Maybe file a Github ticket
on this? If you give me your exact umlaut_services.yml file (and maybe
your Gemfile.lock too), I can maybe try to reproduce on my end.

Jonathan

[Scot Thomas Dalton]

I didn't figure anything out, but it did seem to resolve itself in our case. We deployed other changes at the same time (terrible, I know) and my theory was that as "cached" service responses got cleaned out the new ones did the right thing. But that doesn't explain Dale's issue.

Maybe the upgrade to the OpenURL gem caused an issue.[1][2] I don't see why that would be the case, but that seems to be what changed related to the SFX service adaptor.

Thanks,

Scot

[1] https://github.com/team-umlaut/umlaut/compare/v3.2.0...v3.3.0#diff-9ce563d06c3e200f34111e2f50f51c0aL29

[2] https://github.com/openurl/openurl/compare/v0.4.3...master

On Mon, Aug 18, 2014 at 10:18 AM, Jonathan Rochkind <roch...@jhu.edu> wrote:

So something doesn't seem right there, because an Umlaut upgrade should
not cause your SFX to respond any slower than it did before! And your SFX really ought not to take 120 seconds to respond -- are you actually having to wait up to 2 minutes for your Umlaut page with SFX results to display?

In addition to being a pretty terrible experience for your users, I think that's also going to result in a pretty unstable app, if it's blocking that long before returning results.

Scot, did you figure out anything else about this?

Dale, can you let me know what version of Umlaut you were using previously, and what version you upgraded to? Maybe file a Github ticket on this?  If you give me your exact umlaut_services.yml file (and maybe your Gemfile.lock too), I can maybe try to reproduce on my end.

Jonathan

On 8/18/14 9:35 AM, Poulter, Dale wrote:

We are still having this issue but we were able to at least mask the
issue by increasing the sfx_timeout to 120 so everything is ok right
now.  We will probably move back to the delivered menu shortly with a
few customizations.


-Dale


-----Original Message----- From: Jonathan Rochkind
[mailto:roch...@jhu.edu] Sent: Monday, August 18, 2014 8:30 AM To:

Poulter, Dale; umlaut-software@googlegroups.com Subject: Re: [umlaut]

Umlaut sfx service fails with timeout

Hi Dale, I'm afraid I was away on summer vacation, just returned. Are
you still having this problem?  Did you downgrade to an earlier
version of Umlaut in the meantime?

Jonathan


On 7/21/14 9:28 AM, Dale Poulter wrote:

Good morning,

We upgraded to 3.3 last week and everything was working great
until this morning when all requests have started to fail.  When I
look at the service_errors it looks like timeout errors but we have
not made any changes in the past week after the upgrade.  Any
suggestions?

-- You received this message because you are subscribed to the
Google Groups "Umlaut" group. To unsubscribe from this group and
stop receiving emails from it, send an email to

umlaut-software+unsubscribe@googlegroups.com
<mailto:umlaut-software+unsub...@googlegroups.com>. For more
options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Umlaut" group.

To unsubscribe from this group and stop receiving emails from it, send an email to umlaut-software+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.