mendeley provider behaving erratically

[Jason Priem]

I'm getting all kinds of stuff out of the Mendeley provider. When I run the default Dryad sample, I get this out of the backend stdout: 

120515 205025    ERROR backend - exception for item(e7cfbaf6b5d747eb958b92d6c613638b): '' (client_server_error on Provider(mendeley) aliases)

120515 205025    ERROR backend - exception for item(c5ee87b378d245368637371d81402937): 'groups' (unknown_error on Provider(mendeley) metrics)

120515 205025    ERROR backend - Error processing item for provider Provider(mendeley) metrics: Unknown exception 'groups', aborting

120515 205025    ERROR backend - exception for item(5698ec43affe45b8b98d0bb1f6d3645a): '' (client_server_error on Provider(mendeley) aliases)

120515 205026    ERROR backend - exception for item(d4d9dfe3e7ea4168bb977d1125a03dec): '' (client_server_error on Provider(mendeley) aliases)

120515 205026    ERROR backend - exception for item(f831b5752f68479c83e1b6b628ea3f41): '' (client_server_error on Provider(mendeley) aliases)

I'm not clear what's causing that error (there's actually much more, I'm just snipping out a sample) Note the one line that complains about an unknown "groups" exception...that's consistently the only one like that.  Some other odd behavior is going along with this:

1. At first I was getting Mendeley metrics for select dryad dois...no more. Now Mendeley is saving as "<ts>: null" metric value in the db. Do not know what changed...tested the Mendeley API call and it works fine, so not out of calls.

2. The Dryad metrics that do work get populated much more slowly than if I disable the Mendeley provider. When the Mendeley provider is active, Dryad seems to happen one-at-a-time...Wikipedia, too. Not sure I understand why this is; I get that Mendeley threads should be taking a while, since they're busy failing and retrying. But why should that slow the Dryad or Wikipedia threads down? I'm thinking that it is that it's related to item-level locking, and the failing threads keep the locks on for longer?

--
Jason Priem
UNC Royster Scholar
School of Information and Library Science
University of North Carolina at Chapel Hill

[Heather Piwowar]

Cool I'll have a look tonight

--
You received this message because you are subscribed to the Google Groups "total-impact-dev" group.
To post to this group, send email to total-im...@googlegroups.com.
To unsubscribe from this group, send email to total-impact-d...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/total-impact-dev?hl=en.

[Kevin Campbell]

On Wed, May 16, 2012 at 2:06 AM, Jason Priem <j...@jasonpriem.org> wrote:

I'm getting all kinds of stuff out of the Mendeley provider. When I run the default Dryad sample, I get this out of the backend stdout: 

120515 205025    ERROR backend - exception for item(e7cfbaf6b5d747eb958b92d6c613638b): '' (client_server_error on Provider(mendeley) aliases)

120515 205025    ERROR backend - exception for item(c5ee87b378d245368637371d81402937): 'groups' (unknown_error on Provider(mendeley) metrics)

120515 205025    ERROR backend - Error processing item for provider Provider(mendeley) metrics: Unknown exception 'groups', aborting

120515 205025    ERROR backend - exception for item(5698ec43affe45b8b98d0bb1f6d3645a): '' (client_server_error on Provider(mendeley) aliases)

120515 205026    ERROR backend - exception for item(d4d9dfe3e7ea4168bb977d1125a03dec): '' (client_server_error on Provider(mendeley) aliases)

120515 205026    ERROR backend - exception for item(f831b5752f68479c83e1b6b628ea3f41): '' (client_server_error on Provider(mendeley) aliases)

I'm not clear what's causing that error (there's actually much more, I'm just snipping out a sample) Note the one line that complains about an unknown "groups" exception...that's consistently the only one like that.  Some other odd behavior is going along with this:

Jason,

Can you grep logs/total-impact.log for those items? You should be able to get traces of what's going on for processing each.
 

1. At first I was getting Mendeley metrics for select dryad dois...no more. Now Mendeley is saving as "<ts>: null" metric value in the db. Do not know what changed...tested the Mendeley API call and it works fine, so not out of calls.

2. The Dryad metrics that do work get populated much more slowly than if I disable the Mendeley provider. When the Mendeley provider is active, Dryad seems to happen one-at-a-time...Wikipedia, too. Not sure I understand why this is; I get that Mendeley threads should be taking a while, since they're busy failing and retrying. But why should that slow the Dryad or Wikipedia threads down? I'm thinking that it is that it's related to item-level locking, and the failing threads keep the locks on for longer?

The item level locking shouldn't affect anything other than writes to the database on the same item. From the errors you've posted it looks more like the provider code is raising exceptions and causing retries.

K

[Heather Piwowar]

Hey Kev and Jason, 

I've tracked down the Mendeley bugs and have fixes.  Have to do a bit of test-fixing/writing before I check in, but zonked tonight.  Will commit and push tomorrow.

h

[Jason Priem]

Great, thanks!

--
You received this message because you are subscribed to the Google Groups "total-impact-dev" group.
To post to this group, send email to total-im...@googlegroups.com.
To unsubscribe from this group, send email to total-impact-d...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/total-impact-dev?hl=en.