Problem with listrecords on node01

[Jerome Grimmer]

Hey guys,

I’ve been looking into why we haven’t received any data from the LR since April 8^th, 2013.  I do a listrecords against node01.public.learningregistry.net and I see nothing after 2013-04-08T11:42:09.049822Z.  I find it hard to believe that nobody has published to the LR in over a month, so something must be going awry somewhere.

 

For https://node01.public.learningregistry.net/harvest/listrecords?from=2013-04-08T11:42:09Z&until=2013-04-08T11:42:10Z I get all kinds of data back from node01, ending with timestamp 2013-04-08T11:42_09.049822Z.  Using this URL, https://node01.public.learningregistry.net/harvest/listrecords?from=2013-04-08T11:43:00Z&until=2013-05-20, this is all I get back:

 

{

  "listrecords": [],

  "request": {

    "HTTP_request": "",

    "verb": "listrecords",

    "from": "2013-04-08T11:43",

    "until": "2013-05-20T00:00:00Z"

  },

  "OK": true,

  "responseDate": "2013-05-20T14:15:02.514428Z",

  "error": "",

  "resumption_token":"null"

}

 

Any idea what’s going on?

 

Jerome Grimmer

Southern Illinois University Carbondale

2450 Foundation Drive Suite 100

Springfield, IL

Phone: 217-786-3010 ext. 5857

Toll-free: 1-800-252-4822 ext. 5857

NOTE: My E-mail address has changed

jgri...@siuccwd.com

"If you think you're too small to make a difference, you've never spent a night with a mosquito." - An African Proverb

 

[Jim Klo]

Hi Jerome,

Walt and I just had a call and don't believe that there is a bug at this time. We believe that this is one of the 'eventual consistency' side effects that NoSQL DB's frequently encounter.  We think, as the issue you reported is no longer reproducible, that the /status (from which I presume you are using to track document counts) uses a different view than that of /harvest/listrecords) of which both use 'stale' results to ensure a response from the server with little latency (as views in couchdb are indexed upon request).

Additionally to note, granularity of the harvest services is to the second, however we store nanosecond.  It's possible the range got truncated.  For example:

Say /status says:

{

• node_name: "node01",
• node_id: "bf3e2bf1e40e4b9db68325fa4269b55b",
• active: true,
• timestamp: "2013-05-29T21:27:29.440161Z",
• start_time: "2013-05-21T18:31:03.431516Z",
• install_time: "2013-03-28T20:23:22.199350Z",
• earliestDatestamp: "2011-10-26T15:14:02",
• doc_count: 619210

}

You need to use an until of: 2013-05-29T21:27:30Z with any of the harvest interfaces.

The release that is "eminent" includes a couple minor fixes that should help with some of the latency in indexing.

Jim Klo

Senior Software Engineer

Center for Software Engineering

SRI International

t. @nsomnac

-- 
You received this message because you are subscribed to the Google Groups "Learning Registry Developers List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to learningreg-d...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.