Reduce Function Error

[Kurt Didenhover]

I'm getting an error on a simple reduce function. All my test nodes are on the same server with the db sharded into 16 parts. I'm getting an error from the smartproxy.

Map Function:
function(doc){ if(doc.type=='session') {emit(doc.aID, 1);} }

Reduct Function:
function(keys,values){return sum(values);}

I'm seeing the following stack trace

Unable to fetch data from node [Failure instance: Traceback: <type 'exceptions.KeyError'>: 0
/usr/lib/python2.6/site-packages/twisted/web/client.py:152:handleResponse
/usr/lib/python2.6/site-packages/twisted/web/client.py:304:page
/usr/lib/python2.6/site-packages/twisted/internet/defer.py:243:callback
/usr/lib/python2.6/site-packages/twisted/internet/defer.py:312:_startRunCallbacks
--- <exception caught here> ---
/usr/lib/python2.6/site-packages/twisted/internet/defer.py:328:_runCallbacks
/usr/lib/python2.6/site-packages/smartproxy/fetcher.py:171:_onsuccess
/usr/lib/python2.6/site-packages/smartproxy/reducer.py:303:process_map
/usr/lib/python2.6/site-packages/smartproxy/reducer.py:320:queue_data
/usr/lib/python2.6/site-packages/smartproxy/reducer.py:373:__reduce
/usr/lib/python2.6/site-packages/smartproxy/reducer.py:335:_do_reduce
/usr/lib/python2.6/site-packages/smartproxy/reducer.py:204:enqueue
/usr/lib/python2.6/site-packages/smartproxy/reducer.py:226:next
/usr/lib/python2.6/site-packages/smartproxy/reducer.py:247:feed
/usr/lib/python2.6/site-packages/twisted/internet/process.py:677:writeToChild
]

Any ideas on how to fix this?

Thanks,

Kurt

--
Subscription settings: http://groups.google.com/group/couchdb-lounge/subscribe?hl=en

[Randall Leeds]

Before I look into this deeply, can you make sure that your design
document has replicated to all shards and that you can run the reduce
on each by contacted it directly on the standard couch port?

[Kurt Didenhover]

Yes, my design doc is replicated and the reduce works on each shard if run individually. I'm using couchdb 0.11.0 and twisted 8.2 on Fedora Core 12.

After turning on debug logging, it appears that each shard is giving back its data correctly, but when smartproxy tried to aggregate the results it died.

Thanks for the help!

> --
> To unsubscribe, reply using "remove me" as the subject.

[Randall Leeds]

I'll take a look at those lines in the code in just a bit, but if I
don't see anything obvious that could trigger it it'll take me a
little longer to try to repro since I don't have 0.11 installed right
now.

In the meantime, macfergus rewrote the reducer if you'd like to live
on the edge.
fetch macfergus/couchdb-lounge and cherry-pick
4e1253db4ef7ca51c22b0709f4804a8eef83bfd1
It might not be a totally clean merge, so tread carefully ;)

[Kurt Didenhover]

So, I figured out my problem. Nothing wrong with the code. In smartproxy.xml the couchjs executable was installed in /usr/local/bin and not /usr/bin so Twisted wasn't finding that, but when I specified the path to the couchjs file, it worked.

Kurt

[Randall Leeds]

Hurray! Glad it was that easy.
Ill look at detecting and logging that problem shortly so it doesn't lead more people on long hunts.

More evidence that we have a serious lack of documentation.

On Apr 13, 2010 2:13 PM, "Kurt Didenhover" <kdide...@gmail.com> wrote:

So, I figured out my problem. Nothing wrong with the code. In smartproxy.xml the couchjs executable was installed in /usr/local/bin and not /usr/bin so Twisted wasn't finding that, but when I specified the path to the couchjs file, it worked.

Kurt

On Apr 12, 2010, at 3:58 PM, Randall Leeds wrote:

> I'll take a look at those lines in the code in...