I've been hitting my head against a wall for a good 4 hours trying to get ApplyQuantile to work. Can someone give me the most basic working example possible? All I need is to take an input stream, determine the quantiles for a value in that stream, and then replace that value with the quantile. Here's what I have so far:
register 'java/include/datafu-0.0.5.jar';
define Quantile datafu.pig.stats.StreamingQuantile('101');
define ApplyQuantile datafu.pig.stats.ApplyQuantile('101');
prefs = LOAD '$input' USING PigStorage() AS (user_id:chararray, item_id:chararray, preference:float);
grouped_prefs = GROUP prefs ALL;
quantiles = FOREACH grouped_prefs GENERATE Quantile(prefs.preference) as value;
processed = FOREACH prefs GENERATE ApplyQuantile(prefs.preference, quantiles.value);
STORE processed INTO '$output';
Generating quantiles works fine, but as soon as I try to use them I start getting errors. This code throws "ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: Expected a tuple", but I've tried about 200 variations and none of them seem to work either.
What am I missing? :(
-Ryan