step 3 of ccc-gistemp

[vinay]

Hi,

I am working on porting the ccc-gistemp to MapReduce framework.
Till date I have been quite successful with Step 1 and I am now trying
Step 3 as its compute intensive.

Step 3 is a little confusing and I was wondering what files from /
input are required exclusively by step 3.? Is SBBX.HadR2 referred to
in Step 3 execution.?

This is important as I need to write code to specifically move only
the required files into the MR space while running Mapreduce.

Any information in this regard will be very useful.

Thanks,
VS

[Nick Barnes]

On Wed, Jun 29, 2011 at 18:33, vinay <vinaysu...@gmail.com> wrote:
>   I am working on porting the ccc-gistemp to MapReduce framework.

Cool.

> Till date I have been quite successful with Step 1 and I am now trying
> Step 3 as its compute intensive.
>
> Step 3 is a little confusing and I was wondering what files from /
> input are required exclusively by step 3.? Is SBBX.HadR2 referred to
> in Step 3 execution.?

No. SBBX.HadR2 is the sea-surface temperature dataset, and is only
read by step 4. The only step 3 inputs are the step 2 outputs.

Please don't hesitate to ask more questions. We're very short of documentation.
--
Nick Barnes, Climate Code Foundation, http://climatecode.org/

[Vinay Sudhakaran]

Hi Nick,

    Quite strange....If I do not feed in the input directory to MapReduce Job then I get an error in step3.py, function inbox(station_records, lats, latn, longw, longe)

for record in station_records:
     st = record.station

st = record.station
AttributeError: 'Series' object has no attribute 'station'

I was wondering if while create the Series object, are there any file/attribute referred to in order to differentiate station records from subbox series?
I do not have any problems in Step1 and thats why I'm confused.!

Thanks,
VS

--
You received this message because you are subscribed to the Google Groups "CCC GISTEMP discussion" group.
To post to this group, send email to ccc-giste...@googlegroups.com.
To unsubscribe from this group, send email to ccc-gistemp-dis...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/ccc-gistemp-discuss?hl=en.

--
Regards,
Vinay Sudhakaran

Ph:+44 7901 037094
     +44 7821 551979

[Vinay Sudhakaran]

Hi Nick,

    I got it working. The v2meta wasn't getting created as I forgot to move the relevant files in Step3.!

Thanks,
VS