Re: Rounding in step2
Nick Barnes
> I've been working on porting ccc-gistemp to use the MapReduce
> programming model to parallelize the compute-intensive tasks. Quite
> successful with steps 1 and 3. Step 2 seem to be okie but I am getting
> the same rounding error that you've talked about in this discussion.
What ccc-gistemp sources are you basing your work on? I got rid of a
lot of rounding, back in early 2010.
> It can be seen that the temperature for December changes by 0.01 in
> 1959 and this continues for all the months in 1960. Similar trends
> were observed for many stations.
> Can you please tell me why this could be happening? This is evident
> only when the code is parallelized and I see no issue with
> parallelization as the system seems to be working fine for other
> stations that down show this error trend.
Step 2 computes UHI adjustments based on annual anomaly series, and
then applies them on a year-by-year basis. So if the computation is
incorrect, you will exactly see this sort of error.
You will need to drill down into rural_difference (which computes a
rural-urban series between a given urban series and whatever nearby
rural series can be found), getfit (which calculates a two-part linear
fit to the rural-urban series), and adjust_record (which adds the
two-part linear fit to the urban series). A good start might be to
look at the return values of getfit().
Nick B
Nick Barnes
> I had been working on porting the cccgistemp code to MapReduce
> framework using mrjob API for parallelizing data and compute intensive
> tasks of Steps 1, 2 and 3.
>
> If I have to release this modified version so as be useful for others
> working in this area to do some performance improvement does it become a
> joint copyright or do I have to sign a contribution agreement handing over
> my copyright?
>
> In either case, I have the copyright (C) 2008 Ravenbrook Limited maintained
> in all the source files.
Assuming that you are happy to release your code under the same BSD
license that it all exists under at present, you should simply add a
copyright statement with your name and date to any files to which you
have made a significant contribution. Something like this:
Copyright (C) 2008 Ravenbrook Limited.
Portions copyright (C) 2012 Vinay Sudhakaran.
Or similar.
We haven't been very good at adding and maintaining copyright
statements and license texts. Our intention is that all copyrights
remain with the original creators, and all the code is BSD licensed.
Copyrights of stuff that I wrote before we created the Foundation
should probably belong to Ravenbrook Limited; for stuff that I wrote
after that they should probably belong to Climate Code Foundation.
I might go through the code-base and make this a bit clearer.
Nick B