Detailed examination of one station's adjustments?

[Greg Freemyer]

Over on WUWT, there is a discussion of the adjustments to Kathmandu's
temp series in GISS.

With CCC-gistemp, is it easy to target one station and see what the
adjustments are for each step.

The discussion there is that it is "assumed" Kathmandu is being
heavily adjusted in Step 2. And that the main station causing that
adjustment is Tingri. Tingri apparently shows a discontinuity in the
raw temp record that is suspected of triggering.

http://data.giss.nasa.gov/cgi-bin/gistemp/gistemp_station.py?id=205556640000&data_set=2&num_neighbors=1

(I gather the above chart is raw, unadjusted data).

Can step 2 be drilled down into to confirm that assumption?

If Tingri is source of the admjustment, are there corresponding
examples of sites with discontinuities in the down direction that
would average out effects like this?

If this sort of analysis is not yet possible, I suggest that
discussions like this are exactly the sort of question CCC-gistemp can
help clarify.

Thanks
Greg

[David Jones]

On 12 Aug 2010, at 16:22, Greg Freemyer wrote:

> Over on WUWT, there is a discussion of the adjustments to Kathmandu's
> temp series in GISS.

This discussion:
http://wattsupwiththat.com/2010/08/11/more-gunsmoke-this-time-in-nepal/

(URLs are helpful for wattsupwiththat because: a) I generally don't
read it; and, b) articles are posted so frequently a "recent"
discussion quickly requires scrolling past twenty more recent articles)

>
> With CCC-gistemp, is it easy to target one station and see what the
> adjustments are for each step.
>
> The discussion there is that it is "assumed" Kathmandu is being
> heavily adjusted in Step 2. And that the main station causing that
> adjustment is Tingri. Tingri apparently shows a discontinuity in the
> raw temp record that is suspected of triggering.
>
> http://data.giss.nasa.gov/cgi-bin/gistemp/gistemp_station.py?
> id=205556640000&data_set=2&num_neighbors=1
>
> (I gather the above chart is raw, unadjusted data).
>
> Can step 2 be drilled down into to confirm that assumption?
>
> If Tingri is source of the admjustment, are there corresponding
> examples of sites with discontinuities in the down direction that
> would average out effects like this?
>
> If this sort of analysis is not yet possible, I suggest that
> discussions like this are exactly the sort of question CCC-gistemp can
> help clarify.

Agreed on all counts.

It so happens that one of my recent passes over the code was to add a
bit more logging to Step 2 (the so called peri-urban adjustment).

It was on the basis of that logging that I made this comment:

http://wattsupwiththat.com/2010/08/11/more-gunsmoke-this-time-in-
nepal/#comment-454873

The log from Step 2 is in log/step2.log (once ccc-gistemp has run)

grepping it for the station ID gives:

$ grep 21744454000 log/step2.log

217444540000 annual-anomaly {'series': [0.3022916666666674,
-0.13810185185185095, 0.02023148148148228, 0.48689814814814847,
-0.23810185185185131, 0.4118981481481489, -0.22143518518518487,
0.03689814814814904, 0.84523148148148231, 0.82023148148148217,
-0.66310185185185144, -0.36310185185185107, -0.14643518518518422,
-0.18810185185185091, -0.35187499999999899, -0.39643518518518422,
-0.27143518518518467, -0.52976851851851792, 0.27856481481481543,
0.27023148148148179], 'year': 1961}
217444540001 annual-anomaly {'series': [0.28361281752085649,
-0.037104775071736359, 0.2462285582615974, 0.44622855826159707,
0.021228558261597336, 0.45456189159493043, 0.27064985455789375,
0.22122855826159693, -0.062104775071736583, -0.27043810840506932,
9999.0, 9999.0, 9999.0, 9999.0, 9999.0, 9999.0, 9999.0, 9999.0,
9999.0, 9999.0, 9999.0, 9999.0, 9999.0, 9999.0, 9999.0, 9999.0,
9999.0, 9999.0, 0.21011378082888499, 9999.0, 9999.0,
-0.4362307319476213, -0.4954381084050698, -0.17224677030740992,
0.10261366164505606, -0.48083541242901778, -0.38951584880074486],
'year': 1951}
217444540002 step2-action "short"
217444540000 step2-action "adjusted"
217444540000 neighbours ['207424750000', '207425870003',
'207424040000', '207422950000', '207425990000', '205557730000',
'205555780000', '205554720000', '205556640000']
217444540000 adjustment {'series': [-0.10227654553042161,
-0.39388666564380032, -0.29081760755497854, -0.47153920678988326,
-0.92729804498483548, -0.82572059099261064, -0.44177291324246332,
-0.70641675208821753, -0.33472259382950947, 0.046779120713967663,
-0.29192730365340996, -0.59807665718303471, 0.013485224660954757,
-1.3350264207505282, -0.13784574410919806, -0.39139089315970066,
0.083636507778385313, 0.10995470599806509, -0.36228637277301434,
-0.64169091600496264, -0.26706999423090105, -0.22525667230128296,
-0.2753937814539078, -0.18833661698129872, -0.4560265613608166,
-0.97093890922576864, -0.28588166688906202, -0.43501636236773322,
0.1831597997701962, -0.53567876651465052, -0.32838464115862753,
-0.33510843011626879, -0.33810763278103279, -0.62316425095798045,
-0.44760965043962264, 0.55172938150191397, 0.20434335346484428,
-0.57885092671018323, -0.2558637250758366, -0.01056007805280494,
0.06652808326567082, 0.29692234615494723, 0.049391985604336761,
-0.11792853898778023, 0.20753754603213295, -0.44698422470028271,
-0.15103455796374035, -0.17888272272678085, 0.41521685250790696,
0.017029015033566831, -0.3579738688813856, 0.11161996839726496,
0.078759564803737248, -0.61370415937425515, -0.43697470540797856,
-0.28283047017029228, -0.51090151062389155, -0.5281372382967332,
-0.11007335214241164, -0.11334938402902876, -0.20732572949809661,
0.2976743589329291, 0.32930394565423521, -0.058207545593726068,
-0.22924258422007215, -0.095316063203213633, -0.018270531744214301,
0.45479175401361033, 0.20750250707535098, -0.17985011565689071,
-0.28948224098690362, 0.033466771060746463, 0.38360985786086282,
0.58334311916861425, 0.66290994874731146, 0.44921064388347559,
0.29813204969921198, 0.27084897110506556, 1.0011015572561897,
0.24029070030912544, 0.53471247241601438, 0.43350847533587356,
-0.1886772522194497, -0.43456120238192536, 0.061405637032171501,
-0.51622915938456415, 0.055330192557411145, -0.55086332184764408,
-0.7254754726080761, 0.19241851427780746, -0.27398074426742863,
-0.47850013186756396, 0.19970198917679069, 0.11073615347776652,
0.47109110396110587, 0.24639996450412588, 0.49178679652356044,
0.52396325299099467, -0.013301952717718408, 0.75078150496729368,
0.36784136091454683, 0.5609638836545966, 0.1752732011212815,
0.0083557732135654435, 0.44847658300828913, 0.93104614125200424,
0.12538137964875104, 0.13397819206597397, 0.78904891521602494,
0.55060523728779032, 0.460040946308576, 0.52894381124910861,
0.15566334566396939, 0.92914946932716824, 0.42914946932716791,
0.22164579576827201, 0.49388646168649075, 0.062970289311410979,
0.60330129710782876, 0.61795833352406504, 0.59092304522417372,
0.47845132112305178, 1.084711434122885, 1.0098310813767919,
0.77589014117374511, 0.88059007540315304, 0.80183607225086018,
0.1369134390937293, -0.366185699922916, 1.1706598496352305, 9999.0],
'difference': [(1961, 0.13121680866920615), (1962,
-0.05057540036759875), (1963, -0.45479268386340765), (1964,
-0.42549251111597697), (1965, -0.27812730753271286), (1966,
-0.35656795559073773), (1967, -0.32942813666245918), (1968,
-0.76237362075622517), (1969, -0.65281296720367488), (1970,
-1.0942122257489109), (1971, 0.18460171998428748), (1972,
0.56280384102864178), (1973, 0.25717133866295072), (1974,
0.65919295581295678), (1975, 0.59827496450412487), (1976,
0.88822198170874467), (1977, 0.79539843817617939), (1978,
0.51646656580079953), (1979, 0.47221669015247825), (1980,
0.097609879433065039)], 'year': 1880}
217444540001 step2-action "dropped"

There's lots of arcane detail here, and I'd love to be able to
present this in a whizzy tool.

Volunteers anyone?

drj

[Greg Freemyer]

On Fri, Aug 13, 2010 at 11:39 AM, David Jones <d...@ravenbrook.com> wrote:

On 12 Aug 2010, at 16:22, Greg Freemyer wrote:

Over on WUWT, there is a discussion of the adjustments to Kathmandu's
temp series in GISS.

This discussion:
http://wattsupwiththat.com/2010/08/11/more-gunsmoke-this-time-in-nepal/

(URLs are helpful for wattsupwiththat because: a) I generally don't read it; and, b) articles are posted so frequently a "recent" discussion quickly requires scrolling past twenty more recent articles)

Yes, that one.

That's not very penetrable.  A whizzy tool is definitely in order.  In addition to showing info about which stations are impacting step2 by how much, it would be nice to have tool that output a graphic of Kathmandu in this case after each step.

ie. One graph with raw, post step-1, post step-2, etc. would be great.
 
Even better is that since the underlying data/logs is pretty static, a web page front in could be put together to drill into these details dynamically.  Pulling out the data from the logs for just one station seems like it would be a relatively quick grep and then some formatting.  The underlying logs would only have to be generated once a month.

Does ccc-gistemp have plans for an interactive site like that?  Can it be done at http://clearclimatecode.org/

Volunteers anyone?

drj

 
I'm not much of web developer, or I'd consider it.  I've got some perl based weather displays running for a weather station at my Dad's house, but to be honest, they don't look very good.  And they are really slow to generate.  (Like over a minute for one set of graphs.)

Greg

[David Jones]

Right. GISTEMP's own station browser almost does that: http://
data.giss.nasa.gov/cgi-bin/gistemp/gistemp_station.py?
id=217444540000&data_set=1&num_neighbors=1

And our stationplot.py can _almost_ do that. There is no "etc" for
stations by the way, for Step 3 and beyond temperature data are
gridded, stations lose their identity.

>
> Even better is that since the underlying data/logs is pretty
> static, a web page front in could be put together to drill into
> these details dynamically. Pulling out the data from the logs for
> just one station seems like it would be a relatively quick grep and
> then some formatting. The underlying logs would only have to be
> generated once a month.
>
> Does ccc-gistemp have plans for an interactive site like that? Can
> it be done at http://clearclimatecode.org/

Well, we have plans. But as it's entirely pro bono, such plans
progress very slowly.

Cheers,
drj