Auditing the auditors #1
Josh Halpern
the M&M audit. Anyone who seriously wants to join in should go to
http://www.climate2003.com/ to about the middle of the page where it
says Audit Issues. I will be going through these as time allows, but
fair warning, it took about four hours (including a visit to a local
library that had the necessary journals, they were not on line) to get
through #1. It's taking about 1.5 hours to do this, because I have to
relocate a number of web sites, etc.
Some preliminaries. The file sent by Mann's group to McIntyre is called
pcproxy.txt and can be found at http://www.climate2003.com. To save
space, I will call this site C2300. Also to save space, unless
specifically noted, I will use pcproxy to refer to the file pcproxy.txt
as found at C2300. The file was compiled by Scott Rutherford.
Rutherford also sent a roster showing which proxys were in each column
of pcproxy. You can find this at
http://www.climate2003.com/data/backto1820.txt. This file is a
modification of the file multiproxy.inf sitting on
ftp://holocene.evsc.virginia.edu/pub/MBH98 which I will call /pub/MBH98
to save space. multiproxy.inf lists the input files for the multiproxy
analysis. backto1820.txt simply adds a serial number to the file names
in order.
The issues of C2300 refer to the file pcproxy and the paper MBH98 as
published in Nature, we will call that MBH98 and take care to
differentiate it from the ftp site /pub/MBH98. We will call the
published supplemental materials NSM.
OK, from C2300 inquiring minds wish to know.
#1 is Does the database contain truncations of series 10, 11 and 100?
(and of the version of series 65 used by MBH98)?
which is actually three questions, but wth. Let us call them 1a, 1b and
1c. Note that this is a point that has been heavily relied on in the
criticism of MBH98.
1a. #10 is the Central England historical record, originally published
by Manley. pcproxy lists values between 1730 and 1987.
The data base is maintained by the Hadley center and can be downloaded
from
http://www.met-office.gov.uk/research/hadleycentre/CR_data/Monthly/HadCET_act.txt
and the series runs from 1659 to the present.
The reference given in the supplemental materials of MBH98 are
sufficient to locate this source. The header on the data file is:
MONTHLY MEAN CENTRAL ENGLAND TEMPERATURE (DEGREES C)
1659-1973 MANLEY (Q.J.R.METEOROL.SOC., 1974)
1974ON PARKER ET AL. (INT.J.CLIM., 1992)
There is at least one Hadley Center web page which links to the data set
http://www.met-office.gov.uk/research/hadleycentre/obsdata/CET.html
That page has the following references at the bottom
Manley, G., 1953: The mean temperature of Central England, 1698 to 1952.
Quart J Roy Meteorol Soc, 79, 242-261.
Manley, G. 1974: Central England Temperatures: monthly means 1659 to
1973. Quart J Roy Meteorolol Soc, 100, 389-405.
Parker, D. E., T. P. Legg, and C. K. Folland, 1992: A new daily Central
England Temperature Series, 1772-1991. Int J Climatol, 12, 317-342.
I went and got the last two today, and read them. The reason that MBH98
used does not use the data before 1730 is clearly explained in the first
paragraph of the 1992 Parker, Legg and Folland paper:
"Manley1953) published a time series of monthly mean temperatures
representative of central England for 1698-1952, followed (Manley 1974)
by an extended and revised series for 1659-1973. Up to 1814 his data
are based mainly on overlapping sequences of observations from a variety
of carefully chosen and documented locations. Up to 1722, available
instrumental records fail to overlap and Manley needs to use
non-instrumental series for Utrecht compiled by Labrijn (1945), in order
to mate the monthly central England temperature (CET) series
complete.Between 1723 and the 1760s there are no gaps in the composite
instrumental record, but the observations generally were taken in
unheated rooms rather than with a truly outdoor exposure...."
Which means that the Manley reconstruction is only continuous from 1722
on, but the information upon which it relies from 1723-28 has further
difficulties, essentially absolute values were not reliable, and the
series was constructed by taking the difference between measurements
made by those thermometers and ones thought to be more reliable after
1727, and then repeatedly differenced to get values before 1727.
In the light of this, it is perfectly reasonable to truncate the CET
series at 1730. On the other end, 1987 is the cut off for the
analysis. We note that the cut offs were listed in the NSM.
On balance this illustrates the principal of RTFR (Read the effing
references) and the danger of someone unfamiliar with an area trying to
do an "audit". To use the CET before 1730 would clearly have been a
mistake, and I think you could make a good argument that it should have
been cut off at 1772.
1b. #11 is the Central Europe historical record, originally published
by Pfister. pcproxy lists values between 1550 and 1987. The series
goes to 1525. The data can be picked up from
ftp://ftp.ngdc.noaa.gov/paleo/historical/switzerland/clinddef.txt
and information on the compilation can be found in the same folder as
readme_swissindices.txt
The references for this data are:
Pfister, C. (1984): Das Klima der Schweiz von 1525-1860 und seine Bedeutung
in der Geschichte von Bevoelkerung und Landwirtschaft. Bern
Pfister, C. (1992): Monthly temperature and precipitation patterns in Central
Europe from 1525 to the present. A methodology for quantifying man made
evidence on weather and climate. In: Bradley R.S., Jones P.D. (eds.)
Climate since 1500A.D., pp. 118-143. London
Pfister C., Kington J., Kleinlogel G., Schuele H., Siffert E. (1994):
The creation of high resolution spatio- temporal reconstructions of past
climate from direct meteorological observations and proxy data.
Methodological considerations and results. In: Frenzel, B., Pfister C.,
Glaeser, B. (eds), Climate in Europe 1675-1715.
of which I could locate the second. On page 121, second paragraph from
the bottom, one reads:
"The evidence increases in volume, density and diversity over time. For
the period 1525-1549 the entries originate mainly from chronicles and
annals. Accordingly, weather sequences are mainly described at a
seasonal level; information is missing for 43% of the months and the
enphasis is on anomalous rather than ordinary weather. In the second
period 1550-1658 monthly data from weather diaries and personal papers
are abundant....."
Again, On balance this illustrates the principal of RTFR and the danger
of someone unfamiliar with an area trying to do an "audit". To use the
central European index before 1550 would clearly have been a mistake.
Note also that Bradley was one of the two book editors, so he surely
knew a great deal about these series. Phil Jones was the other.
1c #100 is a chinese tree ring series which can be found from the
NOAA paleoclimate site
ftp://ftp.ngdc.noaa.gov/paleo/treering/chronologies/asia/chin004.crn
that runs from 1540-1989. You can also find this data set from the noaa
paleoclimate site http://www.ngdc.noaa.gov/paleo/.
1540-1989 is the period for this data set that one finds in the NSM.
In pcproxy the first two years (1540 and 1541) are absent. Two files,
called chin04.txt and chin04-orig.txt are found in the
/pub/MBH98/TREE/ITRDB/MISC folder. They are identical to each other and
appear identical to the data set at the noaa paleo site.
In short, this is almost certainly a copying error in the construction
of pcproxy.
The urge to snark is strong, beware the jabberwock my son.
josh halpern
<ftp://holocene.evsc.virginia.edu/pub/MBH98/TREE/ITRDB/MISC/chin04-orig.txt>
Phil Hays
> Well, time consuming though I knew it would be I have started to audit
Thank you for your effort.
--
Phil Hays
Jim Norton
Thank you very much.
Roger Coppock
Mann has published several times since then, even
this year. Why aren't M&M 'auditing' more recent
work?
--
"One who joyfully guards his mind
And fears his own confusion
Can not fall.
He has found his way to peace."
-- Buddha, in the "Pali Dhammapada,"
~5th century BCE
-.-. --.- Roger Coppock (rcop...@adnc.com)
-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 100,000 Newsgroups - 19 Different Servers! =-----
Steve Schulin
Josh Halpern <j.ha...@incoming.verizon.net> wrote:
Your abc breakdown ignores the M&M question about series 65?
Interesting info, Josh. Thanks. From what you've reported, it looks like
some other truncation point would have been more perfect than 1730,
however: 1723, 1728, or somewhere in 1760s. Your use of the root perfect
seems an exaggeration in this case. I agree that the supplemental info
lists 1730, but note that the column is described as showing the "first
year of record available". That does not seem like an accurate
description of the start date used by Mann et al. for this series.
I think you're on the right track in answering M&M's question 1a. It
seems likely that Mann et al did indeed decide to truncate the early
years, including a quite cold series of values, from this series. I
don't recall Mann et al discussing any systematic approach to validating
the data in all the 112 or 159 or however many there were series.
I think you're again on the right track in answering M&M's question 1b.
It seems likely that Mann et al did indeed decide to truncate the early
years from this series.
>
> 1c #100 is a chinese tree ring series which can be found from the
> NOAA paleoclimate site
> ftp://ftp.ngdc.noaa.gov/paleo/treering/chronologies/asia/chin004.crn
> that runs from 1540-1989. You can also find this data set from the noaa
> paleoclimate site http://www.ngdc.noaa.gov/paleo/.
>
> 1540-1989 is the period for this data set that one finds in the NSM.
> In pcproxy the first two years (1540 and 1541) are absent. Two files,
> called chin04.txt and chin04-orig.txt are found in the
> /pub/MBH98/TREE/ITRDB/MISC folder. They are identical to each other and
> appear identical to the data set at the noaa paleo site.
chin04.txt is not identical to chin04-orig.txt -- the first two years of
the series, high values both, appear in chin04-orig.txt but not
chin04.txt
>
> In short, this is almost certainly a copying error in the construction
> of pcproxy.
It's not clear whether you think Mann et al actually used the 1540-1541
values for this time series in producing the hockey stick.
>
> The urge to snark is strong, beware the jabberwock my son.
M&M ask reasonable questions under the heading of issue #1. Some of what
you write here is illuminatory, some is obfuscatory, and some looks just
snarky itself.
>
> josh halpern
> <ftp://holocene.evsc.virginia.edu/pub/MBH98/TREE/ITRDB/MISC/chin04-orig.txt>
>
David Ball
Josh spent 4 hours getting the information he posted. What are
you doing, besides posting "what have you done lately" comments? Is
there anything preventing you from getting off your ass and doing some
of the legwork or do you plan to just sit here and complain about
someone else's efforts?
What I find illuminating about Josh's post is that the basic
information about why certain series were truncated is available in
the reference material. When the answer to each question posed by M&M
boils down to their inability to RTFR, there is a problem. That likely
stems from the unseemly rush they were in to get something in print. I
can't think of a researcher anywhere who can knock off a peer-reviewed
paper in a matter of weeks, without taking some serious shortcuts and
at least in the case of their question 1, it appears that those
shortcuts caused them to overlook basic information needed to do their
"audit".
>>
>> Which means that the Manley reconstruction is only continuous from 1722
>> on, but the information upon which it relies from 1723-28 has further
>> difficulties, essentially absolute values were not reliable, and the
>> series was constructed by taking the difference between measurements
>> made by those thermometers and ones thought to be more reliable after
>> 1727, and then repeatedly differenced to get values before 1727.
>>
>> In the light of this, it is perfectly reasonable to truncate the CET
>> series at 1730. On the other end, 1987 is the cut off for the
>> analysis. We note that the cut offs were listed in the NSM.
>>
>> On balance this illustrates the principal of RTFR (Read the effing
>> references) and the danger of someone unfamiliar with an area trying to
>> do an "audit". To use the CET before 1730 would clearly have been a
>> mistake, and I think you could make a good argument that it should have
>> been cut off at 1772.
>
>Interesting info, Josh. Thanks. From what you've reported, it looks like
>some other truncation point would have been more perfect than 1730,
>however: 1723, 1728, or somewhere in 1760s. Your use of the root perfect
>seems an exaggeration in this case. I agree that the supplemental info
>lists 1730, but note that the column is described as showing the "first
>year of record available". That does not seem like an accurate
>description of the start date used by Mann et al. for this series.
And you are basing this on what? Josh's point about the
problematic nature of the 1723-1728 data is well taken. Using any of
the data in that period would be a mistake. There is nothing
inherently wrong with using 1730 - it's a nice round number - and as
Josh pointed out other truncation points could have been adopted.
Let's face it, the proxy data are all reconstructed back to nice round
numbers. In the tree ring data, why go back to 1700? Why not 1699? Or
1697? It's a computational convenience, nothing more.
Your arbitrary choice of a point in the 1760's is noted. In
addition, your quibbling about the description of 1730 being the first
year or record is just that: a quibble. When an analysis is done, bad
data are routinely thrown out. That's a fact of life. The paper in
question is NOT meant for the general public but for other experts in
the field, experts who persumably are, unlike M&M, aware of the
problematic nature of some of the data. That being said, such a
notation can be considered a given.
>
>>
>> 1c #100 is a chinese tree ring series which can be found from the
>> NOAA paleoclimate site
>> ftp://ftp.ngdc.noaa.gov/paleo/treering/chronologies/asia/chin004.crn
>> that runs from 1540-1989. You can also find this data set from the noaa
>> paleoclimate site http://www.ngdc.noaa.gov/paleo/.
>>
>> 1540-1989 is the period for this data set that one finds in the NSM.
>> In pcproxy the first two years (1540 and 1541) are absent. Two files,
>> called chin04.txt and chin04-orig.txt are found in the
>> /pub/MBH98/TREE/ITRDB/MISC folder. They are identical to each other and
>> appear identical to the data set at the noaa paleo site.
>
>chin04.txt is not identical to chin04-orig.txt -- the first two years of
>the series, high values both, appear in chin04-orig.txt but not
>chin04.txt
>
>>
>> In short, this is almost certainly a copying error in the construction
>> of pcproxy.
>
>It's not clear whether you think Mann et al actually used the 1540-1541
>values for this time series in producing the hockey stick.
>
>>
>> The urge to snark is strong, beware the jabberwock my son.
>
>M&M ask reasonable questions under the heading of issue #1. Some of what
>you write here is illuminatory, some is obfuscatory, and some looks just
>snarky itself.
>
And the reasonable responses given by Josh show that the
information M&M are inquiring about was available in the paper's
reference material, material that they apparently didn't bother to
read. That isn't a snark, nor is it obfuscatory. It is simply the
truth. Nor are your minor quibbles adding anything to the discussion,
except noise.
Steve Schulin
David Ball <wra...@mb.sympatico.ca> wrote:
> you doing, besides posting "what have you done lately" comments? ...
My question was prompted because I got the impression that Josh was
attempting to fully address the questions in issue #1. His use of the
phrase "to get through #1" contributed to that impression. My question
was politely stated. If you had been asking question to me under similar
circumstances, it would have been no surprise to see you be quite
abusive about only seeing abc instead of abcd.
> ... Is
> there anything preventing you from getting off your ass and doing some
> of the legwork or do you plan to just sit here and complain about
> someone else's efforts?
There's plenty of legwork to be done, that's for sure. I had done enough
legwork previously to recognize that Josh was incorrect in stating,
below, that the chin04.txt and chin04-orig.txt files at ftp site were
identical. Are you not grateful to have that error corrected?
> What I find illuminating about Josh's post is that the basic
> information about why certain series were truncated is available in
> the reference material. When the answer to each question posed by M&M
> boils down to their inability to RTFR, there is a problem. ...
It seems clear enough that Mann et al chose to use subsets of series 10
and 11. There is no indication that they systematically applied some
evaluative methodology to all the series. And your suggestion that "the
answer to each question posed by M&M boils down to their inability to
RTFR" seems quite an exaggeration.
> ... That likely
> stems from the unseemly rush they were in to get something in print. I
> can't think of a researcher anywhere who can knock off a peer-reviewed
> paper in a matter of weeks, without taking some serious shortcuts and
> at least in the case of their question 1, it appears that those
> shortcuts caused them to overlook basic information needed to do their
> "audit".
You have often mistaken question marks for exclamation points, so your
take on the current matter comes as no surprise. Your apparent argument
that "of course Mann et al modified the source data" seems quite at odds
with the impression given by all those who have been echoing Mann's
recent claim that the data necessary to replicate their work is
available from the referenced sources of the data.
>
> >>
> >> Which means that the Manley reconstruction is only continuous from 1722
> >> on, but the information upon which it relies from 1723-28 has further
> >> difficulties, essentially absolute values were not reliable, and the
> >> series was constructed by taking the difference between measurements
> >> made by those thermometers and ones thought to be more reliable after
> >> 1727, and then repeatedly differenced to get values before 1727.
> >>
> >> In the light of this, it is perfectly reasonable to truncate the CET
> >> series at 1730. On the other end, 1987 is the cut off for the
> >> analysis. We note that the cut offs were listed in the NSM.
> >>
> >> On balance this illustrates the principal of RTFR (Read the effing
> >> references) and the danger of someone unfamiliar with an area trying to
> >> do an "audit". To use the CET before 1730 would clearly have been a
> >> mistake, and I think you could make a good argument that it should have
> >> been cut off at 1772.
> >
> >Interesting info, Josh. Thanks. From what you've reported, it looks like
> >some other truncation point would have been more perfect than 1730,
> >however: 1723, 1728, or somewhere in 1760s. Your use of the root perfect
> >seems an exaggeration in this case. I agree that the supplemental info
> >lists 1730, but note that the column is described as showing the "first
> >year of record available". That does not seem like an accurate
> >description of the start date used by Mann et al. for this series.
>
> And you are basing this on what? ...
On the meaning of the phrase "first year of record available", as
opposed to the meaning of "first year of record chosen to be included
although prior years were available".
> ... Josh's point about the
> problematic nature of the 1723-1728 data is well taken. ...
I too think it may very well have been reasonable to truncate this
series. I find it quite silly that you apparently chastise M&M for
asking the question, however.
> ... Using any of
> the data in that period would be a mistake. There is nothing
> inherently wrong with using 1730 - it's a nice round number - and as
> Josh pointed out other truncation points could have been adopted...
And once its verified what Mann et al actually did, the effects of other
treatments will be interesting to explore.
> ...
> Let's face it, the proxy data are all reconstructed back to nice round
> numbers. In the tree ring data, why go back to 1700? Why not 1699? Or
> 1697? It's a computational convenience, nothing more. ...
Are you saying that other time series were truncated by Mann et al
besides the four mentioned in issue #1?
> ...
> Your arbitrary choice of a point in the 1760's is noted. ...
I quite purposefully used the precision quoted from the reference by
Josh. How would you improve upon my choice of language?
> ... In
> addition, your quibbling about the description of 1730 being the first
> year or record is just that: a quibble. ...
I thought about elaborating, but didn't want to clutter. In the body of
the Nature paper, Mann et al claim to have included five series in their
analysis that apparently were not included. It is in the context of such
apparent sloppiness that M&M's questions warrant clear answers, untinged
by the insults and invective that have become rote from Mann and his
apologists.
> ... When an analysis is done, bad
> data are routinely thrown out. That's a fact of life. The paper in
> question is NOT meant for the general public but for other experts in
> the field, experts who persumably are, unlike M&M, aware of the
> problematic nature of some of the data. That being said, such a
> notation can be considered a given.
So you think it inappropriate that IPCC used the hocky stick graph in
Fig 1 of the WG1 TAR policymaker summary?
I've seen how prone some folks here are to exaggerate. I thought it
worthwhile to nip some of that in the bud in this case.
David Ball
<j.ha...@incoming.verizon.net> wrote:
Hi, Josh,
I took a quick look at M&M's second question:
2. Are the 1980 values of series #73 through #80 identical to
7 decimal places? Similarly for the 1980 values of series #81-83? And
for the 1980 values of series #84 and #90-92? What is the reason for
this?
I went to the PC data on Mann's website in
ftp://holocene.evsc.virginia.edu/pub/MBH98/TREE/STAHLE/SWM/BACKTO_1700
which encompasses series 73 through 80 and compared the series to
those highlighted by M&M in their excel spreadsheet. The short answer
is that the spreadsheet data are shifted one row up on each of the
PC's. In other words, the 1980 data given on the above ftp site are
shown in the spreadsheet as being in 1979. The 1979 are shifted to
1978 and so on. This is true for all of the PC's. I'm not sure where
the 0.023030 that appears in the 1980 position on the spreadsheet
comes from, but it looks like an error in putting the spreadsheet
together. Certainly, the data on the Mann website are consistent with
what is on the spreadsheet except for this point.
The same thing appears to have happened with series 81-83 in
the spreadsheet when compared to the data at:
ftp://holocene.evsc.virginia.edu/pub/MBH98/TREE/VAGANOV/BACKTO_1750
as well as series 84 and 90-92 at:
ftp://holocene.evsc.virginia.edu/pub/MBH98/TREE/ITRDB/NOAMER/BACKTO_1750
Each series is shifted down a year in the spreadsheet. Again, I don't
know where the 1980 data come from, but it looks like an artifact of
putting the spreadsheet together and nothing nefarious at all.
Dave.
Josh Halpern
David Ball wrote:
Thanks, I was going to do that..........
Well that was the point of starting.
While I am at this, let me acknowledge that Steve Sculin was correct
about the chin004.txt and chin004-orig.txt. Which means that we still
have to understand if there was any reason for deleting the first two
years of the series, which probably means that I am going to have to
hunt down the original reference. The state of play wrt to that point
is that MBH HAD the complete file but they also had a file with two
years truncated at the beginning
josh halpern
>
>
>
>
Nigel Persaud
> NOAA paleoclimate site
> ftp://ftp.ngdc.noaa.gov/paleo/treering/chronologies/asia/chin004.crn
> that runs from 1540-1989. You can also find this data set from the noaa
> paleoclimate site http://www.ngdc.noaa.gov/paleo/.
>
> 1540-1989 is the period for this data set that one finds in the NSM.
> In pcproxy the first two years (1540 and 1541) are absent. Two files,
> called chin04.txt and chin04-orig.txt are found in the
> /pub/MBH98/TREE/ITRDB/MISC folder. They are identical to each other and
> appear identical to the data set at the noaa paleo site.
>
> In short, this is almost certainly a copying error in the construction
> of pcproxy.
As Steve Schulin pointed out, your information here is false. Steve is
correct that the first two years of chin04-orig.txt are deleted in
chin.04 txt, exactly as stated by M&M. I hope your other research is
more accurate than what you've stated above.
BTW have you figured out the 159 series yet? I guess not or you'd have
told us. Why don't you finish that before you start something else?
David Ball
I took a look at that as well and he is right, but I think we
have to keep things in perspective. We're talking about two years from
a single series. Steve makes much of the fact that the missing years
were apparently warm, but we're really talking about nothing more than
noise. If two warm years in one record from a single part of the world
is sufficient to make MBH's results do a 180 turn there is a serious
problem somewhere.
David Ball
>Where are the calculations of principal components for series in the range
>#73-92 that would show that these have been collated into the correct year?
>Do you have any working papers that show these, and if so, would you make
>them FTP or otherwise publicly available?
much of an issue anymore. The software used for calculating the PC's
used in series 72-83 are publically available on the Mann website.
I've yet to be able to reproduce Mann's results, but I'm making
progress. I haven't had much time in more than a week to even look at
it. Having said that, M&M are not asking about the validity of the
code, but rather whether the code exists and it does. The one thing I
do want to check is whether the fortran code used with the SWM and
Vaganov series is the functionally the same as that used in producing
the PC's for series 84-92.
David Ball
wrote:
>> 1c #100 is a chinese tree ring series which can be found from the
>> NOAA paleoclimate site
>> ftp://ftp.ngdc.noaa.gov/paleo/treering/chronologies/asia/chin004.crn
>> that runs from 1540-1989. You can also find this data set from the noaa
>> paleoclimate site http://www.ngdc.noaa.gov/paleo/.
>>
>> 1540-1989 is the period for this data set that one finds in the NSM.
>> In pcproxy the first two years (1540 and 1541) are absent. Two files,
>> called chin04.txt and chin04-orig.txt are found in the
>> /pub/MBH98/TREE/ITRDB/MISC folder. They are identical to each other and
>> appear identical to the data set at the noaa paleo site.
>>
>> In short, this is almost certainly a copying error in the construction
>> of pcproxy.
>
>As Steve Schulin pointed out, your information here is false. Steve is
>correct that the first two years of chin04-orig.txt are deleted in
>chin.04 txt, exactly as stated by M&M. I hope your other research is
>more accurate than what you've stated above.
Might I suggest you change your tone. It is completely
uncalled for, especially since you don't seem to want to get off your
ass and do any of the work. Josh has already given Steve credit for
this point, as have I. I guess you missed that part. Let's keep things
in perspective, too. So far, M&M have not come across as very
competent, especially when much of the information that they were
looking for was in the reference material, material they obviously
didn't read.
>
>BTW have you figured out the 159 series yet? I guess not or you'd have
>told us. Why don't you finish that before you start something else?
That's what YOU are after. Do some of the work for yourself
and look for the answers. I guess you simply don't like seeing M&M
cast in an unfavourable light. Why not address the rest of Josh's
post? The information was available in the references. Why didn't M&M
read them? Seems like a pretty simple thing to do, especially since it
only took Josh 4 hours or so.
James Acker
much work M&M are doing to figure out the differences (as well as
Mann et al.)
One question, Josh: what library did you go to for all the
references? UMD-College Park?
Jim Acker
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
Jim Acker
jac...@gl.umbc.edu
"Since we are assured that an all-wise Creator has observed the
most exact proportions, of number, weight, and measure, in the
make of all things, the most likely way therefore, to get any
insight into the nature of those parts of the creation, which
come within our observation, must in all reason be to number,
weigh, and measure." - Stephen Hales
Josh Halpern
James Acker wrote:
SNIP...
> One question, Josh: what library did you go to for all the
>references? UMD-College Park?
>
Yep. Easily the best university library in the neighborhood. Open on
Sunday unlike the Library of Congress (We are too poor)
josh halpern
>
>
>
>
David Ball
>Subject: Re: Auditing the auditors #1
>
>View this article only
>Newsgroups: sci.environment
>Date: 2003-12-16 07:16:14 PST
>
>I'd like to commend Josh on this effort -- one wonders how
>much work M&M are doing to figure out the differences (as well as
>Mann et al.)
I've been wondering why M&M, when they noticed the problems,
assumed that there was something nefarious going on with MBH and also
why they went ahead and published their article? When one sees gross
errors in data analysis, one doesn't immediately leap to the most
extreme explanation as the reason. The first thing I do is check my
own work. I certainly wouldn't go ahead and publish something until I
was absolutely sure that my results were above reproach.
The frustrating thing is that there are actually a few people
taking shots at Josh for attempting to clarify this instead of
helping. That helps do nothing except perpetuate the discrepency
between the two positions.
David Ball
<j.ha...@incoming.verizon.net> wrote:
>ftp://holocene.evsc.virginia.edu/pub/MBH98
A clarification on M&M's question 6....
What is the source for your data for series #37 (precipitation in
grid-box 42.5N, 72.5W)? Did you use the data from Jones-Bradley
Paris, France and if so, in which series? More generally, please
provide, identifications of the exact Jones-Bradley locations for each
of the series #21-42. Where are the original source data?
M&M appear to have gotten the first part of the question
screwed up. According to:
ftp://holocene.evsc.virginia.edu/pub/MBH98/INSTR/PREC/prec.loc
1813 11 12.5 82.5 prec-1820-01.dat
1817 11 17.5 72.5 prec-1820-03.dat
1809 11 37.5 282.5 prec-1820-04.dat
1749 11 42.5 2.5 prec-1820-05.dat
1804 11 42.5 7.5 prec-1820-06.dat
1770 11 42.5 287.5 prec-1820-07.dat
1813 11 47.5 2.5 prec-1820-08.dat
1805 11 47.5 12.5 prec-1820-09.dat
1697 11 52.5 12.5 prec-1820-10.dat
1800 11 52.5 357.5 prec-1820-11.dat
1785 11 57.5 352.5 prec-1820-12.dat
series 37 ( prec-1820-07.dat ) corresponds to a lat/lon of 42.5 N
287.5E (72.5W) or which would put it somewhere in North America. (I
don't have an atlas handy so maybe someone could pin-point exactly
where this is). As near as I can tell, it appears to be somewhere
northwest of Boston (42'22N 71'02W). I don't see how they could have
uesd a European series to construct this. The data from series 34
looks to be in the same area.
Indeed the locations of the series correspond to series 32 to
42. The locations of series 21 to 31 can be found in:
ftp://holocene.evsc.virginia.edu/pub/MBH98/INSTR/TEMP/temp.loc
1820 10 42.5 267.5 temp-1820-01.dat
1757 10 47.5 2.5 temp-1820-02.dat
1753 10 47.5 7.5 temp-1820-03.dat
1767 10 47.5 12.5 temp-1820-04.dat
1775 10 47.5 17.5 temp-1820-05.dat
1792 10 52.5 17.5 temp-1820-07.dat
1756 10 57.5 17.5 temp-1820-09.dat
1752 10 57.5 32.5 temp-1820-10.dat
1816 10 62.5 7.5 temp-1820-11.dat
1761 10 62.5 12.5 temp-1820-12.dat
1814 10 62.5 42.5 temp-1820-13.dat
Most of these appear to be in Western Europe, except for the first
one.
Nigel Persaud
Josh, this is an ingenious rationalization.
However, first of all, it is common ground now that MBH deleted the
first 25 years of this series. Prior to M&M, no one knew that. (The
undeleted series are also at ftp/MBH98. Secondly, MBH did not disclose
that they deleted the first 25 years.) The deletion may or may not be
justifiable in climatological terms, but people relying on MBH should
be advised MBH did this without having to parse through original
sources. The deletion should have been annotated in the Supplementary
Information directly and justified. Some readers might not agree with
this approach and are entitled to know. Third, it is far from obvious
that it would be a "mistake" to use the first 25 years. After all, MBH
is using PROXIES. The first 25 years of the historical record are a
better PROXY than some tree ring. Fourth, there is no way of knowing
whether your rationalization is the same as Mann's. The periods which
he deleted in Central England do not match exactly to the periods
which you purport to rationalize. Finally, while you're at this, can
you figure out why MBH used summer versions instead of annual versions
for these series?
Nigel.
David Ball
wrote:
>> 1b. #11 is the Central Europe historical record, originally published
Nigel, you appear desperate to try and rationalize M&M's
position. Why? First of all, the author's are under no obligation to
show all their work. A peer-reviewed paper is not intended for the
masses but for experts in the field. As such, many things may be taken
as given, espeicially since the experts should be familiar with the
reference material used in the study. If I write a paper on imaging
satellite products, I sure as hell don't need to put vast amounts of
filler in explaining how the satellite sensors work. It is assumed
that anyone reading the paper will have the expertise to understand
the basics. The real question is why didn't M&M read the reference
material. Care to answer that one?
Secondly, bad data are routinely thrown out of any analysis. I
do it all the time. Indeed many robust estimation techniques require
that outliers NOT be considered. As long as this is done in a rigorous
fashion it is entirely acceptable. No data were DELETED. They simply
were not used in the analysis. Big difference. Your intentional use of
this particular phrasing is perjorative and unnecessary. ANY analysis
requires that data be pre-processed prior to the analysis being done.
If there is any chance that the data are suspect, they should never be
used.
Thirdly, your claims about the first 25 years of the record
being better are wishful thinking on your part and nothing more. Prove
it! Josh's arguments are entirely appropriate. Better to use a
slightly smaller subset of the data than risk contaminating it. Your
argument has no justification that I can see.
Nigel Persaud
prec-1820-07.dat. This doesn't prove anything about the source data.
MBH98 refers to Bradlye-Jones 1992 as a source but fail to identify
the corresponding locations. M&M note that the correlation between #37
and the precipitation from Bradley-Jones Paris, France (archived at
WDCP) is extraordinarily high and the start years coincide. You've
not dealt with this issue at all and it is still outstanding. Nigel
David Ball <wra...@mb.sympatico.ca> wrote in message news:<n4p0uvgaj8jb102d6...@4ax.com>...
Nigel Persaud
> do it all the time. Indeed many robust estimation techniques require
> that outliers NOT be considered. As long as this is done in a rigorous
> fashion it is entirely acceptable. No data were DELETED. They simply
> were not used in the analysis. Big difference. Your intentional use of
> this particular phrasing is perjorative and unnecessary. ANY analysis
> requires that data be pre-processed prior to the analysis being done.
> If there is any chance that the data are suspect, they should never be
> used.
David, your comments about robust estimation are a reasonable way of
approaching this issue and your phrase "in a rigorous fashion" is a
good criterion. However, there's no indication in MBH98 that they
carried out such a procedure. Even if the publication was directed at
the classes and not the masses, some description is surely required.
Nigel
David Ball
wrote:
>David, the MBH proxy series #37 corresponds to ftp/MBH98 series
David Ball
wrote:
>David, the MBH proxy series #37 corresponds to ftp/MBH98 series
>prec-1820-07.dat. This doesn't prove anything about the source data.
>MBH98 refers to Bradlye-Jones 1992 as a source but fail to identify
>the corresponding locations. M&M note that the correlation between #37
>and the precipitation from Bradley-Jones Paris, France (archived at
>WDCP) is extraordinarily high and the start years coincide. You've
>not dealt with this issue at all and it is still outstanding. Nigel
>
Not according to the latitude/longitude of the points in the
data files. Try not to be so single-minded. Look at the information,
not your dogmatic interpretation of the information. There are a
number of points in the data files that are CLEARLY not located in
Europe.
David Ball
wrote:
>Secondly, bad data are routinely thrown out of any analysis. I
Why? It is relatively easy to see from the reference material
why the data in the early years was deemed suspect. In the absence of
anything suggesting that these data are valid, they should be treated
as suspect, not the other way around. I ask the question again: why
were M&M unable to read the reference material?
David Ball
>Subject: Re: Auditing the auditors #1
>Original Format
>Newsgroups: sci.environment
>Date: 2003-12-15 09:56:27 PST
>
>> >
>> >Your abc breakdown ignores the M&M question about series 65?
>>
>> Josh spent 4 hours getting the information he posted. What are
>> you doing, besides posting "what have you done lately" comments? ...
>
>My question was prompted because I got the impression that Josh was
>attempting to fully address the questions in issue #1. His use of the
>phrase "to get through #1" contributed to that impression. My question
>was politely stated. If you had been asking question to me under similar
>circumstances, it would have been no surprise to see you be quite
>abusive about only seeing abc instead of abcd.
What exactly is preventing you from investigating the question
of series 65? Laziness? Indifference?
>
>> ... Is
>> there anything preventing you from getting off your ass and doing some
>> of the legwork or do you plan to just sit here and complain about
>> someone else's efforts?
>
>There's plenty of legwork to be done, that's for sure. I had done enough
>legwork previously to recognize that Josh was incorrect in stating,
>below, that the chin04.txt and chin04-orig.txt files at ftp site were
>identical. Are you not grateful to have that error corrected?
Most of the legwork seems to stem from M&M's inability to read
reference material, get the proper data in the proper format and a
number of other housekeeping issues. It doesn't seem to have much to
do with Mann et al so far.
As for being grateful, it was nice to have that pointed out. I
wouldn't make too much out of it. I doubt very much it has a lot to do
with anything in the grand scheme of things.
>
>> What I find illuminating about Josh's post is that the basic
>> information about why certain series were truncated is available in
>> the reference material. When the answer to each question posed by M&M
>> boils down to their inability to RTFR, there is a problem. ...
>
>It seems clear enough that Mann et al chose to use subsets of series 10
>and 11. There is no indication that they systematically applied some
>evaluative methodology to all the series. And your suggestion that "the
>answer to each question posed by M&M boils down to their inability to
>RTFR" seems quite an exaggeration.
If they had bothered to read the references, they would have
phrased their questions in a completely different way. Your posturing
on the matter has been noted for some time as has your penchant for
overstating the value of this paper - as one of the most important...
As for retaining/throwing away suspect data, it is done all
the time. In the event that data are suspect, and in this case, the
references clearly show they are suspect, common practice is to throw
them out unless some compelling reason can be found to retain them.
Since there is no way to judge their worth, it makes sense to get rid
of them, but retain as much information as possible, so carving the
series back to the 60's or 70's doesn't make a lot of sense.
>
>> ... That likely
>> stems from the unseemly rush they were in to get something in print. I
>> can't think of a researcher anywhere who can knock off a peer-reviewed
>> paper in a matter of weeks, without taking some serious shortcuts and
>> at least in the case of their question 1, it appears that those
>> shortcuts caused them to overlook basic information needed to do their
>> "audit".
>
>You have often mistaken question marks for exclamation points, so your
>take on the current matter comes as no surprise. Your apparent argument
>that "of course Mann et al modified the source data" seems quite at odds
>with the impression given by all those who have been echoing Mann's
>recent claim that the data necessary to replicate their work is
>available from the referenced sources of the data.
LOL. Spin in any way you want, Steve, but as of now, it's M&M
that are looking pretty bad.
>
>>
>> >>
>> >> Which means that the Manley reconstruction is only continuous from 1722
>> >> on, but the information upon which it relies from 1723-28 has further
>> >> difficulties, essentially absolute values were not reliable, and the
>> >> series was constructed by taking the difference between measurements
>> >> made by those thermometers and ones thought to be more reliable after
>> >> 1727, and then repeatedly differenced to get values before 1727.
>> >>
>> >> In the light of this, it is perfectly reasonable to truncate the CET
>> >> series at 1730. On the other end, 1987 is the cut off for the
>> >> analysis. We note that the cut offs were listed in the NSM.
>> >>
>> >> On balance this illustrates the principal of RTFR (Read the effing
>> >> references) and the danger of someone unfamiliar with an area trying to
>> >> do an "audit". To use the CET before 1730 would clearly have been a
>> >> mistake, and I think you could make a good argument that it should have
>> >> been cut off at 1772.
>> >
>> >Interesting info, Josh. Thanks. From what you've reported, it looks like
>> >some other truncation point would have been more perfect than 1730,
>> >however: 1723, 1728, or somewhere in 1760s. Your use of the root perfect
>> >seems an exaggeration in this case. I agree that the supplemental info
>> >lists 1730, but note that the column is described as showing the "first
>> >year of record available". That does not seem like an accurate
>> >description of the start date used by Mann et al. for this series.
>>
>> And you are basing this on what? ...
>
>On the meaning of the phrase "first year of record available", as
>opposed to the meaning of "first year of record chosen to be included
>although prior years were available".
More spin and posturing, I'm afraid. Only you could take
standard practice and something very innocent and try to make
something nefarious out of it.
>
>> ... Josh's point about the
>> problematic nature of the 1723-1728 data is well taken. ...
>
>I too think it may very well have been reasonable to truncate this
>series. I find it quite silly that you apparently chastise M&M for
>asking the question, however.
Asking the question? Never. I chastise them for their rush to
print because it is a likely cause in their overlooking crucial
information (i.e. reading the references).
>
>> ... Using any of
>> the data in that period would be a mistake. There is nothing
>> inherently wrong with using 1730 - it's a nice round number - and as
>> Josh pointed out other truncation points could have been adopted...
>
>And once its verified what Mann et al actually did, the effects of other
>treatments will be interesting to explore.
We're making progress. Certainly question 2, 3 and 12 have
been put to bed. Question 6 is still a bit of a problem, not because
of the data used but because some of the positions appear to be in the
US. There's nothing inherently wrong with that, although Nigel seems
to feel that isn't likely because some of the series resemble some
European series.
>
>> ...
>> Let's face it, the proxy data are all reconstructed back to nice round
>> numbers. In the tree ring data, why go back to 1700? Why not 1699? Or
>> 1697? It's a computational convenience, nothing more. ...
>
>Are you saying that other time series were truncated by Mann et al
>besides the four mentioned in issue #1?
I'm saying that if you look at all 22 SWM series for example,
they don't all stop at 1700, yet the author's chose 1700 as a
break-point. 1600 was also chosen. It was a convenience. They could
have used 1699, but 1700 is a nice round number.
>
>> ...
>> Your arbitrary choice of a point in the 1760's is noted. ...
>
>I quite purposefully used the precision quoted from the reference by
>Josh. How would you improve upon my choice of language?
1760 is no better or worse than 1730. Again, in any analysis
containing suspect data you want to get of the suspect data, while
retaining as much good data as possible. As Josh said, an argument
could be made that 1770 was a better starting point. The author's
apparently didn't agree.
>
>> ... In
>> addition, your quibbling about the description of 1730 being the first
>> year or record is just that: a quibble. ...
>
>I thought about elaborating, but didn't want to clutter. In the body of
>the Nature paper, Mann et al claim to have included five series in their
>analysis that apparently were not included. It is in the context of such
>apparent sloppiness that M&M's questions warrant clear answers, untinged
>by the insults and invective that have become rote from Mann and his
>apologists.
ROTFL. You've got a couple of hack who knock off a paper,
submit it completely full of errors to a third-rate "journal" then
whine when they are shown to be less than competent and you are
posturing about Mann et al's sloppiness? Too funny.
>
>> ... When an analysis is done, bad
>> data are routinely thrown out. That's a fact of life. The paper in
>> question is NOT meant for the general public but for other experts in
>> the field, experts who persumably are, unlike M&M, aware of the
>> problematic nature of some of the data. That being said, such a
>> notation can be considered a given.
>
>So you think it inappropriate that IPCC used the hocky stick graph in
>Fig 1 of the WG1 TAR policymaker summary?
No, I think it entirely appropriate. Mann et al have done
nothing wrong and I've looked at this far harder than you have. All
you've done is posture and try to spin and frankly it's wearing thin.
As for your perjorative terminology, it's what I've come to expect
from you: no facts and lots of snarkiness.
>> >
>> >M&M ask reasonable questions under the heading of issue #1. Some of what
>> >you write here is illuminatory, some is obfuscatory, and some looks just
>> >snarky itself.
>> >
>> And the reasonable responses given by Josh show that the
>> information M&M are inquiring about was available in the paper's
>> reference material, material that they apparently didn't bother to
>> read. That isn't a snark, nor is it obfuscatory. It is simply the
>> truth. Nor are your minor quibbles adding anything to the discussion,
>> except noise.
>
>I've seen how prone some folks here are to exaggerate.
Look in the mirror for the single worst offender.
> I thought it
>worthwhile to nip some of that in the bud in this case.
See that you don't resort to any more of it and this thread
will remain quite pleasant.
Nigel Persaud
> data files. Try not to be so single-minded. Look at the information,
> not your dogmatic interpretation of the information. There are a
> number of points in the data files that are CLEARLY not located in
> Europe.
>
I agree that ftp/MBH98 locates the series in North America. So what?
ftp/MBH98 is not source data. Can you show where the data in #37 comes
from? Nigel
David Ball
wrote:
>> Not according to the latitude/longitude of the points in the
Huh? That's funny, but we've been crunching the numbers from
that site, comparing M&M's spreadsheet with the numbers from that
site, recalculating the PC's from that site and now that there's some
discrepencies your answer is that it doesn't contain source data? We
know series 37 is prec-1820-07.dat posted to:
ftp://holocene.evsc.virginia.edu/pub/MBH98/INSTR/PREC
and in that directory is a file called:
prec.loc
which contains weighting factors and lat/lon points for the various
series, 2 of which are NOT in Europe, affecting series 34 and 37.
Similarly, for the temperature series, it appears that series 21 is
not located in Europe but in the US midwest. Now, do you have
something specific that precludes these three sites from being in the
US? Give something specific, not just a correlation between...
Steve Schulin
David Ball <wra...@mb.sympatico.ca> wrote:
> >From: Steve Schulin (steve....@nuclear.com)
> >Subject: Re: Auditing the auditors #1
> >View: Complete Thread (18 articles)
> >Original Format
> >Newsgroups: sci.environment
> >Date: 2003-12-15 09:56:27 PST
> >
> >> >
> >> >Your abc breakdown ignores the M&M question about series 65?
> >>
> >> Josh spent 4 hours getting the information he posted. What are
> >> you doing, besides posting "what have you done lately" comments? ...
> >
> >My question was prompted because I got the impression that Josh was
> >attempting to fully address the questions in issue #1. His use of the
> >phrase "to get through #1" contributed to that impression. My question
> >was politely stated. If you had been asking question to me under similar
> >circumstances, it would have been no surprise to see you be quite
> >abusive about only seeing abc instead of abcd.
>
> What exactly is preventing you from investigating the question
> of series 65? Laziness? Indifference?
Priorities.
>
> >
> >> ... Is
> >> there anything preventing you from getting off your ass and doing some
> >> of the legwork or do you plan to just sit here and complain about
> >> someone else's efforts?
> >
> >There's plenty of legwork to be done, that's for sure. I had done enough
> >legwork previously to recognize that Josh was incorrect in stating,
> >below, that the chin04.txt and chin04-orig.txt files at ftp site were
> >identical. Are you not grateful to have that error corrected?
>
> Most of the legwork seems to stem from M&M's inability to read
> reference material, get the proper data in the proper format and a
> number of other housekeeping issues. It doesn't seem to have much to
> do with Mann et al so far.
The world already knows much more about Mann et al's methodology,
because of McIntyre and McKitrick's efforts, than it knew for the 5
years after Nature published the article. Your exclusive praise for
Josh's efforts is quite a contrast to your exclusive derision of M&M's.
At this point, we don't actually even know what data Mann et al used.
McIntyre, using the error-filled pcproxy.txt file, without corrections,
came up with a graph quite similar to the Mann et al hockey stick
product. I wouldn't be surprised if Mann's peers let him avoid ever
publicly providing enough info to allow his work to be replicable.
> As for being grateful, it was nice to have that pointed out. I
> wouldn't make too much out of it. I doubt very much it has a lot to do
> with anything in the grand scheme of things.
That reminds me of a comment made by one of the relief pitchers of my
beloved Philadelphia Phillies: "Ten million years from now, when the sun
burns out and the Earth is just a frozen iceball hurtling through space,
nobody's going to care whether or not I got this guy out."
> >> What I find illuminating about Josh's post is that the basic
> >> information about why certain series were truncated is available in
> >> the reference material. When the answer to each question posed by M&M
> >> boils down to their inability to RTFR, there is a problem. ...
> >
> >It seems clear enough that Mann et al chose to use subsets of series 10
> >and 11. There is no indication that they systematically applied some
> >evaluative methodology to all the series. And your suggestion that "the
> >answer to each question posed by M&M boils down to their inability to
> >RTFR" seems quite an exaggeration.
>
> If they had bothered to read the references, they would have
> phrased their questions in a completely different way. Your posturing
> on the matter has been noted for some time as has your penchant for
> overstating the value of this paper - as one of the most important...
That was quite appropriate phrase to use as title of that
thread-starting message. The initial post contained excerpts from a
hot-off-the-press op ed by paleoclimate researcher Bob Carter:
* " ... it is clear that the McIntyre and McKitrick paper is set to
become one of the most important that has been published in recent
years. Apart from the light that it throws on the climate debate, the
paper raises profound issues to do with the integrity of scientific
publication, how data which underpins published papers should be
archived and made available, and whether science advice given to
governments on policy matters should be rigorously audited."
* "... unlike Mann, Bradley and Hughes for their 1998 paper, McIntyre
and McKitrick have made full disclosure of all the assumptions made and
techniques used in their manipulation of the data, have posted the data
they used on a freely-accessible website, and have invited other
scientists to comment on or check their conclusions."
* "Mann, Bradley and Hughes used complex statistical averaging methods
to combine no fewer than 112 such proxy records into an inferred climate
curve for 1400-1902. Controversially, for the period between 1902 and
1980 they then spliced on an averaged curve of actual thermometer
temperature measurements. The result was a combined curve which showed
little evidence of either the Mediaeval Warm Period or the Little Ice
Age and a dramatic upturn to higher temperatures after 1900 and
continuing to 1980. Thus was born the famous 'hockey stick curve' of
recent climate change. Though it was immediately adopted as the received
truth by global warming lobbyists, many scientists were sharply critical
of the conclusions of Mann, Bradley and Hughes. Critics pointed out that
the graph was based on datasets which were heavily manipulated
statistically and, in combining at the year 1902 two datasets of
different derivation, Mann, Bradley and Hughes had transgressed good
statistical practice."
* "In an initial response, Mann has asserted that the dataset which he
provided McIntyre and McKitrick is not the same dataset he used in his
Nature paper, and anyway contains errors. Such a response does not
inspire confidence in Mann's other work, and, anyway, McIntyre and
McKitrick used for their analysis a recompiled, accurate dataset. It
will obviously be some time before the argument is concluded and the
dust settles."
* "Australia should consider following Denmark's example and set up a
science audit unit to verify the soundness of advice the federal
government receives. Such an audit unit can be funded with the money
saved by closing down the Australian Greenhouse Office."
[Source: Bob Carter (prof geology, Marine Geophysical Laboratory, James
Cook U), "Global Warming May Turn Out To Be Just Hot Air", Australian
Financial Review, November 3, 2003, p. 71]
> As for retaining/throwing away suspect data, it is done all
> the time. In the event that data are suspect, and in this case, the
> references clearly show they are suspect, common practice is to throw
> them out unless some compelling reason can be found to retain them.
> Since there is no way to judge their worth, it makes sense to get rid
> of them, but retain as much information as possible, so carving the
> series back to the 60's or 70's doesn't make a lot of sense.
In the 1998 Nature paper, Mann et al note what looks to be a general
principle: "... the mutual information contained in a diverse and widely
distributed set of independent climate indicators can more faithfully
capture the consistent climate signal that is present, reducing the
compromising effects of biases and weaknesses in the individual
indicators." Finding reason to drop cold-year data from Little Ice Age
and warm-year data from prior begs the question of whether all the
series were treated the same.
> >> ... That likely
> >> stems from the unseemly rush they were in to get something in print. I
> >> can't think of a researcher anywhere who can knock off a peer-reviewed
> >> paper in a matter of weeks, without taking some serious shortcuts and
> >> at least in the case of their question 1, it appears that those
> >> shortcuts caused them to overlook basic information needed to do their
> >> "audit".
> >
> >You have often mistaken question marks for exclamation points, so your
> >take on the current matter comes as no surprise. Your apparent argument
> >that "of course Mann et al modified the source data" seems quite at odds
> >with the impression given by all those who have been echoing Mann's
> >recent claim that the data necessary to replicate their work is
> >available from the referenced sources of the data.
>
> LOL. Spin in any way you want, Steve, but as of now, it's M&M
> that are looking pretty bad.
They have raised many interesting points.
And they have discovered facts which prompt other interesting points,
too. Take the five series (fran003, ital015, ital015x, spai026, and
spai047) that were listed amongst the proxies used in the 1998 paper but
which were not amongst the pcproxy.txt file. M&M ask Mann et al if those
series were in fact used. Mann hasn't answered that specifically yet,
but he does say that all the data for the 1998 paper is available in the
newly disclosed ftp site. Those five series don't appear in the
directories at the ftp site. I'm curious as to how this matter plays
out. If it turns out that these series were not used, I'm sure I'm not
the only one who would be interested to learn the particulars as to why
they are listed. Did Mann et al intend to use them originally? If so,
what changed their mind?
Having seen you often misrepresent stuff, I can understand why you find
such to be standard practice. Mann et al wrote their own paper. Nobody
forced them to use that word "available" in describing the column.
> >> ... Josh's point about the
> >> problematic nature of the 1723-1728 data is well taken. ...
> >
> >I too think it may very well have been reasonable to truncate this
> >series. I find it quite silly that you apparently chastise M&M for
> >asking the question, however.
>
> Asking the question? Never. I chastise them for their rush to
> print because it is a likely cause in their overlooking crucial
> information (i.e. reading the references).
As it turns out, we're lucky that M&M did exactly what they did, and
that Mann reacted as he did. Otherwise, we wouldn't have been able to
contemplate on the sight of Mann's newly disclosed ftp site being
vacuumed like Hillary Clinton's former law firm's files were.
> >> ... Using any of
> >> the data in that period would be a mistake. There is nothing
> >> inherently wrong with using 1730 - it's a nice round number - and as
> >> Josh pointed out other truncation points could have been adopted...
> >
> >And once its verified what Mann et al actually did, the effects of other
> >treatments will be interesting to explore.
>
> We're making progress. Certainly question 2, 3 and 12 have
> been put to bed. ...
The recent revelations by Mann in response to M&M have already enabled
much to be known that was not known before.
> ... Question 6 is still a bit of a problem, not because
> of the data used but because some of the positions appear to be in the
> US. There's nothing inherently wrong with that, although Nigel seems
> to feel that isn't likely because some of the series resemble some
> European series.
> >> ...
> >> Let's face it, the proxy data are all reconstructed back to nice round
> >> numbers. In the tree ring data, why go back to 1700? Why not 1699? Or
> >> 1697? It's a computational convenience, nothing more. ...
> >
> >Are you saying that other time series were truncated by Mann et al
> >besides the four mentioned in issue #1?
>
> I'm saying that if you look at all 22 SWM series for example,
> they don't all stop at 1700, yet the author's chose 1700 as a
> break-point. 1600 was also chosen. It was a convenience. They could
> have used 1699, but 1700 is a nice round number.
If that's what you were saying, then it wasn't analogous to the
truncation issue.
> >> ...
> >> Your arbitrary choice of a point in the 1760's is noted. ...
> >
> >I quite purposefully used the precision quoted from the reference by
> >Josh. How would you improve upon my choice of language?
>
> 1760 is no better or worse than 1730. ...
Neither one warrants the root "perfect" in his context, fer sure.
> ... Again, in any analysis
> containing suspect data you want to get of the suspect data, while
> retaining as much good data as possible. As Josh said, an argument
> could be made that 1770 was a better starting point. ...
He said 1772. I agree with him.
> ... The author's
> apparently didn't agree.
Nor did they think it appropriate to give any insight into whether they
subjected all the series to the same criteria.
> >> ... In
> >> addition, your quibbling about the description of 1730 being the first
> >> year or record is just that: a quibble. ...
> >
> >I thought about elaborating, but didn't want to clutter. In the body of
> >the Nature paper, Mann et al claim to have included five series in their
> >analysis that apparently were not included. It is in the context of such
> >apparent sloppiness that M&M's questions warrant clear answers, untinged
> >by the insults and invective that have become rote from Mann and his
> >apologists.
>
> ROTFL. You've got a couple of hack who knock off a paper,
> submit it completely full of errors to a third-rate "journal" then
> whine when they are shown to be less than competent and you are
> posturing about Mann et al's sloppiness? Too funny.
I'm happy to disagree with every bit of your opinion on this.
> >> ... When an analysis is done, bad
> >> data are routinely thrown out. That's a fact of life. The paper in
> >> question is NOT meant for the general public but for other experts in
> >> the field, experts who persumably are, unlike M&M, aware of the
> >> problematic nature of some of the data. That being said, such a
> >> notation can be considered a given.
> >
> >So you think it inappropriate that IPCC used the hocky stick graph in
> >Fig 1 of the WG1 TAR policymaker summary?
>
> No, I think it entirely appropriate. ...
LOL - you just got through saying the paper "is NOT meant for the
general public but for other experts in the field".
> ... Mann et al have done
> nothing wrong and I've looked at this far harder than you have. ...
But you're blinded by your bias, having long ago drank the Mann-made
Kool-aid. You may not even recognize when you're displaying how little
regard you have for the scientific method.
> ... All
> you've done is posture and try to spin and frankly it's wearing thin.
> As for your perjorative terminology, it's what I've come to expect
> from you: no facts and lots of snarkiness.
I've been quite restrained in my comments about Mann et al 1998 and most
of its apologists. Probably too easy-going on some aspects so far.
>
>
> >> >
> >> >M&M ask reasonable questions under the heading of issue #1. Some of what
> >> >you write here is illuminatory, some is obfuscatory, and some looks just
> >> >snarky itself.
> >> >
> >> And the reasonable responses given by Josh show that the
> >> information M&M are inquiring about was available in the paper's
> >> reference material, material that they apparently didn't bother to
> >> read. That isn't a snark, nor is it obfuscatory. It is simply the
> >> truth. Nor are your minor quibbles adding anything to the discussion,
> >> except noise.
> >
> >I've seen how prone some folks here are to exaggerate.
>
> Look in the mirror for the single worst offender.
There you go exaggerating again.
>
> > I thought it
> >worthwhile to nip some of that in the bud in this case.
>
> See that you don't resort to any more of it and this thread
> will remain quite pleasant.
The thread included unpleasantness before I joined in. I can understand
your nostalgia, however, for the unpleasantness in which you revel.
Very truly,
David Ball
<steve....@nuclear.com> wrote:
>In article <dg02uvc3ele66v652...@4ax.com>,
> David Ball <wra...@mb.sympatico.ca> wrote:
>
>> >From: Steve Schulin (steve....@nuclear.com)
>> >Subject: Re: Auditing the auditors #1
>> >View: Complete Thread (18 articles)
>> >Original Format
>> >Newsgroups: sci.environment
>> >Date: 2003-12-15 09:56:27 PST
>> >
>> >> >
>> >> >Your abc breakdown ignores the M&M question about series 65?
>> >>
>> >> Josh spent 4 hours getting the information he posted. What are
>> >> you doing, besides posting "what have you done lately" comments? ...
>> >
>> >My question was prompted because I got the impression that Josh was
>> >attempting to fully address the questions in issue #1. His use of the
>> >phrase "to get through #1" contributed to that impression. My question
>> >was politely stated. If you had been asking question to me under similar
>> >circumstances, it would have been no surprise to see you be quite
>> >abusive about only seeing abc instead of abcd.
>>
>> What exactly is preventing you from investigating the question
>> of series 65? Laziness? Indifference?
>
>Priorities.
Incompetence more likely.
>>
>> Most of the legwork seems to stem from M&M's inability to read
>> reference material, get the proper data in the proper format and a
>> number of other housekeeping issues. It doesn't seem to have much to
>> do with Mann et al so far.
>
>The world already knows much more about Mann et al's methodology,
>because of McIntyre and McKitrick's efforts, than it knew for the 5
>years after Nature published the article.
No, people in this forum are farther ahead today because of
people like Josh who are doing the legwork necessary to correct M&M's
screw-ups. M&M haven't clarified anything, especially since they were
looking at the wrong bloody data.
>Your exclusive praise for
>Josh's efforts is quite a contrast to your exclusive derision of M&M's.
>At this point, we don't actually even know what data Mann et al used.
>McIntyre, using the error-filled pcproxy.txt file, without corrections,
>came up with a graph quite similar to the Mann et al hockey stick
>product. I wouldn't be surprised if Mann's peers let him avoid ever
>publicly providing enough info to allow his work to be replicable.
Incorrect, I'm afraid. You can't use bad data, jiggle it and
then claim that it resembles the original author's work. I looked very
hard and I don't recall seeing strong warming during the LIA, which
shows up on M&M's "analysis".
>
>> As for being grateful, it was nice to have that pointed out. I
>> wouldn't make too much out of it. I doubt very much it has a lot to do
>> with anything in the grand scheme of things.
>
>That reminds me of a comment made by one of the relief pitchers of my
>beloved Philadelphia Phillies: "Ten million years from now, when the sun
>burns out and the Earth is just a frozen iceball hurtling through space,
>nobody's going to care whether or not I got this guy out."
LOL. No, it comes from knowing what PCA techniques do, Steve,
and realizing that two years of missing data are doing to do jack.
>
>> >> What I find illuminating about Josh's post is that the basic
>> >> information about why certain series were truncated is available in
>> >> the reference material. When the answer to each question posed by M&M
>> >> boils down to their inability to RTFR, there is a problem. ...
>> >
>> >It seems clear enough that Mann et al chose to use subsets of series 10
>> >and 11. There is no indication that they systematically applied some
>> >evaluative methodology to all the series. And your suggestion that "the
>> >answer to each question posed by M&M boils down to their inability to
>> >RTFR" seems quite an exaggeration.
>>
>> If they had bothered to read the references, they would have
>> phrased their questions in a completely different way. Your posturing
>> on the matter has been noted for some time as has your penchant for
>> overstating the value of this paper - as one of the most important...
>
>That was quite appropriate phrase to use as title of that
>thread-starting message. The initial post contained excerpts from a
>hot-off-the-press op ed by paleoclimate researcher Bob Carter:
>
<silly op-ed deleted>
Sorry, Steve, M&M's paper is a mess and anything based on it
is necessarily flawed. I wonder why Mr. Carter didn't notice the
warming that M&M showed during the LIA. That should have set off alarm
bells for anyone with an ounce of understanding of the problem.
>
>> As for retaining/throwing away suspect data, it is done all
>> the time. In the event that data are suspect, and in this case, the
>> references clearly show they are suspect, common practice is to throw
>> them out unless some compelling reason can be found to retain them.
>> Since there is no way to judge their worth, it makes sense to get rid
>> of them, but retain as much information as possible, so carving the
>> series back to the 60's or 70's doesn't make a lot of sense.
>
>In the 1998 Nature paper, Mann et al note what looks to be a general
>principle: "... the mutual information contained in a diverse and widely
>distributed set of independent climate indicators can more faithfully
>capture the consistent climate signal that is present, reducing the
>compromising effects of biases and weaknesses in the individual
>indicators." Finding reason to drop cold-year data from Little Ice Age
>and warm-year data from prior begs the question of whether all the
>series were treated the same.
Apparently, you didn't understand what I said to you. Quality
control of any data MUST be done. It is a fact of life. If there is a
suspicion that data are compromised, YOU DON'T USE THEM! Pretty
straight forward, eh?
>
>> >> ... That likely
>> >> stems from the unseemly rush they were in to get something in print. I
>> >> can't think of a researcher anywhere who can knock off a peer-reviewed
>> >> paper in a matter of weeks, without taking some serious shortcuts and
>> >> at least in the case of their question 1, it appears that those
>> >> shortcuts caused them to overlook basic information needed to do their
>> >> "audit".
>> >
>> >You have often mistaken question marks for exclamation points, so your
>> >take on the current matter comes as no surprise. Your apparent argument
>> >that "of course Mann et al modified the source data" seems quite at odds
>> >with the impression given by all those who have been echoing Mann's
>> >recent claim that the data necessary to replicate their work is
>> >available from the referenced sources of the data.
>>
>> LOL. Spin in any way you want, Steve, but as of now, it's M&M
>> that are looking pretty bad.
>
>They have raised many interesting points.
Really? From a screwed up data set? How do you figure? Aren't
you the one who often preaches garbage in, garbage out? A wonderful
example of it here. Does this mean that Mann et al are perfect? I
doubt it.
>
>And they have discovered facts which prompt other interesting points,
>too. Take the five series (fran003, ital015, ital015x, spai026, and
>spai047) that were listed amongst the proxies used in the 1998 paper but
>which were not amongst the pcproxy.txt file. M&M ask Mann et al if those
>series were in fact used. Mann hasn't answered that specifically yet,
>but he does say that all the data for the 1998 paper is available in the
>newly disclosed ftp site.
You mean the ftp site that was available BEFORE M&M ever
started working on their paper?
>Those five series don't appear in the
>directories at the ftp site. I'm curious as to how this matter plays
>out. If it turns out that these series were not used, I'm sure I'm not
>the only one who would be interested to learn the particulars as to why
>they are listed. Did Mann et al intend to use them originally? If so,
>what changed their mind?
I don't know. The answer will be interesting.
>>
>> More spin and posturing, I'm afraid. Only you could take
>> standard practice and something very innocent and try to make
>> something nefarious out of it.
>
>Having seen you often misrepresent stuff, I can understand why you find
>such to be standard practice. Mann et al wrote their own paper. Nobody
>forced them to use that word "available" in describing the column.
Posturing again, Steve. You haven't got a clue what the
analysis is doing, but you're absolutely sure that it's something bad.
When told that it's pretty common, it becomes a grand conspiracy. Your
comments would carry a wee bit more weight if you could understand
what was happening.
>
>> >> ... Josh's point about the
>> >> problematic nature of the 1723-1728 data is well taken. ...
>> >
>> >I too think it may very well have been reasonable to truncate this
>> >series. I find it quite silly that you apparently chastise M&M for
>> >asking the question, however.
>>
>> Asking the question? Never. I chastise them for their rush to
>> print because it is a likely cause in their overlooking crucial
>> information (i.e. reading the references).
>
>As it turns out, we're lucky that M&M did exactly what they did, and
>that Mann reacted as he did. Otherwise, we wouldn't have been able to
>contemplate on the sight of Mann's newly disclosed ftp site being
>vacuumed like Hillary Clinton's former law firm's files were.
LOL. Paul Farrar has already pointed out that the ftp site in
question was available long ago as have I. Your ridiculous attempts to
show it otherwise are quite pathetic.
>
>> >> ... Using any of
>> >> the data in that period would be a mistake. There is nothing
>> >> inherently wrong with using 1730 - it's a nice round number - and as
>> >> Josh pointed out other truncation points could have been adopted...
>> >
>> >And once its verified what Mann et al actually did, the effects of other
>> >treatments will be interesting to explore.
>>
>> We're making progress. Certainly question 2, 3 and 12 have
>> been put to bed. ...
>
>The recent revelations by Mann in response to M&M have already enabled
>much to be known that was not known before.
Yeah verily, and Schulin laid his holy message upon the
peasants of usenet and he saw that all was good.
>
>> ... Question 6 is still a bit of a problem, not because
>> of the data used but because some of the positions appear to be in the
>> US. There's nothing inherently wrong with that, although Nigel seems
>> to feel that isn't likely because some of the series resemble some
>> European series.
>> >> ...
>> >> Let's face it, the proxy data are all reconstructed back to nice round
>> >> numbers. In the tree ring data, why go back to 1700? Why not 1699? Or
>> >> 1697? It's a computational convenience, nothing more. ...
>> >
>> >Are you saying that other time series were truncated by Mann et al
>> >besides the four mentioned in issue #1?
>>
>> I'm saying that if you look at all 22 SWM series for example,
>> they don't all stop at 1700, yet the author's chose 1700 as a
>> break-point. 1600 was also chosen. It was a convenience. They could
>> have used 1699, but 1700 is a nice round number.
>
>If that's what you were saying, then it wasn't analogous to the
>truncation issue.
I never said it was. I said that there is a certain amount of
fluidity available in the data (i.e. analyst preferences) and choosing
break-points is up to them.
>
>> >> ...
>> >> Your arbitrary choice of a point in the 1760's is noted. ...
>> >
>> >I quite purposefully used the precision quoted from the reference by
>> >Josh. How would you improve upon my choice of language?
>>
>> 1760 is no better or worse than 1730. ...
>
>Neither one warrants the root "perfect" in his context, fer sure.
But both are vastly superior to using the flawed data prior to
1730. Live with it, unless you are willing to show a detailed analysis
of your own backing up your contention that they should have been
included.
>
>> ... Again, in any analysis
>> containing suspect data you want to get of the suspect data, while
>> retaining as much good data as possible. As Josh said, an argument
>> could be made that 1770 was a better starting point. ...
>
>He said 1772. I agree with him.
Now back it up. Your opinion matters little. Josh stated that
it was arguable whether 1772 was a better date. Fine. That statement I
can agree with. Your absolute statement has zero to back it up.
>
>> ... The author's
>> apparently didn't agree.
>
>Nor did they think it appropriate to give any insight into whether they
>subjected all the series to the same criteria.
Spin again.
>
>> >> ... In
>> >> addition, your quibbling about the description of 1730 being the first
>> >> year or record is just that: a quibble. ...
>> >
>> >I thought about elaborating, but didn't want to clutter. In the body of
>> >the Nature paper, Mann et al claim to have included five series in their
>> >analysis that apparently were not included. It is in the context of such
>> >apparent sloppiness that M&M's questions warrant clear answers, untinged
>> >by the insults and invective that have become rote from Mann and his
>> >apologists.
>>
>> ROTFL. You've got a couple of hack who knock off a paper,
>> submit it completely full of errors to a third-rate "journal" then
>> whine when they are shown to be less than competent and you are
>> posturing about Mann et al's sloppiness? Too funny.
>
>I'm happy to disagree with every bit of your opinion on this.
The facts state otherwise. Live with it.
>
>> >> ... When an analysis is done, bad
>> >> data are routinely thrown out. That's a fact of life. The paper in
>> >> question is NOT meant for the general public but for other experts in
>> >> the field, experts who persumably are, unlike M&M, aware of the
>> >> problematic nature of some of the data. That being said, such a
>> >> notation can be considered a given.
>> >
>> >So you think it inappropriate that IPCC used the hocky stick graph in
>> >Fig 1 of the WG1 TAR policymaker summary?
>>
>> No, I think it entirely appropriate. ...
>
>LOL - you just got through saying the paper "is NOT meant for the
>general public but for other experts in the field".
The TAR is a summary of the science. Big difference, fool.
>
>> ... Mann et al have done
>> nothing wrong and I've looked at this far harder than you have. ...
>
>But you're blinded by your bias, having long ago drank the Mann-made
>Kool-aid. You may not even recognize when you're displaying how little
>regard you have for the scientific method.
ROTFL. Spoken like someone who decided his preferences a long
time ago. The facts say otherwise, but you've never let facts get in
the way before.
>>
>> > I thought it
>> >worthwhile to nip some of that in the bud in this case.
>>
>> See that you don't resort to any more of it and this thread
>> will remain quite pleasant.
>
>The thread included unpleasantness before I joined in. I can understand
>your nostalgia, however, for the unpleasantness in which you revel.
>
You're the one trolling, Steve. Don't blame me for catching
you at again.
David Ball
I finally found some time to go through a PCA analysis of
Mann's data in a fairly rigorous fashion. As I mentioned in an earlier
post, I downloaded their data, preprocessed it to get common starting
points as they did and then ran it through a completely different
algorithm. I didn't bother filling in the missing years at the end of
each series as MBH just used persistence and the PCA technique would
likely treat this as noise. I'd feel more comfortable extrapolating
values using polynomial or rational functions, but that's a lot of
work for a couple of points, at least in the basic series I've
processed so far.
Given the slightly different datasets and the different
algorithm - using Jacobi rotations to iterate toward the correct
eigenvalues and vectors - there are slight differences in the output
EOF's and eigenvalues in most of the cases I've looked at, but the
results are quite similar. The main difference between Mann et al and
my routine is in the number of PC's retained. Since I'm using Kaiser's
criterion - retaining only those PC's having eigenvalues greater than
or equal to 1 - there are differences. For the OK series in /STAHLE/OK
I retain 3 PC's as well as for the SOAMER series in /ITRDB/SOAMER. The
SWM series I retain far few PC's - only 4 - but there are a number
with eigenvalues hovering just below 1 so this slight difference is
not surprising. For the Australia series, the reverse happens. Where
MBH retained 3 PC's I actually retain 6, but the last 3 are hovering
just above 1.0. From what I've seen so far, there is not a huge
difference, certainly not the kind of difference hinted at by M&M.
M&M were not very honest with their Table 7 on page 762 of E&E
when they didn't disclose that the number of PC's listed in MBH were
THE NUMBER RETAINED and not the number of PC's. There should be a PC
for each dataset, but only a subset of them are retained. They make it
appear as if Mann et al were doing something wrong and they weren't.
That is the whole point of doing the PCA analysis in the first place:
to reduce the dimensionality of the problem. In addition, the 6% of
the variance explained that they list for the SWM series is clearly
wrong as is their 76%. That is way too high, but they were using
corrupt data. MBH's numbers are much different and agree within a few
percent of mine. I haven't finished processing the NOAMER data yet - I
didn't malloc an array large enough and got a segmentation fault when
I attempted to read in the array. I'll fix that up tomorrow. I'm also
going to compare some of Mann's PC's with my own and hopefully with
some of M&M's and plot them up. It will be interesting to see what
shakes out when I plot them.
Steve Schulin
David Ball <wra...@mb.sympatico.ca> wrote:
> On Thu, 18 Dec 2003 10:00:38 -0500, Steve Schulin
> <steve....@nuclear.com> wrote:
>
> >In article <dg02uvc3ele66v652...@4ax.com>,
> > David Ball <wra...@mb.sympatico.ca> wrote:
> >
> >> >From: Steve Schulin (steve....@nuclear.com)
> >> >Subject: Re: Auditing the auditors #1
> >> >View: Complete Thread (18 articles)
> >> >Original Format
> >> >Newsgroups: sci.environment
> >> >Date: 2003-12-15 09:56:27 PST
> >> >
> >> >> >
> >> >> >Your abc breakdown ignores the M&M question about series 65?
> >> >>
> >> >> Josh spent 4 hours getting the information he posted. What are
> >> >> you doing, besides posting "what have you done lately" comments? ...
> >> >
> >> >My question was prompted because I got the impression that Josh was
> >> >attempting to fully address the questions in issue #1. His use of the
> >> >phrase "to get through #1" contributed to that impression. My question
> >> >was politely stated. If you had been asking question to me under similar
> >> >circumstances, it would have been no surprise to see you be quite
> >> >abusive about only seeing abc instead of abcd.
> >>
> >> What exactly is preventing you from investigating the question
> >> of series 65? Laziness? Indifference?
> >
> >Priorities.
>
> Incompetence more likely.
You're free to speculate all you want, bub. What particular competency
do you imagine would be needed to investigate the of series 65?
BTW, from the many quite obvious errors you make in this post, and some
not for the first time, it's clear enough that you aren't yet interested
in discovering whether Mann et al 1998 is replicable as published.
> >> Most of the legwork seems to stem from M&M's inability to read
> >> reference material, get the proper data in the proper format and a
> >> number of other housekeeping issues. It doesn't seem to have much to
> >> do with Mann et al so far.
> >
> >The world already knows much more about Mann et al's methodology,
> >because of McIntyre and McKitrick's efforts, than it knew for the 5
> >years after Nature published the article.
>
> No, people in this forum are farther ahead today because of
> people like Josh who are doing the legwork necessary to correct M&M's
> screw-ups. M&M haven't clarified anything, especially since they were
> looking at the wrong bloody data.
If M&M hadn't published, and Mann hadn't responded, how would Josh know
to look for 159 columns-worth of the so-called multiproxy series?
>
> >Your exclusive praise for
> >Josh's efforts is quite a contrast to your exclusive derision of M&M's.
> >At this point, we don't actually even know what data Mann et al used.
> >McIntyre, using the error-filled pcproxy.txt file, without corrections,
> >came up with a graph quite similar to the Mann et al hockey stick
> >product. I wouldn't be surprised if Mann's peers let him avoid ever
> >publicly providing enough info to allow his work to be replicable.
>
> Incorrect, I'm afraid. You can't use bad data, jiggle it and
> then claim that it resembles the original author's work. I looked very
> hard and I don't recall seeing strong warming during the LIA, which
> shows up on M&M's "analysis".
You're ballocksing things up in confusion. I agree that there's a big
difference between the graphs to which you refer. However, I was
speaking of yet a different graph. i was quite clear in describing it.
> >> As for being grateful, it was nice to have that pointed out. I
> >> wouldn't make too much out of it. I doubt very much it has a lot to do
> >> with anything in the grand scheme of things.
> >
> >That reminds me of a comment made by one of the relief pitchers of my
> >beloved Philadelphia Phillies: "Ten million years from now, when the sun
> >burns out and the Earth is just a frozen iceball hurtling through space,
> >nobody's going to care whether or not I got this guy out."
>
> LOL. No, it comes from knowing what PCA techniques do, Steve,
> and realizing that two years of missing data are doing to do jack.
Your peculiar juxtaposition of condescension and misstatement is quite
typical of you. The series we have discussed with two years of missing
data were not PC series.
>
> >
> >> >> What I find illuminating about Josh's post is that the basic
> >> >> information about why certain series were truncated is available in
> >> >> the reference material. When the answer to each question posed by M&M
> >> >> boils down to their inability to RTFR, there is a problem. ...
> >> >
> >> >It seems clear enough that Mann et al chose to use subsets of series 10
> >> >and 11. There is no indication that they systematically applied some
> >> >evaluative methodology to all the series. And your suggestion that "the
> >> >answer to each question posed by M&M boils down to their inability to
> >> >RTFR" seems quite an exaggeration.
> >>
> >> If they had bothered to read the references, they would have
> >> phrased their questions in a completely different way. Your posturing
> >> on the matter has been noted for some time as has your penchant for
> >> overstating the value of this paper - as one of the most important...
> >
> >That was quite appropriate phrase to use as title of that
> >thread-starting message. The initial post contained excerpts from a
> >hot-off-the-press op ed by paleoclimate researcher Bob Carter:
> >
> <silly op-ed deleted>
>
> Sorry, Steve, M&M's paper is a mess and anything based on it
> is necessarily flawed. I wonder why Mr. Carter didn't notice the
> warming that M&M showed during the LIA. That should have set off alarm
> bells for anyone with an ounce of understanding of the problem.
Of all the mistakes you make in this exchange, it's this one that really
displays how incredibly out-of-touch with reality you are. McIntyre and
McKitrick don't claim that Mann's method will result in a useful
product.
> >> As for retaining/throwing away suspect data, it is done all
> >> the time. In the event that data are suspect, and in this case, the
> >> references clearly show they are suspect, common practice is to throw
> >> them out unless some compelling reason can be found to retain them.
> >> Since there is no way to judge their worth, it makes sense to get rid
> >> of them, but retain as much information as possible, so carving the
> >> series back to the 60's or 70's doesn't make a lot of sense.
> >
> >In the 1998 Nature paper, Mann et al note what looks to be a general
> >principle: "... the mutual information contained in a diverse and widely
> >distributed set of independent climate indicators can more faithfully
> >capture the consistent climate signal that is present, reducing the
> >compromising effects of biases and weaknesses in the individual
> >indicators." Finding reason to drop cold-year data from Little Ice Age
> >and warm-year data from prior begs the question of whether all the
> >series were treated the same.
>
> Apparently, you didn't understand what I said to you. Quality
> control of any data MUST be done. It is a fact of life. If there is a
> suspicion that data are compromised, YOU DON'T USE THEM! Pretty
> straight forward, eh?
You and i apparently agree that Mann et al decided not to use some of
the data in some of the series. You seem to think it not necessary to
document (nor bother to answer questions about) what was decided to be
dropped and what criteria was used in coming to that decision.
> >> >> ... That likely
> >> >> stems from the unseemly rush they were in to get something in print. I
> >> >> can't think of a researcher anywhere who can knock off a peer-reviewed
> >> >> paper in a matter of weeks, without taking some serious shortcuts and
> >> >> at least in the case of their question 1, it appears that those
> >> >> shortcuts caused them to overlook basic information needed to do their
> >> >> "audit".
> >> >
> >> >You have often mistaken question marks for exclamation points, so your
> >> >take on the current matter comes as no surprise. Your apparent argument
> >> >that "of course Mann et al modified the source data" seems quite at odds
> >> >with the impression given by all those who have been echoing Mann's
> >> >recent claim that the data necessary to replicate their work is
> >> >available from the referenced sources of the data.
> >>
> >> LOL. Spin in any way you want, Steve, but as of now, it's M&M
> >> that are looking pretty bad.
> >
> >They have raised many interesting points.
>
> Really? From a screwed up data set? How do you figure? ...
For starters, I happily refer you back to the op ed by paleoclimate
researcher Bob Carter. You remember, the one which included
" ... it is clear that the McIntyre and McKitrick paper is set to
become one of the most important that has been published in recent
years. Apart from the light that it throws on the climate debate, the
paper raises profound issues to do with the integrity of scientific
publication, how data which underpins published papers should be
archived and made available, and whether science advice given to
governments on policy matters should be rigorously audited."
> ... Aren't
> you the one who often preaches garbage in, garbage out? ...
No. Perhaps you're misconstruing my oft-repeated theme that, for
purposes of public policy, climate models are of dubious predictive
value.
> ... A wonderful
> example of it here. Does this mean that Mann et al are perfect? I
> doubt it.
I can't imagine you not doubting it, given that the garbage to which you
appear to be referring was produced and distributed by Mann or his
designee.
> >And they have discovered facts which prompt other interesting points,
> >too. Take the five series (fran003, ital015, ital015x, spai026, and
> >spai047) that were listed amongst the proxies used in the 1998 paper but
> >which were not amongst the pcproxy.txt file. M&M ask Mann et al if those
> >series were in fact used. Mann hasn't answered that specifically yet,
> >but he does say that all the data for the 1998 paper is available in the
> >newly disclosed ftp site.
>
> You mean the ftp site that was available BEFORE M&M ever
> started working on their paper?
It was apparently so "available" that nary a single googled web page or
usenet post referenced it as of the day of Mann's recent disclosure.
> >Those five series don't appear in the
> >directories at the ftp site. I'm curious as to how this matter plays
> >out. If it turns out that these series were not used, I'm sure I'm not
> >the only one who would be interested to learn the particulars as to why
> >they are listed. Did Mann et al intend to use them originally? If so,
> >what changed their mind?
>
> I don't know. The answer will be interesting.
>
> >> More spin and posturing, I'm afraid. Only you could take
> >> standard practice and something very innocent and try to make
> >> something nefarious out of it.
> >
> >Having seen you often misrepresent stuff, I can understand why you find
> >such to be standard practice. Mann et al wrote their own paper. Nobody
> >forced them to use that word "available" in describing the column.
>
> Posturing again, Steve. You haven't got a clue what the
> analysis is doing, but you're absolutely sure that it's something bad.
> When told that it's pretty common, it becomes a grand conspiracy. ...
What are you babbling about? Does it in any way relate to the matter of
whether or not Mann et al were accurate in describing the contents of
the column?
> ... Your
> comments would carry a wee bit more weight if you could understand
> what was happening.
I understand enough to recognize at least some of your ignorances.
> >> >> ... Josh's point about the
> >> >> problematic nature of the 1723-1728 data is well taken. ...
> >> >
> >> >I too think it may very well have been reasonable to truncate this
> >> >series. I find it quite silly that you apparently chastise M&M for
> >> >asking the question, however.
> >>
> >> Asking the question? Never. I chastise them for their rush to
> >> print because it is a likely cause in their overlooking crucial
> >> information (i.e. reading the references).
> >
> >As it turns out, we're lucky that M&M did exactly what they did, and
> >that Mann reacted as he did. Otherwise, we wouldn't have been able to
> >contemplate on the sight of Mann's newly disclosed ftp site being
> >vacuumed like Hillary Clinton's former law firm's files were.
>
> LOL. Paul Farrar has already pointed out that the ftp site in
> question was available long ago as have I. Your ridiculous attempts to
> show it otherwise are quite pathetic.
I recall his claims as less than persuasive, especially given
Rutherford's description of the existance of several less than
comprehensive ftp sites. As for whatever activities of your own you're
referring to here, I confess to not recalling them at all. I'll keep an
eye out for details if I ever have the misfortune to review old posts of
yours.
> >> >> ... Using any of
> >> >> the data in that period would be a mistake. There is nothing
> >> >> inherently wrong with using 1730 - it's a nice round number - and as
> >> >> Josh pointed out other truncation points could have been adopted...
> >> >
> >> >And once its verified what Mann et al actually did, the effects of other
> >> >treatments will be interesting to explore.
> >>
> >> We're making progress. Certainly question 2, 3 and 12 have
> >> been put to bed. ...
> >
> >The recent revelations by Mann in response to M&M have already enabled
> >much to be known that was not known before.
>
> Yeah verily, and Schulin laid his holy message upon the
> peasants of usenet and he saw that all was good.
It's, uh, very interesting to see such fruits of your cognitive
workings. Despite your character flaws and whatnot, I'd like to stress
my hope that you aren't seriously mentally ill.
>
> >
> >> ... Question 6 is still a bit of a problem, not because
> >> of the data used but because some of the positions appear to be in the
> >> US. There's nothing inherently wrong with that, although Nigel seems
> >> to feel that isn't likely because some of the series resemble some
> >> European series.
> >> >> ...
> >> >> Let's face it, the proxy data are all reconstructed back to nice round
> >> >> numbers. In the tree ring data, why go back to 1700? Why not 1699? Or
> >> >> 1697? It's a computational convenience, nothing more. ...
> >> >
> >> >Are you saying that other time series were truncated by Mann et al
> >> >besides the four mentioned in issue #1?
> >>
> >> I'm saying that if you look at all 22 SWM series for example,
> >> they don't all stop at 1700, yet the author's chose 1700 as a
> >> break-point. 1600 was also chosen. It was a convenience. They could
> >> have used 1699, but 1700 is a nice round number.
> >
> >If that's what you were saying, then it wasn't analogous to the
> >truncation issue.
>
> I never said it was. I said that there is a certain amount of
> fluidity available in the data (i.e. analyst preferences) and choosing
> break-points is up to them.
We were discussing truncation. Your "computational convenience" line of
argument seems pretty irrelevant to the truncated series.
> >> >> ...
> >> >> Your arbitrary choice of a point in the 1760's is noted. ...
> >> >
> >> >I quite purposefully used the precision quoted from the reference by
> >> >Josh. How would you improve upon my choice of language?
> >>
> >> 1760 is no better or worse than 1730. ...
> >
> >Neither one warrants the root "perfect" in his context, fer sure.
>
> But both are vastly superior to using the flawed data prior to
> 1730. Live with it, unless you are willing to show a detailed analysis
> of your own backing up your contention that they should have been
> included.
All the series were from work by other researchers. Mann et al. did not
hint at any systematic effort to independently evaluate the series for
internal quality. Are you not curious as to whether Mann et al applied
truncation criteria in systematic fashion?
>
> >
> >> ... Again, in any analysis
> >> containing suspect data you want to get of the suspect data, while
> >> retaining as much good data as possible. As Josh said, an argument
> >> could be made that 1770 was a better starting point. ...
> >
> >He said 1772. I agree with him.
>
> Now back it up. Your opinion matters little. Josh stated that
> it was arguable whether 1772 was a better date. Fine. That statement I
> can agree with. Your absolute statement has zero to back it up.
LOL - you sure are confused. My absolute statements, including "I agree
with him" seems clear enough. It was your claim (your claim that that
Josh said 1770) that I was contrasting with the other, BTW.
> >> ... The author's
> >> apparently didn't agree.
> >
> >Nor did they think it appropriate to give any insight into whether they
> >subjected all the series to the same criteria.
>
> Spin again.
Spinning a web of truth, as I attempt, is preferable to the purposeful
lies of yours.
> >> >> ... In
> >> >> addition, your quibbling about the description of 1730 being the first
> >> >> year or record is just that: a quibble. ...
> >> >
> >> >I thought about elaborating, but didn't want to clutter. In the body of
> >> >the Nature paper, Mann et al claim to have included five series in their
> >> >analysis that apparently were not included. It is in the context of such
> >> >apparent sloppiness that M&M's questions warrant clear answers, untinged
> >> >by the insults and invective that have become rote from Mann and his
> >> >apologists.
> >>
> >> ROTFL. You've got a couple of hack who knock off a paper,
> >> submit it completely full of errors to a third-rate "journal" then
> >> whine when they are shown to be less than competent and you are
> >> posturing about Mann et al's sloppiness? Too funny.
> >
> >I'm happy to disagree with every bit of your opinion on this.
>
> The facts state otherwise. Live with it.
It's you who has been ignoring the material facts in this case, Mr. Ball.
> >> >> ... When an analysis is done, bad
> >> >> data are routinely thrown out. That's a fact of life. The paper in
> >> >> question is NOT meant for the general public but for other experts in
> >> >> the field, experts who persumably are, unlike M&M, aware of the
> >> >> problematic nature of some of the data. That being said, such a
> >> >> notation can be considered a given.
> >> >
> >> >So you think it inappropriate that IPCC used the hocky stick graph in
> >> >Fig 1 of the WG1 TAR policymaker summary?
> >>
> >> No, I think it entirely appropriate. ...
> >
> >LOL - you just got through saying the paper "is NOT meant for the
> >general public but for other experts in the field".
>
> The TAR is a summary of the science. Big difference, fool.
I'm sure you have a point in mind, but it's not clear what it could be.
I can understand why you think of foolishness shortly after calling the
TAR a summary. It brings to mind your own revealed foolishness in the
thread some time ago where it was appropriate to remind you that the A
in TAR stands for 'Assessment', not 'Abridgement'.
By the way, I notice that the IPCC's acronym for Fourth Assessment
Report is AR4. I'm not trying to infer that you don't already know this,
but I don't think it's been mentioned here in usenet yet, and perhaps
someone else will be glad to know this.
> >> ... Mann et al have done
> >> nothing wrong and I've looked at this far harder than you have. ...
> >
> >But you're blinded by your bias, having long ago drank the Mann-made
> >Kool-aid. You may not even recognize when you're displaying how little
> >regard you have for the scientific method.
>
> ROTFL. Spoken like someone who decided his preferences a long
> time ago. The facts say otherwise, but you've never let facts get in
> the way before.
I've never tried to hide behind a veneer of objectivity. I'm quite open
to learning from others who are interested in climate. I've even learned
stuff about weather and climate from you, although your signal-to-noise
ratio is pretty poor. Fact is, I know that you are nowhere near the
worst of the lying bastards promoting notions similar to your expression
of "the horror" of "what we've done" climate-wise. Not even at your
worst have I seen you say something as outrageous as the comment by Karl
and Trenberth in recent Science: "In the absence of climate mitigation
policies, the 90% probability interval for warming from 1990 to 2100 is
1.7? to 4.9?C."
>
> >>
> >> > I thought it
> >> >worthwhile to nip some of that in the bud in this case.
> >>
> >> See that you don't resort to any more of it and this thread
> >> will remain quite pleasant.
> >
> >The thread included unpleasantness before I joined in. I can understand
> >your nostalgia, however, for the unpleasantness in which you revel.
> >
> You're the one trolling, Steve. Don't blame me for catching
> you at again.
I've forgotten your peculiar definition of trolling. What do you mean
here?
Very truly,
David Ball
<steve....@nuclear.com> wrote:
>> >
>> >Priorities.
>>
>> Incompetence more likely.
>
>You're free to speculate all you want, bub. What particular competency
>do you imagine would be needed to investigate the of series 65?
You tell us.
>
>BTW, from the many quite obvious errors you make in this post, and some
>not for the first time, it's clear enough that you aren't yet interested
>in discovering whether Mann et al 1998 is replicable as published.
LOL. That's why I'm re-doing Mann et al from the ground up,
including re-doing their PCA calculations. Given that you lack that
wit to even understand what Mann is doing and lack the integrity to
acknowledge that M&M's amatuerish efforts, with a corrupt dataset no
less, show little of significance, it's pretty hard to take you
seriously.
>
>> >> Most of the legwork seems to stem from M&M's inability to read
>> >> reference material, get the proper data in the proper format and a
>> >> number of other housekeeping issues. It doesn't seem to have much to
>> >> do with Mann et al so far.
>> >
>> >The world already knows much more about Mann et al's methodology,
>> >because of McIntyre and McKitrick's efforts, than it knew for the 5
>> >years after Nature published the article.
>>
>> No, people in this forum are farther ahead today because of
>> people like Josh who are doing the legwork necessary to correct M&M's
>> screw-ups. M&M haven't clarified anything, especially since they were
>> looking at the wrong bloody data.
>
>If M&M hadn't published, and Mann hadn't responded, how would Josh know
>to look for 159 columns-worth of the so-called multiproxy series?
If M&M hadn't published, the media wouldn't have gotten in a
lather about a "study" that shows nothing.
>>
>> >Your exclusive praise for
>> >Josh's efforts is quite a contrast to your exclusive derision of M&M's.
>> >At this point, we don't actually even know what data Mann et al used.
>> >McIntyre, using the error-filled pcproxy.txt file, without corrections,
>> >came up with a graph quite similar to the Mann et al hockey stick
>> >product. I wouldn't be surprised if Mann's peers let him avoid ever
>> >publicly providing enough info to allow his work to be replicable.
>>
>> Incorrect, I'm afraid. You can't use bad data, jiggle it and
>> then claim that it resembles the original author's work. I looked very
>> hard and I don't recall seeing strong warming during the LIA, which
>> shows up on M&M's "analysis".
>
>You're ballocksing things up in confusion. I agree that there's a big
>difference between the graphs to which you refer. However, I was
>speaking of yet a different graph. i was quite clear in describing it.
Garbage in, garbage out, Steve.
>
>> >> As for being grateful, it was nice to have that pointed out. I
>> >> wouldn't make too much out of it. I doubt very much it has a lot to do
>> >> with anything in the grand scheme of things.
>> >
>> >That reminds me of a comment made by one of the relief pitchers of my
>> >beloved Philadelphia Phillies: "Ten million years from now, when the sun
>> >burns out and the Earth is just a frozen iceball hurtling through space,
>> >nobody's going to care whether or not I got this guy out."
>>
>> LOL. No, it comes from knowing what PCA techniques do, Steve,
>> and realizing that two years of missing data are doing to do jack.
>
>Your peculiar juxtaposition of condescension and misstatement is quite
>typical of you. The series we have discussed with two years of missing
>data were not PC series.
ROTFL. Posturing again, I see. Two years OF ANY DATA, are not
going to affect the results. Hell, they fill in missing data at the
end of some of the series with persistence values. No result there
either.
>> >>
>> >> If they had bothered to read the references, they would have
>> >> phrased their questions in a completely different way. Your posturing
>> >> on the matter has been noted for some time as has your penchant for
>> >> overstating the value of this paper - as one of the most important...
>> >
>> >That was quite appropriate phrase to use as title of that
>> >thread-starting message. The initial post contained excerpts from a
>> >hot-off-the-press op ed by paleoclimate researcher Bob Carter:
>> >
>> <silly op-ed deleted>
>>
>> Sorry, Steve, M&M's paper is a mess and anything based on it
>> is necessarily flawed. I wonder why Mr. Carter didn't notice the
>> warming that M&M showed during the LIA. That should have set off alarm
>> bells for anyone with an ounce of understanding of the problem.
>
>Of all the mistakes you make in this exchange, it's this one that really
>displays how incredibly out-of-touch with reality you are. McIntyre and
>McKitrick don't claim that Mann's method will result in a useful
>product.
LOL. It is M&M's own graphs that show strong warming during
the LIA. Mann's does not. The LIA was characterized by COLD
conditions. That should have set of alarm bells for M&M, especially
since a number of other researchers have built upon Mann's results, up
to and including a recent JofC article. In addition, vast numbers of
other papers have discussed the LIA. Given the problems M&M
encountered with the input data and the obvious output problem, the
author's should have said, "Oh, oh, we've got a problem with our input
data". Instead, they bulled forward in an attempt to make Mann et al
look bad and ended up with egg on their faces.
>
>> >> As for retaining/throwing away suspect data, it is done all
>> >> the time. In the event that data are suspect, and in this case, the
>> >> references clearly show they are suspect, common practice is to throw
>> >> them out unless some compelling reason can be found to retain them.
>> >> Since there is no way to judge their worth, it makes sense to get rid
>> >> of them, but retain as much information as possible, so carving the
>> >> series back to the 60's or 70's doesn't make a lot of sense.
>> >
>> >In the 1998 Nature paper, Mann et al note what looks to be a general
>> >principle: "... the mutual information contained in a diverse and widely
>> >distributed set of independent climate indicators can more faithfully
>> >capture the consistent climate signal that is present, reducing the
>> >compromising effects of biases and weaknesses in the individual
>> >indicators." Finding reason to drop cold-year data from Little Ice Age
>> >and warm-year data from prior begs the question of whether all the
>> >series were treated the same.
>>
>> Apparently, you didn't understand what I said to you. Quality
>> control of any data MUST be done. It is a fact of life. If there is a
>> suspicion that data are compromised, YOU DON'T USE THEM! Pretty
>> straight forward, eh?
>
>You and i apparently agree that Mann et al decided not to use some of
>the data in some of the series. You seem to think it not necessary to
>document (nor bother to answer questions about) what was decided to be
>dropped and what criteria was used in coming to that decision.
Not for the masses, no.
>> >>
>> >> LOL. Spin in any way you want, Steve, but as of now, it's M&M
>> >> that are looking pretty bad.
>> >
>> >They have raised many interesting points.
>>
>> Really? From a screwed up data set? How do you figure? ...
>
>For starters, I happily refer you back to the op ed by paleoclimate
>researcher Bob Carter.
Geologist, I thought.
>You remember, the one which included
>" ... it is clear that the McIntyre and McKitrick paper is set to
>become one of the most important that has been published in recent
>years. Apart from the light that it throws on the climate debate, the
>paper raises profound issues to do with the integrity of scientific
>publication, how data which underpins published papers should be
>archived and made available, and whether science advice given to
>governments on policy matters should be rigorously audited."
It addresses none of those points, but feel free to use your
vivid imagination any way you want.
>
>> ... Aren't
>> you the one who often preaches garbage in, garbage out? ...
>
>No. Perhaps you're misconstruing my oft-repeated theme that, for
>purposes of public policy, climate models are of dubious predictive
>value.
LOL. Wouldn't it have been simpler just to admit that you've
said it instead of dissembling?
>
>> ... A wonderful
>> example of it here. Does this mean that Mann et al are perfect? I
>> doubt it.
>
>I can't imagine you not doubting it, given that the garbage to which you
>appear to be referring was produced and distributed by Mann or his
>designee.
Funny, I've found no garbage on their website. It looks very
good.
>
>> >And they have discovered facts which prompt other interesting points,
>> >too. Take the five series (fran003, ital015, ital015x, spai026, and
>> >spai047) that were listed amongst the proxies used in the 1998 paper but
>> >which were not amongst the pcproxy.txt file. M&M ask Mann et al if those
>> >series were in fact used. Mann hasn't answered that specifically yet,
>> >but he does say that all the data for the 1998 paper is available in the
>> >newly disclosed ftp site.
>>
>> You mean the ftp site that was available BEFORE M&M ever
>> started working on their paper?
>
>It was apparently so "available" that nary a single googled web page or
>usenet post referenced it as of the day of Mann's recent disclosure.
Paul found it. I found it and it took just a couple of
minutes. I guess M&M just weren't trying very hard.
>> >
>> >Having seen you often misrepresent stuff, I can understand why you find
>> >such to be standard practice. Mann et al wrote their own paper. Nobody
>> >forced them to use that word "available" in describing the column.
>>
>> Posturing again, Steve. You haven't got a clue what the
>> analysis is doing, but you're absolutely sure that it's something bad.
>> When told that it's pretty common, it becomes a grand conspiracy. ...
>
>What are you babbling about? Does it in any way relate to the matter of
>whether or not Mann et al were accurate in describing the contents of
>the column?
It is standard practice to throw out suspect data in any
analysis, Steve. Think you can grasp that simple concept? You've only
been told repeatedly.
>
>> ... Your
>> comments would carry a wee bit more weight if you could understand
>> what was happening.
>
>I understand enough to recognize at least some of your ignorances.
So when can we expect your audit of Mann? LOL.
>> >As it turns out, we're lucky that M&M did exactly what they did, and
>> >that Mann reacted as he did. Otherwise, we wouldn't have been able to
>> >contemplate on the sight of Mann's newly disclosed ftp site being
>> >vacuumed like Hillary Clinton's former law firm's files were.
>>
>> LOL. Paul Farrar has already pointed out that the ftp site in
>> question was available long ago as have I. Your ridiculous attempts to
>> show it otherwise are quite pathetic.
>
>I recall his claims as less than persuasive, especially given
>Rutherford's description of the existance of several less than
>comprehensive ftp sites. As for whatever activities of your own you're
>referring to here, I confess to not recalling them at all. I'll keep an
>eye out for details if I ever have the misfortune to review old posts of
>yours.
More posturing. Is that all you can do?
>> >The recent revelations by Mann in response to M&M have already enabled
>> >much to be known that was not known before.
>>
>> Yeah verily, and Schulin laid his holy message upon the
>> peasants of usenet and he saw that all was good.
>
>It's, uh, very interesting to see such fruits of your cognitive
>workings. Despite your character flaws and whatnot, I'd like to stress
>my hope that you aren't seriously mentally ill.
I'm fine. Thanks for asking. And you? How's life in
Schulin-land. Still the cesspool it's always been?
>> >
>> >If that's what you were saying, then it wasn't analogous to the
>> >truncation issue.
>>
>> I never said it was. I said that there is a certain amount of
>> fluidity available in the data (i.e. analyst preferences) and choosing
>> break-points is up to them.
>
>We were discussing truncation. Your "computational convenience" line of
>argument seems pretty irrelevant to the truncated series.
On the contrary. As Josh pointed out, an argument COULD have
been made that the series should have been truncated in the 1770's.
You claim 1760 - without anything to support your POV naturally -
would have been better. The author's clearly thought a third thing and
acted upon that. It was their choice to make and since you have been
unable to show anything wrong with that choice other than you dislike
absolutely everything to do with the paper, without having to get into
the touchy area of actually understanding it, their choice is the one
that matters. Your hubris doesn't enter into the picture anywhere.
>> But both are vastly superior to using the flawed data prior to
>> 1730. Live with it, unless you are willing to show a detailed analysis
>> of your own backing up your contention that they should have been
>> included.
>
>All the series were from work by other researchers. Mann et al. did not
>hint at any systematic effort to independently evaluate the series for
>internal quality. Are you not curious as to whether Mann et al applied
>truncation criteria in systematic fashion?
With series of this type, it would be impossible to apply
systematic criteria to all the series. They need to be evaluated one
by one.
>
>>
>> >
>> >> ... Again, in any analysis
>> >> containing suspect data you want to get of the suspect data, while
>> >> retaining as much good data as possible. As Josh said, an argument
>> >> could be made that 1770 was a better starting point. ...
>> >
>> >He said 1772. I agree with him.
>>
>> Now back it up. Your opinion matters little. Josh stated that
>> it was arguable whether 1772 was a better date. Fine. That statement I
>> can agree with. Your absolute statement has zero to back it up.
>
>LOL - you sure are confused. My absolute statements, including "I agree
>with him" seems clear enough. It was your claim (your claim that that
>Josh said 1770) that I was contrasting with the other, BTW.
I could have sworn you said: From what you've reported, it
looks like some other truncation point would have been more perfect
than 1730, however: 1723, 1728, or somewhere in 1760s. An absolute
statement, Steve.
>> >> ... The author's
>> >> apparently didn't agree.
>> >
>> >Nor did they think it appropriate to give any insight into whether they
>> >subjected all the series to the same criteria.
>>
>> Spin again.
>
>Spinning a web of truth, as I attempt, is preferable to the purposeful
>lies of yours.
LOL. Steve, you wouldn't know the truth if it walked up and
bit you on the ass. Lies, now. Those are your specialty and you're
very good at telling them.
>
>> >> >> ... In
>> >> >> addition, your quibbling about the description of 1730 being the first
>> >> >> year or record is just that: a quibble. ...
>> >> >
>> >> >I thought about elaborating, but didn't want to clutter. In the body of
>> >> >the Nature paper, Mann et al claim to have included five series in their
>> >> >analysis that apparently were not included. It is in the context of such
>> >> >apparent sloppiness that M&M's questions warrant clear answers, untinged
>> >> >by the insults and invective that have become rote from Mann and his
>> >> >apologists.
>> >>
>> >> ROTFL. You've got a couple of hack who knock off a paper,
>> >> submit it completely full of errors to a third-rate "journal" then
>> >> whine when they are shown to be less than competent and you are
>> >> posturing about Mann et al's sloppiness? Too funny.
>> >
>> >I'm happy to disagree with every bit of your opinion on this.
>>
>> The facts state otherwise. Live with it.
>
>It's you who has been ignoring the material facts in this case, Mr. Ball.
Please enlighten me, Perfesser. And please, no nonsense about
how M&M's results show what Mann's results would have been if they'd
done their analysis properly.
>
>> >> ... Mann et al have done
>> >> nothing wrong and I've looked at this far harder than you have. ...
>> >
>> >But you're blinded by your bias, having long ago drank the Mann-made
>> >Kool-aid. You may not even recognize when you're displaying how little
>> >regard you have for the scientific method.
>>
>> ROTFL. Spoken like someone who decided his preferences a long
>> time ago. The facts say otherwise, but you've never let facts get in
>> the way before.
>
>I've never tried to hide behind a veneer of objectivity.
Nor one of honesty.
> I'm quite open
>to learning from others who are interested in climate. I've even learned
>stuff about weather and climate from you, although your signal-to-noise
>ratio is pretty poor. Fact is, I know that you are nowhere near the
>worst of the lying bastards promoting notions similar to your expression
>of "the horror" of "what we've done" climate-wise. Not even at your
>worst have I seen you say something as outrageous as the comment by Karl
>and Trenberth in recent Science: "In the absence of climate mitigation
>policies, the 90% probability interval for warming from 1990 to 2100 is
>1.7? to 4.9?C."
Actually, the worst lying bastard in this forum, is you. Have
a good look in the mirror.
David Ball
You mentioned some time back that you'd found a problem with
the Vaganov series principle components. Do you recall what they were?
I've finished processing all the PC's for Mann's data and have
retrieved all of the remaining proxy datasets from their website. I
haven't found any problems with the PC's, though my method retains
many more than Mann does (10 in the case of the Vaganov series). I
just want to make sure that I haven't overlooked anything in the
meantime.
Josh Halpern
problem today which probably will require my learning principal
component analysis....Never say never.
Josh Halpern
(It's also a very busy time for me so posting is slow)
Josh Halpern
Nigel Persaud wrote:
Bradley? (Actually Jones and Bradley) as in Mann, Bradley and Hughes.
josh halpern
Josh Halpern
Nigel Persaud wrote:
SNIP....
>>Again, On balance this illustrates the principal of RTFR and the danger
>>of someone unfamiliar with an area trying to do an "audit". To use the
>>central European index before 1550 would clearly have been a mistake.
>>Note also that Bradley was one of the two book editors, so he surely
>>knew a great deal about these series. Phil Jones was the other.
>>
>>
>
>
>Josh, this is an ingenious rationalization.
>
>However, first of all, it is common ground now that MBH deleted the
>first 25 years of this series. Prior to M&M, no one knew that. (The
>undeleted series are also at ftp/MBH98. Secondly, MBH did not disclose
>that they deleted the first 25 years.)
>
How about anyone who was familiar with the two historical series, and
BTW, by stating the years of the Manley and Pfister records that they
used MBH did disclose. The only people who would not know are those not
in the field (Manley and Pfister are very famous records) and those who
did not know should have read the references.
>The deletion may or may not be
>justifiable in climatological terms, but people relying on MBH should
>be advised MBH did this without having to parse through original
>sources. The deletion should have been annotated in the Supplementary
>Information directly and justified. Some readers might not agree with
>this approach and are entitled to know. Third, it is far from obvious
>that it would be a "mistake" to use the first 25 years. After all, MBH
>is using PROXIES. The first 25 years of the historical record are a
>better PROXY than some tree ring. Fourth, there is no way of knowing
>whether your rationalization is the same as Mann's. The periods which
>he deleted in Central England do not match exactly to the periods
>which you purport to rationalize. Finally, while you're at this, can
>you figure out why MBH used summer versions instead of annual versions
>for these series?
>
>
An underlying issue is that M&M did their reconstruction using the full
record. However, as we have seen this is almost certainly a bad choice
and would therefore make their reconstruction less accurate.
josh halpern
Nigel Persaud
> >I agree that ftp/MBH98 locates the series in North America. So what?
> >ftp/MBH98 is not source data. Can you show where the data in #37 comes
> >from? Nigel
> >
>
> Bradley? (Actually Jones and Bradley) as in Mann, Bradley and Hughes.
>
Can you identify which series in Jones and Bradley?
David Ball
<j.ha...@incoming.verizon.net> wrote:
>Thanks David, this is well beyond me, although I ran into a strange
>problem today which probably will require my learning principal
>component analysis....Never say never.
>
>Josh Halpern
>
>(It's also a very busy time for me so posting is slow)
I understand completely. I finished processing the NOAMER data
and retain WAAAY more PC's than Mann et al, 40 to be exact, though
many are hovering just above the noise level.
I'm going take a stab at a complete reconstruction a-la Mann
et al, using my input data and see what I come up with. In inspecting
some of the PC's there are some noticable differences, primarily at
the ends of the series. At the start of some series, the PC's show
more interannual variability, while it is difficult to gauge how the
ends of the series will impact the consolidated series because I've
made no attempt at all to fill in the missing years pre-1980 with
persistence values.
Nigel Persaud
and so there was little correlation between the ftp/MBH98 PC2 and my
PC2 and I think I may have posted that up. But I went back and
compared the PC2 and PC3 and those correlated.
In my calculations, as I mentioned before, I find discrepancies
between the NOAMER, SOAMER and AUSTRAL PCs and ftp/MBH98 PCs, while
the Vaganov and Stahle tie together much closer. Nigel
David Ball <wra...@mb.sympatico.ca> wrote in message news:<mnl7uvo5jlr3p6fdh...@4ax.com>...
David Ball
wrote:
>David, when I re-did the Vaganov PCs, the order of some PCs flipped
>and so there was little correlation between the ftp/MBH98 PC2 and my
>PC2 and I think I may have posted that up. But I went back and
>compared the PC2 and PC3 and those correlated.
>
>In my calculations, as I mentioned before, I find discrepancies
>between the NOAMER, SOAMER and AUSTRAL PCs and ftp/MBH98 PCs, while
>the Vaganov and Stahle tie together much closer. Nigel
>
I've found a lot of differences, but those, I think, stem from
the different approach I'm using to determining them. My eigenvalues
are very similar, though the first two PC's from the AUSTRAL series
are much smaller, and the percentage of variance explained is very
close - 17% to 16%. Visual inspection of the PC's is not going to tell
much, however, so I have to complete the entire MBH procedure to see
what can be seen. Since I've not used any fills I should be able to
address some of Mr. Schulin's concerns directly.
Joshua Halpern
No, not at this point, Frankly I have not looked, but I can identify
the Bradley, which is why I think this issue is almost certainly a
non-issue. It is somewhat amusing the way Mann has been picked on,
Bradley and Hughes are if anything more prominent and associated with
better known climate research organizations than Mann, who was a
post-doc at the time of publication.
Kind of like Rasool and Schneider V2.1 :)
josh halpern
Nigel Persaud
> the Bradley, which is why I think this issue is almost certainly a
> non-issue. It is somewhat amusing the way Mann has been picked on,
> Bradley and Hughes are if anything more prominent and associated with
> better known climate research organizations than Mann, who was a
> post-doc at the time of publication.
>
> Kind of like Rasool and Schneider V2.1 :)
>
> josh halpern
Then this problem is still outstanding.
David Ball
MBH used as their 1080+ series? I've downloaded the data, truncated it
into the 1902-1995 period but need to select a subset. MBH are not
clear how this is done and M&M merely refer to data file containing
the points which I haven't been able to locate.
David Ball
of series appearing in series 21 to 42 come from "Climate Since AD
1500" Collected reconstructions, 500 Years, Bradley and Jones 1992.
Subsets can be viewed at:
ftp://ftp.ngdc.noaa.gov/paleo/climate1500ad/
The readme file:
ftp://ftp.ngdc.noaa.gov/paleo/climate1500ad/readme_climate1500ad.txt
details what each file in the directory contains. Pay special
attention to the file ch13.txt. The readme calls the file CH13.DAT so
I'm not sure if the two are one and the same. In that ch13.txt file
are a list of stations that are used and you can clearly see that
there are stations in North America - New Haven, Toronto and a site in
Minnesota. Now, getting from these data to the series listed in MBH
will take a little work, but I don't think it there is any requirement
that the Bradley and Jones series be in Europe. It would be
interesting to see if series 37 in the M&M spreadsheet could be
reconstructed from these data so that we could pinpoint exactly if
there is an error.
Josh Halpern
identify which files are used by the starting dates.
Josh Halpern
Nigel Persaud
and related series, which discusses the start dates, correlations of
Bradley-Jones series in much more detail than you've got to so far? It
has exact references to the WDCP locations. Nigel
Nigel Persaud
http://www.climate2003.com/scripts/read.mann.txt. If you go to
http://www.climate2003.com/computations.html, M&M also give
description of calculations to recover Jones data and a script in R to
do it. It's a pretty reasonable description. M&M do much more than
merely refer to the data file; they give you a script for getting it.
What more do you want? Nigel
David Ball <wra...@mb.sympatico.ca> wrote in message news:<4e4fuv003ccemmbh7...@4ax.com>...
David Ball
wrote:
>M&M have a complete description of downloading Mann data: see
>http://www.climate2003.com/scripts/read.mann.txt. If you go to
>http://www.climate2003.com/computations.html, M&M also give
>description of calculations to recover Jones data and a script in R to
>do it. It's a pretty reasonable description. M&M do much more than
>merely refer to the data file; they give you a script for getting it.
>What more do you want? Nigel
Already done. Neither program specifies WHAT or HOW the subset
of the CRU data are selected. In
http://www.climate2003.com/scripts/read.jones.txt
the process is done by:
#PART 2 - TRUNCATING, SCALING AND LAT-WEIGHTING JONES DATA PER MBH98
RECIPE
#this requires prior downloading of Mann data from his FTP site. See
"read.mann"
base.out<-"c:/climate/data/mann"
#LOAD MANN GRIDPOINTS
load("c:/climate/data/mann/gridpoints.tab"); #loads grid2 with
grid2[,5] being 1082 Jones grid-box IDs
LOOK AT THE CODE ABOVE. I'M LOOKING THE 1082 JONES GRID-BOX ID'S.
tempmann<-sort(grid2[,5])##loads Mann grid-boxes as sorted Jones ID
numbers
#LOAD RAW JONES DATA AND MAKE 1902-95 SUBSET FOR MANN
load("c:/climate/data/jones/hadcruv.tab") ###month 553 is 1902 Jan;
v<-array(v,dim=c(dim(v)[1],36*72));####this puts the time series for
grid box (i,j) into column 36*(i-1)+(j-1) and takes about 5 seconds
#truncate to Mann period of 1902-1995
n1<-1995-1901; ##this is period described in MBH98,786
period<-(1901-1855)*12+1:(n1*12) ##this selects month-index for 1902
Jan to 1995 Dec in JOnes v-matrix #length 1128
v<-v[period,]## this truncates v to 1902-1995 dim=1128x2592
dim(v); ##1128x2592
#truncate to Mann's 1082 cells
v<-v[,tempmann]
dim(v) # 1128 x 1082
M&M, for all their griping about Mann et al disclosing key
aspects of their analysis, do a very poor job of their own. Why aren't
ALL the relevent files made available on M&M's website? Where is their
full disclosure?
Nigel Persaud
> of the CRU data are selected.
>
That's up to Mann to explain, not M&M. I'm not aware of any
explanation by MBH of the selection of the 1082 grid-boxes. Nigel
David Ball
wrote:
>> Already done. Neither program specifies WHAT or HOW the subset
On the contrary. M&M have taken MBH to task for not disclosing
their methodology. Why haven't M&M come clean with theirs. What's
sauce for the goose, after all. It's pretty hypocritical to say one
thing then turn around and do the very thing that you accuse others
of.
Josh Halpern
I've changed the subject line, to reflect which of M&Ms questions in
www.climate2003.com is being discussed. Some abbreviations are used in
this series of posts which are explained at the bottom. Because this
post contains a table, the reader would do well to view it with an
equispaced font. It may also be more that you want to know, in which
case arbitrary truncation, or a nice glass of wine are recommended.
Nigel Persaud wrote:
Well yes, and we will do so in two sections. First today we will
discuss the long instrumental data station temperature series which were
used. We will IDENTIFY the series, and where they can be found in these
two posts. The inquiry from M&M is
***********************
6. What is the source for your data for series #37 (precipitation
in grid-box 42.5N, 72.5W)? Did you use the data from Jones-Bradley
Paris, France and if so, in which series? More generally, please
provide, identifications of the exact Jones-Bradley locations for each
of the series #21-42. Where are the original source data?
***********************
One finds 13 such temperature series in ...../pub/MBH98/INSTR/TEMP, with
file names temp-1820-XX.dat These files are referred to in the NSM
listing of MBH98 (see below for abbreviations used). Two of these
thirteen files, #06 and #08 are not used and are not listed in the NSM
or pcproxy. The reference given is to a chapter by P. Jones and R.S.
Bradley (the B of MBH, a professor at UMass and M. Mann's postdoctoral
advisor at UMass) "Climatic variations in the longest instrumental
records," in a book edited by them "Climate Since AD 1500, pp 246-268
(Chapter 13).
As David Ball discovered, the data files for the climate temperature
records used in Chapter 13 are on the noaa paleoclimate site in a file
ftp://ftp.ngdc.noaa.gov/paleo/climate1500ad/ch13.txt. The seasonal
anomolies in oC that are listed are for
A. Seasonal anomalies (øC) for:
1761 1980 TRONDHEIM
1756 1980 STOCKHOLM
1701 1980 CENTRAL ENGLAND
1753 1980 GENEVA
1701 1980 BERLIN
1743 1980 LENINGRAD
1831 1980 SVERDLOVSK
1838 1980 BARNAUL
1778 1980 TORONTO
1832 1981 SITKA
1781 1970 NEW HAVEN
18201982 MINNESOTA
of these TRONDHEIM(12), STOCKHOLM (09), GENEVA (03), BERLIN (07),
LENINGRAD (10), and MINNESOTA (01) are used in MBH98 The numbers in
parentheses refer to the number in the NSM and /pub/MBH98/INSTR/TEMP. I
suspect that the data files in this repository are smoothed. Let me
insert a caveat here: I have found these files and the others, but to
understand them will take more time.
Where are the others and why are they not here?
Where they are is (relatively) easy to find. It took me about a day to
think of how to track them and a couple of hours afterwards to find
them. What I did was go to the library and find a copy of "Climate
since AD1500" (CAD1500 below). By looking at the referenced in CAD1500,
we find that the actual data series themselves were never "published" in
the peer reviewed literature, but were made available through a series
of DOE technical reports and eventually on various web sites. The
closest publication as such is "PD Josen, SCB Raper, RS Bradley, HF
Diaz, PM Kelly and TML Wigley, "Northern Hemisphere Surface Air
Temperature Variations: 1851-1984" J Climate and Applied Meteorology 25
(1986) 161. One piece of important information that we pick up is that
the data set is gridded, on a 5 x 5 degree grid...well ok, I think MBH
said that, but this paper actually shows the grid.
But where is the data?
Well, googling around with variations of Bradley, Jones, gridded dataset
and temperature it was finally located at the Carbon Dioxides IAC at
ORNL, in ftp directory http://cdiac.ornl.gov/ftp/ndp020/ in the file
jonesnh.dat (the precipitation files are there too along with programs
to extract them, but leave that story for tomorrow unless you want to
spent a long time working out what is where. (It took me about 30-45
minutes to dig this set out from among the other interesting climate
data records that are available, including a nice updated one of Jones
at the Climate Research Unit in England, and another one at the Climate
Data Center in the US. Tomorrow I will try and figure out whether I can
get my hands on the underlying 1985/86 DOE reports).
But why was this ftp site not cited. I would suggest two reasons.
First, what were Nature's rules about citing such web material in 1998.
Second, I believe that there may have been a slightly different site in
1998, but also at CDIAC.
The format of the temperature recors is explained in the file
ndp020r1.doc, as well as how to extract individual station records.
What remains to do is to scroll through the dataset for individual
stations, and find those stations whose start year agrees with (or is
close to) the date given in the NSM and the files in
/pub/MBH98/INST/TEMP/temp.loc/ and would be close to the gridded
coordinates given in MBH98. The coordinates are the coordinates of the
grid squares in which the station used is located, NOT the coordinates
of the station.
Let us look at the header for the TRONDHEIM station (one of those used)
in jonesnh.dat
12580 634 -105 115 TRONDHEIM/TYHOLT NORWAY 1 1761 1991 10 1761
The first number is the station number assigned by the World
Meteorological Organization.
The second number 634 is the latitude in tenths of degrees (ie. 63.4
degrees north)
The second number -15 is the longitude in tenthat of degree (ie 10.5
degrees east)
We then get the name of the station, the country and the first year in
the record (1761), the last year in the record (1991) and the first year
of "quality" data (1761).
We compare this with the data given in the NSM, and find that file # 12,
has a start date of 1761, in the grid box 62.5 N, 12.5 E, OK, that is
pretty clearly Trondheim. Well, I went through the list of a few
hundred stations, and matched them up. My conclusions are:
1 42.5 92.5W minnesota USA 1820 1820
2 47.5 2.5E paris France 1757 48.8 2.5 1757
3 47.5 7.5E geneva Switzerland 1758 46.2 6.2 1753
4 47.5 12.5E kremunster Austria 1767 48.1 14.1 1767
5 47.5 17.5E wien Austria 1776 48.2 16.4 1775 10
1777
6 de bilt Netherlands 1706 52.1 5.2 1706
7 52.5 17.5E berlin Germany 1792 52.5 13.4 1701
holes to 1775
8 vilnus Lithuania 1777 54.6 25.3 1777
miss 1964,39,15-17
9 57.5 17.5E stockholm Sweden 1756 59.4 18.1 1756
10 57.5 32.5E st. pertersburg Russia 1752 60.0 30.3 1743
miss btw 1745-51 1801-04
11 62.5 7.5E oslo Norway 1816 59.9 10.7 1816
12 62.5 12.5E trondheim Norway 1761 63.4 10.5 1761
13 62.5 42.5E archangel Russia 1814 64.6 40.6 1814
miss 1832
The first number is the file number as given in the MNH98 NSM, and the
/PUB/MBH98 files. The second pair are the gridded data lat and long, as
given in the NSM. Following the name of the station and the country is
the data given in the NSM as the start date. The next pair of numbers
are the actual lat/long of the station, followed by the actual start
date of the record.
Doubts: File 3 could be Basel (47.6 -7.6), which starts in 1755, but
1838-41 are missing for Basel, and Geneva is included in the CAD1500
files.
The record for berlin starts much later (1792) than the record (1701),
but there are many missing years in the early part of the century, and
there are comments in some of the papers I have read to a critical
evaluation. By position, this has to be Berlin, Warsaw is too far east
(52.2, -21.0) and missing the WWII years. Another possiblity is
Stuttgart, which starts in 1792
107390 488 -92 315 STUTTGART/CANNST W.GERMANY 1 1792 1991 10 1961
but is too far south and west (48.4 -9.2) as well as being considered
reliable only in 1961 and has missing data between 1931 and 1960.
Frankfurt is missing data between 1796 and 1825 and is too far west
(50.1 -8.7) Has to be Berlin.
The minnesota record is NOT in the CDIAC data set (OTOH, it is in the
CAD1500 dataset, and that is the reference nah). I suspect it was
discovered by Bradley and Jones sometime in the 1990s.
Enjoy. Remember to RTFR and RTFRR before saying that A should have done
B instead of C
josh halpern
Some preliminaries. The file sent by Mann's group to McIntyre is called
pcproxy.txt and can be found at http://www.climate2003.com. To save
space, I will call this site C2003. Also to save space, unless
specifically noted, I will use pcproxy to refer to the file pcproxy.txt
as found at C2003. The file was compiled by Scott Rutherford.
Rutherford also sent a roster showing which proxys were in each column
of pcproxy. You can find this at
http://www.climate2003.com/data/backto1820.txt. This file is a
modification of the file multiproxy.inf sitting on
ftp://holocene.evsc.virginia.edu/pub/MBH98 which I will call /pub/MBH98
to save space. multiproxy.inf lists the input files for the multiproxy
analysis. backto1820.txt simply adds a serial number to the file names
in order.
The issues of C2300 refer to the file pcproxy and the paper MBH98 as
published in Nature, we will call that MBH98 and take care to
differentiate it from the ftp site /pub/MBH98. We will call the
published supplemental materials NSM.
David Ball
Josh, I noticed that the lat/lon positions supplied with the
temperature series on the paleoclimate ftp site were nice round
numbers and had a pretty good idea the central point in a 5X5 degree
box was being given. Is there any chance that some of these series are
composites of multiple sites - assuming a given grid box had more than
one site in it?
The other thing that kind of puzzles me is why MBH (and M&M
for that matter) would include those temperature series in the first
place. The whole methodology of MBH revolves around using proxy data.
I suppose it could be that these records provide a longer-term
baseline than the CRU data - which only extends back to the 19th
Century - but I wish that had been made clearer.
Steve Schulin
Josh Halpern <j.ha...@incoming.verizon.net> wrote, in part:
> ... the file pcproxy.txt
> as found at C2003... was compiled by Scott Rutherford.
> ...
How is this known? The date of the pcproxy.txt file predates the request
by McIntyre. Rutherford did indeed forward the file to McIntyre, but I
wonder why you state that Rutherford compiled the file. If anything, the
evidence seems to indicate that Rutherford wasn't aware that a compiled
file existed when McIntyre asked. He thought the data was not available
from any one ftp site.
David Ball
LOL. Conspiracy theorists of the world unite! Schulin is on
the case. God's man, here's Josh doing the legwork that M&M should
have bloody well done and this is all you can say? Why is he doing all
this donkey work while you sit and huff and puff and play the
blowhard?
Steve Schulin
David Ball <wra...@mb.sympatico.ca> wrote:
Gee whiz. I was polite in my question. Not event a hint of an insult.
I'm glad he's interested in the subject. As to your suggestion that M&M
were remiss in asking the question instead of doing exactly as Josh, I
note that Josh was aided by what I've come to refer to as the newly
disclosed Mann et al ftp site. I know that you contend that the ftp site
was available before. I don't mind in the least publicly discounting
your claim as lacking in credibility.
David Ball
<steve....@nuclear.com> wrote:
Here we are attempting to fit all the pieces of the puzzle
together - something anyone attempting to do an "audit" should have
bloody well done - and you are posturing about who said what and when.
God's man, is that the best you can do? The "newly" discovered Mann
website has been around for a hell of a long time, you fool. I found
it over a year ago. Other's have told you the same thing. Your idiotic
comments on this matter help absolutely no one. Either get off your
fat pimply backside and offer something constructive or shut the hell
up. For God's sake, don't get in the way just for the sake of being an
obstructionist.
Steve Schulin
David Ball <wra...@mb.sympatico.ca> wrote:
> bloody well done - and you are posturing about who said what and when. ...
I asked a direct question about a direct statement made by another. I
don't mind that you're apparently not interested in the answer to my
question. But your reaction to the asking of it is pretty laughable.
> ... God's man, is that the best you can do? The "newly" discovered Mann
> website has been around for a hell of a long time, you fool. I found
> it over a year ago. ...
Maybe you did. Maybe you think you did. The lack of any prior mentions
of that ftp directory showing up in google search of web and usenet
seems a more reliable indicator than your quite sparsely described
recollection.
> ... Other's have told you the same thing. ...
Farrar's description of his recollection leaves room for doubt that he
saw the particular directory we're discussing.
> ... Your idiotic
> comments on this matter help absolutely no one. Either get off your
> fat pimply backside and offer something constructive or shut the hell
> up. For God's sake, don't get in the way just for the sake of being an
> obstructionist.
If Josh or anyone else has info related to the genesis of the ballocksed
pcproxy files, I'm sure I'm not the only person who'd be interested to
learn the details. My question was quite straightforward about a quite
unambiguous claim by Josh. Your characterization of my post as "getting
in the way" seems a stretch, although I must admit that it does appear
to have provoked you to imbecilic behavior.
Very truly,
David Ball
http://cdiac.ornl.gov/ftp/tr051/
I haven't been able to look at the raw data yet, but if you go through
the inventory file:
http://cdiac.ornl.gov/ftp/tr051/station.inv
and match up the starting years for the various stations some
interesting things come to light. The precipitation series, 32 to 42,
appear to fall out thusly, using MBH's series start years...
32: Madras, India
33: Bombay, India
34: Toulouse, France
35: Marseille, France
36: New Haven, CT
37: Paris, France
38: Klagenfurt, Austria
39: Prague, Czech
40: Kew Gardens (London)
41: Eallabus, UK
42: Edinburgh, Scotland
Now, there are some rather obvious problems with this, and I won't be
able to confirm my hypothesis until I can download the data and look
at it, but here goes...
1. Notice that series 36 and 37 have been reversed from what
is on the MBH website. This would confirm what M&M found with series
37 - it is Paris. There is still a series northwest of Boston, but it
is in series 36.
2. Series 34 is problematic as MBH list it at 37.5 282.5. The
only series that starts in 1809, however is in Toulouse, France,
rather a long way from the eastern US.
3. Series 38 appears to be Klagenfurt, Austria, however the
longitude given in MBH's ftp site is off by 10 degrees.
4. The longitude for series 40 also appears to be off. The
only series starting in 1697 is for Kew Gardens, London, which is
definitely not in the 12.5E bin.
David Ball
<steve....@nuclear.com> wrote:
LOL. Your question is completely immaterial. What's important
is what the bloody file contained, not your he said/they said
arguments. It's pretty conclusive evidence that you have nothing to
offer the discussion when you have to resort to this type of nonsense.
>
>> ... God's man, is that the best you can do? The "newly" discovered Mann
>> website has been around for a hell of a long time, you fool. I found
>> it over a year ago. ...
>
>Maybe you did. Maybe you think you did. The lack of any prior mentions
>of that ftp directory showing up in google search of web and usenet
>seems a more reliable indicator than your quite sparsely described
>recollection.
ROTFL. I did. And Paul Farrar told you as much also. But, hey,
you are entirely free to ignore what you're being told. I'd expect
nothing less from someone who believes that the Earth is only a few
thousand years old, despite the fact that the science says something
else.
>
>> ... Other's have told you the same thing. ...
>
>Farrar's description of his recollection leaves room for doubt that he
>saw the particular directory we're discussing.
Who can argue with silly logic like that.
>
>> ... Your idiotic
>> comments on this matter help absolutely no one. Either get off your
>> fat pimply backside and offer something constructive or shut the hell
>> up. For God's sake, don't get in the way just for the sake of being an
>> obstructionist.
>
>If Josh or anyone else has info related to the genesis of the ballocksed
>pcproxy files, I'm sure I'm not the only person who'd be interested to
>learn the details.
LOL. Steve, we're so far passed the cocked up data that M&M
got that it doesn't matter. Haven't you been paying attention? The
fact is, their data was corrupt. It happens. That's why data need to
be quality controled. Even M&M recognized there were problems. Read
their paper. Instead of assuming that the data were corrupt by some
misadventure, they assumed that this was the real data that MBH used
and proceeded with their analysis. Needless to say, they arrived at
erroneous conclusions. The fact that their analysis showed erroneous
warming during the LIA should have set off warning bells that there
was a problem with the input data. Instead, they assumed that it was
MBH's analysis that was flawed. That now appears unlikely.
>My question was quite straightforward about a quite
>unambiguous claim by Josh. Your characterization of my post as "getting
>in the way" seems a stretch, although I must admit that it does appear
>to have provoked you to imbecilic behavior.
>
LOL. Since you lack the wherewithal to understand what MBH
did, I can certainly understand your inability to offer anything to
the discussion aside from silly posturing and bold-faced innuendo, the
latter of which, as is usually the case with you, has nothing to
support it.
There are a lot more basic questions that need to be
addressed, like why MBH used some temperature series as inputs. Why
were the precipitation series chosen the way they were? Why not use
all of the data. That is likely immediately obvious to anyone working
routinely with the data, but for someone who is not, it isn't clear.
That needs to be addressed.
Nigel Persaud
> records used in Chapter 13 are on the noaa paleoclimate site in a file
> ftp://ftp.ngdc.noaa.gov/paleo/climate1500ad/ch13.txt.
This identification was set out by M&M.
>
> of these TRONDHEIM(12), STOCKHOLM (09), GENEVA (03), BERLIN (07),
> LENINGRAD (10), and MINNESOTA (01) are used in MBH98 The numbers in
> parentheses refer to the number in the NSM and /pub/MBH98/INSTR/TEMP.
I presume that you are using M&M's suggested identifications here, as
I have not seen these identifications (site to number) in MBH.
>
> Where are the others and why are they not here?
While there is undoubtedly some connection between the MBH series and
prior Jones data, I don't see that you've connected the dots just yet.
The values in MBH don't seem to tie together to the ndp020 data.
David Ball
wrote:
>> As David Ball discovered, the data files for the climate temperature
>> records used in Chapter 13 are on the noaa paleoclimate site in a file
>> ftp://ftp.ngdc.noaa.gov/paleo/climate1500ad/ch13.txt.
>
>This identification was set out by M&M.
>
>>
>> of these TRONDHEIM(12), STOCKHOLM (09), GENEVA (03), BERLIN (07),
>> LENINGRAD (10), and MINNESOTA (01) are used in MBH98 The numbers in
>> parentheses refer to the number in the NSM and /pub/MBH98/INSTR/TEMP.
>
>I presume that you are using M&M's suggested identifications here, as
>I have not seen these identifications (site to number) in MBH.
These sites fall out from the Jones data, given the starting
year given by MBH for each series, I think you'll find.
>
>>
>> Where are the others and why are they not here?
>
>While there is undoubtedly some connection between the MBH series and
>prior Jones data, I don't see that you've connected the dots just yet.
>The values in MBH don't seem to tie together to the ndp020 data.
And of course, you've subtracted the 1902-19xx mean, removed
the linear trend, and re-normalized the data? Here's a more important
question, Nigel: why are these temperature series used at all?
Everything else is using proxy data. Why use these data, fit to 5x5
bins, when you are going to later reduce the CRU temperature data down
to 16 principle components? In addition, why select the precipitation
sites they did? Using precipitation anomalies as a proxy for
temperature isn't necessarily the best thing, but at least they were
using proxy data. Why select the stations they did? Why not use more
stations or fewer?
David Ball
<j.ha...@incoming.verizon.net> wrote:
If you use the data from
ftp://holocene.evsc.virginia.edu/pub/MBH98/INSTR/TEMP/temp.loc
you get somewhat different results.
1. ???
2. Paris
3. Geneva
4. Kremuenster, Austria
5. Wien Hohe, Austria
` 7. Wroclaw, Poland
9. Stockholm
10. ???
11. Bergen, Norway
12. Trondheim
13. ???
I used the series start data from the above MBH file and went through
http://cdiac.ornl.gov/ftp/ndp020/jonesnh.dat
looking for matching start years. Where there was more than one
station, I matched the lat/lon position with the station name. For
example, the above file for series three could have also been for
Turin, Italy, but the lat/lon didn't match. Similarly, for series 2,
Frankfurt also started in the correct year. It could be that these
series are composites of the available data within each grid box and
the assigned name is given to the major centre in each box regardless
of whether there is data for that point.
Josh Halpern
David Ball wrote:
>On Tue, 30 Dec 2003 04:31:40 GMT, Josh Halpern
><j.ha...@incoming.verizon.net> wrote:
> I think you'll find the precipitation data in:
>
>http://cdiac.ornl.gov/ftp/tr051/
>
>I haven't been able to look at the raw data yet, but if you go through
>the inventory file:
>
>http://cdiac.ornl.gov/ftp/tr051/station.inv
>
>and match up the starting years for the various stations some
>interesting things come to light. The precipitation series, 32 to 42,
>appear to fall out thusly, using MBH's series start years...
>
>32: Madras, India
>33: Bombay, India
>34: Toulouse, France
>35: Marseille, France
>36: New Haven, CT
>37: Paris, France
>38: Klagenfurt, Austria
>39: Prague, Czech
>40: Kew Gardens (London)
>41: Eallabus, UK
>42: Edinburgh, Scotland
>
1 12.5 82.5E Madras India 1813 13.1 80.3 1813
2
3 17.5 72.5E Bombay India 1817 18.9 -72.9 1817
4 37.5 77.5W Charlestown USA 1809 32.8 79.9 1738
5 42.5 2.5E Marseille France 1749 43.3 -5.4 1749
6 42.5 7.5E Bologna Italy 1804 44.5 -11.5 1813
7 42.5 72.5W New Haven USA 1770 41.3 72.9 1804
8 47.5 2.5E Paris France 1813 48.8 -2.5 1770
9 47.5 12.5E Prague Czech 1805 50.1 -14.3 1805
10 52.5 12.5E Kew Gardens UK 1697 51.5 -3 1697
11 52.5 2.5W Eallabus UK 1800 55.6 -6.2 1800
12 57.5 7.5W Edinburgh UK 1785 55.9 -3.2 1785
Darn, you stole my fire. My list is a bit different
Note that the starting dates for 6, 7, and 8 are cyclically permuted.
>
>Now, there are some rather obvious problems with this, and I won't be
>able to confirm my hypothesis until I can download the data and look
>at it, but here goes...
>
> 1. Notice that series 36 and 37 have been reversed from what
>is on the MBH website. This would confirm what M&M found with series
>37 - it is Paris. There is still a series northwest of Boston, but it
>is in series 36.
>
Bologna, New Haven and Paris have been cyclically rotated.
> 2. Series 34 is problematic as MBH list it at 37.5 282.5. The
>only series that starts in 1809, however is in Toulouse, France,
>rather a long way from the eastern US.
>
Charleston
> 3. Series 38 appears to be Klagenfurt, Austria, however the
>longitude given in MBH's ftp site is off by 10 degrees.
>
Try Prague
> 4. The longitude for series 40 also appears to be off. The
>only series starting in 1697 is for Kew Gardens, London, which is
>definitely not in the 12.5E bin.
>
True, but 2.5 would work.
My how time flies when you are having fun
josh halpern
David Ball
<j.ha...@incoming.verizon.net> wrote:
How did you figure out that the rotation in dates had taken
place, Josh? Also, I take it the Charleston data from 1738 to 1809 is
incomplete in some way. I haven't uncompressed the file to have a look
yet. This should still account for how M&M got Paris in series 37, if
I'm not mistaken.
Josh Halpern
David Ball wrote:
I could have said rotation of files, but I was looking mostly at the NSM
listing, not the files in pcproxy and /pub/MBH98
First, I pretty much identified all of the others, except 6, 7, and 8.
If you look at prec-1820-06.dat in the ../pub/MBH98 it starts in 1804.
Similarly prec-1820-07.dat starts in 1770 and prec-1820-08 starts in
1813. If you sort the inventory file at
http://cdiac.ornl.gov/ftp/tr051/station.inv there is only sone station
with a start date of 1770, Paris. Similarly there is only one station
with a start date of 1813, New Haven. There are three files with start
dates of 1813: Madras (eliminated because it is #1 in the list above),
Bologna, or Klagenfurth. If you look at the coordinates, the
coordinates for New Haven match 7 in the list above, and Paris matches
8. That leaves something in the (42.5 -7.5) lat long box. Klagenfurth
is (46.7, -14.3) , Bolognia is (44.5, -11.5) which is a better match.
That identifies prec-1820-06.dat as New Haven, prec-1820-07.dat as
Paris and prec-1820-07.dat as Bologna.
You can check these identifications a bit by blowing up Fig 1a from
MBH98 and looking very carefully at the screen under maximum
magnification. The problem is that the resolution of the figures suck
and the area encompassing England, France, Switzerland, etc has too many
symbols. which overlay each other. There should have been an insert.
josh halpern
josh halpern
<ftp://holocene.evsc.virginia.edu/pub/MBH98/INSTR/PREC/prec-1820-06.dat>
Nigel Persaud
> >prior Jones data, I don't see that you've connected the dots just yet.
> >The values in MBH don't seem to tie together to the ndp020 data.
>
> And of course, you've subtracted the 1902-19xx mean, removed
> the linear trend, and re-normalized the data?
I looked at the Trondheim data. The NDP020 data is not "standardized",
but is a temperature reading. The MBH98 data is provided to eight (!)
digits and has been transformed somehow from the NDP020 (or similar)
data source.
It's easy to link the NDP020 data to the C1500 data in this case. The
C1500 annual average is equal to the NDP020 annual average less the
1901-50 mean, as rounded to one digit. It's possible that the editions
vary for other stations.
All three versions commence in 1761 and end in 1980.
I tried normalizing the C1500 version on 1902-80 and that didn't match
to MBH. The 1902-80 mean of the MBH98 version is not zero so it's
normalized some other way. Also when you plot the MBH version minus
the C1500 version, you get consistent differences.
While the data is presumably related to NDP020 and this can be used
for station identification, it also looks like MBH (1) may have used
some other edition of the data - this is supported by the Minnesota
series; (2) they have carried out some transformation of the data
which is not explained. The use of another edition is also supported
by the discrepancies between the JB92 versions and MBH versions
identified by M&M, in the cases where they had made station
identifications.
I suspect that the same will prove true of the precipitation data.
In the case of the Trondheim station, there is a slight negative
temperature trend in the 1902-80 period. Why would you remove this
trend and re-normalize? What would you then propose be done to the
pre-1902 data? Nigel
Nigel Persaud
> >>><j.ha...@incoming.verizon.net> wrote:
> >>> I think you'll find the precipitation data in:
> >>>
> >>>http://cdiac.ornl.gov/ftp/tr051/
> >>>
When I unzipped the data file, it gave me an unintelligible file. Did
anyone else have better luck?
David Ball
wrote:
>> >>>On Tue, 30 Dec 2003 04:31:40 GMT, Josh Halpern
You have to use a unix uncompress utility. WinDoze unzipping
utilities will decode the file, but WinDoze uses a different carriage
return than unix systems do, so the file you unzip will look god
awful. Doing it on a Unix or Linux system will get you the correctly
formatted file.
Josh Halpern
David Ball
<j.ha...@incoming.verizon.net> wrote:
>
>
>David Ball wrote:
>
>>On 31 Dec 2003 08:04:28 -0800, pers...@yahoo.com (Nigel Persaud)
>>wrote:
>>
>>
>>
>>>>>>>On Tue, 30 Dec 2003 04:31:40 GMT, Josh Halpern
>>>>>>><j.ha...@incoming.verizon.net> wrote:
>>>>>>> I think you'll find the precipitation data in:
>>>>>>>
>>>>>>>http://cdiac.ornl.gov/ftp/tr051/
>>>>>>>
>>>>>>>
>>>>>>>
>>>When I unzipped the data file, it gave me an unintelligible file. Did
>>>anyone else have better luck?
>>>
>>>
>>
>> You have to use a unix uncompress utility. WinDoze unzipping
>>utilities will decode the file, but WinDoze uses a different carriage
>>return than unix systems do, so the file you unzip will look god
>>awful. Doing it on a Unix or Linux system will get you the correctly
>>formatted file.
>>
>
>gzip works on most platforms http://www.gzip.org/
>
You bet, but I think the carriage return problem still exists.
Nigel Persaud
Josh and David, If you look at the data series as well as the starting
dates, one notices the following: the MBH data is a linear
transformation of the annualized TR051 data using some undisclosed
transformation. David's identifications are better than Josh's and I
will comment on these. David's identifications of 32,34,35,37,39,41
and 42 are secure with close to 100% correlation (but with a linear
transformation). TR051 Kew Gardens and Klagenfurt have high, but not
exact matches to MBH series 38 and 40, pointing to some related, but
not identical source. Boston correlates as closely to #36 as
Klagenfurt or Kew Gardens do to their matches(0.82), but the source of
the pre-1818 data in MBH is a puzzle. Bombay does not tie in to #33 at
all. Philadelphia has the highest correlation (0.55) to #33 of all
series commencing between 1800 and 1830.
The above pretty much shows that M&M were completely right in this
issue.
Nigel
David Ball
wrote:
>> >>>I haven't been able to look at the raw data yet, but if you go through
M&M haven't been right on a single thing so far, Nigel. Have
you reconstructed MBH's input data from the TR051 data? That's what
needs to be done. Trying to assess the inputs in terms of correlations
is not the correct thing to be doing here. Show me how to get from
point a to b, then we'll be on to something.
Nigel Persaud
> >issue.
> >
> M&M haven't been right on a single thing so far, Nigel. Have
> you reconstructed MBH's input data from the TR051 data? That's what
> needs to be done. Trying to assess the inputs in terms of correlations
> is not the correct thing to be doing here. Show me how to get from
> point a to b, then we'll be on to something.
In fact, I have. If you take the TR051 series for 1770-1980 for
Paris/Le Bourget (wmo 61507150000) and standardize by subtracting its
mean and dividing by its s.d., then multiply the resulting series by
the s.d. of MBH #37 and add in the mean of MBH#37, you get MBH#37 to
within 3 decimal places. Exactly why MBH transformed the TR051 series
in this way remains a mystery as does why they used the wrong
location, but that they did so is beyond doubt. MBH#37 is TR051 Paris,
q.e.d. This should take you from point a to point b. Likewise, MBH#34
is TR051 Toulouse, as you surmised. Nigel
David Ball
wrote:
>> >The above pretty much shows that M&M were completely right in this
I will try and confirm your results tomorrow. Subtracting the
mean and normalizing to unit standard deviation is entirely
appropriate, especially working with multi-variate data. All of the
tree data were transformed in this fashion. That's why I asked you
earlier if that was the case. You cannot compare tree data with
temperatures or precipitation on a one to one basis as these data all
have different magnitudes. They all have to be put into the same frame
of reference. All of the tree data are transformed in this fashion
prior to the EOF's being calculated.
Nigel Persaud
Actually, I used the precipitation data from
ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/v2/zipd/v2.prcp.zip. I think
that this is the updated version of TR051. It's a big file (80 MB). I
forgot to mention that I took annual totals as a first step, but this
is presumably obvious.
Lubo? Motl
re-check all the claims and alleged errors.
> > Again, On balance this illustrates the principal of RTFR and the danger
> > of someone unfamiliar with an area trying to do an "audit". To use the
> > central European index before 1550 would clearly have been a mistake.
Could you please justify every sentence like that by a rational
argument? Why do you want to exclude central Europe before 1550? I was
born in the very center of Europe :-), and as far as I know, our 14th
and 15th century was very warm! Grapes were grown everywhere, and the
Czech king Charles IV as well as the hussites loved to drink wine all
the time! Today we prefer to drink beer and grapes are only grown in
Moravia. :-)
I have absolutely no confidence in claims like "people unfamiliar with
an area should not try to do an audit" unless it is also said why:
what they're doing wrong and why it is wrong. I won't ever believe
that I am uncapable to understand climatology, for example. It is
exactly the arrogant sort of the statements made by the people who
have no argument to say, and therefore they argue that they are - by
definition - the only ones who should be trusted. I am a physicist who
does not like the economists too much ;-), but according to the
available data (so far), McKitrick and McIrtyne did a better job than
MBH.
I don't care whether they were trained or paid as economists or
businessmen. Whoever seems to do a better analysis of climatological
data, is a better climatologist! A scientific result must be
repeatable by everyone - certainly by a physicist - and if someone
will tell me that I can't repeat it because I am a physicist (and
McKitrick can't do it because he is an economist), then it is no
science.
> However, first of all, it is common ground now that MBH deleted the
> first 25 years of this series. Prior to M&M, no one knew that. (The
> undeleted series are also at ftp/MBH98. Secondly, MBH did not disclose
> that they deleted the first 25 years.) The deletion may or may not be
> justifiable in climatological terms,
I wonder how anyone can justify something like that. What a scientific
fraud is supposed to mean if this is not one? Read Feynman's "Cargo
Cult Science", http://www.lhup.edu/~dsimanek/cargocul.htm - to see
what a scientific fraud is. The history of the measurements of the
elementary electric charge (following Millikan's setup) are
embarassing for physics: the physicists used to exclude the results
that they did not like - that were far from the believed value - and
therefore they were getting wrong results even though their data would
give them much more accurate result, had they been honest.
McKitrick and McIrtyne show that if the analysis is done in a way that
seems much more natural and honest to me (at least after one day of
analyses), then they obtain very different results. It is a secondary
question whether their conclusions are correct; what seems more
important to me is that the original results are proved not to be
robust. They depend on small modifications, and therefore no
conclusions are really justified.
A different issue. I think that the modified conclusions are correct.
There has been a medieval warm period. Greenland used to be a green
land after Vikings occupied it in 950, and it became colder after
1300. There has been a little ice age between 1550-1800, and the
chronicles say clearly that Thames used to freeze in London on regular
basis, and so on.
> but people relying on MBH should
> be advised MBH did this without having to parse through original
> sources. The deletion should have been annotated in the Supplementary
> Information directly and justified.
OK, I don't believe it. If they admitted that they erased 25 years, it
would be like admitting that they were cheating.
> which you purport to rationalize. Finally, while you're at this, can
> you figure out why MBH used summer versions instead of annual versions
> for these series?
Unfortunately I guess that the reason is that the summer versions lead
to "better" results.
All the best
Lubos Motl
David Ball
<j.ha...@incoming.verizon.net> wrote:
>
>Well, googling around with variations of Bradley, Jones, gridded dataset
>and temperature it was finally located at the Carbon Dioxides IAC at
>ORNL, in ftp directory http://cdiac.ornl.gov/ftp/ndp020/ in the file
>jonesnh.dat (the precipitation files are there too along with programs
>to extract them, but leave that story for tomorrow unless you want to
>spent a long time working out what is where. (It took me about 30-45
>minutes to dig this set out from among the other interesting climate
>data records that are available, including a nice updated one of Jones
>at the Climate Research Unit in England, and another one at the Climate
>Data Center in the US. Tomorrow I will try and figure out whether I can
>get my hands on the underlying 1985/86 DOE reports).
I downloaded Jones' data from the above website. I was
interested, not so much in getting a match between a subset of the
data with what is posted on the MBH website, but rather to explore why
those particular sites and only those were chosen.
If you look at the data, and weed out those sites that do not
have a quality control flag, and remove those series that do not
extend all the way to 1980, then sort the data by the start time of
the record, a few interesting things pop out. The stations with the
longest series are:
1. Berlin 1756
2. Geneva 1753
3. Stockholm 1756
4. Trondheim 1761
5. Paris 1757
Starting to look familiar? If you then look at each series and
identify the periods over which there are continuous observations
(i.e. no -999) flags, you can adjust the start time of the series. In
cases where there is a single missing month, interpolation can fill in
the blanks so that an annual mean can be identified. When longer-term
absences in the data are considered, you don't start your series.
You'll notice some differences in the start times I've identified for
the 5 stations above and those on the MBH website.
After the first 5 stations, things start to get muddy, because
6 through 10 are:
6. Copenhagen 1798
7. Wien 1775
8. Budapest 1780
9. Munich 1781
10. Kremeunster 1802
and these are not the same stations that MBH used. It could be that
they picked one long-term record per 5x5 grid box that was
representative. I need to check on that. There is a lot of data that
do not appear in MBH and I'd like to figure out why.
Lubo? Motl
> No, people in this forum are farther ahead today because of
> people like Josh who are doing the legwork necessary to correct M&M's
> screw-ups. M&M haven't clarified anything, especially since they were
> looking at the wrong bloody data.
Could you be more specific what was wrong with their choice of the
data? You know, if an impartial observer reads M&M, he or she can see
every statement (about the errors of MBH98) to be documented by very
specific numbers and quotations of MBH98, and so on. I've spent one
day with this analysis so far. Is it so difficult to say what exactly
is wrong about the M&M statements? If it's difficult, why do you
expect the impartial people not to believe essentially every word of
M&M?
If you say that the M&M criticism is not fair, I want to know whether
you think that it is correct to copy the data many times, omit the
first 25 years (especially the warm years in the warm period, and the
cold years in the cold period, to make the average more constant), use
the summer averages instead of the annual ones for a selected subset,
and so on, and why. Or alternatively, you can show that MBH did not do
these errors and the M&M claims are based on some misunderstandings.
What you are showing is nothing - it is just wasting of our time. Do
you think that all the people around are stupid not to be able to
check these things for themselves?
> Sorry, Steve, M&M's paper is a mess and anything based on it
> is necessarily flawed. I wonder why Mr. Carter didn't notice the
> warming that M&M showed during the LIA. That should have set off alarm
> bells for anyone with an ounce of understanding of the problem.
Look, a lot of attacks, but - again - not a single argument. Your
scientific (?) field seems sort of sick to me. It is much like Alan
Sokal's affair that revealed that the scientific standards of the
"Social Text" follow from the requirements that everyone should
flatter the editors and their prejudices. This is how the "greenhouse
science" seems to work, too. No arguments, just mysterious secret
procedures that none is allowed to know and only the "chosen ones" can
perform. Whoever shows something that disagrees with the "chosen ones"
is a heretic.
Sorry, if this is how it really works, then it is no science.
> >In the 1998 Nature paper, Mann et al note what looks to be a general
> >principle: "... the mutual information contained in a diverse and widely
> >distributed set of independent climate indicators can more faithfully
> >capture the consistent climate signal that is present, reducing the
> >compromising effects of biases and weaknesses in the individual
> >indicators." Finding reason to drop cold-year data from Little Ice Age
Well, yes and no. If a method XY gives systematically colder results
than the method CD, then we can compare the years T1 and T2 more
reliably if both of them are measured by both methods, or a single
method only, but nevertheless the same set of methods is applied in
both cases. However: It is absolutely obvious that this strategy can't
be followed in this case because the different centuries tend to
provide us with different sorts of proxies - even though comparing the
different centuries is precisely what we want to achieve.
Therefore: if we want to figure out something about the 15th century,
we must take *all* available methods into account, with the possible
weights reflecting how much we trust each single method, and we must
try to compare the results with the proxies from other centuries (such
as the 20th century), especially with the proxies of the same type.
But if we use such weights, they should be used universally if
possible. If they weights are time-dependent, OK, but the results
should be more or less independent of these choices otherwise they're
not reliable. Mann's claim seems to be a magic statement that is meant
to justify an arbitrary selection of the data, which is absolutely
unnacceptable, and you won't fool me that it is an illusion caused by
my not being a climatologist. This is just a question of being a
scientist.
> Apparently, you didn't understand what I said to you. Quality
> control of any data MUST be done. It is a fact of life. If there is a
> suspicion that data are compromised, YOU DON'T USE THEM! Pretty
> straight forward, eh?
That's right. A great idea. I wonder why so many people used MBH98
despite the rule above! Another comment about this idea: the suspicion
whether some data are compromised must be described in that paper, and
justified (therefore also justifiable) by objective, repeatable and
verifiable arguments. Michael Mann's or anyone else's private
suspicion that some data might be compromised is absolutely
irrelevant, and any result based on similar subjective suspicions is
compromised and should not be used, as you wrote correctly.
> Really? From a screwed up data set? How do you figure? Aren't
> you the one who often preaches garbage in, garbage out? A wonderful
> example of it here. Does this mean that Mann et al are perfect? I
> doubt it.
M&M apparently do not claim that the methodology and the proxies
chosen by MBH98 are perfect. M&M just accept that the data sets and
the methodology are well-defined, and they follow it properly - or at
least, more properly, following the standard rules of careful
statistics and scientific integrity. Whether we believe the modified
conclusions of M&M or not, is a different issue. I have various
reasons - the older insights about the medieval climate etc. - to
believe that the corrected results are pretty good. Nevertheless it is
clear that neither results are really robust, and the conclusions are
a matter of belief. I do believe the medieval warm period, for
example, but the available data are slightly unconvincing.
> >which were not amongst the pcproxy.txt file. M&M ask Mann et al if those
> >series were in fact used. Mann hasn't answered that specifically yet,
> >but he does say that all the data for the 1998 paper is available in the
> >newly disclosed ftp site.
All these things make it pretty clear that Mann knows pretty well what
was going wrong, but he does not want to tell us. BTW Mann's FTP site
does not work for me which also looks a bit suspicious to me. From the
public relation perspective, M&M just look perfect to me: sort of
neutral, honest, and they publish everything that anyone could find
relevant. Why can't MBH do the same thing? What do you think that my
answer can be?
> >> More spin and posturing, I'm afraid. Only you could take
> >> standard practice and something very innocent and try to make
> >> something nefarious out of it.
Could you explain why you think that it was innocent? Maybe I do not
know *something*, but with my current understanding it does not look
innocent, and until someone explains me some point that I am missing,
I will think that it was something nefarious.
> LOL. Paul Farrar has already pointed out that the ftp site in
> question was available long ago as have I. Your ridiculous attempts to
> show it otherwise are quite pathetic.
Haha, but it does not seem available now.
> I never said it was. I said that there is a certain amount of
> fluidity available in the data (i.e. analyst preferences) and choosing
> break-points is up to them.
It is not such a disaster if someone chooses break-points even though
I don't think that they had too much data so that it would be
reasonable to throw some random years away (in order to start with the
"round" numbers). But anyway if the results depend on such
break-points in a significant enough way, then the results are not
robust and should not be treated seriously. 1691/2 by the way was a
very cold winter in Boston and Salem - for those who know the history
of the witches here. ;-) Is it really an accident that seemingly all
these subjective choices of MBH98 are almost always done in order to
reduce the relatively well-known large fluctuations of the temperature
in the past - in this case the little ice age around 1700?
You can truncate the data to omit a couple of very cold winters before
1700, and make the average look more constant. But it's the last thing
that you can do against the fact that these years were very cold.
> >> ROTFL. You've got a couple of hack who knock off a paper,
> >> submit it completely full of errors to a third-rate "journal" then
> >> whine when they are shown to be less than competent and you are
> >> posturing about Mann et al's sloppiness? Too funny.
You seem to rely on the belief that your seemingly dirty tricks to
continue with your greenhouse pseudoscience will never be looked at by
the people outside the greenhouse field. You're wrong. M&M showed that
the economists can do it very well, and be sure that the physicists at
many places are capable to do the job, too. It does not really matter
whether the editors of all these "well-established journals" are
irrational inquisitors who only reject the articles that contradict
the "well-established" dogmas, regardless of the scientific qualities.
M&M were able to publish their work, it is obviously an important
work, and now everyone has the opportunity to study the case! One
thing is clear to me: if it's revealed that a scientific department of
a university dramatically twists scientific methods in order to
achieve some political agenda, and uses the peer-review process only
to strengthen its own influence, such a department should be
abolished.
> >> >So you think it inappropriate that IPCC used the hocky stick graph in
> >> >Fig 1 of the WG1 TAR policymaker summary?
This looks increasingly scary to me, especially in light of the huge
influence of these documents. Why no one paid as an environmental
scientist has tried to reproduce all the calculations of MBH98, and
publish a more detailed analysis? What are the tens of thousands of
these people paid for? Why it must be outsider economists who does
this important job, and why is it 5 years later?
All the best,
Luboš
Lubo? Motl
> >That's up to Mann to explain, not M&M. I'm not aware of any
> >explanation by MBH of the selection of the 1082 grid-boxes. Nigel
>
> On the contrary. M&M have taken MBH to task for not disclosing
> their methodology. Why haven't M&M come clean with theirs. What's
> sauce for the goose, after all. It's pretty hypocritical to say one
> thing then turn around and do the very thing that you accuse others
> of.
What are you talking about? M&M precisely describe, step by step,
their process of selection and scaling of 1082 MBH cells at
Josh Halpern
Lubo? Motl wrote:
>First of all, let me admire Josh Halpern and others who try to re-check all the claims and alleged errors.
>
>
>>>Again, On balance this illustrates the principal of RTFR and the danger of someone unfamiliar with an area trying to do an "audit". To use the central European index before 1550 would clearly have been a mistake.
>>>
>>>
>
>Could you please justify every sentence like that by a rational argument?
>
Before doing your Emily Latella imitation go read C. Pfister "Monthly
temperature and precipitation in central Europe, 1525-1979; quantifying
documentary evidence on weather and its effects", in Climate since AD
500, pp118, Routledge,. 1992)
>Why do you want to exclude central Europe before 1550?
>
Maybe because the particular data series that was being referred to is
not reliable before 1550? According to the person who put it together,
but what does he know?
>I was born in the very center of Europe :-), and as far as I know, our 14th and 15th century was very warm! Grapes were grown everywhere, and the Czech king Charles IV as well as the hussites loved to drink wine all the time! Today we prefer to drink beer and grapes are only grown in Moravia. :-)
>
While I am very partial to Czech beer, as far as I am concerned your
fondness for Moravian wine is as the peace of God.
>I have absolutely no confidence in claims like "people unfamiliar with an area should not try to do an audit" unless it is also said why: what they're doing wrong and why it is wrong.
>
An audit assumes that one is familiar with the underlying material. M&M
claim to evaluate a calculation that uses a mass of climatological data,
much of which has been created by, collected and evaluated by two
experts in the field.
>I won't ever believe that I am uncapable to understand climatology, for example. It is exactly the arrogant sort of the statements made by the people who have no argument to say, and therefore they argue that they are - by definition - the only ones who should be trusted. I am a physicist who does not like the economists too much ;-), but according to the available data (so far), McKitrick and McIrtyne did a better job than MBH.
>
You have not been following the bouncing ball. You can, perhap,
understand climatology. To reach a useful understanding might take two
to five years, not the five months or so the M&M took.
>I don't care whether they were trained or paid as economists or businessmen. Whoever seems to do a better analysis of climatological data, is a better climatologist! A scientific result must be repeatable by everyone - certainly by a physicist - and if someone will tell me that I can't repeat it because I am a physicist (and McKitrick can't do it because he is an economist), then it is no science.
>
>
No what I am telling you is that if you use datasets you need to
understand the strengths and weaknesses of the data sets. One of the
points that is continually missed in this hoo-ha is that Bradley is
involved in constructing and maintaining many of the most reliable
climatological data sets, his (and Mann's) collaborator Jones is equally
involved and that Hughes is an expert in tree ring analysis. To
McKitrick it is just a set of numbers. RTFR and understand them.
>>However, first of all, it is common ground now that MBH deleted the first 25 years of this series. Prior to M&M, no one knew that. (The undeleted series are also at ftp/MBH98. Secondly, MBH did not disclose that they deleted the first 25 years.) The deletion may or may not be justifiable in climatological terms,
>>
>>
>
>I wonder how anyone can justify something like that. What a scientific fraud is supposed to mean if this is not one?
>
MBH98 specifically wrote that they did not use the first 25 years of
Pfister's central european series. It is obvious to anyone who RTFR why
this was done, and is almost certainly well known to anyone who does
climatology.
>Read Feynman's "Cargo Cult Science", http://www.lhup.edu/~dsimanek/cargocul.htm - to see what a scientific fraud is. The history of the measurements of the elementary electric charge (following Millikan's setup) are
>embarassing for physics: the physicists used to exclude the results that they did not like - that were far from the believed value - and therefore they were getting wrong results even though their data would give them much more accurate result, had they been honest.
>
You have that one screwed up also. First of all, the error in
Millikan's work was the value used for the viscosity of air. Secondly
the so called discarded data was a training set, where Millikan was
trying out the apparatus and learning how to use it. Anyone who has
done this experiment needs to practice before getting reliable data.
http://www1.umn.edu/ships/updates/fraud.htm
>McKitrick and McIrtyne show that if the analysis is done in a way that seems much more natural and honest to me (at least after one day of analyses), then they obtain very different results. It is a secondary question whether their conclusions are correct; what seems more important to me is that the original results are proved not to be robust. They depend on small modifications, and therefore no conclusions are really justified.
>
>
The problem with this one is that there are several independent
calculations of the same things, and they agree with MBH98 or are below
MBH98 until the 20th century.
>A different issue. I think that the modified conclusions are correct. There has been a medieval warm period. Greenland used to be a green land after Vikings occupied it in 950, and it became colder after 1300. There has been a little ice age between 1550-1800, and the chronicles say clearly that Thames used to freeze in London on regular
>basis, and so on.
>
>
As they say, you should look up the meaning of the word GLOBAL.
>>but people relying on MBH should be advised MBH did this without having to parse through original
>>sources. The deletion should have been annotated in the Supplementary Information directly and justified.
>>
>>
>
>OK, I don't believe it. If they admitted that they erased 25 years, it would be like admitting that they were cheating.
>
>
As I pointed out above, MBH98 specifically stated that they omitted the
first 25 years of the Pfister series, and reading Pfister clearly states
why for this period his series does not contain reliable data and no
instrumental data. To repeat
"For the perid 1525-1549 the entries originate mainly from chronicles
and annals. Accordingly, weather sequences are mainly described at a
seasonal level; information is missing for 43% of the months and the
emphasis is on anomalous rather than on ordinary weather."
>>which you purport to rationalize. Finally, while you're at this, can you figure out why MBH used summer versions instead of annual versions for these series?
>>
>>
>
>Unfortunately I guess that the reason is that the summer versions lead to "better" results.
>
I am less than impressed by both claims above. All I can say is that
the summer series is found in the /pub/ directory and pcproxy, but what
was used for the MBH98 calculation, has not been established.
josh halpern
David Ball
wrote:
>
>I have absolutely no confidence in claims like "people unfamiliar with
>an area should not try to do an audit" unless it is also said why:
>what they're doing wrong and why it is wrong. I won't ever believe
>that I am uncapable to understand climatology, for example. It is
>exactly the arrogant sort of the statements made by the people who
>have no argument to say, and therefore they argue that they are - by
>definition - the only ones who should be trusted. I am a physicist who
>does not like the economists too much ;-), but according to the
>available data (so far), McKitrick and McIrtyne did a better job than
>MBH.
Your statement would be valid if the M&M had actually done an
audit. They didn't. That's a huge problem. To do an audit, M&M would
have needed to use the same data that MBH did. They did not, both by
getting ahold of a corrupt dataset and not having the wherewithal to
realize that you cannot do any kind of good science with corrupt data.
It turns out that many of the problem M&M encountered would
have been fixable had they bothered to read the reference material.
>
>I don't care whether they were trained or paid as economists or
>businessmen. Whoever seems to do a better analysis of climatological
>data, is a better climatologist!
Silly comment noted. Whoever DOES a better analysis is a
better ANALYST. SEEMS never enters the picture. Interpretation of the
analysis takes expertise. You're a physicist. That's terrific. I doubt
very much you could look at a radar image and tell me the
meteorological implications of what you were seeing without a lot of
training.
>A scientific result must be
>repeatable by everyone - certainly by a physicist - and if someone
>will tell me that I can't repeat it because I am a physicist (and
>McKitrick can't do it because he is an economist), then it is no
>science.
You're certainly going to have a lot of trouble repeating it
when you don't use the same data.
>
>McKitrick and McIrtyne show that if the analysis is done in a way that
>seems much more natural and honest to me (at least after one day of
>analyses), then they obtain very different results. It is a secondary
>question whether their conclusions are correct; what seems more
>important to me is that the original results are proved not to be
>robust. They depend on small modifications, and therefore no
>conclusions are really justified.
LOL. You'll have to justify that comment. It's more natural
and honest to use corrupt data, leave out certain series and arrive at
results that are patently wrong? Care to explain how you arrive at a
series that shows warming during the Little Ice Age and NOT ask
yourself the all important question: "Oh, oh. We may have a problem
here."
David Ball
wrote:
>David Ball <wra...@mb.sympatico.ca> wrote:
>
>> No, people in this forum are farther ahead today because of
>> people like Josh who are doing the legwork necessary to correct M&M's
>> screw-ups. M&M haven't clarified anything, especially since they were
>> looking at the wrong bloody data.
>
>Could you be more specific what was wrong with their choice of the
>data? You know, if an impartial observer reads M&M, he or she can see
>every statement (about the errors of MBH98) to be documented by very
>specific numbers and quotations of MBH98, and so on. I've spent one
>day with this analysis so far. Is it so difficult to say what exactly
>is wrong about the M&M statements? If it's difficult, why do you
>expect the impartial people not to believe essentially every word of
>M&M?
First of all, read through this entire thread and the
subsequent threads. You will see a lot of problems with M&M's data.
The spreadsheet they used was completely corrupted. They didn't read
the reference material.
Secondly, your use of the word "impartial" implies that you
are and anyone who disagrees with your POV is not. Many readers have
come down on either side - some agreeing with MBH, others with M&M.
Few have attempted to reconstruct from the ground up what both M&M and
MBH have done. That is being impartial.
So far, I haven't seen anything that leads me to believe that
there is anything wrong with MBH. I have seen much wrong with M&M's
approach, up to and including an unseemly rush to get something in
print when there were clear problems with their analysis.
>
>If you say that the M&M criticism is not fair, I want to know whether
>you think that it is correct to copy the data many times, omit the
>first 25 years (especially the warm years in the warm period, and the
>cold years in the cold period, to make the average more constant), use
>the summer averages instead of the annual ones for a selected subset,
>and so on, and why. Or alternatively, you can show that MBH did not do
>these errors and the M&M claims are based on some misunderstandings.
>What you are showing is nothing - it is just wasting of our time. Do
>you think that all the people around are stupid not to be able to
>check these things for themselves?
I suggest that the M&M analysis is piss-poor and the
conclusions derived from it are necessarily flawed.
My goal is to re-do MBH from the ground up, hopefully
addressing, as has been done on a number of occasions, some of the
questions that M&M raise. I'm not interested in doing a so-called
"audit" of their work, but rather I'm interested in reproducing MBH's
results using MY analysis.
>
>> Sorry, Steve, M&M's paper is a mess and anything based on it
>> is necessarily flawed. I wonder why Mr. Carter didn't notice the
>> warming that M&M showed during the LIA. That should have set off alarm
>> bells for anyone with an ounce of understanding of the problem.
>
>Look, a lot of attacks, but - again - not a single argument. Your
>scientific (?) field seems sort of sick to me. It is much like Alan
>Sokal's affair that revealed that the scientific standards of the
>"Social Text" follow from the requirements that everyone should
>flatter the editors and their prejudices. This is how the "greenhouse
>science" seems to work, too. No arguments, just mysterious secret
>procedures that none is allowed to know and only the "chosen ones" can
>perform. Whoever shows something that disagrees with the "chosen ones"
>is a heretic.
And your sort of comments are nothing more than posturing.
Anyone doing an analysis who arrives at a result that is fundamentally
at odds with KNOWN facts has to ask themselves a number of questions
the first and most important being: "Did I screw up?" When you fail to
do that, you have a major problem on your hands.
>
>Sorry, if this is how it really works, then it is no science.
Real science takes a phenominal attention to detail and a
thorough knowledge of what you are doing.
>
>> Apparently, you didn't understand what I said to you. Quality
>> control of any data MUST be done. It is a fact of life. If there is a
>> suspicion that data are compromised, YOU DON'T USE THEM! Pretty
>> straight forward, eh?
>
>That's right. A great idea. I wonder why so many people used MBH98
>despite the rule above! Another comment about this idea: the suspicion
>whether some data are compromised must be described in that paper, and
>justified (therefore also justifiable) by objective, repeatable and
>verifiable arguments. Michael Mann's or anyone else's private
>suspicion that some data might be compromised is absolutely
>irrelevant, and any result based on similar subjective suspicions is
>compromised and should not be used, as you wrote correctly.
Feel free at any point to offer your insights into the
problematic nature of the data. Please do more than use the corrupt
dataset used by MBH.
>
>> Really? From a screwed up data set? How do you figure? Aren't
>> you the one who often preaches garbage in, garbage out? A wonderful
>> example of it here. Does this mean that Mann et al are perfect? I
>> doubt it.
>
>M&M apparently do not claim that the methodology and the proxies
>chosen by MBH98 are perfect. M&M just accept that the data sets and
>the methodology are well-defined, and they follow it properly - or at
>least, more properly, following the standard rules of careful
>statistics and scientific integrity.
LOL. By not using the correct data? How do you figure? By not
reading the supplied reference material? How do you figure?
>Whether we believe the modified
>conclusions of M&M or not, is a different issue. I have various
>reasons - the older insights about the medieval climate etc. - to
>believe that the corrected results are pretty good. Nevertheless it is
>clear that neither results are really robust, and the conclusions are
>a matter of belief. I do believe the medieval warm period, for
>example, but the available data are slightly unconvincing.
Then you should have no trouble providing us with detailed
analysis showing why your results are superior to everyone elses.
Please, no hand-waving arguments.
>
>> >which were not amongst the pcproxy.txt file. M&M ask Mann et al if those
>> >series were in fact used. Mann hasn't answered that specifically yet,
>> >but he does say that all the data for the 1998 paper is available in the
>> >newly disclosed ftp site.
>
>All these things make it pretty clear that Mann knows pretty well what
>was going wrong, but he does not want to tell us. BTW Mann's FTP site
>does not work for me which also looks a bit suspicious to me. From the
>public relation perspective, M&M just look perfect to me: sort of
>neutral, honest, and they publish everything that anyone could find
>relevant. Why can't MBH do the same thing? What do you think that my
>answer can be?
ROTFL. This is what you call being "impartial"?
>
>> >> More spin and posturing, I'm afraid. Only you could take
>> >> standard practice and something very innocent and try to make
>> >> something nefarious out of it.
>
>Could you explain why you think that it was innocent? Maybe I do not
>know *something*, but with my current understanding it does not look
>innocent, and until someone explains me some point that I am missing,
>I will think that it was something nefarious.
Could you explain why it isn't?
>
>> LOL. Paul Farrar has already pointed out that the ftp site in
>> question was available long ago as have I. Your ridiculous attempts to
>> show it otherwise are quite pathetic.
>
>Haha, but it does not seem available now.
ftp://holocene.evsc.virginia.edu/pub/MBH98
Lubo? Motl
> Before doing your Emily Latella imitation go read C. Pfister "Monthly
> temperature and precipitation in central Europe, 1525-1979; quantifying
> documentary evidence on weather and its effects", in Climate since AD
> 500, pp118, Routledge,. 1992)
Well, I suppose you meant "Climate since AD 1500", did not you? I am
sort of stunned by your selective approach to the literature. Why
don't you follow other prescriptions and works by the same author
(Pfister, in this case) that indicate that the European climate
probably used to be very different to contemporary climate? Lamb 1982,
Pfister 1984, Grove 1988 at
http://www.cx.unibe.ch/hist/fru/fru-adv.htm
Pfister's methodology from 1992 is certainly not something canonical
that one can follow without explicitly mentioning it. This MBH
approach seems to be a selective application of the subset of the
rules that give the results that the authors like. Sorry, we
physicists might have much higher scientific standards than your
field, but all these things that you exposed just seem to be an
obvious fraud. Removing some data (25 years!) from the ensemble is a
very nontrivial decision, and the reader must be informed that it was
done and why it was done.
> Maybe because the particular data series that was being referred to is
> not reliable before 1550? According to the person who put it together,
> but what does he know?
Yes, I suppose that none really knows how the medieval weather should
be properly reconstructed, and which data are really reliable. But it
does not matter. Once you decide that you want to follow some hints
anyway, you must do it properly, which does not seem to be the case
here. I don't care about the artificially created fame of your
colleagues: the available hard data simply suggest that the Czech
chronicles from 1500 or so are more reliable than articles in Nature
1998 about the same subject because the chronicles were written as a
description of reality while MBH98 is a result of a random selective
process of the data meant to find the desired results.
> An audit assumes that one is familiar with the underlying material. M&M
> claim to evaluate a calculation that uses a mass of climatological data,
> much of which has been created by, collected and evaluated by two
> experts in the field.
Well, the mere non-technical statement that "someone is an expert"
does not mean much. Well, if there are serious problems with their
work, then these people should *not* be treated as experts, in fact
they should be fired. The rule "22" in a list is "don't always believe
experts", see e.g.
http://www.engin.umich.edu/dept/aero/people/faculty/bernstein/research_advice.pdf
You probably don't understand how serious the situation is. Most of
your field now seems to be a bubble containing arrogant
pseudoscientists that call one another "experts" even though they are
doing worse job than the outsider economists, and the question is
whether it is the case or not. If it is the case, various people
should forget about being called an "expert", and if a deliberate
fraud was done in some important cases - whose impact might have been
wasting trillions of dollars worldwide - such people should definitely
be arrested, I think.
> To reach a useful understanding might take two
> to five years, not the five months or so the M&M took.
If this is the sort of "argument" that should impress us, I am scared.
You know, if you need five years for this relatively low-brow and
ill-defined field, it does not mean that everyone needs five years.
You know, there has been a history in our field about a person who
eventually became our colleague. But before it happened, he had rocked
the world of physics even though he was an outsider, a patent officer
in Bern - much like McKitrick is an economist. As you probably know,
the name of the person was Einstein, and there are many more examples
of this sort. The relevance of a statement that someone is an expert
is simply equal to zero, as soon as some technical details can be
discussed explicitly, and I encourage you not to waste my time with
such "arguments" in the future.
> No what I am telling you is that if you use datasets you need to
> understand the strengths and weaknesses of the data sets.
No, sorry, but this is not science.
> One of the
> points that is continually missed in this hoo-ha is that Bradley is
> involved in constructing and maintaining many of the most reliable
> climatological data sets,
Why do you call all this stuff "reliable", especially if it
permanently leads to mutually contradicting conclusions? All your
argument are cyclical. Each individual procedure of MBH should be
trusted because they are experts, and they are experts because their
papers are trusted by other experts, which are trusted by MBH, and so
on. If you don't enrich this line of reasoning by concrete scientific
evidence, then it is just a stupid paranormal game of the children who
want to feel important.
I repeat that the people outside your field are not idiots (at least
some of them are not), and we know that if there were some real
technical evidence justifying the steps, we would have already heard
them.
> his (and Mann's) collaborator Jones is equally
> involved and that Hughes is an expert in tree ring analysis. To
> McKitrick it is just a set of numbers. RTFR and understand them.
Because it IS a set of numbers. Every legitimate number must be
counted, it must be counted once, it must be corretly copied from page
to page, it must be assigned a correct geographic location, and so on.
All these rules have been violated. Conclusions about the 15th century
will always be controversial, but if they depend on a particular,
randomly chosen selection process of the available data, then the
conclusions are clearly worthless, regardless how many times you call
your troubled colleagues "experts".
You seem to disagree with this basic statement that it does not matter
whether someone is called an "expert" once we can judge the specific
technical details - which I find scary. M&M might not be natural
scientists by training, but they obviously - unlike you - understand
and follow the basic rules of science, which are very different rules
from the religion and where the authority is obtained because of
interesting and verifiable results and well-done scientific work, not
from God.
> MBH98 specifically wrote that they did not use the first 25 years of
> Pfister's central european series. It is obvious to anyone who RTFR why
> this was done, and is almost certainly well known to anyone who does
> climatology.
Arrogant and empty statement, once again.
> You have that one screwed up also. First of all, the error in
> Millikan's work was the value used for the viscosity of air.
Yes, I know so very well.
> Secondly
> the so called discarded data was a training set, where Millikan was
> trying out the apparatus and learning how to use it. Anyone who has
> done this experiment needs to practice before getting reliable data.
> http://www1.umn.edu/ships/updates/fraud.htm
You totally missed the point, and it seems that you endorse the
fraudulent experiments. The whole issue - try to read Feynman's
lecture again - is that the Millikan-like experiments done after him
might have gotten the right value immediately because they were
dealing differently with the air viscosity. But the next experiment
only got 55% of the correct value, the following one 60%, and it
continued and only after very many experiments, the measured value
started to oscillate around 100% of the currently believed value. Why
was that? Why did not they obtain 100% already in the second
experiment?
Because these experiments were done in a fraudulent way. If they
obtained a result that was too far from the "generally believed" value
- essentially Millikan's incorrect value - they usually excluded it,
while when it was close to Millikan's value, they were not too
stringent and accepted it. You know, this is bad science, it is fraud.
Do you agree with this judgement? It seems to me that you explicitly
endorse this approach which I find totally unbelievable and
outrageous.
Is not it exactly what you are saying? "Every climatologist knows why
these data should be excluded". Maybe every climatologist knows it,
but it can also be that every climatologist is a bad scientist if they
follow these very strange rules. You only confirmed what I have been
afraid of for some time.
> The problem with this one is that there are several independent
> calculations of the same things, and they agree with MBH98 or are below
> MBH98 until the 20th century.
You have just showed us what sort of "independence" you are talking
about. Every climatologist knows that the results that would disagree
with this dogma must be excluded, right? Really disgusting, and every
concerned scientist should try hard to eliminate this sort of dirty
unscientific methodology from our universities!
By the way. There have been decades of convincing works pointing to
the medieval warm period and little ice age, and the only reason why
so many people suddenly found the new shocking hypothesis of Mann et
al. plausible is that it agreed with some of their ideological
prejudices. You make it more than clear that there does not exist a
single convincing technical argument that would justify all these
steps. You can only praise the "authorities" and "experts".
> As they say, you should look up the meaning of the word GLOBAL.
This is precisely another thing where I've checked that they
understand very well what's going on, while you don't seem to. The
concrete data are *never* GLOBAL, and the temperature is not one
number per the Universe, it is roughly a scalar function of the
position. If one decides to compute some average, there are many
subtleties about the variance and the fluctuations that can have
impact on the conclusions, and in fact there exists no good
methodology to extract a single index from non-uniformly distributed
local data.
> "For the perid 1525-1549 the entries originate mainly from chronicles
> and annals. Accordingly, weather sequences are mainly described at a
> seasonal level; information is missing for 43% of the months and the
> emphasis is on anomalous rather than on ordinary weather."
Well, this is the sort of thinking about the past texts that people
will use to refer to MBH98 in the future. But in the case of the 16th
century chronicles it just does not seem correct to omit the data -
the data that were obviously giving them the warm 15th century (and
the first half of the 16th century, before the little ice age started)
and MBH knew very well that it was so.
All the best
Lubos
Nigel Persaud
> On Tue, 30 Dec 2003 04:31:40 GMT, Josh Halpern
> <j.ha...@incoming.verizon.net> wrote:
>
>
> >
> >Well, googling around with variations of Bradley, Jones, gridded dataset
> >and temperature it was finally located at the Carbon Dioxides IAC at
> >ORNL, in ftp directory http://cdiac.ornl.gov/ftp/ndp020/ in the file
> >jonesnh.dat (the precipitation files are there too along with programs
> >to extract them, but leave that story for tomorrow unless you want to
> >spent a long time working out what is where. (It took me about 30-45
> >minutes to dig this set out from among the other interesting climate
> >data records that are available, including a nice updated one of Jones
> >at the Climate Research Unit in England, and another one at the Climate
> >Data Center in the US. Tomorrow I will try and figure out whether I can
> >get my hands on the underlying 1985/86 DOE reports).
>
> I downloaded Jones' data from the above website.
I suggest that you work with the GISS-2 data. It seems to be
consistent with NDP020, but a later edition and MBH seems to have used
a GISS dataset in between NDP020 and GISS2 (presumably GISS-1). For
example, the Minneapolis data in GISS2, starting in 1820, is pretty
similar to proxy #21 (and perhaps would be identical with GISS1). As
noted before, the Paris precipitation data from GISS (up to linear
transformation) matches exactly to MBH proxy #37, but the earlier
editions are only very close.
I agree that the selection procedures merit much attention.
David Ball
wrote:
>David Ball <wra...@mb.sympatico.ca> wrote:
LOL. No they don't. They go through the mechanics of what they
did, but they in no way show every step they took. For example, they
decided to exactly follow MBH's methodology - they are auditing the
latter's work - by opting NOT to use some of MBH's data. Why was this
done?
David Ball
wrote:
>David Ball <wra...@mb.sympatico.ca> wrote in message news:<2d6fvvca7vv9duisl...@4ax.com>...
No. I'm attempting to reproduce MBH's results independently.
I'm not attempting to update their results using newer data. M&M's
botched analysis has raised a number of questions that need
clarification. MBH is a seminal work in temperature reconstruction and
many others have adopted their techniques. If there are problems with
the author's methodology, then that will have implications down the
line with all the subsequent work.
Steve Schulin
David Ball <wra...@mb.sympatico.ca> wrote:
To illustrate the impact of the audit findings.
Nigel Persaud
CEng and CEur data may be of lower quality than subsequent data, it
surely qualifies as a valid "proxy" and indeed of a higher quality
than virtually any other series in the entire MBH98 repertoire. Would
any of you seriously argue that information from some of the retained
tree ring sites is a better proxy for temperature in 1525-50 than the
1525-50 CEng and CEur series? Of course not.
The suggestion that this deletion is "obvious" to any climatologist
should also be considered in the context of retained Chinese data in
other studies. Crowely-Lowery 2000 use Chu's 1973 phenological series,
which is stated in Zhang 1994 to have been based on incorrect lunar
datings and to have been inaccurate. Even Chinese series based on
correct calendar interpretations are based on flimsier information
than the deleted CEng and CEur series. None of the Chinese data would
survive the exclusion process allegedly used in CEng and CEur.
The periods are also deleted in Bradley-Jones 1993, purporting to
disprove the Little Ice Age, again without annotation. It looks like
MBH simply used leftover digital versions from the earlier article
(the use of summer data is telltale) and no specific consideration was
given to the matter, however ingenious the rationalizations may be at
this time. Nigel
David Ball
<j.ha...@incoming.verizon.net> wrote:
>
>Well, googling around with variations of Bradley, Jones, gridded dataset
>and temperature it was finally located at the Carbon Dioxides IAC at
>ORNL, in ftp directory http://cdiac.ornl.gov/ftp/ndp020/ in the file
>jonesnh.dat (the precipitation files are there too along with programs
>to extract them, but leave that story for tomorrow unless you want to
>spent a long time working out what is where. (It took me about 30-45
>minutes to dig this set out from among the other interesting climate
>data records that are available, including a nice updated one of Jones
>at the Climate Research Unit in England, and another one at the Climate
>Data Center in the US. Tomorrow I will try and figure out whether I can
>get my hands on the underlying 1985/86 DOE reports).
>
>
<snipped>
>We compare this with the data given in the NSM, and find that file # 12,
>has a start date of 1761, in the grid box 62.5 N, 12.5 E, OK, that is
>pretty clearly Trondheim. Well, I went through the list of a few
>hundred stations, and matched them up. My conclusions are:
>
>1 42.5 92.5W minnesota USA 1820 1820
>
>2 47.5 2.5E paris France 1757 48.8 2.5 1757
>3 47.5 7.5E geneva Switzerland 1758 46.2 6.2 1753
>
>4 47.5 12.5E kremunster Austria 1767 48.1 14.1 1767
>
>5 47.5 17.5E wien Austria 1776 48.2 16.4 1775 10
>1777
>6 de bilt Netherlands 1706 52.1 5.2 1706
>
>7 52.5 17.5E berlin Germany 1792 52.5 13.4 1701
>holes to 1775
>8 vilnus Lithuania 1777 54.6 25.3 1777
>miss 1964,39,15-17
>9 57.5 17.5E stockholm Sweden 1756 59.4 18.1 1756
>10 57.5 32.5E st. pertersburg Russia 1752 60.0 30.3 1743
>miss btw 1745-51 1801-04
>11 62.5 7.5E oslo Norway 1816 59.9 10.7 1816
>12 62.5 12.5E trondheim Norway 1761 63.4 10.5 1761
>13 62.5 42.5E archangel Russia 1814 64.6 40.6 1814
>miss 1832
>
I finished going through the raw data on the above website.
Before giving the results, I applied some constraints, rigorous ones I
might add.
1. I made no predetermination about what stations would be
used.
2. While more recent data are available, the goal is to use
the data available to MBH for their 1998 study.
3. Only series starting prior to 1820 were considered as per
MBH. If they started after 1820, they were not considered.
4. Only series that were complete (i.e. no missing years) were
considered. If there was one monthly temperature missing from a given
year, an interpolated value (using rational functions) was fitted in
and the yearly average calculated. I opted not to interpolate missing
yearly values so as not to introduce errors. This rule is strict.
Leningrad was filtered out because it had 2 months of missing data in
1980. Yes, that's picky, but you have to draw the line somewhere.
5. Series were assumed to start after the last missing year in
the record. This has led to some adjustments in the starting times of
the series.
The filtered sites can be seen at:
http://www.mts.net/~shade3/temp.loc
You'll notice that many of the same sites appear in this new list as
on the one Josh put together above, but there are some minor
differences. For example, Berlin starts in 1756 instead of 1792.
Minneapolis disappears since its records don't start until 1859.
Vilnius disappears as it has a lot of missing data and I've already
mentioned my hard-assed filtering of Leningrad. Incidentally, I also
ran this algorithm on the southern hemisphere data and no stations
made it through the filtering process.
I also plotted the data, both the MBH sites (green stars) and
the new sites I've come up with (red circles). You'll notice right
away that there is a narrowing of the filtered records to Western
Europe. The Russian and North American sites disappear.
http://www.mts.net/~shade3/tmp.jpg
Josh Halpern
Lubo? Motl wrote:
>Josh Halpern <j.ha...@incoming.verizon.net> wrote:
>
>
>
>>Before doing your Emily Latella imitation go read C. Pfister "Monthly temperature and precipitation in central Europe, 1525-1979; quantifying documentary evidence on weather and its effects", in Climate since AD 500, pp118, Routledge,. 1992)
>>
>>
>
>Well, I suppose you meant "Climate since AD 1500",
>
Fire that editor
>did not you? I am sort of stunned by your selective approach to the literature.
>
Because what we were concerned with is the data series generated by
Pfister and used by MBH98 and M&M in their reconstructions. Not every
paper Pfister has written, or Mann has written, or Bradley has written,
or Huges has writter.
>Why don't you follow other prescriptions and works by the same author (Pfister, in this case) that indicate that the European climate probably used to be very different to contemporary climate? Lamb 1982, Pfister 1984, Grove 1988 at
>http://www.cx.unibe.ch/hist/fru/fru-adv.htm
>
An amusing link there. You did read it, did you not? Did you notice that
one of the partners in the research program described at the link
(Annual Decadal Variability in Climate in Europe) is Philip Jones from
the Climate Research Unit and also co-author with Michael Mann of a new
multiproxy study which extends the MBH98 work back to the year dot (AD
0) using ice borehole data, more tree rings, etc? Using the methods of
MBH98.
And do you understand the sentence which you have paraphrased: "Previous
studies have provided indications that, European climate was often very
different to contemporary climate (e.g. Lamb 1982, Pfister 1984, Grove
1988)." Why is that in anyway a contradiction of what MBH98 have said?
Have you RTFRs?
>Pfister's methodology from 1992 is certainly not something canonical that one can follow without explicitly mentioning it.
>
RTFR. BTW, it is not his methodology, but the nature of the data from
the period between 1525 and 1550. We did not discuss how he went from
journal entries to numerical values for temperature which would be his
methodology. I know it is enjoyable to throw around the occasional
cannonical, Hamiltonian, spin matrix or whatever, but do stick to the
question under discussion.
>This MBH approach seems to be a selective application of the subset of the rules that give the results that the authors like. Sorry, we physicists might have much higher scientific standards than your field, but all these things that you exposed just seem to be an obvious fraud. Removing some data (25 years!) from the ensemble is a
>very nontrivial decision, and the reader must be informed that it was done
>
Again, the careful, expert readers WERE informed. The rest is an
argument with the Nature editors, but certainly the short form of a
Nature article, even with supplementary material does not allow space
for a detailed discussion of each data series when there are hundreds of
them, whether it meets your standards or not, and frankly my dear who
appointed you editor?. Those interested in understanding the choices are
welcome to RTFRs.
>and why it was done.
>
Why is simple. The data before 1525 in his series was neither
instrumental nor trustworthy, as Pfister acknowledged, as MBH98 knew,
and as M&M apparently did not.
>>Maybe because the particular data series that was being referred to is not reliable before 1550? According to the person who put it together, but what does he know?
>>
>>
>
>Yes, I suppose that none really knows how the medieval weather
>
1550 is at best about 100 years after the medieval period ended. You
either need a history lessons or you are trying to obscure the issue
with an I don't know, no one can know argument. You are also ignoring
the proxy data, which is what we have. The issue is less that we don't
have data, but that the geographical spread of the data is barely
sufficient to support a global analysis. Gobal, you know that word you
have a problem with, not Czech, global, although sometimes we have to
make do with Northern Hemisphere.
>should be properly reconstructed, and which data are really reliable.
>
RTFR Clearly people who have spent a great deal of their professional
lives doing so have a better idea that two guys who spent their spare
time on it for five months, or a couple of astrophysics types or you.
That does not stop the couple of guys, the daytime astrophysics types,
or you from spewing all sorts of libel and opinion though
> But it does not matter.
>
It sure does. Ever hear about GIGO? Data checking and assimilation are
the vital and first part of any analysis. And learning about
paleoclimate does matter if you are not living on Mars, given the macro
anthropic driven changes in our atmosphere and to the surface of the
Earth and the oceans in the last 150 years.
>Once you decide that you want to follow some hints anyway, you must do it properly, which does not seem to be the case here.
>
Frankly, I weary of the self appointed data police. In this point it is
clear that MBH98 did the proper job wrt the Pfister central european
record. If you insist on beating your head against that particular wall
you will only expose yourself as foolish and an ideologue. From what you
are saying, even I, who at best am an amature in this business, can tell
that you do not have a clue of what the best possible analysis would be.
Now true, we have not yet traced every thread, but for those we have
MBH98 made good choices.
>I don't care about the artificially created fame of your colleagues: the available hard data simply suggest that the Czech chronicles from 1500
>
Which Czech chronicles are you talking about now? I am sure Pfister and
his merry band of ADVICE folk would be interested in looking at them,
especially if there is a daily weather log. Drop them a line. Of course,
one mention in the middle of 1000 pages of the weather is not very
useful. "It was a warm May day and we tossed a couple of guys out the
window". Diaries are good, which record such things as the start of
harvests over many years, etc. Some hard work with a good diary can
produce a useful record. Though perhaps you have not noticed, the globe
is not the Czech Republic.
> or so are more reliable than articles in Nature 1998 about the same subject because the chronicles were written as a description of reality while MBH98 is a result of a random selective process of the data meant to find the desired results.
>
>
Right, sure. Take your meds and see a doctor in the morning. In case you
have not noticed, your chronicles are about the Czech and Moravian
lands, MBH98 is a GLOBAL reconstruction. It appears that believing one's
country is the entire world is an illness not restricted to the US.
>>An audit assumes that one is familiar with the underlying material. M&M claim to evaluate a calculation that uses a mass of climatological data, much of which has been created by, collected and evaluated by two experts in the field.
>>
>>
>
>Well, the mere non-technical statement that "someone is an expert" does not mean much.
>
Actually, I went beyond that. I pointed out that both Bradley and
Hughes, in 1998 were experts in creating and evaluating the particular
kinds of data series used in MBH98, that they had created many of the
data sets that were used, or were collaborators with many of the others
who had.
>Well, if there are serious problems with their work,
>
A WAGNEA (Wild assed guess no explanation available)
>then these people should *not* be treated as experts, in fact they should be fired. The rule "22" in a list is "don't always believe experts", see e.g.
>
>
Which you and your ilk interpret as never believe someone who knows more
than you.
>http://www.engin.umich.edu/dept/aero/people/faculty/bernstein/research_advice.pdf
>
>You probably don't understand how serious the situation is. Most of your field now seems to be a bubble containing arrogant pseudoscientists that call one another "experts" even though they are doing worse job than the outsider economists, and the question is whether it is the case or not. If it is the case, various people should forget about being called an "expert", and if a deliberate fraud was done in some important cases - whose impact might have been
>wasting trillions of dollars worldwide - such people should definitely be arrested, I think.
>
>
Well, you are wrong about just everything else, and this to. Your naked
assertions above are merely attitude, and I don't work in the area of
climate studies.
>>To reach a useful understanding might take two to five years, not the five months or so the M&M took.
>>
>>
>
>If this is the sort of "argument" that should impress us, I am scared. You know, if you need five years for this relatively low-brow and ill-defined field, it does not mean that everyone needs five years.
>
Wanna bet? Judging from your spew I estimate that you will need forever.
The arrogance of physicists is well known, but you ain't Pauli.
> You know, there has been a history in our field about a person who eventually became our colleague. But before it happened, he had rocked the world of physics even though he was an outsider, a patent officer in Bern
>
Oh, you mean the fellow who got his Ph.D from the University of Zurich.
Contrary to what you think he was a well educated theoretical physicist.
>- much like McKitrick is an economist. As you probably know, the name of the person was Einstein, and there are many more examples of this sort.
>
Mostly when you start looking at them in detail they turn out to be
urban legends, like your statements about Millikan and the oil drop
experiment which was snipped.
> The relevance of a statement that someone is an expert is simply equal to zero, as soon as some technical details can be discussed explicitly, and I encourage you not to waste my time with such "arguments" in the future.
>
>
Posturing on your part, is your problem
>>No what I am telling you is that if you use datasets you need to understand the strengths and weaknesses of the data sets.
>>
>>
>
>No, sorry, but this is not science.
>
>
Remember that little statement GIGO. If M&M used the first 25 years of
the Pfister record they made a mistake. David Ball has pointed this out
to you.
>>points that is continually missed in this hoo-ha is that Bradley is involved in constructing and maintaining many of the most reliable climatological data sets,
>>
>>
>
>Why do you call all this stuff "reliable",
>
Ah, I did not call it reliable, I called it the most reliable. Now here
is an introduction to real science friend, not the pretend stuff one
does in beginner laboratories which always work. Many times one has
large data sets with a great deal of variability and even some errors as
well as statistical noise. Unfortunately, one cannot get better data
sets and often one needs to use these data sets to answer important
questions. Statistical methods of treating such situations are vital in
many fields including weather, climate, medicine (pretty much anything
biological) and yes, even physics. Using these methods requires careful
analysis of the data and an understanding of how errors in the data
could affect the data set. MBH98 is pretty much all a discussion of one
such method. It is not trivial.
>especially if it permanently leads to mutually contradicting conclusions?
>
Such as?
>All your argument are cyclical. Each individual procedure of MBH should be trusted because they are experts, and they are experts because their papers are trusted by other experts, which are trusted by MBH, and so
>on. If you don't enrich this line of reasoning by concrete scientific evidence, then it is just a stupid paranormal game of the children who want to feel important.
>
>
A perfect description of yourself. Note that contrary to what you
erroniously claim, what I have been doing here is carefully going over
what is available and identifying what was done in detail. As a
non-paleoclimatologist I have to spend considerable time on each point,
but I have been able to clear up some of the questions M&M asked, and
explain what was done in MBH98. For example, we now understand why the
first 25 years of the Pfister central european record were not used in
MBH98. David Ball has been looking at the statistical analysis, where he
has more practice. I might go learn about principal component analysis,
but I would be very prone to learners mistakes. David has experience
with weather data sets (closely related) and this kind of analysis, so
he is much better suited to that.
In looking for answers to the questions that M&M asked, I do have to be
aware that it would be hard to find two people (Bradley and Hughes) who
would be more informed about the various data series used, and one has
to respect that. I also have to acknowledge that Bradley and Hughes
would have the best versions of the various data sets they were involved
in creating.
Again, one of the most interesting things in this whole to-do, is how
denialists have pushed Mann forward and ignored Bradley and Hughes (and
now Jones).
>I repeat that the people outside your field are not idiots (at least some of them are not),
>
Someone who is clearly naive about a field (M&M, Soon and Baliunas) who
jumps in to challenge a bunch of people who clearly are experts, and
produces pieces of muck such as were published in E&E is either
asymototically approaching idiocy or trolling for those who have already
reached the limit. BTW what do you think my field is? RTFR
>and we know that if there were some real technical evidence justifying the steps, we would have already heard
>them.
>
>
You did, and if you RTFRs you would understand.
>>his (and Mann's) collaborator Jones is equally involved and that Hughes is an expert in tree ring analysis. To
>>McKitrick it is just a set of numbers. RTFR and understand them.
>>
>>
>
>Because it IS a set of numbers. Every legitimate number must be counted, it must be counted once, it must be corretly copied from page to page, it must be assigned a correct geographic location, and so on.
>
Note the word legitimate. Also note that if you are seeking global
coverage (you know, that part of the earth's surface not in the Czech
Republic) information from an oversampled area may not be very
important, you might use the most reliable (that is why they are called
experts fella) series and exclude others that are less reliable.
>All these rules have been violated. Conclusions about the 15th century will always be controversial, but if they depend on a particular, randomly chosen selection process of the available data, then the conclusions are clearly worthless, regardless how many times you call your troubled colleagues "experts".
>
And you read the rules in the big book of science. Which edition?
You also need to look at
http://www.ngdc.noaa.gov/paleo/pubs/mann2003b/mann2003b.html and see all
the new data that has been added, and that it has not changed the
reconstruction much.
You also appear to think that there are many records extending back to
1500. Hint: there are not
>You seem to disagree with this basic statement that it does not matter whether someone is called an "expert" once we can judge the specific technical details -
>
From everything you have written you are ill equipped to look at the
technical details. Let me put it this way, often you have to be an
expert to judge the technical details. This appears to be such a case
>which I find scary. M&M might not be natural scientists by training, but they obviously - unlike you - understand and follow the basic rules of science, which are very different rules from the religion and where the authority is obtained because of interesting and verifiable results and well-done scientific work, not from God.
>
>
To believe that M&M were disinterested seekers after truth is to repeat
Mann's error. As to the "rules of science" I would again ask which big
book of science you have been reading.
>>MBH98 specifically wrote that they did not use the first 25 years of Pfister's central european series. It is obvious to anyone who RTFR why this was done, and is almost certainly well known to anyone who does
>>climatology.
>>
>>
>
>Arrogant and empty statement, once again.
>
>
Right. Arrogant and empty, to point out that if you RTFR, you will see
why MBH98 was correct in excluding the period 1525-1549 in the Pfister
record.
>>You have that one screwed up also. First of all, the error in
>>Millikan's work was the value used for the viscosity of air.
>>
>>
>
>Yes, I know so very well.
>
>
>>Secondly the so called discarded data was a training set, where Millikan was trying out the apparatus and learning how to use it. Anyone who has done this experiment needs to practice before getting reliable data. http://www1.umn.edu/ships/updates/fraud.htm
>>
>>
>
>You totally missed the point, and it seems that you endorse the fraudulent experiments. The whole issue - try to read Feynman's lecture again - is that the Millikan-like experiments done after him might have gotten the right value immediately because they were dealing differently with the air viscosity. But the next experiment only got 55% of the correct value, the following one 60%, and it continued and only after very many experiments, the measured value started to oscillate around 100% of the currently believed value. Why was that? Why did not they obtain 100% already in the second experiment?
>
>Because these experiments were done in a fraudulent way. If they obtained a result that was too far from the "generally believed" value - essentially Millikan's incorrect value - they usually excluded it, while when it was close to Millikan's value, they were not too stringent and accepted it. You know, this is bad science, it is fraud.
>Do you agree with this judgement? It seems to me that you explicitly endorse this approach which I find totally unbelievable and outrageous.
>
>
First of all, the check on the Millikan oil drop experiment was an X-ray
scattering measurement which did not depend on the viscosity of air.
http://www.whfreeman.com/modphysics/PDF/3-1c.pdf top of the third page.
I learned this from a professor who was involved in the X-Ray
experiments. Feynman was telling you a story, and if you doubt me, go
find the experimental papers he talked about. You could do the oil drop
experiment forever, and you would get the same value of the electron
charge that Millikan did. What you would have to do is measure the air
viscosity, a very different kettle of fish.
>Is not it exactly what you are saying? "Every climatologist knows why
>these data should be excluded". Maybe every climatologist knows it,
>but it can also be that every climatologist is a bad scientist if they
>follow these very strange rules. You only confirmed what I have been
>afraid of for some time.
>
>
>
>>The problem with this one is that there are several independent
>>calculations of the same things, and they agree with MBH98 or are below
>>MBH98 until the 20th century.
>>
>>
>
>You have just showed us what sort of "independence" you are talking about. Every climatologist knows that the results that would disagree with this dogma must be excluded, right? Really disgusting, and every concerned scientist should try hard to eliminate this sort of dirty unscientific methodology from our universities!
>
>
Blather.
>By the way. There have been decades of convincing works pointing to the medieval warm period and little ice age, and the only reason why so many people suddenly found the new shocking hypothesis of Mann et al. plausible is that it agreed with some of their ideological prejudices. You make it more than clear that there does not exist a single convincing technical argument that would justify all these steps. You can only praise the "authorities" and "experts".
>
>
There is an old saying that it is not what you don't know that hurts
you, but what you think you know and is false that gets you. The best
answer to this, is of course, in the IPCC TAR
http://www.grida.no/climate/ipcc_tar/wg1/070.htm
**********************************
The terms “Little Ice Age” and “Medieval Warm Period” have been used to
describe two past climate epochs in Europe and neighbouring regions
during roughly the 17th to 19th and 11th to 14th centuries,
respectively. The timing, however, of these cold and warm periods has
recently been demonstrated to vary geographically over the globe in a
considerable way (Bradley and Jones, 1993; Hughes and Diaz, 1994;
Crowley and Lowery, 2000). Evidence from mountain glaciers does suggest
increased glaciation in a number of widely spread regions outside Europe
prior to the 20th century, including Alaska, New Zealand and Patagonia
(Grove and Switsur, 1994). However, the timing of maximum glacial
advances in these regions differs considerably, suggesting that they may
represent largely independent regional climate changes, not a
globally-synchronous increased glaciation (see Bradley, 1999). Thus
current evidence does not support globally synchronous periods of
anomalous cold or warmth over this timeframe, and the conventional terms
of “Little Ice Age” and “Medieval Warm Period” appear to have limited
utility in describing trends in hemispheric or global mean temperature
changes in past centuries.
*****************************
>
>
>>As they say, you should look up the meaning of the word GLOBAL.
>>
>>
>
>This is precisely another thing where I've checked that they understand very well what's going on, while you don't seem to. The concrete data are *never* GLOBAL, and the temperature is not one number per the Universe, it is roughly a scalar function of the position. If one decides to compute some average, there are many subtleties about the variance and the fluctuations that can have impact on the conclusions, and in fact there exists no good methodology to extract a single index from non-uniformly distributed local data.
>
>
Been there, done that (not me, but MBH and co.)
http://www.ngdc.noaa.gov/paleo/ei/ei_reconsb.html
http://www.ngdc.noaa.gov/cgi-bin/paleo/mannplot2.pl
and, of course, the several papers starting with MBH98.
I am glad to finally understand that what is said in some Czech journal
is not representative of global temperatures.
>>"For the perid 1525-1549 the entries originate mainly from chronicles and annals. Accordingly, weather sequences are mainly described at a seasonal level; information is missing for 43% of the months and the emphasis is on anomalous rather than on ordinary weather."
>>
>>
>
>Well, this is the sort of thinking about the past texts that people will use to refer to MBH98 in the future. But in the case of the 16th century chronicles it just does not seem correct to omit the data - the data that were obviously giving them the warm 15th century (and the first half of the 16th century, before the little ice age started) and MBH knew very well that it was so.
>
>
Again, RTFR: First of all the Pfister record does not include any data
from the 15th century. Second the little ice age in Europe started about
1600. MBH98 start using the Pfister data in 1550. Pfister finds the
period from 1530 to 1560 to be warm, "almost as warm as the 20th
century", so by starting in 1550, MBH98 have used a series that starts
on a relatively high point. If you are talking about other journals,
hey, call Pfister and the ADVICE folk up.
If you are talking about Pfister's series including journal information
from the middle 16th to 17th century: from Pfister: "In the second
period, 1550-1658 monthly data from weather diaries and personal papers
are abundant. The few missing monthly data 7% are concentrated in the
months from October to December."
David Ball
<j.ha...@incoming.verizon.net> wrote:
>
>
>David Ball wrote:
>
>>On Tue, 30 Dec 2003 04:31:40 GMT, Josh Halpern
>><j.ha...@incoming.verizon.net> wrote:
>> I think you'll find the precipitation data in:
>>
>>http://cdiac.ornl.gov/ftp/tr051/
>>
>Darn, you stole my fire. My list is a bit different
>
>Note that the starting dates for 6, 7, and 8 are cyclically permuted.
>
I finished going through the precipitation data. I can find no
permutation unless the data file has the wrong names associated with
the wrong data. Series 37 is indeed Paris. The data have the 1902-1980
mean removed and are divided by 10. Nothing significant. Many of the
sites we found in common are still there after I processed the data,
but some have disappeared due to missing data. Bombay, Charleston,
Prague, New Haven. Instead of the latter, Boston appears. I have a
complete file and map of the my sites and MBH's. I will post it later
today.
On another note, I did a little thinking about Nigel's point
about using updated data - he suggested the GISS-2 data. I'm curious
to see what, if any, the differences are between them. Does anyone
have a url for where one can find the data?
Nigel Persaud
David, here is the url for the data.
ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/v2/zipd/v2.prcp.zip. The GISS2
temperature data has Minneapolis starting in 1820, so it's closer to
what Mann used than Climate since 1500, which was his (incorrect)
reference.
My take is as follows:
32- Madras
33 - Definitely not Bombay data. The closest match that I found was
to Philadelphia data, but the match was not very good.
34 - Definitely Toulouse. This is mislocated in North America by
Mann.
35 - Definitely Marseilles.
36 - I agree that Boston is the closest match. The correlation to
GISS2 is over 0.82, which is strong evidence that this is the right
series. He probably is using GISS1. The source of the data before the
start of Boston is a puzzle.
37 - Definitely Paris, mislocated to North America.
38 - Klagenfurt in an as yet unlocated version
39 - definitely Prague
40 - Kew Gardens in an unlocated version
41 - Edinburh in an unlocated version
In addition to the big geographical errors for Paris, Toulouse, Boston
and the pending large geographical error for the series located at
Bombay, there are small geographical errors (typically 2 grid boxes
off) in all sites except Madras.
This is definitely a total screw-up by MBH. Your spotting on Toulouse
and others was good. Nigel
David Ball
wrote:
>David Ball <wra...@mb.sympatico.ca> wrote in message news:<p4hlvv0f0f501bh7k...@4ax.com>...
Damn! I was hoping there was another dataset that I'd missed.
I downloaded this data last night. There are actually two sets of
data, one raw the other quality controlled. The quality controlled
data is much different than the raw one with Minneapolis starting in
1859.
>My take is as follows:
>32- Madras
>33 - Definitely not Bombay data. The closest match that I found was
>to Philadelphia data, but the match was not very good.
>34 - Definitely Toulouse. This is mislocated in North America by
>Mann.
>35 - Definitely Marseilles.
>36 - I agree that Boston is the closest match. The correlation to
>GISS2 is over 0.82, which is strong evidence that this is the right
>series. He probably is using GISS1. The source of the data before the
>start of Boston is a puzzle.
>37 - Definitely Paris, mislocated to North America.
>38 - Klagenfurt in an as yet unlocated version
>39 - definitely Prague
>40 - Kew Gardens in an unlocated version
>41 - Edinburh in an unlocated version
You don't understand. I didn't make any attempt by way of
correlation to match the sites from MBH with the data from tr051
files. I physically went through the files and calculated the sites
from the data using the same criteria I used for the temperature
data....
1. Only series starting before 1820 were used.
2. Only series with data to 1980 were used.
3. If a yearly record was flagged as missing and only one
monthly value was needed, an interpolated value was determined using
rational functions. No attempt was made to interpolate from year to
year.
4. A continuous record was required for each site. If that
didn't happen the data were thrown out.
>
>In addition to the big geographical errors for Paris, Toulouse, Boston
>and the pending large geographical error for the series located at
>Bombay, there are small geographical errors (typically 2 grid boxes
>off) in all sites except Madras.
>
>This is definitely a total screw-up by MBH. Your spotting on Toulouse
>and others was good. Nigel
Incorrect. In looking at the various datasets, whether they be
USHCN, GHCN, CRU, the tree-ring data, ... each and every one shows
nuances. That is one of the points the various authors working with
these data make. They are not identical and because of that, they
allow for checks to be made on each other.
Having said that, the fact that I can easily reproduce much of
the data is meaningful. We're hoping we have exactly the same data
that MBH used, but we don't know for sure. For me, in trying to
reproduce the author's work, it is enough that there is significant
overlap in the site data. You simply cannot make the leap from having
the central lat/lon of a 5x5 box to knowing exactly what site the
author's were working with.
Josh Halpern
David Ball wrote:
>On 6 Jan 2004 (Nigel Persaud) wrote:
>
>
>>David Ball <wra...@mb.sympatico.ca> wrote
>>
>>
>>>On Wed, 31 Dec 2003 02:41:59 GMT, Josh Halpern
>>>
>>>
>>>>David Ball wrote:
>>>>
>>>>
>>>>>On Tue, 30 Dec 2003 04:31:40 GMT, Josh Halpern
>>>>>SNIP....
SNIP....
>
> Damn! I was hoping there was another dataset that I'd missed.
>I downloaded this data last night. There are actually two sets of
>data, one raw the other quality controlled. The quality controlled
>data is much different than the raw one with Minneapolis starting in
>1859.
>
>
>
>>My take is as follows:
>>32- Madras
>>
Agreed
>>33 - Definitely not Bombay data. The closest match that I found was
>>to Philadelphia data, but the match was not very good.
>>
Take a look at Figure 1 in MBH98
http://www.ngdc.noaa.gov/paleo/pubs/mann1998/frames.htm
It definitely shows Bombay as a precipitation site. The best version of
this figure can be found at
ftp://eclogite.geo.umass.edu/pub/mann/ONLINE-PREPRINTS/MultiProxy/ as
fig1a-color.ps Let's put it this way, I have a lot less trust in
pcproxy and the ftp sites then I do in MBH98. Ideally one wants a match
between everything, but you cannot always get it.
<ftp://eclogite.geo.umass.edu/pub/mann/ONLINE-PREPRINTS/MultiProxy/fig1a-color.ps>
>>34 - Definitely Toulouse. This is mislocated in North America by
>>Mann.
>>
Again, look at the figure (do yourself a favor use Ghostview and the
postscript file) Charleston is definitely there.
>>35 - Definitely Marseilles.
>>
Agreed
>>36 - I agree that Boston is the closest match. The correlation to
>>GISS2 is over 0.82, which is strong evidence that this is the right
>>series. He probably is using GISS1. The source of the data before the
>>start of Boston is a puzzle.
>>
Don't forget New Haven. The figure shows a better match to New Haven, at
least to someone who spent a lot of time on the train between NY and
Providence.
>>37 - Definitely Paris, mislocated to North America.
>>
Agreed
>>38 - Klagenfurt in an as yet unlocated version
>>
or Wien
>>39 - definitely Prague
>>40 - Kew Gardens in an unlocated version
>>41 - Edinburh in an unlocated version
>>
Where did Ellabus go? It's clearly in the NSM and the figure.
josh halpern
Nigel Persaud
independent test. For example, the map shows Kew Gardens 2 grid-boxes
to the east of where it really is - but in the location of the grid
box. See also below.
> >>My take is as follows:
> >>32- Madras
> >>
> Agreed
>
> >>33 - Definitely not Bombay data. The closest match that I found was
> >>to Philadelphia data, but the match was not very good.
> >>
>
> Take a look at Figure 1 in MBH98
> http://www.ngdc.noaa.gov/paleo/pubs/mann1998/frames.htm
> It definitely shows Bombay as a precipitation site. The best version of
> this figure can be found at
> ftp://eclogite.geo.umass.edu/pub/mann/ONLINE-PREPRINTS/MultiProxy/ as
> fig1a-color.ps Let's put it this way, I have a lot less trust in
> pcproxy and the ftp sites then I do in MBH98. Ideally one wants a match
> between everything, but you cannot always get it.
> <ftp://eclogite.geo.umass.edu/pub/mann/ONLINE-PREPRINTS/MultiProxy/fig1a-color.ps>
The map renders the grid box. However, the data does not tie in at all
to actual Bombay data.
> >>34 - Definitely Toulouse. This is mislocated in North America by
> >>Mann.
> >>
> Again, look at the figure (do yourself a favor use Ghostview and the
> postscript file) Charleston is definitely there.
Again, the map renders the grid-box file. The data has a 100% match to
data from Toulouse.
> >>35 - Definitely Marseilles.
> >>
> Agreed
>
> >>36 - I agree that Boston is the closest match. The correlation to
> >>GISS2 is over 0.82, which is strong evidence that this is the right
> >>series. He probably is using GISS1. The source of the data before the
> >>start of Boston is a puzzle.
> >>
> Don't forget New Haven. The figure shows a better match to New Haven, at
> least to someone who spent a lot of time on the train between NY and
> Providence.
I've compared the New Haven data to the MBH98 data and it doesn't
match. Boston matches closer. But I think that MBH is using data in
between TR051 and GISS2 - probably GISS1.
> >>37 - Definitely Paris, mislocated to North America.
> >>
> Agreed
>
> >>38 - Klagenfurt in an as yet unlocated version
> >>
> or Wien
The data ties better to Klagenfurt, but it is not the exact match that
one finds for Toulouse and PAris.
> >>39 - definitely Prague
> >>40 - Kew Gardens in an unlocated version
> >>41 - Edinburh in an unlocated version
> >>
> Where did Ellabus go? It's clearly in the NSM and the figure.
41,42 - my bad. 41 - Definitely Eallabus.
42- definitely Edinburgh.
Again, the map in MBH98 just renders the grid-box information. Series
#33 does not have Bombay precipitation information. I don't know where
the data comes from, but not Bombay. Of all the sites starting between
1800 and 1830, the series correlates most closely to Philadelphia, but
especially well.
As above, a different version for Boston, Klagenfurt and Kew Gardens
needs to be identified. The other series are firmly tied to GISS data,
although not necessarily the location stated. Nigel.
Josh Halpern
Nigel Persaud wrote:
>Josh, the map is a rendering of the grid-box locations. It is not an
>independent test. For example, the map shows Kew Gardens 2 grid-boxes
>to the east of where it really is - but in the location of the grid
>box. See also below.
>
True, but the issue of Bombay comes up.....
>>>>My take is as follows:
>>>>32- Madras
>>>>
>>>>
>>>>
>>Agreed
>>
>>
>>
>>>>33 - Definitely not Bombay data. The closest match that I found was
>>>>to Philadelphia data, but the match was not very good.
>>>>
>>>>
>>>>
>>Take a look at Figure 1 in MBH98
>>http://www.ngdc.noaa.gov/paleo/pubs/mann1998/frames.htm
>>It definitely shows Bombay as a precipitation site. The best version of
>>this figure can be found at
>>ftp://eclogite.geo.umass.edu/pub/mann/ONLINE-PREPRINTS/MultiProxy/ as
>>fig1a-color.ps Let's put it this way, I have a lot less trust in
>>pcproxy and the ftp sites then I do in MBH98. Ideally one wants a match
>>between everything, but you cannot always get it.
>><ftp://eclogite.geo.umass.edu/pub/mann/ONLINE-PREPRINTS/MultiProxy/fig1a-color.ps>
>>
>>
>
>The map renders the grid box. However, the data does not tie in at all
>to actual Bombay data.
>
>
There is where, IMHO, there is a problem. How do you know what you are
calling the "actual data" is the actual data used in MBH98. In the
absence of a real audit trail, no one does and no scientific paper,
other than those connected to drug trails has a real audit trail. This
is why David's approach is the one usually taken, and one that works.
josh halpern
David Ball
<j.ha...@incoming.verizon.net> wrote:
>
>Take a look at Figure 1 in MBH98
>http://www.ngdc.noaa.gov/paleo/pubs/mann1998/frames.htm
>It definitely shows Bombay as a precipitation site. The best version of
>this figure can be found at
>ftp://eclogite.geo.umass.edu/pub/mann/ONLINE-PREPRINTS/MultiProxy/ as
>fig1a-color.ps Let's put it this way, I have a lot less trust in
>pcproxy and the ftp sites then I do in MBH98. Ideally one wants a match
>between everything, but you cannot always get it.
><ftp://eclogite.geo.umass.edu/pub/mann/ONLINE-PREPRINTS/MultiProxy/fig1a-color.ps>
At this stage, Josh, I'm beginning to think it doesn't much
matter. I've gone through a lot of different datasets and crunched the
numbers and there seems to be a hard core of sites that are easily
identifiable from every one with a number of wanna-be's that appear
and disapper depending on the dataset being used.
You're right about the figure, but I don't think the
differences we're discovering are going to matter much in the grand
scheme of things by the time we're done. I've posted the stations I
found by crunching the TR051 file at:
http://www.mts.net/~shade3/precip.loc
and the associated image showing the locations (red dots) with the MBH
sites (green stars) at:
http://www.mts.net/~shade3/jones_precip.gif
What's important, IMHO, is that I've been able to duplicate the data
from MBH for the common stations.
A couple of other points - might as well post this here rather
than starting a new thread - regarding MBH's PC's...
1. I've had to make some adjustments to refine the PC
algorithm I'm using. I was retaining way too many PC's to make them
functional and too many were hovering right above the noise level.
I've plotted so-called scree diagrams for each of the eigenvalues in
an effort fine-tune the determination of exactly which PC's to retain.
Rule N that MBH uses has the annoying habit of being too
conservative at times. A scree diagram basically plots the eigenvalues
in descending order with the number of the eigenvalue on the abscissa
http://www.mts.net/~shade3/scree.gif
The diagram is so-named for the area of a mountainside where the
flotsam and jetsam that tumbles down the slope collects. In the
example, this would be at or about the 4th eigen value. The theory
states that you should only retain eigenvalues above the scree
position, the remaining positions be contaminated by noise. This
example is the BACKTO_1600 case from the noamer tree ring dataset. The
red line is the position dictated by Kaiser's Criterion. In this case,
it would suggest retaining a much smaller number PC's (3 in this
case). Kaiser's criterion gives a number in the 40's. Three is much
lower than that retained by MBH, but this illustrates one of the
problems with using Rule N.
2. This is where things get really dicey. I downloaded the
HadCRUT2 dataset from the CRU website and made my way through it.
According to MBH, they were able to identify 1082 continuous series
from 1902 to 1995 covering 1128 consecutive months. They also retained
a subset of continuous data from 1854 to 1995 amounting to 271(?).
I couldn't even get close to those numbers with the CRU data.
For the 1902 to 1995 period, I found 291 series from the global
dataset that were continous throughout the period. For the 1854 to
1995 period, I only found 61 series. I'm not sure what they did to
fill in the blanks but there's definitely a problem here. If you look
at MBH's figure 1 and the distribution of the sites, there are an
awful lot of sites over marine areas. Given the reliance on SST
values, it's difficult to imagine being able to get soooo many
complete series over the water.
Compare the layout in Figure 1 with my two figures...
http://www.mts.net/~shade3/c20.gif
which shows the points retained for the 1902 to 1995 period and
http://www.mts.net/~shade3/all.gif
which shows the points retained for the 1854 to 1995 period. While
they maintain the same general pattern, you can see the envelope
created used by MBH is much greater. I'm not sure that is justified.
It will be interesting to see what the PC's from this smaller set of
data look like.
Nigel Persaud
> HadCRUT2 dataset from the CRU website and made my way through it.
> According to MBH, they were able to identify 1082 continuous series
> from 1902 to 1995 covering 1128 consecutive months. They also retained
> a subset of continuous data from 1854 to 1995 amounting to 271(?).
> I couldn't even get close to those numbers with the CRU data.
> For the 1902 to 1995 period, I found 291 series from the global
> dataset that were continous throughout the period. For the 1854 to
> 1995 period, I only found 61 series. I'm not sure what they did to
> fill in the blanks but there's definitely a problem here.
This issue is specifically raised in the M&M correspondence with Mann.
Sounds like you're agreeing with them more and more on the problems in MBH98. Nigel
David Ball
wrote:
>> 2. This is where things get really dicey. I downloaded the
LOL. If you think one thing is more and more, Nigel. Given
that I've found myriad problems with M&M ranging from sloppiness to
downright neglect, I think MBH are pretty safe. My question to you is,
why are you in such a rush to pass judgement on MBH when we're still
crunching the numbers?
Per
> Your statement would be valid if the M&M had actually done an
> audit. They didn't. That's a huge problem. To do an audit, M&M would
> have needed to use the same data that MBH did. They did not, both by
> getting ahold of a corrupt dataset and not having the wherewithal to
> realize that you cannot do any kind of good science with corrupt data.
M&M asked for and got the dataset from Mann, and asked Mann and
colleague to help them with their reconstruction. Moreover, they had
problems with the dataset, and directly asked Mann to confirm that it
was the correct dataset. It seems M&M tried to act in good faith...
You can see the e-mail correspondence between M&M and Mann here:
http://www.uoguelph.ca/~rmckitri/research/Response.Oct29.pdf
Re: the "Pfister data" of 1525, I am somewhat bemused. It may be that
it is correct to exclude that data, but I don't see that as the
relevant issue. For me, the issue is that you are meant to be able to
use the information in a paper to repeat the study, to be able to
repeat and verify it; that means, can you unequivocally determine from
the MBH paper what (parts of) datasets were used and what was done ?
If you are having to read a reference cited by MBH, then guess that
bits were omitted, that is in my opinion poor practice.