Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

a problem with temperature records

38 views
Skip to first unread message

josephus

unread,
Apr 13, 2013, 9:00:09 AM4/13/13
to

I �have some data sets ��ENGLAND 1729-1970, �NOAA 1880-2012 �GISS ==
NOAA data. �I even have �a data set for �ICE �but that is not my
problem.

the problem �is �when I �run a �least squares algorithm over the �data
set. �should I change �the �extent, �the slope �increases (??).

so I �plotted the aggregate list of slopes �// mod X. �then I plot that
list. �and �this is were the problem exceeds my programming and
mathematics. �I can take slope data �at any scale ��(all slopes are
2012 to N). ��I take all my slopes upward �If I reversed the process I
would expect to get negative slopes.


this is a plot of �the slopes over �the data NOAA ��and �excluding data
set downward slopes. ���this is an artifact of GW. ���the ranges of the
data are �increasing �together. �the result is my BEST FIT line �also
is
positive and ��increasing.

the big question is �WHAT KIND of PROCESSING �caused �the RAW DATA to
show irregular slopes like these

If you want I can send you the RAW data �and �the slope list.

this is �a much more complex problem than just displaying odd data.
every data set I have processed agrees with the general terms of this
program ��they all plot upward �except for ENGLAND �it slopes downward

I believe this data set was conformed by a process. ��but �the
artifacts
have strange properties �it may be a trap door function, �but I am
interesstd in ��understanding �the slope data that I have �produced.

josephus


MY regular plot [scatter gram]

���My linear data is +
���������The data is *
�����Collisions are �%
Yscale = 4.92133509966079E+0001
-0.1371415| ���������0.4063938| ���������0.6756460|
--------------------------------------------------------
2012 40 | �������������������. ������������������%|

2010 39 | �������������������. �����������������*+|

2008 39 | �������������������. �����������������% |

2006 38 | �������������������. ����������������% �|

2004 37 | �������������������. ���������������*+ �|

2002 36 | �������������������. ��������������*+ ��|

2000 35 | �������������������. �������������*+ ���|

1998 34 | �������������������. ������������* + ���|

1996 33 | �������������������. �����������* + ����|

1994 32 | �������������������. ����������* + �����|

1992 32 | �������������������. ����������* + �����|

1990 31 | �������������������. ���������* + ������|

1988 31 | �������������������. ���������*+ �������|

1986 29 | �������������������. �������* �+ �������|

1984 29 | �������������������. �������* + ��������|

1982 28 | �������������������. ������* �+ ��������|

1980 27 | �������������������. �����* �+ ���������|

1978 26 | �������������������. ����* �+ ����������|

1976 26 | �������������������. ����* �+ ����������|

1974 26 | �������������������. ����* + �����������|

1972 26 | �������������������. ����*+ ������������|

1970 27 | �������������������. �����% ������������|

1968 27 | �������������������. ����+* ������������|

1966 28 | �������������������. ���+ �* �����������|

1964 29 | �������������������. ���+ ��* ����������|

1962 29 | �������������������. ��+ ���* ����������|

1960 28 | �������������������. �+ ���* �����������|

1958 27 | �������������������. �+ ��* ������������|

1956 26 | �������������������. + ��* �������������|

1954 27 | �������������������. + ���* ������������|

1952 26 | �������������������.+ ���* �������������|

1950 27 | �������������������+ �����* ������������|

1948 28 | �������������������+ ������* �����������|

1946 28 | ������������������+. ������* �����������|

1944 27 | �����������������+ . �����* ������������|

1942 24 | �����������������+ . ��* ���������������|

1940 20 | ����������������+ �* �������������������|

1938 16 | ���������������% ��. �������������������|

1936 12 | �����������* ��+ ��. �������������������|

1934 9 �| ��������* ����+ ���. �������������������|

1932 8 �| �������* ����+ ����. �������������������|

1930 3 �| ��* ���������+ ����. �������������������|

1928 3 �| ��* ��������+ �����. �������������������|

1926 1 �| * ���������+ ������. �������������������|

--------------------------------------------------------
�7 lines
�����������������������������| ������*
�����������������������������| �����**
�����������������������������| �����***
�����������������������������| �����****
�����������������������������| �����****
������������* ���������������| �����**** ** �����*
����������* * ���** �* ��* ��|* ��* **** **********
�������������������1111111111|22222222223333333333
����������1234567890123456789|01234567890123456789

Axel Vogt

unread,
Apr 13, 2013, 5:35:26 PM4/13/13
to
...

I tried to understand you question - but I am a bit lost (may be it is
my English,
and I do not understand you abbreviations).

Could you provide your data (by a link?) and clearly say what you do or
expect or
want? It is not clear what "least squares algorithm over the data set"
should be,
or "negative slopes".

Do you want some approximating 'curve' to your data? Or just a 'line'?
Or what?

Herman Rubin

unread,
Apr 13, 2013, 5:36:45 PM4/13/13
to
The way you entered your data, I cannot get your plot into my
reply. �However, the big error you made is assuming that the
data should be explained by a linear process. �It obviously
is not.

--
This address is for information only. �I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Department of Statistics, Purdue University
hru...@stat.purdue.edu ��������Phone: (765)494-6054 ��FAX: (765)494-0558

JohnF

unread,
Apr 15, 2013, 8:50:26 AM4/15/13
to

Axel Vogt <&nor...@axelvogt.de> wrote:
> josephus wrote:
> > I have some data sets ENGLAND 1729-1970, NOAA 1880-2012 ?GISS ==
> > NOAA data. I even have a data set for ICE but that is not my
> > problem.
> > the problem is when I run a least squares algorithm over the data
> > set. should I change the extent, the slope increases (??).
> > the big question is WHAT KIND of PROCESSING caused the RAW DATA to
> > show irregular slopes like these
> > josephus
>
> I tried to understand you question - but I am a bit lost

Me, too, a bit lost. But there are lots of geostatistical
techniques to accommodate lots of less-than-obvious behaviors,
correlations, etc, e.g., google the term kriging and/or see
http://wikipedia.org/wiki/Kriging
You shouldn't be surprised seeing unexpected statistical
artifacts just using straightforward least squares.
You've probably got lots more literature work ahead of you
before deciding how best to reduce, analyze, etc your data.
--
John Forkosh �( mailto: �j...@f.com �where j=john and f=forkosh )

Herman Rubin

unread,
Apr 15, 2013, 8:53:13 AM4/15/13
to

On 2013-04-13, Axel Vogt <&nor...@axelvogt.de> wrote:

> On 13.04.2013 15:00, josephus wrote:
> >
> > I �have some data sets ��ENGLAND 1729-1970, �NOAA 1880-2012 �GISS ==
> > NOAA data. �I even have �a data set for �ICE �but that is not my
> > problem.
> >
> > the problem �is �when I �run a �least squares algorithm over the �data
> > set. �should I change �the �extent, �the slope �increases (??).
> >
> > so I �plotted the aggregate list of slopes �// mod X. �then I plot that
> > list. �and �this is were the problem exceeds my programming and
> > mathematics. �I can take slope data �at any scale ��(all slopes are
> > 2012 to N). ��I take all my slopes upward �If I reversed the process I
> > would expect to get negative slopes.
> >
> >
> > this is a plot of �the slopes over �the data NOAA ��and �excluding data
> > set downward slopes. ���this is an artifact of GW. ���the ranges of the
> > data are �increasing �together. �the result is my BEST FIT line �also
> > is
> > positive and ��increasing.
> >
> > the big question is �WHAT KIND of PROCESSING �caused �the RAW DATA to
> > show irregular slopes like these
> >
> > If you want I can send you the RAW data �and �the slope list.
> >
> > this is �a much more complex problem than just displaying odd data.
> > every data set I have processed agrees with the general terms of this
> > program ��they all plot upward �except for ENGLAND �it slopes downward
> >
> > I believe this data set was conformed by a process. ��but �the
> > artifacts
> > have strange properties �it may be a trap door function, �but I am
> > interesstd in ��understanding �the slope data that I have �produced.
> >
> > josephus
> >
> >
> > MY regular plot [scatter gram]
> > ...
>
> I tried to understand you question - but I am a bit lost (may be it is
> my English,
> and I do not understand you abbreviations).

Could you provide your data (by a link?) and clearly say what you do or
expect or
want? It is not clear what "least squares algorithm over the data set"
should be,
or "negative slopes".

Looking at the pattern of deviations, it is unclear how meaningful
is a straight line fit. �Possibly one could try to fit one with
temporally dependent deviations, which MIGHT be reasonable. �But
the semi-routine fitting of a straight line is not sound statistical
reasoning.

When the data form an ordered set, ignoring the dependence of the
deviations from the model is unsound. �Independence of the
residuals is an important part of the hypotheses needed to
justify the use of least squares. �Using a model derived from
thinking about the underlying causes is far better, and one
does not get the model from the data, but data only help to
choose between models.
Do you want some approximating 'curve' to your data? Or just a 'line'?
Or what?


josephus

unread,
Apr 15, 2013, 8:58:34 AM4/15/13
to
Herman Rubin wrote:
> The way you entered your data, I cannot get your plot into my
> reply. �However, the big error you made is assuming that the
> data should be explained by a linear process. �It obviously
> is not.
>


the �temperature records are a �LINEAR scatter of point one per year.

the data �is the record SLOPES of all the data processed by a program.
��the �question is �why would �the slope data �show �an increase until
��1922 or so �and then would be stable until 1976 �then it would
increase in �a nonlinear manner �to ��the current time. ���but in all
cases �the �temperatures �are �rising. ��if you cant �understand the
question �GO AWAY

my question was �what process could have �changed the temparature �data
�so that the BEST FIT slopes would �show this pattern


this is my DATA �and �the plot of that data ����X is the year ��Y is
the
data ���Y_Calc is a computed �LINEAR fit �of this data. �not real
useful.

I truncated the �data �to be smaller than �133 yesrs.


josephus

�input data FROM TRIAL

�44 items
�A= -12.506116926 �B= 0.006558967 slope of f(N)
���UT �16 �lt �28

INDEX year ���T-DATA ������������A+B*I ������������Ycalc-Y
���I ���X �������Y ��������������Y_Calc �����������residuals
���1 �2012 ��0.67564604 �����0.69052522 ���������-0.01487918
���2 �2010 ��0.67512411 �����0.67740728 ���������-0.00228317
���3 �2008 ��0.66179562 �����0.66428935 ���������-0.00249373
���4 �2006 ��0.64898528 �����0.65117141 ���������-0.00218613
���5 �2004 ��0.63101075 �����0.63805348 ���������-0.00704273
���6 �2002 ��0.60898221 �����0.62493554 ���������-0.01595333
���7 �2000 ��0.59069909 �����0.61181761 ���������-0.02111852
���8 �1998 ��0.57266059 �����0.59869968 ���������-0.02603909
���9 �1996 ��0.54883184 �����0.58558174 ���������-0.03674990
��10 �1994 ��0.52947211 �����0.57246381 ���������-0.04299170
��11 �1992 ��0.52751347 �����0.55934587 ���������-0.03183240
��12 �1990 ��0.50738856 �����0.54622794 ���������-0.03883938
��13 �1988 ��0.49413400 �����0.53311000 ���������-0.03897600
��14 �1986 ��0.47077725 �����0.51999207 ���������-0.04921482
��15 �1984 ��0.46964545 �����0.50687413 ���������-0.03722868
��16 �1982 ��0.44794184 �����0.49375620 ���������-0.04581436
��17 �1980 ��0.42604543 �����0.48063826 ���������-0.05459284
��18 �1978 ��0.41000618 �����0.46752033 ���������-0.05751415
��19 �1976 ��0.40278508 �����0.45440240 ���������-0.05161731
��20 �1974 ��0.41122620 �����0.44128446 ���������-0.03005826
��21 �1972 ��0.40710513 �����0.42816653 ���������-0.02106140
��22 �1970 ��0.42683548 �����0.41504859 ����������0.01178689
��23 �1968 ��0.42136534 �����0.40193066 ����������0.01943468
��24 �1966 ��0.43919953 �����0.38881272 ����������0.05038681
��25 �1964 ��0.46062146 �����0.37569479 ����������0.08492667
��26 �1962 ��0.45978338 �����0.36257685 ����������0.09720653
��27 �1960 ��0.44394761 �����0.34945892 ����������0.09448869
��28 �1958 ��0.43062317 �����0.33634098 ����������0.09428219
��29 �1956 ��0.40559441 �����0.32322305 ����������0.08237136
��30 �1954 ��0.42460882 �����0.31010512 ����������0.11450370
��31 �1952 ��0.41139084 �����0.29698718 ����������0.11440366
��32 �1950 ��0.41723675 �����0.28386925 ����������0.13336751
��33 �1948 ��0.44128608 �����0.27075131 ����������0.17053477
��34 �1946 ��0.45031527 �����0.25763338 ����������0.19268189
��35 �1944 ��0.42941434 �����0.24451544 ����������0.18489889
��36 �1942 ��0.36059908 �����0.23139751 ����������0.12920157
��37 �1940 ��0.27551560 �����0.21827957 ����������0.05723603
��38 �1938 ��0.20151958 �����0.20516164 ���������-0.00364206
��39 �1936 ��0.10792066 �����0.19204370 ���������-0.08412304
��40 �1934 ��0.06421356 �����0.17892577 ���������-0.11471221
��41 �1932 ��0.04378326 �����0.16580784 ���������-0.12202457
��42 �1930 ��-0.06633484 �����0.15268990 ���������-0.21902474
��43 �1928 ��-0.05653061 �����0.13957197 ���������-0.19610258
��44 �1926 ��-0.13714154 �����0.12645403 ���������-0.26359557
�Intercept is -12.506116926 Sigma A is 1.175600577
�slope ����is 0.006558967 Sigma B is ��������0
2011 0.006558967
MY regular plot [scatter gram]

Axel Vogt

unread,
Apr 15, 2013, 8:03:29 PM4/15/13
to
On 15.04.2013 14:50, JohnF wrote:

> Axel Vogt <&nor...@axelvogt.de> wrote:
> > josephus wrote:
> > > I have some data sets ENGLAND 1729-1970, NOAA 1880-2012 ?GISS ==
> > > NOAA data. I even have a data set for ICE but that is not my
> > > problem.
> > > the problem is when I run a least squares algorithm over the data
> > > set. should I change the extent, the slope increases (??).
> > > the big question is WHAT KIND of PROCESSING caused the RAW DATA to
> > > show irregular slopes like these
> > > josephus
> >
> > I tried to understand you question - but I am a bit lost
> >
> Me, too, a bit lost. But there are lots of geostatistical
> techniques to accommodate lots of less-than-obvious behaviors,
> correlations, etc, e.g., google the term kriging and/or see
> http://wikipedia.org/wiki/Kriging
> You shouldn't be surprised seeing unexpected statistical
> artifacts just using straightforward least squares.
> You've probably got lots more literature work ahead of you
> before deciding how best to reduce, analyze, etc your data.

No, I do not want to do that, but want to know what you do
(and why only for each 2nd year?).

Anyway, here are your data, plotted in a linked Excel sheet

Roughly one can say: data have 3 periods, are always a bit
oscillating. And can never be approximated by a 'line'.

http://www.axelvogt.de/temp/temp_data.xls

Herman Rubin

unread,
Apr 15, 2013, 8:05:24 PM4/15/13
to
My editor cannot manage your data in reasonable form, so I
must delete it. �One of the most standard tests for the appropriateness
of a linear model is to look at the time pattern of the residuals;
this is enough to show that a linear least squares procedure is not
an appropriate explanation of the data.

Statistics is not merely a collection of algorithms into which
data can be plugged. �Each algorithm has its group of assumptions,
without which its conclusions are not reasonable.

....................
0 new messages