"Cannellino" <fagi...@cannellino.it> wrote in message
news:nyupnd769lae.5...@40tude.net...
| Hi,
| I'm looking for a software that can perform linear fitting with weights
| both on X data and Y data.
Microsoft Excel.
Ignore the advice to use Microsoft EXCEL. Several studies have found
MS EXCEL to be inaccurate, unreliable, sometimes wrong; it has been
said that "friends do not let friends use EXCEL for statistical
analysis". Here are some relevant links:
http://www.daheiser.info/excel/frontpage.html
http://www-unix.oit.umass.edu/~evagold/excel.html
http://www.cs.uiowa.edu/~jcryer/JSMTalk2001.pdf
http://www.coventry.ac.uk/ec/~nhunt/pottel.pdf
http:/www.forecastingprinciples.com/paperpdf/McCullough.pdf
http://www.burns-stat.com/
In the latter link, click on 'tutorials' then on 'spreadsheet
addiction'.
R.G. Vickson
--
This message is brought to you by Androcles
http://www.androcles01.pwp.blueyonder.co.uk/
"Ray Vickson" <RGVi...@shaw.ca> wrote in message
news:315f0d1a-4b7c-47b8...@1g2000prg.googlegroups.com...
| On May 2, 2:23 pm, Cannellino <fagiol...@cannellino.it> wrote:
| > Hi,
| > I'm looking for a software that can perform linear fitting with weights
| > both on X data and Y data. I mean tha most general linear fitting, where
| > the x1, x2, x3, ... values have different errors, and the same for
| > y1,y2,y3,.. values.
| > Thanks in advance and sorry for my english
|
| Ignore the advice to use Microsoft EXCEL. Several studies have found
| MS EXCEL to be inaccurate, unreliable, sometimes wrong;
HAHAHA!
A least squares curve fit that is "inaccurate", "unreliable" and
"sometimes wrong" doesn't trouble anyone with any fuckin' sense
in the slightest!
This in my inaccurate, unreliable, and sometimes wrong blood pressure
with a line through it, you wanker.
http://www.androcles01.pwp.blueyonder.co.uk/BloodPressure.xls
I guess that professional statisticians have no fuckin' sense, then.
Well, it should trouble anyone who wants to use the results in a
SERIOUS study (not just for recreation or for solving a homework
problem). Anyway, it is not necessarily so that the line itself is
wrong, but more that the analysis and interpretation could be
erroneous. Actually, simple enough problems are dealt with well enough
by EXCEL, so fitting a straight line is probably safe. It is really in
the realm of more involved problems (such as the error-in-both-
variables model of the OP) that trouble can occur.
> This in my inaccurate, unreliable, and sometimes wrong blood pressure
> with a line through it, you wanker.
It sounds like you don't have your temper under control, and that may
lead to dangerous levels of high blood pressure. Relax and don't take
yourself so seriously. You will be happier for it (after breaking your
bad habit of bitterness and aggression) and will live a longer and
more fulfilling life. But maybe you don't' give a shit and just want
to be negative and obstructive.
R.G. Vickson
> http://www.androcles01.pwp.blueyonder.co.uk/BloodPressure.xls
"Ray Vickson" <C6...@shaw.ca> wrote in message
news:fa198acd-b114-4019...@q1g2000prf.googlegroups.com...
| On May 2, 4:53 pm, "Androcles" <Headmas...@Hogwarts.physics> wrote:
| > --
| > This message is brought to you by Androcles
| > http://www.androcles01.pwp.blueyonder.co.uk/
| >
| > "Ray Vickson" <RGVick...@shaw.ca> wrote in message
| >
| > news:315f0d1a-4b7c-47b8...@1g2000prg.googlegroups.com...
| > | On May 2, 2:23 pm, Cannellino <fagiol...@cannellino.it> wrote:
| > | > Hi,
| > | > I'm looking for a software that can perform linear fitting with
weights
| > | > both on X data and Y data. I mean tha most general linear fitting,
where
| > | > the x1, x2, x3, ... values have different errors, and the same for
| > | > y1,y2,y3,.. values.
| > | > Thanks in advance and sorry for my english
| > |
| > | Ignore the advice to use Microsoft EXCEL. Several studies have found
| > | MS EXCEL to be inaccurate, unreliable, sometimes wrong;
| >
| > HAHAHA!
| >
| > A least squares curve fit that is "inaccurate", "unreliable" and
| > "sometimes wrong" doesn't trouble anyone with any fuckin' sense
| > in the slightest!
|
| I guess that professional statisticians have no fuckin' sense, then.
AHAHAHA!
Very professional, is guessing.
Surely it is obvious that professional statisticians have no fuckin' sense,
then?
|
| Well, it should trouble anyone who wants to use the results in a
| SERIOUS study (not just for recreation or for solving a homework
| problem). Anyway, it is not necessarily so that the line itself is
| wrong, but more that the analysis and interpretation could be
| erroneous.
So we ought to guess as professional statisticians with no fuckin' sense
(and you) do, huh?
| Actually, simple enough problems are dealt with well enough
| by EXCEL,
So you concede the point and were merely prejudiced against
Microsoft, you guess. Are you a professional statistician who
likes to guess?
| so fitting a straight line is probably safe.
What's the exact probability of danger in fitting a straight line , then?
Try not to guess, I want an accurate, reliable, always right answer.
Unless you have no fuckin' sense, that is.
| It is really in
| the realm of more involved problems (such as the error-in-both-
| variables model of the OP) that trouble can occur.
Come on, then, tell us what the trouble is in my blood pressure
chart. I recorded time to the nearest 15 minutes and whatever
the meter readout was, that's error-in-both-variables.
|
| > This in my inaccurate, unreliable, and sometimes wrong blood pressure
| > with a line through it, you wanker.
|
| It sounds like you don't have your temper under control, and that may
| lead to dangerous levels of high blood pressure. Relax and don't take
| yourself so seriously.
HAHAHA!
Did you guess that because you are a wanker? I'll remind you that
your analysis and interpretation could be erroneous.
| You will be happier for it (after breaking your
| bad habit of bitterness and aggression) and will live a longer and
| more fulfilling life. But maybe you don't' give a shit and just want
| to be negative and obstructive.
Let's summarize.
1) The OP said "I'm looking for a software that can perform linear fitting"
2) I said "Microsoft Excel"
3) Some guessing wanker said "Ignore the advice to use Microsoft EXCEL",
"Actually, simple enough problems are dealt with well enough by EXCEL"
4) This makes ME negative and obstructive.
5) You are not just a wanker, you are a stooopid cunt, thicker than shit.
"hanson" <han...@quick.net> wrote in message
news:wTRSj.2050$qW.1457@trnddc06...
OP: "I'm looking for a software that can perform linear fitting ... "
A: "Microsoft Excel" (giving negative and obstructive advice).
V: "Ignore the advice to use Microsoft EXCEL", "simple enough problems are
dealt with well enough by EXCEL."
V: "I guess..."
V: "But maybe you don't' give a shit and just want to be negative and
obstructive."
Does that define what a wanker is or is Vickson just an exemplar of one?
"hanson" <han...@quick.net> wrote in message
news:xg_Sj.967$Fv.878@trnddc03...
I'll tell you what it is...
Excel doesn't give the results an Einstein Dingleberry wants, so Excel
is at fault.
See http://www.androcles01.pwp.blueyonder.co.uk/Doppler/V0.866.JPG
I'll dig out that program from my old computer some day...
or recreate it.
"hanson" <han...@quick.net> wrote in message
news:GG%Sj.513$Ve.427@trnddc08...
|
| "Androcles" <Headm...@Hogwarts.physics> wrote in message
| news:YX_Sj.6116$NZ7....@newsfe10.ams2...
| > This message is brought to you by Androcles
| > http://www.androcles01.pwp.blueyonder.co.uk/
| http://groups.google.com/group/sci.physics/msg/06a1da4e2fa1b80b
| wherein Androcles wrote:
| > I'll tell you what it is...
| > Excel doesn't give the results an Einstein Dingleberry wants,
| > so Excel is at fault. See [1] here:
| > http://www.androcles01.pwp.blueyonder.co.uk/Doppler/V0.866.JPG
| > I'll dig out that program from my old computer some day...
| > or recreate it.
| >
| hanson wrote:
| Yeah, I remember that. Great job!. Lead you to Algol, IIRC?
Nah, that was a 1987 DOS "c" version of Copernicus.exe,
a c++ program I wrote way back in 1993. It's still running today,
but I scrapped the DOS stuff when it became obvious that
Windows was going to be the industry leader.
http://www.androcles01.pwp.blueyonder.co.uk/Copernicus.exe
That was before the days of viruses when I was living under the
delusion that people were intelligent and would actually understand it.
Nowadays I suppose you'd call it a classic, it's 15 years old.
There is serendipity here, I scrapped the first DOS version when I
"realised" (after a few pints of strong ale) that it wasn't ever going to
reproduce the light curve of Algol even though it did reproduce
some cepheids, and I was pretty busy back then anyway as a project
leader on an industrial vision system working with Motorola processors
and a Gould minicomputer that had its own air-conditioned room.
I only recreated a Windows version because I needed to familiarise
myself with Intel processors and c++, and then purely by chance I
saw something that approximated Algol and that led me to research
and investigate it further, or I'd still be an ED.
| But spare yourself future efforts about [1] because REL-gious
| conversions from ED'ism to rationality in real world existence
| only happens in the school of Hard Knox. We went thru that.
Yeah... I've got a few dozen blood-stained T-shirts that read
"Property of H.M. Government, School of Hard Knox Dept."
Her Majesty still waves and smiles to me, but oy wey, no
knighthood has been forthcoming. Go figure... I did get a
blue tie with a gold emblem on it though (now blood-stained).
http://www.queensawards.org.uk/business/Award_Categories/International_Trade.html| We posted about it and told how we, in grad school, both| threatened Einstein doubters with beating the shit out of them...| ... and then came our gradual exodus out of the darkness of| Einstein's cul-de sac into the light of the rising sun...ahaha...| eventually realizing what is really going on...Once I knew how smart I was. Now I know how stupid I am.| http://groups.google.com/group/sci.physics.relativity/msg/b09ab8bdc8f2ae70| .... hence Einstein Dingleberries are here to linger on in their| fantasy world of contstructs of cons and out-right lies as can be| seen in the heuristic use of REL in the real world where:| == mil/indust. Eng, R&D....................."does not need REL shit"| == *.edu and grantology ..................."does need REL, No shit"| == Promo, Sales & Movies..............."loves REL by the shitload"| == Jews protect it as cultural heritage whether "REL is shit or not".| >| Ergo: REL is a con, a story, a fable, a tale, called a theory.| Washing dishes or cars for an hour will earn you enough| money to buy you a hamburger... But no amount of promoting| & proselytizing REL will ever buy you anything in the real world| unless you belong in the upper strata of the hierarchies in| the appropriate places as summarized in the 4-liner above...| Remember, here in these NG's it's mostly (a) the geriatric| "cultural" ilk that is so pathetically and fanatically trying to sell| REL & (b) many times it's their goyim victims, known as| Einstein Dingleberries, whom they were able to brainwash| and do their bidding.... Thanks for the laughs dude...ahahahaha...| ahahaha... ....ahahanson|Yep...
Your problem is far from simple, and it has an extensive history of
several different attempts to deal with it. The problem is that
standard regression packages (i.e., standard least-squares fits) do
NOT account properly for the fact that you have errors in both x and
y. Standard methods are OK when you have errors in y alone, but may
give a significantly incorrect line fit when there are errors in x as
well as in y. A quite accessible account of the problem is given in
http://en.wikipedia.org/wiki/Total_least_squares . It presents the
problem as one of minimizing a quadratic function subject to m linear
constraints (where m = number of observations). The standard EXCEL
Analysis Toolpack cannot deal with this problem, but you COULD try to
use the EXCEL Solver tool to deal with the problem, as that tool is
able to reasonably well deal with constrained linear and nonlinear
optimization. That package is quite reliable, having been written on a
subcontracted basis by some of the leading optimization experts. It
uses sophisticated techniques such as the generalized reduced-gradient
method, BFGS or conjugate-gradient direction-setting methods---your
choice, by setting some parameters---as well as sophisticated line-
search algorithms. Of course, it does suffer from the natural
limitations of a package that estimates gradients by finite-
differences, although there, too, the user has the choice of forward-
difference or central-difference formulas. It can handle problems with
a few hundred variables and a couple of hundred constraints. I have
used it numerous times in the past with great success, and if I were
to recommend a way to solve your problem in EXCEL it would be to apply
the Solver Tool to the nonlinear programming formulation, but to stay
far away from the standard regression package. My objections are not
some anti-Microsoft bias, but are rather an anti-using-the-wrong-tool
bias.
Alternatively, you cold apply an iterative method as explained in
Reference 1 the link provided.
A nice summary of the problem and the issues that crop up in yhour
type of problem is given in the report "An Historical Review of Linear
Regression with Errors in both Variables", by J.W Gillard (1996),
obtainable for free at http://jppsg.ac.uk/maths/resources/Gillard_Tech_Report.pdf
.
R.G. Vickson
"hanson" <han...@quick.net> wrote in message
news:v%1Tj.1315$sp.1156@trnddc02...
That's an OE bug.
When a string of characters as in http://www.queensaw...
above exceeds width limit the CR/LF vanishes. Usually
I use tinyurl for long strings but I missed it this time.
Sorry about that.
Most don't have a web page either. Schwartzschit and Dork Van de
merde are exceptions rather than the rule, but I leave you to judge the
quality.
http://users.telenet.be/vdmoortel/dirk/Physics/TwinsEvents.html
He actually draws in text on his own web page, though what it is
supposed to be only Dork knows.
As for carrying out a calculation, ya gotta love this line:
"So if T = 5 years and v = 0.8c, then the stay at home twin will
have aged 10 years" -- and he's the fumble mumbler.
Another Dingleberry is ASS-istant professor Andersen (Tusseladd,
or Norwegian troll) who really DID say Excel was in error on a
minus sign.
| All they do is to parrot
| what someone else has said, most notoriously rect-Al Schwartz
| who believes that if he scours the web and does cut and paste
| jobs and screams loud enough he will become smart... ahaha...
Not hard to see through, is it? Totally transparent.
| Astonishingly some goyim Einstein Dingleberries do believe in
| rect-Al's world which he daily visits and urges his follower to do
| the same: <http://www.mazepath.com/uncleal/sunshine.jpg>
| ahahaha.. ahahaha... ahahahanon
Yes, but it's no different to gazillions of Pope, Prophet, Messiah or
Buddha followers who believe and repeat what they are told to
believe, and mostly in fear of what their neighbours will think of
them or do to them if they fail to conform to orthodoxy.
Kings, Presidents and Priests know it, of course, and manipulate
themselves into power by hiring thugs to do their bidding;
whether the name is Mugabe or Elizabeth or Bush or Hitler or
Stalin or Hussein, Einstein is essentially no different, the thug
mentality of sheep following the leader doesn't change. Let's
face it, Schwartz is a bullying thug.
This problem has been studied for a long time. (Some early examples:
R. J. Adcock, "Note on the method of least squares," The Analyst (Des
Moines), vol. 4, 183-184 (1877)
R. J. Adcock, "A problem in least squares," The Analyst (Des Moines),
vol. 5, 53-54 (1878)
K. Pearson, "On lines and planes of closest fit to systems of points in
space," Phil. Mag. vol. 2, 559-572 (1901) )
I don't know whether you can easily find software that treats this case
(strange, since the problem has been well-known for a long time, and can
easily be handled with a computer). I hope this summary will be of some
help.
The equations are inherently nonlinear even for straight-line fitting,
so implementation of the theory was difficult in the pre-computer era.
One of the best references from the period just before the development
of digital computers is a book (well worth reading):
W. E. Deming, "Statistical Adjustment of Data," Wiley, 1943.
There was a flurry of interest within the last few decades. Here is a
sampling:
D. York, Can. J. Phys vol.44 1079 (1966)--Just the straight line, but a
good analysis.
M. Lybanon, "A better least-squares method when both variables have
uncertainties," Am. J. Phys. vol 52, 22-26 (1984)--This method works for
straight lines or more complicated functions.
M. Lybanon, "A simple generalized least-squares algorithm," Computers &
Geosciences vol. 11, no. 4, 501-508 (1985)--A program implementing the
algorithm of the preceding paper.
S. D. Christian, E. E. Tucker, and E. Enwall, "east squares analysis: A
primer," American Laboratory, vol. 8, no. 6, 41-49 (1986)--Good
formulation of the straight line problem (also some material on more
general functions).
P. L. Jolivette, "Least-squares fits when there are errors in X,"
Computers in Physics vol. 7 no. 2, 208-212 (1993)--Jolivette offered the
program via e-mail, but the same e-mail address
(joli...@physics.hope.edu) may no longer be a valid address.
J. R. Macdonald and W. J. Thompson, "Least-squares fitting when both
variables contain errors: Pitfalls and possibilities," Am. J. Phys.
vol. 60, 66-72 (1992)--This article has a section on straight-line
fitting, and includes a survey of available (at the time) software.
W. H. Jefferys, M. J. Fitzpatrick, and B. E. McArthur, "GaussFit--a
system for least squares and robust estimation," Celestial Mech. vo..
41, 39-49 (1988)--The software,, developed for the Hubble Space
Telescope program, used to be available from Jefferys at the University
of Texas Department of Astronomy. Jefferys published some earlier
papers, but this article is a compilation, puls some new material.
B. P. Miller and He. E. Dunn, "Orthogonal least-squares line fit with
variable scaling," Computers in Physics July/Aug. 1988 (not sure of the
volume number)--An interesting geometrical interpretation, applied
specifically to the straight line. The authors offered software.
M. Lybanon and K. C. Messa, Jr., "Genetic Algorithm Model Fitting,"
Chapter 8 (pp. 269-345) in L. D. Chambers (ed.), Practical Handbook of
Genetic Algorithms, Complex Coding Systems, Volume III, CRC Press, ISBN
0-8493-2539-0 (1999)--The chapter discusses application of the method to
the case of arbitrary uncertainties in both variables, and includes a
program listing. The software was (and may still be) available from CRC
Press.