Farewell to Conjoint Measurement
Trendler, Guenter
Nowadays it is usually believed that the road to measurement is composed of two distinct, consecutive steps. The first step comprises the scientific task and it is followed by the instrumental task. We must find out if an attribute is measurable before we can measure it. It is also believed that the first task involves testing only qualitative laws (or axioms of measurement) like in conjoint measurement.
However, what is usually ignored is that under these circumstances solving the scientific task does not lead automatically to measurement. In order to measure we need a standard sequence. Measurement necessarily requires the construction of standard sequences. Conversely, if the construction is impossible measurement is also impossible no matter how many times the axioms are successfully verified. But, more importantly, if we (can) construct a standard sequence we can test measurability the traditional way by searching for constants in a law; a method which is well known as "derived measurement". This procedure also has the great advantage that it allows us to discover measurability and attain measurement at the same time, in one step.
Hence, if we only test axioms we will never attain measurement. In the end we must construct a standard sequence anyway. It is therefore best if we proceed with the construction directly. But if we do so there is no need to test qualitative axioms anymore. We can use the much simpler method of derived measurement to verify measurability and simultaneously attain measurement. That's why conjoint measurement is superfluous. The instrumental task of standard sequence construction eliminates the necessity of the scientific task. Conjoint measurement never was a real alternative to derived measurement. At a closer look all the alleged advantages of conjoint measurement disintegrate in thin air and no more remains of it as from the last night's dream.
G.
Andrew Kyngdon
Hello Guenter,
Systems of derived measurements, as far as I can see, require base quantities. A derived quantity, either through definition or law, is the product of at least two other quantities. Force and density are prime examples here.
The problem for derived measurement in psychology is this: where are the base psychological quantities? Where are the psychological analogues to length, time, plane angle and mass? As far as I can tell, no compelling case has been made that such things exist (which is not to say that they do not). In the absence of base psychological quantities, derived measurement is of no use at all. It may be of use in some cases, such as psychophysics, where physical quantities are involved. But not in the case of cognitive abilities, attitudes and personality traits, for example, which are assessed by psychometric tests which produce nothing more than partial orders of test score response patterns.
I disagree with your statement that the theory of conjoint measurement is superfluous. I tend more to agree with Steve Humphry’s argument in his Measurement rejoinder paper that the case for the theory needs to be strengthened. I believe it can be, especially since Steve alerted me to Kyburg’s (1984) work relating the quantity calculus and conjoint measurement. I have been putting some thought lately to the role of units in conjoint measurement and I believe the theory can be reformulated so as to make the role of the unit clear. Typical formal presentations of conjoint measurement demand that only levels of attributes be identified. In my mind this is far too weak. I consider that one level of an attribute must be specified as a “tentative unit” and that all other levels of the same attribute be hypothesised multiples of this level. This would make the relation between conjoint measurement, derived measurement and concatenation much clearer than what it is at present. I as yet have not put my thoughts down on paper as I am very busy with other things, however, I do not see it as an insurmountable task.
In my opinion, the theory of conjoint measurement has followed a rather unfortunate and at times hostile historical trajectory. For some reason, its discoverers did not strongly made the case for it. Luce & Narens (1993) stated that they assumed that the theory would become standard textbook material in quantitative psychology shortly after it was proposed. This did not happen. Being representationalists, its discoverers couched the theory in highly technical mathematics and set theory, which has rendered it largely inaccessible to most behavioural scientists. Moreover, the connection between the classical/ standard definition of measurement and conjoint measurement was not elaborated as representationalists do not believe that definition to be true. In the minds of those who do believe that measurement is the product of a real number and a unit, the representationalism in which conjoint measurement has been couched has called into question the worth of the theory. I believe that it was only by historical accident that conjoint measurement was discovered by representationalists.
The psychometric world has been hostile to conjoint measurement. Ever since Ramsay’s review of the first volume of Foundations of Measurement, psychometricians have dismissed the theory because it is not stochastic. Moreover, given the ordinal constraints imposed upon data by the cancellation axioms, the statistics that behavioural scientists are familiar with are simply not suitable for the probabilistic testing of these axioms. However, rather than develop the suitable probabilistic frameworks, and then embark on a rigorous empirical program, psychometricians seemed to be content with letting the theory of conjoint measurement flounder in favour of psychometric models. Indeed, the non-stochastic nature of conjoint measurement was used at times to support psychometric models (e.g., van der Linden, 1994). Hence, ordered restricted inference frameworks were worked out only relatively recently (e.g., Karabatsos, 2001; Davis-Stober, 2009), many decades after conjoint measurement was first proposed. Thus the effect of these developments was limited. But even these developments are still not good enough for the psychometricians. I’ve heard a senior Rasch figure dismiss these as “statistical afterthoughts” (whatever the hell that means), a view which has been mindlessly parroted by some on Rasch listserves.
Another problem is that applications to date have mostly been by people who do not sufficiently understand the theory. Most often they construct empirical tests of the cancellation axioms which are highly biased towards a positive result. Conjoint measurement has been used, as noted by Steve, in a largely post hoc sense. In most empirical applications psychologists have used it much like they have used null hypothesis significance testing – as something that one does after one has designed the study and collected data. However, if the role of the unit in conjoint measurement is considered, what can be interpreted from such post hoc applications is limited. I believe the theory of conjoint measurement must be used to inform the design of experiments, rather than as post hoc data analysis.
I believe that a compelling case can be made that theory of conjoint measurement has not been properly understood by behavioural scientists, and that its empirical worth in the quantification of psychological attributes has not been properly investigated. This is due to the failure of those who discovered the theory to make the case for it, the hostility of the psychometricians (who should have known better) and historical factors. Conjoint measurement has had a better reception in decision making under risk and uncertainty, where it has been mainly used in formal arguments (e.g., Kahneman & Tversky, 1979; Wakker & Tversky, 1993).
Hence I do not believe that a compelling case can be made for the rejection of conjoint measurement as a means of quantification. The complete failure of other ways of producing scientific measurements of psychological attributes underscores this.
Cheers,
Andrew
Andrew Kyngdon, PhD
MetaMetrics, Inc.
My website: https://sites.google.com/site/drandrewkyngdon/home
Measurement Forum: http://groups.google.com/group/talking-measurement
--
You received this message because you are subscribed to the Google Groups "Talking Measurement" group.
To post to this group, send email to talking-m...@googlegroups.com.
To unsubscribe from this group, send email to talking-measure...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/talking-measurement?hl=en.
Trendler, Guenter
A. I consider that one level of an attribute must be specified as a “tentative unit” and that all other levels of the same attribute be hypothesised multiples of this level.
G. Exactly! The practical realization of this idea leads to the construction of a standard sequence. Once such a standard sequence can be constructed derived measurement is applicable. Correct? Hence why should we make use of conjoint measurement anymore since derived measurement is much simpler in application? (That psychological attributes do not sustain the construction of standard sequences is another topic.)
Where exactly do we disagree?
GT
-----Ursprüngliche Nachricht-----
Von: talking-m...@googlegroups.com im Auftrag von Andrew Kyngdon
Gesendet: Mo 04.04.2011 03:34
An: talking-m...@googlegroups.com
Betreff: [talking-measurement] RE: Farewell to Conjoint Measurement
Hello Guenter,
Systems of derived measurements, as far as I can see, require base quantities. A derived quantity, either through definition or law, is the product of at least two other quantities. Force and density are prime examples here.
The problem for derived measurement in psychology is this: where are the base psychological quantities? Where are the psychological analogues to length, time, plane angle and mass? As far as I can tell, no compelling case has been made that such things exist (which is not to say that they do not). In the absence of base psychological quantities, derived measurement is of no use at all. It may be of use in some cases, such as psychophysics, where physical quantities are involved. But not in the case of cognitive abilities, attitudes and personality traits, for example, which are assessed by psychometric tests which produce nothing more than partial orders of test score response patterns.
I disagree with your statement that the theory of conjoint measurement is superfluous. I tend more to agree with Steve Humphry's argument in his Measurement rejoinder paper that the case for the theory needs to be strengthened. I believe it can be, especially since Steve alerted me to Kyburg's (1984) work relating the quantity calculus and conjoint measurement. I have been putting some thought lately to the role of units in conjoint measurement and I believe the theory can be reformulated so as to make the role of the unit clear. Typical formal presentations of conjoint measurement demand that only levels of attributes be identified. In my mind this is far too weak. I consider that one level of an attribute must be specified as a "tentative unit" and that all other levels of the same attribute be hypothesised multiples of this level. This would make the relation between conjoint measurement, derived measurement and concatenation much clearer than what it is at present. I as yet have not put my thoughts down on paper as I am very busy with other things, however, I do not see it as an insurmountable task.
In my opinion, the theory of conjoint measurement has followed a rather unfortunate and at times hostile historical trajectory. For some reason, its discoverers did not strongly made the case for it. Luce & Narens (1993) stated that they assumed that the theory would become standard textbook material in quantitative psychology shortly after it was proposed. This did not happen. Being representationalists, its discoverers couched the theory in highly technical mathematics and set theory, which has rendered it largely inaccessible to most behavioural scientists. Moreover, the connection between the classical/ standard definition of measurement and conjoint measurement was not elaborated as representationalists do not believe that definition to be true. In the minds of those who do believe that measurement is the product of a real number and a unit, the representationalism in which conjoint measurement has been couched has called into question the worth of the theory. I believe that it was only by historical accident that conjoint measurement was discovered by representationalists.
The psychometric world has been hostile to conjoint measurement. Ever since Ramsay's review of the first volume of Foundations of Measurement, psychometricians have dismissed the theory because it is not stochastic. Moreover, given the ordinal constraints imposed upon data by the cancellation axioms, the statistics that behavioural scientists are familiar with are simply not suitable for the probabilistic testing of these axioms. However, rather than develop the suitable probabilistic frameworks, and then embark on a rigorous empirical program, psychometricians seemed to be content with letting the theory of conjoint measurement flounder in favour of psychometric models. Indeed, the non-stochastic nature of conjoint measurement was used at times to support psychometric models (e.g., van der Linden, 1994). Hence, ordered restricted inference frameworks were worked out only relatively recently (e.g., Karabatsos, 2001; Davis-Stober, 2009), many decades after conjoint measurement was first proposed. Thus the effect of these developments was limited. But even these developments are still not good enough for the psychometricians. I've heard a senior Rasch figure dismiss these as "statistical afterthoughts" (whatever the hell that means), a view which has been mindlessly parroted by some on Rasch listserves.
Another problem is that applications to date have mostly been by people who do not sufficiently understand the theory. Most often they construct empirical tests of the cancellation axioms which are highly biased towards a positive result. Conjoint measurement has been used, as noted by Steve, in a largely post hoc sense. In most empirical applications psychologists have used it much like they have used null hypothesis significance testing - as something that one does after one has designed the study and collected data. However, if the role of the unit in conjoint measurement is considered, what can be interpreted from such post hoc applications is limited. I believe the theory of conjoint measurement must be used to inform the design of experiments, rather than as post hoc data analysis.
Andrew Kyngdon
G. Where exactly do we disagree?
A. In derived measurement, it is unambiguous that the units concerned are well defined magnitudes of extant continuous quantities. My "tentative units", by contrast, are mere hypotheses as it is not known that the relevant attributes are continuous quantities. It may turn out that they are not real units at all.
Andrew
Trendler, Guenter
Unfortunately I was not able to answer in more detail sooner. I'll give it now a try.
A. The problem for derived measurement in psychology is this: where are the base psychological quantities? Where are the psychological analogues to length, time, plane angle and mass? As far as I can tell, no compelling case has been made that such things exist (which is not to say that they do not). In the absence of base psychological quantities, derived measurement is of no use at all. It may be of use in some cases, such as psychophysics, where physical quantities are involved. But not in the case of cognitive abilities, attitudes and personality traits, for example, which are assessed by psychometric tests which produce nothing more than partial orders of test score response patterns.
G. Derived measurement may indeed be a problem in psychology, but how does this affect the question of the usefulness of conjoint measurement? I see no connection. From the fact that derived measurement doesn't work it does not follow that conjoint measurement is necessarily a useful alternative (see also bellow).
A. I disagree with your statement that the theory of conjoint measurement is superfluous. I tend more to agree with Steve Humphry’s argument in his Measurement rejoinder paper that the case for the theory needs to be strengthened. I believe it can be, especially since Steve alerted me to Kyburg’s (1984) work relating the quantity calculus and conjoint measurement. I have been putting some thought lately to the role of units in conjoint measurement and I believe the theory can be reformulated so as to make the role of the unit clear. Typical formal presentations of conjoint measurement demand that only levels of attributes be identified. In my mind this is far too weak. I consider that one level of an attribute must be specified as a “tentative unit” and that all other levels of the same attribute be hypothesised multiples of this level. This would make the relation between conjoint measurement, derived measurement and concatenation much clearer than what it is at present. I as yet have not put my thoughts down on paper as I am very busy with other things, however, I do not see it as an insurmountable task.
G. In my view the role of the unit in conjoint measurement is already clear. It is there at the core of conjoint measurement. Without the concept of unit the axioms of conjoint measurement would not make any sense. The axioms are chosen is such a way as to logically legitimize the construction of standard sequences (i.e. combinations of units). There is no essential difference between conjoint and derived measurement. That this is not generally recognized is not necessarily only the fault of representationalism.
A. In my opinion, the theory of conjoint measurement has followed a rather unfortunate and at times hostile historical trajectory. For some reason, its discoverers did not strongly made the case for it. Luce & Narens (1993) stated that they assumed that the theory would become standard textbook material in quantitative psychology shortly after it was proposed. This did not happen. Being representationalists, its discoverers couched the theory in highly technical mathematics and set theory, which has rendered it largely inaccessible to most behavioural scientists. Moreover, the connection between the classical/ standard definition of measurement and conjoint measurement was not elaborated as representationalists do not believe that definition to be true. In the minds of those who do believe that measurement is the product of a real number and a unit, the representationalism in which conjoint measurement has been couched has called into question the worth of the theory. I believe that it was only by historical accident that conjoint measurement was discovered by representationalists.
G. In my view there is no essential difference between the classical and representational theory of measurement; that is, in both cases measurement is understood as the product of a real number and a unit. The difference is only a matter of linguistic expression. Also, I don't believe that conjoint measurement was discovered by representationalists by chance. On the contrary, since it is a typical abstract mathematical construction with no practical value it could only be discovered the abstract, mathematical way.
A. Hence I do not believe that a compelling case can be made for the rejection of conjoint measurement as a means of quantification. The complete failure of other ways of producing scientific measurements of psychological attributes underscores this.
G. Your concluding "hence" is obviously only rhetorical (or an expression of creed) since you did not address any of my arguments. How should we therefore know if a compelling case for the rejection of conjoint measurement can be made or not? The failure of other ways of producing scientific measurement is not really of any argumentative relevance. Taken seriously it is just a error of logic.
Best,
GT
-----Ursprüngliche Nachricht-----
Von: talking-m...@googlegroups.com im Auftrag von Andrew Kyngdon
Gesendet: Do 07.04.2011 01:01
An: talking-m...@googlegroups.com
Betreff: RE: [talking-measurement] RE: Farewell to Conjoint Measurement
Paul Barrett
In a special issue of Theory and Psychology,
http://tap.sagepub.com/content/current
Is there really a boundary between quantitative and qualitative research
approaches? Examples of quantitative research in an interpretive vein
two "commentary" papers caught my eye:
Maracek, J. (2011) Numbers and interpretations: What is at stake in our ways
of knowing? Theory and Psychology, 21, 2, 220-240.
Abstract
This article reflects on a set of target articles concerned with the use of
quantitative procedures in interpretive research. The authors of those
articles (Osatuke & Stiles; Westerman; and Yanchar) discuss ways that
numerical procedures can be brought into interpretive studies, using
illustrations from research programs on psychotherapy process, schools, law
courts, and work life. Instead of the usual quantitative-qualitative
distinction, I use Geertz's distinction between experimental science and
interpretive science and Kidder and Fine's distinction between Big-Q and
small-q research to reflect on several procedural and epistemological
differences among target papers. The diversity of approaches under the
umbrella of qualitative methods is described, along with some recent
developments. Even though US psychology continues to mount stiff resistance
against incorporating interpretive approaches into its knowledge-producing
practices, such approaches are flowering in other parts of the world.
And
Michell, J. (2011) Qualitative research meets the ghost of Pythagoras.
Theory and Psychology, 21, 2, 241-259.
Abstract
The issue of qualitative versus quantitative methods is rooted first and
foremost in the character of the phenomena investigated and not in an
investigator's methodological preferences. If the phenomenon under
investigation is non-quantitative, then it cannot be studied successfully by
attempting to use quantitative methods because trying to impose quantitative
concepts upon qualitative phenomena misrepresents them. If the target
articles provide any guide, these truths are ignored as much by
psychologists wanting to mix quantitative with qualitative methods as by
mainstream quantitative researchers. These articles display both the power
of the modernist fantasy that measurement is always a discretionary choice
of any investigator and the power of the persistent delusion that
psychological attributes must be measurable. In psychology, as ever, the
ghost of Pythagoras rules.
The article by Michell is perhaps his most powerful critique of
"quantitative psychology" to date.
Regards .. Paul
Andrew Kyngdon
G. Derived measurement may indeed be a problem in psychology, but how does this affect the question of the usefulness of conjoint measurement? I see no connection. From the fact that derived measurement doesn't work it does not follow that conjoint measurement is necessarily a useful alternative (see also bellow).
As I said, one does not have to have extant base quantities measured in well defined units to use the theory of conjoint measurement. Base psychological quantities are, however, necessary for the use of derived measurement in psychology, unless one has physical quantities at one's disposal, such as in psychophysics.
In my view, when using conjoint measurement, one has to explicitly identify a level as a "tentative unit" prior to empirical application, but one does not have to know that the "tentative unit" is a genuine unit. Therein lies the usefulness of conjoint measurement - we do not have to have prior knowledge that the attributes concerned are continuous quantities. All we must do is hypothesise that they are quantitative; and the defining of a "tentative unit" is consistent with such a hypothesis.
G. In my view the role of the unit in conjoint measurement is already clear. It is there at the core of conjoint measurement. Without the concept of unit the axioms of conjoint measurement would not make any sense. The axioms are chosen is such a way as to logically legitimize the construction of standard sequences (i.e. combinations of units). There is no essential difference between conjoint and derived measurement. That this is not generally recognized is not necessarily only the fault of representationalism.
I disagree that units are at the core of current axiomatic presentations of conjoint measurement. Typically in such presentations, units only get a mention in passing in the "uniqueness theorem" section following the axioms and statement of the representation theorem. That these theorems empirically hold for a given natural system logically implies that the axioms have already been tested. In my view one has to design the whole experiment prior to the collection of data with the concept of a "tentative unit" in mind.
One can apply conjoint measurement to the levels of attributes where there is no explicit unit defined, and find that no such unit is obtained after application. Have a read of some of my work applying conjoint measurement to attitudes via Coombs (1964) theory of unidimensional unfolding. There you will find instances where the cancellation axioms hold but only unequal scale differences between attitude levels are obtained (the so - called "ordered metric" scale). See Kyngdon (2006) and Kyngdon & Richards (2007). Obtaining standard sequences from the post hoc application of conjoint measurement is by no means a certainty, as you will see.
G. In my view there is no essential difference between the classical and representational theory of measurement; that is, in both cases measurement is understood as the product of a real number and a unit. The difference is only a matter of linguistic expression. Also, I don't believe that conjoint measurement was discovered by representationalists by chance. On the contrary, since it is a typical abstract mathematical construction with no practical value it could only be discovered the abstract, mathematical way.
Here you are very much mistaken. There are important and non-trivial differences between the classical and representational definitions of measurement. Measurement in the classical/standard sense must only involve the estimation of ratios between magnitudes and unit magnitudes of continuous quantities. This does not involve the act of assigning anything nor the representation of anything.
Restricting representational measurement to ratio scales is arbitrary and an argument which contradicts the internal logic of representationalism. Representationalism basically states that measurement consists of the assignment of numbers to objects or events such that the behaviour of the relevant "qualitative" empirical relational system is reflected in a "numerical relational system". In this way merely ordinal and categorical phenomena can be "measured". In the classical theory, only continuous quantities can be measured. In the mind of the realist, the only work of the representationalists that is relevant to measurement is the work concerning "interval" and "ratio" scales.
I did not argue that the representationalists who discovered conjoint measurement did so by chance, merely that it was an historical accident that it was discovered by such thinkers. Luce & Tukey (1964) had an explicit, non-stochastic motivation to explore the possibility of quantification when no natural concatenation operation was present. The theory may have been discovered at some point previously by someone whom would not necessarily subscribe to representationalism or even had knowledge of it. But seeing that we cannot replay history we'll never know.
G. Your concluding "hence" is obviously only rhetorical (or an expression of creed) since you did not address any of my arguments. How should we therefore know if a compelling case for the rejection of conjoint measurement can be made or not? The failure of other ways of producing scientific measurement is not really of any argumentative relevance. Taken seriously it is just a error of logic.
You stated in your initial post that conjoint measurement is "superfluous" and that derived measurement be used instead. I argued that conjoint measurement is not superfluous because we do not currently have the system of extant, base psychological quantities that derived measurement requires. If this is not directly addressing your arguments then I do not know what is.
I concluded that a compelling case for the rejection of conjoint measurement cannot be made. I mentioned the complete failure of other methods to achieve psychological measurement as a means of emphasising my conclusion, not as a means of providing logical justification for it.
Andrew
Andrew Kyngdon, PhD
MetaMetrics, Inc.
www.lexile.com
My website: https://sites.google.com/site/drandrewkyngdon/home
Measurement Forum: http://groups.google.com/group/talking-measurement
Stephen Humphry
Interesting exchange. I'm interested in Andrew's response, especially in that it concurs with my understanding of additive conjoint measurement and how units of measurement fit within it. Guenter, you say:
G. In my view there is no essential difference between the classical and representational theory of measurement; that is, in both cases measurement is understood as the product of a real number and a unit. The difference is only a matter of linguistic expression. Also, I don't believe that conjoint measurement was discovered by representationalists by chance. On the contrary, since it is a typical abstract mathematical construction with no practical value it could only be discovered the abstract, mathematical way.
S. I wonder whether you could briefly explain why you think measurement is understood as a product of a real number and a unit in the representational theory. I start to wonder, which representational theory. I understand "the" representational theory as the idea that we 'use numbers' to represent qualitative relations. I suppose the key question is: how does one define a unit, such as the kg, in purely representational terms? Given there must be reference to operations in both the standard/classical and representational conceptions of measurement, there may be common ground; however, I find it difficult to see how a unit such as the kg or volt can be interpreted in the same way in the representational and standard/classical conceptions.
I fully agree also that the difficulty in psychology is establishing base quantities of some kind in some fashion.
Steve
Stephen Humphry | Associate Professor
Graduate School of Education
The University of Western Australia
M428, 35 Stirling Highway, Crawley, WA, 6009
Telephone: +61 8 6488 7008
Fax: +61 8 6488 1052
www.gse.uwa.edu.au
CRICOS Code: 00126G
Trendler, Guenter
G. In my view there is no essential difference between the classical and representational theory of measurement; that is, in both cases measurement is understood as the product of a real number and a unit. The difference is only a matter of linguistic expression.
S. I wonder whether you could briefly explain why you think measurement is understood as a product of a real number and a unit in the representational theory. I start to wonder, which representational theory. I understand "the" representational theory as the idea that we 'use numbers' to represent qualitative relations. I suppose the key question is: how does one define a unit, such as the kg, in purely representational terms? Given there must be reference to operations in both the standard/classical and representational conceptions of measurement, there may be common ground; however, I find it difficult to see how a unit such as the kg or volt can be interpreted in the same way in the representational and standard/classical conceptions.
G. How is a unit interpreted in the classical conception?
A. Here you are very much mistaken. There are important and non-trivial differences between the classical and representational definitions of measurement. Measurement in the classical/standard sense must only involve the estimation of ratios between magnitudes and unit magnitudes of continuous quantities. This does not involve the act of assigning anything nor the representation of anything. Restricting representational measurement to ratio scales is arbitrary and an argument which contradicts the internal logic of representationalism. Representationalism basically states that measurement consists of the assignment of numbers to objects or events such that the behaviour of the relevant "qualitative" empirical relational system is reflected in a "numerical relational system". In this way merely ordinal and categorical phenomena can be "measured". In the classical theory, only continuous quantities can be measured. In the mind of the realist, the only work of the representationalists that is relevant to measurement is the work concerning "interval" and "ratio" scales.
G. Let us forget for a moment ordinal measurement… isn’t it clear that in the case of unit-based measurement (e.g. interval and ratio scale measurement) the assignment of numbers to (properties) of objects is accomplished by means of a standard sequence? For example, in the case of length measurement the empirical relational system is a set of objects (e.g. fish), the standard sequence is a ruler (1) and the numerical relational system is the system of positive real numbers. Note that the procedure of length measurement is the same as in classical theory. That is, in practice the proceeding is identical (2). It is only described differently. This way numbers are associated with objects by means of the standard sequence and in this sense measurement is a product of a real number and a unit in representational theory.
GT
(1) http://visual.merriam-webster.com/science/measuring-devices/measure-length.php
(2) http://www.tpwd.state.tx.us/publications/annual/fish/tag_measure_release/
-----Ursprüngliche Nachricht-----
Von: talking-m...@googlegroups.com im Auftrag von Stephen Humphry
Gesendet: Fr 15.04.2011 08:16
Trendler, Guenter
G. In my view there is no essential difference between the classical and representational theory of measurement; that is, in both cases measurement is understood as the product of a real number and a unit. The difference is only a matter of linguistic expression. Also, I don't believe that conjoint measurement was discovered by representationalists by chance. On the contrary, since it is a typical abstract mathematical construction with no practical value it could only be discovered the abstract, mathematical way.
S. I wonder whether you could briefly explain why you think measurement is understood as a product of a real number and a unit in the representational theory. I start to wonder, which representational theory. I understand "the" representational theory as the idea that we 'use numbers' to represent qualitative relations. I suppose the key question is: how does one define a unit, such as the kg, in purely representational terms? Given there must be reference to operations in both the standard/classical and representational conceptions of measurement, there may be common ground; however, I find it difficult to see how a unit such as the kg or volt can be interpreted in the same way in the representational and standard/classical conceptions.
G. Representational theory describes (ratio) measurement as follows: “The sequence a, 2a = a*a', 3a = (2a) * a", 4a, 5a, ... is called a standard sequence based on a. A meter stick graded in millimeters provides, in convenient form, the first 1000 members of a standard sequence constructed from a one-millimeter rod. If we observe that rod b falls between na and (n + 1)a, say, between 480 and 481 mm, then we assign it a length between nPHI(a)and (n + 1) PHIa (…).The value of PHI(a) depends on the selection of a particular rod (say, e) to have unit length. If e ~ ma, then PHI(a) = 1/m. Thus, if e is the meter stick, then m = 1000 and the length assigned to b must be between 0.480 and 0.481 meters; if e is a centimeter rod, then m = 10 and PHI(b) must be between 48.0 and 48.1 cm.” (FM, Vol. 1, p. 4)
Where exactly is the difficulty with the interpretation of the concept of unit in representationalism?
Guenter
Stephen Humphry
You ask how a unit is interpreted in the classical conception. A unit is a particular magnitude of a kind of quantity. The kg is a particular mass. The metre is a particular length. The second is a particular duration. Then a derived unit is stated in terms of these, probably most simply, the unit of speed is that of an object travelling exactly 1 metre per second. There has to be a means of estimating the ratio of a magnitude to another. This depends on relations among different kinds of quantities and observable regularities.
G. This way numbers are associated with objects by means of the standard sequence and in this sense measurement is a product of a real number and a unit in representational theory.
S. Measurement is a product of a real number? What do you think is measured: an object, or a property of an object?
S. The ratio of a magnitude to a unit is established using instruments and procedures that allow the comparison. In practice, something like a standard sequence is mostly involved in a mesaurement process as a whole to estimate a magnitude; e.g. M = 5 kg. However, ongologically, in principle it is possible to count successive wave crests or cycles in the measurement of magnitudes of kinds of quantities. Time is defined in terms of periods of waves. In principle, lengths are measuremable though the observable manifestions of waves; e.g. through interferometry. Then, given the relations E = h f, and E = m {c}^2, energy and mass are mesurable in terms of wave frequencies (or periods). So these are naturally occurring manifestations that relate a fundamental countable phenomena (wave cycles) to the various continuous quantities in a way that allows measurement. Measurement depends on systems of quantitative relations, and to precisely define units requires using those systems, units must be defined within a system, where the system reflects the various quantitative relations, mutual, definitional and causal, that are described by physical science. (Pure) numbers are not assigned to anything in measuring magnitude relative to units; neither are (pure) numbers "associated with" anything. Numbers of entities, such as collinearly juxtaposed rods, indicate numbers of units of length because each rod possesses the unit of length; i.e. it has a particular length.
G. If we observe that rod b falls between na and (n + 1)a, say, between 480 and 481 mm, then we assign it a length between nPHI(a)and (n + 1) PHIa (…).
S. We can't assign lengths. We can estimate and express how large a length is relative to a unit using a standard sequence, but no assignment is involved--not of lengths or pure numbers. There is an association, between the number of entities with a particular length and the length that is measured, because entities actually have lengths (particularly things like rods for which the meaning of length is quite well-defined).
Steve
________________________________________
From: talking-m...@googlegroups.com [talking-m...@googlegroups.com] On Behalf Of Trendler, Guenter [guenter....@zi-mannheim.de]
Sent: Saturday, 16 April 2011 4:58 AM
Trendler, Guenter
The discussion drifted of again to the question about the philosophical validity of representationalism. By and large I see it as valid description of und useable prescription to measurement practice while realists don’t. As I already emphasized on this forum I have no quarrel with representationalism or realism. I fully agree with realism as a valid description of und useable prescription to measurement practice. Hence there is no need to discuss this topic. However, my farewell to conjoint measurement does not rely on representational philosophy. Hence there should be no difficulty for the realist to understand it. Nevertheless it has not been properly addressed; maybe because it has not been properly understood.
Andrew wrote: You stated in your initial post that conjoint measurement is "superfluous" and that derived measurement be used instead. I argued that conjoint measurement is not superfluous because we do not currently have the system of extant, base psychological quantities that derived measurement requires. If this is not directly addressing your arguments then I do not know what is. I concluded that a compelling case for the rejection of conjoint measurement cannot be made. I mentioned the complete failure of other methods to achieve psychological measurement as a means of emphasising my conclusion, not as a means of providing logical justification for it.
G. Actually I argued that conjoint measurement is no better than derived measurement because both involve the construction of standard sequences. (For details please return to my original email.) That is why the former is superfluous, and not because “we do not currently have the system of extant, base psychological quantities that derived measurement requires”. I’m not aware that this argument has been addressed resp. invalidated. The question is: does the application of conjoint measurement involve the construction of a standard sequences in order to obtain measurement values? This is where I’m looking for a direct rejoinder.
GT
-----Ursprüngliche Nachricht-----
Von: talking-m...@googlegroups.com im Auftrag von Stephen Humphry
Gesendet: Sa 16.04.2011 12:52
An: talking-m...@googlegroups.com
Betreff: RE: [talking-measurement] RE: Farewell to Conjoint Measurement
Hi Guenter,
You ask how a unit is interpreted in the classical conception. A unit is a particular magnitude of a kind of quantity. The kg is a particular mass. The metre is a particular length. The second is a particular duration. Then a derived unit is stated in terms of these, probably most simply, the unit of speed is that of an object travelling exactly 1 metre per second. There has to be a means of estimating the ratio of a magnitude to another. This depends on relations among different kinds of quantities and observable regularities.
G. This way numbers are associated with objects by means of the standard sequence and in this sense measurement is a product of a real number and a unit in representational theory.
S. Measurement is a product of a real number? What do you think is measured: an object, or a property of an object?
S. The ratio of a magnitude to a unit is established using instruments and procedures that allow the comparison. In practice, something like a standard sequence is mostly involved in a mesaurement process as a whole to estimate a magnitude; e.g. M = 5 kg. However, ongologically, in principle it is possible to count successive wave crests or cycles in the measurement of magnitudes of kinds of quantities. Time is defined in terms of periods of waves. In principle, lengths are measuremable though the observable manifestions of waves; e.g. through interferometry. Then, given the relations E = h f, and E = m {c}^2, energy and mass are mesurable in terms of wave frequencies (or periods). So these are naturally occurring manifestations that relate a fundamental countable phenomena (wave cycles) to the various continuous quantities in a way that allows measurement. Measurement depends on systems of quantitative relations, and to precisely define units requires using those systems, units must be defined within a system, where the system reflects the various quantitative relations, mutual, definitional and causal, that are described by physical science. (Pure) numbers are not assigned to anything in measuring magnitude relative to units; neither are (pure) numbers "associated with" anything. Numbers of entities, such as collinearly juxtaposed rods, indicate numbers of units of length because each rod possesses the unit of length; i.e. it has a particular length.
G. If we observe that rod b falls between na and (n + 1)a, say, between 480 and 481 mm, then we assign it a length between nPHI(a)and (n + 1) PHIa (.).
S. We can't assign lengths. We can estimate and express how large a length is relative to a unit using a standard sequence, but no assignment is involved--not of lengths or pure numbers. There is an association, between the number of entities with a particular length and the length that is measured, because entities actually have lengths (particularly things like rods for which the meaning of length is quite well-defined).
Steve
________________________________________
From: talking-m...@googlegroups.com [talking-m...@googlegroups.com] On Behalf Of Trendler, Guenter [guenter....@zi-mannheim.de]
Sent: Saturday, 16 April 2011 4:58 AM
To: talking-m...@googlegroups.com
Subject: AW: [talking-measurement] RE: Farewell to Conjoint Measurement
Hi Andrew, Steve,
G. In my view there is no essential difference between the classical and representational theory of measurement; that is, in both cases measurement is understood as the product of a real number and a unit. The difference is only a matter of linguistic expression.
S. I wonder whether you could briefly explain why you think measurement is understood as a product of a real number and a unit in the representational theory. I start to wonder, which representational theory. I understand "the" representational theory as the idea that we 'use numbers' to represent qualitative relations. I suppose the key question is: how does one define a unit, such as the kg, in purely representational terms? Given there must be reference to operations in both the standard/classical and representational conceptions of measurement, there may be common ground; however, I find it difficult to see how a unit such as the kg or volt can be interpreted in the same way in the representational and standard/classical conceptions.
G. How is a unit interpreted in the classical conception?
A. Here you are very much mistaken. There are important and non-trivial differences between the classical and representational definitions of measurement. Measurement in the classical/standard sense must only involve the estimation of ratios between magnitudes and unit magnitudes of continuous quantities. This does not involve the act of assigning anything nor the representation of anything. Restricting representational measurement to ratio scales is arbitrary and an argument which contradicts the internal logic of representationalism. Representationalism basically states that measurement consists of the assignment of numbers to objects or events such that the behaviour of the relevant "qualitative" empirical relational system is reflected in a "numerical relational system". In this way merely ordinal and categorical phenomena can be "measured". In the classical theory, only continuous quantities can be measured. In the mind of the realist, the only work of the representationalists that is relevant to measurement is the work concerning "interval" and "ratio" scales.
G. Let us forget for a moment ordinal measurement. isn't it clear that in the case of unit-based measurement (e.g. interval and ratio scale measurement) the assignment of numbers to (properties) of objects is accomplished by means of a standard sequence? For example, in the case of length measurement the empirical relational system is a set of objects (e.g. fish), the standard sequence is a ruler (1) and the numerical relational system is the system of positive real numbers. Note that the procedure of length measurement is the same as in classical theory. That is, in practice the proceeding is identical (2). It is only described differently. This way numbers are associated with objects by means of the standard sequence and in this sense measurement is a product of a real number and a unit in representational theory.
Trendler, Guenter
sorry, missed this...
G. In my view the role of the unit in conjoint measurement is already clear. It is there at the core of conjoint measurement. Without the concept of unit the axioms of conjoint measurement would not make any sense. The axioms are chosen is such a way as to logically legitimize the construction of standard sequences (i.e. combinations of units). There is no essential difference between conjoint and derived measurement. That this is not generally recognized is not necessarily only the fault of representationalism.
A. I disagree that units are at the core of current axiomatic presentations of conjoint measurement. Typically in such presentations, units only get a mention in passing in the "uniqueness theorem" section following the axioms and statement of the representation theorem. That these theorems empirically hold for a given natural system logically implies that the axioms have already been tested. In my view one has to design the whole experiment prior to the collection of data with the concept of a "tentative unit" in mind.
G. I also disagree. The measurement axioms are used to mathematically prove the representation theorem. In the case of metric measurement (e.g. ratio measurement) the representation theorem asserts that if a standard sequence can be established, then a homomorphism into a numerical relational structure can be constructed. A standard sequence is nothing but a different expression for a sequence of standard units. The axioms ensure that such a standard sequence can be constructed in a logically consistent manner. Hence the unit is the key element of all metric measurement system (e.g. extensive measurement, difference measurement, conjoint measurement etc.).
A. One can apply conjoint measurement to the levels of attributes where there is no explicit unit defined, and find that no such unit is obtained after application. Have a read of some of my work applying conjoint measurement to attitudes via Coombs (1964) theory of unidimensional unfolding. There you will find instances where the cancellation axioms hold but only unequal scale differences between attitude levels are obtained (the so - called "ordered metric" scale). See Kyngdon (2006) and Kyngdon & Richards (2007). Obtaining standard sequences from the post hoc application of conjoint measurement is by no means a certainty, as you will see.
G. Of course, equal differences between attitude levels can be obtain only through the construction of a standard sequence. Hoping that the unit is already somehow out there and can be discovered by testing cancellation axioms is totally illusory.
G.
-----Ursprüngliche Nachricht-----
Von: talking-m...@googlegroups.com im Auftrag von Andrew Kyngdon
Gesendet: Fr 15.04.2011 07:04
Trendler, Guenter
On p. 249 Joel Michell writes: “Of course, any researcher is free to speculate about what might be concluded were the attributes involved quantitative (rather than ordinal) and were the scale values used actually measurements of those attributes (rather than ordinal indices) and the sorts of analyses referred to above then done. However, in that case, the conclusions would follow from a combination of observation and speculation. Presenting such conclusions as valid inferences from the observations alone would reflect, at best, profound ignorance. Those who support scientific research financially and those whose lives are affected by the conclusions scientists arrive at are entitled to expect scientists to alert them when conclusions stand, even partly, on speculation, and they have a right to be assured that all other conclusions are supported exclusively by evidence.”
I think Joel Michell touches here a delicate but extremely important topic, namely the responsibility of the scientist, which I believe is not sufficiently enough discussed in the scientific community (see 1 for an exception). Obviously in the context of the “pathological science syndrome” the issue of responsibility is of critical importance. I believe that one of the most effective methods to combat pathological science is to arouse the novice attention to the possibility that the “science” he is about to study may be pathological. For instance, I can imagine that an elementary course on “The Ethics of Science” becomes obligatory for every freshman. Among other things such a course would train him in recognizing a branch of science as pathological. Thus the freshman will get the means to recognize a “science” as pathological and decide in due time to do something else instead of squandering the tax payers money. Raising critical awareness and responsibility will also establish a protection mechanism against undesired developments in science. This is the more important since it is of almost no use to discuss these matters with someone embarked on the pathological ship. Then he is already as good as lost for the scientific cause. Exceptions confirm the rule.
On p. 253 Michell writes: “The underlying cause here is the failure of mainstream psychology to sustain a culture of uncompromising critical inquiry. In a talk in 1974, the physicist Richard Feynman (1985) drew attention to the fact that what psychology lacked was “a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty—a kind of leaning over backwards” (p. 341) to accommodate criticisms. Feynman was not referring to personal qualities of individual psychologists. Psychologists, in general, have as much personal integrity as other scientists, being drawn from the same population. Feynman was referring to a cultural phenomenon: the failure of mainstream psychology as a social movement to sustain institutional mechanisms of rigorous criticism.”
Feynman’s talk is of course a most read for every scientist (2). However, I also believe that in his speech Feynman offers a powerful metaphor to understand and analyse the anatomy of a pathological science, namely the phenomenon of “cargo cult” (3). The first question any student must ask before he begins his studies is: is the science I’m interested in a cargo-cult science?
(2) http://calteches.library.caltech.edu/51/2/CargoCult.pdf
(3) http://en.wikipedia.org/wiki/Cargo_cult_science
-----Ursprüngliche Nachricht-----
Von: talking-m...@googlegroups.com im Auftrag von Paul Barrett
Gesendet: Fr 15.04.2011 00:55
An: talking-m...@googlegroups.com
Betreff: [talking-measurement] Mixed Qualitative and Quantitative Research
And
Regards .. Paul
Paul Barrett
Psychological Measurement is openly acclaimed as being different from
physical measurement.
IRT is its saviour.
A position piece from Klaus Sijtsma ...
Sijtsma, K. (2011) Introduction to the measurement of psychological
attributes. Measurement, 44, 7, 1209-1219.
Abstract
This article introduces the measurement of psychological attributes, such as
intelligence and extraversion. Examples of measurement instruments are
discussed, as well as a deterministic measurement model. Error sources that
threaten measurement precision and validity are discussed, and also ways to
control their detrimental influence. Statistical measurement models describe
the random error component in empirical data and impose a structure that, if
the model fits the data, implies particular measurement properties for the
scale. The well-known Rasch model is discussed along with other models, and
using a sample of data collected with 612 students who solved 13 arithmetic
tasks it is demonstrated how a scale for arithmetic ability is calibrated.
The difference between psychological measurement and physical measurement is
briefly discussed.
Joel Michell gets one reference and no discussion.
The man has never heard of Pace, V.L., Brannick, M.T. (2010) How similar are
personality scales of the "same" construct? A meta-analytic investigation.
Personality and Individual Differences, 49, 7, 669-676.
But then, this is merely a statistician defining measurement quite
subjectively.
It's as though Wood, R. (1978) Fitting the rasch model - a heady tale.
British Journal of Mathematical and Statistical Psychology, 31, 27-32, never
existed. The Rasch model fit's random coin-tosses near perfectly. And
Sijtsma now calls this "psychological measurement"?
And, why worry about Michell, J. (2004) Item Response Models, pathological
science, and the shape of error. Theory and Psychology, 14, 1, 121-129?
Maybe if he read something like:
Courville, T.G. (2004) An empirical comparison of item response theory and
classical test theory item/person statistics. PhD Thesis - Texas A & M
University : http://txspace.tamu.edu/handle/1969.1/1064, 1-129, he might
decide to join the rest of us in the real world.
But, none of those many articles out there which might give a serious
scholar pause for deep concern are even mentioned in this article.
Instead, psychological measurement is now stated by fiat as being
'different' from physical measurement. The word "quantity" never enters the
discussion. It's not even defined. Measurement is simply defined by the
operations of the mathematical model fit to data. That's all there is to it.
22 years of careful exploration and elaboration, through to Andrew's
forthcoming article:
Kyngdon, A. (2011) Plausible measurement analogies to some psychometric
models of test performance. British Journal of Mathematical and Statistical
Psychology
(http://onlinelibrary.wiley.com/doi/10.1348/2044-8317.002004/full), in
press.
This is what it comes down to ... a simple-minded redefinition of
measurement especially for psychology, built on the back of
questionnaire-item response patterns. It's so pathetic words fail me.
If it wasn't 8:40am here, I'd be reaching for a stiff drink!
Regards .. Paul
---------------------------
Advanced Projects R&D Ltd.
---------------------------
W: www.pbarrett.net
E: pa...@pbarrett.net
M: +64-(0)21-415625
Andrew Kyngdon
Just grab a good coffee instead. :)
We cannot really expect anything different from senior figures like Sijtsma who are nearing retirement. They are too set in their ways and thinking to change their minds. Paul Kline was the exception that proved the rule.
As far as I see it, it is unscientifically untenable to claim measurement and endorse some definition of it other than the classical/standard one. Creating another definition of measurement raises far more questions than what it answers. The representational theory of measurement is the only coherent alternative the classical/standard definition, but it runs into enormous problems of its own making and lacks the conceptual parsimony of the classical theory. Perhaps the most fundamental problem is that it completely fails to explain the success of the classical definition of measurement in physics, engineering and everyday life. Stonehenge and the Pyramids were built without any knowledge at all of representation theorems and set theory, yet they were not built without well defined units of measurement and the estimation of ratios. How did this happen if the classical theory is false?
Avoidance of the classical theory of measurement is just an evasion of some of the fundamental scientific questions facing psychometrics. Are psychological attributes quantitative? If they are, how can they be measured in well defined units? Are some psychological attributes measurable whilst others are not? Can tests actually measure or do we need to develop other observational methodologies? These questions will simply remain unanswered if, like Sijtsma, psychometricians continue to avoid discussion of the relevant issues.
I've been drafting a commentary on a paper where the authors went to considerable effort to create a descriptive theory of student progression on a component of the science curriculum. Seemingly unbeknownst to them, their compelling descriptive theory entailed that progression was a partial order. Phenomena that are partial orders cannot be measured, but the authors considered that their theory actually strengthened application of the Rasch model. It was amazing to read how strongly they stressed that progression was qualitative and ordinal, only in the next breath to say that it was quantitative and measureable. Obviously they had not much of an idea of the differences between order, quality and quantity and their paper suffered from the confusion of these concepts.
If it's any consolation, there is still a thirst out there for knowledge and clarity regarding psychological measurement. Joel Michell recently told me that, despite several years of retirement now, he still gets an email once a fortnight or so from some corner of the world asking about measurement. When working he used to get an email almost once per day. Obviously there are many out there who know that there are problems with psychological measurement and are keen enough to start seeking answers. We can only encourage them.
Cheers,
Andrew
-----Original Message-----
From: talking-m...@googlegroups.com [mailto:talking-m...@googlegroups.com] On Behalf Of Paul Barrett
Sent: Tuesday, 31 May 2011 6:38 AM
To: talking-m...@googlegroups.com
IRT is its saviour.
Regards .. Paul
--
Paul Barrett
Dominican monk to endure!
Some on the list might enjoy my current Barrett View #3: Shades of Milgram
and Meehl: The obedient psychologist
(http://www.pbarrett.net/tbv/Index.html#BV3 )
And the forthcoming one: Psychologists, spin, and Feynman's "utter honesty".
On a more profound note, James' Grice's personality lab website now contains
more details about Observational Oriented Modeling, including an excellent
video presenting the philosophy, the binary procrustes methodology & logic,
and a worked example of OOM on a typical psychology dataset.
http://psychology.okstate.edu/faculty/jgrice/personalitylab/methods.htm
The book reference itself (plus software) is:
Grice, J. (2011) Observation Oriented Modeling: Analysis of cause in the
behavioral sciences. New York: Academic Press. ISBN: 978-012-385194-9.
Regards .. Paul
Nick Connolly
Andrew Kyngdon
--