Embracing Subjectivity
10 views
Skip to first unread message
CuppoJava
Sep 23, 2011, 2:00:58 PM9/23/11
to PatricksScratchpad
Comments on the article "Embracing Subjectivity"
Alexander Farley
Sep 23, 2011, 2:52:33 PM9/23/11
to PatricksScratchpad
I disagree that the interpretation of that picture as a dalmation is
subjective. In fact, I bet that a fast enough image-recognition AI
would arrive at the same conclusion. My weak little brain recognized a
4-legged creature, but I interpreted it as facing the opposite
direction (to the right). After reading the "real answer", I agree
that a dalmation is a better fit, but I don't see any need for
admitting subjectivity into the picture. You wouldn't be convinced
that the picture was actually a clown, even if I insisted that I saw a
clown, would you? Preference for the color blue vs. purple is
subjective, since I don't think I make the argument that either is a
better color. Whatever someone says, in a subjective statement, is the
truth. On the other hand, I think it's easy to demonstrate that a
dalmation explains more of the picture than a clown does.
It seems to me like this article mixes up subjectivity and statistical
interpretation, unless you consider them to be the same thing. Of
course, in daily life it's hard to actually be certain about things
since most of our impressions are interpretations of noisy sensory
inputs, but does that make them "subjective" or does it just make them
the best objective guess we can make, with noisy inputs and simple
priors?
subjective. In fact, I bet that a fast enough image-recognition AI
would arrive at the same conclusion. My weak little brain recognized a
4-legged creature, but I interpreted it as facing the opposite
direction (to the right). After reading the "real answer", I agree
that a dalmation is a better fit, but I don't see any need for
admitting subjectivity into the picture. You wouldn't be convinced
that the picture was actually a clown, even if I insisted that I saw a
clown, would you? Preference for the color blue vs. purple is
subjective, since I don't think I make the argument that either is a
better color. Whatever someone says, in a subjective statement, is the
truth. On the other hand, I think it's easy to demonstrate that a
dalmation explains more of the picture than a clown does.
It seems to me like this article mixes up subjectivity and statistical
interpretation, unless you consider them to be the same thing. Of
course, in daily life it's hard to actually be certain about things
since most of our impressions are interpretations of noisy sensory
inputs, but does that make them "subjective" or does it just make them
the best objective guess we can make, with noisy inputs and simple
priors?
CuppoJava
Sep 24, 2011, 12:17:42 PM9/24/11
to PatricksScratchpad
Thanks for the comments Alex. You mentioned statistical interpretation
which actually is quite confusing as to how it relates to this
article. I was worried for a little bit about whether my point has
been nullified, but after some more thought, I think I'm safe.
So let us say there are two competing theories for explaining this
picture. Theory 1 says that it is a dalmatian. Theory 2 says that it
is a clown.
I personally think Theory 1 makes more sense, and as you said, would
think so even if you insisted otherwise. However, I cannot think of
any objective reason why Theory 1 is a "better" theory than Theory 2.
It just makes more sense to *me*. How exactly would you demonstrate
that a dalmatian explains more of the picture than a clown does? By
what and whose objective function?
In response to your point about statistical interpretation, I believe
that they run into the exact same problem actually. Which is that,
having already observed the data, there isn't a satisfying way to say
that one model is better than another. This leads to the classical
problem of "overfitting". There are heuristics that people end up
using (eg. regularizers, or cross-validation), but they *are*
heuristics and which one you use is a matter of personal choice.
(Otherwise we would all end up using the same thing and model
selection would be a solved problem.)
I would like to say, that the problem of being given a 2d image of
black and white pixels, and being asked "What is it a picture of?" -
is analogous to the problem of being given a set of points and being
asked "What order polynomial are these points drawn from?". And we
just don't have a good answer to that question.
An example of a similar but objective problem would be the computer-
vision problem you mentioned where we have a big collection of
pictures, got a hundred people to figure out what is in them, and our
*objective* is to come up with a classifier that agrees with as many
people as possible. That is a perfectly objective task, where noisy
inputs is an added difficulty. But that is not the reason I prefer
Theory 1. I prefer Theory 1 simply because it makes sense to me. Even
if a thousand other people and computer-vision algorithms insist
otherwise, I would still prefer Theory 1. Wouldn't you?
On Sep 23, 11:52 am, Alexander Farley <alexander.s.far...@gmail.com>
wrote:
which actually is quite confusing as to how it relates to this
article. I was worried for a little bit about whether my point has
been nullified, but after some more thought, I think I'm safe.
So let us say there are two competing theories for explaining this
picture. Theory 1 says that it is a dalmatian. Theory 2 says that it
is a clown.
I personally think Theory 1 makes more sense, and as you said, would
think so even if you insisted otherwise. However, I cannot think of
any objective reason why Theory 1 is a "better" theory than Theory 2.
It just makes more sense to *me*. How exactly would you demonstrate
that a dalmatian explains more of the picture than a clown does? By
what and whose objective function?
In response to your point about statistical interpretation, I believe
that they run into the exact same problem actually. Which is that,
having already observed the data, there isn't a satisfying way to say
that one model is better than another. This leads to the classical
problem of "overfitting". There are heuristics that people end up
using (eg. regularizers, or cross-validation), but they *are*
heuristics and which one you use is a matter of personal choice.
(Otherwise we would all end up using the same thing and model
selection would be a solved problem.)
I would like to say, that the problem of being given a 2d image of
black and white pixels, and being asked "What is it a picture of?" -
is analogous to the problem of being given a set of points and being
asked "What order polynomial are these points drawn from?". And we
just don't have a good answer to that question.
An example of a similar but objective problem would be the computer-
vision problem you mentioned where we have a big collection of
pictures, got a hundred people to figure out what is in them, and our
*objective* is to come up with a classifier that agrees with as many
people as possible. That is a perfectly objective task, where noisy
inputs is an added difficulty. But that is not the reason I prefer
Theory 1. I prefer Theory 1 simply because it makes sense to me. Even
if a thousand other people and computer-vision algorithms insist
otherwise, I would still prefer Theory 1. Wouldn't you?
On Sep 23, 11:52 am, Alexander Farley <alexander.s.far...@gmail.com>
wrote:
Alexander Farley
Sep 24, 2011, 12:59:37 PM9/24/11
to PatricksScratchpad
By what and whose objective function: I think, for instance, that
simple correlation with scaled and translated dog and clown pictures
from different perspectives would find the lowest cost as being a dog
in the position you describe. If someone presented an alternative
objective function, with established performance in image
classification, and it yielded some other answer, then I would
consider the alternative suggestion.
If you were to claim that it was just a picture of a picture of a
dalmation, I would consider that subjective or indeterminable. If you
claim that the picture represents a different phenomena which would
yield the same sensory input, then there doesn't seem to be an
objective way to discern the two.
I agree that use of heuristics decreases certainty, but I don't think
that they decrease objectivity when they are applied. I would just
claim that you have to make an objective choice of classifier
algorithm. Most heuristic choices are defensible on the grounds of
providing low classification error. If a case appeared where a less-
heuristic algorithm yielded one answer and a more-heuristic algorithm
yielded another answer, I would probably go with the results of the
less-heuristic algorithm. I still don't see where subjectivity is
involved because I'm objectively recognizing that less-heuristic
algorithms tend to have lower classification error.
So, my understanding of your argument is basically this: noise leads
to uncertainty in classification which leads to subjectivity. I agree
that there is not one provably correct answer for i.e. polynomial
fitting, but that doesn't mean that there aren't objectively better
and worse possibilities. Also, I would have to see the alternative
conclusion suggested by the thousand people/computer vision
algorithms, to decide if I still prefer Theory 1. If the alternative
was something that explains the picture equally well, i.e. "it's a
picture of a picture of a dalmation", then I would not be convinced,
but I would give up arguing. If a thousand people claimed they agreed
with Theory 2, I would question which objective function they were
choosing, and whether it was designed to provide low classification
error or just claim that everything looks like a clown.
simple correlation with scaled and translated dog and clown pictures
from different perspectives would find the lowest cost as being a dog
in the position you describe. If someone presented an alternative
objective function, with established performance in image
classification, and it yielded some other answer, then I would
consider the alternative suggestion.
If you were to claim that it was just a picture of a picture of a
dalmation, I would consider that subjective or indeterminable. If you
claim that the picture represents a different phenomena which would
yield the same sensory input, then there doesn't seem to be an
objective way to discern the two.
I agree that use of heuristics decreases certainty, but I don't think
that they decrease objectivity when they are applied. I would just
claim that you have to make an objective choice of classifier
algorithm. Most heuristic choices are defensible on the grounds of
providing low classification error. If a case appeared where a less-
heuristic algorithm yielded one answer and a more-heuristic algorithm
yielded another answer, I would probably go with the results of the
less-heuristic algorithm. I still don't see where subjectivity is
involved because I'm objectively recognizing that less-heuristic
algorithms tend to have lower classification error.
So, my understanding of your argument is basically this: noise leads
to uncertainty in classification which leads to subjectivity. I agree
that there is not one provably correct answer for i.e. polynomial
fitting, but that doesn't mean that there aren't objectively better
and worse possibilities. Also, I would have to see the alternative
conclusion suggested by the thousand people/computer vision
algorithms, to decide if I still prefer Theory 1. If the alternative
was something that explains the picture equally well, i.e. "it's a
picture of a picture of a dalmation", then I would not be convinced,
but I would give up arguing. If a thousand people claimed they agreed
with Theory 2, I would question which objective function they were
choosing, and whether it was designed to provide low classification
error or just claim that everything looks like a clown.
Patrick Li
Sep 24, 2011, 2:08:37 PM9/24/11
to patrickss...@googlegroups.com
I agree with everything you have said. You can create a well-defined objective task (such as lowering classification error). Then any of these various heuristics can be evaluated, and you just simply choose the best one.
The question is, in the absence of an objective task, such as in the case of recognizing this picture, is there anything worthwhile that can be learned? We are not concerned with lowering classification error - there is only a single picture here. We are just trying to "understand" this one picture, whatever that means. By your own reasoning, you have found the dalmatian theory to be more satisfying than the clown theory.
Let us suppose that I have my own objective function. And for every single other picture in the world *except* this one, my classification agrees with yours. I insist it's a clown, and you insist it's a dalmatian. For any practical purpose, whether its a clown or dalmatian makes no difference. But it makes a difference to *you*: one theory makes sense, the other just doesn't. Isn't this a subjective argument?
I mean, I am sure that there are lots of people who will insist that that actually is a clown and not a dalmatian. On what grounds can I say they're wrong except to say that more people agree with me than agree with them?
Here is a fun little example involving those 3d "magic-eye" stereoscopic images. About half the world, can cross their eyes, and "see" a three dimensional object pop out from the pages, and about half the world simply sees a flat patterned page. How does one side objectively say that the other side is blind? And how does the other side objectively say that the other side is deluding themselves?
Alexander Farley
Oct 3, 2011, 3:18:16 PM10/3/11
to PatricksScratchpad
I think that if a pair of classifiers existed such that they gave the
exact same response to every input except one case, even including
arbitrarily similar but not exactly equal inputs, it could be shown
that one of those classifiers decided on its response to that disputed
input based on something other than a learning process. I can't think
of a really easy way to prove this, but it seems to make sense.
Another way of saying this would be that the classification boundary
in the input space is extremely non-convex, i.e. it includes grainy
points which could correspond to preset responses, not points with any
geometric relationship to anywhere else in the input space.
Also, I argue that the "hypothesis" that a dalmation is in the picture
vs a clown is a valid scientific hypothesis in the sense that it can
be tested. The test is performed by comparing our brain's degree of
recognition of a dog in the hypothesized orientation in the picture
under consideration. The process of "looking for the dog" in the image
is a sequence of hypothesis tests where in each case, the hypothesis
is a dog in a different orientation/perspective/pose etc.
This isn't purely internal, either; it would be possible, once someone
claims to see the dog, to ask what orientation the dog is in. For the
rotating wheels example you presented, I would expect that people see
the same patterns of relative rotation (as in, which are rotating in
opposite directions with regard to each other). This could be used to
determine whether they really have recognized the same thing, or are
just deluding themselves.
exact same response to every input except one case, even including
arbitrarily similar but not exactly equal inputs, it could be shown
that one of those classifiers decided on its response to that disputed
input based on something other than a learning process. I can't think
of a really easy way to prove this, but it seems to make sense.
Another way of saying this would be that the classification boundary
in the input space is extremely non-convex, i.e. it includes grainy
points which could correspond to preset responses, not points with any
geometric relationship to anywhere else in the input space.
Also, I argue that the "hypothesis" that a dalmation is in the picture
vs a clown is a valid scientific hypothesis in the sense that it can
be tested. The test is performed by comparing our brain's degree of
recognition of a dog in the hypothesized orientation in the picture
under consideration. The process of "looking for the dog" in the image
is a sequence of hypothesis tests where in each case, the hypothesis
is a dog in a different orientation/perspective/pose etc.
This isn't purely internal, either; it would be possible, once someone
claims to see the dog, to ask what orientation the dog is in. For the
rotating wheels example you presented, I would expect that people see
the same patterns of relative rotation (as in, which are rotating in
opposite directions with regard to each other). This could be used to
determine whether they really have recognized the same thing, or are
just deluding themselves.
CuppoJava
Oct 10, 2011, 7:34:32 PM10/10/11
to PatricksScratchpad
Hmm that argument does actually throw a wrench in my argument. I'm
gonna have to reconsider exactly what I wanted to say and rewrite it I
think now..
I disagree with your assertion about the pair of classifiers, where a
sensible classifier that gets everything right except for one must be
disregarding the "geometric relationships" with other points. I'm not
as quick to assume that our pre-conceptions about what is and what
isn't dogs can be expressed in any well behaving geometric manner. In
fact, that is part of the problem, that simple objective functions
with intuitive geometric interpretations (such as sum of square
differences) coincide so incredibly poorly with our concept of "dog"-
ness.
The point you made that I find pretty hard to defend against is the
assertion that the hypothesis is actually testable in some way. As you
mentioned, in the example of the rotating disks, I could probe them by
asking for what direction the disks are rotating in. Then if they
answer "correctly" (as in, they see what I see), then I can be
somewhat reassured that they are indeed experiencing the same
phenomenon I am.
So I actually don't know what to say against that point right now. I
will keep on pondering on that.
But I feel that there is an analogy to be made, and an absurdem to be
reached. Is there a fundamental difference between the following 2
scenarios? I would like to say there are, but can't think of the
difference right now:
Person 1 says: Look at these pictures of disks. They are actually
rotating. You might not see them as rotating, but give it some time, I
can teach you to see it.
Person 2 says: Close your eyes and meditate on the balance of the
universe. Can you flow the energy flowing through your muscles? If you
cannot, give it some time, I can teach you how.
On Oct 3, 12:18 pm, Alexander Farley <alexander.s.far...@gmail.com>
gonna have to reconsider exactly what I wanted to say and rewrite it I
think now..
I disagree with your assertion about the pair of classifiers, where a
sensible classifier that gets everything right except for one must be
disregarding the "geometric relationships" with other points. I'm not
as quick to assume that our pre-conceptions about what is and what
isn't dogs can be expressed in any well behaving geometric manner. In
fact, that is part of the problem, that simple objective functions
with intuitive geometric interpretations (such as sum of square
differences) coincide so incredibly poorly with our concept of "dog"-
ness.
The point you made that I find pretty hard to defend against is the
assertion that the hypothesis is actually testable in some way. As you
mentioned, in the example of the rotating disks, I could probe them by
asking for what direction the disks are rotating in. Then if they
answer "correctly" (as in, they see what I see), then I can be
somewhat reassured that they are indeed experiencing the same
phenomenon I am.
So I actually don't know what to say against that point right now. I
will keep on pondering on that.
But I feel that there is an analogy to be made, and an absurdem to be
reached. Is there a fundamental difference between the following 2
scenarios? I would like to say there are, but can't think of the
difference right now:
Person 1 says: Look at these pictures of disks. They are actually
rotating. You might not see them as rotating, but give it some time, I
can teach you to see it.
Person 2 says: Close your eyes and meditate on the balance of the
universe. Can you flow the energy flowing through your muscles? If you
cannot, give it some time, I can teach you how.
On Oct 3, 12:18 pm, Alexander Farley <alexander.s.far...@gmail.com>
Reply all
Reply to author
Forward
0 new messages