Hey guys, I'm looking for some feedback by the end of today on some work I'm doing for Anna Salamon at SingInst.  

I'm developing a "rationality checklist". The ideal would be: the more items you can answer 'yes' to, the better off you likely are as a rationalist. It's a way to 'measure' your rationality—to help give a sense of where you stand, where there's room for improvement. It's also a way to track 'progress' over time.  

The skill of Curiosity is basically done. Any and all feedback is much appreciated, such as noting confusing, ambiguous, irrelevant, or missing questions. Where do you stand? My score would be terrible. :P

If you don't have time, no worries.  

Alex

---

*Curiosity*

Do you have specific habits for getting curious when you notice you're not curious about something important?
Do you, in every situation, endeavor to have an accurate map of the territory?
Do you regularly acknowledge/accept the possible worlds that may exist? (e.g. if the iron is hot, I desire to believe it it hot; if it is cool, I desire to believe it is cool)
Do you regularly ask, “What are the causes of my beliefs? Why do I think this? What’s the source?”
Do you regularly ask, “What would I expect to see differently if x was or was not the case?”
Do you regularly ask, when unexpected things happen, “Why didn’t I expect x to happen?”
When you sit down to think, or to look something up, do you regularly ask, "What am I chasing? Why am I doing this? What is most likely to help me learn this and figure it out? Am I asking myself questions about this?"
Do you frequently stop to consider what information will be most valuable to achieving your goals?
Do you frequently ask, "What do I most want to accomplish? What do I need to know, in order to achieve that thing?"
Do you focus your curiosity on information that will be helpful in achieving your goals?  
Do you tend to be curious about what you need to reach your goals?
Do you stop reading when a source becomes irrelevant?
Do you then actively seek out more useful information?
Do you gravitate to inquiries that seem most promising of producing shifts in belief?
Do you gravitate to inquiries that are least like the ones you've tried before?
Do you ever call topics or ideas boring, shallow, crazy, beneath you, or confusing (or other words that close off thought)?
Do you notice when conflicting emotions cut off your curiosity?
Do you, in every social interaction, ask what that person can teach you?
Do you, in every situation where you receive feedback, treat it as potentially valuable?

--  
Alex Vermeer

Hi Alex,

I'm delighted to be able to help test your metric.

I'm posting this to the LW group to answer others' curiosity of me (if
any). Critiques on my philosophy or strategy are welcome.

After doing this quiz, I feel that I've been prompted to do some additional
thinking. This was a valuable exercise.

My score is 9/19 yesses. I found that two of the questions would use "No."
as a better indicator of rationality.
Final score: 7/19.

Stefan

--------------------------------------------------------------------------- --------------------------------------------------------------------------- --------------------

Do you have specific habits for getting curious when you notice you're not

-Yes.
-First, evaluate: "If I'm not curious about this, is it really important?"
-Second, find someone who is curious about this, and have them infect me
with passion.
--This works best in person, but can be accomplished through reading.
-Comment: I did not immediately read this as a yes or no question.
Immediately, I wrote down my thoughts and strategy, instead of answering
the question.

-Yes.
-However: I am willing to settle for a sufficiently accurate map, which
will allow me to manipulate reality to accomplish my goals.
--A perfect map is in principle impossible. Good enough is good enough.

-Yes.
-I do find the litany of Tarski a useful framework to think about why I
want to believe.
-I think that I thought similarly even before reading about this idea on LW
--This is not generally the case. LW has modified some of my thinking, and
has introduced new ideas that I might not have encountered otherwise.
-Comment: I don't understand the phrasing of the questions. The example
clued me in to what you were asking.
-Comment: This question seems to be identical to the previous one.

-No.
-Self-evaluation does not happen on any periodic timescale. Mostly, I
continue accumulating new ideas and placing them in my personal worldview. *
Occasionally* I will either realise a contradiction or be otherwise
prompted to evaluate either the causes of my beliefs or whether there is
any cognitive dissonance.
-It is my hope that by learning about learning that I will be able to
perform this sort of updating as a matter of course when encountering new
ideas that are not so different from my worldview as to require more
protracted reflection. (i.e. I want regular epiphanies until my map becomes
much closer to the territory)

-No.

-No.
-I tend to "react and continue" rather than "react and reflect."

-Yes.
-Although I frequently discover that my desired end-goal was a
less-interesting end-point, and that I will have to shift "what I am
chasing" when I learn more about the subject I am studying.
-On second thought, it would be useful to maintain a log of when I do shift
my learning goals, so that I also study why I had those end-goals in the
first place, which might help me understand how to teach more effectively.

-No.
-I do this every once in a while. For instance, my goal is to become an
astronaut. Five years ago, I decided that my best next step would be an
undergraduate degree in physics. While pursuing my degree, I did not
seriously reevaluate that this was the best step to be currently taking. At
the end of my degree, I decided on a different next step, which diverges
from the physicist's path.
-Frequency = 3 years << Frequently

-No.
-Same answer as above.

-No.
-I tend to become curious about ideas for their own sake, and get diverted
from information that will help achieve my goals.

-Yes.
-And this makes some other fields of study seem boring by comparison.

-Sometimes. I am currently transitioning from No to Yes.
-Final answer: No.
-I get a sense of satisfaction from completeness, so I like to continue
reading even past usefulness.
-I am currently realising that timeliness is also important.

-Yes.

-No.
-I gravitate toward inquiries that help explain away questions. Whether or
not my belief shifts is not actually a measure of whether my map is
becoming more accurate.

-No.
-Generally, I apply methods I have previously learned.
-Generally, I approach questions from ideas that I already have in hand.

-Yes.
-Sometimes, I feel that my "level" is not sufficiently high to begin
reading a particular treatment of a topic.
-Recent example: Wald's text on General Relativity was too fast until I got
a grip on tensors using other sources.
-I don't think a "Yes" response on this is an indicator of a rationalist.
--Rationalists should be able to consider all ideas, and should consider
all knowledge when making decisions.

-Yes.
-A lot of times I question my "cause" (Manned Space Exploration) and
whether or not it's a field where I can make significant contributions. I'm
driven to think about my cause because I want to work on it, but I feel
inadequate to the task, and so I don't deal with it in a seriously curious
manner.
-Upon reflection, the low-frequency with which I think about "What
information will be most important to my goals" is probably an unconscious
strategy to deal with this cut-off problem.
--By precommitting to taking a class or doing a degree or taking a job, I
constrain myself to working on my cause (or some activity related to my
cause) in spite of my feelings of inadequacy.
-I don't think a "Yes" response on this is an indicator of a rationalist.
--For it is written that Curiosity is a Virtue.

-No.
-This happens in many social interactions. However, I view some social
interactions as mere politeness.
-This is a good point. I'll think on how to always be learning from others.

-Yes.
-Either the feedback is legitimate and I should modify myself, or the
feedback is misguided and I have learned something about the way in which
the commenter argues, which I can use to better understand them.

Hey Alex,

This was a fun exercise in self-examination.

Like Stefan did, I'm posting *my own results* as well. I read one or two of
his and decided to respond similarly with a "Yes." or "No." and then a
brief explanation. I also numbered the questions to refer to them more
easily, which is something I would suggest for any future surveys emailed
out, for ease of discussion.

Also like Stefan, I found that questions 16 and 17 confusing with regard to
the polarity of their answers. Specifically, I agree that 16 seems to be
no-positive, but I'm unsure about question 17 (about conflicting emotions).
It strikes me that since conflicting emotions will naturally inhibit the
desire to learn more about something, noticing the conflicting emotions and
their effects is a sign of a rationalist. Having them is not, but which of
these is the question asking about, the having or the noticing? At any
rate, I think it's clear that this question is ambiguous since Stefan and I
interpreted it differently.

I'm going to score myself out of 18, omitting question 17 as I'm unsure how
to grade it.
Final score: *8/18*. Lots of room for improvement.

*General comments on the checklist:*
First of all, I don't really like the checklist format, since all of these
are a matter of degree. While a checklist may be simpler than a likert
scale (eg. 1-5 or even 1-3) it loses the possibility of a "sometimes"
answer or any assessment of degree. Skills are hardly a binary.

In any case, these questions are subjective, and so we will fall victim to
our own standards; that is, if I have a much stricter idea of what
"frequently" or "regularly" means, then I'm going to rank myself much lower
on those questions. This effect is mitigated by using a sliding scale
because it will just turn a 3 into a 2 rather than a 1 into a 0.
Furthermore, the scale could actually be one of frequency: eg *"When
unexpected things happen, how often do you ask "**Why didn’t I expect x to
happen?""* with possible answers of "(almost) never", "<25% of the time",
"less than half of the time", "more than half of the time", ">75% of the
time", and "(almost) always". This removes the ambiguity of the word
"regularly", with the compromise that the quiz becomes much more
complicated.

Still, if this tool is designed to be used for self-improvement and to "track
'progress' over time", it's much easier to say "I used to do this a quarter
of the time and now I do it half of the time" than to say "I used to to
this rarely, but now I do it regularly" especially for rationalist types
that are used to working with percentages for predictions and the like. The
other issue is that if it is used in an ongoing manner, we may find that
our standards for excellence change as well delve more deeply into
self-improvement, and so numerical phrasing helps us have a consistent
measurement tool.

The specific questions seemed quite good, although some appeared to be
repeated or at least too similar when placed adjacently. Notably 8, 9 and
10. If these are supposed to be different, they could use a rephrasing to
make it clear how one would apply to a given situation and the others not.

I just noticed that your email had a deadline on it. I would also say that
you might have received more timely responses if "by the end of today" or
some similarly note of urgency had appeared in the subject line of your
email. I glanced at the message and thought "Oh, this looks like fun but
I'll do it later when I have more time." I hope my feedback has still been
valuable to you.

*My responses*
1 Do you have specific habits for getting curious when you notice you're
not curious about something important?
 No.
To be honest, I can't think of times when I've "noticed I'm not curious
about something important".
 2 Do you, in every situation, endeavor to have an accurate map of the
territory?
 No.
Most of the time, absolutely, but not in every situation.
 (I had to look up these terms, because I haven't spent that much time on
LW and didn't recognize what it was referring to.)
 3 Do you regularly acknowledge/accept the possible worlds that may exist?
(e.g. if the iron is hot, I desire to believe it it hot; if it is cool, I
desire to believe it is cool)
 No.
But the Litany of Tarski is something I'm working to infuse into my regular
thought-patterns.
 4 Do you regularly ask, “What are the causes of my beliefs? Why do I think
this? What’s the source?”
 No.
I do it fairly often when I get into an argument/debate/discussion with
someone, but not just when I'm thinking.
 5 Do you regularly ask, “What would I expect to see differently if x was
or was not the case?”
 No.
At least, not in such words.
 6 Do you regularly ask, when unexpected things happen, “Why didn’t I
expect x to happen?”
 Yes.
I find when I'm in a situation where I'm trying to reason about something
with partial evidence and then I discover that it was actually not what I
expected, I wonder "How should I have drawn my conclusions from the given
evidence to reach the right answer? Was this possible, given the evidence
available?"
 7 When you sit down to think, or to look something up, do you regularly
ask, "What am I chasing? Why am I doing this? What is most likely to help
me learn this and figure it out? Am I asking myself questions about this?"
 No.
I regularly get sidetracked when doing research or when trying to think
through something.
 8 Do you frequently stop to consider what information will be most
valuable to achieving your goals?
 Yes.
When I'm focused on my goals, I can become pretty ruthless in cutting out
irrelevance.
 9 Do you frequently ask, "What do I most want to accomplish? What do I
need to know, in order to achieve that thing?"
 No.
This seems like a similar question to the one above though.
 10 Do you focus your curiosity on information that will be helpful in
achieving your goals?
 Yes.
Also seems similar to the one two above.
 11 Do you tend to be curious about what you need to reach your goals?
 Yes.
If I have a goal to do X, I become much more interested in any blogs or
articles or other sources of information that might help me, and I tend to
notice them more.
 12 Do you stop reading when a source becomes irrelevant?
 Yes.
Well, yes when I realize it's irrelevant.
 I also walk out of movies when I'm not enjoying them.
 13 Do you then actively seek out more useful information?
Yes.
 14 Do you gravitate to inquiries that seem most promising of producing
shifts in belief?
 No.
This reminds me of Wason's 2-4-6 task.
 15 Do you gravitate to inquiries that are least like the ones you've tried
before?
 No.
Well, sometimes.
 Frustratingly, I've noticed myself punching in the same search query (or
something very similar) multiple times in a row. This is more due to
forgetting that that's what I searched for last time than thinking it will
produce different results.
 16 Do you ever call topics or ideas boring, shallow, crazy, beneath you,
or confusing (or other words that close off thought)?
 No.
Only to focus on more pertinent topics, but I will otherwise explore topics
or ideas ad absurdium.
 17 Do you notice when conflicting emotions cut off your curiosity?
 Yes.
I had to think about this, but yeah, I do notice when my feelings about a
possible result cause me to not want to know the answer.
 18 Do you, in every social interaction, ask what that person can teach
you?
 No.
But I would like to develop this habit. I already do it sometimes, but not
consistently.
 19 Do you, in every situation where you receive feedback, treat it as
potentially valuable?
 Yes.
I'm really good at taking criticism. Obviously, it's harder to actually
apply it to produce a change in behaviour, but I'm always open to the idea
of improving and I always try to enact change based on feedback.

Cheers,
Malcolm

On Thu, Apr 19, 2012 at 3:47 PM, Alex Vermeer <vermeer.a...@gmail.com>wrote:

...

read more »