rating your curiosity (looking for feedback)

[Alex Vermeer]

Hey guys, I'm looking for some feedback by the end of today on some work I'm doing for Anna Salamon at SingInst.

I'm developing a "rationality checklist". The ideal would be: the more items you can answer 'yes' to, the better off you likely are as a rationalist. It's a way to 'measure' your rationality—to help give a sense of where you stand, where there's room for improvement. It's also a way to track 'progress' over time.

The skill of Curiosity is basically done. Any and all feedback is much appreciated, such as noting confusing, ambiguous, irrelevant, or missing questions. Where do you stand? My score would be terrible. :P

If you don't have time, no worries.

Alex

---

*Curiosity*

Do you have specific habits for getting curious when you notice you're not curious about something important?
Do you, in every situation, endeavor to have an accurate map of the territory?
Do you regularly acknowledge/accept the possible worlds that may exist? (e.g. if the iron is hot, I desire to believe it it hot; if it is cool, I desire to believe it is cool)
Do you regularly ask, “What are the causes of my beliefs? Why do I think this? What’s the source?”
Do you regularly ask, “What would I expect to see differently if x was or was not the case?”
Do you regularly ask, when unexpected things happen, “Why didn’t I expect x to happen?”
When you sit down to think, or to look something up, do you regularly ask, "What am I chasing? Why am I doing this? What is most likely to help me learn this and figure it out? Am I asking myself questions about this?"
Do you frequently stop to consider what information will be most valuable to achieving your goals?
Do you frequently ask, "What do I most want to accomplish? What do I need to know, in order to achieve that thing?"
Do you focus your curiosity on information that will be helpful in achieving your goals?
Do you tend to be curious about what you need to reach your goals?
Do you stop reading when a source becomes irrelevant?
Do you then actively seek out more useful information?
Do you gravitate to inquiries that seem most promising of producing shifts in belief?
Do you gravitate to inquiries that are least like the ones you've tried before?
Do you ever call topics or ideas boring, shallow, crazy, beneath you, or confusing (or other words that close off thought)?
Do you notice when conflicting emotions cut off your curiosity?
Do you, in every social interaction, ask what that person can teach you?
Do you, in every situation where you receive feedback, treat it as potentially valuable?

--
Alex Vermeer

[Stefan De Young]

Hi Alex,

I'm delighted to be able to help test your metric.

I'm posting this to the LW group to answer others' curiosity of me (if any). Critiques on my philosophy or strategy are welcome.

After doing this quiz, I feel that I've been prompted to do some additional thinking. This was a valuable exercise.

My score is 9/19 yesses. I found that two of the questions would use "No." as a better indicator of rationality.
Final score: 7/19.

Stefan

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Do you have specific habits for getting curious when you notice you're not curious about something important?

-Yes.
-First, evaluate: "If I'm not curious about this, is it really important?"
-Second, find someone who is curious about this, and have them infect me with passion.
--This works best in person, but can be accomplished through reading.
-Comment: I did not immediately read this as a yes or no question. Immediately, I wrote down my thoughts and strategy, instead of answering the question.
 

Do you, in every situation, endeavor to have an accurate map of the territory?

-Yes.
-However: I am willing to settle for a sufficiently accurate map, which will allow me to manipulate reality to accomplish my goals.
--A perfect map is in principle impossible. Good enough is good enough.
 

Do you regularly acknowledge/accept the possible worlds that may exist? (e.g. if the iron is hot, I desire to believe it it hot; if it is cool, I desire to believe it is cool)

-Yes.
-I do find the litany of Tarski a useful framework to think about why I want to believe.
-I think that I thought similarly even before reading about this idea on LW
--This is not generally the case. LW has modified some of my thinking, and has introduced new ideas that I might not have encountered otherwise.
-Comment: I don't understand the phrasing of the questions. The example clued me in to what you were asking.
-Comment: This question seems to be identical to the previous one.
 

Do you regularly ask, “What are the causes of my beliefs? Why do I think this? What’s the source?”

-No.
-Self-evaluation does not happen on any periodic timescale. Mostly, I continue accumulating new ideas and placing them in my personal worldview. Occasionally I will either realise a contradiction or be otherwise prompted to evaluate either the causes of my beliefs or whether there is any cognitive dissonance.
-It is my hope that by learning about learning that I will be able to perform this sort of updating as a matter of course when encountering new ideas that are not so different from my worldview as to require more protracted reflection. (i.e. I want regular epiphanies until my map becomes much closer to the territory)
 

Do you regularly ask, “What would I expect to see differently if x was or was not the case?”

-No.
 

Do you regularly ask, when unexpected things happen, “Why didn’t I expect x to happen?”

-No.
-I tend to "react and continue" rather than "react and reflect."
 

When you sit down to think, or to look something up, do you regularly ask, "What am I chasing? Why am I doing this? What is most likely to help me learn this and figure it out? Am I asking myself questions about this?"

-Yes.
-Although I frequently discover that my desired end-goal was a less-interesting end-point, and that I will have to shift "what I am chasing" when I learn more about the subject I am studying.
-On second thought, it would be useful to maintain a log of when I do shift my learning goals, so that I also study why I had those end-goals in the first place, which might help me understand how to teach more effectively.
 

Do you frequently stop to consider what information will be most valuable to achieving your goals?

-No.
-I do this every once in a while. For instance, my goal is to become an astronaut. Five years ago, I decided that my best next step would be an undergraduate degree in physics. While pursuing my degree, I did not seriously reevaluate that this was the best step to be currently taking. At the end of my degree, I decided on a different next step, which diverges from the physicist's path.
-Frequency = 3 years << Frequently
 

Do you frequently ask, "What do I most want to accomplish? What do I need to know, in order to achieve that thing?"

-No.
-Same answer as above.
 

Do you focus your curiosity on information that will be helpful in achieving your goals?

-No.
-I tend to become curious about ideas for their own sake, and get diverted from information that will help achieve my goals.
 

Do you tend to be curious about what you need to reach your goals?

-Yes.
-And this makes some other fields of study seem boring by comparison.
 

Do you stop reading when a source becomes irrelevant?

-Sometimes. I am currently transitioning from No to Yes.
-Final answer: No.
-I get a sense of satisfaction from completeness, so I like to continue reading even past usefulness.
-I am currently realising that timeliness is also important.
 

Do you then actively seek out more useful information?

-Yes.
 

Do you gravitate to inquiries that seem most promising of producing shifts in belief?

-No.
-I gravitate toward inquiries that help explain away questions. Whether or not my belief shifts is not actually a measure of whether my map is becoming more accurate.
 

Do you gravitate to inquiries that are least like the ones you've tried before?

-No.
-Generally, I apply methods I have previously learned.
-Generally, I approach questions from ideas that I already have in hand.
 

Do you ever call topics or ideas boring, shallow, crazy, beneath you, or confusing (or other words that close off thought)?

-Yes.
-Sometimes, I feel that my "level" is not sufficiently high to begin reading a particular treatment of a topic.
-Recent example: Wald's text on General Relativity was too fast until I got a grip on tensors using other sources.
-I don't think a "Yes" response on this is an indicator of a rationalist.
--Rationalists should be able to consider all ideas, and should consider all knowledge when making decisions.
 

Do you notice when conflicting emotions cut off your curiosity?

-Yes.
-A lot of times I question my "cause" (Manned Space Exploration) and whether or not it's a field where I can make significant contributions. I'm driven to think about my cause because I want to work on it, but I feel inadequate to the task, and so I don't deal with it in a seriously curious manner.
-Upon reflection, the low-frequency with which I think about "What information will be most important to my goals" is probably an unconscious strategy to deal with this cut-off problem.
--By precommitting to taking a class or doing a degree or taking a job, I constrain myself to working on my cause (or some activity related to my cause) in spite of my feelings of inadequacy.
-I don't think a "Yes" response on this is an indicator of a rationalist.
--For it is written that Curiosity is a Virtue.
 

Do you, in every social interaction, ask what that person can teach you?

-No.
-This happens in many social interactions. However, I view some social interactions as mere politeness.
-This is a good point. I'll think on how to always be learning from others.
 

Do you, in every situation where you receive feedback, treat it as potentially valuable?

-Yes.
-Either the feedback is legitimate and I should modify myself, or the feedback is misguided and I have learned something about the way in which the commenter argues, which I can use to better understand them.

[Malcolm McCulloch]

Hey Alex,

This was a fun exercise in self-examination.

Like Stefan did, I'm posting my own results as well. I read one or two of his and decided to respond similarly with a "Yes." or "No." and then a brief explanation. I also numbered the questions to refer to them more easily, which is something I would suggest for any future surveys emailed out, for ease of discussion.

Also like Stefan, I found that questions 16 and 17 confusing with regard to the polarity of their answers. Specifically, I agree that 16 seems to be no-positive, but I'm unsure about question 17 (about conflicting emotions). It strikes me that since conflicting emotions will naturally inhibit the desire to learn more about something, noticing the conflicting emotions and their effects is a sign of a rationalist. Having them is not, but which of these is the question asking about, the having or the noticing? At any rate, I think it's clear that this question is ambiguous since Stefan and I interpreted it differently.

I'm going to score myself out of 18, omitting question 17 as I'm unsure how to grade it.

Final score: 8/18. Lots of room for improvement.

General comments on the checklist:

First of all, I don't really like the checklist format, since all of these are a matter of degree. While a checklist may be simpler than a likert scale (eg. 1-5 or even 1-3) it loses the possibility of a "sometimes" answer or any assessment of degree. Skills are hardly a binary.

In any case, these questions are subjective, and so we will fall victim to our own standards; that is, if I have a much stricter idea of what "frequently" or "regularly" means, then I'm going to rank myself much lower on those questions. This effect is mitigated by using a sliding scale because it will just turn a 3 into a 2 rather than a 1 into a 0. Furthermore, the scale could actually be one of frequency: eg "When unexpected things happen, how often do you ask "Why didn’t I expect x to happen?"" with possible answers of "(almost) never", "<25% of the time", "less than half of the time", "more than half of the time", ">75% of the time", and "(almost) always". This removes the ambiguity of the word "regularly", with the compromise that the quiz becomes much more complicated.

Still, if this tool is designed to be used for self-improvement and to "track 'progress' over time", it's much easier to say "I used to do this a quarter of the time and now I do it half of the time" than to say "I used to to this rarely, but now I do it regularly" especially for rationalist types that are used to working with percentages for predictions and the like. The other issue is that if it is used in an ongoing manner, we may find that our standards for excellence change as well delve more deeply into self-improvement, and so numerical phrasing helps us have a consistent measurement tool.

The specific questions seemed quite good, although some appeared to be repeated or at least too similar when placed adjacently. Notably 8, 9 and 10. If these are supposed to be different, they could use a rephrasing to make it clear how one would apply to a given situation and the others not.

I just noticed that your email had a deadline on it. I would also say that you might have received more timely responses if "by the end of today" or some similarly note of urgency had appeared in the subject line of your email. I glanced at the message and thought "Oh, this looks like fun but I'll do it later when I have more time." I hope my feedback has still been valuable to you.

My responses

1 Do you have specific habits for getting curious when you notice you're not curious about something important?

No.

To be honest, I can't think of times when I've "noticed I'm not curious about something important".

2 Do you, in every situation, endeavor to have an accurate map of the territory?

No.

Most of the time, absolutely, but not in every situation.

(I had to look up these terms, because I haven't spent that much time on LW and didn't recognize what it was referring to.)

3 Do you regularly acknowledge/accept the possible worlds that may exist? (e.g. if the iron is hot, I desire to believe it it hot; if it is cool, I desire to believe it is cool)

No.

But the Litany of Tarski is something I'm working to infuse into my regular thought-patterns.

4 Do you regularly ask, “What are the causes of my beliefs? Why do I think this? What’s the source?”

No.

I do it fairly often when I get into an argument/debate/discussion with someone, but not just when I'm thinking.

5 Do you regularly ask, “What would I expect to see differently if x was or was not the case?”

No.

At least, not in such words.

6 Do you regularly ask, when unexpected things happen, “Why didn’t I expect x to happen?”

Yes.

I find when I'm in a situation where I'm trying to reason about something with partial evidence and then I discover that it was actually not what I expected, I wonder "How should I have drawn my conclusions from the given evidence to reach the right answer? Was this possible, given the evidence available?"

7 When you sit down to think, or to look something up, do you regularly ask, "What am I chasing? Why am I doing this? What is most likely to help me learn this and figure it out? Am I asking myself questions about this?"

No.

I regularly get sidetracked when doing research or when trying to think through something.

8 Do you frequently stop to consider what information will be most valuable to achieving your goals?

Yes.

When I'm focused on my goals, I can become pretty ruthless in cutting out irrelevance.

9 Do you frequently ask, "What do I most want to accomplish? What do I need to know, in order to achieve that thing?"

No.

This seems like a similar question to the one above though.

10 Do you focus your curiosity on information that will be helpful in achieving your goals?

Yes.

Also seems similar to the one two above.

11 Do you tend to be curious about what you need to reach your goals?

Yes.

If I have a goal to do X, I become much more interested in any blogs or articles or other sources of information that might help me, and I tend to notice them more.

12 Do you stop reading when a source becomes irrelevant?

Yes.

Well, yes when I realize it's irrelevant.

I also walk out of movies when I'm not enjoying them.

13 Do you then actively seek out more useful information? 

Yes.

14 Do you gravitate to inquiries that seem most promising of producing shifts in belief?

No.

This reminds me of Wason's 2-4-6 task.

15 Do you gravitate to inquiries that are least like the ones you've tried before?

No.

Well, sometimes.

Frustratingly, I've noticed myself punching in the same search query (or something very similar) multiple times in a row. This is more due to forgetting that that's what I searched for last time than thinking it will produce different results.

16 Do you ever call topics or ideas boring, shallow, crazy, beneath you, or confusing (or other words that close off thought)?

No.

Only to focus on more pertinent topics, but I will otherwise explore topics or ideas ad absurdium.

17 Do you notice when conflicting emotions cut off your curiosity? 

Yes.

I had to think about this, but yeah, I do notice when my feelings about a possible result cause me to not want to know the answer.

18 Do you, in every social interaction, ask what that person can teach you? 

No.

But I would like to develop this habit. I already do it sometimes, but not consistently.

19 Do you, in every situation where you receive feedback, treat it as potentially valuable? 

Yes.

I'm really good at taking criticism. Obviously, it's harder to actually apply it to produce a change in behaviour, but I'm always open to the idea of improving and I always try to enact change based on feedback.

Cheers,

Malcolm