against UD+ASSA, part 2
Wei Dai
argument against UD, then against the general possibility of any single
objective measure.
1. There is an infinite number of universal Turing machines, so there
is an infinite number of UD. If we want to use one UD as an objective
measure, there has to be a universal Turing machine that is somehow uniquely
suitable for this purpose. Why that UTM and not some other? We don't even
know what that justification might look like.
2. Computation is just a small subset of math. I knew this was the case,
having learned about oracle machines in my theory of computation class. But
I didn't realize just how small a subset until I read _Theory of Recursive
Functions and Effective Computability_, by Hartley Rogers. Given that there
is so much mathematical structure outside of computation, why should they
not exist? How can we be *sure* that they don't exist? If we are not *sure*,
then we have to take the possibility of their existence into account when
making decisions, in which case we still need a measure in which they have
non-zero measures.
3. At this point I started looking for another measure that can replace UD.
I came up with what I called "set theoretic universal measure", where the
measure of a set is inversely related to the length of its description in a
formal set theory. Set theory covers a lot more math, but otherwise we still
have the same problems. Which formal set theory do we use? And how can we be
sure that all structures that can possibly exist possible can be formalized
as sets? (An example of something that can't would be a device that can
decide the truth value of any set theoretic statement.)
4. Besides the lack of good candidates, the demise of ASSA means we don't
need an objective measure anymore. There is no longer an issue of sampling,
so we don't need an objective measure to sample from. The thought experiment
in part 1 of "against UD+ASSA" points out that in general, it's not the
measure of one's observer-moment that matters, but the measures of the
outcomes that are causally related to one's decisions. Those measures
can be interpreted as indications of how much one cares about the outcomes,
and therefore can be subjective.
So where does this chain of thought lead us? I think UD+ASSA, while flawed,
can serve as a kind of stepping stone towards a more general rationality.
Somehow UD+ASSA is more intuitively appealing, whereas truly generalized
rationality looks very alien to us. I'm not sure any of us can really
practice the latter, even if we can accept it philosophically. But perhaps
our descendents
can. One danger I see with UD+ASSA is we'll program it into an AI, and the
AI will be forever stuck with the idea that non-computable phenomenon can't
exist,
no matter what evidence it might observe.
Bruno Marchal
Le 26-sept.-07, à 14:40, Wei Dai a écrit :
> So where does this chain of thought lead us? I think UD+ASSA, while
> flawed,
> can serve as a kind of stepping stone towards a more general
> rationality.
I would suggest to use the Hal Finney "UDist" term, instead of UD for
helping the new people on the list to not confuse the UD = Universal
Dovetailer, with the UDist = Universal Distribution. But this is a
detail. Roughly speaking I agree with you.
> Somehow UD+ASSA is more intuitively appealing, whereas truly
> generalized
> rationality looks very alien to us. I'm not sure any of us can really
> practice the latter, even if we can accept it philosophically. But
> perhaps
> our descendents
> can.
This seems rather mysterious for me.
> One danger I see with UD+ASSA is we'll program it into an AI, and the
> AI will be forever stuck with the idea that non-computable phenomenon
> can't
> exist,
> no matter what evidence it might observe.
And this is still more mysterious. An "AI" as simple as the already
existing "Peano Arithmetic" has enough cognitive abilities to
understand that non-computable "phenomenon" have to exist (relatively
to itself).
I will say more when I come back on Church thesis.
Bruno