I've been thinking about probabilistic logic recently, and I want to hold a MIRIx workshop to discuss it on the weekend after next. If you are interested in participating, you can provide you availability at this doodle
. We will start with lunch at Kismet
in the UW plaza at 12, and then head over to MC 1085 on the UW campus to discuss math among whiteboards. (If you don't want lunch, you can meet us in MC 1085 afterward. I do recommend eating something before doing math though rather than being hungry all afternoon. There will be snacks though.)
I have a few papers for people to read to prepare for the workshop. I'm not sure how much of this people have the background to understand; I will try to shift the balance of discussing these papers vs new ideas depending on people's comfort. If you have any questions that can be better answered by a short post than discussion at the workshop, or if you have anything more general to say about the choice of papers or the workshop, then reply to this post.
The first link is A model of UDT with a concrete prior over logical statements
(Benja Fallenstein, 2012), which is actually a LW post rather than a paper. This describes a simple model that (using a large but finite amount of computation) gets the right answer on many decision-theoretic and at least a few game-theoretic problems. My suggested research direction would be, briefly, to investigate properties of this and similar models. This particular model is a bit of a response to earlier work based on proofs rather than probability measures. This previous work ran into what is called the problem of spurious proofs (1
are brief introductions, I can also find more sources motivating this agent if people want). In this and other formalisms, we have a partial solution to that problem, but we don't have a good reason to expect the agent's beliefs to be reasonable in general. In fact, it's easy to come up with simple examples where models analogous to Benja's behave in ways that we intuitively think of as irrational. I'm interested in seeing how far we can extend these counterexamples, possibly finding problems where Benja's original model, rather than analogous examples, fails.
The second link is Non-Omniscience, Probabilistic Inference, and Metamathematics
(Christiano, 2014). For agents with a halting oracle, this proposes an alternative prior that doesn't make the same simple mistakes that Benja's agent does. I am interested in whether there are other things that this agent does wrong, or whether there are theorems that we can prove about this agent that show that it does act the way we want it to. Some ideas along these lines are discussed in the conclusion. This paper spends a lot of time talking about how to make computable agents using the same ideas, but those parts require a lot more math background, and they are less relevant to the ideas that I want to discuss, so feel free to skip them if you're not interested. Some other parts are also less relevant to my current ideas, such as sections 4 and 5, which mostly focus on applications and motivation.
The last link is Logical Prior Probability
(Demski, 2012). This proposes a number of priors over logical theories, and discusses their properties and relationships. This paper requires some algorithmic information theory, which you can ask me about, skip, or read about, if you're unfamiliar with it. A few things are conjectured, but I think a lot of the conjectures are easy to prove or refute, so we can discuss that. This isn't discussed in the paper, but some of these priors satisfy interesting notions of optimality. It would be interesting to check whether Benja's or (more likely) Paul's agent, or minor modifications, are also optimal in the same way. If not, we can ask why, and investigate whether any departures from "optimality" are actually things that we want to avoid.
, but the important thing for this workshop is to get the main points of the three papers I linked above. Again, I find it hard to estimate people's comfort with this material, and I intend to calibrate that based on responses to this post and on how the discussion goes at the workshop.