MIRIxUW workshop - Feb 7th or 8th

[Sam Eisenstat]

Hi everyone,

I've been thinking about probabilistic logic recently, and I want to hold a MIRIx workshop to discuss it on the weekend after next. If you are interested in participating, you can provide you availability at this doodle. We will start with lunch at Kismet in the UW plaza at 12, and then head over to MC 1085 on the UW campus to discuss math among whiteboards. (If you don't want lunch, you can meet us in MC 1085 afterward. I do recommend eating something before doing math though rather than being hungry all afternoon. There will be snacks though.)

I have a few papers for people to read to prepare for the workshop. I'm not sure how much of this people have the background to understand; I will try to shift the balance of discussing these papers vs new ideas depending on people's comfort. If you have any questions that can be better answered by a short post than discussion at the workshop, or if you have anything more general to say about the choice of papers or the workshop, then reply to this post.

The first link is A model of UDT with a concrete prior over logical statements (Benja Fallenstein, 2012), which is actually a LW post rather than a paper. This describes a simple model that (using a large but finite amount of computation) gets the right answer on many decision-theoretic and at least a few game-theoretic problems. My suggested research direction would be, briefly, to investigate properties of this and similar models. This particular model is a bit of a response to earlier work based on proofs rather than probability measures. This previous work ran into what is called the problem of spurious proofs (1 and 2 are brief introductions, I can also find more sources motivating this agent if people want). In this and other formalisms, we have a partial solution to that problem, but we don't have a good reason to expect the agent's beliefs to be reasonable in general. In fact, it's easy to come up with simple examples where models analogous to Benja's behave in ways that we intuitively think of as irrational. I'm interested in seeing how far we can extend these counterexamples, possibly finding problems where Benja's original model, rather than analogous examples, fails.

The second link is Non-Omniscience, Probabilistic Inference, and Metamathematics (Christiano, 2014). For agents with a halting oracle, this proposes an alternative prior that doesn't make the same simple mistakes that Benja's agent does. I am interested in whether there are other things that this agent does wrong, or whether there are theorems that we can prove about this agent that show that it does act the way we want it to. Some ideas along these lines are discussed in the conclusion. This paper spends a lot of time talking about how to make computable agents using the same ideas, but those parts require a lot more math background, and they are less relevant to the ideas that I want to discuss, so feel free to skip them if you're not interested. Some other parts are also less relevant to my current ideas, such as sections 4 and 5, which mostly focus on applications and motivation.

The last link is Logical Prior Probability (Demski, 2012). This proposes a number of priors over logical theories, and discusses their properties and relationships. This paper requires some algorithmic information theory, which you can ask me about, skip, or read about, if you're unfamiliar with it. A few things are conjectured, but I think a lot of the conjectures are easy to prove or refute, so we can discuss that. This isn't discussed in the paper, but some of these priors satisfy interesting notions of optimality. It would be interesting to check whether Benja's or (more likely) Paul's agent, or minor modifications, are also optimal in the same way. If not, we can ask why, and investigate whether any departures from "optimality" are actually things that we want to avoid.

Further resources are available on this LW post, but the important thing for this workshop is to get the main points of the three papers I linked above. Again, I find it hard to estimate people's comfort with this material, and I intend to calibrate that based on responses to this post and on how the discussion goes at the workshop.

See you this weekend,

Sam

[Sandy Maguire]

Hey Sam,

Thanks for setting this up! I'd love to get involved but that weekend
is the iron ring ceremony; 100% won't be able to make it =(

Sandy

> --
> You received this message because you are subscribed to the Google Groups "lesswrong waterloo" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to lesswrongwater...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Sam Eisenstat]

Hi Sandy,

I'm sorry to hear that. I'm going to be away for reading week, so the only other options are to push it up to this weekend or push it back to late February. I would consider having it this weekend, but I'm not sure if that gives people enough time to prepare. Are you fine with it being that soon? Is everyone else? Later in February seems strictly worse than just having more workshops; the workshops will be small and informal, and I will post notes from them, so it hopefully won't be too hard to catch up. It is hard to find a weekend that works for everyone.

Best,
Sam

[William Saunders]

I wouldn't have time to prepare and wouldn't come if it was pushed up to
this weekend (but feel free to have it then if that works best for
everyone else).

--
William Saunders
wil...@williamsaunders.net