This is the write-up of my MATS project so far, as a mentee of Abram Demski . Those who would prefer a more formal/academic/mathematical write-up should look at the AAMAS submission here .

TL:DR, a formal framework for representing CDT, EDT and UDT on the majority of classical Newcomblike problems, plus some accompanying results about how CDT can be made to pay for ignorance.

I present my framework for hosting classical decision theories (Causal, Evidential and Updtateless /Functional decision theories) in classical Newcomblike problems. The framework is intended to be faithful (the outputs of the framework regarding what theories act in which ways in which problems agree with the consenus), fair (each theory's intuitions about "why this is rational behaviour" can be expressed as formal properties of the framework, the framework tries to avoid "begging the question" in favour or against any particular theory) and computable (given a formal description of any theory and any problem (at least those that the framework can express) there is a simple algorithm do determine what that theory would do in that problem. To the best of my knowledge, there is no other framework that fulfills these three properties while hosting CDT, EDT and UDT. We can broadly divide Newcomblike problems into two categories: read problems (the world varies with the agent by reading the agent, e.g Newcomb's Problem ) and write problems (the world varies with the agent by writing some portion of the agent, e.g Smoking Lesion ). My framework only handles read problems, rejecting write problems mostly as a consequence of giving agents perfect introspection.

The most significant finding from my framework thus far is that a calibrated [1] CDT agent will in some cases pay to receive reduced information, and objectively benefit for doing so. To the best of my knowledge this is an orginal result.

A Decision Space is comprised of the following elements: • 𝑆 is a finite set of states containing the “invalid” state 𝑠0. The state set is partitioned into start states and end states, . Since this framework is concerned only with single-step problems, this split encompasses all possible states. • 𝐴 is a finite set of actions. • Acc : 𝑆′ → 2𝐴 \ ∅ is the accessibility function, assigning to each start state its available actions. • 𝛾 : 𝑆′ × 𝐴 → 𝑆′′ is the outcome function, such that . • is an observability set that partitions 𝑆′, such that . It follows that we may write Acc(𝑜) without ambiguity. • 𝑈 : 𝑆′′ → R is a utility function.

𝐶 is a predictor mechanism, written as a tuple 𝐶 = (1−𝜀, 𝑜′, 𝐹 ). 1 − 𝜀 is the predictor’s reliability, 𝑜′ the predicted observation (the observation that the predictor "cares about"), and 𝐹 a mapping from the realised prediction to a distribution over start states, such that:

Given a decision space with observation set and action set , a decision policy for is a function such that:

Let be the set of all decision policies for .

An important property of predictors in Newcomblike problems is that they are decision-sensitive, meaning that the prediction is correlated with the agent's eventual realised action over any other property of the agent. Correspondingly, the predictor reads (with some random noise) the policy's action at the predicted observation o'.

Let be the set of all pairs where is a decision policy for , for all possible .

The predictor function is a map such that for with from and , is the distribution on :

By the construction of the preidctor mechanism and the predictor function, we ensure that any decision procedure encountering any (expressible-within-the-framework) problem has a greater-than-zero chance of being in any of the possible start states. This is mostly acheived by enforcing that the predictor can be arbitrarily close to perfectly accurate, but cannot actually be perfectly accurate.

This choice avoids phenomena such as an agent's action at a particular observation rendering that observation logically inconsistent with the agent (and thus making the choice of action difficult to analyse) or an agent making a certain observation and concluding with logical certainty how they are about to act. I am reasonably condifent that truly perfect prediction is probably impossible in practice, so I do not consider this a high price to pay.

Given , a nonempty set is a decision problem for .

There is some debate over whether it is legitimate to consider only some sub-branches of a decision space when comparing two theories on that space. For maximum flexibility I allow this, though I do also formalise why we might want to say that some such comparisons are illegitimate (mostly if the two theories reach those branches at very different rates).

Agents believe that the problem functions exactly as described by . They are not required to hold any beliefs about the explicit probability of themselves performing certain actions.

Note that the agent only has access to the predictor through its description in , which gives only the parameters of the predictor (accuracy, predicted observation, response to prediction) and describes nothing of how the predictor works or what it will actually predict. The predictor, on the other hand, has direct access to the decision policy of the agent and its output on . This prevents circular dependency or the agent trying to "game" the predictor. It also preserves the decision-sensitivity of the predictor.

Since agents do not (necessarily) have any beliefs about the action they will take, nor any beliefs about what exact action will be predicted, the agent cannot straightforwardly evaluate the probability of being in any particular start state or the expected utility of the various accessible actions. An epistemology is the device that fixes this gap. Different epistemologies emit different data shapes as required for different decision procedures.

The following list of epistemologies is not considered exhaustive, but is what is required to formalise the theories I want to study.

Prediction-Skeptic epistemology refuses to accept and/or make use of the decision-sensitivity of the predictor. In particular, Prediction-Skeptic epistemology holds that it is appropriate to determine about your beliefs regarding what the prediction is likely to be, and then to reason about what action you should take without ever updating your beliefs about the prediction .

For simplicity, I model Prediction-Skeptic epistemology as always having uniform uncertainty over what the prediction could be [2] , though a more complete model would allow for more variety in Prediction-Skeptic beliefs.

Its output is a vector over representing the (presumed) expected utility of each accessible action. Its generative law is the uniform mixture ; conditioning on gives the within-cell posterior . For each a \in \Acc(o) emit

Fully Transparent epistemology accepts decision-sensitive predictions (in particular, the notion that you cannot separate reasoning from what the prediction was from reasoning about what you will do ), and believes that an agent can meaningfully and reliably reason about how they would act given observations they did not make (this is especially important for when the agent is not at the predicted observation ).

Its output at is a matrix over which represents "the expected value of if I knew I was about to do ". EDT cares only about the diagonal but CDT needs the whole matrix.

For each supposed resolved action , the credal state is , with start-state law ; conditioning on gives . For each actual act a', emit

It's output at is a vector over which represents "the expected value of , given that I know that I am the kind of agent who would do at ".

Query the paired theory for its resolved -action . The generative law is , and for each emit

Partially Transparent epistemology also respects decision-sensitivity, but is not confident in its ability to reason about how it would act at observations not made. At it outputs the same as Fully Transparent, off it outputs the same as Prediction-Skeptic.

Does not query which observation was made, simply outputs the expected utility (over as a whole ) of each possible decision policy . The output decision policies do read the observation, so it is never in danger of (for example) selecting an action that is not accessible.

Outputs a vector over . For each policy , emit:

where is the unique observation cell containing .

Each Decision Theory must be formalised in this rule as a Decision Procedure which is the combination of an Epistemology (above) with a simple Decision Rule (below).

Once again, the list below is not claimed to be exhaustive. Due to the differently-shaped outputs, different decision rules can only accept certain epistemologies, as denoted in the square brackets.

Pick the action with the highest expexted value according to the epistemology.

At , call ratifiable iff its diagonal entry is its own row-max:

Equivalently, an action is ratifiable iff, conditional on knowing you are about to perform , no other accessible act beats . This ties in to the ratifiable causalist argument that while you may use your imminent action as evidence about the world, it is not rational to use your action as a lever on the world via your beliefs .

if you are about to perform a non-ratifiable action , then in the instant before you perform , your beliefs about the world will be conditioned on the fact that you are about to perform . Holding this state fixed (a requirement of CDT, since the state is causally independent of your action) , there is some action that would be better than (this follows from the definition of ratifiability). Therefore, your action is not an optimal action according to your current beliefs about the state of the world, therefore (according to CDT) your act is not rational.

If no action is ratifiable, the rule returns [4] .

Off , , using the off-cell vector supplied by the paired epistemology (self-knowledge under Fully Transparent; uniform random under Partially Transparent). This also returns if there is no ratifiable action at .

Unlike CDT, EDT claims that the appropriate way to calculate the expected utility of an action is "if I knew I was going to perform , what would be my expected utility?" Thus EDT uses the same matrix as NR-CDT, but only cares about the diagonal. This follows the EDT intuition that "the best action is the action that maximises my expectation of utility, conditional on performing that action".