The current source text is the original text and can be switched to the Chinese translation.
SaveShare
报道配图 · 来源:Alignment Forum
Overview
Linkpost for https://arxiv.org/html/2611.14454v1 Our PIBBSS fellow Damini Kusum wrote an excellent paper on prediction with reflective oracle Solomonoff Induction (rOSI). Reflective oracles are best known for their game-theoretic applications, where they are often dismissed because "why would all players choose the same reflective oracle?" It turns out that reflective oracles can also be used to construct a version of Solomonoff induction (called rOSI) with very nice sequence prediction properties - roughly speaking, it can learn to extract computable structure from (partly) uncomputable sequences. In fact, its properties are quite reminiscent of logical induction, but this paper is much shorter and more accessible. In this context, there is only one "player" to choose an oracle, which is just a fixed-point (though you can sort of consider the set of hypotheses entertained by that player as subplayers who share the oracle). I think this paper is the clearest exposition of a very neat but previously obscure idea in agent foundations: self-reflection can overcome unrealizability. Discuss
Linkpost for https://arxiv.org/html/2611.14454v1
Our PIBBSS fellow Damini Kusum wrote an excellent paper on prediction with reflective oracle Solomonoff Induction (rOSI).
Reflective oracles are best known for their game-theoretic applications, where they are often dismissed because "why would all players choose the same reflective oracle?"
It turns out that reflective oracles can also be used to construct a version of Solomonoff induction (called rOSI) with very nice sequence prediction properties - roughly speaking, it can learn to extract computable structure from (partly) uncomputable sequences. In fact, its properties are quite reminiscent of logical induction, but this paper is much shorter and more accessible. In this context, there is only one "player" to choose an oracle, which is just a fixed-point (though you can sort of consider the set of hypotheses entertained by that player as subplayers who share the oracle).
I think this paper is the clearest exposition of a very neat but previously obscure idea in agent foundations: self-reflection can overcome unrealizability.
READ THE ORIGINAL
Read the original for the full context.
The content of this article comes from Alignment Forum,Copyright belongs to the original author and publisher.
Tl;dr : Intuitions we have about human organisations, like how we think of making deals or how we think about the power or values of an organisation, can be productively applied to understand groups of AIs taking coordinated action in the world. An important consideration in understanding the type of AI organisations we expect is which notion of self AIs will adopt.
Business or Consumer? Everyone you sell your product or service to is a customer. However, the question is: do you treat them as a business or a consumer? You might think that’s a pointless distinction since your goal is simply to sell your product, right? Wrong. That question determines your product’s market fit.
The Ban RSI protest at Anthropic’s IPO was encouraging: well-attended, covered by media, high-energy. It was focused on a single message – that we should ban AI systems that are capable of building more capable AI systems. It was a promising outcome for a protest planned only a couple of days in advance. But there was something weird about it, something I couldn’t put my finger on while it was going on.
Loading comments…