Anixx

@DavidGao There are no contradictions in the 1D case besides Hilbert's hotel, but as I said it can be mitigated by dropping translation-invariance regarding unbounded sets.

..and $A∩B$ is not a zero set.

@DavidGao we want our measure also be ordered in this way: $\mu(A\cup B)>\mu(A)$ iff $B$ is not an empty set.

@DavidGao a measure based on cardinal numbers would not be additive.

@DavidGao Hilbert's hotel prohibits a translation-invariant finitely-additive measure that is non-zero for any non-zero set. That's why we have to make it non-translation invariant for nonbounded sets.

@DavidGao in 1D case there is no paradox besides the Hilbert's hotel, which can be properly mitigated by making the measure non-translation-invariant regarding non-bounded sets.

@DavidGao I mean surreals belonging to $No(\omega_2)$, which includes countable surreals and continuum. Those are enough for an Euclidean space.

@DavidGao Hilbert's hotel paradox happens only with unbounded sets: a set can be made into a strict subset of itself by translation. With bounded sets this is impossible in 1D case.

@DavidGao but why should it evaluate to finite numbers? I prefer surreal numbers.

@DavidGao my point is, in 1D case due to Hilbert's hotel paradox we have to make our measure non-translation-invariant, and this is OK. In 2D case, we can fix the things by making the measure non-rotation invariant, but it seems too much. And in 3D case, only a theory of surreal integration will help...

@DavidGao why bounded sets should have finite measures?...

@DavidGao suppose, we want our measure to be non-null for any non-empty set. Then we get this paradox and need do something, like introduce non-measurable sets or make it non-rotation-invariant.

@DavidGao the point is it is null in your terminology only under Lebesgue measure. If we want to chose another measure, it would not necessarily be null. Particularly, if we want our measure to be non-null for any non-empty set.

@DavidGao if B is a strict subset of A, then A \ B cannot be null, no? As to your other comment, my point is, this paradox is already surprising in 2D case and without Axiom of Choice, if we want to have a finitely-additive (for bounded sets) and rotation-invariant measure. The unbounded case is already paradoxical in 1D case (Hilbert's hotel paradox).

@DavidGao I did not mention Lebesgue measure at all. Neither I meant it. Why are you talking about it?

@md2perpe thanks for pointing it, clarified. Irrational degrees, of course. Not radians.

Hello, guys! I want to attract attention to this old answer which I spotted only recently: mathoverflow.net/a/200866/10059 I think, it is THE answer to the posed question, and should be the chosen one. I also call to upvote it. Unfortunately, the author is inactive now.

@RonJohn so, if the AIs cannot modify own aim, they are not dangerous and will always remain aligned?

@JD I interpret ethics as everything concerning the rules of behavior, not only answers on the question "should?"

@Hokon this question is important for understanding of AI ethics.

@mudskipper Would an agent decide to manually change the goals if the changing environment made the goals inconsistent, meaningless or ambiguous? Or he will stick to them regardless and will try to find meaning and interpretation? Of meaningless goals would incaacitate the agent so that he would not have any desire to fix the things?

@mudskipper yes, but the agent may anticipate this and so, refuse the idea because it hampers the original goals. Or not? This is a question.

@mudskipper quite similar, yes, but this is a reverse. Can one ever come with a decision that is contrary to the current goals, anticipating that the decision will change the goals and as such will not be regretted?

@ScottRowe no. An AI can come with new ideas without changing its own goals. But how can he come with an idea to change the goals?

@mudskipper well, let's assume the agent can get experience, and in a changing environment, and otherwise completely free, including in decisions that can introduce randomness (to the extent he thinks it is desirable), in creating other AIs and clones, new models, as long as he thinks this benefits his stated aims.

@ScottRowe yes, I am asking about it, and the idea of a random though could be a workaround, but would not an agent with clear goals make efforts to eliminate random thoughts?... Or abide by them?...

@tkruse I meant he has access to any kind of own functionality, be it source code, training data or whatever.

@Hokon one can think about the original aims as of embedded moral.

Materialism is absolutely logically consistent.

@Feeds He proved that the future of a system where the observer is properly included is not probabilistically or deterministically predictable. Since there there are systems unpredictable by a deterministic theory (whatever theory), the conclusion is that the most complete physical description of the universe is non-deterministic (and non-Bayesian). This is mathematics.

@Feeds You said "Or it presupposes that our personality or consciousness already exists non-deterministically (somehow), in which case it's a trivial inference from something they already accept, and it just kicks the can down the road into why they think THOSE things are non-deterministic and why that would be meaningful." - this is simple, and this was shown by Thomas Breuer that no deterministic or probabilistic (Bayesian) theory can be universally valid.

@safesphere oh if you are at this, I should point that the very formation of the event horizon in finite time is impossible. So, if we do not just assume that a BH "just exists" with certain mass (that never fell in), then the picture is that the horizon never forms. No BH can be fully formed in finite time.

@safesphere "forever" means also after the BH is no more. Hardly Bob can survive if he is close enough to the BH though by the time the BH explodes. Yet, being destroyed by an explosion does not mean the information gets lost.

@D.Halsey I do not understand what you wrote.

@D.Halsey their short time corresponds to infinite time by the clock of the distant observer, which exceeds the time of the black hole evaporation. You are using pure GR outside its domain of applicability. The remote observer will allways see the falling observer outside the BH even after the BH ceases to exist. So, he will be able to meet with the falling observer again after the BH is no more (if he is not destroyed by the explosion).

@safesphere something reaching under the event horizon and the very black hole forming also violates the information paradox. As to the firewall, I did not say it does not exist. It is evident that if the BH explodes ahead, you are a subject of intense radiation that evaporates you. So, I believe firewall exists but it is out of the scope of this question.

@safesphere Classical GR is not applicable to the timespans, comparable to the BH evaporation time. From the OP question it seems, he is interested in what will happen in reality.

@rschwieb ah, I see. Thanks for the clarification.

Well, it is not isomorphic to the 2x2 matrices but does not match the description in the post either.

@rschwieb anyway, your construction does not fit the description, because either xy=0 or yx=0 I think, your construction is isomorphic to the 2x2 matrices.

Or maybe, not. Seems non-commutative.

@rschwieb But your construction should be commutative.

@rschwieb Okay, you are saying, it is split-complex plus two dual units.

@rschwieb let's see. There should be a matrix representation if this is correct, as for any finite-dimensional associative algebra.

@JoshuaTilley the only one non-commutative associative 4-dimensional algebra with nilpotents and idempotents are $2\times2$ matrices.

@JoshuaTilley where from do you know it is not commutative? Or it is just a postulate?

I think, either $xy=0$ or $yx=0$

@CaptainLama $xy$ is a nilpotent.

You should specify that $xy\ne 0$, otherwise, $\mathbb{R}^2$ satisfies athe multiplication table.

@toolforger, there are, Cesaro, Borel, Abel and any other will give 1/2. wolframalpha.com/… Also, the numerosity of moves up is 1/2 greater than the numerosity down.