"Suppose these are truly fair, truly random dice." - Wait a minute. You are smuggling a very strong assumption into your thought experiment here, which is precisely the whole point of my question. First of all, examples involving dice and similar objects are problematic for the reasons I recently added to the OP (take a look). Secondly, how can truly random dice be fair? That seems question-begging, which is exactly what my question is asking in the first place. How can something truly random be forced to follow a uniform (i.e., "fair") distribution?

@keshlam "if the dice aren't fair, you will get a different probability distribution" - If the dice in your thought experiment are obeying rigid body dynamics, the thought experiment is then off-topic (see the recent edit to the OP to understand why).

@keshlam I guess you mean "no more improbable"? Yes, all of that exactly.

@user80226 The thought experiment doesn't change if the dice aren't fair, only the numbers (perhaps 7 could be five or sixteen times as likely as 12, with unfair dice). Either way, it is the numbers you get on the two dice, and the rule for combining those into one outcome, which determines the result - not some kind of hidden state remembered by the system of two dice which is somehow separate to the individual dice. And if no "keeping track" is required for the system of two dice to obey a probability distribution, why should any "keeping track" be required for the individual dice?

@kaya3 You are once again talking about dice. Dice obey rigid body dynamics. That's determinism, not randomness.

@user80226 It doesn't matter where your random inputs come from. If you don't believe in dice, add the radioactive decay times of two unstable nuclei, or whatever else you prefer. You are picking at things which are not relevant to the point.

I think the part you are hung up on is the idea that probability distributions must be manifested in physical objects, such as dice. The answer is that they don't need to be, because "random processes" are not the same thing as "probability distributions". As the last part of my answer states, "It also doesn't have to be physically possible to take independent samples from a distribution."

When you don't abstract away the usual physical details, it may well be the case that repeated rolls of a real, non-ideal, physical die are not independent or identically distributed. Nonetheless, that doesn't refute the concept of a probability distribution, because a probability distribution is an abstract mathematical object, not something which exists in reality. Just as the number 17 exists in the ontology of mathematics, so does the uniform discrete distribution on six elements. How can we know a physical die exhibits that distribution? Only by statistical tests and inductive reasoning.

But even if a real, non-ideal, physical die does not roll independent, identically distributed outcomes, there is still a probability distribution associated with each separate roll of that die. It's just that the distributions for each separate roll might be different, and we don't necessarily know what those distributions are, and it might not be physically possible to sample any individual one of those distributions more than once (if doing so somehow changes the physical state of the die, or the universe in which the die is rolled). But they are distributions nonetheless.

@kaya3 Your answer proposes a deterministic protocol/function F(X), where X is some noisy input. If X is deterministic, the whole thing is deterministic, and so it's not relevant to my question. Dice following deterministic mechanical rules would be an example of a deterministic X that is not relevant to my question. If X is truly random, then F(X) would be a deterministic transformation of truly random input. However, I don't see how F(X) would itself be random following a probability distribution unless X itself already follows a probability distribution.

But if you assume that, then that shifts the problem to now having to explain why X follows a probability distribution in the first place. That's why I don't see how your answer can solve the problem. It only shifts the problem to the inputs. Why do the inputs follow a probability distribution in the first place?

@user80226 The point is to show that distributions don't need anything "kept track of" in order to be followed. A system of two dice obeys a distribution which neither die individually obeys, neither die is responsible for or capable of "keeping track" of what that distribution is supposed to produce in the long run, and a human adding the result of the two dice in their head doesn't magically create some other physical entity which "keeps track" of the dice together to ensure the distribution is obeyed in the long run. So a distribution is followed without any "keeping track".

Then, given that a distribution can be obeyed without anything keeping track for that distribution, why should we suppose it is necessary that anything keeps track for any other distribution? We have already shown that it is not necessary for this particular distribution, therefore you cannot claim it is necessary for distributions in general.

@kaya3 But that's question begging, because you haven't explained yet how your inputs are able to be (1) truly random and (2) followers of a probability distribution in the first place without keeping track of information. Assuming that for your inputs in order to explain how something else doesn't need that is question-begging with respect to the inputs.

It is not question begging, you are just reading my words as if they are an argument for a different conclusion than what I have stated. It doesn't matter how the individual dice are able to obey a distribution; so long as you accept that the distribution of the sum of the two dice is different to the distributions of the individual dice, then there is a new distribution with no physical entity keeping track of that distribution. Even if you believe the individual dice "keep track" of their own distributions, what physical entity keeps track of the distribution of their sum?

If it helps, imagine I roll a rainbow assortment of dice, and then decide to sum the blue and green ones only. Is there a physical entity which enforces the distribution of the sum of those two dice particularly? Is there another physical entity enforcing the distribution of the sum of the red and purple dice, and so on for every pair of dice? Are there yet more entities keeping track of all the individual distributions of sums of three dice, or differences between each pair, or so on?

@kaya3 There are two options for your inputs: (1) your inputs are deterministic or (2) your inputs are random (well, technically there is a whole debate on whether free will is a valid third option: philosophy.stackexchange.com/q/18424/80226). Option #1 boils down to a whole deterministic mechanism, a deterministic machine evolving over time. My question is not about deterministic machines.

Option #2 would assume that there are random inputs following a distribution. I cannot grant that assumption because that's what you are supposed to show it is possible in the first place. Otherwise it is circular.

This is regardless of whether those dice do keep track of anything in order to enforce a distribution on their long-term outcomes. Because in the experiment, I only rolled them once. But also the conclusion of this argument is only that there are distributions (such as the sums and differences) which don't correspond with any physical entity that could keep track of them.

Your question doesn't ask "how do we know there is anything random in the universe?" so it's not circular to suppose that some sources of randomness exist

You asked about how random things in the universe are able to obey a distribution over the long term.

@kaya3 I can grant something random, but I cannot grant something random following a distribution.

Well, sorry, but if something is random (or even if it is not), then it follows a distribution, as that is mathematically what it means for something to be random

The word "random" has no formal meaning except that

But your question was not how we can know that there is a distribution, you asked about how we know it obeys that distribution in the long term over many samples.

In my experiment, I only rolled the dice once each.

Although, even if you doubt that randomness exists in the universe, the mathematical theory of probability still applies to situations of imperfect information, for example people can still play Poker in a deterministic universe and make bets which they can determine to be profitable in the long-run, as long as it's assumed that neither player knows what the unseen cards will be

If you don't believe that the dice are fair or that they maintain their distribution in the long term, simply take the distribution for one roll and label the probabilities of each outcome p_1, p_2, p_3, p_4, p_5, p_6, then q_1...q_6 for the second die. Add p_7 and q_7 for the probabilities that the dice will respectively fall without a clear result, collapse into a black hole, or any other outcome than a number from 1 to 6.

Disregard all notions that these probabilities apply to any future rolls beyond the one roll I am about to make. Suppose that if I were to roll them a second time (which I will not do), the probabilities would be wildly different.

If you believe the dice are deterministic such that the outcome "3 and 5" is preordained, then these distributions still exist, it's just that p_3 = 1 and q_5 = 1 and the rest are 0.

If you believe the dice are deterministic but you don't know which outcome is preordained, then these numbers represent your subjective judgement of how likely each outcome is, or your best estimate according to a simulation, or whatever.

Whatever you believe, these numbers exist, even if nobody knows or agrees what they actually equal. And those numbers are a probability distribution.

Then the numbers p1q1, p1q2 + p2q1, p1q3 + p2q2 + p3q1, and so on, are another probability distribution. And there is no physical entity corresponding to that distribution.

So it follows that a probability distribution can be associated with a physical outcome but not correspond to a physical entity.

@kaya3 A deterministic process has a fixed outcome given fixed inputs. The only way for the deterministic process to empirically behave according to a non-trivial probability distribution over many runs is that the inputs are not kept fixed but modified in such a way that the outcomes of the whole deterministic process also change. It's always possible to manipulate the inputs adversarially in order to make the distribution fail to emerge.

You are fundamentally misunderstanding what a probability distribution is

A discrete probability distribution is literally just an assignment of numbers to outcomes such that the total of the numbers equals 1.

The individual numbers also have to be between 0 and 1.

@kaya3 But anyways, my whole question is about making sense of the assumption that you can have truly random phenomena following a probability distribution. You say that I'm forced to assume that by definition. I was checking articles such as this one: plato.stanford.edu/entries/chance-randomness/#RandWithChan and I don't see that said definition is necessarily granted, but anyways, feel free to replace "random" with "non-deterministic".

Probability distributions are mathematical objects, to assert that they exist is no more than to assert that numbers like 0.5 exist

Real-world outcomes can be associated with probability distributions in exactly the same way that they can be associated with numbers. If I have five apples, then I have associated something in the real world with a mathematical object, the number 5

Likewise if one of those apples is poisoned, and I reach blindly for one to eat, I can associate the number 0.2 with the outcome that I get poisoned.

@kaya3 I understand the concept of probability distribution, but given a function F(X), you cannot assert a priori that F(X) follows a distribution Y unless you make assumptions about X first. If F is deterministic, an adversarial agent can manipulate X in order to make F output whatever they want, without following any particular distribution.

If you need X to be uniformly distributed, and if X is deterministic, then whatever deterministic process is producing X would need to be such that it outputs X uniformly empirically. I can make sense of that if X comes from a chaotic process that is very sensitive to initial conditions (coin flips are like that). But that's deterministic.

@kaya3 What I cannot understand is how something non-deterministic (i.e., "random", although you are taking issue with that word) can follow a probability distribution.

That statement is simply false; of course I can assert that F(X) follows a distribution Y, if I make no assertions about what Y actually is.

I would need to know something about X's distribution to know more about Y's distribution, but it is unassailable that X and Y have distributions. Simply, X and Y represent random variables, and random variables by definition have distributions.

If you think that statement is at all controversial then you are taking those words to mean something they do not mean

@kaya3 Suppose that Y exists. Then imagine an adversarial agent that chooses X such that the actual distribution fails to be Y, but ends up being Y' instead. Contradiction.

That is like saying, suppose f(x) is a number y. Now imagine an adversarial agent chooses x to be a different number, now the output is no longer y. Contradiction

You have just disproved that numbers exist

You can make it simpler. Suppose x is a number. Now suppose an adversarial agent chooses x to be a different number instead. Now it is no longer x. Contradiction

Of course, since you defined Y to be F(X), then Y is still F(X), whatever X is

@kaya3 "Simply, X and Y represent random variables, and random variables by definition have distributions" - This keeps begging the question. Let's forget about the word "random", let's use the word non-deterministic instead. Why would something non-deterministic necessarily follow a probability distribution?

Why would a collection of apples necessarily follow a number?

Suppose you said there are five apples, and an adversarial agent eats one of them. Now there aren't five any more, proving that the collection of apples does not follow a number.

If you accept that there was a number of apples before and a different number of apples afterwards, then you can just as well accept that there was a distribution before and a different distribution afterwards.

That's what my answer is about, the fact that there can be a different distribution associated with each step of a random process, and in fact that is the definition of a random process

It is not begging the question to say that X and Y represent random variables, any more than it is to say x and y represent numbers. If you doubt that random variables exist, then what did you mean X and Y to stand for? Of course, if you don't doubt that numbers exist, then it is hard to doubt that random variables do

I think you are stuck on the idea of probabilities representing "long-term" rates of non-deterministic outcomes over repeated trials, but there is no need to interpret probabilities that way. Things can have probabilities even if you only ever sample them once. And deterministic things can also have probabilities, firstly because 0 and 1 are probabilities, secondly because of imperfect information

@kaya3 Well, technically you can always affirm the trivial distribution "at each time step t, P(F(X_t)=Y_t) = 1 and P(everything else) = 0", which is a more convoluted way of saying that "there is a 100% probability that what will happen will happen", but that's not very useful and interesting and informative. Normally, you want to know the probability distribution of a phenomenon in order to make informed decisions about its expected values.

You cannot do that if the phenomenon doesn't follow any consistent pattern at all (like an adversarial agent messing up all your predictions).

You can still say that there is a distribution which you know nothing about.

Just as I can say there is an unknown number of cookies in this jar. There is still a number even though I don't know what it is

That number might be 0.

Probabilities are not very good models for adversarial situations where the adversary makes decisions based on strategies which you decide by assigning probabilities to the adversary's actions, and there are a number of paradoxes based on that kind of thing. But that's really just saying that self-referential definitions often lead to paradoxes.

But without self-reference (which usually requires the adversary to be omniscient, or telepathic, or be able to predict the future), you can still assign a probability distribution to a deterministic situation when you don't know what the determined outcome will be. You can even associate a probability distribution when the outcome is already determined but you haven't learned it yet.

Like the case of five apples with one poisoned. It might be the case that you always choose the nearest apple and the furthest one is poisoned. But without me knowing those two facts, nothing stops me from assigning the number 0.2 to the outcome that you take the poisoned one and 0.8 to the outcome that you don't.

And it is meaningless to talk about the long-run distribution of outcomes, because this whole scenario is a setup and nobody is ever going to put us in a room with five apples again.

@kaya3 The only way for you to affirm in the present moment that F(X) follows a distribution Y and be right, is to have access to an oracle with omniscience that can see the infinite future and collect all the statistics about F(X), and condense that information into a distribution Y and tell you "yes, there is a distribution Y that F(X) will follow empirically".

The number 0.2 doesn't need to be interpreted as representing anything about repeated trials. It could be interpreted as meaning, if I were forced to offer bets on the outcome to other people who have the same knowledge as me, then I'd offer odds of 1 in 5.

No, you are confusing the statement "F(X) follows a distribution Y" with "F(X) follows a distribution Y and I know exactly what Y is"

Forget about the future, forget about repeated trials, probabilities don't have to mean that and usually don't

Anyway, I am going to eat

Enjoy your day.