I don't think that this solution is any more rigorous than the one given in the question. Yes, for this type of problems one should go to the formal definitions, define the probability space and the $\sigma$-algebra which is being used and show that the event (1st player wins) is in the $\sigma$-algebra (plus, in this case, to show that they event "nobody wins" has probability 0). Both solutions have their merits.

@user8268 it all comes down to the definition of $p$. In my case, I am defining $p$ as $\sum p_n$. Not sure how else one might define it...

This is not an issue of convergence, though, like you make it seem. At worst, the issue is wether the event in question is in fact an event at all. But since we are in a discrete probability space (the samples are basically all the positive natural numbers (there's a winner after $n$ tries) and $\infty$ (no winner ever)), where all sets of samples are valid events. Any and all probabilities of valid events are guaranteed to exist.

@Vercassivelaunos Again, for me this is a matter of definition. I am defining $p$ as the sum of that series. In words, I am declaring that winning the game means winning in a finite number of moves.

And that's a fine definition. But the issue you are mentioning, namely wether that definition is sound, is a non-issue. In a discrete probability space, the probabilities of all samples already have a finite sum. The sum of any subset of those samples is then also guaranteed to have a finite sum. This is true for all discrete probability spaces and doesn't need to be proved all over again for every exercise.

If you want to be very rigorous for a problem like this, then the rigor should come in defining the elements of $\sigma$-algebra that corresponds to our desired outcome and characterizing its probability using the independence of the flips, law of total probability, etc.

@Vercassivelaunos I don't understand. A priori, I don't think it is obvious that a probability exists for this problem. It has to be defined somehow and, if one wants to use recursive methods, we need to know that the definition is compatible with those.

@user8268 Again, an appeal to the axioms is not a definition any more than you could define the sum of a series by declaring that it must satisfy the usual axioms of addition. Instead, one defines the sum (as the limit of the partial sums) and then argues that this definition satisfies the axioms. This matters if, say, one wants to look at methods of "summing" divergent series. Interesting stuff, but those definitions do tend to violate the standard rules of arithmetic.

I don't think it really has much to do with axioms. A probability exists if the event takes the form of a measureable subset of the sample space. The probability existing or not comes down to setting up our probability space

@whpowell96 Not sure what point you are making. My point is that the axioms come second to the definition. You first define the probability and then argue that your definition satisfies your axiom set. In this case, as I am defining it by a convergent series, I already know that the axioms are satisfied (just a matter of the usual arithmetic of absolutely convergent series).

"I don't think it's obvious that a probability exists for this problem". That's my point: It is obvious in discrete probability spaces as long as the probability space is sound - which you do seem to take as obvious, since you say that $p_n$ obviously exists. If the sum of probabilities of disjoint events does not converge, then their probabilities weren't valid probabilities in the first place.

@Vercassivelaunos Not following. Of course $p_n$ exists. It's a finite problem, so counting measure is defined. It's extending that notion to allow infinite paths that causes the conceptual problem. And you seem to be agreeing with me that convergence is what lets me extend the definition.

Tempted to delete my post, as I didn't intend for the comment section to get so long. Not the way the site is meant to work. I'll leave it up for now.

I guess in my mind, proceeding rigorously begins with: $\Omega$ is the set of all finite strings over the alphabet $\{H,T\}$, our event is $\omega = \{H,TTH,TTTH,\dots\}$. Now we apply independence, sigma-additivity, etc. to determine what $P(\omega)$ is using only the known probability information (fair independent coin flips). We will find that $P(\omega)$ is a geometric series, but that is not its definition.

@whpowell96 Then what is the definition? You appear to simply assert that it makes sense to "countably add" the probabilities attached to the terminating strings but of course not all countable sums are defined. To stress: I don't see any basis for simply asserting that there has to be a well defined probability theory in this case. It needs to be defined.

@whpowell96 yeah but realise, that 1/2 of all strings begin with T, i.e. A winning, it doesnt matter what follows after T, $TH, TTHH,. ..$ all of these cases A wins. of course the game does not proceed in those cases but it can be used to compute the probability if $n \to \infty$

@lulu if you want me to be precise & concrete: we take the space $\Omega=\{0,1\}^{\mathbb N}$ (of all infinite sequences of flips), then $A$ the smallest $\sigma$-algebra containing the events "the $n$-the flip was 1", and finally the unique probability measure for which these basic events are independent and of probability 1/2. The existence & uniqueness of that measure is nontrivial (it cannot be defined as sum of series) but it is a standard result.

@wolfgangmozart12 I don't see the point. Yes, that helps with intuition. As a practical matter, I could replace "winning in the unbounded game" with "winning in the first billion turns." And, yes, it would be surprising if the theory broke down in that gap. But surprising things happen sometimes.

@user8268 I claim that anything you wrote along those lines would be equivalent to the convergence of the series. Whatever you define, you need to show that it is compatible with countable addition. How is that not asserting that $p=\sum p_n$?

@wolfgangmozart12 I stated things like this so that we could consider our event as a countable set of finite strings. Each of these is easy to consider and working with a countable number of them is much easier than jumping straight to the set of all infinite binary strings

You dont need to jump to infinity: what is the probability of the first player winning by his $n$-th turn? it is the sum of geometric sequence. These are finitie strings $2^{n}$ and $1/2$ of them have H as a first character, $1/8$ as second et cetera. These are the cases the first player wins...

@lulu I was saying that the standard (=in any textbook and for any mathematician) definition of probability is via a probability measure and that with such a setup both your solution and the solution given in the question are correct.

@wolfgangmozart12 This is just repeating my point. The debate, to the extent there is one, is that some people would prefer to say that the reason the convergent sum gives the answer is that this follows from the standard axioms of probability. I object to that on the grounds that it still leaves the core probability undefined.

@user8268 And nobody is disagreeing with that. My definition literally constructs that measure (and uniqueness follows from the basic arithmetic of convergent series). I think you would prefer to deduce the existence of such a measure from some general principle, but at some point there must be a definition.

@lulu the disagreement is: you don't define/ construct the measure (you do it only for some sets, which doesn't imply the existence of a measure, and no, this is not just nitpicking) and so you cannot say that your solution is any more rigorous than the original one. Knowing that the measure exists, you can use its $\sigma$-additivity, which is what you do - and it is ok, but the original solution is equally ok.

@user8268 It may not be more rigorous but the OP is saying: "I don't know how to methodically produce such equations, and hence for more complicated scenarios I have a low level of certainty that they are correct, and", I would argue that using the sum of geometric series for this problem is a better methodical approach than the proposed equation, especially if you were to answer this problem with for instance 20 players. It is not difficult to derive a formula for X players from the geometric sequence.

@lulu I think my main issue is our starting point and the assumptions that come with it. If we start immediately by calculating the probability of a specific event like you do, the existence of a probability space should be assumed because that's where we get the probabilities $p_n$. In that case, convergence of any sum of disjoint probabilities is a given. If, however, the probability space needs to be constructed first, one would start (in the discrete case), with the probabilities of all possible samples and show that their sum is 1. From there, convergence of your sum is again a given.

Addendum: Considering my previous comment, my objection is not so much that you are wrong - you are not. But more so that the objection you give to the OP's rigorousness comes at a point in the argument where all the necessary steps to neutralize your objection have usually already been taken and can be assumed.

@Vercassivelaunos not following. The definition of $p_n$ is routine, just counting measure. the definition of $p$ is not routine as it, in principle, considers infinite collections. At some point, some definition of $p$ must be introduced. You can't simply assert that there must be some definition that lets me compute using $p$. Sure, you can invoke some general principle, and there's nothing wrong about that, but it just defers the issue. Phrased differently, if we had never considered the convergence of series, this new definition would give me a theory for that.

I'm sorry but this discussion is quite beside the point of the original question. OP is clearly asking for a more practical approach to the problem and a general solution which is very much provided by the geometric sequence approach. I suppose it's been misinterpreted what rigorous means in the original question, as well as the fact that the yielded result is correct and doesn't require simulation even for arbitrary number of players, which OP had to resort to ...