I have a question, and I feel like it may be enough of a "high-level" question to be suitable for MO instead of MSE, but I'm nevertheless not sure it would really be a good question for this site.

The question is:

What first-order statements about the integers are provable?

Or, more precisely: what is the most appropriate definition of the word "provable", when talking about first-order statements?

I can think of two candidate answers off the top of my head, but neither one is satisfying.

One is that such a statement is "provable" if and only if it's a consequence of Peano arithmetic. But that's clearly an incorrect definition, because Goodstein's theorem is definitely "provable", but it's not a consequence of Peano arithmetic.

Another is that such a statement is "provable" if and only if it's a consequence of ZFC. My problem with that is that I don't really trust ZFC. A lot of mathematicians think that the axioms of ZFC are all "true", but I don't. (I don't think they're false, either.) Most mathematicians probably think that ZFC is arithmetically sound, but it seems hard to come up with a justification for that belief.