Even the electrostatic potential isn't "observable" in your sense, yet we routinely measure it. The trick is to define a reference potential. Similarly, there is nothing preventing you from defining a gauge and measuring the vector potential in that gauge.

@JohnDoty sure you can "measure" in that sense (measure magnetic field, choose a gauge convention and calculate the corresponding vector potential). But the point here is the resulting values are arbitrary, they have no physical relevance, just as the value of electric potential is arbitrary and has no physical relevance. Only derivatives or integrals of these potentials do have definite values independent of the convention.

@JohnDoty we measure potential difference.

@JánLalinský Pretty much everything we measure is arbitrary in your sense. What longitude are you at? What time is it? How much power is your computer consuming? All dependent on arbitrary conventions. Does that mean they have no physical relevance? I believe you're confusing the mathematical formalism with physics.

@ProfRob And we only measure time difference, yet we can agree that today is August 7.

@ProfRob, one has to be careful here, because voltmeters actually measure differences in chemical potential, not electric potential. See for example, a diode in thermal equilibrium which has a built in E field and potential difference, but the chemical potential is uniform so a voltmeter will see zero.

@ProfRob Many things that we measure don't have well-defined zero points.

@JohnDoty no, those other things have definite agreed upon definition. In SI, vector potential has only definite agreed upon unit - T.m - but the gauge is not definite, there are several common definitions, and in fact there is infinity of them.

@JánLalinský No, they often don't have definite agreed on zero points. For, example, for time we have TT, TAI, GPS time and others with different zero points. And as ProfRob points out, there are time zones and a date line. The number of possibilities is infinite here, too.

@ProfRob Note that civil time is based on UTC, and for that you can't even reliably take the time difference between two events and expect a clock to agree.

with regards the comment by @ProfRob: If you have defined a zero of potential, then you are measuring potential difference

@jim And if you have defined a gauge, then you may measure the vector potential.

In ac induction the B-field is never observed, instead the time rate of its surface integral is observed. This is no different from observing the time rate of the line integral of the vector potential. There is a reason why even Maxwell thought $\mathbf A$ is primary over $\mathbf B$, and this is why he denoted them in straight alphabetical order...

@JohnDoty Have you ever actually found it useful to measure the vector potential in the way you're describing?

@Andrew I've found it useful to calculate the vector potential in antenna theory. And as hyportnex's excellent answer implies, any time you're using a transformer with a high $\mu$ core, you're effectively using a changing vector potential to drive the output. Which pretty much is all the time if you're using electronics.

@JohnDoty Yes, with time, different people use different points of origin, so stating number of seconds without further context is not informative. So you found another example of the problem that I pointed out with vector potential - just units and value are not enough to have physical significance. We need elaboration on the definition - in your example with time, the null event assigned time 0, and in the case of vector potential, we need the exact definition of the vector potential function.

@JohnDoty When you choose some exact definition of vector potential to do calculations, this choice has no physical significance. The same results can be arrived using different exact definition. Vector potential is an auxiliary mathematical function whose definition is chosen to suit human preferences/calculational skills, not the physical situation. That's the point.

@JánLalinský That's just like choosing the center of mass frame to do calculations. "The same results can be arrived using different exact definition."

@JohnDoty there is only one center of mass, and it is physically significant - for example, center of mass can't move without external force. There is infinity of vector potential functions and the choice is purely arbitrary. We have experiments to determine the CM, but there is none to determine the value of $\mathbf A$.

@JánLalinský It is routine in physics to measure and use quantities that require "elaboration on the definition". Time, location, velocity, energy, momentum, voltage, ... There's nothing special about the vector potential here.

@JohnDoty I don't argue it is special, in fact I've confirmed here electric potential and time have the same deficiency that different values can describe the same situation. In contrast to quantities such as force, electric field, and so on, which are definite so different values mean different physical state.

@JohnDoty nobody argues that electric potential value or time value has definite meaning, since everybody knows they relate to a zero reference. So why do it for vector potential? It requires a gauge fixing condition.

I never argued that it has a definite meaning. I only argued that a lack of a definite meaning in your sense has no impact on its status as a useful abstraction for modeling reality. $A$ is no more or less "real" than $B$.

@JohnDoty "useful abstraction for modeling reality" and "A is no more or less real than B" are very different goalposts. I agree with the first one.

@JánLalinský Do you somehow imagine that the $B$ that a magnetometer registers is the same thing as the $B$ in your whiteboard models? The first is real, but it isn't the same thing as the abstraction.

@JohnDoty there is a difference between $B$ in real situation and the concept of $B$ in my thoughts, but I fail to see the relevance. My point is it makes sense to build and use a magnetometer to measure $B$, but it makes no sense to build and use a vector-potential-meter to measure $A$.

@JánLalinský I've been known to wind transformers.

@JohnDoty transformers are not measuring instruments that could show value of vector potential.

@JánLalinský See hyportnex's excellent answer.

@JohnDoty I did, see my comments there.

@JohnDoty OK, fair enough. Of course I agree it's useful to compute the vector potential in radiation problems. And I also agree in specific problems the vector potential has a nice interpretation in some special gauge. However, it's also true that there is an unphysical part to $A$ in the sense that you can change $A$ by a gauge transformation and not affect any observable quantities. This fact is super useful as a practical check on calculations, and not appreciating the relationship between different gauges led to much confusion about how to interpret results, historically.

Sometimes being precise on the math actually does make it possible to be more clear about the physics and isn't just math for math's sake :)

@Andrew There are plenty of cases where unstated arbitrary choices can get you in trouble. Even for scalar potential, confusion about what "ground" is occurs commonly in real life. And indeed, it isn't just math for math's sake if you're sane about it. But part of being sane about it is to recognize that all the math is unreal. Mathematical objects exist only in human minds.

Yes, A is unreal, but so is B. For modeling, a sane physicist uses whichever gets the result. On the bench, you'll use B, but recognize that the thing a magnetometer measures is not the mathematical idealization.