I've convinced myself that "overfitting" and "underfitting" are not useful vocabulary words. In any applied setting, the only relevant test is whether the model solves the problem to the desired level of quality (which can be assessed by whatever criteria you please; pick the one that makes sense for your problem and organization). So instead of asking "Is my model overfitting?" we should be training students to ask the question "How precise does the model need to be? How can we achieve that?"

Instead of just pointing at a graph of epochs over time, or at hyperparameters, this question also implies the wider universe of possible reasons that a model could be doing poorly. Perhaps the data isn't very good, or there's not really a relationship between what you have and what you want to measure.

It also is the only question that matters. "Is it worth our time and money to deploy this model?" and the merit of the deployment is implied by the answer to the question "Does this do a good enough job solving our problem?"

I look forward to hearing your feedback.

@ReinstateMonica One virtue of the "*fitting" terminology is that it reminds people to consider the domain of application. The additional questions you ask--which are important--require (among other things) one to consider the possible ranges of future inputs (the "domain") to which the model will be applied.

@whuber ah! So an argument in favor of the -fitting language is that we care about "what comes next" (either in deployment or temporally or both). Whereas "Does the model do a good job" hides the contextual question because it doesn't say "Does the model do a good job in a future scenario?"