@hxd perhaps you could include the information somewhere at that post, for the next person. I recall a time (many decades past) when I asked myself the same question. But not having a StackExchange at the time, I answered it as well.

--- --- ---

I'm reminded of Fisher's comment about consulting the statistician after the experiment is finished

@hxd1011: I also work in the industry and daily interact with people from non-technical backgrounds. Black-boxes do work beautifully in certain fields like CV and NLP that have reasonably well-defined structures, they work because we build them so they recognise the features. It is not a coincide that if you look at NN zoos you have NNs for faces, translations, emotions and so on. Try customer analytics; (almost?) nothing is there because a happy customer (ie. human behaviour) is much fuzzier.

The problem with black-boxes especially if we deal with non-technical people is that if they break, we are lame ducks in terms of criticism. We said $X$ and $Y$ happened, they are unhappy and we have no excuse. We then sit around trying to think what happened, trying to deduce why a model gave a particular output or not. Don't get me wrong, I routinely use black-boxes; I rave on how amazing GAMMs are but out-of-the-box gradient boosters do amazingly well and I use them all the time. :)

@usεr11852 yes, I agree, and in fact, I am a very conservative person and like the model with high bias low variance. I do against blackbox model without totally understand how the data are collected and data distribution.

in that way it is no way to tell if the inputs are changed in production.

I recently wants to open my mind a little bit more to use some of these black box thing.

the data changes too fast, e.g., technology platform change.

so, the model will be trashed in few years anyway.

@Glen_b I couldn't even find a question in that post. We ought to start by elucidating that. A person who only has data, but has no objective and no knowledge of methods to evaluate those data, is a person with nothing at all. We can't even attempt Fisher's post mortem until we have some clue about what the experiment was for in the first place!