Imagine an arbitrary function f and a neural net with a single hidden layer but infinite neurons in that layer (and use ReLU or max(0,x) as activation).
Now think of f as broken up into small linear segments that approximate it locally.
If the NN has infinite neurons in that single layer, then it can approximate each of the little line segments.
This is very clear since the ReLU makes all the rest 0 besides the new weight, which corrects the previous ones.
Is this correct? (at least directionally)