@OleksandrR. Sorry for pinging you so boldfaced on an old, totally unrelated message. I would really like to have your opinion:
I'm running a model-fit with NMinimize. It's a constrained problem and everything is working fine but since our last comment-discussion I'm aware how slow simulated annealing is. I switched to NelderMead which runs faster: 31 s compared to 216s. For my (almost) 500 datasets this makes a difference of 28h compared to 4h.
I just tested one data-set but one drawback is that the quality seems to drop from 5micrometer to 13micrometer. Even if I raise MaxIterations from 100 to 200 it's not getting better.
When I decrease MaxIterations to 50 and use Automatic Method (it uses diff evolution not sim ann, too late here..) again, I get an an error of 11micrometer which is not really better than the NelderMead solution.
Do you have any further tips except from using the slow method and wait 1+ day for the results..
@halirutan Nelder-Mead is a quasi-local optimizer, so it depends on your problem and how well you can estimate the initial guesses whether it can be successful or not. Also, how many dimensions does your objective function have?
Differential evolution should give the best result but it usually requires tuning the parameters. The default values Mathematica uses are not universal; they are not even particularly good choices for most problems. But it's difficult to say what would be appropriate without knowing more about the problem.
Differential evolution is however very wasteful of objective function evaluations as we discussed, and NMinimize makes it even worse by not remembering the objective function value even within an iteration. It evaluates it at least twice at every point! So obviously you can gain a factor of 2 using memoization.
Well, unless the function is stochastic, of course, in which case memoization is usually a bad idea
@halirutan Oleksandr has probably covered much of it. Perhaps some of the information under Nelder-Mead here reference.wolfram.com/language/tutorial/… could be of use. E.g. post-processing with a local optimizer to improve the fit, or changing the simplex parameters a bit.
