@halirutan hello

@OleksandrR. Hi.

I have turned my algorithm upside down and now, even if I use standard values for NMinimiz I'm down to 17 seconds (from 280).

Question:

Sounds like good progress!

I have from a former run of NMinimize pretty good initial parameters for my fit. I want to use them as initial agent points for DE. Is it now better to use larger scale factor to make the agents move around more?

Because the parameters calculated with NelderMead describe my data pretty well and are very narrow in the parameter space.

@halirutan hard to say. Reasoning about DE is often very difficult for any particular problem. Its behavior is very opaque and it often behaves unexpectedly. However, I would say that a scale factor of 0.8-0.9 would not be out of the ordinary. Much greater than 1 is normally counterproductive. The crossover probability is normally a lot more important than the scale factor, depending on how big of a parameter space you plan to explore.

Small crossover probability (=large Mma's crossover probability) favors local search. The opposite changes more parameters simultaneously, so it is better for finding new parts of the search space.

@OleksandrR. So a higher crossover will produce a more agile agent, because more of the new positions is used.

Exactly.

@OleksandrR. I really worked out how the algorithm works because I needed a better understanding.

The reason I ask is the following:

By the way, if it can be useful to you, you're welcome/encouraged to try my Python differential evolution. It has no constraints so you would have to come up with your own scheme for that, but in general it is more versatile and powerful than Mma's implementation.

I have seen that sometimes (not often) DE doesn't find a good solution and I could show (meaning, I saw it in many runs) that when I use good initial points, DE works far better.

@OleksandrR. I would, but I need to make some progress. So tonight I will definitely start fit for all my datasets.. this damn paper needs to be published.

The initial points are all chosen to be in the region {0,1} by default, so this will produce a rather badly directed search for most problems

@OleksandrR. Really? I thought it will use my constrains.

Sorry, yes. In the case of constrained problems it will start with feasible points only.

@OleksandrR. OK, this is good to know.

@OleksandrR. I have one last question: Is there a superior method for a global constrained problem where I can provide (exact) gradients of my target function?

Anyway, yf you specify the initial points yourself you can certainly save a lot of time. With sufficiently large F and CR you will probably find the same solution no matter where you start, but this is at the expense of many function evaluations. DE was not really intended to minimize this -- it is a "brute force at all costs" type of algorithm.

Not for NMinimize. All its methods are direct search (gradient free). And anyway, if you follow the gradient, won't you end up with a local search anyway? You could do this with FindMinimum.

BTW "RandomSearch" is basically "call FindMinimum at lots of points". If a quasi-local search is good enough for you but you need a different constraint handling implementation than in the interior point method.

@OleksandrR. That's why I ask. Because everything I used so far with gradients was local. But it kind of feels that there must be a superior method if I have so much information about my target function (and not only a black box).

Some methods use line search or some form of interpolation to move in potentially productive parts of the parameter space. This is usually retrofitted on top of other methods. The thing is, if you have a function with deceptive local minima, that approach will not help you, and it might just as often lead to premature convergence.

@OleksandrR. OK, then thank you very much for all the details!

If you have good starting points you can try them with FindMinimum and see what you get by following the gradient. The thing is, FindMinimum is so fast, that usually it won't hurt you to try.

@OleksandrR. But some kind of local search is done by NMinimize too. Do you know the possible settings for the "PostProcess" option?

PostProcess may be KKT, None, or FindMinimum

AFAIK

@OleksandrR. And the choice for Automatic is what?

This means that when the global search has converged, the result is passed to FindMinimum to be polished to high precision.

Don't know. You'd have to read the NMinimize code or turn on the diagnostics to find that out.

@OleksandrR. OK... I guess I'd just use FindMinimum with the answer from NMinimize to see if a further improvement is possible after all.

If so then you probably want to turn off PostProcess, otherwise it obscures what result NMinimize has really found. Sometimes NMinimize finds a really bad result that FindMinimum ends up doing all the work to polish

@OleksandrR. yes.

If you don't mind me asking, what about the fovea are you trying to model? There must be more to it than your function I assume

"CrossProbability" -> 0.05, "ScalingFactor" -> 0.95 is rather good for deceptive problems BTW

@OleksandrR. It's a simple model of the fovea centralis.

Actually the biggest problem I have with NMinimize is that its convergence detection is too eager. The criteria seem to me more complex and not as effective relative to my preferred one, i.e. sustained decrease of the function value over a number of successive iterations

You are doing some sort of simulation for each function evaluation?

@OleksandrR. Look here.

@OleksandrR. We have so-called optical coherence tomography scans of patients and from these volumes you can extract important retinal layers.

I was just trying to get your paper. The Elsevier website is just as horrible as it ever was!

@OleksandrR. Are you on research gate?

(which is even worse than facebook)

No, I'm not. I always considered ResearchGate as a kind of seedy place with copyright issues, but maybe I misjudged it

@OleksandrR. Researcher just don't give a sh** about copyright I guess. Someone (not me) uploaded the full elsevier version of the paper there.

@OleksandrR. But if you're really keen to look at it just drop me a mail and I send it.

I got it now, thanks. I just find the website very slow and awkward to use. But it's been like this forever

@OleksandrR. As long as community.wolfram is the slowest of my webpages, everything is under control.

I think you have nothing to worry about in that case!

I don't even visit Wolfram Community regularly. As well as being infuriatingly slow, I find that the payoff is quite often not worth the wait, because people post all sorts of semi-relevant discussion and half formed thoughts. The signal to noise is not as good as SE.

@OleksandrR. Me neither. I look from time to time, but when I answered a post the last time, I wrote about an hour and then hit "send". I was logged out in the mean time and everything was lost.

That's extremely horrible. Enough to put one off it forever.

@halirutan okay, I should go to bed. I have a 10.00 meeting tomorrow. Goodnight!

@OleksandrR. Thanks again. Good night.

@Rojo Why 137 points for a star?

Uhuu! Mathematica is Back in Tiobe index

Now in the last 50th from top 100

That's me woken up this morning - trying to figure out why R was faster than C++. Turns out if you compare two different things, R is indeed faster :-) mathematica.stackexchange.com/questions/75303/…

@bobthechemist I wanted to make the player as happy as possible, but after a careful analysis, I concluded 138 was too much.

@Rojo Dynamic@dude should probably be in more people's notebooks.

@MichaelHale Haha

Hahaha, I earned Tumbleweed badge…

@xslittlegrass 我就知道你是同胞

@Rojo Very good point. Are you sharing the code? I have a pile of papers to grade and am running out of ways to procrastinate.

@bobthechemist hey are you around?

@bobthechemist I don't mind, but it's not a neat notebook, I was trying some stuff to see if it would work.

@VitaliyKaurov Yep, at the moment.

@bobthechemist Cleaning it up a bit

for my own sake

@Rojo Cool - I've been toying around with some M-based games and would like to see your approach (whenever you find it fit enough to share).

@bobthechemist In a couple of mins

@bobthechemist @Rojo I did a simple test with game loops in Mathematica about a year ago. I started a simple clone of Spacewar. It was going great until I realized CurrentValue only works on modifier keys. So to turn the ship you have to repeatedly tap the left or right arrow keys, you can't hold it down and have it turn smoothly. Shift does the accelerating though and that works with CurrentValue so it's nice.

t0 = AbsoluteTime[]; dt = 0; {width, height} = {600,
  400}; stars = {White, PointSize@Tiny,
    Point@Transpose@{RandomReal[#, #3],
       RandomReal[#2, #3]}} & @@ {width, height, 30}; ship1 := {White,
   Line[4 {{{0, -3}, {1, -1}, {1, 1}, {0, 3}, {-1,
       1}, {-1, -1}, {0, -3}}, {{1, -1}, {2, -2}, {2, -3}, {1/
       2, -2}}, {{-1, -1}, {-2, -2}, {-2, -3}, {-(1/2), -2}},
     If[CurrentValue[
       "ShiftKey"], {{1/
        2, -2}, {0, -4}, {-(1/2), -2}}, {}]}]}; {position1, velocity1,
   angle1} = {2./3 {width, height}, {0, 0}, 0}; EventHandler[

I could use my gamepad I guess, but a lot of people don't have those and even fewer have two of them.

@MichaelHale Exactly, it is offputting to have such limitted current values

@MichaelHale Nice game :)

@MichaelHale But I just happened to be able to rotate it smoothly keeping the arrow keys pressed

Not with shift at the same time however

@Rojo Thanks. Wish I could take credit for the design.

@MichaelHale Don't mind that, they are just proofs of concept

If one worked nicely, then we can start being creative

@Rojo It depends on your OS. I'm on Windows and holding the arrow key will stutter and then go smoothly like if you hold down a letter key in a text field. I think on Mac though you have to keep pressing the arrow key.

I also just didn't do anything creative. Just trying to implement an idea and see how it went in MMA

@Rojo They are good tests. Several parts of writing that code made Mathematica really stand out. I love how I could set the ship graphic to a delayed definition and then put conditional logic for drawing the ship in different states in line with the original shape.

Or how short the basic game loop is with a Dynamic wrapping some code that just updates the time.

But like you said, the input limitations kind of damper things. I was hoping they would prioritize that because games are what got so many people to install Flash and maybe they could help CDF get more adoption.

@xzczd 新年快乐！哈哈

@MichaelHale Heh, that's fun. I've been thinking about using a controller for that very reason.

@bobthechemist I'm actually controlling the mouse with a controller from my bed right now. Then I hold down the left stick to disable mouse/keyboard functions when I start playing an actual game.

@bobthe (my boss called but I'll finish cleaning it up as soon as I can)

@Rojo Cool - I definitely don't want you to get fired.

@bobthechemist Ahh

I was getting confused wasting time because it didn't work

after some commenting

but its surely because I am on 10.0.0 now

and ReplacePart didn't work an associations. What a waste of time. I'll send it now assuming it works, and you let me know

or someone

(or me when I get home)

I did a little refactoring too so I might have broken some detail

dropbox.com/s/kzgsxq6nbkfi860/dude.nb?dl=0