Can the function 'lmdif' handle complex nonlinear equation systems?

[lutein]

Especially when the number of problems m is greater than that of variables n. I've tried some simple nonlinear equations and it runs very well, but once the problem becomes complex, results are not correct always. I adjusted the initial value of variables x and some parameters like factor, but it turned out useless.
So I wonder if there is any improvement of the function 'lmdif' or 'lmder' to deal with more complex problems? Or what I can modify to make it work better?

[devernay]

m >> n works very well in practice. The Levenberg-Marquardt algorithm was used a lot for camera calibration and bundle adjustment, where m=2 timesnumber_of_points times number_of_cameras and n=3 times number_of_points + 6 times number_of_cameras. With 100 point and 4 cameras, that gives m=800 and n=324.

The problem may come from the fact that the function you're trying to optimize has several local minima and the initialization is too far from the solution.

You may want to check how sensitive your problem is to the initial position, but trying on a case where you jknow what the solution is.
If it is too sensitive, then you can try one of the following:

• try several initial positions and keep the best solution
• work on a better method to find the initial solution (there are recent methods which use machine learning for this)
• use something else than L-M to optimize (genetic algorithms, stochastic optimization...)

[lutein]

I think the reason may be, there are too many sqrt() which can't be simplified in my function. Once I add some sqrt() then the optimize result would be incorrect. It's strange when I use a similar optimize algorithm in scipy, the results are pretty perfect, then I check the source code and find that it's also a LM algorithm.

[devernay]

are your residuals derivable? this is important that all residuals are C1 for LM. For example, you cannot optimize abs(x) by passing sqrt(abs(x)) to LM. Are you sure scipy uses LM and not another technique (simplex, powell...) to solve your problem?

[devernay]

which function from scipy are you using?