Trust-region nonlinear solver
Hi, I just added a new nonlinear solver for the steady command: it is a trust-region solver, available under solve_algo=4 (it replaces the old value of that option, which was a Newton without the correction for singular Jacobian; I had added that variant a long time ago, and it was neither really useful nor theoretically grounded). The implementation of the solver is essentially the fsolve.m from Octave, that I have adapted for MATLAB compatibility and that I have also slightly simplified. I am wondering whether we may want to use this new solver as the default, since trust-region algorithms are nowadays considered to be the state of the art for solving systems of nonlinear equations. Said otherwise, should we modify the default to be solve_algo=4 instead of solve_algo=2 ? Since I have not thoroughly tested the new solver, a first step would probably to test it on various relevant examples. Best, -- Sébastien Villemot Researcher in Economics Dynare developer http://www.dynare.org/sebastien
Hi, I think that we can change the default and put a note on the Forum asking for return of experiments with the new algorithm. I don't have a problem that would be a good case, but we should try to get some. Best Michel Sébastien Villemot writes:
Hi,
I just added a new nonlinear solver for the steady command: it is a trust-region solver, available under solve_algo=4 (it replaces the old value of that option, which was a Newton without the correction for singular Jacobian; I had added that variant a long time ago, and it was neither really useful nor theoretically grounded).
The implementation of the solver is essentially the fsolve.m from Octave, that I have adapted for MATLAB compatibility and that I have also slightly simplified.
I am wondering whether we may want to use this new solver as the default, since trust-region algorithms are nowadays considered to be the state of the art for solving systems of nonlinear equations. Said otherwise, should we modify the default to be solve_algo=4 instead of solve_algo=2 ?
Since I have not thoroughly tested the new solver, a first step would probably to test it on various relevant examples.
Best,
-- Michel Juillard
Le mercredi 05 février 2014 à 16:43 +0100, Michel Juillard a écrit :
I think that we can change the default and put a note on the Forum asking for return of experiments with the new algorithm. I don't have a problem that would be a good case, but we should try to get some.
I re-ran the testsuite with the trust region as default and the only issue is with tests/expectations/expectation.mod. The initial values are far from the solution (they are actually at mid-point between zero and the solution), and the Jacobian is singular at that initial point. Curiously the trust region fails with that starting point, but succeeds after a small perturbation of the initial point that makes the Jacobian non-singular. I am quite surprised by this result, because I thought the point of trust region was precisely to avoid that kind of situation. Maybe this is a very pathological case. However note that this does not mean that the trust region performs worse than the newton. By construction, all the examples in the testsuite have been made to work with the newton. Considering the above, do you still think I should make the switch? As I said, the problematic MOD file can easily be fixed. -- Sébastien Villemot Researcher in Economics Dynare developer http://www.dynare.org/sebastien
I think we should definitely make the switch. After a two months of playing around with it I find that many mod-files encountered in the forum profit from it. The trust-region particularly performs better when people estimate models but only have initvals from calibration. Am 21.02.2014 16:03, schrieb Sébastien Villemot:
Le mercredi 05 février 2014 à 16:43 +0100, Michel Juillard a écrit :
I think that we can change the default and put a note on the Forum asking for return of experiments with the new algorithm. I don't have a problem that would be a good case, but we should try to get some. I re-ran the testsuite with the trust region as default and the only issue is with tests/expectations/expectation.mod. The initial values are far from the solution (they are actually at mid-point between zero and the solution), and the Jacobian is singular at that initial point. Curiously the trust region fails with that starting point, but succeeds after a small perturbation of the initial point that makes the Jacobian non-singular.
I am quite surprised by this result, because I thought the point of trust region was precisely to avoid that kind of situation. Maybe this is a very pathological case.
However note that this does not mean that the trust region performs worse than the newton. By construction, all the examples in the testsuite have been made to work with the newton.
Considering the above, do you still think I should make the switch? As I said, the problematic MOD file can easily be fixed.
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
Le mercredi 05 mars 2014 à 14:34 +0100, Johannes Pfeifer a écrit :
I think we should definitely make the switch. After a two months of playing around with it I find that many mod-files encountered in the forum profit from it. The trust-region particularly performs better when people estimate models but only have initvals from calibration.
Thanks for your feedback. I have therefore changed the default to the trust-region solver.
Am 21.02.2014 16:03, schrieb Sébastien Villemot:
Le mercredi 05 février 2014 à 16:43 +0100, Michel Juillard a écrit :
I think that we can change the default and put a note on the Forum asking for return of experiments with the new algorithm. I don't have a problem that would be a good case, but we should try to get some. I re-ran the testsuite with the trust region as default and the only issue is with tests/expectations/expectation.mod. The initial values are far from the solution (they are actually at mid-point between zero and the solution), and the Jacobian is singular at that initial point. Curiously the trust region fails with that starting point, but succeeds after a small perturbation of the initial point that makes the Jacobian non-singular.
I am quite surprised by this result, because I thought the point of trust region was precisely to avoid that kind of situation. Maybe this is a very pathological case.
However note that this does not mean that the trust region performs worse than the newton. By construction, all the examples in the testsuite have been made to work with the newton.
Considering the above, do you still think I should make the switch? As I said, the problematic MOD file can easily be fixed.
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-- Sébastien Villemot Researcher in Economics Dynare developer http://www.dynare.org/sebastien
HI Sebastien, I agree with this proposal. thank you Marco On 2/4/2014 6:03 PM, Sébastien Villemot wrote:
Hi,
I just added a new nonlinear solver for the steady command: it is a trust-region solver, available under solve_algo=4 (it replaces the old value of that option, which was a Newton without the correction for singular Jacobian; I had added that variant a long time ago, and it was neither really useful nor theoretically grounded).
The implementation of the solver is essentially the fsolve.m from Octave, that I have adapted for MATLAB compatibility and that I have also slightly simplified.
I am wondering whether we may want to use this new solver as the default, since trust-region algorithms are nowadays considered to be the state of the art for solving systems of nonlinear equations. Said otherwise, should we modify the default to be solve_algo=4 instead of solve_algo=2 ?
Since I have not thoroughly tested the new solver, a first step would probably to test it on various relevant examples.
Best,
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-- Marco Ratto, Joint Research Centre The European Commission, TP 361, 21027 ISPRA(VA), ITALY Tel: +39 0332 78 9217 Fax: +39 0332 78 5752 marco.ratto@jrc.ec.europa.eu
participants (4)
-
Johannes Pfeifer -
Marco Ratto -
Michel Juillard -
Sébastien Villemot