estimation with analytic scores and hessian
Dear Friends, I just completed the first tests with estimation of DYNARE with analytic scores and hessian and everything seems to work. I have implemented it so far for stationary models both for univariate and multivariate KF. Before committing the changes, I would like to anticipate you some decisions I took: I particular I had to change the list of output of dsge_likelihood (and similarly for consistency of dsgevar_likelihood). We had discussed this a bit some time ago and I envisaged the possibility to keep the current list simply changing the type of output LIK = scalar for numerical derivs LIK= cell = {LIK, GRAD, HESS} Stephan in particular supported this last option. However, the standard form of objective functions in matlab is [fval, grad, hess] = func( ... ) and this form would allow, for example, a direct use of mode_compute=1 or 3. This means I have adapted all fval calls, e.g. in csminwel1, so that now the exit_flag is 4th output of dsge_likelihood. This will also allow to use Sims optimizer with analytic derivatives The same for mode_compute=5. By the way, this will also allow to eliminate the annoying dsge_likelihood_hh. I have also thought of a way to adapt mode_compute=5 for dsge-var as well. what do you think? best Marco -- Marco Ratto, Joint Research Centre The European Commission, TP 361, 21027 ISPRA(VA), ITALY Tel: +39 0332 78 5639 Fax: +39 0332 78 5733 marco.ratto@jrc.ec.europa.eu
Hi Marco, thanks a lot for this new development. I agree with your changes. All the best, Michel On 04/24/2012 06:12 PM, Marco Ratto wrote:
Dear Friends,
I just completed the first tests with estimation of DYNARE with analytic scores and hessian and everything seems to work. I have implemented it so far for stationary models both for univariate and multivariate KF.
Before committing the changes, I would like to anticipate you some decisions I took: I particular I had to change the list of output of dsge_likelihood (and similarly for consistency of dsgevar_likelihood).
We had discussed this a bit some time ago and I envisaged the possibility to keep the current list simply changing the type of output
LIK = scalar for numerical derivs
LIK= cell = {LIK, GRAD, HESS}
Stephan in particular supported this last option.
However, the standard form of objective functions in matlab is
[fval, grad, hess] = func( ... )
and this form would allow, for example, a direct use of mode_compute=1 or 3. This means I have adapted all fval calls, e.g. in csminwel1, so that now the exit_flag is 4th output of dsge_likelihood. This will also allow to use Sims optimizer with analytic derivatives
The same for mode_compute=5.
By the way, this will also allow to eliminate the annoying dsge_likelihood_hh. I have also thought of a way to adapt mode_compute=5 for dsge-var as well.
what do you think? best Marco
Hi Marco, On the basic standard output for optimizers in matlab, I think it is [X,FVAL,EXITFLAG,OUTPUT] = FUNC(...) at least for fminsearch. Other output arguments like GRAD and HESS come after OUTPUT when the optimizers allow for them. Maybe we should also use this to standardize the ouput arguments a user-defined optimizer should provide. In my opinion, the first two X and FVAL are the most important. I welcome the changes. Best, Junior On Tue, Apr 24, 2012 at 3:39 PM, Michel Juillard <michel.juillard@mjui.fr>wrote:
Hi Marco,
thanks a lot for this new development. I agree with your changes.
All the best,
Michel
On 04/24/2012 06:12 PM, Marco Ratto wrote:
Dear Friends,
I just completed the first tests with estimation of DYNARE with analytic scores and hessian and everything seems to work. I have implemented it so far for stationary models both for univariate and multivariate KF.
Before committing the changes, I would like to anticipate you some decisions I took: I particular I had to change the list of output of dsge_likelihood (and similarly for consistency of dsgevar_likelihood).
We had discussed this a bit some time ago and I envisaged the possibility to keep the current list simply changing the type of output
LIK = scalar for numerical derivs
LIK= cell = {LIK, GRAD, HESS}
Stephan in particular supported this last option.
However, the standard form of objective functions in matlab is
[fval, grad, hess] = func( ... )
and this form would allow, for example, a direct use of mode_compute=1 or 3. This means I have adapted all fval calls, e.g. in csminwel1, so that now the exit_flag is 4th output of dsge_likelihood. This will also allow to use Sims optimizer with analytic derivatives
The same for mode_compute=5.
By the way, this will also allow to eliminate the annoying dsge_likelihood_hh. I have also thought of a way to adapt mode_compute=5 for dsge-var as well.
what do you think? best Marco
______________________________**_________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-**bin/mailman/listinfo/dev<https://www.dynare.org/cgi-bin/mailman/listinfo/dev>
-- "You can never know everything", Lan said quietly, "and part of what you know is always wrong. Perhaps even the most important part. A portion of wisdom lies in knowing that. A portion of courage lies in going on anyway." Robert Jordan, Winter's Heart, Book IX of the Wheel of Time. We have not succeeded in answering all of our problems. The answers we have found only serve to raise a whole set of new issues. In some ways we are as confused as ever, but we believe we are confused on a higher level and about more important things. (cited in Øksendal, 1985)
HI Junior, indeed you are right about the standard output of optimizers and may be useful to conform to these as well. However, I was addressing the issue of the objective funcions to be fed to optimizers, so I think we can agree on that? thanks a lot Marco On 4/25/2012 2:07 AM, Junior Maih wrote:
Hi Marco, On the basic standard output for optimizers in matlab, I think it is [X,FVAL,EXITFLAG,OUTPUT] = FUNC(...) at least for fminsearch. Other output arguments like GRAD and HESS come after OUTPUT when the optimizers allow for them.
Maybe we should also use this to standardize the ouput arguments a user-defined optimizer should provide. In my opinion, the first two X and FVAL are the most important.
I welcome the changes.
Best, Junior
On Tue, Apr 24, 2012 at 3:39 PM, Michel Juillard <michel.juillard@mjui.fr <mailto:michel.juillard@mjui.fr>> wrote:
Hi Marco,
thanks a lot for this new development. I agree with your changes.
All the best,
Michel
On 04/24/2012 06:12 PM, Marco Ratto wrote:
Dear Friends,
I just completed the first tests with estimation of DYNARE with analytic scores and hessian and everything seems to work. I have implemented it so far for stationary models both for univariate and multivariate KF.
Before committing the changes, I would like to anticipate you some decisions I took: I particular I had to change the list of output of dsge_likelihood (and similarly for consistency of dsgevar_likelihood).
We had discussed this a bit some time ago and I envisaged the possibility to keep the current list simply changing the type of output
LIK = scalar for numerical derivs
LIK= cell = {LIK, GRAD, HESS}
Stephan in particular supported this last option.
However, the standard form of objective functions in matlab is
[fval, grad, hess] = func( ... )
and this form would allow, for example, a direct use of mode_compute=1 or 3. This means I have adapted all fval calls, e.g. in csminwel1, so that now the exit_flag is 4th output of dsge_likelihood. This will also allow to use Sims optimizer with analytic derivatives
The same for mode_compute=5.
By the way, this will also allow to eliminate the annoying dsge_likelihood_hh. I have also thought of a way to adapt mode_compute=5 for dsge-var as well.
what do you think? best Marco
_______________________________________________ Dev mailing list Dev@dynare.org <mailto:Dev@dynare.org> https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-- "You can never know everything", Lan said quietly, "and part of what you know is always wrong. Perhaps even the most important part. A portion of wisdom lies in knowing that. A portion of courage lies in going on anyway." Robert Jordan, Winter's Heart, Book IX of the Wheel of Time.
We have not succeeded in answering all of our problems. The answers we have found only serve to raise a whole set of new issues. In some ways we are as confused as ever, but we believe we are confused on a higher level and about more important things. (cited in Øksendal, 1985)
-- Marco Ratto, Joint Research Centre The European Commission, TP 361, 21027 ISPRA(VA), ITALY Tel: +39 0332 78 5639 Fax: +39 0332 78 5733 marco.ratto@jrc.ec.europa.eu
Hi Marco, Sorry I misunderstood what you meant. I now understand better and agree with the improvements. thanks, Junior On Thu, Apr 26, 2012 at 4:07 AM, Marco Ratto <marco.ratto@jrc.ec.europa.eu>wrote:
** HI Junior,
indeed you are right about the standard output of optimizers and may be useful to conform to these as well. However, I was addressing the issue of the objective funcions to be fed to optimizers, so I think we can agree on that?
thanks a lot Marco
On 4/25/2012 2:07 AM, Junior Maih wrote:
Hi Marco, On the basic standard output for optimizers in matlab, I think it is [X,FVAL,EXITFLAG,OUTPUT] = FUNC(...) at least for fminsearch. Other output arguments like GRAD and HESS come after OUTPUT when the optimizers allow for them.
Maybe we should also use this to standardize the ouput arguments a user-defined optimizer should provide. In my opinion, the first two X and FVAL are the most important.
I welcome the changes.
Best, Junior
On Tue, Apr 24, 2012 at 3:39 PM, Michel Juillard <michel.juillard@mjui.fr>wrote:
Hi Marco,
thanks a lot for this new development. I agree with your changes.
All the best,
Michel
On 04/24/2012 06:12 PM, Marco Ratto wrote:
Dear Friends,
I just completed the first tests with estimation of DYNARE with analytic scores and hessian and everything seems to work. I have implemented it so far for stationary models both for univariate and multivariate KF.
Before committing the changes, I would like to anticipate you some decisions I took: I particular I had to change the list of output of dsge_likelihood (and similarly for consistency of dsgevar_likelihood).
We had discussed this a bit some time ago and I envisaged the possibility to keep the current list simply changing the type of output
LIK = scalar for numerical derivs
LIK= cell = {LIK, GRAD, HESS}
Stephan in particular supported this last option.
However, the standard form of objective functions in matlab is
[fval, grad, hess] = func( ... )
and this form would allow, for example, a direct use of mode_compute=1 or 3. This means I have adapted all fval calls, e.g. in csminwel1, so that now the exit_flag is 4th output of dsge_likelihood. This will also allow to use Sims optimizer with analytic derivatives
The same for mode_compute=5.
By the way, this will also allow to eliminate the annoying dsge_likelihood_hh. I have also thought of a way to adapt mode_compute=5 for dsge-var as well.
what do you think? best Marco
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-- "You can never know everything", Lan said quietly, "and part of what you know is always wrong. Perhaps even the most important part. A portion of wisdom lies in knowing that. A portion of courage lies in going on anyway." Robert Jordan, Winter's Heart, Book IX of the Wheel of Time.
We have not succeeded in answering all of our problems. The answers we have found only serve to raise a whole set of new issues. In some ways we are as confused as ever, but we believe we are confused on a higher level and about more important things. (cited in Øksendal, 1985)
-- Marco Ratto, Joint Research Centre The European Commission, TP 361, 21027 ISPRA(VA), ITALY Tel: +39 0332 78 5639 Fax: +39 0332 78 5733marco.ratto@jrc.ec.europa.eu
-- "You can never know everything", Lan said quietly, "and part of what you know is always wrong. Perhaps even the most important part. A portion of wisdom lies in knowing that. A portion of courage lies in going on anyway." Robert Jordan, Winter's Heart, Book IX of the Wheel of Time. We have not succeeded in answering all of our problems. The answers we have found only serve to raise a whole set of new issues. In some ways we are as confused as ever, but we believe we are confused on a higher level and about more important things. (cited in Øksendal, 1985)
Hi there, would it be possible to add an option to estimation analytic_derivation default = 0 (already set in global_initialization) =1, triggers estimation with analytic gradient and the final hessian is also computed analytically. thanks a lot Marco On 4/24/2012 9:39 PM, Michel Juillard wrote:
Hi Marco,
thanks a lot for this new development. I agree with your changes.
All the best,
Michel
On 04/24/2012 06:12 PM, Marco Ratto wrote:
Dear Friends,
I just completed the first tests with estimation of DYNARE with analytic scores and hessian and everything seems to work. I have implemented it so far for stationary models both for univariate and multivariate KF.
Before committing the changes, I would like to anticipate you some decisions I took: I particular I had to change the list of output of dsge_likelihood (and similarly for consistency of dsgevar_likelihood).
We had discussed this a bit some time ago and I envisaged the possibility to keep the current list simply changing the type of output
LIK = scalar for numerical derivs
LIK= cell = {LIK, GRAD, HESS}
Stephan in particular supported this last option.
However, the standard form of objective functions in matlab is
[fval, grad, hess] = func( ... )
and this form would allow, for example, a direct use of mode_compute=1 or 3. This means I have adapted all fval calls, e.g. in csminwel1, so that now the exit_flag is 4th output of dsge_likelihood. This will also allow to use Sims optimizer with analytic derivatives
The same for mode_compute=5.
By the way, this will also allow to eliminate the annoying dsge_likelihood_hh. I have also thought of a way to adapt mode_compute=5 for dsge-var as well.
what do you think? best Marco
-- Marco Ratto, Joint Research Centre The European Commission, TP 361, 21027 ISPRA(VA), ITALY Tel: +39 0332 78 9217 Fax: +39 0332 78 5733 marco.ratto@jrc.ec.europa.eu
Marco Ratto <marco.ratto@jrc.ec.europa.eu> writes:
would it be possible to add an option to estimation
analytic_derivation
default = 0 (already set in global_initialization)
=1, triggers estimation with analytic gradient and the final hessian is also computed analytically.
Done. -- Sébastien Villemot Researcher in Economics & Debian Maintainer http://www.dynare.org/sebastien Phone: +33-1-40-77-84-04 - GPG Key: 4096R/381A7594
Dear all, To close ticket 59, I would require feedback on the treatment of complex likelihoods, see http://www.dynare.org/pipermail/dev/attachments/20120425/8000f828/attachment... To recap: currently complex likelihoods are assigned a penalty and continue estimation. They do not result in an error. 1. Hence, I would suggest to delete the text part of the error message referring to complex likelihoods. 2. On a related issue, the treatment of NaN-Likelihoods in linear and non-linear estimation is not consistent. In the former they results in an error, while in the latter they result in a penalty. Is this distinction on purpose? And if not, how should we treat this case in the future. Best, Johannes -------- Johannes Pfeifer Haußerstr. 29 72076 Tübingen Tel.: +49-(0)7071-6396184 Mobil: +49-(0)170-6936820 jpfeifer@gmx.de -----Ursprüngliche Nachricht----- Von: dev-bounces@dynare.org [mailto:dev-bounces@dynare.org] Im Auftrag von Sébastien Villemot Gesendet: Mittwoch, 6. Juni 2012 11:35 An: Marco Ratto Cc: List for Dynare developers Betreff: Re: [DynareDev] estimation with analytic scores and hessian Marco Ratto <marco.ratto@jrc.ec.europa.eu> writes:
would it be possible to add an option to estimation
analytic_derivation
default = 0 (already set in global_initialization)
=1, triggers estimation with analytic gradient and the final hessian is also computed analytically.
Done. -- Sébastien Villemot Researcher in Economics & Debian Maintainer http://www.dynare.org/sebastien Phone: +33-1-40-77-84-04 - GPG Key: 4096R/381A7594
1) The current implementation is buggy in dsge_likelihood.m and in non_linear_dsge_likelihood.m. Writing likelihood = penalty; DOESN'T penalize the likelihood because penalty is a base value to which it is necessary to ADD a positive amount proportional to the violation. 2) I don't understand the initialization penalty = xparam1 in non_linear_dsge_likelihood.m 3) for linear models, we make sure that the solution (dr.ghx and dr.ghu) are real. Complex likelihood can appear only if matrix F in the Kalman filter is not positive definite. I'm not sure that this problem would arise only for some value of the estimated parameters that we want to eliminate. I rather think it is the result of a problem with the model or the priors that we should reject with an error. But others may disagree. 4) For nonlinear models, maybe it makes sense to filter out parameters generating complex likelihood. Stéphane and Frédéric should tell their opinion. Best Michel On 06/06/2012 12:09 PM, Johannes Pfeifer wrote:
Dear all,
To close ticket 59, I would require feedback on the treatment of complex likelihoods, see http://www.dynare.org/pipermail/dev/attachments/20120425/8000f828/attachment...
To recap: currently complex likelihoods are assigned a penalty and continue estimation. They do not result in an error.
1. Hence, I would suggest to delete the text part of the error message referring to complex likelihoods.
2. On a related issue, the treatment of NaN-Likelihoods in linear and non-linear estimation is not consistent. In the former they results in an error, while in the latter they result in a penalty. Is this distinction on purpose? And if not, how should we treat this case in the future.
Best,
Johannes
-------- Johannes Pfeifer Haußerstr. 29 72076 Tübingen Tel.: +49-(0)7071-6396184 Mobil: +49-(0)170-6936820 jpfeifer@gmx.de
-----Ursprüngliche Nachricht----- Von: dev-bounces@dynare.org [mailto:dev-bounces@dynare.org] Im Auftrag von Sébastien Villemot Gesendet: Mittwoch, 6. Juni 2012 11:35 An: Marco Ratto Cc: List for Dynare developers Betreff: Re: [DynareDev] estimation with analytic scores and hessian
Marco Ratto<marco.ratto@jrc.ec.europa.eu> writes:
would it be possible to add an option to estimation
analytic_derivation
default = 0 (already set in global_initialization)
=1, triggers estimation with analytic gradient and the final hessian is also computed analytically. Done.
-- Sébastien Villemot Researcher in Economics& Debian Maintainer http://www.dynare.org/sebastien Phone: +33-1-40-77-84-04 - GPG Key: 4096R/381A7594
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
participants (5)
-
Johannes Pfeifer -
Junior Maih -
Marco Ratto -
Michel Juillard -
Sébastien Villemot