Hi Marco,
I am trying to factorize the mode-finding code to make it reusable in other parts of Dynare. But I am having a hard time with newrat in master. The reason is that newrat is the only optimizer that endogenously changes other options used inside of the objective function. I would really like to get rid of this behavior as it makes the generic passing of input arguments to objective functions impossible. One always needs to make sure that the updated options is passed.
To be specific, the problem in master derives from the fact that the analytic Hessian requires options_.kalman_algo to be either 2 or 4. If this is not the case, newrat sets one of these values. My proposal to remedy this would be to only allow using newrat without options_.analytic_derivation when kalman_algo=2 or kalman_algo=4. That is, when not using analytical derivatives the user must at least specify a Kalman algo that allows for analytical derivatives. Such a check would make changing the options redundant. Or do you envision cases where people want an analytical Hessian based univariate filters while wanting to compute the likelihood in mode-finding based numerical derivatives derived from the multivariate Kalman filter.
An additional issue that popped up: currently, newrat with analytical_derivation seems to be broken for most Kalman algorithms. Thus, I cannot test the features/changes I propose. I created a pull request with a unit test for these issues. I attach the files you need to trace out the errors.
Best,
Johannes
Hi Johannes,
I have to think about this. Not sure how much quickly, however. Analytic derivation should only work with multivariate stationary filter. All other filters should be broken (no analytic derivs in all other filters). best Marco
On 12/4/2014 6:20 PM, Johannes Pfeifer wrote:
Hi Marco,
I am trying to factorize the mode-finding code to make it reusable in other parts of Dynare. But I am having a hard time with newrat in master. The reason is that newrat is the only optimizer that endogenously changes other options used inside of the objective function. I would really like to get rid of this behavior as it makes the generic passing of input arguments to objective functions impossible. One always needs to make sure that the updated options is passed.
To be specific, the problem in master derives from the fact that the analytic Hessian requires options_.kalman_algo to be either 2 or 4. If this is not the case, newrat sets one of these values. My proposal to remedy this would be to only allow using newrat without options_.analytic_derivation when kalman_algo=2 or kalman_algo=4. That is, when not using analytical derivatives the user must at least specify a Kalman algo that allows for analytical derivatives. Such a check would make changing the options redundant. Or do you envision cases where people want an analytical Hessian based univariate filters while wanting to compute the likelihood in mode-finding based numerical derivatives derived from the multivariate Kalman filter.
An additional issue that popped up: currently, newrat with analytical_derivation seems to be broken for most Kalman algorithms. Thus, I cannot test the features/changes I propose. I created a pull request with a unit test for these issues. I attach the files you need to trace out the errors.
Best,
Johannes
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
Hi Marco,
Thanks for the quick reply. That is fine with me. The compatibility was nowhere documented and judging from the code you had already done precautions for these other algorithms. The reason I send you the unit test is that for some algorithms it break down with seemingly easily fixable problem like unconformable matrices where a transpose seem to be missing. But if they are not implemented at all, we should probably just document this and filter out these cases. If I understand you correctly, only kalman_algo=1 is supported. I would then add a check for this.
For now I would appreciate if you could have a look at the newrat-implementation in https://github.com/DynareTeam/dynare/pull/817. We could then work from there.
Best,
Johannes
--
Johannes Pfeifer
Friesenwall 104
50672 Köln
Tel.: +49-221-29873852
Mobil.: +49-170-6936820
Germany
Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Marco Ratto Gesendet: Freitag, 5. Dezember 2014 08:32 An: List for Dynare developers Betreff: Re: [DynareDev] newrat
Hi Johannes,
I have to think about this. Not sure how much quickly, however. Analytic derivation should only work with multivariate stationary filter. All other filters should be broken (no analytic derivs in all other filters). best Marco
On 12/4/2014 6:20 PM, Johannes Pfeifer wrote:
Hi Marco,
I am trying to factorize the mode-finding code to make it reusable in other parts of Dynare. But I am having a hard time with newrat in master. The reason is that newrat is the only optimizer that endogenously changes other options used inside of the objective function. I would really like to get rid of this behavior as it makes the generic passing of input arguments to objective functions impossible. One always needs to make sure that the updated options is passed.
To be specific, the problem in master derives from the fact that the analytic Hessian requires options_.kalman_algo to be either 2 or 4. If this is not the case, newrat sets one of these values. My proposal to remedy this would be to only allow using newrat without options_.analytic_derivation when kalman_algo=2 or kalman_algo=4. That is, when not using analytical derivatives the user must at least specify a Kalman algo that allows for analytical derivatives. Such a check would make changing the options redundant. Or do you envision cases where people want an analytical Hessian based univariate filters while wanting to compute the likelihood in mode-finding based numerical derivatives derived from the multivariate Kalman filter.
An additional issue that popped up: currently, newrat with analytical_derivation seems to be broken for most Kalman algorithms. Thus, I cannot test the features/changes I propose. I created a pull request with a unit test for these issues. I attach the files you need to trace out the errors.
Best,
Johannes
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
Hi Johannes,
sorry for the very late reply.
1) yes, only kalman_algo=1 is supported by analytic derivatives;
2) I looked at your pull request. I can see you had an head-ache with my 'flag' option, which is quite obscure, apologies. This option is in fact linked to numerical derivatives and not to analytic ones. when flag=1, the standard dynare hessian.m is used, while when flag=0 the outer product gradient hessian is computed with mr_hessian (outer product gradient is not very precise, but for large models numerical hessian is not very good either and it is much more expensive, so to start MCMC it can be more efficient to use the former). Now comes the issue of kalman_algo: for outer product gradient it is best to have the max number of individual densities as possible. This is obtained using kalman_algo = 2 (or 4 for non-stationary models). This becomes crucial for large models, large data sets and large number of estimated parameters. In those case, the matrix of data becomes 'fat' and it can be that
T<number_estim_params (making outer product hessian rank deficient for multivariate filters)
but
T*varobs_nbr> number_esti_params
This implies that only with univariate filters the outer product gradient is positive definite and can be used to start a metropolis.
This explains the 'endogenous' variation of the kalman_algo option by mode_compute=5.
However, this is not needed for the optimizer step and this step can be moved below, where the hessian is computed.
However, to do this, I need to understand what is the meaning of the 'Hessian' option that you introduced in your pull request. It seems that the old 'flag' option is duplicated into
options_.newrat.hess (for numerical derivs, which should be in fact the only case for using it)
and
another 'Hessian' field into options_list.
I think the latter should be eliminated and only options_.newrat.hess should be used, to replicate the same behaviour as before?
thanks a lot, Best wishes Marco
On 05/12/14 10:04, Johannes Pfeifer wrote:
Hi Marco,
Thanks for the quick reply. That is fine with me. The compatibility was nowhere documented and judging from the code you had already done precautions for these other algorithms. The reason I send you the unit test is that for some algorithms it break down with seemingly easily fixable problem like unconformable matrices where a transpose seem to be missing. But if they are not implemented at all, we should probably just document this and filter out these cases. If I understand you correctly, only kalman_algo=1 is supported. I would then add a check for this.
For now I would appreciate if you could have a look at the newrat-implementation in https://github.com/DynareTeam/dynare/pull/817. We could then work from there.
Best,
Johannes
--
Johannes Pfeifer
Friesenwall 104
50672 Köln
Tel.: +49-221-29873852
Mobil.: +49-170-6936820
Germany
*Von:*Dev [mailto:dev-bounces@dynare.org] *Im Auftrag von *Marco Ratto *Gesendet:* Freitag, 5. Dezember 2014 08:32 *An:* List for Dynare developers *Betreff:* Re: [DynareDev] newrat
Hi Johannes,
I have to think about this. Not sure how much quickly, however. Analytic derivation should only work with multivariate stationary filter. All other filters should be broken (no analytic derivs in all other filters). best Marco
On 12/4/2014 6:20 PM, Johannes Pfeifer wrote:
Hi Marco, I am trying to factorize the mode-finding code to make it reusable in other parts of Dynare. But I am having a hard time with newrat in master. The reason is that newrat is the only optimizer that endogenously changes other options used inside of the objective function. I would really like to get rid of this behavior as it makes the generic passing of input arguments to objective functions impossible. One always needs to make sure that the updated options is passed. To be specific, the problem in master derives from the fact that the analytic Hessian requires options_.kalman_algo to be either 2 or 4. If this is not the case, newrat sets one of these values. My proposal to remedy this would be to only allow using newrat without options_.analytic_derivation when kalman_algo=2 or kalman_algo=4. That is, when not using analytical derivatives the user must at least specify a Kalman algo that allows for analytical derivatives. Such a check would make changing the options redundant. Or do you envision cases where people want an analytical Hessian based univariate filters while wanting to compute the likelihood in mode-finding based numerical derivatives derived from the multivariate Kalman filter. An additional issue that popped up: currently, newrat with analytical_derivation seems to be broken for most Kalman algorithms. Thus, I cannot test the features/changes I propose. I created a pull request with a unit test for these issues. I attach the files you need to trace out the errors. Best, Johannes _______________________________________________ Dev mailing list Dev@dynare.org <mailto:Dev@dynare.org> https://www.dynare.org/cgi-bin/mailman/listinfo/dev-- Marco Ratto, Financial and Economic Analysis Joint Research Centre The European Commission, marco.ratto@jrc.ec.europa.eu mailto:marco.ratto@jrc.ec.europa.eu
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
Dear Marco,
thanks a lot for the explanations. They help a lot to understand the code.
Regarding your question: I did not duplicate the flag-option. Its value is now stored in options_.newrat.hess. The Hessian option of options_list just provides an interface to easily change this value. By specifying
estimation(mode_compute=5,optim=(Hessian,1))
the option
options_.newrat.hess
will be set. Unless I am missing something, there is currently no other way for users to set this option. My code first checks whether the user explicitly specified a Hessian via the optim-option. If that is not the case, this flag is set according to the other specified options.
If you send me a proposal for changes, I will amend the pull request accordingly.
Best,
Johannes
--------
Johannes Pfeifer
Friesenwall 104
50672 Köln
Mobil: +49-(0)170-6936820
jpfeifer@gmx.de
Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Marco Ratto Gesendet: Dienstag, 23. Dezember 2014 22:36 An: List for Dynare developers Betreff: Re: [DynareDev] newrat
Hi Johannes,
sorry for the very late reply.
1) yes, only kalman_algo=1 is supported by analytic derivatives;
2) I looked at your pull request. I can see you had an head-ache with my 'flag' option, which is quite obscure, apologies. This option is in fact linked to numerical derivatives and not to analytic ones. when flag=1, the standard dynare hessian.m is used, while when flag=0 the outer product gradient hessian is computed with mr_hessian (outer product gradient is not very precise, but for large models numerical hessian is not very good either and it is much more expensive, so to start MCMC it can be more efficient to use the former). Now comes the issue of kalman_algo: for outer product gradient it is best to have the max number of individual densities as possible. This is obtained using kalman_algo = 2 (or 4 for non-stationary models). This becomes crucial for large models, large data sets and large number of estimated parameters. In those case, the matrix of data becomes 'fat' and it can be that
T<number_estim_params (making outer product hessian rank deficient for multivariate filters)
but
T*varobs_nbr> number_esti_params
This implies that only with univariate filters the outer product gradient is positive definite and can be used to start a metropolis.
This explains the 'endogenous' variation of the kalman_algo option by mode_compute=5.
However, this is not needed for the optimizer step and this step can be moved below, where the hessian is computed.
However, to do this, I need to understand what is the meaning of the 'Hessian' option that you introduced in your pull request. It seems that the old 'flag' option is duplicated into
options_.newrat.hess (for numerical derivs, which should be in fact the only case for using it)
and
another 'Hessian' field into options_list.
I think the latter should be eliminated and only options_.newrat.hess should be used, to replicate the same behaviour as before?
thanks a lot, Best wishes Marco
On 05/12/14 10:04, Johannes Pfeifer wrote:
Hi Marco,
Thanks for the quick reply. That is fine with me. The compatibility was nowhere documented and judging from the code you had already done precautions for these other algorithms. The reason I send you the unit test is that for some algorithms it break down with seemingly easily fixable problem like unconformable matrices where a transpose seem to be missing. But if they are not implemented at all, we should probably just document this and filter out these cases. If I understand you correctly, only kalman_algo=1 is supported. I would then add a check for this.
For now I would appreciate if you could have a look at the newrat-implementation in https://github.com/DynareTeam/dynare/pull/817. We could then work from there.
Best,
Johannes
--
Johannes Pfeifer
Friesenwall 104
50672 Köln
Tel.: +49-221-29873852
Mobil.: +49-170-6936820
Germany
Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Marco Ratto Gesendet: Freitag, 5. Dezember 2014 08:32 An: List for Dynare developers Betreff: Re: [DynareDev] newrat
Hi Johannes,
I have to think about this. Not sure how much quickly, however. Analytic derivation should only work with multivariate stationary filter. All other filters should be broken (no analytic derivs in all other filters). best Marco
On 12/4/2014 6:20 PM, Johannes Pfeifer wrote:
Hi Marco,
I am trying to factorize the mode-finding code to make it reusable in other parts of Dynare. But I am having a hard time with newrat in master. The reason is that newrat is the only optimizer that endogenously changes other options used inside of the objective function. I would really like to get rid of this behavior as it makes the generic passing of input arguments to objective functions impossible. One always needs to make sure that the updated options is passed.
To be specific, the problem in master derives from the fact that the analytic Hessian requires options_.kalman_algo to be either 2 or 4. If this is not the case, newrat sets one of these values. My proposal to remedy this would be to only allow using newrat without options_.analytic_derivation when kalman_algo=2 or kalman_algo=4. That is, when not using analytical derivatives the user must at least specify a Kalman algo that allows for analytical derivatives. Such a check would make changing the options redundant. Or do you envision cases where people want an analytical Hessian based univariate filters while wanting to compute the likelihood in mode-finding based numerical derivatives derived from the multivariate Kalman filter.
An additional issue that popped up: currently, newrat with analytical_derivation seems to be broken for most Kalman algorithms. Thus, I cannot test the features/changes I propose. I created a pull request with a unit test for these issues. I attach the files you need to trace out the errors.
Best,
Johannes
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
@Johannes apologies for the very late feedback. I just pulled a request that fixes issues related to optimizer 5. thank you for your endeavor in factorizing optimizers Marco
-
Johannes Pfeifer -
Marco Ratto