Optimization routines
Hi Marco, I have just pushed a fix for your optimization routines (mode_compute==5) and added in the testsuite a mod file (fs2000d.mod) calling your routines. Best, Stéphane. -- Stéphane Adjemian Université du Maine, Gains et Cepremap Tel(Gains): +33(0)2 43 83 31 35 Tel(Cepremap): +33(0)1 40 77 84 19
Hi Stephane, Thanks a lot. By the way, last week in Lisbon I hve completed the routines to compute the analytic hessian (not only the analytic asymptotic hessian). If the likelihood functions are now fully updated, I can try to plug-in the analytic scores in optimization, if desired by the user. For the moment just for testing, with a syntax in the end. Moreover, I can allow for analytic HEssian in place of numerical one. OK? best Marco On 10/3/2011 12:37 PM, Stéphane Adjemian wrote:
Hi Marco,
I have just pushed a fix for your optimization routines (mode_compute==5) and added in the testsuite a mod file (fs2000d.mod) calling your routines.
Best, Stéphane.
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-- Marco Ratto, (http://easu.jrc.ec.europa.eu/), Joint Research Centre The European Commission, TP 361, 21027 ISPRA(VA), ITALY Tel: +39 0332 78 5639 Fax: +39 0332 78 5733 marco.ratto@jrc.ec.europa.eu
Hi Marco, what exactly do you call the analytic hessian and the analytic asymptotic hessian. I seem to remember from Harvey that you need to run the Kalman filter for each derivative of the likelihood with respect to one parameter. Could you vectorize these operations? Best Michel On 10/4/2011 3:23 PM, Marco Ratto wrote:
Hi Stephane, Thanks a lot. By the way, last week in Lisbon I hve completed the routines to compute the analytic hessian (not only the analytic asymptotic hessian). If the likelihood functions are now fully updated, I can try to plug-in the analytic scores in optimization, if desired by the user. For the moment just for testing, with a syntax in the end. Moreover, I can allow for analytic HEssian in place of numerical one.
OK? best Marco
On 10/3/2011 12:37 PM, Stéphane Adjemian wrote:
Hi Marco,
I have just pushed a fix for your optimization routines (mode_compute==5) and added in the testsuite a mod file (fs2000d.mod) calling your routines.
Best, Stéphane.
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Asymptotic Hessian is with the Gaussian approximation (sandwitch formula), no ? Stéphane. On 04/10/2011 15:28, Michel Juillard wrote:
Hi Marco,
what exactly do you call the analytic hessian and the analytic asymptotic hessian. I seem to remember from Harvey that you need to run the Kalman filter for each derivative of the likelihood with respect to one parameter. Could you vectorize these operations?
Best
Michel
On 10/4/2011 3:23 PM, Marco Ratto wrote:
Hi Stephane, Thanks a lot. By the way, last week in Lisbon I hve completed the routines to compute the analytic hessian (not only the analytic asymptotic hessian). If the likelihood functions are now fully updated, I can try to plug-in the analytic scores in optimization, if desired by the user. For the moment just for testing, with a syntax in the end. Moreover, I can allow for analytic HEssian in place of numerical one.
OK? best Marco
On 10/3/2011 12:37 PM, Stéphane Adjemian wrote:
Hi Marco,
I have just pushed a fix for your optimization routines (mode_compute==5) and added in the testsuite a mod file (fs2000d.mod) calling your routines.
Best, Stéphane.
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
- -- Stéphane Adjemian Université du Maine, GAINS & CEPREMAP http://www.dynare.org/stepan Tél(Chevaleret) +33(0)1 40 77 84 19 Tél(Le Mans) +33(0)2 43 83 31 35 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iF4EAREIAAYFAk6LC6gACgkQrQOaeEU5AZf4QwEAhB/ZF4+bhcH6KFzhGo9kyu+R F4ckIh7tEnXYTQBpI9AA/0njAKkNSaeFfiTsKD+iI7RWzxnZP3GTTVgA4XQPrubo =o+5D -----END PGP SIGNATURE-----
Hi Michel, by analytic Hessian I mean the matrix of second derivatives of the likelihood. the analytic asymptotic hessian is the limit of the hessian for infinite number of observations, where all second order derivative terms drop out in the formulas, and only first order derivatives are used to compute asymptotic hessian. In the routines, for every time step of the kalman filter, I do a loop for every parameter value to compute the analytic score. If the hessian is desired, I also add the second loop for all cross derivatives. In the end this is a loop similar to Harvey, since there is not true vectorization, but all common computations for the filter are done only once, instead of doing them many times for each derivative. So, computationally not a bad solution. I have to think if I can really vectorize the formulas and implement them as tensor/algebraic equations, but I am not sure if this really possible. In the end, with one single call of kalman filter with the analytic scores/ hessian, I get the full result. best Marco On 10/4/2011 3:28 PM, Michel Juillard wrote:
Hi Marco,
what exactly do you call the analytic hessian and the analytic asymptotic hessian. I seem to remember from Harvey that you need to run the Kalman filter for each derivative of the likelihood with respect to one parameter. Could you vectorize these operations?
Best
Michel
On 10/4/2011 3:23 PM, Marco Ratto wrote:
Hi Stephane, Thanks a lot. By the way, last week in Lisbon I hve completed the routines to compute the analytic hessian (not only the analytic asymptotic hessian). If the likelihood functions are now fully updated, I can try to plug-in the analytic scores in optimization, if desired by the user. For the moment just for testing, with a syntax in the end. Moreover, I can allow for analytic HEssian in place of numerical one.
OK? best Marco
On 10/3/2011 12:37 PM, Stéphane Adjemian wrote:
Hi Marco,
I have just pushed a fix for your optimization routines (mode_compute==5) and added in the testsuite a mod file (fs2000d.mod) calling your routines.
Best, Stéphane.
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-- Marco Ratto, (http://easu.jrc.ec.europa.eu/), Joint Research Centre The European Commission, TP 361, 21027 ISPRA(VA), ITALY Tel: +39 0332 78 5639 Fax: +39 0332 78 5733 marco.ratto@jrc.ec.europa.eu
Thanks Marco, this is maybe an area where we can thing about low level parallelization. Do you have already timing results? I always wondered about the gain in speed as the computation of the analytical derivatives is rather complex. Best Michel On 10/4/2011 3:36 PM, Marco Ratto wrote:
Hi Michel,
by analytic Hessian I mean the matrix of second derivatives of the likelihood. the analytic asymptotic hessian is the limit of the hessian for infinite number of observations, where all second order derivative terms drop out in the formulas, and only first order derivatives are used to compute asymptotic hessian.
In the routines, for every time step of the kalman filter, I do a loop for every parameter value to compute the analytic score. If the hessian is desired, I also add the second loop for all cross derivatives.
In the end this is a loop similar to Harvey, since there is not true vectorization, but all common computations for the filter are done only once, instead of doing them many times for each derivative. So, computationally not a bad solution. I have to think if I can really vectorize the formulas and implement them as tensor/algebraic equations, but I am not sure if this really possible.
In the end, with one single call of kalman filter with the analytic scores/ hessian, I get the full result.
best Marco
On 10/4/2011 3:28 PM, Michel Juillard wrote:
Hi Marco,
what exactly do you call the analytic hessian and the analytic asymptotic hessian. I seem to remember from Harvey that you need to run the Kalman filter for each derivative of the likelihood with respect to one parameter. Could you vectorize these operations?
Best
Michel
On 10/4/2011 3:23 PM, Marco Ratto wrote:
Hi Stephane, Thanks a lot. By the way, last week in Lisbon I hve completed the routines to compute the analytic hessian (not only the analytic asymptotic hessian). If the likelihood functions are now fully updated, I can try to plug-in the analytic scores in optimization, if desired by the user. For the moment just for testing, with a syntax in the end. Moreover, I can allow for analytic HEssian in place of numerical one.
OK? best Marco
On 10/3/2011 12:37 PM, Stéphane Adjemian wrote:
Hi Marco,
I have just pushed a fix for your optimization routines (mode_compute==5) and added in the testsuite a mod file (fs2000d.mod) calling your routines.
Best, Stéphane.
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
Hi Michel, not yet profiled the routines. With small models like ls2003 etc. is it quite competitive with numerical ones. Also the actual dynare parallelization with the 'always-open' mode could be tried (also for numerical derivs). best Marco On 10/4/2011 3:40 PM, Michel Juillard wrote:
Thanks Marco,
this is maybe an area where we can thing about low level parallelization. Do you have already timing results? I always wondered about the gain in speed as the computation of the analytical derivatives is rather complex.
Best
Michel
On 10/4/2011 3:36 PM, Marco Ratto wrote:
Hi Michel,
by analytic Hessian I mean the matrix of second derivatives of the likelihood. the analytic asymptotic hessian is the limit of the hessian for infinite number of observations, where all second order derivative terms drop out in the formulas, and only first order derivatives are used to compute asymptotic hessian.
In the routines, for every time step of the kalman filter, I do a loop for every parameter value to compute the analytic score. If the hessian is desired, I also add the second loop for all cross derivatives.
In the end this is a loop similar to Harvey, since there is not true vectorization, but all common computations for the filter are done only once, instead of doing them many times for each derivative. So, computationally not a bad solution. I have to think if I can really vectorize the formulas and implement them as tensor/algebraic equations, but I am not sure if this really possible.
In the end, with one single call of kalman filter with the analytic scores/ hessian, I get the full result.
best Marco
On 10/4/2011 3:28 PM, Michel Juillard wrote:
Hi Marco,
what exactly do you call the analytic hessian and the analytic asymptotic hessian. I seem to remember from Harvey that you need to run the Kalman filter for each derivative of the likelihood with respect to one parameter. Could you vectorize these operations?
Best
Michel
On 10/4/2011 3:23 PM, Marco Ratto wrote:
Hi Stephane, Thanks a lot. By the way, last week in Lisbon I hve completed the routines to compute the analytic hessian (not only the analytic asymptotic hessian). If the likelihood functions are now fully updated, I can try to plug-in the analytic scores in optimization, if desired by the user. For the moment just for testing, with a syntax in the end. Moreover, I can allow for analytic HEssian in place of numerical one.
OK? best Marco
On 10/3/2011 12:37 PM, Stéphane Adjemian wrote:
Hi Marco,
I have just pushed a fix for your optimization routines (mode_compute==5) and added in the testsuite a mod file (fs2000d.mod) calling your routines.
Best, Stéphane.
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-- Marco Ratto, (http://easu.jrc.ec.europa.eu/), Joint Research Centre The European Commission, TP 361, 21027 ISPRA(VA), ITALY Tel: +39 0332 78 5639 Fax: +39 0332 78 5733 marco.ratto@jrc.ec.europa.eu
Thanks Michel On 10/4/2011 4:31 PM, Marco Ratto wrote:
Hi Michel,
not yet profiled the routines. With small models like ls2003 etc. is it quite competitive with numerical ones.
Also the actual dynare parallelization with the 'always-open' mode could be tried (also for numerical derivs).
best Marco
On 10/4/2011 3:40 PM, Michel Juillard wrote:
Thanks Marco,
this is maybe an area where we can thing about low level parallelization. Do you have already timing results? I always wondered about the gain in speed as the computation of the analytical derivatives is rather complex.
Best
Michel
On 10/4/2011 3:36 PM, Marco Ratto wrote:
Hi Michel,
by analytic Hessian I mean the matrix of second derivatives of the likelihood. the analytic asymptotic hessian is the limit of the hessian for infinite number of observations, where all second order derivative terms drop out in the formulas, and only first order derivatives are used to compute asymptotic hessian.
In the routines, for every time step of the kalman filter, I do a loop for every parameter value to compute the analytic score. If the hessian is desired, I also add the second loop for all cross derivatives.
In the end this is a loop similar to Harvey, since there is not true vectorization, but all common computations for the filter are done only once, instead of doing them many times for each derivative. So, computationally not a bad solution. I have to think if I can really vectorize the formulas and implement them as tensor/algebraic equations, but I am not sure if this really possible.
In the end, with one single call of kalman filter with the analytic scores/ hessian, I get the full result.
best Marco
On 10/4/2011 3:28 PM, Michel Juillard wrote:
Hi Marco,
what exactly do you call the analytic hessian and the analytic asymptotic hessian. I seem to remember from Harvey that you need to run the Kalman filter for each derivative of the likelihood with respect to one parameter. Could you vectorize these operations?
Best
Michel
On 10/4/2011 3:23 PM, Marco Ratto wrote:
Hi Stephane, Thanks a lot. By the way, last week in Lisbon I hve completed the routines to compute the analytic hessian (not only the analytic asymptotic hessian). If the likelihood functions are now fully updated, I can try to plug-in the analytic scores in optimization, if desired by the user. For the moment just for testing, with a syntax in the end. Moreover, I can allow for analytic HEssian in place of numerical one.
OK? best Marco
On 10/3/2011 12:37 PM, Stéphane Adjemian wrote:
Hi Marco,
I have just pushed a fix for your optimization routines (mode_compute==5) and added in the testsuite a mod file (fs2000d.mod) calling your routines.
Best, Stéphane.
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi Marco, It's OK for me. This will allow the implementation of non informative priors à la Jeffrey... Best, Stéphane. On 04/10/2011 15:23, Marco Ratto wrote:
Hi Stephane, Thanks a lot. By the way, last week in Lisbon I hve completed the routines to compute the analytic hessian (not only the analytic asymptotic hessian). If the likelihood functions are now fully updated, I can try to plug-in the analytic scores in optimization, if desired by the user. For the moment just for testing, with a syntax in the end. Moreover, I can allow for analytic HEssian in place of numerical one.
OK? best Marco
On 10/3/2011 12:37 PM, Stéphane Adjemian wrote:
Hi Marco,
I have just pushed a fix for your optimization routines (mode_compute==5) and added in the testsuite a mod file (fs2000d.mod) calling your routines.
Best, Stéphane.
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
- -- Stéphane Adjemian Université du Maine, GAINS & CEPREMAP http://www.dynare.org/stepan Tél(Chevaleret) +33(0)1 40 77 84 19 Tél(Le Mans) +33(0)2 43 83 31 35 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iF4EAREIAAYFAk6LDEsACgkQrQOaeEU5AZddhgD/etvePBrWyKuen5BXlQEtRNmM ndKffH7iSTsc6O4JJIkA/2ph/uIzczY6IeDeOjaU9iRd1L7mgEa9IzV361mo/IJX =lgru -----END PGP SIGNATURE-----
participants (3)
-
Marco Ratto -
Michel Juillard -
Stéphane Adjemian