September 2015 - Dev

Re: [DynareDev] mode_compute 5
by Marco Ratto 27 Sep '15

27 Sep '15

Dear Michel, I think the code is OK without your changes, but indeed I changed the behaviour of the code w.r.t. stable version. Let me try to explain: 1) indeed the 6th input argument in newrat triggers how the hessian is computed in the final step. However, I realized that this is not the optimal way to go, so in dynare_minimize_objective this is better forced to be zero. The point is that with argument 0 (or 1) the best way to compute outer product gradient is with univarite kalman filter, since it maximizes the number of density points used in outer product. However, when as usual multivariate filter is used for estimation, this would imply to change kalman_algo from within optimizer which is not a good thing to do. So, it is best to force zero in optimization. This implies in practice no extra computation in newrat. Then, in dynare_estimation_1.m, the need for that piece of code you mention at point 2). If newratflag = 0 and univariate filter was already used, nothing is done, and no further hessian computation is performed. If multivariate filter was used, then outer product gradient is re-done using univariate algorithm. Then, it seems also OK for me to trigger there full hessian computation with mr_hessian using newratflag=2 [although I must say I never do that]. 2) as discussed in point 1), newratflag is indeed an option that triggers how the hessian in newrat is computed in place of the standard dynare one, but it needs to be used in ./matlab/dynare_estimation_1. lines 262-288 and not in ./matlab/optimization/dynare_minimize_objective.m essentially linked to univariate kalman filter use in outer product gradient 3) I think re-setting analytic_derivative can be useful for subsequent computation, in order not to change options_ w.r.t. the user had set in the estimation() command. thanks a lot best Marco -- Marco Ratto, Financial and Economic Analysis Joint Research Centre The European Commission, TP 361, 21027 ISPRA(VA), ITALY Tel: +39 0332 78 3794 Fax: +39 0332 78 5752, marco.ratto(a)jrc.ec.europa.eu <marco.ratto(a)jrc.ec.europa.eu> http://www.macfinrobods.eu/ On 9/20/2015 2:44 PM, Michel Juillard wrote: > > > Dear Marco, > > I'm trying to remove global variable objective_function_penalty_base > that is used by mode_compute 4 and 5. > > Doing that, I remarked things that I don't understand concerning > mode_compute 5 in master: > > 1) in ./matlab/optimization/dynare_minimize_objective.m, newrat() is > called on line 247 with the 6th input argument set to zero. My guess is > that it should be newflagrat, otherwise I don't understand what > newflgrat is used for. In any case, the current behavior and results > seem different from what we had in the stable version > > 2) looking at the code in > ./matlab/optimization/newrat.m, I understand that the > final value of the hessian is computed at the end of this function according to user's options. I > don't understand the reason for the code in > ./matlab/dynare_estimation_1. lines 262-288 > > 3) Is it necessary for newrat() to reset options_.analytic_derivation > and have ./matlab/dynare_estimation_1.m lines 241-242? > > My impression is that we have changed the code without fully > understanding how newrat.m is working. > > Best > > >

2 2

Status of testsuite in master branch
by Dynare Robot 26 Sep '15

26 Sep '15

d43534d9f36f0247e2590f2c98753a7bbeaf5019 Fixed compilation of org files to html (dynare-internals.org) Dynare failed to compile or MATLAB testsuite failed to run Dynare failed to compile or Octave testsuite failed to run A full log can be found at http://www.dynare.org/testsuite/master

1 0

Fwd: Cron <dynbot@karaba> /home/dynbot/snapshot/make-snapshot.sh >/home/dynbot/snapshot/lastlog.txt 2>&1 || echo "Snapshot build failed, see the log."
by Stéphane Adjemian 24 Sep '15

24 Sep '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Dear all, I made a distribution upgrade on karaba today (moving from debian wheezy to jessie). This explains why the build of the snapshot failed (the testsuite will most likely fail also). Apparently, the problem is related to emacs (when we compile the org files related to the internal documentation). Best, Stéphane. - -------- Forwarded Message -------- Subject: Cron <dynbot@karaba> /home/dynbot/snapshot/make-snapshot.sh > /home/dynbot/snapshot/lastlog.txt 2>&1 || echo "Snapshot build failed, see the log." Date: Thu, 24 Sep 2015 22:27:14 +0200 (CEST) From: Cron Daemon <root(a)cepremap.org> To: dynbot(a)cepremap.org Snapshot build failed, see the log. -----BEGIN PGP SIGNATURE----- iQQcBAEBCgAGBQJWBGFqAAoJEKbUTLnGTOd7JTAf/1OdulU4eIyWZpKWAU0jPMwX DtdQNmP9LlENlJpZKlZY6843oR/qQb8zFhFGlppuqUXLrn0cQ+E1t5TwdV89ocdj QVrs3Ya0X1DmwMsJV2SqojJsAxtd3BdeFYMlyKaC2XDtEfsCIwDICaB+7SY/TW/X dg8spSZYh0fi/z2EPHLpw8CytG8cgFHvM3jJyYf8gkY4V4hBCv6Llz0JQ39/jZMs UmGpFTOkctX8RW0eHgFivyF+JpPvFKw00woWOxo5schScjWGCz6F4e1E66poQ3Xa z0lwqumMiEGDtm27MR/eRcXY12wznZY8ZWLP1YFJpvWT3gIvnEohR4Ml7619jahN Fi5HI5SDuR0qp0Fu5R3Zm8N/TT6RLSChDqka5LYRSt3Y2ubYIwGk2omvlAHCHsnn eYRYLNPCwjgrTbozFG4ejdH43kVeUb7C3QnUF2b+O3eVVpDYP6FVaTDVwJ2N5trB 8Ff3KBB4gCvhp89TRIWMErRfutJi7BQyzPMLbpICI6p3INrOV4QfQG5oAbwBUau0 B7WUFu/shUbxIVQOdhYgRkRilXb+TIteOya/GUqXFls0CNpeUvx87XXLpxB9Yppc Usf6lqwwGLJwEzXvX3d2Mdc9IFjiPTYB6Eii7FEvzTmbSZYdM6TBTi7bRTyRa/qK dLcnkH8lexY2vIyXZdb9ijBsmSRpDwZADTQNKgC+2GV0ZnZgvfOrOzIgPNOQCQfF KOfjx2sUU9VJWYhRZUcue6eHsaGCPVGZWI/sLz39f8Vgz7ushvPe8MQ98edFYHtc D7u/4bGdOjeF8OJjxKsKP6gPxXbs8Q7UCnCW2DYxqjTzCdJnXBHSwuCqosziXd4N 8JgPLEbj5lKHXFIHP022ZxXZDZWwm95WdUm5vnmX270JvAN+M9YgaHj+0JZWf7RD S6rl38wTODK5Hk8u2kANDmZilVjLSK4fkqnxupf8kDHqfkTG2O0y3OMK7yvnJ6dl cHemlnLqJUOLwjvT+lY+MbQyYdn/s4MdNj5oMhsa20/DYN2R6Lju5PNuLzTicjNd +ua1rpoFZc8KBK68cKz/kInaer9g2wGASilK10VYCIHmEai9J+gXFK7MeJ97Plvs Uy8siOoISW+WhBpIAjkDbtB7+0+Qxqn24+36VEpWQeFM0A1RjpBB8lXPvOMCbuRC 4J62warC8mlYvEJGAtJlr6uEekYd+fmagc5/fnDGhushhRB6Uvmknr564mJ+Ikf6 qHQQFHZKJ1RHfP3ikD1Zq+vsuav51vu8xZb9xVNAG4tWCNVihd/oW+ouD3SoRreD cmHFihBC3elo4zAXfob+7LsMAfs2ACPOKk0sl76nxTc2WGkG4iHkrakVHypNydQ= =lhl7 -----END PGP SIGNATURE-----

1 0

Status of testsuite in master branch
by Dynare Robot 22 Sep '15

22 Sep '15

d7d8af34a01ce158ea06abafe87aec3dbe920454 preprocessor: remove create_timing_csv.sh (moved to testsuite.git) ================================ DYNARE MAKE CHECK MATLAB RESULTS ================================ | TOTAL: 555 | PASS: 540 | FAIL: 10 | XFAIL: 5 | XPASS: 0 | LIST OF FAILED TESTS: | * estimation/MH_recover/fs2000_recover.mod | * identification/rbc_ident/rbc_ident_varexo_only.mod | * kalman/likelihood_from_dynare/fs2000_corr_ME.mod | * kalman/likelihood_from_dynare/fs2000_corr_ME_missing.mod | * kalman/likelihood_from_dynare/fs2000_uncorr_ME.mod | * kalman/likelihood_from_dynare/fs2000_uncorr_ME_missing.mod | * ep/rbc2.mod | * ../matlab/modules/dseries/src/(a)dseries/subsasgn.m#21 | * ../matlab/modules/dseries/src/(a)dseries/subsasgn.m#22 | * ../matlab/prior_draw.m#1 | | LIST OF 10 SLOWEST TESTS: | * optimizers/fs2000_6.mod - 2261.630000 | * walsh.mod - 1119.110000 | * gsa/ls2003.mod - 865.990000 | * optimizers/fs2000_2.mod - 759.790000 | * estimation/TaRB/fs2000_tarb.mod - 298.750000 | * optimizers/fs2000_10.mod - 258.190000 | * ms-sbvar/test_ms_variances_repeated_runs.mod - 203.280000 | * conditional_forecasts/2/fs2000_est.mod - 184.530000 | * gsa/ls2003a.mod - 162.270000 | * optimal_policy/osr_example_obj_corr_non_stat_vars.mod - 138.160000 ================================ DYNARE MAKE CHECK OCTAVE RESULTS ================================ | TOTAL: 549 | PASS: 530 | FAIL: 14 | XFAIL: 5 | XPASS: 0 | LIST OF FAILED TESTS: | * estimation/fs2000_MCMC_jumping_covariance.mod | * estimation/MH_recover/fs2000_recover.mod | * gsa/cod_ML_morris/cod_ML_morris.mod | * kalman/likelihood_from_dynare/fs2000_corr_ME.mod | * kalman/likelihood_from_dynare/fs2000_corr_ME_missing.mod | * kalman/likelihood_from_dynare/fs2000_uncorr_ME.mod | * kalman/likelihood_from_dynare/fs2000_uncorr_ME_missing.mod | * ep/rbc2.mod | * optimizers/fs2000_7.mod | * ../matlab/lyapunov_symm.m#1 | * ../matlab/modules/dseries/src/(a)dseries/dseries.m#5 | * ../matlab/modules/dseries/src/(a)dseries/subsasgn.m#21 | * ../matlab/modules/dseries/src/(a)dseries/subsasgn.m#22 | * ../matlab/prior_draw.m#1 | | LIST OF 10 SLOWEST TESTS: | * optimizers/fs2000_6.mod - 45917.921690 | * gsa/ls2003.mod - 17213.831797 | * optimizers/fs2000_2.mod - 16979.489151 | * estimation/TaRB/fs2000_tarb.mod - 6492.473755 | * optimizers/fs2000_10.mod - 5678.338875 | * conditional_forecasts/2/fs2000_est.mod - 4223.023922 | * ms-sbvar/test_ms_variances_repeated_runs.mod - 3200.472017 | * TeX/fs2000_corr_ME.mod - 3189.907357 | * fs2000/fs2000.mod - 2915.702220 | * measurement_errors/fs2000_corr_me_ml_mcmc/fs2000_corr_ME.mod - 2758.232379 A full log can be found at http://www.dynare.org/testsuite/master

1 0

removing global objective_function_penalty_base
by Michel Juillard 21 Sep '15

21 Sep '15

I have made a new branch 'penalty' where I removed global objective_function_penalty_base Tell me what you think and I will merge it in master. Best -- Michel Juillard

2 8

penalty size
by Michel Juillard 21 Sep '15

21 Sep '15

In dsge_likelihood, there are several instances of penalty set to 100 (or 1) when there is no way to compute an amount proportional to the violation. Such values seem large to me and of nature of creating cliffs in the objective function. It seems that a value such as 0.1 should be sufficient as it is always added to the last accepted value of the objective function. What do you think? -- Michel Juillard

2 1

mode_compute 5
by Michel Juillard 21 Sep '15

21 Sep '15

Dear Marco, I'm trying to remove global variable objective_function_penalty_base that is used by mode_compute 4 and 5. Doing that, I remarked things that I don't understand concerning mode_compute 5 in master: 1) in ./matlab/optimization/dynare_minimize_objective.m, newrat() is called on line 247 with the 6th input argument set to zero. My guess is that it should be newflagrat, otherwise I don't understand what newflgrat is used for. In any case, the current behavior and results seem different from what we had in the stable version 2) looking at the code in ./matlab/optimization/newrat.m, I understand that the final value of the hessian is computed at the end of this function according to user's options. I don't understand the reason for the code in ./matlab/dynare_estimation_1. lines 262-288 3) Is it necessary for newrat() to reset options_.analytic_derivation and have ./matlab/dynare_estimation_1.m lines 241-242? My impression is that we have changed the code without fully understanding how newrat.m is working. Best -- Michel Juillard

2 2

Status of testsuite in master branch
by Dynare Robot 21 Sep '15

21 Sep '15

c598f4dde3675d3b3e1f4ba5be4b4c9abaeed3e9 fixed another bug in col_id computation ================================ DYNARE MAKE CHECK MATLAB RESULTS ================================ | TOTAL: 555 | PASS: 540 | FAIL: 10 | XFAIL: 5 | XPASS: 0 | LIST OF FAILED TESTS: | * estimation/MH_recover/fs2000_recover.mod | * identification/rbc_ident/rbc_ident_varexo_only.mod | * kalman/likelihood_from_dynare/fs2000_corr_ME.mod | * kalman/likelihood_from_dynare/fs2000_corr_ME_missing.mod | * kalman/likelihood_from_dynare/fs2000_uncorr_ME.mod | * kalman/likelihood_from_dynare/fs2000_uncorr_ME_missing.mod | * ep/rbc2.mod | * ../matlab/modules/dseries/src/(a)dseries/subsasgn.m#21 | * ../matlab/modules/dseries/src/(a)dseries/subsasgn.m#22 | * ../matlab/prior_draw.m#1 | | LIST OF 10 SLOWEST TESTS: | * optimizers/fs2000_6.mod - 2268.840000 | * walsh.mod - 1116.430000 | * gsa/ls2003.mod - 869.490000 | * optimizers/fs2000_2.mod - 752.130000 | * estimation/TaRB/fs2000_tarb.mod - 301.800000 | * optimizers/fs2000_10.mod - 257.130000 | * ms-sbvar/test_ms_variances_repeated_runs.mod - 203.010000 | * conditional_forecasts/2/fs2000_est.mod - 188.610000 | * gsa/ls2003a.mod - 156.310000 | * optimal_policy/osr_example_obj_corr_non_stat_vars.mod - 127.110000 Dynare failed to compile or Octave testsuite failed to run A full log can be found at http://www.dynare.org/testsuite/master

1 0

Status of testsuite in master branch
by Dynare Robot 18 Sep '15

18 Sep '15

51ab885e68c929274d8dd5f94187da2831954ba5 test suite: fix naming in graph when a point is highlighted ================================ DYNARE MAKE CHECK MATLAB RESULTS ================================ | TOTAL: 555 | PASS: 540 | FAIL: 10 | XFAIL: 5 | XPASS: 0 | LIST OF FAILED TESTS: | * estimation/MH_recover/fs2000_recover.mod | * identification/rbc_ident/rbc_ident_varexo_only.mod | * kalman/likelihood_from_dynare/fs2000_corr_ME.mod | * kalman/likelihood_from_dynare/fs2000_corr_ME_missing.mod | * kalman/likelihood_from_dynare/fs2000_uncorr_ME.mod | * kalman/likelihood_from_dynare/fs2000_uncorr_ME_missing.mod | * ep/rbc2.mod | * ../matlab/modules/dseries/src/(a)dseries/subsasgn.m#21 | * ../matlab/modules/dseries/src/(a)dseries/subsasgn.m#22 | * ../matlab/prior_draw.m#1 | | LIST OF 10 SLOWEST TESTS: | * optimizers/fs2000_6.mod - 2256.990000 | * walsh.mod - 1170.820000 | * gsa/ls2003.mod - 866.760000 | * optimizers/fs2000_2.mod - 764.270000 | * estimation/TaRB/fs2000_tarb.mod - 298.300000 | * optimizers/fs2000_10.mod - 258.940000 | * ms-sbvar/test_ms_variances_repeated_runs.mod - 203.260000 | * conditional_forecasts/2/fs2000_est.mod - 185.380000 | * gsa/ls2003a.mod - 158.080000 | * gsa/cod_ML_morris/cod_ML_morris.mod - 112.360000 Dynare failed to compile or Octave testsuite failed to run A full log can be found at http://www.dynare.org/testsuite/master

1 0

Status of testsuite in master branch
by Dynare Robot 17 Sep '15

17 Sep '15

ad3d00f0da38452c479c604aaa4ddf34c8f1521f preprocessor: fix bug when printing model variables containing underscores in latex file. closes #1059 ================================ DYNARE MAKE CHECK MATLAB RESULTS ================================ | TOTAL: 555 | PASS: 540 | FAIL: 10 | XFAIL: 5 | XPASS: 0 | LIST OF FAILED TESTS: | * estimation/MH_recover/fs2000_recover.mod | * identification/rbc_ident/rbc_ident_varexo_only.mod | * kalman/likelihood_from_dynare/fs2000_corr_ME.mod | * kalman/likelihood_from_dynare/fs2000_corr_ME_missing.mod | * kalman/likelihood_from_dynare/fs2000_uncorr_ME.mod | * kalman/likelihood_from_dynare/fs2000_uncorr_ME_missing.mod | * ep/rbc2.mod | * ../matlab/modules/dseries/src/(a)dseries/subsasgn.m#21 | * ../matlab/modules/dseries/src/(a)dseries/subsasgn.m#22 | * ../matlab/prior_draw.m#1 | | LIST OF 10 SLOWEST TESTS: | optimizers/fs2000_6.mod - 2387.680000 | gsa/ls2003.mod - 916.030000 | walsh.mod - 892.550000 | optimizers/fs2000_2.mod - 810.930000 | estimation/TaRB/fs2000_tarb.mod - 319.340000 | optimizers/fs2000_10.mod - 275.660000 | ms-sbvar/test_ms_variances_repeated_runs.mod - 208.310000 | conditional_forecasts/2/fs2000_est.mod - 204.320000 | optimal_policy/osr_example_obj_corr_non_stat_vars.mod - 176.630000 | gsa/ls2003a.mod - 153.320000 ================================ DYNARE MAKE CHECK OCTAVE RESULTS ================================ | TOTAL: 549 | PASS: 529 | FAIL: 15 | XFAIL: 5 | XPASS: 0 | LIST OF FAILED TESTS: | * estimation/MH_recover/fs2000_recover.mod | * gsa/cod_ML_morris/cod_ML_morris.mod | * fs2000/fs2000a.mod | * fs2000/fs2000_dseries_b.mod | * kalman/likelihood_from_dynare/fs2000_corr_ME.mod | * kalman/likelihood_from_dynare/fs2000_corr_ME_missing.mod | * kalman/likelihood_from_dynare/fs2000_uncorr_ME.mod | * kalman/likelihood_from_dynare/fs2000_uncorr_ME_missing.mod | * ep/rbc2.mod | * optimizers/fs2000_7.mod | * ../matlab/lyapunov_symm.m#1 | * ../matlab/modules/dseries/src/(a)dseries/dseries.m#5 | * ../matlab/modules/dseries/src/(a)dseries/subsasgn.m#21 | * ../matlab/modules/dseries/src/(a)dseries/subsasgn.m#22 | * ../matlab/prior_draw.m#1 | | LIST OF 10 SLOWEST TESTS: | optimizers/fs2000_6.mod - 48287.037751 | gsa/ls2003.mod - 17462.043309 | optimizers/fs2000_2.mod - 16804.018185 | optimizers/fs2000_10.mod - 5517.404816 | estimation/TaRB/fs2000_tarb.mod - 5490.983166 | conditional_forecasts/2/fs2000_est.mod - 3839.111930 | ms-sbvar/test_ms_variances_repeated_runs.mod - 3327.375948 | TeX/fs2000_corr_ME.mod - 3046.410390 | measurement_errors/fs2000_corr_me_ml_mcmc/fs2000_corr_ME.mod - 2523.745724 | fs2000/fs2000.mod - 2512.309009 A full log can be found at http://www.dynare.org/testsuite/master

1 0