Re: [DynareDev] Bug in dynare_estimation_init in 4.2.5
Actually, the correct solution is if options_.prefilter == 1 if options_.missing_data data=rawdata; bayestopt_.mean_varobs = zeros(n_varobs,1); for variable=1:n_varobs rdx = find(~isnan(rawdata(:,variable))); m = mean(rawdata(rdx,variable)); data(rdx,variable) = rawdata(rdx,variable)-m; bayestopt_.mean_varobs(variable) = m; end else bayestopt_.mean_varobs = mean(rawdata,1)'; data = rawdata-repmat(bayestopt_.mean_varobs',gend,1); end else data=rawdata; end % Transpose the dataset array. data = transpose(data); I pushed the fix to the 4.2 branch. Best, Johannes -------- Johannes Pfeifer Haußerstr. 29 72076 Tübingen Tel.: +49-(0)7071-6396184 Mobil: +49-(0)170-6936820 jpfeifer@gmx.de Von: Johannes Pfeifer [mailto:jpfeifer@gmx.de] Gesendet: Freitag, 11. Mai 2012 20:14 An: List for Dynare developers (dev@dynare.org) Betreff: Bug in dynare_estimation_init in 4.2.5 Dear all, dynare_estimation_init.m in 4.2.5 currently reads if options_.prefilter == 1 if options_.missing_data bayestopt_.mean_varobs = zeros(n_varobs,1); for variable=1:n_varobs rdx = find(~isnan(rawdata(:,variable))); m = mean(rawdata(rdx,variable)); rawdata(rdx,variable) = rawdata(rdx,variable)-m; bayestopt_.mean_varobs(variable) = m; end else bayestopt_.mean_varobs = mean(rawdata,1)'; rawdata = rawdata-repmat(bayestopt_.mean_varobs',gend,1); end end % Transpose the dataset array. data = transpose(rawdata); Hence, rawdata is overwritten with the demeaned data. This leads to wrong smoother plots. The corresponding part should be: if options_.prefilter == 1 if options_.missing_data data=rawdata; bayestopt_.mean_varobs = zeros(n_varobs,1); for variable=1:n_varobs rdx = find(~isnan(rawdata(:,variable))); m = mean(rawdata(rdx,variable)); data(rdx,variable) = rawdata(rdx,variable)-m; bayestopt_.mean_varobs(variable) = m; end else bayestopt_.mean_varobs = mean(rawdata,1)'; data = rawdata-repmat(bayestopt_.mean_varobs',gend,1); end end % Transpose the dataset array. data = transpose(data); I tried to push a fix for cherrypicking to 4.2 but I am not sure I succeeded. Best, Johannes -------- Johannes Pfeifer Haußerstr. 29 72076 Tübingen Tel.: +49-(0)7071-6396184 Mobil: +49-(0)170-6936820 jpfeifer@gmx.de
Thanks Johannes. I have pushed your commits to master All the best, Michel On 05/14/2012 07:07 PM, Johannes Pfeifer wrote:
Actually, the correct solution is
ifoptions_.prefilter == 1
ifoptions_.missing_data
data=rawdata;
bayestopt_.mean_varobs = zeros(n_varobs,1);
forvariable=1:n_varobs
rdx = find(~isnan(rawdata(:,variable)));
m = mean(rawdata(rdx,variable));
data(rdx,variable) = rawdata(rdx,variable)-m;
bayestopt_.mean_varobs(variable) = m;
end
else
bayestopt_.mean_varobs = mean(rawdata,1)';
data = rawdata-repmat(bayestopt_.mean_varobs',gend,1);
end
else
data=rawdata;
end
% Transpose the dataset array.
data = transpose(data);
I pushed the fix to the 4.2 branch.
Best, Johannes
--------
Johannes Pfeifer
Haußerstr. 29
72076 Tübingen
Tel.: +49-(0)7071-6396184
Mobil: +49-(0)170-6936820
jpfeifer@gmx.de
*Von:*Johannes Pfeifer [mailto:jpfeifer@gmx.de] *Gesendet:* Freitag, 11. Mai 2012 20:14 *An:* List for Dynare developers (dev@dynare.org) *Betreff:* Bug in dynare_estimation_init in 4.2.5
Dear all,
dynare_estimation_init.m in 4.2.5 currently reads
ifoptions_.prefilter == 1
ifoptions_.missing_data
bayestopt_.mean_varobs = zeros(n_varobs,1);
forvariable=1:n_varobs
rdx = find(~isnan(rawdata(:,variable)));
m = mean(rawdata(rdx,variable));
rawdata(rdx,variable) = rawdata(rdx,variable)-m;
bayestopt_.mean_varobs(variable) = m;
end
else
bayestopt_.mean_varobs = mean(rawdata,1)';
rawdata = rawdata-repmat(bayestopt_.mean_varobs',gend,1);
end
end
% Transpose the dataset array.
data = transpose(rawdata);
Hence, rawdata is overwritten with the demeaned data. This leads to wrong smoother plots. The corresponding part should be:
ifoptions_.prefilter == 1
ifoptions_.missing_data
data=rawdata;
bayestopt_.mean_varobs = zeros(n_varobs,1);
forvariable=1:n_varobs
rdx = find(~isnan(rawdata(:,variable)));
m = mean(rawdata(rdx,variable));
data(rdx,variable) = rawdata(rdx,variable)-m;
bayestopt_.mean_varobs(variable) = m;
end
else
bayestopt_.mean_varobs = mean(rawdata,1)';
data = rawdata-repmat(bayestopt_.mean_varobs',gend,1);
end
end
% Transpose the dataset array.
data = transpose(data);
I tried to push a fix for cherrypicking to 4.2 but I am not sure I succeeded.
Best,
Johannes
--------
Johannes Pfeifer
Haußerstr. 29
72076 Tübingen
Tel.: +49-(0)7071-6396184
Mobil: +49-(0)170-6936820
jpfeifer@gmx.de <mailto:jpfeifer@gmx.de>
_______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
Dear All, I have recently looked at posterior MCMC things and see that since dynare 4.2. ReshapeMatFiles is still alive for posterior irfs, while for all other posterior analysis reshape is no longer done and pm3 is used to manage the raw files saved during posterior sampling. ReshapeMatFiles in posteriorIRF is still a big bottleneck for large models (for my last metropolis with QUEST it took 5.5 hrs to get the reshaped files, while the posterior sampling per se was much more rapid!). If reshape is needed for posterior irfs, I think there are ways to make it quicker. I have yesterday tried an option which reduced the 5.5 hrs to 1hr20m. On the other hand, could we envisage the use of pm3 also for irfs and skip the use of Reshape as a whole? best Marco -- Marco Ratto, Joint Research Centre The European Commission, TP 361, 21027 ISPRA(VA), ITALY Tel: +39 0332 78 9217 Fax: +39 0332 78 5733 marco.ratto@jrc.ec.europa.eu
Marco Ratto <marco.ratto@jrc.ec.europa.eu> writes:
I have recently looked at posterior MCMC things and see that since dynare 4.2. ReshapeMatFiles is still alive for posterior irfs, while for all other posterior analysis reshape is no longer done and pm3 is used to manage the raw files saved during posterior sampling.
ReshapeMatFiles in posteriorIRF is still a big bottleneck for large models (for my last metropolis with QUEST it took 5.5 hrs to get the reshaped files, while the posterior sampling per se was much more rapid!).
If reshape is needed for posterior irfs, I think there are ways to make it quicker. I have yesterday tried an option which reduced the 5.5 hrs to 1hr20m. On the other hand, could we envisage the use of pm3 also for irfs and skip the use of Reshape as a whole?
I am not very familiar with this part of the code, but I would say that it is a good idea to factorize code and to drop obsolete routines. -- Sébastien Villemot Researcher in Economics & Debian Maintainer http://www.dynare.org/sebastien Phone: +33-1-40-77-84-04 - GPG Key: 4096R/381A7594
participants (4)
-
Johannes Pfeifer -
Marco Ratto -
Michel Juillard -
Sébastien Villemot