- Dev - dynare.org

Re: [DynareDev] [Trac] #20: Create new Dynare web site (using Plone)
by Trac Dynare 22 Dec '09

22 Dec '09

#20: Create new Dynare web site (using Plone) ----------------------------+----------------------------------------------- Reporter: sebastien | Owner: Type: enhancement | Status: closed Priority: major | Milestone: Component: Documentation | Version: Resolution: fixed | Keywords: ----------------------------+----------------------------------------------- Changes (by sebastien): * status: new => closed * resolution: => fixed Comment: The new website is up, and accessible on http://www.dynare.org The old address gives a redirection to the new one. Snapshots are now built on Karaba, and transferred to Kirikou via SSH. There are two flavours of snapshot: source snapshost and Windows binary snapshot. The M2HTML documentation has moved: see the new link in the "Developers' corner" of the Dynare website. -- Ticket URL: <https://www.dynare.org/trac/ticket/20#comment:2> Dynare <http://www.dynare.org> The Dynare project

1 0

Re: [DynareDev] Proposed specification for estimation functions
by G. Perendia 21 Dec '09

21 Dec '09

Hi Sorry for this stream of emails - you may ignore the previous two today (and this one too actually) as I cleared out now how we will link the different priors to the individual parameters and their individual sub-period life-spans in calculation of the posterior, as stated under Parameters and Sub-Samples on the EstimationModule wiki page. Regards George

1 0

[Trac] #67: Bug in gamrnd (prior_draw)
by Trac Dynare 21 Dec '09

21 Dec '09

#67: Bug in gamrnd (prior_draw) --------------------------+------------------------------------------------- Reporter: ratto | Owner: Type: bug | Status: new Priority: major | Milestone: Component: Core M-files | Version: Keywords: | --------------------------+------------------------------------------------- For some types of gamma samples, like mean=1.5 and std=0.25, which implies a=36, b= 0.0417 the gamrnd dynare routine provides wrong values (too high), always rejected since the values are always above the upper bound. -- Ticket URL: <https://www.dynare.org/trac/ticket/67> Dynare <http://www.dynare.org> The Dynare project

1 1

[Trac] #68: Option stack_solve_algo with values 2, 3 and 4 are not working properly
by Trac Dynare 21 Dec '09

21 Dec '09

#68: Option stack_solve_algo with values 2, 3 and 4 are not working properly --------------------------+------------------------------------------------- Reporter: sebastien | Owner: ferhat Type: bug | Status: new Priority: major | Milestone: 4.1 Component: Core M-files | Version: Keywords: | --------------------------+------------------------------------------------- See the following files in tests/block_bytecode: fs2000_bicgstab.mod fs2000_gmres.mod fs2000_optpath.mod ramst_a.mod In the first two files, computation succeeds but gives wrong results. In the last two files, MATLAB fails with an error. -- Ticket URL: <https://www.dynare.org/trac/ticket/68> Dynare <http://www.dynare.org> The Dynare project

1 1

[Trac] #44: The preprocessor can't normalize GIMF model with block+bytecode (but succeeds with only block)
by Trac Dynare 21 Dec '09

21 Dec '09

#44: The preprocessor can't normalize GIMF model with block+bytecode (but succeeds with only block) --------------------------+------------------------------------------------- Reporter: sebastien | Owner: ferhat Type: bug | Status: new Priority: major | Milestone: 4.1 Component: Preprocessor | Version: Keywords: | --------------------------+------------------------------------------------- -- Ticket URL: <https://www.dynare.org/trac/ticket/44> Dynare <http://www.dynare.org> The Dynare project

1 3

Re: [DynareDev] Proposed specification for estimation functions
by G. Perendia 21 Dec '09

21 Dec '09

PS: or are priors going to be "true" priors, given once for each parameter and valid across all sub-periods? ----- Original Message ----- From: "G. Perendia" <george(a)perendia.orangehome.co.uk> > One issue about the design is, are the priors going to be dependent on the > individual parameters and linked to their lifespans (from start to end of > each period, all the way through), or, after the initial estimation step, > priors will be dependent and related to the common estimation periods - as I > understand it is the latter - and e.g. the posteriors from the previous > (common) sub-sample estimation will become priors for the next - are those > correct views? > > Best regards > > George > > > ----- Original Message ----- > From: "G. Perendia" <george(a)perendia.orangehome.co.uk> > To: <michel.juillard(a)ens.fr> > Cc: <sebastien.villemot(a)ens.fr> > Sent: Saturday, December 12, 2009 11:42 AM > Subject: Re: [DynareDev] Proposed specification for estimation functions > > > > Dear Michel and Sebastien > > > > 1) Perhaps I need to explain better what my line of thought was: > > > > Michel wrote: > > > > > "The estimation algorithm (optimization or MCMC) see all the > > parameters > > > > > for all sub-periods as a single vector. > > > > > > > > > I agree in part. But where is the vector of parameter sub-sample periods > and > > associated priors? > > What I see our Wiki page is describing is what estimation for each period > > (time slot) needs - i.e. just one of the ("longest" common) time-periods > for > > estimation, and the additional info is required to define the start and > the > > end of that period.(which has its start higher/equal than any individual > > parameter's start and its end lower|equal than any individual parameter's > > end in for period slot). > > > > In the (still rough draft of the) model I suggest that information is > mainly > > held (or referred from for the integration as a "view") by the structure > > EstPeriod and all estimation periods in the vector of EstPeriod-s. held in > > the EstimatedParametersDescription together with the time-invariant > > attributes (e.g. IDs..) for integration.. > > > > > > > In updateQHparams(), it is possible to compare the date of the > > relevant > > > > > subperiod with StartPeriod and EndPeriod. I don't think that such > > tests > > > > > would be very costly." > > > > I am also sure it is more efficient to construct those periods once rather > > than repeating time slot/period tests in each call to Likelihood/posterior > > calculation (and there may be 100,000s of those in one estimation) and we > > should use a design that supports that from the start. > > > > Those EstPeriod-s should be constructed at some point (I guess once at a > > start of estimation by EstPeriod constructor ) from the information such > as > > individual parameter start-end periods and the relevant associated priors > > for those periods. That initial info-set is held in the vector of vectors > of > > EstParamSubSampleDescription. > > > > The EstimatedParametersDescription. structure (or class ) integrates all > > that information and makes it available to (friend, for efficiency) class > > LogPosteriorDensity on as needed basis and updates all parameters (inc. H > > and Q) when time slot/period related xparam1 and priors sets are > supplied. > > > > I hope this clarification helps. > > > > > > 2) I however agree that it is often difficult to exchange thoughts over > > email when person-to-person brain-storming discussion is often more > > efficient and contributes to better understanding. If you would like me > to > > come for a meeting in Paris I will and I guess the earlier the better so I > > suggest Monday 21st if that is the earliest suitable for Sebastien and I > can > > stay till Tuesday if you also think that having a spare day for possible > > additional discussion or clarification may be beneficial (or I can just > work > > in the office instead). > > > > Shall I book the tickets for Monday only or Monday and Tuesday (with > > accommodation) then?. > > > > > > PS: I believe that my time spent on the design using the diagramming tool > is > > not wasted since it provides a better overview of the model and it can > also > > produce all the header and .cpp function interface code with the comments > > automatically , thus saving some time on the initial coding side. > > > > Best regards > > George > > > > ----- Original Message ----- From: "G. Perendia" > > <george(a)perendia.orangehome.co.uk> > > > > > > Dear Michel and Sebastien > > > > > > > > I am not sure if it is that simple. > > > > I enclose a draft diagram of basic estimation model with new Estim > > > > parameters description class and its associated (public ? data) > > > structures > > > > that should be created by constructors once and as much as poss. > reused > > > > through repeated calls to CalcPosteriorDensity (e.g. EstPeriods so > that > > we > > > > know how many periods/time slots we need to estimate) ). There is also > > > some > > > > redundancy in the schema so to improve efficiency too. > > > > > > > > I wanted your opinion before I upload it to Wiki page - let me know if > > you > > > > would like more info. > > > > > > > > Best regards > > > > > > > > George > > > > ----- Original Message ----- > > > > From: "Michel Juillard" <michel.juillard(a)ens.fr> > > > > To: "List for Dynare developers" <dev(a)dynare.org> > > > > Sent: Thursday, December 10, 2009 8:11 PM > > > > Subject: Re: [DynareDev] Proposed specification for estimation > functions > > > > > > > > > > > > > Dear George, > > > > > > > > > > I'm not sure that it needs to be that complicated. > > > > > The estimation algorithm (optimization or MCMC) see all the > parameters > > > > > for all sub-periods as a single vector. > > > > > > > > > > In updateQHparams(), it is possible to compare the date of the > > relevant > > > > > subperiod with StartPeriod and EndPeriod. I don't think that such > > tests > > > > > would be very costly. > > > > > > > > > > If I'm wrong and that we need after all the structures that you > > suggest, > > > > > we will add them later. > > > > > > > > > > All the best, > > > > > > > > > > Michel > > > > > > > > > > G. Perendia wrote: > > > > > > Thanks Michel > > > > > > As parameters can have different time slots I think we will need > to > > > add > > > > two > > > > > > structures and: > > > > > > 1 one to )organise data around individual parameters having a > vector > > > of > > > > > > descriptions for each of their time slots, and > > > > > > 2) an additional structure listing all identified longest common > > > periods > > > > > > pointing to each parameter's description for that slot. > > > > > > > > > > > > Thus, > > > > > > re (1) instead of xparam1 being vector of doubles and e.g. > > > start_period > > > > > > vectors having n entries, xpramss1 will be a vector of vectors of > > > > > > structures, and, > > > > > > re (2) timeslots will be a vector of common periods of > > > unpredetermined > > > > > > size, of vectors of length n containing n indices to the relevant > > > > > > parameter's description for that period including its value and > > > priors. > > > > > > > > > > > > Best regards > > > > > > > > > > > > George > > > > > > > > > > > > ----- Original Message ----- > > > > > > From: "Michel Juillard" <michel.juillard(a)ens.fr> > > > > > > To: "List for Dynare developers" <dev(a)dynare.org> > > > > > > Sent: Thursday, December 10, 2009 10:58 AM > > > > > > Subject: Re: [DynareDev] Proposed specification for estimation > > > functions > > > > > > > > > > > > > > > > > > > > > > > >> Hi George, > > > > > >> > > > > > >> performance improvement was the idea. Lets keep the scalars as > they > > > are > > > > > >> used to distinguish different types of estimation problems. You > can > > > > > >> leave aside the vector of indices until we encounter the need for > > > them. > > > > > >> > > > > > >> Best > > > > > >> > > > > > >> Michel > > > > > >> > > > > > >> G. Perendia wrote: > > > > > >> > > > > > >>> Hi > > > > > >>> > > > > > >>> It seems to me that the following values and vectors described > on > > > the > > > > > >>> > > > > > > (new) > > > > > > > > > > > >>> EstimationModule wiki page are in a way redundant as it looks > like > > > > that > > > > > >>> > > > > > > they > > > > > > > > > > > >>> can > > > > > >>> be derived from the parameter_description type vector. If so, is > > > there > > > > > >>> > > > > > > any > > > > > > > > > > > >>> other > > > > > >>> reason for keeping them explicitly as part of the structure > except > > > > that > > > > > >>> > > > > > > it > > > > > > > > > > > >>> is useful for performance improvement and for easier > understanding > > > > > >>> of the structure? > > > > > >>> > > > > > >>> - nvx integer scalar (number of estimated structural shock > > standard > > > > > >>> deviations). > > > > > >>> -. nvx_id nvx by 1 vector of integers. > > > > > >>> -. nvn integer scalar (number of estimated measurement error > > > standard > > > > > >>> deviations). > > > > > >>> -. nvn_id nvn by 1 vector of integers. > > > > > >>> -. ncx integer scalar (number of estimated structural shock > > > > > >>> > > > > > > correlations). > > > > > > > > > > > >>> -. ncx_id ncx by 1 vector of integers. > > > > > >>> -. ncn integer scalar (number of estimated measurement error > > > > > >>> > > > > > > correlations). > > > > > > > > > > > >>> -. ncn_id ncn by 1 vector of integers. > > > > > >>> -. np integer scalar (number of estimated "deep" parameters). > > > > > >>> -. np_id np by 1 vector of integers. > > > > > >>> > > > > > >>> Best regards > > > > > >>> > > > > > >>> George > > > > > >>> > > > > > >>> > > > > > >>> > > > > > >>> > > > > > >>> > > > > > >>> _______________________________________________ > > > > > >>> Dev mailing list > > > > > >>> Dev(a)dynare.org > > > > > >>> http://www.dynare.org/cgi-bin/mailman/listinfo/dev > > > > > >>> > > > > > >>> > > > > > >>> > > > > > >> _______________________________________________ > > > > > >> Dev mailing list > > > > > >> Dev(a)dynare.org > > > > > >> http://www.dynare.org/cgi-bin/mailman/listinfo/dev > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > Dev mailing list > > > > > > Dev(a)dynare.org > > > > > > http://www.dynare.org/cgi-bin/mailman/listinfo/dev > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > Dev mailing list > > > > > Dev(a)dynare.org > > > > > http://www.dynare.org/cgi-bin/mailman/listinfo/dev > > > > > > > > > >

1 0

Re: [DynareDev] Proposed specification for estimation functions
by G. Perendia 21 Dec '09

21 Dec '09

One issue about the design is, are the priors going to be dependent on the individual parameters and linked to their lifespans (from start to end of each period, all the way through), or, after the initial estimation step, priors will be dependent and related to the common estimation periods - as I understand it is the latter - and e.g. the posteriors from the previous (common) sub-sample estimation will become priors for the next - are those correct views? Best regards George ----- Original Message ----- From: "G. Perendia" <george(a)perendia.orangehome.co.uk> To: <michel.juillard(a)ens.fr> Cc: <sebastien.villemot(a)ens.fr> Sent: Saturday, December 12, 2009 11:42 AM Subject: Re: [DynareDev] Proposed specification for estimation functions > Dear Michel and Sebastien > > 1) Perhaps I need to explain better what my line of thought was: > > Michel wrote: > > > > "The estimation algorithm (optimization or MCMC) see all the > parameters > > > > for all sub-periods as a single vector. > > > > > > I agree in part. But where is the vector of parameter sub-sample periods and > associated priors? > What I see our Wiki page is describing is what estimation for each period > (time slot) needs - i.e. just one of the ("longest" common) time-periods for > estimation, and the additional info is required to define the start and the > end of that period.(which has its start higher/equal than any individual > parameter's start and its end lower|equal than any individual parameter's > end in for period slot). > > In the (still rough draft of the) model I suggest that information is mainly > held (or referred from for the integration as a "view") by the structure > EstPeriod and all estimation periods in the vector of EstPeriod-s. held in > the EstimatedParametersDescription together with the time-invariant > attributes (e.g. IDs..) for integration.. > > > > > In updateQHparams(), it is possible to compare the date of the > relevant > > > > subperiod with StartPeriod and EndPeriod. I don't think that such > tests > > > > would be very costly." > > I am also sure it is more efficient to construct those periods once rather > than repeating time slot/period tests in each call to Likelihood/posterior > calculation (and there may be 100,000s of those in one estimation) and we > should use a design that supports that from the start. > > Those EstPeriod-s should be constructed at some point (I guess once at a > start of estimation by EstPeriod constructor ) from the information such as > individual parameter start-end periods and the relevant associated priors > for those periods. That initial info-set is held in the vector of vectors of > EstParamSubSampleDescription. > > The EstimatedParametersDescription. structure (or class ) integrates all > that information and makes it available to (friend, for efficiency) class > LogPosteriorDensity on as needed basis and updates all parameters (inc. H > and Q) when time slot/period related xparam1 and priors sets are supplied. > > I hope this clarification helps. > > > 2) I however agree that it is often difficult to exchange thoughts over > email when person-to-person brain-storming discussion is often more > efficient and contributes to better understanding. If you would like me to > come for a meeting in Paris I will and I guess the earlier the better so I > suggest Monday 21st if that is the earliest suitable for Sebastien and I can > stay till Tuesday if you also think that having a spare day for possible > additional discussion or clarification may be beneficial (or I can just work > in the office instead). > > Shall I book the tickets for Monday only or Monday and Tuesday (with > accommodation) then?. > > > PS: I believe that my time spent on the design using the diagramming tool is > not wasted since it provides a better overview of the model and it can also > produce all the header and .cpp function interface code with the comments > automatically , thus saving some time on the initial coding side. > > Best regards > George > > ----- Original Message ----- From: "G. Perendia" > <george(a)perendia.orangehome.co.uk> > > > > Dear Michel and Sebastien > > > > > > I am not sure if it is that simple. > > > I enclose a draft diagram of basic estimation model with new Estim > > > parameters description class and its associated (public ? data) > > structures > > > that should be created by constructors once and as much as poss. reused > > > through repeated calls to CalcPosteriorDensity (e.g. EstPeriods so that > we > > > know how many periods/time slots we need to estimate) ). There is also > > some > > > redundancy in the schema so to improve efficiency too. > > > > > > I wanted your opinion before I upload it to Wiki page - let me know if > you > > > would like more info. > > > > > > Best regards > > > > > > George > > > ----- Original Message ----- > > > From: "Michel Juillard" <michel.juillard(a)ens.fr> > > > To: "List for Dynare developers" <dev(a)dynare.org> > > > Sent: Thursday, December 10, 2009 8:11 PM > > > Subject: Re: [DynareDev] Proposed specification for estimation functions > > > > > > > > > > Dear George, > > > > > > > > I'm not sure that it needs to be that complicated. > > > > The estimation algorithm (optimization or MCMC) see all the parameters > > > > for all sub-periods as a single vector. > > > > > > > > In updateQHparams(), it is possible to compare the date of the > relevant > > > > subperiod with StartPeriod and EndPeriod. I don't think that such > tests > > > > would be very costly. > > > > > > > > If I'm wrong and that we need after all the structures that you > suggest, > > > > we will add them later. > > > > > > > > All the best, > > > > > > > > Michel > > > > > > > > G. Perendia wrote: > > > > > Thanks Michel > > > > > As parameters can have different time slots I think we will need to > > add > > > two > > > > > structures and: > > > > > 1 one to )organise data around individual parameters having a vector > > of > > > > > descriptions for each of their time slots, and > > > > > 2) an additional structure listing all identified longest common > > periods > > > > > pointing to each parameter's description for that slot. > > > > > > > > > > Thus, > > > > > re (1) instead of xparam1 being vector of doubles and e.g. > > start_period > > > > > vectors having n entries, xpramss1 will be a vector of vectors of > > > > > structures, and, > > > > > re (2) timeslots will be a vector of common periods of > > unpredetermined > > > > > size, of vectors of length n containing n indices to the relevant > > > > > parameter's description for that period including its value and > > priors. > > > > > > > > > > Best regards > > > > > > > > > > George > > > > > > > > > > ----- Original Message ----- > > > > > From: "Michel Juillard" <michel.juillard(a)ens.fr> > > > > > To: "List for Dynare developers" <dev(a)dynare.org> > > > > > Sent: Thursday, December 10, 2009 10:58 AM > > > > > Subject: Re: [DynareDev] Proposed specification for estimation > > functions > > > > > > > > > > > > > > > > > > > >> Hi George, > > > > >> > > > > >> performance improvement was the idea. Lets keep the scalars as they > > are > > > > >> used to distinguish different types of estimation problems. You can > > > > >> leave aside the vector of indices until we encounter the need for > > them. > > > > >> > > > > >> Best > > > > >> > > > > >> Michel > > > > >> > > > > >> G. Perendia wrote: > > > > >> > > > > >>> Hi > > > > >>> > > > > >>> It seems to me that the following values and vectors described on > > the > > > > >>> > > > > > (new) > > > > > > > > > >>> EstimationModule wiki page are in a way redundant as it looks like > > > that > > > > >>> > > > > > they > > > > > > > > > >>> can > > > > >>> be derived from the parameter_description type vector. If so, is > > there > > > > >>> > > > > > any > > > > > > > > > >>> other > > > > >>> reason for keeping them explicitly as part of the structure except > > > that > > > > >>> > > > > > it > > > > > > > > > >>> is useful for performance improvement and for easier understanding > > > > >>> of the structure? > > > > >>> > > > > >>> - nvx integer scalar (number of estimated structural shock > standard > > > > >>> deviations). > > > > >>> -. nvx_id nvx by 1 vector of integers. > > > > >>> -. nvn integer scalar (number of estimated measurement error > > standard > > > > >>> deviations). > > > > >>> -. nvn_id nvn by 1 vector of integers. > > > > >>> -. ncx integer scalar (number of estimated structural shock > > > > >>> > > > > > correlations). > > > > > > > > > >>> -. ncx_id ncx by 1 vector of integers. > > > > >>> -. ncn integer scalar (number of estimated measurement error > > > > >>> > > > > > correlations). > > > > > > > > > >>> -. ncn_id ncn by 1 vector of integers. > > > > >>> -. np integer scalar (number of estimated "deep" parameters). > > > > >>> -. np_id np by 1 vector of integers. > > > > >>> > > > > >>> Best regards > > > > >>> > > > > >>> George > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> _______________________________________________ > > > > >>> Dev mailing list > > > > >>> Dev(a)dynare.org > > > > >>> http://www.dynare.org/cgi-bin/mailman/listinfo/dev > > > > >>> > > > > >>> > > > > >>> > > > > >> _______________________________________________ > > > > >> Dev mailing list > > > > >> Dev(a)dynare.org > > > > >> http://www.dynare.org/cgi-bin/mailman/listinfo/dev > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > Dev mailing list > > > > > Dev(a)dynare.org > > > > > http://www.dynare.org/cgi-bin/mailman/listinfo/dev > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > Dev mailing list > > > > Dev(a)dynare.org > > > > http://www.dynare.org/cgi-bin/mailman/listinfo/dev > > > > > >

1 0

[Trac] #48: Incorrect treatment of log-linear transformation for purely backward looking models
by Trac Dynare 19 Dec '09

19 Dec '09

#48: Incorrect treatment of log-linear transformation for purely backward looking models --------------------------+------------------------------------------------- Reporter: sebastien | Owner: Type: bug | Status: new Priority: major | Milestone: 4.1 Component: Core M-files | Version: 4.0.4 Keywords: | --------------------------+------------------------------------------------- In dr1.m, log-linear test and the log-linear transformation of the model ghx and ghu matrices at line 449 are not reached for a purely backward looking model, because of a return statement at line 266 -- Ticket URL: <https://www.dynare.org/trac/ticket/48> Dynare <http://www.dynare.org> The Dynare project

1 1

(no subject)
by Tao Zha 19 Dec '09

19 Dec '09

Dear Michel: Attached is my revised documentation for the Dynare interface of SWZ. Feel free to expand or modify it. Two more things. 1) The request for the Atlanta Fed's commitment to Dynare's membership is moving forward. I'm awaiting for an official answer from the financial planning department here. 2) Perhaps we can arrange Houton to come to the Atlanta fed for a few things to wrap up the SWZ stuff. January and February can be busy for me as I have to help recruiting for both institutions (Fed and Emory). I wonder if some time in March is good for Houton. If not, we can certainly have Houton visiting earlier. I'm just stating my preference and do not want to rule out any particular month. Hope all is well with you and your family. Merry Christmas, Tao

1 0

[Trac] #70: USE_DLL option affected by the timestamp directory bug of Octave 3.2
by Trac Dynare 18 Dec '09

18 Dec '09

#70: USE_DLL option affected by the timestamp directory bug of Octave 3.2 --------------------------+------------------------------------------------- Reporter: sebastien | Owner: sebastien Type: bug | Status: new Priority: major | Milestone: 4.1 Component: Preprocessor | Version: Keywords: | --------------------------+------------------------------------------------- -- Ticket URL: <https://www.dynare.org/trac/ticket/70> Dynare <http://www.dynare.org> The Dynare project

1 1

[Trac] #71: rplot command fails on model with no lags
by Trac Dynare 18 Dec '09

18 Dec '09

#71: rplot command fails on model with no lags --------------------------+------------------------------------------------- Reporter: sebastien | Owner: Type: bug | Status: new Priority: minor | Milestone: 4.1 Component: Core M-files | Version: Keywords: | --------------------------+------------------------------------------------- -- Ticket URL: <https://www.dynare.org/trac/ticket/71> Dynare <http://www.dynare.org> The Dynare project

1 1

Re: [DynareDev] 4.1 branched out
by Sébastien Villemot 18 Dec '09

18 Dec '09

Le jeudi 17 décembre 2009 à 23:42 +0000, G. Perendia a écrit : > Running make with new trunk code I am getting an error when it reach > building the k_order_perturbation: > > make[2]: *** No rule to make target > '../../../sources/k_order_perturbation/k_order_perturbation.cpp' needed by > 'k_order_perturbation.o' Stop. > > although the .cpp and .h files in sources/k_order_perturbation have been > replaced by the new .cc and .hh files. You need to run "make distclean", then re-configure, and re-compile. It seems that automake doesn't handle well such renaming of source files... Best -- Sébastien Villemot

1 0

4.1 branched out
by Sébastien Villemot 16 Dec '09

16 Dec '09

Hi, After having reindented and beautified all Dynare code, I created the branch for Dynare 4.1. This marks the beginning of the test phase for Dynare 4.1, and the opening of a new development cycle (until Dynare 4.2 is released). You are therefore free to add new features to the trunk or to start deep refactoring. Please test the 4.1 branch. If you spot bugs, please fix them in the trunk, and I will merge them in the branch. There are still 5 open bugs against Dynare 4.1 in Trac, we need to fix them before we can release Dynare 4.1.0. Best, -- Sébastien Villemot

1 0

Re: [DynareDev] [Trac] #16: Reindent all source code (C++, M-files)
by Trac Dynare 16 Dec '09

16 Dec '09

#16: Reindent all source code (C++, M-files) ---------------------------+------------------------------------------------ Reporter: sebastien | Owner: sebastien Type: enhancement | Status: closed Priority: trivial | Milestone: 4.1 Component: Core M-files | Version: Resolution: fixed | Keywords: ---------------------------+------------------------------------------------ Changes (by sebastien): * status: accepted => closed * resolution: => fixed -- Ticket URL: <https://www.dynare.org/trac/ticket/16#comment:3> Dynare <http://www.dynare.org> The Dynare project

1 0

[Trac] #66: Document the fast deterministic simulations in the reference manual
by Trac Dynare 16 Dec '09

16 Dec '09

#66: Document the fast deterministic simulations in the reference manual ---------------------------+------------------------------------------------ Reporter: sebastien | Owner: sebastien Type: feature | Status: new Priority: major | Milestone: 4.1 Component: Documentation | Version: Keywords: | ---------------------------+------------------------------------------------ At least we should incoporate the new options described on: http://www.dynare.org/DynareWiki/FastDeterministicSimulationAndSteadyStateC… -- Ticket URL: <https://www.dynare.org/trac/ticket/66> Dynare <http://www.dynare.org> The Dynare project

1 1

Re: [DynareDev] [Trac] #12: Compute temporary terms when using option block_mfs of steady
by Trac Dynare 16 Dec '09

16 Dec '09

#12: Compute temporary terms when using option block_mfs of steady ---------------------------+------------------------------------------------ Reporter: sebastien | Owner: sebastien Type: enhancement | Status: closed Priority: minor | Milestone: 4.1 Component: Preprocessor | Version: Resolution: fixed | Keywords: ---------------------------+------------------------------------------------ Changes (by sebastien): * status: accepted => closed * resolution: => fixed Comment: Fixed in r3244. -- Ticket URL: <https://www.dynare.org/trac/ticket/12#comment:2> Dynare <http://www.dynare.org> The Dynare project

1 0

[Trac] #64: Fix USE_DLL option at order 2
by Trac Dynare 14 Dec '09

14 Dec '09

#64: Fix USE_DLL option at order 2 --------------------------+------------------------------------------------- Reporter: sebastien | Owner: Type: bug | Status: new Priority: minor | Milestone: 4.1 Component: Core M-files | Version: Keywords: | --------------------------+------------------------------------------------- Currently the MEX returns a hessian made of 3 columns (sparse representation: two columns for indices, one column for values). A call to sparse() function needs to be added in dr1.m when USE_DLL option is used. Maybe a field options_.use_dll should be added by the preprocessor to facilitate the test in dr1.m. -- Ticket URL: <https://www.dynare.org/trac/ticket/64> Dynare <http://www.dynare.org> The Dynare project

3 3

Re: [DynareDev] [Trac] #3: Add interface to k-order DLL, up to order 3, using model compiled in C
by Trac Dynare 11 Dec '09

11 Dec '09

#3: Add interface to k-order DLL, up to order 3, using model compiled in C --------------------------+------------------------------------------------- Reporter: sebastien | Owner: michel Type: feature | Status: closed Priority: major | Milestone: 4.1 Component: K-order DLL | Version: Resolution: fixed | Keywords: --------------------------+------------------------------------------------- Changes (by sebastien): * status: accepted => closed * resolution: => fixed -- Ticket URL: <https://www.dynare.org/trac/ticket/3#comment:6> Dynare <http://www.dynare.org> The Dynare project

1 0

3rd order and USE_DLL option under Windows
by Sébastien Villemot 11 Dec '09

11 Dec '09

Hi, I made a change in the preprocessor so that 3rd order approx, and USE_DLL option, which both require the on-the-fly compilation of a MEX file. These options now correctly work with both Cygwin and Microsoft Visual C++ under Windows. For those of you who use Cygwin, you need to update your mexopts.bat with the one I just put in the SVN in windows/ subdirectory. I also wrote a wiki page on the topic: http://www.dynare.org/DynareWiki/ConfigureMatlabWindowsForMexCompilation Best -- Sébastien Villemot

1 0

Release of 4.1
by Sébastien Villemot 11 Dec '09

11 Dec '09

Hi, We are planning to release 4.1 very soon, before Christmas. I have reassigned to milestone 4.2 various Trac tickets which were initially planned for 4.1, and that we won't have the time to deal with before Christmas. Please pay particular attention not to break anything in the trunk, until we create the branch for 4.1. Also note that before creating the 4.1 branch, I plan to reindent all the C++ and Matlab code. Best -- Sébastien Villemot

1 0

Re: [DynareDev] [Trac] #5: Implement explicit writing of first order conditions of Ramsey problem
by Trac Dynare 11 Dec '09

11 Dec '09

#5: Implement explicit writing of first order conditions of Ramsey problem ---------------------------+------------------------------------------------ Reporter: sebastien | Owner: sebastien Type: enhancement | Status: accepted Priority: major | Milestone: 4.2 Component: Preprocessor | Version: Resolution: | Keywords: ---------------------------+------------------------------------------------ Changes (by sebastien): * milestone: 4.1 => 4.2 -- Ticket URL: <https://www.dynare.org/trac/ticket/5#comment:2> Dynare <http://www.dynare.org> The Dynare project

1 0

Re: [DynareDev] Proposed specification for estimation functions
by G. Perendia 10 Dec '09

10 Dec '09

PS: Re naming suggestions: As M_.params is updated too in the 50-123 block, name update_params is also adequate though something like updateQHMparams() would still be more precise . ----- Original Message ----- From: G. Perendia To: List for Dynare developers Sent: Monday, September 21, 2009 10:27 AM Subject: Re: [DynareDev] Proposed specification for estimation functions Thanks Stephane Re: 3&4) Sorry for confusion: Overlapping variables refer to the results of Kalman from the previous period sub-sample block such as a(t|t-1) and P(t|t-1) and will not they be used as starting point entries for the next period block estimation. However, I am not sure if then their "meaning" for the next sub-sample may be distorted by separate data filtering (DataPreparation) for that sub-sample. I am however also not sure if I can take this into C++ DsgeLikelihood as yet in full - there are many unresolved items (and even more not yet properly understood by me, see below) - but I can at least start building bones for its future integration. E.g. have quite a few more questions: 5) How are we going to calculate and report overall (or sub-sample?) likelihood(s) to the parameter optimising/sampling DsgeLikelihood driving function (eg csminwel, fminunc or MCMC MH) and how to get it to optimise/sample parameters for different sub-samples? Or, is the parameter differentiation parametric/functional and planned to be included in "update_parameters" block? 6) How will the system know which name structure to target ID1 if "ID1 is a n by 1 vector of indices targeting to the (parameter, exogenous variable or endogenous variable) names (in M_.param_names, M_.exo_names and M_.endo_names" 7) If "the considered subsamples for different parameters may be different", then I believe we need to split the data subsamples and the estimation blocks into shorter common sub-samples. I.e., when you say: "xparam1 is a column n by 1 vector holding the current values of the estimated parameters" does "current" mean value at time t such that max(StartPeriod)< t < min(EndPeriod) for the set? 8) Are priors time-dependent, defined for each sub-sample (or the time slot of the parameter? Are they updated on the run, conditional on previous sub-sample or preset? 9) what do nv*_id nv* by 1 vector of integers represent? If shock IDs, then how the nv* shocks/errors can be linked to individual params' periods. I.e. are they time dependent too and if not, what is their role in this time-slot-dependent structure? 10) In the loop you refer to "NP" as number of periods: "LOOP on periods 1 to NP" but in the structure (lowercase) "np" seem to refer to the number of deep parameters- use of same word/literal (despite of the case difference) may be a bit confusing 11) which optimiser is the method of indirect inference in Dynare? May I also make some suggestions? re 4) Block of lines 50-123 updates and checks shocks and obs. errors cov kalman matrices Q and H using relevant subsets of the parameters vector xparam1 (but not changing xparam1). I believe a more precise and descriptive name for the block (e.g. updateQH() or updateShocksAndErrorMatrices() or something on those lines) may be less confusing with e.g. generating or updating the whole new set of all parameters. Also, DataCleanup, DataDetrend or DataPreparation instead of "data_filtering()" I believe may be less confusing with e.g. Kalman filtering. I hope this helps to clarify rather than confuse the issues even more. Best regards George ----- Original Message ----- From: Stéphane Adjemian To: List for Dynare developers Sent: Friday, September 18, 2009 12:09 PM Subject: Re: [DynareDev] Proposed specification for estimation functions Hi, I am not sure that it would be a good idea to implement the unexpected breaks stuff in the c++ version of the DsgeLikelihood... Because it's a new feature and we have no experience with it. I Changed Period1 to StartPeriod and Period2 to EndPeriod, thanks for the suggestion. In the data filtering step we remove the "deterministic part" of the data (constant and linear trends), so that we do not have to treat this in the kalman filter recursions. This is already done like this in the matlab version of DsgeLikelihood. I do not understand 3) and 4). What is an overlapping variable. The parameter update step corresponds to block between lines 50 to 123 in DsgeLikelihood.m I think that we do have a c version of the sims optimization routine (provided by Tao Zha). Best, Stéphane. 2009/9/18 G. Perendia <george(a)perendia.orangehome.co.uk> Hi I would like to start implementing period-dependent likelihood calculation within the new C++ DsgeLikelihood but that will add a bit of time. 1) I suggest that parameters description is defined by Start (or StartPeriod) and End (or EndPeriod) instead of a.. Period1 (beginning of sub-period) a.. Period2 (end of sub-period) respectively because we may have more than 2 periods and using terms Period1 and 2 as delimiters for other periods can bee a bit confusing in the context (as it did confuse me for a start) . 2) What is data_filtering (for period 1) step as opposed to Kalman filter - is it Kalman smoother? We have no working C++ implementation for smoother as yet. 3) Would it be right to say that the new period divided likelihood estimation requires overlapping variables Y(-/+n) for n>0 among the sub-period blocks? 4) What is parameters update? Is it param likelihood, hill climbing optimisation (or MCMC step) for each sub-period block separately (conditional on the previous period block?)? We do not have working optimiser in C++ yet. 4) Would the interim period-blocks' Kalman inputs be a(t|t-1) and P(t|t-1) as well as overlapping variables Y(-/+n) for n>0 from previous block Kalman output? Best regards George _______________________________________________ Dev mailing list Dev(a)dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev -- Stéphane Adjemian CEPREMAP & Université du Maine Tel: (33)1-43-13-62-39 ---------------------------------------------------------------------------- _______________________________________________ Dev mailing list Dev(a)dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev

2 7

Re: [DynareDev] [Trac] #3: Add interface to k-order DLL, up to order 3, using model compiled in C
by Trac Dynare 09 Dec '09

09 Dec '09

#3: Add interface to k-order DLL, up to order 3, using model compiled in C --------------------------+------------------------------------------------- Reporter: sebastien | Owner: michel Type: feature | Status: accepted Priority: major | Milestone: 4.1 Component: K-order DLL | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment(by sebastien): Almost done as of r3216. The last thing to do is to try to make Microsoft Visual C++ work with k_order_solver, this is mandatory for 64-bit MATLAB for Windows. This involves modifying the mexopts.bat for testing purposes, and if successful, update the preprocessor (maybe with a preprocessor option to switch between g++/MSVC), and the reference manual. -- Ticket URL: <https://www.dynare.org/trac/ticket/3#comment:5> Dynare <http://www.dynare.org> The Dynare project

1 0

k_order_perturbation under Windows
by Michel Juillard 08 Dec '09

08 Dec '09

k_order_perturbation DLL works under Windows XP and Octave when compiled under Cygwin (with gcc-4), but it is necessary to add -D_WIN32 when compiling, in order to avoid the dlopen/dlclose functions. If these functions are present, Octave refuses to load the DLL. I'm going to check with Matlab now Best Michel

3 8

Re: [DynareDev] SWZ Question regarding max_lag_length r
by Michel Juillard 04 Dec '09

04 Dec '09

Hi Houtan, > Dear Michel and Sébastien, > > Everything is coded and tested for the svar_identification. However, I > want to check my implementation of the maximum lag length (r) with > you. In order to obtain r, I have introduced a temporary variable > entitled max_endo_lag in ParsingDriver, which updates this variable > every time add_model_variable() is called. Is this the best way to do > this or can you think of a better way. I should point out that it is > my understanding that the variables entitled like max_endo_lag in > DynamicModel do not actually contain the original maximum endogenous > lag in the model (which is what I understand should be the value for > r). Otherwise, I would have simply included dynamic_model in the > SvarIdentificationStatement class. > this is true. (S)VAR will be the only type of models that are left with lags on more than one period. > One problem with the implementation I have chosen is that we would > need to ensure that the model block is declared before the > svar_identification block in the .mod file. I noticed that, currently, > though in the manual we are instructed as to the proper declaration > order within the .mod file (i.e. vars, params, model, ..., > estimation/stoch_simul), this order is not enforced within Dynare. > In fact, certain mixes of this ordering work (e.g. putting the initval > statement above the model block) whereas certain orderings fail > without any sort of descriptive message (e.g., ... shock block, > initval block, model block ...). In my opinion, enforcing some sort of > ordering and exiting with a warning to the user would be nicer than > simply allowing dynare to crash. > > So, if the implementation described in paragraph 1 that obtains the > maximum endogenous variable lag length by constantly updating this > variable in add_model_variable is correct, we must ensure that the > model block has been declared before the svar_identification block so > that this variable will have been set. That said, I welcome any ideas > that would solve this problem more efficiently than the method > described herein. They are in fact very few strict order dependencies that need to be enforced and checked. The manual gives more an advice than an obligation. For, (S)VAR, we don't need the model block and we need to relax this requirement in the preprocessor. In this particular case, I don't think that we need to enforce any consistency. It makes sense for example to declare identification restrictions on many lags and then decide to estimate with a smaller number of lags in the VAR, without asking the user to adjust the SVAR_IDENTIFICATION block. You probably need to create an SvarIdentificationStatement class similar to i.e. InitParamStatement in NumericalInitialization.hh that record all the identification restrictions before the output stage. You can declare there a private variable with the maximum number of lags used in the svar_identification block. Sebastien, what do you think? Best Michel You need however the maximum number of lags used in the SVAR_IDENTIFICATION block to dimension Ri

3 5

expectational operators
by G. Perendia 04 Dec '09

04 Dec '09

Hi Can anyone let me know what is the status of implementation of the expectational operators (in particular E_t,t ) and when will they be ready for use in our dev version as they are needed for full implementation of Partial Information? Also, is their syntax going to follow closely what is on the wiki page NewOperators? Best regards George

3 3

SWZ
by Michel Juillard 03 Dec '09

03 Dec '09

I have updated SWZ in SVN to include changes that were already present in what I presented last week. I have added to the wiki a detailed description of the interface for SVAR exclusion restriction in http://www.dynare.org/DynareWiki/SvarExclusionInterface the Qi and Ri matrices should be implemented by Houtan in the preprocessor. Karl and Stephane have just to write a small identification/exclusions.m and identification/lower_cholesky.m All the best, Michel

3 7

k_order_perturbation under Windows
by Michel Juillard 03 Dec '09

03 Dec '09

Configuring my Matlab compiler as LCC doesn't work. I can't load fs2000k2_dynamic.mexw32 even directly from Matlab. George, Sebastien, You should first verify that the DLLs createed with use_dll work indeed with Matlab before attemping to debug k_order_perturbation.cpp Even if it isn't that, the problem is most likely with the calling of Dynamic DLL. This is the only part that is different under Windows and Linux. Best Michel

2 1

Re: [DynareDev] Fw: [Trac] #60: Fix memory leaks in k_order_perturbation DLL
by Sébastien Villemot 03 Dec '09

03 Dec '09

Le lundi 30 novembre 2009 à 17:50 +0000, G. Perendia a écrit : > Is there any more info on this issue? Please see my comment associated to the #60 issue on the trac system. Please answer there if you want to continue the discussion. -- Sébastien Villemot

3 6

Re: [DynareDev] C++ DsgeLikelihood and estimation issues
by G. Perendia 02 Dec '09

02 Dec '09

Hi Sebastien I now have a pretty stable working prototype of C++ DsgeLikelihood suitable for initial indicative performance tests. Unfortunately, during the lengthy debugging process, I realised I had to take out some performance improvers built as overloads for the GeneralMatrix copy() method, originally for improving the Kalman Filter performance, since they were breaking (now integrated) K-order-perturbation and hence I will need to devise and develop some alternative performance booster methods. However, some very rough performance comparison tests with fs2000 model even with such degraded KF indicate to be pretty promising and worth of the further effort: whilst 1000 loop calling Matlab DsgeLikelihood.m (all with same parameters) takes randomly somewhere between 40 and 50 sec, calling the equivalent CalcLikelihood method from within parent C++ mexFunction (together with constructing its parent class once as it would be done by the C++ estimation) takes approx. 10 sec (thus, less than 1/4th of time). This was run on an old and slower machine but the relative advantage is pretty portable. That, (together with some additional performance boosting) may result in a rather substantial time saving for whole MLE/MCMC estimation! Best regards George

3 4

[Trac] #61: error message for histval syntax
by Trac Dynare 01 Dec '09

01 Dec '09

#61: error message for histval syntax --------------------------+------------------------------------------------- Reporter: michel | Owner: sebastien Type: bug | Status: new Priority: minor | Milestone: 4.1 Component: Preprocessor | Version: Keywords: | --------------------------+------------------------------------------------- if one writes wrongly histval; y = 1; end; instead of histval; y,1; end; there is no error message with the line number where the error occurs -- Ticket URL: <https://www.dynare.org/trac/ticket/61> Dynare <http://www.dynare.org> The Dynare project

1 1

Fw: disclyap_fast issue
by G. Perendia 30 Nov '09

30 Nov '09

Dear Michel It seems that disclyap_fast is not returning results for good reasons whilst Dynare solver lyapunov_symm still returns some acceptable results which may or may not result in incorrect estimates. The problem with NAN was debugged from C++ version of DsgeLikelihood as I was trying to use fs2000k model and the non-pos-def with fs2000a (though either of which models are not the most suitable anyway) . The pos-def issue occurred in Matlab Dynare as I was running disclyap_fast with check option, something not currently imposed in either C++ DsgeLikelihood nor in lyapunov_symm. But I wander what we should do in such situations if it occur in general - stick to disclyap_fast diagnosis and return penalty from DsgeLikelihood in case results are NAN (which seems adequate solution at least for time being) or poss. change C++ lyapunov solver? Also, - shall we check if Pstar is positive definite and also abandon with penalty or continue as is done at the moment? Best regards George ----- Original Message ----- From: "joe pearlman" <j.pearlman(a)londonmet.ac.uk> To: "G. Perendia" <george(a)perendia.orangehome.co.uk> Sent: Thursday, November 26, 2009 2:38 PM Subject: Re: disclyap_fast issue > > Dear George > > there is quite obviously an eigenvalue of 1 in the T matrix, so the > Lyapunov matrix cannot posibly converge! > > Joe > > G. Perendia wrote: > > > > > Dear Joe > > > > I have encountered a problem with disclyap_fast returning NAN matrix > > when another Dynare solver returns still suitable results for KF. Please > > see the enclosed file giving input and outputs. > > > > Do you have any suggestions on how to overcome such issue? > > > > Best regards > > > > George

3 4

Follow up on Markov Switching meeting
by Sébastien Villemot 26 Nov '09

26 Nov '09

Hi, Following our meeting on Markov Switching code: * Tao, Junior, Karl and Stéphane L. are now subscribed to the list of Dynare developers * On the wiki, I renamed the page on SWZ options, and created a page with the proposal for the interface (incorporating some of the changes that we discussed during the meeting). See: http://www.dynare.org/DynareWiki/MarkovSwitchingOptions http://www.dynare.org/DynareWiki/MarkovSwitchingInterface * I made a change to the preprocessor: using sbvar or ms_sbvar without a model block should now work Best, -- Sébastien Villemot

3 2

[Trac] #57: Translate or remove auxiliary variables in output of commands
by Trac Dynare 25 Nov '09

25 Nov '09

#57: Translate or remove auxiliary variables in output of commands --------------------------+------------------------------------------------- Reporter: sebastien | Owner: sebastien Type: enhancement | Status: new Priority: major | Milestone: 4.1 Component: Core M-files | Version: Keywords: | --------------------------+------------------------------------------------- -- Ticket URL: <https://www.dynare.org/trac/ticket/57> Dynare <http://www.dynare.org> The Dynare project

1 1

[Trac] #36: Write a Wiki page on the various incompatibilities accross MATLAB versions
by Trac Dynare 23 Nov '09

23 Nov '09

#36: Write a Wiki page on the various incompatibilities accross MATLAB versions ---------------------------+------------------------------------------------ Reporter: sebastien | Owner: Type: feature | Status: new Priority: minor | Milestone: Component: Documentation | Version: Keywords: | ---------------------------+------------------------------------------------ This wiki page should only document changes relevant to Dynare, in particular with respect to MEX files. All MATLAB release notes since R14 are available on: http://www.mathworks.com/access/helpdesk/help/techdoc/ -- Ticket URL: <https://www.dynare.org/trac/ticket/36> Dynare <http://www.dynare.org> The Dynare project

1 1

printf in Vector and GeneralMatrix
by G. Perendia 20 Nov '09

20 Nov '09

Hi I intend to change printf() calls in the <object>print() functions of some of the Dynare++ Sylv exceptions and Vector and GeneralMatrix to mexPrintf()and use following to switch between printf and mexPrintf() #ifdef MATLAB #include "mex.h" #else #define mexPrintf printf #endif Is that OK or will this cause any problems? Best regards George

3 5

General Matrix Issue
by G. Perendia 19 Nov '09

19 Nov '09

Hi Sebastien (et all) 1) Whilst trying to use one of already present public functions in sylv/General Matrix.h/cpp (multRightTrans) in DsgeLikelihood, the multiplication was crashing and I realised that there appear to be an issue with two private, underlying functions (gemm_partial_left and gemm_partial_right). They use a (fixed?) private static variable int md_length for number of rows/columns for partial-copy of the host matrix. The variable is in .cpp defined to be 32 i.e. int GeneralMatrix::md_length = 32; and has no apparent mechanism to be dynamically modified so, as it is, it appears that those functions may work only for matrices of specific size(s) - However, I may have missed something and someone may be able to throw some light how is md_length adjusted and/or properly used? 2) The gemm_partial_left and gemm_partial_right are used by the following public functions: multRight, multRightTrans multLeft multLeftTrans And then, all those are used on a couple of places by the (so far unused and untested) C++ Diffuse Kalman filters and in the kalman utils library TSUtils::correctDefinitness (also used by Diffuse Kalman filters) and, as well, by few classes in the sylv library, e.g. MultRight is then used by other classes such as QuasiTriangularZero and SchurDecompZero multiply-constructors. One would need to investigate further whether if any of those functions are being used currently by any of the current Dynare applications or any of the GeneralMatrix children classes (and there are few of those in tl_library) and, if needed poss. carefully modify their behaviour and/or use of md_length. Does anyone know of their direct use by other Dynare apps so far and/or has any experience of them? I also looked into Dynare++ and KroneckerProd but they do not appear there (other than within the sylv library) In meantime, for DsgeLikelihood I may need to write an alternative way of multRightTrans untill this issue is resolved. Should I log this as a bug/issue in Track and write an alternative multRightTranspose for DsgeLikelihood or investigate the issue straight on? Best regards George

3 6

Re: [DynareDev] General Matrix Issue
by G. Perendia 18 Nov '09

18 Nov '09

PS: 3) Glancing at the code it appear to me that the 2 gemm_partial_* functions were an attempt to optimise multiplication by doing it in parts but seem to be incomplete (unfinished?) and their four calling function (mult*) may need to be rewritten. I think that may be poss. done in a very simple, quick (but working) way, replacing calls to gemm_partial_*, far less effort than investigating their possible (mis)use or trying to fix their partial multiplication. ----- Original Message ----- From: G. Perendia To: List for Dynare developers Sent: Wednesday, November 18, 2009 2:16 PM Subject: Re: [DynareDev] General Matrix Issue Thanks Michel 1) My out-of-the-box Dynere build does not work smoothly and it is rather slow and cumbersome on my cygwin system. 2) However, trying to build (my local) k_order_pert test executable or the DLL both fail if gemm_partial_* are not present in GeneralMatrix.cpp as the linker fails to resolve the link from multRight/Left* to them so I guess Dynare++ would not build too although the problematic functions may not be called during the processing. Best regards George ----- Original Message ----- From: Michel Juillard To: List for Dynare developers Sent: Wednesday, November 18, 2009 10:48 AM Subject: Re: [DynareDev] General Matrix Issue Hi 1. Are you able to build whole of Dynare out of the box with configure and make files? 2. if that is the case, remove the two problematic functions and see what doesn' t build If it is only the Kalman filter just fix the functions Best Michel Juillard On Nov 18, 2009, at 11:32, "G. Perendia" <george(a)perendia.orangehome.co.uk> wrote: Hi Sebastien (et all) 1) Whilst trying to use one of already present public functions in sylv/General Matrix.h/cpp (multRightTrans) in DsgeLikelihood, the multiplication was crashing and I realised that there appear to be an issue with two private, underlying functions (gemm_partial_left and gemm_partial_right). They use a (fixed?) private static variable int md_length for number of rows/columns for partial-copy of the host matrix. The variable is in .cpp defined to be 32 i.e. int GeneralMatrix::md_length = 32; and has no apparent mechanism to be dynamically modified so, as it is, it appears that those functions may work only for matrices of specific size(s) - However, I may have missed something and someone may be able to throw some light how is md_length adjusted and/or properly used? 2) The gemm_partial_left and gemm_partial_right are used by the following public functions: multRight, multRightTrans multLeft multLeftTrans And then, all those are used on a couple of places by the (so far unused and untested) C++ Diffuse Kalman filters and in the kalman utils library TSUtils::correctDefinitness (also used by Diffuse Kalman filters) and, as well, by few classes in the sylv library, e.g. MultRight is then used by other classes such as QuasiTriangularZero and SchurDecompZero multiply-constructors. One would need to investigate further whether if any of those functions are being used currently by any of the current Dynare applications or any of the GeneralMatrix children classes (and there are few of those in tl_library) and, if needed poss. carefully modify their behaviour and/or use of md_length. Does anyone know of their direct use by other Dynare apps so far and/or has any experience of them? I also looked into Dynare++ and KroneckerProd but they do not appear there (other than within the sylv library) In meantime, for DsgeLikelihood I may need to write an alternative way of multRightTrans untill this issue is resolved. Should I log this as a bug/issue in Track and write an alternative multRightTranspose for DsgeLikelihood or investigate the issue straight on? Best regards George _______________________________________________ Dev mailing list Dev(a)dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev ---------------------------------------------------------------------------- _______________________________________________ Dev mailing list Dev(a)dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev

1 0

Re: [DynareDev] [Trac] #2: Add finite time variance decomposition
by Trac Dynare 12 Nov '09

12 Nov '09

#2: Add finite time variance decomposition ---------------------------+------------------------------------------------ Reporter: sebastien | Owner: stepan Type: feature | Status: closed Priority: major | Milestone: 4.1 Component: Core M-files | Version: Resolution: fixed | Keywords: ---------------------------+------------------------------------------------ Changes (by sebastien): * status: assigned => closed * resolution: => fixed -- Ticket URL: <https://www.dynare.org/trac/ticket/2#comment:2> Dynare <http://www.dynare.org> The Dynare project

1 0

Re: [DynareDev] partial info + estimation
by Sébastien Villemot 09 Nov '09

09 Nov '09

Hi George, Le samedi 07 novembre 2009 à 10:17 +0000, G. Perendia a écrit : > I will need a number reserved for Part info kalman_algo. I currently use a > literal PartInfoKF set to 11. The status of kalman_algo is somewhat hybrid: it is both an option that can be modified by the user (yet undocumented), and an intermediary result which is modified by the Matlab routines. I don't yet fully understand how this works, so I let Michel or Stéphane answer if this creates a problem. In the future, we should probably create a variable in oo_ which will store the modified value of kalman_algo; values in options_ should not be modified. Michel and Stéphane: by the way, what is the status of dsge_posterior_kernel.m? This function is not called by any other M-function nor by the preprocessor. > PS: Yesterday I uploaded the changes to Part info branch needed for PI > estimation (5 modified and 6 renamed files to avoid name conflicts) but I > did not get root(a)dynare.org email info. Your commit was registered and I got the email. I don't know why you didn't get the email, I suppose it was filtered by your anti-spam. Best -- Sébastien Villemot

3 2

[Trac] #49: User guide: update chapter on installation of Dynare
by Trac Dynare 06 Nov '09

06 Nov '09

#49: User guide: update chapter on installation of Dynare ---------------------------+------------------------------------------------ Reporter: sebastien | Owner: sebastien Type: bug | Status: new Priority: minor | Milestone: 4.1 Component: Documentation | Version: Keywords: | ---------------------------+------------------------------------------------ -- Ticket URL: <https://www.dynare.org/trac/ticket/49> Dynare <http://www.dynare.org> The Dynare project

1 1

[Trac] #39: Make warnings about unitialized params/vars optional
by Trac Dynare 06 Nov '09

06 Nov '09

#39: Make warnings about unitialized params/vars optional --------------------------+------------------------------------------------- Reporter: sebastien | Owner: sebastien Type: bug | Status: new Priority: minor | Milestone: 4.1 Component: Preprocessor | Version: Keywords: | --------------------------+------------------------------------------------- -- Ticket URL: <https://www.dynare.org/trac/ticket/39> Dynare <http://www.dynare.org> The Dynare project

1 1

[Trac] #50: Add dynare_simul_ to build_matlab.m
by Trac Dynare 06 Nov '09

06 Nov '09

#50: Add dynare_simul_ to build_matlab.m --------------------------------------+------------------------------------- Reporter: sebastien | Owner: sebastien Type: feature | Status: new Priority: minor | Milestone: 4.1 Component: Build system / Packaging | Version: Keywords: | --------------------------------------+------------------------------------- -- Ticket URL: <https://www.dynare.org/trac/ticket/50> Dynare <http://www.dynare.org> The Dynare project

1 1

Re: [DynareDev] C++ DsgeLikelihood
by Michel Juillard 04 Nov '09

04 Nov '09

Dear George, thanks for the message. The issue needs to be discussed carefully. 1) We are trying to get rid of most global variables and to pass oo_, options_ and M_ as arguments. There is no good reason, why this has not been done yet in DsgeLikelihood.m 2) Besides a cleaner more robust code, we learned that accessing global variables in Matlab is expensive, and, most likely it is also true for Mex library 3) DsgeLikelihood++ is not primarily meant for calling from Matlab, but from a C++ version of dynare_estimation_1.m 4) There are only two of these variables that are changed in DsgeLikelihood: - M_.params, but a local version of modified params vector should probably be an argument to dynare_resolve - options_.kalman_algo that passes back the choice of Kalman algorithm. We can probably find another mechanism to pass back this information to Matlab. 5) My preference would be to have one C++ class that contains all the parameters from M_, options_ and bayestopt_ (the latter needs to be rethought), needed by the C++ estimation functions. This class should be initialized at the instanciation of the top class (here dynare_estimation++) from M_ and options_ pointers passed as arguments 6) One needs to think about a similar mechanism for output variables. 7) I have three problems with the current structures that we use *M_ contains parameters describing the model (OK) *oo_ contains output. We may want to distinguish output that is only made available to the user and intermediary output that is meant to be used by other routines such as structure dr. It could make accessing these intermediary results faster * options_ contains both real options_ and information on the current state of Dynare computations (such as kalman_algo here), These should probably be separated * bayestopt_ (and estim_params) is currently a mess and should be reconsidered. It is mainly meant to contained parameters describing the priors that are deduced from user input passed by the preprocessor in estim_params 8) In summary, for now, in DsgeLikelihood++, you should simply refer to a C++ version of these objects that we will build later. Best Michel G. Perendia wrote: > Dear Michel > > Regarding design of the interface with Matlab (and Octave?), there are many > variables from global structures that need to be referred to from various > parts of estimation (and some updated too). > > Is it ok if I make DsgeLikelihood++ system refer to options_, oo_ and other > large structures (and some other variable) using mex functions > mexGetVariable (and, if needed, mexPutVariable for updates) rather than > trying to pass whole structures at the call time and creating C++ instances > of all or (large) subsets of their data. > > e.g. > const mxArray *mexExt = mexGetVariable("base", "mexext"); > or > const mxArray *M_ = mexGetVariable("caller", "M_"); > > That will reduce number of variables that need to be passed on call and > writing code on their breaking down and passing around from mexFunction to > the other modules. > > I can optimise access to read-once only to reduce DLL-Matlab traffic too > using a permanent class on the lines of: > > class MexStruct > { > mxArray* mexStruct; > const char *base; // one of: base, caller, global > const char *structName; > MexStruct( char *sBase, char * mexStructName) > : base(sBase), structName(mexStructName) > { > mexStruct=mexGetVariable(base, structName); > } > mxArray * > getMexStructField(char * field) > { > return mxGetField(mexStruct, 0, field); > } > void > putMexStruct(){mexPutVariable(base, structName, mexStruct);} > void > setMexStructField(char * field, mxArray *newVal) > { > mxArray *fp = mxGetField(mexStruct, 0, field); > mxDestroy(fp) > mxSetField(mexStruct,0,field, newVal); > } > } > > Some global variables in those structures also need to be set so the above > class' methods can pass back updated global variables too at the end of the > DsgaLikelihood. Also, as another advantage of this model, they can be passed > back even in real time when if that may be needed for parallelisation. > > Best regards > > George > Mob. +44(0)7951415480 > artilogica(a)btconnect.com > > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > To: "G. Perendia" <george(a)perendia.orangehome.co.uk> > Sent: Monday, August 24, 2009 11:15 AM > Subject: Re: C++ DsgeLikelihood > > > >> Hi George, >> >>> Dear Michel >>> >>> 1) Re email below, due to problem of code for calling >>> > <model>dynamic.dll, > >>> needed by estimation but being in the same file with mexFunction in >>> k_order_perturbation.cpp >>> >>> Alternatives: >>> a) I can use (a different) literal, e.g. >>> KORDERPERT_MEX_FILE >>> and compile k_order_perturbation.cpp for k_order_perturbation.dll using >>> -DKORDERPERT_MEX_FILE >>> and for DsgeLIkelihood/estimation without, - probably the easiest >>> > solution > >>> to implement and maintain until situation may change, - or, >>> >>> b) I can split k_order_perturbation.cpp in two - >>> > k_order_perturbation.cpp > >>> with mexFunction and another file for calling <model>dynamic.dll which >>> > will > >>> be placed into korderpertlib.a and linked to DsgeLikelihood/estimation >>> > too - > >>> probably the optimal solution. or, >>> >>> c) copy k_order_perturbation.cpp code for calling >>> > <model>dynamic.dll.into > >>> another file and keep it under DsgeLikelihood/estimation and have >>> DsgeLikelihood refer to k-orderpertlib for the rest - this may duplicate >>> effort on maintaining the dynamic.dll call management code but it is a >>> > clear > >>> cut. >>> >>> >>> >> I prefer 1.b) too >> >>> 2) Shall I treat C++ DsgeLIkelihood >>> a) as an extension of KF or KORDERPERT and put its code in one of those >>> > 2 > >>> mex/source directories, or >>> >>> b) create a new one, e.g. estimation, with its makefiile referring to >>> > the > >>> code and objects in the other two? >>> >>> >>> I suggest 1.b and to treat DsgeLIkelihood as a subdirectory of kalman >>> directory which will refer to korderpertlib and kalman src. >>> >>> >>> >> I prefer 2.b, because DsgeLikelihood calls Kalman code and it is weird >> to have it in a subdirectory of kalman >> >> All the best, >> >> Michel >> >>> Please let me know if that is OK with you. >>> >>> Best regards >>> >>> George >>> >>> ----- Original Message ----- >>> From: "G. Perendia" <george(a)perendia.orangehome.co.uk> >>> To: "Michel Juillard" <michel.juillard(a)ens.fr> >>> Sent: Friday, August 21, 2009 3:50 PM >>> Subject: Re: C++ DsgeLikelihood >>> >>> >>> >>> >>>> Dear Michel >>>> >>>> In k-order-perturbation makefile someone(you?) defined literal >>>> -DMATLAB_MEX_FILE >>>> for several cpp files, some of which do not contain mexFunction, e.g. >>>> >>>> k_order_test_main.o: k_order_test_main.cpp >>>> gcc -DMATLAB_MEX_FILE ... >>>> >>>> k_ord_dynare.o: k_ord_dynare.cpp k_ord_dynare.h >>>> gcc -DMATLAB_MEX_FILE .... >>>> >>>> k_order_perturbation.o: k_order_perturbation.cpp k_order_perturbation.h >>>> gcc -DMATLAB_MEX_FILE .... >>>> >>>> nlsolve.o: $(DYNAREPPDIR)/src/nlsolve.cpp >>>> gcc -DMATLAB_MEX_FILE .... >>>> >>>> whereas only k_order_perturbation.cpp contains mexFunction(). >>>> >>>> What is that switch/literal used for? >>>> >>>> I wandered if it is OK to use that literal and set a switch in >>>> k_order_perturbation.cpp to exclude mexFunction from compilation when >>>> > not > >>>> defined so that the file can be used with other mexFunctions such as >>>> > the > >>> one >>> >>> >>>> for DsgeLikelihood., e.g. >>>> >>>> #ifdef MATLAB_MEX_FILE // exclude mexFunction for other applications >>>> extern "C" { >>>> void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const >>>> > mxArray > >>>> *prhs[]) >>>> { >>>> ... here goes mexFunction .: >>>> } >>>> }; // end of extern C >>>> #endif // ifdef MATLAB_MEX_FILE >>>> >>>> Oterwise I can set another literal. >>>> >>>> Best regards >>>> >>>> George >>>> >>>> ----- Original Message ----- >>>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>>> To: "G. Perendia" <george(a)perendia.orangehome.co.uk> >>>> Sent: Tuesday, August 18, 2009 12:36 PM >>>> Subject: Re: C++ DsgeLikelihood >>>> >>>> >>>> >>>> >>>>> Hi George, >>>>> >>>>> currently there are 3 different cases: >>>>> >>>>> 1) the steady state is not affected by estimation. This is the case >>>>> > that > >>>>> we are implementing now >>>>> 2) A Matlab function is provided by the user to compute the steady >>>>> > state > >>>>> as a function of model's parameters. >>>>> The implementation is cumbersome for the user and should become a >>>>> feature of the preprocessor. As long as the definitive version is not >>>>> done, there is no point in losing time writing a C interface for that >>>>> 3) The steady state must be solved for by calling a nonlinear equation >>>>> solver. We will need to implement this as well, but it is a lower >>>>> priority. I would like to have the entire estimation chain in C++ >>>>> working first with just a constant steady state. >>>>> >>>>> All the best, >>>>> >>>>> Michel >>>>> >>>>> >>>>> G. Perendia wrote: >>>>> >>>>> >>>>>> Dear Michel >>>>>> >>>>>> Regarding steady state, can you remind me why we can not use ss >>>>>> >>>>>> >>>> calculation >>>> >>>> >>>>>> inside the k-order-perturb ? >>>>>> Can we than instead call Dynare Matlab steady state calculation, >>>>>> >>>>>> >>>> currently >>>> >>>> >>>>>> called from resol.m, from estimate_1.m just before calling the new >>>>>> >>>>>> >>> C++ >>> >>> >>>>>> DsgeLikelihood (or from a new Matlab shell DsgeLikelihood_m.m that is >>>>>> >>>>>> >>>> then >>>> >>>> >>>>>> calling C++ DsgeLikelihood) where new C++ resol will not call ss >>>>>> calculation as the Matlab one does? >>>>>> >>>>>> Best regards >>>>>> >>>>>> George >>>>>> >>>>>> ----- Original Message ----- >>>>>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>>>>> To: "G. Perendia" <george(a)perendia.orangehome.co.uk> >>>>>> Sent: Monday, August 17, 2009 8:37 PM >>>>>> Subject: Re: C++ DsgeLikelihood >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> Hi George, >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Should I suppose that the C++ DsgeLikelihood would also include >>>>>>>> > C++ > >>>>>>>> versions of: >>>>>>>> resol.m >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> yes, this mainly calls first order solution in dr1.m >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> kalman_transition_matrix.m >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> yes >>>>>>> >>>>>>> >>>>>>> >>>>>>>> and >>>>>>>> the 1st order part of the k-ord_pert version of dr1.m that will be >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> calling >>>>>> >>>>>> >>>>>> >>>>>>>> k-order_perturbation for 1st order only, either as a DLL (as it >>>>>>>> >>>>>>>> >>> does >>> >>> >>>>>> now) >>>>>> >>>>>> >>>>>> >>>>>>>> or incorporating it statically? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> first order solution should be linked statically >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> May I then also assume that >>>>>>>> >>>>>>>> a) set_state_space.m and the kstate preamble will in future be >>>>>>>> >>>>>>>> >>>> performed >>>> >>>> >>>>>>>> before calling DsgeLikelihood, and not repeatedly within dr1 and >>>>>>>> > its > >>>>>>>> >>>>>> future >>>>>> >>>>>> >>>>>> >>>>>>>> C++ version - as I remember has already been planned? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> yes, we will have to implement such a mechanism >>>>>>> >>>>>>> >>>>>>> >>>>>>>> b) Ramsey will not be part of it for now >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> correct >>>>>>> >>>>>>> >>>>>>> >>>>>>>> c) I need to check but I think I may not need to code >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> transition_matrix.m >>>>>> >>>>>> >>>>>> >>>>>>>> and use the info 3/4 test currently in dr1: >>>>>>>> if nba > nyf info(1) = 3; >>>>>>>> elseif nba < nyf info(1) = 4; >>>>>>>> as a similar test already exist in k-ord-pert. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> probably correct >>>>>>> >>>>>>> >>>>>>> >>>>>>>> d) Also, we seem to need a new algorithm to handle exogenous >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> deterministic >>>>>> >>>>>> >>>>>> >>>>>>>> variables i.e. when M_.exo_det_nbr > 0 ? They are not currently >>>>>>>> >>>>>>>> >>>> handled >>>> >>>> >>>>>> for >>>>>> >>>>>> >>>>>> >>>>>>>> k-order perturbation. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> no, there is no estimation with exogenous deterministic variables >>>>>>> >>>>>>> >>>>>>> Best >>>>>>> >>>>>>> Michel >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Best regards >>>>>>>> >>>>>>>> George >>>>>>>> >>>>>>>> ----- Original Message ----- >>>>>>>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>>>>>>> To: "G. Perendia" <george(a)perendia.orangehome.co.uk>; "Joe >>>>>>>> > Pearlman" > >>>>>>>> <j.pearlman(a)londonmet.ac.uk> >>>>>>>> Sent: Friday, July 31, 2009 8:46 AM >>>>>>>> Subject: Re: lyapunov kalman filter options >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> Thanks George, >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> .... >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> For the next steps, I think that you have now all elements to code >>>>>>>>> >>>>>>>>> >>> a >>> >>> >>>>>> C++ >>>>>> >>>>>> >>>>>> >>>>>>>>> version of DsgeLikelihood.m Of course you should restrict it to >>>>>>>>> > the > >>>>>>>>> >>>>>> case >>>>>> >>>>>> >>>>>> >>>>>>>>> of stationary models. >>>>>>>>> >>>>>>>>> As we haven't solved yet the issue around computing the steady >>>>>>>>> >>>>>>>>> >>> state, >>> >>> >>>>>>>>> for the time being, limit yourself to the case where the steady >>>>>>>>> >>>>>>>>> >>> state >>> >>> >>>>>>>>> doesn't change throughout estimation. >>>>>>>>> >>>>>>>>> Best >>>>>>>>> >>>>>>>>> Michel >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>>> >>> >>> >>> >>> >> > > > > >

3 17

Re: [DynareDev] C++ DsgeLikelihood and Dynare Parameters
by G. Perendia 04 Nov '09

04 Nov '09

Hi Michel Sorry, did you possibly want to keep estim_params separately anyway for some other reason, e.g. due to possible dynamic changes during the estimation process and/or due to the structural changes sub-periods? ----- Original Message ----- From: "G. Perendia" <george(a)perendia.orangehome.co.uk> To: "List for Dynare developers" <dev(a)dynare.org> Sent: Wednesday, November 04, 2009 5:09 PM Subject: Re: [DynareDev] C++ DsgeLikelihood and Dynare Parameters > > See my comments below > Best regards > George > > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > > > Ho George > > Why don't you keep estim_params_ as an intermediary structure that you > > add to your main structrure ? > > I do not quite understand the notion of "intermediary structure that you > add to your main structrure" and, so far, I am afraid that could complicate > the issue greatly since estim_params is a stand-alone (i.e. not part of > another structure such as dr1 is part of oo_). > > If I create separate structure for estim_params we are going back to many > structures situation. > > E.g. dr1 is special case, handled as a separate "sub-structure" variable > extracted from the main structure as its single field, (initially taken form > oo_). > > In general, I think that it is not good to start making exceptions to policy > agreed after long discussion aimed to achieve the single params structure > and here have one single parameter field which is causing problem which also > appear unused - it is left empty and there appear to be no references to the > estim_params_.param_names in Dynare matlab code! > > > > > Best > > > > Michel Juillard > > > > > > On Nov 4, 2009, at 17:02, "G. Perendia" <george(a)perendia.orangehome.co.uk > > > wrote: > > > > > Hi > > > > > > I am having problem creating single parameters structure for > > > DsgeLikelihood > > > as some names are not unique. > > > > > > I. e. I only encountered a problem with "param_names" field which > > > exists in > > > both, M_ and estim_params_ though it does not seem to be populated > > > and used > > > in the latter - can it be ignored (i.e. unused?), removed from > > > estim_params_ > > > or renamed to enforce name uniqueness? > > > > > > Best regards > > > > > > George > > > > > > > > > > > > > > > _______________________________________________ > > > Dev mailing list > > > Dev(a)dynare.org > > > http://www.dynare.org/cgi-bin/mailman/listinfo/dev > > > > > > > > > _______________________________________________ > > Dev mailing list > > Dev(a)dynare.org > > http://www.dynare.org/cgi-bin/mailman/listinfo/dev >

2 1

Request for feature (for Houtan)
by Sébastien Villemot 04 Nov '09

04 Nov '09

Hi Houtan, We want to add a "predetermined_variables" keyword that let people use predetermined variables at the current period. The problem is the following: currently in Dynare, variables decided at date "t" must be declared with a "t" subscript. The typical case is for capital stock: currently in Dynare, it must be declared using a beginning-of-period basis, which gives the following law of motion: k = i + (1-delta)*k(-1) However, some people want to have it on a end-of-period basis, which would look like: k(+1) = i + (1-delta)*k For the latter case, the user would then just write at the beginning of the MOD file: predetermined_variables k; The idea is that the first law of motion should be strictly equivalent to the second associated to the new keyword. The algorithm is very simple: the preprocessor should simply lag the variables declared as predetermined_variables by one period. I think this should consist in a new substitution pass, which would take place before all the existing substitution passes (expectation operator + Leads/lags). There is no need to create auxiliary variables; just lagging the corresponding variables by one and creating the new corresponding expressions is enough. During the parsing, we should take care that variables declared as predetermined_variables had been previously declared as endogenous in a var statement. The preprocessor should add a list "M_.predetermined_variables" in the generated M-file, for use by MATLAB routines. We need also to thing about the consequences for estimation and which observation period to use when a predetermined_variable is observed, but this is not something to be done from within the preprocessor, so you are not concerned. If possible, it would be great if this feature was implemented before you come to Paris. Don't hesitate to ask for more details. Best, -- Sébastien Villemot

3 3

[Trac] #41: bug in resid.m that reset initial values when used after ENDVAL
by Trac Dynare 29 Oct '09

29 Oct '09

#41: bug in resid.m that reset initial values when used after ENDVAL --------------------------+------------------------------------------------- Reporter: michel | Owner: michel Type: bug | Status: new Priority: critical | Milestone: 4.1 Component: Core M-files | Version: Keywords: | --------------------------+------------------------------------------------- resid shouldn't modify oo_ or any global variables Maybe it should check against <fname>_static rather than <fname>_dynamic -- Ticket URL: <https://www.dynare.org/trac/ticket/41> Dynare <http://www.dynare.org> The Dynare project

1 2

Re: [DynareDev] [Trac] #29: bug with varexo_det
by Trac Dynare 28 Oct '09

28 Oct '09

#29: bug with varexo_det ---------------------------+------------------------------------------------ Reporter: michel | Owner: michel Type: bug | Status: closed Priority: major | Milestone: 4.1 Component: Core M-files | Version: 4.0.4 Resolution: fixed | Keywords: ---------------------------+------------------------------------------------ Changes (by sebastien): * status: accepted => closed * resolution: => fixed Comment: Fixed in r3093. -- Ticket URL: <https://www.dynare.org/trac/ticket/29#comment:5> Dynare <http://www.dynare.org> The Dynare project

1 0

Re: [DynareDev] [Trac] #1: Implement conditional forecasts
by Trac Dynare 23 Oct '09

23 Oct '09

#1: Implement conditional forecasts ---------------------------+------------------------------------------------ Reporter: sebastien | Owner: stepan Type: feature | Status: closed Priority: major | Milestone: 4.1 Component: Core M-files | Version: Resolution: fixed | Keywords: ---------------------------+------------------------------------------------ Changes (by sebastien): * status: assigned => closed * resolution: => fixed -- Ticket URL: <https://www.dynare.org/trac/ticket/1#comment:3> Dynare <http://www.dynare.org> The Dynare project

1 0

Re: [DynareDev] [Trac] #6: Add interface to AIM solver
by Trac Dynare 20 Oct '09

20 Oct '09

#6: Add interface to AIM solver ---------------------------+------------------------------------------------ Reporter: sebastien | Owner: sebastien Type: feature | Status: closed Priority: major | Milestone: 4.1 Component: Preprocessor | Version: Resolution: fixed | Keywords: ---------------------------+------------------------------------------------ Changes (by sebastien): * status: accepted => closed * resolution: => fixed Comment: Implemented in r3059. -- Ticket URL: <https://www.dynare.org/trac/ticket/6#comment:2> Dynare <http://www.dynare.org> The Dynare project

1 0

order of variables
by Michel Juillard 20 Oct '09

20 Oct '09

Stephane makes me remark that the introduction of auxiliary variables create problems for people who use Matlab syntax in Dynare and rely on global such as M_.endo_nbr It is important that we define clearly the position of auxiliary variables in the vectors of variables and that we commit to it. We had decided, to put auxiliary variables after all variables declared explicitly in the model. We can add that the Lagrange multipliers added by ramsey_policy will be added after the auxiliary variables. All the best, Michel

3 3

Re: [DynareDev] [Trac] #1: Implement conditional forecasts
by Trac Dynare 16 Oct '09

16 Oct '09

#1: Implement conditional forecasts ---------------------------+------------------------------------------------ Reporter: sebastien | Owner: stepan Type: feature | Status: assigned Priority: major | Milestone: 4.1 Component: Core M-files | Version: Resolution: | Keywords: ---------------------------+------------------------------------------------ Comment(by sebastien): New interface has been implemented and documented in r3049. Note that for "plot_conditional_forecast", an argument of -1 means to M-function plot_icforecast() means that the user did not specified a number of periods, and that the plot should have as many periods as computed in "conditional_forecast" Still need to: * adapt M-functions according to the new calling syntax created by the preprocessor * verify entries in the reference manual (specially for default values of options) * update the wiki page [http://www.dynare.org/DynareWiki/ConditionalForecast] and move it to the 4.1 section on the "new features" page -- Ticket URL: <https://www.dynare.org/trac/ticket/1#comment:2> Dynare <http://www.dynare.org> The Dynare project

1 0

New identification files
by Sébastien Villemot 15 Oct '09

15 Oct '09

Hi Marco, Thanks for your identification routines. Just to remind you that you need to add copyright notices at the top of your new files, as explained on: http://www.dynare.org/DynareWiki/CodingStandards#Copyrightnotice-1 Best, -- Sébastien Villemot

3 8

loglinear models
by G. Perendia 12 Oct '09

12 Oct '09

Looking at DR1.m I noticed that the dr1.m code will not reach log-linear test and the log-linear transformation of the model ghx and ghu matrices (if needed) on line 449 if it is only backward looking due to a return statement on line 266. Is that intended behaviour? Best regards George

3 4

[Trac] #47: Uninformative error message when failing to create subdirectory
by Trac Dynare 12 Oct '09

12 Oct '09

#47: Uninformative error message when failing to create subdirectory --------------------------+------------------------------------------------- Reporter: sebastien | Owner: Type: bug | Status: new Priority: minor | Milestone: 4.1 Component: Core M-files | Version: 4.0.4 Keywords: | --------------------------+------------------------------------------------- If MOD file is <filename>.mod, CheckPath.m tries to create a directory called <filename>. If there already exists a file (not a directory) with that name (i.e. the same filename than the MOD file, but without extension), Dynare crashes and the error message is not very informative. A more explicit message should be displayed. See: http://www.dynare.org/phpBB3/viewtopic.php?f=1&t=2284&p=4507 -- Ticket URL: <https://www.dynare.org/trac/ticket/47> Dynare <http://www.dynare.org> The Dynare project

1 1

Re: [DynareDev] [Trac] #15: Add order 3 derivatives in USE_DLL mode (for k-order)
by Trac Dynare 12 Oct '09

12 Oct '09

#15: Add order 3 derivatives in USE_DLL mode (for k-order) ---------------------------+------------------------------------------------ Reporter: sebastien | Owner: sebastien Type: enhancement | Status: closed Priority: major | Milestone: 4.1 Component: Preprocessor | Version: Resolution: fixed | Keywords: ---------------------------+------------------------------------------------ Changes (by sebastien): * status: accepted => closed * resolution: => fixed Comment: Fixed in r3036 -- Ticket URL: <https://www.dynare.org/trac/ticket/15#comment:2> Dynare <http://www.dynare.org> The Dynare project

1 0

Re: [DynareDev] operators
by Michel Juillard 08 Oct '09

08 Oct '09

houtan wrote: > Dear Sébastien and Michel, > > I have made the requested changes but have a few minor questions. > > 1) As we are outputing the tsid as opposed to a variable name, I have > created oLatexDynamicSteadyStateOperator but allowed it to perform the > same operation as oMatlabDynamicSteadyStateOperator. Is this OK? > > 2) Though it is not in the specification on the Operator web page, I > have implemented oMatlabDynamicSteadyStateOperator for eExogenousDet. > In this case, it writes "oo_.exo_det_steady_state(...)". > > 3) Having made the changes, I gave it a first test with example1.mod, > placing the steady_state operater in different places to see if it was > parsing correctly. Though it seems to be parsing as per the rules > discussed on the web page, in Matlab I did continually encounter the > errors of the form: > ??? Undefined variable "oo_" or class "oo_.exo_steady_state". > ??? Undefined variable "oo_" or class "oo_.steady_state". > > I just wanted to bring this to your attention. > Thanks Houtan, you are right, there is a flaw in the specification: oo_ is not know to function <fname>_dynamic.m There are two solutions: 1) add steady_state to the list of arguments of this function. This is probably more elegant. The drawback is that we increase the number of arguments of the function for a rarely used feature 2) call Matlab global variable oo_ from inside <fname>_dynamic.m. The preprocessor can add the code only for those models that actually use the STEADY_STATE operator. In this case, it will be slower than 1), but the code doesn't change for the majority of cases. What do you think, everybody? Note that what we choose will need to be also implemented in the evaluator of the bytecode. Best Michel > I think that's it. Otherwise, the changes were pretty straightforward. > I have uploaded it to svn. > > Best, > Houtan > > > > > > 2009/9/10 Sébastien Villemot <sebastien.villemot(a)ens.fr > <mailto:sebastien.villemot@ens.fr>> > > Le mercredi 09 septembre 2009 à 19:47 -0400, houtan a écrit : > > For me, the only way we could have the desired output for a dynamic > > model (using the proposed procedure of modifying only > > UnaryOpNode::writeOutput() and dropping datatree.isDynamic()) is by > > using a modified version of IDEA 4 (shown below). In other words, I > > don't see how changing the output_type from dynamic to static will > > produce the dynamic model output (oo_.(exo_)steady_state(...)), > which > > is only located under case oMatlabOutsideModel. > > You're perfectly right, for the moment oMatlabOutsideModel is the only > one which creates the right output for Matlab code. It is my mistake, > sorry for creating confusion and for not having read carefully your > message. > > I think that we should create new types in the ExprOutputType > enumeration. For example we could create something like > oMatlabDynamicSteadyStateOperator, which would be used when > encountering > a steady state operator in the dynamic model. It might end up to do > exactly the same than oMatlabOutsideModel, but it is probably more > readable to use a different name since it's a different > functionnality. > > We also need to deal with three other types of output: > * C output in oCDynamicModel: since we are going to drop C output > in the > near future, if you encounter a steady state operator in that > mode, just > exit the preprocessor with a message like "not implemented" > * Matlab output for sparse model (oMatlabDynamicModelSparse): I > need to > discuss with Ferhat on how to deal with that: for the moment, just > exit > with a "not implemented" message > * Latex output for dynamic model (oLatexDynamicModel): here again we > should create a new output type (such as > oLatexDynamicSteadyStateOperator), which would add bars over variable > names (for variable $x$, it could write $\bar{x}$) > * Bytecode output: you need not to modify UnaryOpNode::compile(). We > should however modify the bytecode interpreter (bytecode DLL in > mex/sources/bytecode). I also need to discuss that with Ferhat, > just do > nothing for the moment about that. > > > > Also, might we still need the isDynamic method to use in > > UnaryOpNode::composeDerivatives()? > > You're probably right, I think you can't escape using this method at > this point. > > > As you can see, implementing operator is not so straightforward, so > don't worry if it takes times to converge to the best solution. > > Best, > > -- > Sébastien Villemot > > >

4 11

New build system
by Sébastien Villemot 07 Oct '09

07 Oct '09

Hi everyone, I have finished the new build system for Dynare. With a single "make" command you can now build Dynare preprocessor, MEX files for MATLAB and Octave, Dynare++, all the documentation. The procedure is described on: http://www.dynare.org/DynareWiki/BuildingDynareFromSource I would like to get your feedback about this, both in terms of functionality and of documentation. Two more platforms still need to be tested and documented: * Windows 64-bit, using MinGW64 * MacOS I will take care of Windows 64-bit. Houtan: when you have the time, could you test the procedure on MacOS? Maybe you could also write a section on the wiki page when you have finished. I will soon integrate the k-order DLL to that build system. Best, -- Sébastien Villemot

3 18

[Trac] #42: Fix problem of order 2 with leads of two or more
by Trac Dynare 07 Oct '09

07 Oct '09

#42: Fix problem of order 2 with leads of two or more --------------------------+------------------------------------------------- Reporter: sebastien | Owner: michel Type: bug | Status: new Priority: blocker | Milestone: 4.1 Component: Core M-files | Version: 4.0.4 Keywords: | --------------------------+------------------------------------------------- -- Ticket URL: <https://www.dynare.org/trac/ticket/42> Dynare <http://www.dynare.org> The Dynare project

1 2

Leads and lags of 2 or more
by Sébastien Villemot 06 Oct '09

06 Oct '09

Hi all, I have made an important change in the preprocessor, which should fix an important bug that Michel has discovered. It turns out that Dynare gives a wrong result for stochastic approximation at order 2 for models with leads of 2 or more. In order to fix that bug, the preprocessor now removes all leads and lags >=2 for a stochastic model. It does so by creating auxiliary variables and equations, so that the model has only one lead and one lag. This is the strategy used by Dynare++, and it is the easiest way to fix that bug. So don't be surprised if Dynare reports more equations than you declared, and if some variables called "AUXLEAD..." and "AUXLAG..." appear. You can examine the result of the transformation by using the "write_latex_dynamic_model" command (see http://www.dynare.org/DynareWiki/ModelLatexOutput ) The bug should now be fixed (although I now have a crash on GIMF in dr1.m). What remains to be done is the adaptation of the output routines, so that the output remains the same than before (instead of having the auxiliary variables in it). Houtan: for the auxiliary variables related to the expectation operator, you can reuse the framework that I have setup. * in SymbolTable.hh, add a new enum to aux_var_t, a new method for adding your type of aux var, and if needed new fields to AuxVarInfo * add a new method to class DynamicModel (similar to those for substituting leads and lags) * call that new method from ModFile::transformPass(), *before* the methods substituting leads and lags * for the initialization of new aux variables (since they are not declared in the "initval" block, the preprocessor needs to give an initial value to them): you need to add your new auxiliary equations to ModelTree::aux_equations; this should suffice, the mechanism that I have setup in DynamicModel::fillEvalContext() and StaticModel::writeAuxInitval() should take care of it Best, -- Sébastien Villemot

3 5

[Trac] #43: bug in 2nd order approximation
by Trac Dynare 25 Sep '09

25 Sep '09

#43: bug in 2nd order approximation --------------------------+------------------------------------------------- Reporter: michel | Owner: michel Type: bug | Status: new Priority: critical | Milestone: 4.1 Component: Core M-files | Version: Keywords: | --------------------------+------------------------------------------------- Change in the order of declaration of endogenous variables affects 2nd order approximation. Bug reported by Huixin 9/21 -- Ticket URL: <https://www.dynare.org/trac/ticket/43> Dynare <http://www.dynare.org> The Dynare project

1 2

pdf generators in C
by G. Perendia 22 Sep '09

22 Sep '09

Hi Does anyone know of good source code for pdf generators that is available in C/C++ for beta, gamma etc distributions? Best regards George

2 1

Re: [DynareDev] [Trac] #29: bug with varexo_det
by Trac Dynare 21 Sep '09

21 Sep '09

#29: bug with varexo_det ---------------------------+------------------------------------------------ Reporter: michel | Owner: michel Type: bug | Status: accepted Priority: major | Milestone: 4.1 Component: Core M-files | Version: 4.0.4 Resolution: | Keywords: ---------------------------+------------------------------------------------ Changes (by sebastien): * milestone: => 4.1 -- Ticket URL: <https://www.dynare.org/trac/ticket/29#comment:4> Dynare <http://www.dynare.org> The Dynare project

1 0

Re: [DynareDev] [Trac] #10: Integrate build system of DLLs with autoconf
by Trac Dynare 21 Sep '09

21 Sep '09

#10: Integrate build system of DLLs with autoconf ---------------------------------------+------------------------------------ Reporter: sebastien | Owner: sebastien Type: enhancement | Status: closed Priority: major | Milestone: 4.1 Component: Build system / Packaging | Version: Resolution: fixed | Keywords: ---------------------------------------+------------------------------------ Changes (by sebastien): * status: accepted => closed * resolution: => fixed Comment: See: http://www.dynare.org/DynareWiki/BuildingDynareFromSource -- Ticket URL: <https://www.dynare.org/trac/ticket/10#comment:3> Dynare <http://www.dynare.org> The Dynare project

1 0

Proposed specification for estimation functions
by G. Perendia 21 Sep '09

21 Sep '09

Hi I would like to start implementing period-dependent likelihood calculation within the new C++ DsgeLikelihood but that will add a bit of time. 1) I suggest that parameters description is defined by Start (or StartPeriod) and End (or EndPeriod) instead of a.. Period1 (beginning of sub-period) a.. Period2 (end of sub-period) respectively because we may have more than 2 periods and using terms Period1 and 2 as delimiters for other periods can bee a bit confusing in the context (as it did confuse me for a start) . 2) What is data_filtering (for period 1) step as opposed to Kalman filter - is it Kalman smoother? We have no working C++ implementation for smoother as yet. 3) Would it be right to say that the new period divided likelihood estimation requires overlapping variables Y(-/+n) for n>0 among the sub-period blocks? 4) What is parameters update? Is it param likelihood, hill climbing optimisation (or MCMC step) for each sub-period block separately (conditional on the previous period block?)? We do not have working optimiser in C++ yet. 4) Would the interim period-blocks' Kalman inputs be a(t|t-1) and P(t|t-1) as well as overlapping variables Y(-/+n) for n>0 from previous block Kalman output? Best regards George

2 2

mode check + diffuse filter
by Marco Ratto 18 Sep '09

18 Sep '09

Dear All, when running mode check it can happen that, for some parameter values, the model reaches some unit root boundary for which no sufficient information in the data is available. This implies an error message from the DKF routine which and the execution breaks. Shall I put a try-catch trap in mode_check to fix this ? best wishes Marco -- Marco Ratto, Euro-area Economy Modelling Centre (http://eemc.jrc.ec.europa.eu), Joint Research Centre The European Commission, TP 361, 21027 ISPRA(VA), ITALY Tel: +39 0332 78 5639 Fax: +39 0332 78 5733 marco.ratto(a)jrc.ec.europa.eu

1 0

Re: [DynareDev] [Trac] #7: Test for the presence of economics bibtex styles in the configure script
by Trac Dynare 18 Sep '09

18 Sep '09

#7: Test for the presence of economics bibtex styles in the configure script ---------------------------------------+------------------------------------ Reporter: sebastien | Owner: sebastien Type: bug | Status: closed Priority: minor | Milestone: 4.1 Component: Build system / Packaging | Version: Resolution: fixed | Keywords: ---------------------------------------+------------------------------------ Changes (by sebastien): * status: accepted => closed * resolution: => fixed Comment: Fixed in r2948. -- Ticket URL: <https://www.dynare.org/trac/ticket/7#comment:2> Dynare <http://www.dynare.org> The Dynare project

1 0

Re: [DynareDev] Dynare++ build system
by ondra.kamenik＠gmail.com 11 Sep '09

11 Sep '09

Amazing! Ondra K. ------Original Message------ From: Sébastien Villemot To: dev(a)dynare.org Cc: Ondra Kamenik Sent: Sep 11, 2009 19:43 Subject: Dynare++ build system As you can see, I have committed a new build system for Dynare++. It will build dynare++ only if the configure script finds ctangle and a suitable version of blas and lapack. For example on Linux systems il will find Atlas if installed. It is probably possible to compile with MKL using the --with-blas and --with-lapack options of configure script. It will also build PDF documentation of dynare++ with "make pdf", provided you have pdflatex, pdftex and cweave. It will run the various test programs of dynare++ with "make check". Best, -- Sébastien Villemot Sent from my BlackBerry® wireless device

1 0

Dynare++ build system
by Sébastien Villemot 11 Sep '09

11 Sep '09

As you can see, I have committed a new build system for Dynare++. It will build dynare++ only if the configure script finds ctangle and a suitable version of blas and lapack. For example on Linux systems il will find Atlas if installed. It is probably possible to compile with MKL using the --with-blas and --with-lapack options of configure script. It will also build PDF documentation of dynare++ with "make pdf", provided you have pdflatex, pdftex and cweave. It will run the various test programs of dynare++ with "make check". Best, -- Sébastien Villemot

1 0

M files documentation
by Michel Juillard 11 Sep '09

11 Sep '09

I have started a documentation of the M files of Dynare on the wiki. http://www.dynare.org/DynareWiki/DynareToolbox Among other things there is m2html documentation of the /matlab directory and subdirectories. I have build cron files on karaba and kirikou to automatize their update dayly, but it isn't working yet. Don't hesitate to add to it. The exercise was interesting, because it revealed that we were missing smoother routines for several cases. In the "verbal" description, I use a lot of hyperlinks. Could somebody volunteer to install a robot to check that links are alive on the wiki and more generally on the Dynare web site? Thanks in advance. All the best, Michel

1 0

Change in the build system
by Sébastien Villemot 09 Sep '09

09 Sep '09

Hi, In order to simplify the creation of the new build system (in particular for dynare++), I have decided to move to automake (which greatly simplifies the work of creating makefiles). I also decided to remove from SVN all files generated by any autotool (autoconf, automake). So the configure script is no longer in the SVN. To get it back, you need to: * install "autoconf" and "automake" on your machine * run "autoreconf -i" from the top-level (you can add the "-s" switch if you want symbolic links for automake utilities instead of plain copies) Then you can go on with the business of the configure script and so on. Best, -- Sébastien Villemot

1 0

[Trac] #34: Fix the dimension of oo_.mean.
by Trac Dynare 09 Sep '09

09 Sep '09

#34: Fix the dimension of oo_.mean. --------------------------+------------------------------------------------- Reporter: stepan | Owner: stepan Type: bug | Status: new Priority: major | Milestone: 4.1 Component: Core M-files | Version: 4.0.4 Keywords: | --------------------------+------------------------------------------------- Depending on the situation (simulated moments or theoretical moments) oo_.mean is a column or a row vector. -- Ticket URL: <https://www.dynare.org/trac/ticket/34> Dynare <http://www.dynare.org> The Dynare project

1 2

[Trac] #33: Correct bug in imcforecast.
by Trac Dynare 09 Sep '09

09 Sep '09

#33: Correct bug in imcforecast. --------------------------+------------------------------------------------- Reporter: stepan | Owner: stepan Type: bug | Status: new Priority: major | Milestone: 4.1 Component: Core M-files | Version: Keywords: | --------------------------+------------------------------------------------- The call to DsgeSmoother is buggy because of the missing observation new feature. -- Ticket URL: <https://www.dynare.org/trac/ticket/33> Dynare <http://www.dynare.org> The Dynare project

1 2

Dynare++ change
by Sébastien Villemot 08 Sep '09

08 Sep '09

Hi everyone, As agreed by Ondra, Dynare++ has been incorporated as a subdirectory of Dynare SVN. It is therefore now accessible at: https://www.dynare.org/svn/dynare/trunk/dynare++ The old Dynare++ SVN repository (https://www.dynare.org/svn/dynare++) is still accessible, but only for read access. Best, -- Sébastien Villemot

2 2

Re: [DynareDev] [Trac] #4: Add derivatives of model residuals with respect to parameters
by Trac Dynare 08 Sep '09

08 Sep '09

#4: Add derivatives of model residuals with respect to parameters ---------------------------+------------------------------------------------ Reporter: sebastien | Owner: sebastien Type: feature | Status: closed Priority: major | Milestone: 4.1 Component: Preprocessor | Version: Resolution: fixed | Keywords: ---------------------------+------------------------------------------------ Changes (by sebastien): * status: accepted => closed * resolution: => fixed Comment: Fixed in r2902 When the {{{identification;}}} command is present in the MOD file, the preprocessor creates a file ''FILENAME''{{{_params_derivs.m}}}, with the following prototype: {{{ function [rp, gp] = FILENAME_params_derivs(y, x, params, it_) }}} * {{{rp}}} is model residuals derivatives w.r. to params: equations in rows, parameters in columns * {{{gp}}} is jacobian derivatives w.r. to params: equations in dim 1, dynamic endogenous in dim 2, params in dim 3 -- Ticket URL: <https://www.dynare.org/trac/ticket/4#comment:3> Dynare <http://www.dynare.org> The Dynare project

1 0

dynare++
by Michel Juillard 04 Sep '09

04 Sep '09

Hi Ondra, 1) I committed a few changes to dynare++ brought about the George's work on package k_order as a DLL for Dynare Matlab - earlier in the Spring I added an option to set qz_criterium in order to the user to be able to put it slightly above 1 if the model contains unit roots (as in Dynare Matlab). It is at that time that I added by mistake the printing of D and E that you removed today. Sorry about that. - today I added a comment concerning dr_centralize and added a function to be able to write a TwoDMatrix to memory is order to be able to pass results back to the calling program 2) In order to facilitate building k_order DLL, we would like to merge dynare++ and dynare, keeping the same directory architecture. Sebastien would then build an autoconf/configure set of files for compiling both standalone dynare++ and k_order DLL and we would produce binaries for standalone dynare++ at the same time as the other binaries of dynare. Would you agree to that? 3) I observed that for leads on more than 1 period, dynare++ and dynare don't provide the same 2nd order approximation. We should investigate the origin of the difference 4) Is your simulate DLL compatible with dr_centralize option? All the best, Michel

3 3

Re: [DynareDev] [Trac] #8: Create a 2009a subdirectory under mex/, for 64-bits platforms
by Trac Dynare 03 Sep '09

03 Sep '09

#8: Create a 2009a subdirectory under mex/, for 64-bits platforms ---------------------------------------+------------------------------------ Reporter: sebastien | Owner: sebastien Type: bug | Status: closed Priority: minor | Milestone: 4.1 Component: Build system / Packaging | Version: Resolution: fixed | Keywords: ---------------------------------------+------------------------------------ Changes (by sebastien): * status: accepted => closed * resolution: => fixed Comment: Fixed in SVN r2891. -- Ticket URL: <https://www.dynare.org/trac/ticket/8#comment:4> Dynare <http://www.dynare.org> The Dynare project

1 0

Re: [DynareDev] [Trac] #28: Define a syntax for bytecode generation and the various Newton solvers (for steady and simul)
by Trac Dynare 03 Sep '09

03 Sep '09

#28: Define a syntax for bytecode generation and the various Newton solvers (for steady and simul) ---------------------------+------------------------------------------------ Reporter: sebastien | Owner: ferhat Type: enhancement | Status: closed Priority: critical | Milestone: 4.1 Component: Preprocessor | Version: Resolution: fixed | Keywords: ---------------------------+------------------------------------------------ Changes (by sebastien): * status: accepted => closed * resolution: => fixed Old description: > Some discussion on this point is on: > http://www.dynare.org/DynareWiki/ByteCode New description: -- Comment: See the new documentation on: http://www.dynare.org/DynareWiki/FastDeterministicSimulationAndSteadyStateC… -- Ticket URL: <https://www.dynare.org/trac/ticket/28#comment:2> Dynare <http://www.dynare.org> The Dynare project

1 0

[Trac] #30: Correct buggy mh_recover mode
by Trac Dynare 28 Aug '09

28 Aug '09

#30: Correct buggy mh_recover mode --------------------------+------------------------------------------------- Reporter: stepan | Owner: stepan Type: bug | Status: new Priority: major | Milestone: 4.1 Component: Core M-files | Version: Keywords: | --------------------------+------------------------------------------------- The initialization of the mcmc and the number of simulations are wrong when there is only one mcmc chain. -- Ticket URL: <https://www.dynare.org/trac/ticket/30> Dynare <http://www.dynare.org> The Dynare project

1 3

Re: [DynareDev] [Trac] #11: Allow for the possibility of using the bytecode representation of the model for any task
by Trac Dynare 28 Aug '09

28 Aug '09

#11: Allow for the possibility of using the bytecode representation of the model for any task ---------------------------+------------------------------------------------ Reporter: sebastien | Owner: ferhat Type: enhancement | Status: accepted Priority: major | Milestone: 4.2 Component: Preprocessor | Version: Resolution: | Keywords: ---------------------------+------------------------------------------------ Changes (by ferhat): * status: new => accepted -- Ticket URL: <https://www.dynare.org/trac/ticket/11#comment:2> Dynare <http://www.dynare.org> The Dynare project

1 0

Re: [DynareDev] [Trac] #28: Define a syntax for bytecode generation and the various Newton solvers (for steady and simul)
by Trac Dynare 28 Aug '09

28 Aug '09

#28: Define a syntax for bytecode generation and the various Newton solvers (for steady and simul) ---------------------------+------------------------------------------------ Reporter: sebastien | Owner: ferhat Type: enhancement | Status: accepted Priority: critical | Milestone: 4.1 Component: Preprocessor | Version: Resolution: | Keywords: ---------------------------+------------------------------------------------ Changes (by ferhat): * status: new => accepted -- Ticket URL: <https://www.dynare.org/trac/ticket/28#comment:1> Dynare <http://www.dynare.org> The Dynare project

1 0

Re: [DynareDev] [Trac] #21: Add partial information
by Trac Dynare 27 Aug '09

27 Aug '09

#21: Add partial information ---------------------------+------------------------------------------------ Reporter: sebastien | Owner: george Type: feature | Status: accepted Priority: major | Milestone: 4.2-alpha2 Component: Core M-files | Version: Resolution: | Keywords: ---------------------------+------------------------------------------------ Changes (by george): * owner: => george * status: new => accepted -- Ticket URL: <https://www.dynare.org/trac/ticket/21#comment:1> Dynare <http://www.dynare.org> The Dynare project

1 0

Trac for Dynare
by Sébastien Villemot 27 Aug '09

27 Aug '09

Hi, I have just installed a Trac for Dynare, available at the adresse: http://www.dynare.org/trac Trac is a project management software. It helps developers to deal with bugs, features to be added, deadlines, ... The basic item is a "ticket". A ticket is typically a bug report, or a feature to be added. The ticket is closed once the bug is fixed, or the feature is implemented. Users can spawn a discussion on a ticket. A ticket can be assigned: * a priority level * a type (bug, feature...) * a component (the part of Dynare to which it relates: the preprocessor, the core M-files, such DLL, the doc...) * a milestone (to be done for Dynare 4.1, for Dynare 4.2) * an owner (the person who is responsible for fixing the bug or adding the feature) * a version (4.0.1, 4.0.2, ...) (only useful for bug reports) I have already added some tickets, corresponding to the main release goals for Dynare 4.1, which is planned for the end of september. See "View tickets". The "Roadmap" lists the future milestones, currently 4.1 and 4.2. For each milestone, there is a deadline, and also the percentage of the tickets assigned to that milestone that have been closed. You can also browse the subversion repository through "Browse source". The "Timeline" lists all the recent changes (including SVN commits). You can syndicate to this timeline in your favorite RSS aggregator. Finally there is a wiki, which is a little bit redundant with the already existing DynareWiki. We may decide not to use it, or we may use it only for development specific stuff, and dedicate the other wiki to public documentation and other Dynare-related stuff. In order to add/modify tickets or wiki pages you need to be logged in. The usernames and password are exactly the same than for the SVN repository of Dynare. Please don't hesitate to add new tickets for the bugs you discover, or for the features you plan to add. Also please take ownership of existing tickets if you plan to deal with them. At a later date we may allow user to open tickets for bug reports, if we think we can avoid useless or wrong bug reports. Best, -- Sébastien Villemot

1 2

Re: [DynareDev] [Trac] #29: bug with varexo_det
by Trac Dynare 27 Aug '09

27 Aug '09

#29: bug with varexo_det ---------------------------+------------------------------------------------ Reporter: michel | Owner: michel Type: bug | Status: accepted Priority: major | Milestone: Component: Core M-files | Version: 4.0.4 Resolution: | Keywords: ---------------------------+------------------------------------------------ Changes (by sebastien): * version: => 4.0.4 -- Ticket URL: <https://www.dynare.org/trac/ticket/29#comment:3> Dynare <http://www.dynare.org> The Dynare project

1 0

MATLAB bug on Linux/64
by Sébastien Villemot 19 Aug '09

19 Aug '09

Hi everyone, I am hitting a MATLAB bug which is driving me crazy. I attach two files: plouf.m and plouf_static.m. They are derived from preprocessor-generated files with the EAGLE model. The script "plouf.m" calls the function "plouf_static.m". When I run "plouf.m" on a Linux/64 machine, MATLAB crashes with a bus error (tested on Karaba and Hal, with Matlab 7.5, 7.6 and 7.8). This does not happen on Linux/32 or Windows/32. I have not tested other platforms. As can be seen from the displayed marks which I have inserted, MATLAB crashes around line 131 of plouf_static.m. More precisely, it seems to crash just after having computed the expression there. I can't understand exactly what's going on. Can you reproduce this? Note that I have observed similar crashes with other big models on Karaba. Since so far I cannot precisely pinpoint what triggers the bug, I don't know how to make a workaround. So for the moment, for the development of EAGLE, I am going to use Matlab/32 in a virtual machine on Karaba. Probably we should send a report to Mathworks. Best, -- Sébastien Villemot

2 5

model derivatives with respect to parameters
by Michel Juillard 19 Aug '09

19 Aug '09

Dear Nikolay, we have started to work on producing the derivatives with respect to parameters. We are planing to create a new function, that will return a 3-dimensional array, by equation, variable and parameter. You could then select one parameter and squeeze the array in the parameter dimension to obtain a matrix of Jacobian derivatives with respect to a given parameter. Do you see any problem with this implementation? We should be able to provide in about two weeks. All the best, Michel

4 6

Re: [DynareDev] lyapunov kalman filter options
by Michel Juillard 30 Jul '09

30 Jul '09

G. Perendia wrote: > Dear Michel > > Should we include Matlab disclyap_fast.m into (next major release of) Dynare > Matlab suite as an alternative or replacement for lyapunov_symm.m and shall > we/I then also: > > we should just replace lyapunov_symm by (renamed) disclyap_fast.m > a) change Matlab disclyap_fast.m to accept tolerance as parameter and > include it into Dynare Matlab suite, > yes > b) change its name, > yes > d) prepare and include C++ disclyap_fast_dll.dll as separate module, again > as alternative to lyapunov_symm.m, > no, let's not multiply the DLLs called from Matlab. We will use the C code as part of a big DLL that performs all the estimation code > d) change its name too, > e) change DsgeLikelihood.m to use either Matlab and/or C++ disclyap_fast.m > > This is a bit complicated. Dsgelikelihood (and dsgelikelihood_hh) calls lyapunov_symm for stationary models. kalman_algo = 1 or 2 Equivalent code is present in dsgelikelihood.m for nonstationary models kalman_algo = 3 or 4 Best Michel > Best regards > > George > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > To: "List for Dynare developers" <dev(a)dynare.org> > Sent: Wednesday, July 29, 2009 7:36 PM > Subject: Re: [DynareDev] lyapunov kalman filter options > > > Sounds good, thanks > > Michel > > G. Perendia wrote: > >> Dear Michel (et al) >> >> On my slower laptop I run few tests and got timings roughly on lines of >> > the > >> following timings for an 8x8 T matrix and 10,000 loops: >> >> Matlab: >> >> lyapunov_symm.m time = 19.2270 s >> Pearlman Matlab disclyap_fast.m time = 13.0280 s >> >> New C++ disclyap_fast DLL: >> >> disclyap_fast_dll_time = 7.8010 sec for tolerance= 1e-16 >> disclyap_fast_dll_time = 6.5600 sec for tolerance= 1e-12 >> >> Best regards >> >> George >> >> ----- Original Message ----- >> From: "Michel Juillard" <michel.juillard(a)ens.fr> >> To: "G. Perendia" <george(a)perendia.orangehome.co.uk>; "List for Dynare >> developers" <dev(a)dynare.org> >> Sent: Wednesday, July 29, 2009 10:33 AM >> Subject: Re: [DynareDev] lyapunov kalman filter options >> >> >> Tanks for the update. >> The false value when A contains unit root is a fault of design. The >> information on which columns are useable is in the list of stationary >> variables, >> >> Best >> >> Michel >> >> G. Perendia wrote: >> >> >>> Dear Michel and Stephane >>> >>> Re the below warning: the results from lyapunov_symm.m seem now correct - >>> >>> >> It >> >> >>> seems that my Matlab variables memory was corrupted and after a reboot >>> > and > >>> restart it seems ok now - sorry for the alarm. >>> >>> It however reports (an incorrect ) X=[0's] instead of indeterminate or >>> >>> >> NaN >> >> >>> if I pass a=I. >>> >>> PS: and I still need to test and debug new C++lyapunov because it may >>> > have > >>> been causing the variable corruption. >>> >>> >>> Best regards >>> >>> George >>> >>> ----- Original Message ----- >>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>> To: "G. Perendia" <george(a)perendia.orangehome.co.uk> >>> Cc: "List for Dynare developers" <dev(a)dynare.org> >>> Sent: Wednesday, July 29, 2009 9:27 AM >>> Subject: lyapunov kalman filter options >>> >>> >>> >>> >>> >>>> I would use lyapunov_tol = 1e-12 >>>> (in the example disclyap_fat.m, the criterium is 1e-16) >>>> and >>>> kalman_tol = 1e-12 >>>> >>>> Both options should be accessible to users and obtained from >>>> options_. The initialization should take place in >>>> > global_initialization.m > >>>> Best >>>> >>>> Michel >>>> >>>> >>>> >>>> >>>> >>>>> Errata: >>>>> 1) it is options_.lyapunov_complex_threshold=1e-15 that is passed in >>>>> >>>>> >>>>> >>> calls >>> >>> >>> >>>>> to the >>>>> existing lyapunov_symm.m and I can use that as tolerance instead the >>>>> original 1e-6 if needed. >>>>> >>>>> Warning: >>>>> >>>>> If I pass a=T=[0] and b=RQR'~0 to lyapunov_symm.m, I get result Pstar >>>>> >>>>> >>>>> >>> very >>> >>> >>> >>>>> different from b although the solution to Lyapunov x-a*x*a' = b is >>>>> >>>>> >>>>> >>> x=b=RQR' >>> >>> >>> >>>>> and both, the new C++ and old matlab disklyap_fast return X=b=RQR as >>>>> solution!! >>>>> >>>>> Best regards >>>>> >>>>> George >>>>> >>>>> ----- Original Message ----- >>>>> From: "G. Perendia" <george(a)perendia.orangehome.co.uk> >>>>> To: "Michel Juillard" <michel.juillard(a)ens.fr>; "Stéphane Adjemian" >>>>> <stephane.adjemian(a)gmail.com> >>>>> Sent: Wednesday, July 29, 2009 7:46 AM >>>>> Subject: Re: measurement errors + steady KF >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> Hi Michel, Stephane >>>>>> >>>>>> What tolerances should we use to pass: >>>>>> >>>>>> 1)for the new fast iterative C++ Lyapunov solution convergence check >>>>>> >>>>>> if max( max( abs( P1 - P0 ) ) )>tol >>>>>> >>>>>> The original Joe Pearlman's matlab code used 1e-6. which is same as >>>>>> >>>>>> >>>>>> >>> Dynare >>> >>> >>> >>>>>> default ricatti_tol (also used for a similar purpose in calls to the >>>>>> existing lyapunov_symm.m.). So, shall we use the riccati_tol or set a >>>>>> separate, lyapunov_tol one at the same default level? >>>>>> >>>>>> 2) and, as per below, for the Old style DiffuseLikelihood.m like >>>>>> > Pstar > >>>>>> convergence/differences, that has been built into the C++KF i.e.: >>>>>> notsteady = ~(max(max(abs(Pstar-oldPstar)))<crit); >>>>>> where crit is options_.kalman_tol = 1.0000e-012 - I guess we can >>>>>> >>>>>> >>>>>> >>>>>> >>>>> continue >>>>> >>>>> >>>>> >>>>> >>>>>> to use the same >>>>>> though the new likelihood/kalman_filter*.m suite uses riccati_tol >>>>>> >>>>>> >>>>>> >>> instead >>> >>> >>> >>>>>> but for F convergence check. >>>>>> >>>>>> Best regards >>>>>> >>>>>> George >>>>>> Mob. +44(0)7951415480 >>>>>> artilogica(a)btconnect.com >>>>>> >>>>>> ----- Original Message ----- >>>>>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>>>>> To: "G. Perendia" <george(a)perendia.orangehome.co.uk>; "Stéphane >>>>>> >>>>>> >>>>>> >>> Adjemian" >>> >>> >>> >>>>>> <stephane.adjemian(a)gmail.com> >>>>>> Sent: Tuesday, July 14, 2009 11:47 PM >>>>>> Subject: Re: measurement errors + steady KF >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> I don't know. This is something that we need to discuss with >>>>>>> > Stephane. > >>>>>>> All the best, >>>>>>> >>>>>>> Michel >>>>>>> >>>>>>> G. Perendia wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Thanks Michel >>>>>>>> >>>>>>>> I am finishing algorithm for checking if KF P (and/or K) are steady >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>> for >>>>> >>>>> >>>>> >>>>> >>>>>> the >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>> C++KF. >>>>>>>> >>>>>>>> Old DiffuseLikelihood.m uses Pstar differences, i.e.: >>>>>>>> notsteady = ~(max(max(abs(Pstar-oldPstar)))<crit); >>>>>>>> where crit is options_.kalman_tol = 1.0000e-012 >>>>>>>> >>>>>>>> whilst the new kalman_filter.m uses K to check: >>>>>>>> notsteady = max(max(abs(K-oldK))) > riccati_tol; >>>>>>>> where >>>>>>>> K = Pstar*Z'*invF; >>>>>>>> and options_.riccati_tol= 1.0000e-006 >>>>>>>> >>>>>>>> Which tolerance (and check) should I use for C++KF? >>>>>>>> I have already built a system using Pstar -oldPstar and that works >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>> well >>>>> >>>>> >>>>> >>>>> >>>>>> but >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>> question is which tolerance to use since riccati seems sufficient >>>>>>>> > for > >>>>>>>> >>>>>>>> >>>>> P >>>>> >>>>> >>>>> >>>>> >>>>>>>> difference too. >>>>>>>> >>>>>>>> Best regards >>>>>>>> >>>>>>>> George >>>>>>>> Mob. +44(0)7951415480 >>>>>>>> artilogica(a)btconnect.com >>>>>>>> >>>>>>>> ----- Original Message ----- >>>>>>>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>>>>>>> To: "G. Perendia" <george(a)perendia.orangehome.co.uk> >>>>>>>> Sent: Tuesday, July 14, 2009 6:57 AM >>>>>>>> Subject: Re: measurement errors >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> Hi George, >>>>>>>>> >>>>>>>>> it is like for exogenous variables. If GDP is an endogenous >>>>>>>>> > variable > >>>>>>>>> estimated_params; >>>>>>>>> stderr GDP, uniform_pdf,,,0,1; >>>>>>>>> .... >>>>>>>>> end; >>>>>>>>> >>>>>>>>> will introduce a measurement error on GDP. >>>>>>>>> >>>>>>>>> Best >>>>>>>>> >>>>>>>>> Michel >>>>>>>>> G. Perendia wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> Dear Michel >>>>>>>>>> >>>>>>>>>> For testing Part Info (and the other) Kalman filters, I would need >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>> to >>>>> >>>>> >>>>> >>>>> >>>>>>>>>> set situation with measurement errors so that H~=0. >>>>>>>>>> >>>>>>>>>> Can you please let me know how one specifies measurement errors >>>>>>>>>> in Dynare for estimation so that H ~ =0? >>>>>>>>>> Best regards >>>>>>>>>> >>>>>>>>>> George >>>>>>>>>> artilogica(a)btconnect.com <mailto:artilogica@btconnect.com> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>> a >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> >>> >>> >> _______________________________________________ >> Dev mailing list >> Dev(a)dynare.org >> http://www.dynare.org/cgi-bin/mailman/listinfo/dev >> >> >> >> _______________________________________________ >> Dev mailing list >> Dev(a)dynare.org >> http://www.dynare.org/cgi-bin/mailman/listinfo/dev >> >> >> > > > _______________________________________________ > Dev mailing list > Dev(a)dynare.org > http://www.dynare.org/cgi-bin/mailman/listinfo/dev > > > >

2 2

Re: [DynareDev] lyapunov kalman filter options
by Michel Juillard 29 Jul '09

29 Jul '09

Tanks for the update. The false value when A contains unit root is a fault of design. The information on which columns are useable is in the list of stationary variables, Best Michel G. Perendia wrote: > Dear Michel and Stephane > > Re the below warning: the results from lyapunov_symm.m seem now correct - It > seems that my Matlab variables memory was corrupted and after a reboot and > restart it seems ok now - sorry for the alarm. > > It however reports (an incorrect ) X=[0's] instead of indeterminate or NaN > if I pass a=I. > > PS: and I still need to test and debug new C++lyapunov because it may have > been causing the variable corruption. > > > Best regards > > George > Mob. +44(0)7951415480 > artilogica(a)btconnect.com > > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > To: "G. Perendia" <george(a)perendia.orangehome.co.uk> > Cc: "List for Dynare developers" <dev(a)dynare.org> > Sent: Wednesday, July 29, 2009 9:27 AM > Subject: lyapunov kalman filter options > > > >> I would use lyapunov_tol = 1e-12 >> (in the example disclyap_fat.m, the criterium is 1e-16) >> and >> kalman_tol = 1e-12 >> >> Both options should be accessible to users and obtained from >> options_. The initialization should take place in global_initialization.m >> >> Best >> >> Michel >> >> >> >>> Errata: >>> 1) it is options_.lyapunov_complex_threshold=1e-15 that is passed in >>> > calls > >>> to the >>> existing lyapunov_symm.m and I can use that as tolerance instead the >>> original 1e-6 if needed. >>> >>> Warning: >>> >>> If I pass a=T=[0] and b=RQR'~0 to lyapunov_symm.m, I get result Pstar >>> > very > >>> different from b although the solution to Lyapunov x-a*x*a' = b is >>> > x=b=RQR' > >>> and both, the new C++ and old matlab disklyap_fast return X=b=RQR as >>> solution!! >>> >>> Best regards >>> >>> George >>> Mob. +44(0)7951415480 >>> artilogica(a)btconnect.com >>> >>> ----- Original Message ----- >>> From: "G. Perendia" <george(a)perendia.orangehome.co.uk> >>> To: "Michel Juillard" <michel.juillard(a)ens.fr>; "Stéphane Adjemian" >>> <stephane.adjemian(a)gmail.com> >>> Sent: Wednesday, July 29, 2009 7:46 AM >>> Subject: Re: measurement errors + steady KF >>> >>> >>> >>> >>>> Hi Michel, Stephane >>>> >>>> What tolerances should we use to pass: >>>> >>>> 1)for the new fast iterative C++ Lyapunov solution convergence check >>>> >>>> if max( max( abs( P1 - P0 ) ) )>tol >>>> >>>> The original Joe Pearlman's matlab code used 1e-6. which is same as >>>> > Dynare > >>>> default ricatti_tol (also used for a similar purpose in calls to the >>>> existing lyapunov_symm.m.). So, shall we use the riccati_tol or set a >>>> separate, lyapunov_tol one at the same default level? >>>> >>>> 2) and, as per below, for the Old style DiffuseLikelihood.m like Pstar >>>> convergence/differences, that has been built into the C++KF i.e.: >>>> notsteady = ~(max(max(abs(Pstar-oldPstar)))<crit); >>>> where crit is options_.kalman_tol = 1.0000e-012 - I guess we can >>>> >>>> >>> continue >>> >>> >>>> to use the same >>>> though the new likelihood/kalman_filter*.m suite uses riccati_tol >>>> > instead > >>>> but for F convergence check. >>>> >>>> Best regards >>>> >>>> George >>>> Mob. +44(0)7951415480 >>>> artilogica(a)btconnect.com >>>> >>>> ----- Original Message ----- >>>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>>> To: "G. Perendia" <george(a)perendia.orangehome.co.uk>; "Stéphane >>>> > Adjemian" > >>>> <stephane.adjemian(a)gmail.com> >>>> Sent: Tuesday, July 14, 2009 11:47 PM >>>> Subject: Re: measurement errors + steady KF >>>> >>>> >>>> >>>> >>>>> I don't know. This is something that we need to discuss with Stephane. >>>>> >>>>> All the best, >>>>> >>>>> Michel >>>>> >>>>> G. Perendia wrote: >>>>> >>>>> >>>>>> Thanks Michel >>>>>> >>>>>> I am finishing algorithm for checking if KF P (and/or K) are steady >>>>>> >>>>>> >>> for >>> >>> >>>> the >>>> >>>> >>>>>> C++KF. >>>>>> >>>>>> Old DiffuseLikelihood.m uses Pstar differences, i.e.: >>>>>> notsteady = ~(max(max(abs(Pstar-oldPstar)))<crit); >>>>>> where crit is options_.kalman_tol = 1.0000e-012 >>>>>> >>>>>> whilst the new kalman_filter.m uses K to check: >>>>>> notsteady = max(max(abs(K-oldK))) > riccati_tol; >>>>>> where >>>>>> K = Pstar*Z'*invF; >>>>>> and options_.riccati_tol= 1.0000e-006 >>>>>> >>>>>> Which tolerance (and check) should I use for C++KF? >>>>>> I have already built a system using Pstar -oldPstar and that works >>>>>> >>>>>> >>> well >>> >>> >>>> but >>>> >>>> >>>>>> question is which tolerance to use since riccati seems sufficient for >>>>>> >>>>>> >>> P >>> >>> >>>>>> difference too. >>>>>> >>>>>> Best regards >>>>>> >>>>>> George >>>>>> Mob. +44(0)7951415480 >>>>>> artilogica(a)btconnect.com >>>>>> >>>>>> ----- Original Message ----- >>>>>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>>>>> To: "G. Perendia" <george(a)perendia.orangehome.co.uk> >>>>>> Sent: Tuesday, July 14, 2009 6:57 AM >>>>>> Subject: Re: measurement errors >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> Hi George, >>>>>>> >>>>>>> it is like for exogenous variables. If GDP is an endogenous variable >>>>>>> >>>>>>> estimated_params; >>>>>>> stderr GDP, uniform_pdf,,,0,1; >>>>>>> .... >>>>>>> end; >>>>>>> >>>>>>> will introduce a measurement error on GDP. >>>>>>> >>>>>>> Best >>>>>>> >>>>>>> Michel >>>>>>> G. Perendia wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Dear Michel >>>>>>>> >>>>>>>> For testing Part Info (and the other) Kalman filters, I would need >>>>>>>> >>>>>>>> >>> to >>> >>> >>>>>>>> set situation with measurement errors so that H~=0. >>>>>>>> >>>>>>>> Can you please let me know how one specifies measurement errors >>>>>>>> in Dynare for estimation so that H ~ =0? >>>>>>>> Best regards >>>>>>>> >>>>>>>> George >>>>>>>> Mob. +44(0)7951415480 >>>>>>>> artilogica(a)btconnect.com <mailto:artilogica@btconnect.com> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>> a >>>>> >>>>> >>>>> >>> >>> >>> >>> >> > > > >

2 2

lyapunov kalman filter options
by Michel Juillard 29 Jul '09

29 Jul '09

I would use lyapunov_tol = 1e-12 (in the example disclyap_fat.m, the criterium is 1e-16) and kalman_tol = 1e-12 Both options should be accessible to users and obtained from options_. The initialization should take place in global_initialization.m Best Michel > Errata: > 1) it is options_.lyapunov_complex_threshold=1e-15 that is passed in calls > to the > existing lyapunov_symm.m and I can use that as tolerance instead the > original 1e-6 if needed. > > Warning: > > If I pass a=T=[0] and b=RQR'~0 to lyapunov_symm.m, I get result Pstar very > different from b although the solution to Lyapunov x-a*x*a' = b is x=b=RQR' > and both, the new C++ and old matlab disklyap_fast return X=b=RQR as > solution!! > > Best regards > > George > Mob. +44(0)7951415480 > artilogica(a)btconnect.com > > ----- Original Message ----- > From: "G. Perendia" <george(a)perendia.orangehome.co.uk> > To: "Michel Juillard" <michel.juillard(a)ens.fr>; "Stéphane Adjemian" > <stephane.adjemian(a)gmail.com> > Sent: Wednesday, July 29, 2009 7:46 AM > Subject: Re: measurement errors + steady KF > > > >> Hi Michel, Stephane >> >> What tolerances should we use to pass: >> >> 1)for the new fast iterative C++ Lyapunov solution convergence check >> >> if max( max( abs( P1 - P0 ) ) )>tol >> >> The original Joe Pearlman's matlab code used 1e-6. which is same as Dynare >> default ricatti_tol (also used for a similar purpose in calls to the >> existing lyapunov_symm.m.). So, shall we use the riccati_tol or set a >> separate, lyapunov_tol one at the same default level? >> >> 2) and, as per below, for the Old style DiffuseLikelihood.m like Pstar >> convergence/differences, that has been built into the C++KF i.e.: >> notsteady = ~(max(max(abs(Pstar-oldPstar)))<crit); >> where crit is options_.kalman_tol = 1.0000e-012 - I guess we can >> > continue > >> to use the same >> though the new likelihood/kalman_filter*.m suite uses riccati_tol instead >> but for F convergence check. >> >> Best regards >> >> George >> Mob. +44(0)7951415480 >> artilogica(a)btconnect.com >> >> ----- Original Message ----- >> From: "Michel Juillard" <michel.juillard(a)ens.fr> >> To: "G. Perendia" <george(a)perendia.orangehome.co.uk>; "Stéphane Adjemian" >> <stephane.adjemian(a)gmail.com> >> Sent: Tuesday, July 14, 2009 11:47 PM >> Subject: Re: measurement errors + steady KF >> >> >> >>> I don't know. This is something that we need to discuss with Stephane. >>> >>> All the best, >>> >>> Michel >>> >>> G. Perendia wrote: >>> >>>> Thanks Michel >>>> >>>> I am finishing algorithm for checking if KF P (and/or K) are steady >>>> > for > >> the >> >>>> C++KF. >>>> >>>> Old DiffuseLikelihood.m uses Pstar differences, i.e.: >>>> notsteady = ~(max(max(abs(Pstar-oldPstar)))<crit); >>>> where crit is options_.kalman_tol = 1.0000e-012 >>>> >>>> whilst the new kalman_filter.m uses K to check: >>>> notsteady = max(max(abs(K-oldK))) > riccati_tol; >>>> where >>>> K = Pstar*Z'*invF; >>>> and options_.riccati_tol= 1.0000e-006 >>>> >>>> Which tolerance (and check) should I use for C++KF? >>>> I have already built a system using Pstar -oldPstar and that works >>>> > well > >> but >> >>>> question is which tolerance to use since riccati seems sufficient for >>>> > P > >>>> difference too. >>>> >>>> Best regards >>>> >>>> George >>>> Mob. +44(0)7951415480 >>>> artilogica(a)btconnect.com >>>> >>>> ----- Original Message ----- >>>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>>> To: "G. Perendia" <george(a)perendia.orangehome.co.uk> >>>> Sent: Tuesday, July 14, 2009 6:57 AM >>>> Subject: Re: measurement errors >>>> >>>> >>>> >>>> >>>>> Hi George, >>>>> >>>>> it is like for exogenous variables. If GDP is an endogenous variable >>>>> >>>>> estimated_params; >>>>> stderr GDP, uniform_pdf,,,0,1; >>>>> .... >>>>> end; >>>>> >>>>> will introduce a measurement error on GDP. >>>>> >>>>> Best >>>>> >>>>> Michel >>>>> G. Perendia wrote: >>>>> >>>>> >>>>>> Dear Michel >>>>>> >>>>>> For testing Part Info (and the other) Kalman filters, I would need >>>>>> > to > >>>>>> set situation with measurement errors so that H~=0. >>>>>> >>>>>> Can you please let me know how one specifies measurement errors >>>>>> in Dynare for estimation so that H ~ =0? >>>>>> Best regards >>>>>> >>>>>> George >>>>>> Mob. +44(0)7951415480 >>>>>> artilogica(a)btconnect.com <mailto:artilogica@btconnect.com> >>>>>> >>>>>> >>>> >>>> >>>> >>> a >>> >>> > > > > >

1 0

next step
by Michel Juillard 23 Jul '09

23 Jul '09

Dear George, the next step in the C implementation of the estimation routine is to have a compiled version of the Lyapunov equation solver. The corresponding Matlab code is in lyapunov_symm.m or around lines 185-242 of DsgeLikelihood.m. I probably got the idea of the algorithm in "Numerical Recepees in C++" or "Matrix Computations", this last one by Golub and van Loan. lyapunov_symm.m returns the covariance matrix (and the list of stationary variables) while in DsgeLikelihood, we keep the Schur decomposition of the state space transition matrix in order to have a quasi-triangular transition matrix in the Kalman filter (matrix T) and the Schur vectors to build matrix Z. Currently we only use it for the diffuse filter, but there is no reason not to use for the regular filter if it is more efficient. I see two ways to implement it. Write it from scratch in C++ using direct calls to the necessary Lapack functions and/or use RECSY http://www.cs.umu.se/~isak/recsy/ (we need of course to evaluate it first). The problem/advantage of RECSY is that it uses parallel computing via OpenMP, but currently (as of 2008 versions) MKL distributed with Matlab is incompatible with OpenMP (see the discussion lead by Stephane Adjemian a few months ago). My feeling is that building the estimation DLL will be easier with a Lyapunov solver built from scratch and that we can keep dealing with RESY for further improvments of efficiency. Note that there must be a Lyapunov solver in dynare++, but I'm afraid that it will be plagued with the same problems as the Kalman filter and my feeling is that it takes longer to modify it than to write clean code from the start. Please tell me your thoughts, Best Michel

1 0

Fast steady state computation
by Sébastien Villemot 13 Jul '09

13 Jul '09

Hi everyone, I have added a new option to the "steady" command, called "block_mfs". This option does the following: * it performs a normalization and a block decomposition of the static model * then for each block it computes the minimum feedback set (MFS) of variables, in order to reduce the size of the steady-state sub-problem; in the current state of the code, variables which are not written in a normalized form (x=f(y) where x is not in y), are automatically included in the MFS The option is currently operational. Tested on EAGLE model, under Octave, it is 3 times faster than the traditional steady command. With MATLAB on Karaba, I hit a MATLAB crash which needs further investigation. The current performance bottle-neck is that we only exploit hand-written normalizations in the model. Ferhat has already added some code which tries linear transformations in order to put more equations in normalized form; I will incoporate this to the current code. Further improvements needs to be done: * adding temporary terms to the generated *_static.m file * improving the performance of the preprocessor (it is a bit slow on big models such as GIMF) I will be on vacations for about one month, but I will read my e-mail. All the best, -- Sébastien Villemot

1 0

Preprocessor and g++
by Sébastien Villemot 11 Jul '09

11 Jul '09

Hi, In my last commit, I use an extension of the C++ standard template library that is only implemented in g++ (and also in a few other compilers). Practically, that means that currently the preprocessor cannot be compiled with a compiler different from g++. Is this a problem for you? If it is, then I can provide an alternative implementation of this extension, for compilers other than g++. -- Sébastien Villemot

2 1

Re: [DynareDev] DiffuseLikelihod kalman routines
by G. Perendia 08 Jul '09

08 Jul '09

Hi Marco (et al) I uploaded the compatibility and consistency changes outlined below I made to the trunk and the branch versions of: DiffuseLikelihoodH3 DiffuseLikelihood3 DiffuseLikelihoodH3_Z DiffuseLikelihood3_Z DiffuseLikelihoodH3corr DsgeLikelihood_hh I also added lik to the output of both instances of DiffuseLikelihoodH3corr.m as it seem needed by DsgeLikelihood_hh.m and, removed the incompatible .5* in front of "lik" in the both instances of DsgeLikelihood_hh.m, i.e. llik=[-lnprior; lik(start:end)] instead llik=[-lnprior; .5*lik(start:end)] I did not however correct the call to the non-existent DiffuseLikelihoodH3corr_Z in the latest, trunk, version of DsgeLikelihood_hh.m as I am not yet fully sure of the intention. If the older versions need to be reinstated, that can be done from SVN archive (and I have kept copies too). Let me know if that is OK. Best regards George ----- Original Message ----- From: "G. Perendia" <george(a)perendia.orangehome.co.uk> To: "List for Dynare developers" <dev(a)dynare.org> Sent: Wednesday, July 08, 2009 2:30 PM Subject: Re: [DynareDev] DiffuseLikelihod kalman routines > Hi All (and Marco!) > > 1) I made and committed the changes to both the trunk and the branch > versions of DiffuseLikelihood1, H1, 1_Z and H1_Z.m routines and released > locked routines earlier today. > > 2) However, things are never so simple and the resulting chain reaction / > snowball goes on: > > Looking at the other DiffuseLikelihood* functions called within > DsgeLikilihood_hh that initially did not need the bug fix: > > DiffuseLikelihoodH3_Z > DiffuseLikelihood3_Z > DiffuseLikelihoodH3corr_Z > > The first two use inherently correct but in the context of DsgeLikilihood_hh > incompatible calculation of the lik array which contains all log(2*pi) in > the last (sample+1) element i.e. > lik(smpl+1) = smpl*pp*log(2*pi); > > so that LIK : > > LIK = .5*(sum(lik(start:end))-(start-1)*lik(smpl+1)/smpl); > > is correctly calculated within the functions but that yields probably > incorrect one for the llik returned from DsgeLikilihood_hh in its last line: > > llik=[-lnprior; .5*lik(start:end)] > > I.e. it will contain too many log(2*pi) 's when start>1. > > 3) Also, the new kalman_filter and corrected DiffuseLikelihood ones all have > > lik = (lik + pp*log(2*pi))/2 > > so the lik is already multiplied with 0.5! > and that is incompatible with the llik calculation in DsgeLikilihood_hh. > > I guess, for consistency, I should make some of the changes to the other > Diffuse functions now too, i.e. replace: > lik(smpl+1) = smpl*pp*log(2*pi); > with > lik = (lik + pp*log(2*pi))/2 > as in all other kalman_filter ones > > and , in DsgeLikilihood_hh, probably replace > > llik=[-lnprior; .5*lik(start:end)] > with > llik=[-lnprior; lik(start:end)] > > This is even more urgent since the new version of > kalman_filter.m, also called within DsgeLikilihood_hh, also pre- halves lik > array! > > 3) The third called function does not exist in the matlab directory but > DiffuseLikelihoodH3corr.m (without "_Z") does and that has the same > compatibility problem. > > Hence, DsgeLikilihood_hh also needs to be corrected to refer to an existing > function (I guess it was a typo in DsgeLikilihood_hh) and I can make that > correction too if that is OK. > > Best regards > > George > > ----- Original Message ----- > From: "G. Perendia" <george(a)perendia.orangehome.co.uk> > To: "List for Dynare developers" <dev(a)dynare.org> > Sent: Wednesday, July 08, 2009 12:24 PM > Subject: Re: [DynareDev] DiffuseLikelihod kalman routines > > > Hi > > The trunk ones are still being used by DsgeLikilihood_hh and will be needed > for 4.1 and the branch ones appear to be just their identical copies (e.g. > diff reports no differences), and so will they be fixed by mere copying - > just in case. > > If that is OK , I will lock them this afternoon and let you know when I > finish and unlock them. > > Regards > George > > > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > To: "List for Dynare developers" <dev(a)dynare.org> > Sent: Wednesday, July 08, 2009 12:08 PM > Subject: Re: [DynareDev] DiffuseLikelihod kalman routines > > > I beg to disagree. It's not a good idea to leave bugs behind. I had > already forgotten these ones when we decided to release 4.0.4 > I don't think it is big changes either > > Best > > Michel > > Sébastien Villemot wrote: > > Le mercredi 08 juillet 2009 à 10:56 +0100, G. Perendia a écrit : > > > >> I am scheduling some bug fixing changes to both the trunk and the > >> branch DiffuseLikelihood1, H1, 1_Z and H1_Z.m routines and would like > >> to avoid any potential edit conflicts. > >> > > > > Just to mention that we have decided that there would be no more Dynare > > release in the 4.0.x. We are now focusing our energies on the next major > > release, 4.1, by the end of the summer if everything goes well. > > > > So it's not necessary to fix the 4.0 branch. > > > > Best, > > > > > > > _______________________________________________ > Dev mailing list > Dev(a)dynare.org > http://www.dynare.org/cgi-bin/mailman/listinfo/dev > > > > _______________________________________________ > Dev mailing list > Dev(a)dynare.org > http://www.dynare.org/cgi-bin/mailman/listinfo/dev >

1 0

DiffuseLikelihod kalman routines
by G. Perendia 08 Jul '09

08 Jul '09

Hi I am scheduling some bug fixing changes to both the trunk and the branch DiffuseLikelihood1, H1, 1_Z and H1_Z.m routines and would like to avoid any potential edit conflicts. Is anyone planning or doing work on those routines and is it ok to put a temporary Tortoise SVN lock on them, e.g. for today or tomorrow? Best regards George

4 6

Andrea's files
by Michel Juillard 07 Jul '09

07 Jul '09

Here are the files in a new message. Best Michel

1 0

[Fwd: Paralallel and adaptive MCMC]
by Michel Juillard 07 Jul '09

07 Jul '09

Here are Andrea's file again. I can read the PDF file on my (Windows) machine Best Michel -------- Original Message -------- Subject: Paralallel and adaptive MCMC Date: Mon, 6 Jul 2009 14:39:56 +0200 From: andrea pagano <pagano.andrea(a)gmail.com> To: Michel Juillard <Michel.Juillard(a)ens.fr> Dear Michel just before leaving for a week vacation I am sending you a short note and some rouitnes to do some adaptive and replica exchange MCMC. I have tried to use same input notation as in the metropolis code, Marco and Ivano are using in their work. There are many things to be adjusted, but maybe they can be a good starting point for this line of application. Best Andrea P.S. I have reply to Sebastien asking him top wait untill I return to give me the access to karaba. -- Andrea Pagano via Veratti VARESE tel. +3903321691261 cell.+393403804397

1 0

Compiling korderpert
by Sébastien Villemot 07 Jul '09

07 Jul '09

Hi, I'm currently working on a more elaborate build system for the various MEX files. I've tried to compile korderpert, but it fails: it complains about a missing writeMMap() method on a object coming from Dynare++. I guess that you applied some modifications to Dynare++ original source code. Maybe could you create a patch (a diff file) to be applied to the original Dynare++ code, and send it to me? (I will incoporate it into the build system). Best, -- Sébastien Villemot

3 2

[Fwd: Paralallel and adaptive MCMC]
by Michel Juillard 07 Jul '09

07 Jul '09

Here is a contribution of Andrea on MCMC Best, Michel -------- Original Message -------- Subject: Paralallel and adaptive MCMC Date: Mon, 6 Jul 2009 14:39:56 +0200 From: andrea pagano <pagano.andrea(a)gmail.com> To: Michel Juillard <Michel.Juillard(a)ens.fr> Dear Michel just before leaving for a week vacation I am sending you a short note and some rouitnes to do some adaptive and replica exchange MCMC. I have tried to use same input notation as in the metropolis code, Marco and Ivano are using in their work. There are many things to be adjusted, but maybe they can be a good starting point for this line of application. Best Andrea P.S. I have reply to Sebastien asking him top wait untill I return to give me the access to karaba. -- Andrea Pagano via Veratti VARESE tel. +3903321691261 cell.+393403804397

1 0

C++ Kalman filter
by Michel Juillard 06 Jul '09

06 Jul '09

Dear Andrea and George, than you for your efforts on the Kalman filter. I would like to regroup efforts along the following lines: 1) Kalman filter C++: I would still like to be able to compare with an implementation that removes all object instantiation inside the loop: the lowest level object should be Kalman filter and all necessary work space should be allocated once for ever outside the main loop of the filter. The multiplications should be done using optimized Lapack DGEM. George should continue along this line. I'm suspicious of timings taken in Windows and we should double check them under Linux. For this reason as well, Makefile should also provide for building of tests under Linux. George and Andrea should have accounts on karaba. Sebastien, can you take care of that? 2) Unless I'm missing something, there are only two explanations for not being able to beat DGEMM a) our compilers don't optimize as well as the one used by Matlab b) there is a problem in the algorithm. In order to eliminate and be able to quantify the importance of (a), I would like Andrea to write a standalone test for both operations: - A*x - A*P*A' calling DGEMM (from Matlab's Lapack library) and compare it with his own routines. A minimum modification would be to take DGEMM as it is and modify the loops so as to accomodate a quasi-triangular matrix instead of a general matrix. In this exercise, all lower level BLAS functions should be linked with Matlab's BLAS library. We will make a new attempt to integrate Andrea's routines in the Kalman filter only when we obtain satisfactory standalone test results. I think it is important to spend more time on these tests, because we are evaluating how much speed improvement we can expect from using compiled code instead of Matlab. All the best, Michel

4 4

Re: [DynareDev] Fw: 2nd and 3rd model derivatives + k-order
by Sébastien Villemot 01 Jul '09

01 Jul '09

"G. Perendia" <george(a)perendia.orangehome.co.uk> a écrit : > What is the expected preprocessor mechanism to allow for k-ord-pert to be > used (i.e. by setting matlab/kordpert directory on the top of search path) > and when is it expected to be in place? > > If there is none in place yet, what would be the best place (i.e. which .m > file) to set path to kordpert/dr.1 when > options_.use_k_order=1; > is specified in the mod file, e.g. as per my earlier message below? I don't think it is good practice to override matlab/dr1.m with matlab/kordpert/dr1.m through path manipulations. I would rather dynamically select which version of dr1 to use. For example, dr1.m would be a dispatcher between the original dr1.m and the korder dr1.m (and possibly dr1_sparse.m added by Ferhat). Code that can be factorized would go in top-level dr1.m, and specific code would go into dr1_original.m, dr1_korder.m and dr1_sparse.m. Best, -- Sébastien Villemot

2 1

Re: [DynareDev] 2nd and 3rd model derivatives + k-order
by Sébastien Villemot 29 Jun '09

29 Jun '09

Le jeudi 25 juin 2009 à 12:59 +0100, G. Perendia a écrit : > However, I do not quite understand what do you mean with USE_DLL is > roken - is it because sparse matrix is now returned and the k-order is > broken? USE_DLL is broken in the sense that the matrix returned by the MEX file at the place of g2 is no longer square: it has only 3 columns describing the hessian in a sparse way. So dr1.m is completely confused with USE_DLL option at order 2: it expects a square hessian and it gets a 3-columns matrix. > Can you remind me how is _dynamic MEX/dll file, used by k-order.dll, then > produced - is USE_DLL going to stay as option e.g. for 2nd order or is it > going to be discontinued? We are going to drop USE_DLL option. We will then add an option like "korderpert" to stoch_simul, which will trigger the call to the k-order DLL. And ultimately we will drop the generation of MEX files containing the model, to replace it by a simulated bytecode. The interface to k-order DLL will then need to be changed. > Is jacobia_ still a full matrix? Yes, nothing changed for the Jacobian -- Sébastien Villemot

3 9

gensylv+buid_matlab+check_mode
by Michel Juillard 29 Jun '09

29 Jun '09

Everybody on the dev(a)dynare.org list, please look at 3) below. I need your feedback. Hi Marco, 1) the Sylvester-like functions are in matlab/gensylv -gensylv.m is a replacement when gensylv.dll is missing -sylvester3.m is the function that your are looking for -sylvester3a.m is an iterative refinement of the solution if you consider that the accuracy of the previous function is insufficient 2) the build_matlab problem comes from the fact that the MEX configuration for your compiler does not know option CDEBUGFLAGS= What is you compiler? What is the version of Matlab ? It is also possible that this script doesn't work with older Matalb versions. 3) thank you for this new version of mode_check. I wonder if we shouldn't change the sign and really plot the log-posterior and log-likelihood. Also, as you force the log-likelihood to intersect the log-posterior at the computed mode, it is a log-likelihood kernel (the level of the log-posterior being correct). This should be indicated in the label otherwise, it is confusing. With these changes, I believe it can become the new default for mode_check, but we need to provide an option to get the old graphs. What do the other developers think? All the best, Michel -------- Original Message -------- Subject: Sylverster equation Date: Sun, 14 Jun 2009 10:26:07 +0200 From: rattoma <marco.ratto(a)jrc.it> To: Michel Juillard <michel.juillard(a)ens.fr> References: <4A24238B.1060503(a)ens.fr> <4A2CC2D0.10601(a)jrc.it> Dear Michel, I just would like to remind you, if possible, to dig out a matlab routine for generalized Sylvester equation, so I may try to plug it with Nikolai' staff this week in Lisobon. In themeantime I have two questions: 1) I cannot successfully run the build_matlab script in /mex/source: I get the attached error in windows. Do you have any hint? I probably miss something; 2) I am going to commit a new mode_check with the attacehd graphical layiout. Moreover, I have squeezed the actual tjhree cases with if, into one single for loop, for easier parallelization. Are you OK with this? Many thanks for your help, best wishes Marco

3 8

Re: [DynareDev] 2nd and 3rd model derivatives + k-order
by G. Perendia 29 Jun '09

29 Jun '09

Dear Sebastien Thanks for M_.NNZDerivatives, 1) However, M_.NNZDerivatives(2) ans = 119 should result in a Hessian matrix of 3*119, total 357 elements, however, as can be seen below, it finishes sparse at element 138 instead 357-1=356.Alternatively, if 119 it a total number of elements (including indices) then it should be divisible by 3. Can you explain me where is this discrepancy occurring form? 2) Also, in a related issue, if you, or anyone else, has no objections Re my previous email, would it be then possible to change the behaviour of the preprocessor and have the indices and values of the sparse Hessian from the mex file returned grouped together either 3 by 3 as per my example below (which would then also constitute a 3 column matrix by number of non-zero elements rows as per your email) or, if it is supposed to be a "3 row" matrix, then apart by the total number of non-zero Hessian values, that is,. rather than being apart by no-of-vars/equations as it seems to be the case now? Best regards George ----- Original Message ----- From: "G. Perendia" <george(a)perendia.orangehome.co.uk> To: "List for Dynare developers" <dev(a)dynare.org> Sent: Friday, June 26, 2009 3:26 PM Subject: Re: [DynareDev] 2nd and 3rd model derivatives + k-order > Dear Sebastien > > Am I right to assume from the enclosed sample from the new <mod>_dynamic.c > sparse hessian that row and column indices and values are 10 position apart > with other values in between ( i.e. apart by Number of variables which > happens to be 10 in this case). That seems to me a bit confusing and I am > wandering what to do when the number of non-zero values is not a multiply of > or no of vars - i.e we are going to have zero values to pad the matrices (as > in the example below, positions 119 and 129 are not populated and in model > with 20 or more vars there will be a lot of padding). > > v2[0]=1; > v2[10]=78; > > v2[20]=(-(T21*T16*params[4]*T94)); > > v2[1]=1; > > v2[11]=158; > > v2[21]=v2[20]; > > v2[2]=1; > > v2[12]=73; > > v2[22]=(-(T21*T16*y[9]*params[4]*T172)); > > ..... > > v2[118]=10; > > v2[128]=109; > > v2[138]=(1+params[6])*(1+params[6]-1)*pow(y[6],1+params[6]-1-1)/(1+params[6] > ); > > } > > What is the advantage in grouping by no-of vars? > > What would be disadvantage if we had indices and values grouped together in > 3 by 3 values? e.g. > > v2[0]=1; > v2[1]=78; > > v2[2]=(-(T21*T16*params[4]*T94)); > > ... > > Best regards > > > George > > ----- Original Message ----- > From: "Sébastien Villemot" <sebastien.villemot(a)ens.fr> > To: <dev(a)dynare.org> > Cc: "Pablo Winant" <pablo.winant(a)gmail.com> > Sent: Thursday, June 25, 2009 11:09 AM > Subject: [DynareDev] 2nd and 3rd model derivatives + k-order > > > Hi everyone, > > In the preprocessor, I have applied (in r2787) a change suggested by > Pablo Winant: it consists in optimizing the way that the sparse matrices > for 2nd and 3rd derivatives (g2 and g3) are constructed in dynamic (and > sometimes static) model files. Thanks to Pablo for his very helpful > patch. > > First, note that this breaks the USE_DLL option at order 2. Since we are > going to drop this option in the future, I'm not planning to fix this. > > Second, a change needs to be made in the k-order library: the matrix for > 2nd order derivatives of the dynamic model, as returned by the MEX file, > has changed. It is now a 3 columns matrix, which contain only the > non-null derivatives of the Hessian. Each line represents a non-zero > element: column 1 is the row index of the non-zero element in the > Hessian, column 2 is its column index, and column 3 is its value. Note > that this is the format required by Matlab sparse() function. > > Best, > > -- > Sébastien Villemot > > > > _______________________________________________ > Dev mailing list > Dev(a)dynare.org > http://www.dynare.org/cgi-bin/mailman/listinfo/dev >

2 3

2nd and 3rd model derivatives + k-order
by Sébastien Villemot 26 Jun '09

26 Jun '09

Hi everyone, In the preprocessor, I have applied (in r2787) a change suggested by Pablo Winant: it consists in optimizing the way that the sparse matrices for 2nd and 3rd derivatives (g2 and g3) are constructed in dynamic (and sometimes static) model files. Thanks to Pablo for his very helpful patch. First, note that this breaks the USE_DLL option at order 2. Since we are going to drop this option in the future, I'm not planning to fix this. Second, a change needs to be made in the k-order library: the matrix for 2nd order derivatives of the dynamic model, as returned by the MEX file, has changed. It is now a 3 columns matrix, which contain only the non-null derivatives of the Hessian. Each line represents a non-zero element: column 1 is the row index of the non-zero element in the Hessian, column 2 is its column index, and column 3 is its value. Note that this is the format required by Matlab sparse() function. Best, -- Sébastien Villemot

2 1

Re: [DynareDev] 2nd and 3rd model derivatives + k-order
by G. Perendia 26 Jun '09

26 Jun '09

Dear Sebastien Can the preprocessor identify and set the number of the non-zero items in the Hessian as, e.g.: M_.sparse_hessian_len Best regards George ----- Original Message ----- From: "G. Perendia" <george(a)perendia.orangehome.co.uk> To: "List for Dynare developers" <dev(a)dynare.org> Sent: Friday, June 26, 2009 8:20 AM Subject: Re: [DynareDev] 2nd and 3rd model derivatives + k-order > Yes, writing it in M_ would help greatly. > Best regards > > George > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > To: "List for Dynare developers" <dev(a)dynare.org> > Sent: Friday, June 26, 2009 7:30 AM > Subject: Re: [DynareDev] 2nd and 3rd model derivatives + k-order > > > Good point! So the preprocessor needs to write in the main *.m file the > number of non-zero elements (probably in M_.) and then > k_order_perturbation can make the correct space available. > > Best, > > Michel > > G. Perendia wrote: > > Thanks Michel > > > > I am well aware of the efficiency of sparse matrices being used for > Hessian > > in general. > > > > However, k-order-perturbation dll has problem in finding out what size > > matrix should be passed to dll to get the Hessian back in.. Not knowing > the > > number of non-zero elements in advance can result in having to pass > between > > two DLLs as much as 3 times larger matrix than the flat Hessian (for both > > data and indices) which can be rather large - and mostly empty!! > > > > Best regards > > > > George > > > > ----- Original Message ----- > > From: "Michel Juillard" <michel.juillard(a)ens.fr> > > To: "List for Dynare developers" <dev(a)dynare.org> > > Sent: Friday, June 26, 2009 7:11 AM > > Subject: Re: [DynareDev] 2nd and 3rd model derivatives + k-order > > > > > > Hi George, > > > > it is inefficient to pass full hessian from *_dynamic.m or *_dynamic.dll > > > > In k_ord_dynare, one uses only non-zero elements to populate the tensors. > > > > The details of implementation will change again in the future, but > > passing only non-zero elements is a step in the right direction. > > > > All the best, > > > > Michel > > > > G. Perendia wrote: > > > >> Dear Sebastien > >> > >> If USE_DLL is going to be discontinued why is MEX/DLL now going to return > >> sparse matrix exclusively when its only future use is for > >> > > k-ord-perturbation > > > >> which requires full matrix. Can we than have PERTURB_DLL option now and > >> return full hessian using the existing (old) mex/dll dynamic model (but > >> renamed e.g. <model>_kordpert.mexXX/dll) when that option is used rather > >> than fixing k-ordpert rather unnecessarily and poss. just temporary since > >> > > we > > > >> may revert to the full hessian returned form mex/dll once USE_DLL is > >> abandoned. > >> > >> I think to remember we initially agreed this course of action anyway. In > >> your email from 9th April 09 you mention: > >> > >> ..."For your own purposes, I have understood that you need a full hessian > >> (instead of a sparse one). Can you confirm this? If it is the case, > >> then I shall implement the "pertub_dll" option (as suggested by > >> Michel), so that the preprocessor gives you a full hessian in that case." > >> > >> And, there are actually quite few issues to be resolved for sparse > >> > > matrices > > > >> being returned from dll. For a start, one problem with fixing k-ordpert > >> > > is > > > >> finding out what size matrix should be passed to dll to get hessian in: > >> > > i.e. > > > >> it may be as much as 3 times bigger than the flat hessian - which can be > >> rather large!! > >> > >> Best regards > >> > >> George > >> ----- Original Message ----- > >> From: "Sébastien Villemot" <sebastien.villemot(a)ens.fr> > >> To: "List for Dynare developers" <dev(a)dynare.org> > >> Sent: Thursday, June 25, 2009 1:44 PM > >> Subject: Re: [DynareDev] 2nd and 3rd model derivatives + k-order > >> > >> > >> Le jeudi 25 juin 2009 à 13:41 +0100, G. Perendia a écrit : > >> > >> > >>> Just to recap, both dr1.m and k-order-perturbation.dll are affected but > >>> > >>> > >> only > >> > >> > >>> k-order-perturbation would need to be fixed to cope with this change > >>> > > since > > > >>> dr1.m will not be using USE_DLL mex option anyway. > >>> > >>> > >> Yes, indeed. > >> > >> > >> > > > > > > _______________________________________________ > > Dev mailing list > > Dev(a)dynare.org > > http://www.dynare.org/cgi-bin/mailman/listinfo/dev > > > > > > > > _______________________________________________ > > Dev mailing list > > Dev(a)dynare.org > > http://www.dynare.org/cgi-bin/mailman/listinfo/dev > > > > > > > _______________________________________________ > Dev mailing list > Dev(a)dynare.org > http://www.dynare.org/cgi-bin/mailman/listinfo/dev >

2 1

Re: [DynareDev] Syntax Highlighting for Dynare
by Michel Juillard 26 Jun '09

26 Jun '09

Dear Hakon, thank you very much. This seems very interesting and useful. I'm now working under Linux, so I won't be able to try it right away, but I suggest that you post a message in the Example/contributions section of the Dynare Forum and that you upload your files there. All the best, Michel Håkon Tretvoll wrote: > Hi Michel, > > I've been using Dynare for a while, and one of the things I have been > working on is to replicate the Bayesian estimation of the model in a > paper by Gertler, Sala and Trigari (*). This is a large model, so > there is a lot to keep track of, and the .mod file becomes quite long > as well. One thing that I have found really helpful in making the file > more readable and hence easier to work with, is to use an editor with > syntax highlighting and code folding enabled to recognize the Dynare > commands and keywords. The editor that I have used is Notepad++, which > among other features, has a simple interface to design a new scheme > for syntax highlighting for a new language. I've used that interface > to design such a scheme, and thought I'd pass it on since I think > others could find it quite useful too. > > If this sounds interesting to you, take a look at the attached .zip > file, try it out, and let me know what you think! > > Best, > > Håkon Tretvoll > > > (*) The paper is "An Estimated Monetary DSGE Model with Unemployment > and Staggered Nominal Wage Bargaining" by Gertler, Sala and Trigari > (Journal of Money, Credit and Banking, Vol. 40, No. 8 (December 2008).

1 0

Dynare 4.0.4 sur MacOS
by Sébastien Villemot 24 Jun '09

24 Jun '09

Salut Christophe, Déjà pour te dire que ton paquet Dynare 4.0.3 pour MacOS m'a été bien utile à la summer school, il y a un utilisateur MacOS pour qui j'aurais été bien embêté sinon. Une première question: pourquoi fait-il 21Mo? le paquet windows fait environ 4Mo pour info. Sinon, est-ce que tu pourrais faire un paquet pour Dynare 4.0.4 qui vient de sortir (le source est sur le site web). On pourrait probablement le rajouter sur www.dynare.org pour téléchargement. Amitiés, -- Sébastien Villemot

2 1

Re: [DynareDev] Quasi triangular matrices in Kalman filters
by G. Perendia 24 Jun '09

24 Jun '09

Dear Michel and Andrea Using the updated version of QTV, the QTV_1.f90, we got some 10% improvement on large matrices: Running 1,000,000 loops with 100x100 matrix and vector: Dgemv : 26 sec QTV high level subroutine: 59 sec QTV_1 2nd version of high level subroutine 53sec QT modular: 240 sec Best regards George Mob. +44(0)7951415480 artilogica(a)btconnect.com ----- Original Message ----- From: "G. Perendia" <george(a)perendia.orangehome.co.uk> To: "Michel Juillard" <michel.juillard(a)ens.fr> Cc: "andrea pagano" <pagano.andrea(a)gmail.com>; "List for Dynare developers" <dev(a)dynare.org> Sent: Wednesday, June 24, 2009 2:19 PM Subject: Re: Quasi triangular matrices in Kalman filters > Dear Michel and Andrea > > I run some initial standalone .exe tests with 2 matrices sizes and 3 > optimised programs: > > dgemv_exe - calling Matlab Lapack/Blas dgemv, > qtvmv_exe - calling new high level QT library subroutine QTV.f90, and, > qtamv_exe - calling 4 different low level QT library subroutines in a > highly modular fashion and performing vector addition at the end, > > Whilst the new high-level subroutine is marginally faster than dgemv for > smaller matrices (11x11), both QT subroutines appear however slower than > dgemv for the larger matrices (100x100). > > > QT , Matlab and GM T*a Stand-alone exe's calculations > > Running 10,000,000 loops with 11x11 matrix and vector: > > Dgemv : 10 sec > > QTV high level subroutine: 9 sec > > QT modular: 32 sec > > Running 1,000,000 loops with 100x100 matrix and vector: > > Dgemv : 26 sec > > QTV high level subroutine: 59 sec > > QT modular: 240 sec > > > > It is quite obvious that the modular approach is no-go as it looks like it > is spending too much time calling 4 f90 subroutines and passing data between > C++ and f90, and an overall optimised, (new) high level subroutine (QTV.f90) > is a much (~4 times?) better solution. > > > > NOTES: > > The QT matrices are from qz decomposition or two rand(size) matrices > > The lapse time has been very consistent across several repeated runs! > > The lapse execution time was measured as difference between the unix date > after and before the .exe runs, e.g. as: > > $ date;./qtamvm_exe QT100t_tab.txt aa100_dat 100 1000000;date > > > > PS: For correct results, one needs to supply a tab-delimited file containing > the transpose of the QT matrices at the entry due to the FORTRAN matrix > orientation being different form C++. > > > Shall I upload the data files too? > > Best regards > > George > Mob. +44(0)7951415480 > artilogica(a)btconnect.com > > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > To: "G. Perendia" <george(a)perendia.orangehome.co.uk> > Cc: "andrea pagano" <pagano.andrea(a)gmail.com>; "List for Dynare developers" > <dev(a)dynare.org> > Sent: Monday, June 22, 2009 7:39 PM > Subject: Re: Quasi triangular matrices in Kalman filters > > > > > > Andrea and George, > > > > 1) please write a standalone test/timing of the QT code so that we can > > profile it using standard tools > > 2) Compare with call to dgmev() using Lapack + optimized Blas (possibly > > from a Matlab distribution). > > 3) Upload the code to SVN, so we can test it on other machines. The huge > > variability between 2 runs reported by George, may be due to Windows and > > usually is less important under Linux. > > > > All the best, > > > > Michel > > > > > > G. Perendia wrote: > > > Dear Andrea > > > > > > Thanks for the new libraries. > > > > > > I run some initial performance tests today for the simple T*a > matrix*vector > > > multiplication with 2 different QT matrices sizes but in summary this is > > > what I am, getting: > > > > > > 10000 iteration loop with 100x100 random QT matrix (from qz > decomposition) > > > and a vector: > > > > > > 1st & 2nd run (after > restart) > > > > > > Native matlab matrix multiplication in a loop > > > > > > Ta_time = 0.3010 & 0.6610 > > > > > > Calling dgemv() using Sylv Vector and General Matrix is faster than > Matlab > > > loop: > > > > > > GMcppTaInnrLoop_time = 0.1600 & 0.3310 > > > > > > Calling QT f90 library using of Sylv Vector and General Matrix: > > > > > > QTcppTaInnerLoop_time = 8.5730 & 20.5300 > > > > > > Calling QT f90 library without use of Sylv Vector and General Matrix but > > > using only pure C/C++ double arrays is only marginally faster: > > > > > > QTcpp_noSylv_TaInnerLoop_time = 8.4420 > > > > > > > > > > > > 1000 loop with 10x10 random QT matrix and vector: > > > > > > For a 10x10 matrix, calling QT f90 library takes about twice the time > Matlab > > > loop does but dgemv is still faster. > > > > > > Matlab: 0.0400 > > > > > > GMcppTaInnrLoop_time = 0.0300 > > > > > > QTcppTaInnerLoop_time = 0.0800 > > > > > > > > > > > > It is, however, possible that the MinGW f95 I am using is not the best > > > optimising compiler that can be used and/or that tests for PTP', which I > am > > > planning to do next, may be better.. > > > > > > > > > > > > What are your thoughts? Do you think that we may be able to improve > > > performance of this multiplication somehow. > > > > > > I wander if making many cross-language calls may be rather detrimental > and > > > that we may improve performance if we reduce this high level of > > > modularisation and calling, e.g. by using a higher level subroutine that > > > will perform all operations within f90, passing back only the final Ta? > > > > > > > > > > > > NOTES: > > > > > > After a restart, Matlab appears to be much slower than later! > > > > > > Also, matlab multiplication reports both, the real and the imaginary > part of > > > the result which appear complex but the real part matches QT and dgemv > > > outputs.. > > > > > > > > > > > > Best regards > > > > > > > > > George > > > artilogica(a)btconnect.com > > > > > > ----- Original Message ----- > > > From: "Michel Juillard" <michel.juillard(a)ens.fr> > > > To: "andrea pagano" <pagano.andrea(a)gmail.com> > > > Cc: "G. Perendia" <george(a)perendia.orangehome.co.uk> > > > Sent: Friday, June 19, 2009 1:33 PM > > > Subject: Re: Quasi triangular matrices in Kalman filters > > > > > > > > > > > >> Thanks Andrea > > >> > > >> amities > > >> > > >> Michel > > >> > > >> andrea pagano wrote: > > >> > > >>> Hi all > > >>> I would go for subroutines. I will do it over the weekend while > > >>> looking at other possibilities fortran pointers. > > >>> > > >>> > > >>> Best > > >>> > > >>> Andrea > > >>> > > >>> > > >>> On Fri, Jun 19, 2009 at 10:04 AM, G. > > >>> Perendia<george(a)perendia.orangehome.co.uk> wrote: > > >>> > > >>> > > >>>> Dear Andrea > > >>>> > > >>>> Problem: > > >>>> > > >>>> I have encountered a problem integrating KalmanFilter with the > > >>>> f90 QT library - passing the QT result arrays back to C++. > > >>>> > > >>>> QT Fortran routines have been written in standard Fortran FUNCTION > > >>>> > > > format, > > > > > >>>> (i.e., not SUBROUTINE), so that they are returning double or single > > >>>> dimensional array (they are named by), by value ( not reference). > > >>>> > > > However, > > > > > >>>> as it appears, only simple, single variables seems can be passed from > > >>>> Fortran > > >>>> FUNCTIONs back to C++ (e.g. INT or REAL). > > >>>> > > >>>> On the other hand, NAG, BLAS and LAPACK routines have all been > written > > >>>> > > > as > > > > > >>>> Fortran SUBROUTINEs and they can be integrated with C more easily - > > >>>> > > > they > > > > > >>>> receive parameters > > >>>> and return their results through the variables passed as calling > > >>>> > > > parameters, > > > > > >>>> by references. > > >>>> > > >>>> For example, dgemv.f from BLAS library gets Y by reference and > returns > > >>>> modified Y passed as calling parameter reference. > > >>>> > > >>>> SUBROUTINE DGEMV(TRANS,M,N,ALPHA,A,LDA,X,INCX,BETA,Y,INCY) > > >>>> .... > > >>>> * Y - DOUBLE PRECISION array of DIMENSION at least > > >>>> ... > > >>>> * Before entry .... the incremented array Y > > >>>> * must contain the vector y. On exit, Y is overwritten by > the > > >>>> * updated vector y. > > >>>> .... > > >>>> > > >>>> Poss. Solutions: > > >>>> > > >>>> I could not find any references on how to get arrays from Fortran > > >>>> > > > FUCTION as > > > > > >>>> return value back to C - do you or anyone around you know how to do > it, > > >>>> > > > if > > > > > >>>> at all possible? In any case, passing array by value is also not > > >>>> > > > recommended > > > > > >>>> as rather un-economical, especially for larger matrices. > > >>>> > > >>>> One alternative way I can think of is less explored option of > returning > > >>>> Fortran pointer to the resulting array from the QT functions instead > > >>>> > > > of the > > > > > >>>> array by value > > >>>> and I think I can work one of that out but suggestions are more than > > >>>> welcome. > > >>>> > > >>>> I can see few options: > > >>>> > > >>>> a) to rewrite QT library as SUBROUTINE instead FUCTION routines, or, > > >>>> > > >>>> b) try to use Fortran pointers and, if we can > > >>>> then also rewrite QT library to return pointers, or, > > >>>> > > >>>> c) write 3 or more high level f90 shell SUBROUTINES calling the > > >>>> > > > existing and > > > > > >>>> unmodified QT functions and performing the all operations needed to > > >>>> construct the resulting Ta and > > >>>> TPT' (for both cases 1 and 2) instead of doing low-level QT > > >>>> > > > manipulation in > > > > > >>>> C++. > > >>>> > > >>>> This way QT library need not be changed and those new SUBROUTINEs > will > > >>>> > > > also > > > > > >>>> act as interface with C++. I think this is a more productive and > > >>>> > > > optimal > > > > > >>>> alternative of the three since those combination utilities would have > > >>>> > > > to be > > > > > >>>> written anyway, except it seem to be easier to do that now in Fortran > > >>>> > > > than > > > > > >>>> in C++. > > >>>> > > >>>> If you like and/or are busy, I can by Monday develop the Ta and the > > >>>> > > > first > > > > > >>>> case of TPT' SUBROUTINES whilst the second case may need more > thinking > > >>>> > > > and > > > > > >>>> more granular approach to take advantage of multiple processors. > > >>>> > > >>>> > > >>>> Please let me know your thoughts on this issue and, whether if you > have > > >>>> > > > time > > > > > >>>> to make the needed changes or additions in the f90 files. > > >>>> > > >>>> Best regards > > >>>> > > >>>> George > > >>>> artilogica(a)btconnect.com > > >>>> > > >>>> ----- Original Message ----- > > >>>> From: "andrea pagano" <pagano.andrea(a)gmail.com> > > >>>> To: <george(a)perendia.orangehome.co.uk> > > >>>> Cc: "Michel Juillard" <Michel.Juillard(a)ens.fr> > > >>>> Sent: Monday, June 01, 2009 7:47 PM > > >>>> Subject: Quasi triangular matrices in Kalman filters > > >>>> > > >>>> > > >>>> > > >>>> > > >>>>> Hi all > > >>>>> > > >>>>> I am sending you a set of Fortran routines to calculate the > matricial > > >>>>> expression in Kalman filter together with some explenations. > > >>>>> > > >>>>> Hope they can be a starting point in optimizing the overall > > >>>>> > > > computation > > > > > >>>>> Best > > >>>>> > > >>>>> Andrea > > >>>>> > > >>>>> > > >>>>> -- > > >>>>> Andrea Pagano > > >>>>> via Veratti VARESE > > >>>>> tel. +3903321691261 > > >>>>> cell.+393403804397 > > >>>>> > > >>>>> > > >>>>> > > >>>> > > >>> > > >>> > > >>> > > >> > > > > > > > > > > > > > > > > >

1 0

Re: [DynareDev] Quasi triangular matrices in Kalman filters
by Michel Juillard 24 Jun '09

24 Jun '09

Andrea and George, 1) please write a standalone test/timing of the QT code so that we can profile it using standard tools 2) Compare with call to dgmev() using Lapack + optimized Blas (possibly from a Matlab distribution). 3) Upload the code to SVN, so we can test it on other machines. The huge variability between 2 runs reported by George, may be due to Windows and usually is less important under Linux. All the best, Michel G. Perendia wrote: > Dear Andrea > > Thanks for the new libraries. > > I run some initial performance tests today for the simple T*a matrix*vector > multiplication with 2 different QT matrices sizes but in summary this is > what I am, getting: > > 10000 iteration loop with 100x100 random QT matrix (from qz decomposition) > and a vector: > > 1st & 2nd run (after restart) > > Native matlab matrix multiplication in a loop > > Ta_time = 0.3010 & 0.6610 > > Calling dgemv() using Sylv Vector and General Matrix is faster than Matlab > loop: > > GMcppTaInnrLoop_time = 0.1600 & 0.3310 > > Calling QT f90 library using of Sylv Vector and General Matrix: > > QTcppTaInnerLoop_time = 8.5730 & 20.5300 > > Calling QT f90 library without use of Sylv Vector and General Matrix but > using only pure C/C++ double arrays is only marginally faster: > > QTcpp_noSylv_TaInnerLoop_time = 8.4420 > > > > 1000 loop with 10x10 random QT matrix and vector: > > For a 10x10 matrix, calling QT f90 library takes about twice the time Matlab > loop does but dgemv is still faster. > > Matlab: 0.0400 > > GMcppTaInnrLoop_time = 0.0300 > > QTcppTaInnerLoop_time = 0.0800 > > > > It is, however, possible that the MinGW f95 I am using is not the best > optimising compiler that can be used and/or that tests for PTP', which I am > planning to do next, may be better.. > > > > What are your thoughts? Do you think that we may be able to improve > performance of this multiplication somehow. > > I wander if making many cross-language calls may be rather detrimental and > that we may improve performance if we reduce this high level of > modularisation and calling, e.g. by using a higher level subroutine that > will perform all operations within f90, passing back only the final Ta? > > > > NOTES: > > After a restart, Matlab appears to be much slower than later! > > Also, matlab multiplication reports both, the real and the imaginary part of > the result which appear complex but the real part matches QT and dgemv > outputs.. > > > > Best regards > > > George > artilogica(a)btconnect.com > > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > To: "andrea pagano" <pagano.andrea(a)gmail.com> > Cc: "G. Perendia" <george(a)perendia.orangehome.co.uk> > Sent: Friday, June 19, 2009 1:33 PM > Subject: Re: Quasi triangular matrices in Kalman filters > > > >> Thanks Andrea >> >> amities >> >> Michel >> >> andrea pagano wrote: >> >>> Hi all >>> I would go for subroutines. I will do it over the weekend while >>> looking at other possibilities fortran pointers. >>> >>> >>> Best >>> >>> Andrea >>> >>> >>> On Fri, Jun 19, 2009 at 10:04 AM, G. >>> Perendia<george(a)perendia.orangehome.co.uk> wrote: >>> >>> >>>> Dear Andrea >>>> >>>> Problem: >>>> >>>> I have encountered a problem integrating KalmanFilter with the >>>> f90 QT library - passing the QT result arrays back to C++. >>>> >>>> QT Fortran routines have been written in standard Fortran FUNCTION >>>> > format, > >>>> (i.e., not SUBROUTINE), so that they are returning double or single >>>> dimensional array (they are named by), by value ( not reference). >>>> > However, > >>>> as it appears, only simple, single variables seems can be passed from >>>> Fortran >>>> FUNCTIONs back to C++ (e.g. INT or REAL). >>>> >>>> On the other hand, NAG, BLAS and LAPACK routines have all been written >>>> > as > >>>> Fortran SUBROUTINEs and they can be integrated with C more easily - >>>> > they > >>>> receive parameters >>>> and return their results through the variables passed as calling >>>> > parameters, > >>>> by references. >>>> >>>> For example, dgemv.f from BLAS library gets Y by reference and returns >>>> modified Y passed as calling parameter reference. >>>> >>>> SUBROUTINE DGEMV(TRANS,M,N,ALPHA,A,LDA,X,INCX,BETA,Y,INCY) >>>> .... >>>> * Y - DOUBLE PRECISION array of DIMENSION at least >>>> ... >>>> * Before entry .... the incremented array Y >>>> * must contain the vector y. On exit, Y is overwritten by the >>>> * updated vector y. >>>> .... >>>> >>>> Poss. Solutions: >>>> >>>> I could not find any references on how to get arrays from Fortran >>>> > FUCTION as > >>>> return value back to C - do you or anyone around you know how to do it, >>>> > if > >>>> at all possible? In any case, passing array by value is also not >>>> > recommended > >>>> as rather un-economical, especially for larger matrices. >>>> >>>> One alternative way I can think of is less explored option of returning >>>> Fortran pointer to the resulting array from the QT functions instead >>>> > of the > >>>> array by value >>>> and I think I can work one of that out but suggestions are more than >>>> welcome. >>>> >>>> I can see few options: >>>> >>>> a) to rewrite QT library as SUBROUTINE instead FUCTION routines, or, >>>> >>>> b) try to use Fortran pointers and, if we can >>>> then also rewrite QT library to return pointers, or, >>>> >>>> c) write 3 or more high level f90 shell SUBROUTINES calling the >>>> > existing and > >>>> unmodified QT functions and performing the all operations needed to >>>> construct the resulting Ta and >>>> TPT' (for both cases 1 and 2) instead of doing low-level QT >>>> > manipulation in > >>>> C++. >>>> >>>> This way QT library need not be changed and those new SUBROUTINEs will >>>> > also > >>>> act as interface with C++. I think this is a more productive and >>>> > optimal > >>>> alternative of the three since those combination utilities would have >>>> > to be > >>>> written anyway, except it seem to be easier to do that now in Fortran >>>> > than > >>>> in C++. >>>> >>>> If you like and/or are busy, I can by Monday develop the Ta and the >>>> > first > >>>> case of TPT' SUBROUTINES whilst the second case may need more thinking >>>> > and > >>>> more granular approach to take advantage of multiple processors. >>>> >>>> >>>> Please let me know your thoughts on this issue and, whether if you have >>>> > time > >>>> to make the needed changes or additions in the f90 files. >>>> >>>> Best regards >>>> >>>> George >>>> artilogica(a)btconnect.com >>>> >>>> ----- Original Message ----- >>>> From: "andrea pagano" <pagano.andrea(a)gmail.com> >>>> To: <george(a)perendia.orangehome.co.uk> >>>> Cc: "Michel Juillard" <Michel.Juillard(a)ens.fr> >>>> Sent: Monday, June 01, 2009 7:47 PM >>>> Subject: Quasi triangular matrices in Kalman filters >>>> >>>> >>>> >>>> >>>>> Hi all >>>>> >>>>> I am sending you a set of Fortran routines to calculate the matricial >>>>> expression in Kalman filter together with some explenations. >>>>> >>>>> Hope they can be a starting point in optimizing the overall >>>>> > computation > >>>>> Best >>>>> >>>>> Andrea >>>>> >>>>> >>>>> -- >>>>> Andrea Pagano >>>>> via Veratti VARESE >>>>> tel. +3903321691261 >>>>> cell.+393403804397 >>>>> >>>>> >>>>> >>>> >>> >>> >>> >> > > > >

2 1

Re: [DynareDev] hessian from csminwel
by Michel Juillard 24 Jun '09

24 Jun '09

Dear George, optimziation routines such as csminwel (or fmincon) need only a rough Hessian, and use brute force to keep it definite positive. It is not appropriate to compute the standard error of the posterior mode, that is why we recompute it using a 7-point formula, but, of course, you need to really be at the correct mode to obtain a positive definite hessian. I imagine that in some cases the hessian returned by the optimization routine can do as variance matrix for the proposal, but in other cases, it can't. In doubt, we force the user the refine his/her mode before going into Metropolis. Best, Michel G. Perendia wrote: > Dear Michel > > Working on Part Info integration we are experiencing frequent problem > with models that run only to end of the first, the ML /mode stage > and fail to run MCMC because they fail to calculate chol(vv), > this due to the hessian of the estimated parameters not being positive > definite - this problem is hampering PI tests: > > ??? Error using ==> chol > Matrix must be positive definite. > Error in ==> metropolis_hastings_initialization at 52 > d = chol(vv); > > Why is Dynare using the additional parameter hessian calculation (i.e > using hessian()) in dynare_estimation_1.m when the last one (already > inverted) returned from csminwel (the default gradient optimiser, or > from fmincon) and passes chol(), may be used for MCMC stage > instead (i.e. as is the situation for the optimiser cases 5 - newrat & > 6)? > > What is disadvantage of using hessian_csminwel instead? (I > do understand though that some of the other optimisers do not produce > the Hessian and such calculation is then needed) > > Best regards > > George > artilogica(a)btconnect.com <mailto:artilogica@btconnect.com>

1 0

Release of Dynare 4.0.4
by Sébastien Villemot 20 Jun '09

20 Jun '09

Hi all, I made the packages for Dynare 4.0.4. The announce is as follows: Changes: * Fixed forecasting after stoch_simul * Fixed crash of the smoother when only one exogenous variable * Fixed issue with compilation of MEX files under 64 bits platforms with Matlab R2009a * Fixed name clash when MOD file is called "ramsey.mod" and computes optimal policy * Fixed USE_DLL option at order 2 * Updates to the reference manual in sections on model resolution and estimation Release notes: * for users of Octave under Windows, you must upgrade your Octave installation to 3.0.5 release, for compatibility with the new packages (see http://www.dynare.org/DynareWiki/DynareOctave ) * Linux packages are available for Ubuntu "Jaunty Jackalope" (9.04), Ubuntu "Intrepid Ibex" (8.10), Ubuntu "Hardy Heron" (8.04) and Debian "Lenny" (5.0) as usual (see http://www.dynare.org/DynareWiki/InstallOnDebianOrUbuntu ) @Michel: the stuff you need for the web site is in /home/villemot on Pythie. You can find there: * the Windows package * the source in .tar.gz * the reference manual in PDF and HTML (more recent than those currently on the web site) Best, -- Sébastien Villemot

1 0

Re: [DynareDev] Kalman Filter
by Michel Juillard 18 Jun '09

18 Jun '09

Thanks George, very interesting. Please proceed with simplifying the KF code. All the best, Michel G. Perendia wrote: > Dear Michel > > I already have a stand-alone exe test I used last week (and uploaded it > today too) and I run it through gprof earlier today (though, after wasting > some time last week trying to use a reportedly sophisticated profiler - > CodeAnalyst from AMD - which, so far, I could not make to work at all). > > The other profiling tool I initially used (and reported) last week ("Very > Sleepy") is an "external", not getting into the code details itself but > could be run attached externally to either the stand-alone exe test or the > Matlab running DLL thread too, and that reported for both spending a lot > (~40%) time in ztrsv solver but, at one snapshot, also pointed to a lot of > time spent in dtrsv and GeneralMatrix copy - 50% each. On the contrast, > gprof is more internal, higher resolution profiler and it puts the load > weight (>10%) on housekeeping functions but does not even mention calls to > external library BLAS functions such as dtrsv and ztrsv. > > The both profiling tools, however, seem to confirm what my early code > inspection concluded too: a very high, use of (not very productive) > General Matrix copy constructor -(e.g. the C++ kalman filter stores copy of > few of the main time variant system matrices: F, P and an intermediate L > for each step of the time series evaluation and also creates a copy of the > input T, H and Z at each step as if they may be time invariant too although > this would not be the case for a simple, no-diffuse KF without missing > observations). > > This then resulted in high % of time in GeneralMatrix copy() function, > (which is called by the copy-constructor explicitly) - i.e. as reported by > the both profiling programs: Sleepy: up to 50% at one snapshot point whilst > gprof gives it the 1st rank with 11% on its own or 27.3% of total with its > children. > > The copy() function is followed by the utility functions such as two > varieties (const and non-const) of Vector indexing [] operator and by the > const and variable varieties of GeneralMatrix::get() elements that utilise > the previous Vector indexing[] operator and are themselves directly called > from the heavily used GenaralMatrix copy function among the rest. > > According to gprof, the above high burden of copy constructor and the > related functions are only then followed by the productive functions such as > PLUFact::multInvRight matrix multiplication with inversion (used for > inversion of the F matrix), the GeneralMatrix constructors and the > GeneralMatrix::gemm() - a general matrix multiplication(itself calling > BLAS dgemm) with 4.7, 3.1 and 2.6 % of total time respectively > > NOTE however that gprof profiler paints to an extent different picture and > does not even mention external BLASS functions such as dtrsv and ztrsv > solvers reported as heavy users by the VerySleepy "external" profiler. > > All in all, it appears form the both profiler reports and my initial > inspection that , for start (and as I initially already intended and > suggested to), we should refactor the current heavy use of the un-productive > General Matrix copy constructor and its current reliance on > element-by-element get() function before we get into any further performance > improvements of the productive functions and external libraries. > > Best regards > > George > > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > To: "List for Dynare developers" <dev(a)dynare.org> > Sent: Saturday, June 06, 2009 3:08 PM > Subject: Re: [DynareDev] Kalman Filter > > > >> There are tools to do profiling in C++. All we need is an standalone >> executable calling the filter. Don't loose time adding timing function >> inside the code. It may be difficult to do profiling in Windows. In that >> case, just prepare the code and we will do the profiling in Linux. >> >> Best >> >> Michel >> >> >> G. Perendia wrote: >> >>> Dear Michel >>> >>> 1) Yes, as agreed initially >>> >>> 2) these are the Matlab Dynare KF measures, mainly to show the >>> > proportion > >>> of inversion vs. pure update in Matlab KF. >>> I have not yet done fine profiling for C++, so, not much to upload >>> > either. > >>> 3) I agree.. >>> >>> Best regards >>> >>> George >>> >>> >>> ----- Original Message ----- >>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>> To: "List for Dynare developers" <dev(a)dynare.org> >>> Sent: Saturday, June 06, 2009 2:10 PM >>> Subject: Re: [DynareDev] Kalman Filter >>> >>> >>> >>> >>>> Thanks George >>>> >>>> One of the first thing that we need to establish is whether identical >>>> basic matrix operations take much longer in the C++ implementation than >>>> in Matlab and, if that it is the case, why. >>>> >>>> >>>> >>>>> 2) Indeed, and as a significant part of the overall parcel of updating >>>>> >>>>> >>> P, >>> >>> >>>>> one needs to invert the updated F too : >>>>> >>>>> 100000 loops of small model KF 4x4 F matrix inversion: iF = >>>>> >>>>> >>> inv(F); >>> >>> >>>>> Fmx_inv_time = 2.2530 >>>>> >>>>> 100000 loops of the corresponding core KF 8x8 P matrix update: >>>>> P1 = T*(P-K*P(mf,:))*transpose(T)+QQ; >>>>> Pupdt_time = 3.4450 >>>>> >>>>> (and also, 100000 loops of the preceding K = P(:,mf)*iF; >>>>> Kupdt_time = 0.5910) >>>>> >>>>> >>>>> >>>>> >>>> How do these operations compare with Matlab on your machine? >>>> >>>> >>>> >>>>> The convergence of P exploited in Matlab Dynare KFs (which does not >>>>> >>>>> >>> require >>> >>> >>>>> further update of P and K and inversion of F ), can improve greatly >>>>> performance >>>>> of KF. >>>>> >>>>> e.g.: running Matlab Dynare KF with 57x57 system matrix in 1000 loop >>>>> 1000 of usual matlabKF_time = 337.1650 >>>>> >>>>> and then using P recursively in the loop with a modified >>>>> > kalman_filter.m > >>>>> which returns P too (therefore, utilising P convergence and avoiding >>>>> > its > >>>>> update for the most of the remaining 999 loops): >>>>> 1000 of recursive: Matlab_rec_KF_time = 11.7060 >>>>> >>>>> >>>>> >>>>> 3) And, although the convergence of P in Matlab KF did not take place >>>>> >>>>> >>> for >>> >>> >>>>> the large sw_euro model which had total run much closer to the C++KF >>>>> >>>>> >>> (see 3 >>> >>> >>>>> below), as today's check show, the convergence did however take place >>>>> >>>>> >>> very >>> >>> >>>>> early in the Matlab KF running the small model I initially tested (at >>>>> >>>>> >>> step >>> >>> >>>>> t=3!!!) so it >>>>> certainly did affect and, judging from the above results, rather >>>>> >>>>> >>> greatly >>> >>> >>>>> contribute to the very much faster KF loops we experienced running >>>>> Matlab KF versus C++ KF in the initial tests with the same small >>>>> > model > >>>>> (C++ KF does yet not take >>>>> advantage of convergence and the comparative results were even)!!! >>>>> >>>>> >>>>> >>>>> >>>> OK, we forget the first comparison on the small model, because C++ and >>>> Matlab didn't use the same algorithm (no convergence monitorinig in >>>> > C++). > >>>> Matlab is still faster by 45% on the medium size model. We should focus >>>> on explaining this difference and we don't need to bring monitoring the >>>> convegence of the filter for this particular example. >>>> >>>> Could you please upload on SVN the code that you use for profiling? >>>> >>>> Best >>>> >>>> Michel >>>> >>>> >>>> >>>>> Best regards >>>>> >>>>> George >>>>> >>>>> >>>>> >>>>> >>>> _______________________________________________ >>>> Dev mailing list >>>> Dev(a)dynare.org >>>> http://www.dynare.org/cgi-bin/mailman/listinfo/dev >>>> >>>> >>> >>> _______________________________________________ >>> Dev mailing list >>> Dev(a)dynare.org >>> http://www.dynare.org/cgi-bin/mailman/listinfo/dev >>> >>> >>> >> _______________________________________________ >> Dev mailing list >> Dev(a)dynare.org >> http://www.dynare.org/cgi-bin/mailman/listinfo/dev >>

2 15

Re: [DynareDev] returning arrays from fortran function to C
by G. Perendia 18 Jun '09

18 Jun '09

PS: As an afterthought I think a more productive and optimal alternative is to write 2-3 high level f90 shell SUBROUTINES calling the QT functions and performing the several operations needed to construct the resulting Ta and TPT' and those will then also act as interface with C++. Best regards George artilogica(a)btconnect.com ----- Original Message ----- From: "G. Perendia" <george(a)perendia.orangehome.co.uk> To: "List for Dynare developers" <dev(a)dynare.org> Sent: Thursday, June 18, 2009 8:06 PM Subject: returning arrays from fortran function to C > Dear Andrea and Michel (et al) > > I have encountered an unexpected problem integrating KalmanFilter with the > f90 QT library - passing the QT result arrays back to C++. > > QT Fortran routines have been written in standard Fortran FUNCTION format, > (i.e., not SUBROUTINE), so that they are returning double or single > dimensional array (they are named by), by value ( not reference) and, as it > appears, only simple, single variables seems can be passed from Fortran > FUNCTIONs back to C++ (e.g. INT or REAL). > > On the other hand, NAG, BLAS and LAPACK routines have all been written as > Fortran SUBROUTINEs and the subroutines are the equivalent of "C" functions > returning "(void)". > > SUBROUTINE can be integrated with C more easily as they receive parameters > and return their results through the variables passed as calling parameters > by references. > > For example, dgemv.f from BLAS library gets Y by reference and returns > modified Y passed as calling parameter reference. > > SUBROUTINE DGEMV(TRANS,M,N,ALPHA,A,LDA,X,INCX,BETA,Y,INCY) > .... > * Y - DOUBLE PRECISION array of DIMENSION at least > ... > * Before entry .... the incremented array Y > * must contain the vector y. On exit, Y is overwritten by the > * updated vector y. > .... > > I could not find any references on how to get arrays from Fortran FUCTION as > return value back to C - does anyone know how to do it or, if at all > possible? > > One way I can think of is less explored option of returning Fortran pointer > to the resulting array from the QT functions instead of the array by value > and I think I can work one of that out. > > However, even if there is another way to pass function result arrays back to > C++, I expect it is bound to be less efficient than passing it by reference > (or Fortran pointer), especially for larger matrices. And, if there is no > efficient alternative, I can either easily rewrite QT library as SUBROUTINE > routines instead FUCTION or try to use Fortran pointers and, if ok, I can > then also rewrite QT library to return pointers.. > > Best regards > > George >

1 0

Re: [DynareDev] Modfile en html
by Sébastien Villemot 17 Jun '09

17 Jun '09

Pablo Winant <pablo.winant(a)gmail.com> a écrit : > Je suis tout à fait d'accord pour mathml. C'est globalement mieux. Il > faut juste faire un choix ente le mode sémantique et le mode > présentation. Pour Dynare, je crois qu'il est plus facile d'écrire le > mode sémantique, qui est de plus celui indiqué pour sérialiser le modèle > (en regardant vite fait j'ai l'impression que sérialiser directement le > modfile demande des changements un peu plus intrusifs). Je peux faire un > test rapide de rendu d'un modèle en Mathml, sans toucher au code source > et voir quel rendu on a. > > jsmath est sous license apache 2 c'est écrit sur la page de Download, > mais je n'ai jamais pensé qu'on pouvait le distribuer. Je me suis juste > dit que c'était plus facile à installer et à utiliser qu'une > distribution Latex nécessaire pour compiler le fichier .tex. Et puis ça > évite dans un premier temps de réécrire la partie correspondante dans > ExprNode.cc. > > Du reste, j'ai surtout fait ça pour me faire la main, si Mathml > fonctionne bien je trouve aussi que c'est la meilleure solution. Désolé > pour les changements d'indentation, il faut que je retienne mon éditeur > de faire ça tout seul ! Je pense que ce serait intéressant d'intégrer la fonctionnalité HTML avec le support MathML. Donc si tu as le temps, n'hésite pas à l'implémenter et j'appliquerai le patch. A+ -- Sébastien Villemot

1 0

Dynare 4.0.4
by Sébastien Villemot 17 Jun '09

17 Jun '09

Hi all, We are planning to release Dynare 4.0.4 on friday, in order to have a new version for the summer school. Please merge any bugfix that you made in the trunk and which is applicable to the 4.0 branch. Best, -- Sébastien Villemot

1 0

Re: [DynareDev] Modfile en html
by Sébastien Villemot 17 Jun '09

17 Jun '09

Salut Pablo, Pablo Winant <pablo.winant(a)gmail.com> a écrit : > La dernière fois qu'on s'est vu, je t'avais parlé d'une librairie > Javascript jsmath (http://www.math.union.edu/~dpvc/jsMath/) qui permet > d'afficher des formules latex dans des pages html moyennant > l'installation des fontes latex dans le navigateur. > > En regardant comment tu as implémenté l'option pour sortir des fichiers > latex, j'ai écrit un petit patch (joint) qui ajoute la même chose pour > du html avec les commandes 'write_html_static_model' et > 'write_html_dymamic_model'. Il écrit le fichier dans le répertoire > courant et par convention, il s'attend à trouver dans le même > répertoire, un dossier jsMath (téléchargeable), et un fichier de style > 'style.css' (joint et très améliorable). Je pense que ce chemin devrait > être spécifiable par une option, mais je ne sais pas trop comment faire > pour l'instant. Merci pour le patch, l'idée est intéressante. Je me demandais pourquoi ne pas utiliser MathML? C'est a priori la façon standard d'insérer des maths dans une page web (recommandée par le W3C). Firefox le supporte directement. Pour IE il faut mettre un plugin. Dans les deux cas j'ai l'impression que l'installation est plus simple que pour jsMath. Voir: http://en.wikipedia.org/wiki/Mathml Le gars qui fait jsMath dit que MathML c'est bien, mais qu'il a fait jsMath parce que c'est plus simple à écrire pour un humain. Mais justement on génère le fichier via un programme. Donc quel avantage à utiliser jsMath qui est non-standard? Par ailleurs je n'ai pas réussi à déterminer la licence sous laquelle est distribué jsMath. Sinon un détail: le patch que tu m'as envoyé est constitué pour une bonne part de changements dans l'identation du code, changements qui cassent les conventions d'indentation C++ pour Dynare. Voir: http://www.dynare.org/DynareWiki/CodingStandards A+ -- Sébastien Villemot

1 0

Re: [DynareDev] Kalman Filter
by Michel Juillard 06 Jun '09

06 Jun '09

One more thing: what is the size of your testing model. How many variables total ? How many state variables? How many observed variables? Can you redo the experiments 3 and 4 for a model with more than 100 state variables? Probably 100 calls will be enough Best Michel G. Perendia wrote: > Dear Michel > > Results of run lapse time using etime(clock,t) are rather surprising and > very disappointing (I had to repeat some in a sheer disbelief) - far the > best is Matlab Dynare kalman_filter.m (even after applying some refactoring > of the kalman.cpp's deep inner loop, not applied to all DLL tests yet)!!!. > > Timing tests on Pentium 2.4 GHz > > 1)calling Kalman_filters.m in the 10000 loop, itself calling the optimised > version of DLL with preparation of H and Z matrices in each loop: > total_dll_time = 202.7320 > > 2)calling core optimised version of Kalman_filter_dll in the 10000 loop > after (i.e., loop without) preparation of H&Z matrices in kalman_filters.m: > core_dll_time= = 1st run: 172.5890 2nd run: 195.0500 at ~93% of CPU time > > 3)calling the Kalman_filter_loop.dll with its own inner 10000 loop from > kalman_filters.m (including the one-off call time) > innrloop_dll_time = 1st run 197.8640 2nd run: 192.5970 > 3rd run (after some refactoring of the kalman.cpp filter deep inner loop > not yet used in other DLL tests): 184.9760 > > 4)calling Dynare SVN kalman_filter.m in 10000 loop > matlabKF_time = 1st run: 48.9600 2nd run: 48.4600 > > 5) for a comparison and sanity check: calling Kalman_filters running the > debug version of DLL in the 10000 loop > dbug total_dll_time=etime(clock, t) = 642.4140 running at ~93% of CPU time > > It is possible that using proper, optimised lapack and BLAS may give a > better performance for C++. (NOTE: the DLLs used Matlab lapack and blas > libraries.) > > Shall I nevertheless continue with the C++ version of DsgeLikelihood as > planned or try to revise the performance tests first? > > Best regards > > George > artilogica(a)btconnect.com > > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > To: "G. Perendia" <george(a)perendia.orangehome.co.uk> > Sent: Monday, June 01, 2009 10:32 AM > Subject: Re: [DynareDev] Kalman Filter > > > >> Thanks George, now it is clear. I'm also interested by a timing >> comparison. Could you time >> a) 10'000 calls to the Matlab filter routine >> b) 10'000 calls to the C++ filter routine >> c) add the 10'000 loop inside the DLL and time one call to this modified >> function. >> >> I guess after that we should pass to the coding in C++ of DsgeLikelihood.m >> >> All the best, >> >> Michel >> >> >> G. Perendia wrote: >> >>> Dear Michel >>> >>> No, I do not think there is additional problem (other than with Diffuse >>> Likelihood*.m files when if used) or that correction was not sufficient. >>> >>> Over the weekend I was just trying full 1st stage estimation with a >>> > model we > >>> use for Part Info, with and without the corrections, which just shows >>> > the > >>> scale of the effect of the log likelihood calculation error (now >>> > properly > >>> corrected for the SVN likelihood files) . As you can see in the .rtf >>> > file, > >>> in the few of the tests with SVN version (using >>> > /likelihood/kalman.filter.m) > >>> I used your new code too, but for comparing results in older versions of >>> Dynare- v4.0.2 and 4.0.3 - (using Diffuse Likelihood1.m), I used my >>> > quick > >>> fix applied to the 4.02 DiffuseLikelihod*.m that I also enclosed. >>> >>> Also, some additional tests I did over the weekend show that, after the >>> correction, Dynare now reports estimates more comparable to the new Part >>> Info method than before the correction, also proving that the correction >>> works ! >>> >>> As you suggested, I run a test of your new correction that can be >>> > compared > >>> to C++ and, for the test model I used, F converges at t=3 (start is 41), >>> > and > >>> your new kalman_filter.m code gives total >>> LIK=1640935.5855619563 >>> which is virtually the same as C++ and the quick fix for Diffuse >>> > Likelihood > >>> I tested last week (requoted from below): >>> >>> >>> >>>>>>> the reported likelihood for presample=40 from (quick >>>>>>> >>>>>>> >>> fix) Matlab KF >>> >>> >>>>>>> 1640935.5855267849 >>>>>>> which is nearly the same as that from C++ KF below: >>>>>>> 1640935.5854489324 >>>>>>> >>>>>>> >>> Let me know if you want me to run more tests or work on correcting any >>> > other > >>> Dynare Matlab code! >>> >>> Best regards >>> >>> George >>> >>> ----- Original Message ----- >>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>> To: "G. Perendia" <george(a)perendia.orangehome.co.uk> >>> Sent: Monday, June 01, 2009 9:08 AM >>> Subject: Re: [DynareDev] Kalman Filter >>> >>> >>> >>> >>>> Dear George, >>>> >>>> I'm a bit confused. We agreed that all the versions of Kalman >>>> likelihood, both in ./matlab and in ./matlab/kalman/likelihood were >>>> wrong when presample was larger than the period where the filter >>>> > reaches > >>>> its stationary state. So I don't see the point in still using this code >>>> for testing. >>>> >>>> the code in matlab/kalman/likelihood was corrected last Wednesday on >>>> > SVN > >>>> r2007 >>>> >>>> Do you have reasons to believe that the correction wasn't sufficient? >>>> > Or > >>>> is there another source of discrepancy between the Matlab code and the >>>> C++ code? >>>> Before comparing results of entire estimation procedures, we have to >>>> make sure that all the outputs from the Matlab and C++ functions with >>>> the same inputs are close enough. >>>> >>>> Note that we still have to correct DsgeLikelihood_hh.m that calls the >>>> obsolete DiffuseLikelihood*.m routines instead of the ones in >>>> kalman/likelihood >>>> >>>> Then we should just remove the DiffuseLikelihood*.m functions >>>> >>>> >>>> Best >>>> >>>> Michel >>>> >>>> >>>> G. Perendia wrote: >>>> >>>> >>>>> Dear Michel >>>>> >>>>> Problem and discrepancies with log likelihood calculation with >>>>> >>>>> >>> presampling >>> >>> >>>>> (start>1) is something I have been encountering for some time (i.e. >>>>> > few > >>>>> weeks) already , whilst doing some tests for the new partial >>>>> > information > >>>>> kalman estimation, but, expecting that the problem is in the new >>>>> > Partial > >>>>> info code, I did not get to the bottom of the cause until I >>>>> > encountered > >>>>> discrepancy between the C++ KF (modified to handle presampling >>>>> > properly) > >>> and >>> >>> >>>>> the Dynare KF too. It was that second discrepancy that led me to >>>>> > analyse > >>>>> Dynare KF handling of presampling and to find the problem there. >>>>> >>>>> I did some more preliminary tests yesterday with the older and the >>>>> >>>>> >>> latest >>> >>> >>>>> SVN versions of Dynare, without and with the correction (including >>>>> correction using the fix based around the min() function I initially >>>>> suggested and now applied as a quick fix to the older >>>>> >>>>> >>> DiffuseLikelihood1.m) >>> >>> >>>>> and the estimation results with and without correction are >>>>> > significantly > >>>>> different as can be seen from the enclosed rtf document which uses two >>>>> >>>>> >>> very >>> >>> >>>>> similar models, (i.e. one with gamma>0 the other with gamma=0), to >>>>> >>>>> >>> compare >>> >>> >>>>> the results (also very similar to the one I sent to Stephane). >>>>> >>>>> Although the initial calculations of log-likelihood ("Initial value of >>>>> >>>>> >>> the >>> >>> >>>>> log posterior (or likelihood): ") are same, (i.e. when reste is 0 or >>>>> >>>>> >>> very >>> >>> >>>>> small), it appears that the differences builds up during the >>>>> > estimation > >>> as >>> >>> >>>>> the kalman filter cov matrix start to converges earlier and earlier, >>>>> eventually before the given presample start is reached, as the later >>>>> > in > >>> the >>> >>> >>>>> ML and estimation error minimisation we are (and probably throughout >>>>> > the > >>>>> MCMC estimation process), when the parameters get close to the mode >>>>> (especially if the parameter priors were taken from previously >>>>> > estimated > >>>>> posteriors as is the case with the test models I used). >>>>> >>>>> (I did not yet run MCMC due to estimation failing with the chol() >>>>> non-positive definite error so I would need to change starting >>>>> > values.) > >>>>> Let me know if you would like me to develop and perform more >>>>> > structured > >>>>> tests on a large range of models. >>>>> >>>>> >>>>> Best regards >>>>> >>>>> George >>>>> >>>>> ----- Original Message ----- >>>>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>>>> To: "List for Dynare developers" <dev(a)dynare.org> >>>>> Cc: "G. Perendia" <george(a)perendia.orangehome.co.uk> >>>>> Sent: Wednesday, May 27, 2009 12:16 PM >>>>> Subject: Re: [DynareDev] Kalman Filter >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> In fact the discrepancy found by George occurs only for start ~= 1 >>>>>> So it is clearly related to the handling of the constants. >>>>>> >>>>>> Best >>>>>> >>>>>> Michel >>>>>> >>>>>> >>>>>> Stéphane Adjemian wrote: >>>>>> >>>>>> >>>>>> >>>>>>> George, Can you share your example. In my experience (with medium >>>>>>> scaled models) it takes much more iterations to get to the steady >>>>>>> state kalman filter... >>>>>>> >>>>>>> I didn't look at the cc codes, but one possible source of >>>>>>> > discrepancy > >>>>>>> between matlab and cc may be the steady state KF criterion. I test >>>>>>> > for > >>>>>>> the stationarity of KF considering the changes in the biggest >>>>>>> > elements > >>>>>>> of the Kalman Gain matrix, not the changes in the determinant of the >>>>>>> forecast errors covariance matrix... >>>>>>> >>>>>>> Best, >>>>>>> Stéphane. >>>>>>> >>>>>>> >>>>>>> 2009/5/27 G. Perendia <george(a)perendia.orangehome.co.uk >>>>>>> <mailto:george@perendia.orangehome.co.uk>> >>>>>>> >>>>>>> >>>>>>> 1. In my experience, the filter cov. determinants dF becomes >>>>>>> stationary after several steps (5-10), e.g. 7 >>>>>>> and number of periods the filter is stationary is >>>>>>> reste=smpl-7 >>>>>>> whilst, if start =41 (presample=40) than reste is then larger >>>>>>> > than > >>>>>>> smpl -start+1=smpl-40. >>>>>>> >>>>>>> 2. it is not problematic to add all the determinants at once in >>>>>>> the last period of the stationary filter >>>>>>> as long as we subtract from the total sum >>>>>>> >>>>>>> start-1-(smpl-reste) = 41 -1 -(smpl-smpl+7)=33 >>>>>>> determinants, or, add only >>>>>>> min(reste,(smpl-start+1)) >>>>>>> Best regards >>>>>>> >>>>>>> George >>>>>>> >>>>>>> ----- Original Message ----- >>>>>>> *From:* Stéphane Adjemian >>>>>>> > <mailto:stephane.adjemian@gmail.com> > >>>>>>> *To:* List for Dynare developers <mailto:dev@dynare.org> >>>>>>> *Cc:* G. Perendia <mailto:george@perendia.orangehome.co.uk> >>>>>>> *Sent:* Wednesday, May 27, 2009 11:04 AM >>>>>>> *Subject:* Re: [DynareDev] Kalman Filter >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> I agree, the matlab code is very unclear (even if I had fun >>>>>>> writting it this way ;-) and prone to errors if one uses the >>>>>>> vector lik (Marco is using it). I would rather prefer to add >>>>>>> the constants outside of the loop with a (sub)vector >>>>>>> operation, this should be more efficient. I will do it today >>>>>>> or tomorrow. >>>>>>> >>>>>>> Best, >>>>>>> Stéphane. >>>>>>> >>>>>>> 2009/5/27 Michel Juillard <michel.juillard(a)ens.fr >>>>>>> <mailto:michel.juillard@ens.fr>> >>>>>>> >>>>>>> On closer inspection, I don't think that the expression >>>>>>> pointed by George in kalman_filter.m is wrong: >>>>>>> >>>>>>> 1. reste = smpl-t or the number of periods during which >>>>>>> the filter is stationary. This shouldn't be larger than >>>>>>> T-start+1 >>>>>>> >>>>>>> 2. it is problematic (see below) but not wrong to add >>>>>>> > all > >>>>>>> the determinants at once in the last period of the >>>>>>> stationary filter >>>>>>> >>>>>>> 3. I don't think this explains the difference with the >>>>>>> > C++ > >>>>>>> version of the filter and we still have to look for it. >>>>>>> >>>>>>> 4. it remains that the current code is very unclear and >>>>>>> that if LIK is correct the vector lik doesn't have the >>>>>>> correct constants on each elements. >>>>>>> >>>>>>> 5. I would like to simplify the code and add the correct >>>>>>> constant to each element of the lik vector. It would be >>>>>>> > a > >>>>>>> little bit less efficient in Matlab than the current >>>>>>> > code, > >>>>>>> but I doubt it would be noticeable. >>>>>>> Stephane, what do you think? >>>>>>> >>>>>>> >>>>>>> Best >>>>>>> >>>>>>> Michel >>>>>>> >>>>>>> G. Perendia wrote: >>>>>>> >>>>>>> Dear Michel >>>>>>> >>>>>>> I think I found an error in Dynare Matlab >>>>>>> kalman_filter. suite of utilities >>>>>>> which affects the likelihood LIK results with >>>>>>> > start>1 > >>>>>>> (i.e. presampling>0): >>>>>>> >>>>>>> the calculation speed-up construct which relies on >>>>>>> converged covariance >>>>>>> matrix >>>>>>> >>>>>>> lik(t) = lik(t) + reste*log(dF); >>>>>>> >>>>>>> adds reste * log(dF) to the last-1 (i.e. the smpl) >>>>>>> member of lik >>>>>>> (the last, the lik(smpl+1) one contains >>>>>>> >>>>>>> >>>>>>> >>>>> smpl*pp*log(2*pi)) >>>>> >>>>> >>>>> >>>>>>> but reste is usually larger than T-start+1 so that >>>>>>> >>>>>>> LIK = >>>>>>> >>>>>>> >>>>>>> >>>>> .5*(sum(lik(start:end))-(start-1)*lik(smpl+1)/smpl) >>>>> >>>>> >>>>> >>>>>>> has much more log(dF)s added than required since >>>>>>> > they > >>>>>>> are all concentrated >>>>>>> in the last-1 (the T) member >>>>>>> >>>>>>> For example, if I change the above construct to >>>>>>> lik(t) = lik(t) + min(reste,(smpl-start+1))*log(dF); >>>>>>> >>>>>>> the reported likelihood for presample=40 from Matlab >>>>>>> >>>>>>> >>> KF >>> >>> >>>>> is >>>>> >>>>> >>>>> >>>>>>> 1640935.5855267849 >>>>>>> which is nearly the same as that from C++ KF below: >>>>>>> 1640935.5854489324 >>>>>>> >>>>>>> Shall I make changes to kalman/likelihood/ KFs and >>>>>>> upload the .m files? >>>>>>> This problem affects also the older versions of >>>>>>> DiffuseLikelihood**.m too. >>>>>>> >>>>>>> Best regards >>>>>>> >>>>>>> George >>>>>>> artilogica(a)btconnect.com >>>>>>> >>>>>>> >>>>>>> >>>>> <mailto:artilogica@btconnect.com> >>>>> >>>>> >>>>> >>>>>>> ----- Original Message ----- From: "Michel Juillard" >>>>>>> <michel.juillard(a)ens.fr >>>>>>> >>>>>>> >>> <mailto:michel.juillard@ens.fr>> >>> >>> >>>>>>> To: "G. Perendia" <george(a)perendia.orangehome.co.uk >>>>>>> <mailto:george@perendia.orangehome.co.uk>> >>>>>>> Sent: Tuesday, May 26, 2009 10:32 AM >>>>>>> Subject: Re: Kalman Filter+PS >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Hi George, >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Re 1) below: >>>>>>> I modified C++ KF so that it reports >>>>>>> log-likelihood for given >>>>>>> start/preampling in same/similar manner as >>>>>>> > the > >>>>>>> Matlab KFs do and I am >>>>>>> getting approximately close results, e.g. >>>>>>> >>>>>>> ll= -1640935.5854489324 for C++ and >>>>>>> (-) 1640482.4179242959 for Matlab KF (for >>>>>>> start=41, i.e. >>>>>>> >>>>>>> >>>>>>> presample=40). >>>>>>> >>>>>>> >>>>>>> whilst they appear same for presample=0 >>>>>>> (e.g.2.5906e+006), i.e. >>>>>>> -2590556.989730841 vs >>>>>>> 2590556.989778722 >>>>>>> >>>>>>> Are those results acceptably close or should >>>>>>> > I > >>>>>>> investigate further where >>>>>>> >>>>>>> >>>>>>> the >>>>>>> >>>>>>> >>>>>>> above difference may come form? >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> This indicates a problem . The difference should >>>>>>> be the same with and >>>>>>> without presample. It may come from the >>>>>>> computation of the likelihood >>>>>>> constant. This is done in a very obscure manner >>>>>>> > in > >>>>>>> Dynare Matlab. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> Dev(a)dynare.org <mailto:Dev@dynare.org> >>>>>>> http://www.dynare.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Stéphane Adjemian >>>>>>> CEPREMAP & Université du Maine >>>>>>> >>>>>>> Tel: (33)1-43-13-62-39 >>>>>>> >>>>>>> >>>>>>> >>>>>>> > ------------------------------------------------------------------------ > >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> Dev(a)dynare.org <mailto:Dev@dynare.org> >>>>>>> http://www.dynare.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> Dev(a)dynare.org <mailto:Dev@dynare.org> >>>>>>> http://www.dynare.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Stéphane Adjemian >>>>>>> CEPREMAP & Université du Maine >>>>>>> >>>>>>> Tel: (33)1-43-13-62-39 >>>>>>> >>>>>>> >>>>> ----------------------------------------------------------------------- >>>>> > - > >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> Dev(a)dynare.org >>>>>>> http://www.dynare.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> _______________________________________________ >>>>>> Dev mailing list >>>>>> Dev(a)dynare.org >>>>>> http://www.dynare.org/cgi-bin/mailman/listinfo/dev >>>>>> >>>>>> >>>>>> >>>>>> >>> >>> >>> >> > > > >

2 9

[Fwd: Quasi triangular matrices in Kalman filters]
by Michel Juillard 01 Jun '09

01 Jun '09

Here are the routines prepared by Andrea All the best, Michel -------- Original Message -------- Subject: Quasi triangular matrices in Kalman filters Date: Mon, 1 Jun 2009 20:47:20 +0200 From: andrea pagano <pagano.andrea(a)gmail.com> To: george(a)perendia.orangehome.co.uk CC: Michel Juillard <Michel.Juillard(a)ens.fr> Hi all I am sending you a set of Fortran routines to calculate the matricial expression in Kalman filter together with some explenations. Hope they can be a starting point in optimizing the overall computation Best Andrea -- Andrea Pagano via Veratti VARESE tel. +3903321691261 cell.+393403804397

1 0

Re: [DynareDev] Kalman Filter
by Michel Juillard 01 Jun '09

01 Jun '09

This is indeed disapointing. Could you profile the code for your experiments 3) and 4) in order to try to understand where is the C++ filter at a disadvantage. You probably need to write a main program in C++ setting up the matrices in order to profile the C++ code. If we can't write a C++ filter than runs faster than Matlab, there is no point in trying to write a C++ estimation DLL. All the best, Michel G. Perendia wrote: > Dear Michel > > Results of run lapse time using etime(clock,t) are rather surprising and > very disappointing (I had to repeat some in a sheer disbelief) - far the > best is Matlab Dynare kalman_filter.m (even after applying some refactoring > of the kalman.cpp's deep inner loop, not applied to all DLL tests yet)!!!. > > Timing tests on Pentium 2.4 GHz > > 1)calling Kalman_filters.m in the 10000 loop, itself calling the optimised > version of DLL with preparation of H and Z matrices in each loop: > total_dll_time = 202.7320 > > 2)calling core optimised version of Kalman_filter_dll in the 10000 loop > after (i.e., loop without) preparation of H&Z matrices in kalman_filters.m: > core_dll_time= = 1st run: 172.5890 2nd run: 195.0500 at ~93% of CPU time > > 3)calling the Kalman_filter_loop.dll with its own inner 10000 loop from > kalman_filters.m (including the one-off call time) > innrloop_dll_time = 1st run 197.8640 2nd run: 192.5970 > 3rd run (after some refactoring of the kalman.cpp filter deep inner loop > not yet used in other DLL tests): 184.9760 > > 4)calling Dynare SVN kalman_filter.m in 10000 loop > matlabKF_time = 1st run: 48.9600 2nd run: 48.4600 > > 5) for a comparison and sanity check: calling Kalman_filters running the > debug version of DLL in the 10000 loop > dbug total_dll_time=etime(clock, t) = 642.4140 running at ~93% of CPU time > > It is possible that using proper, optimised lapack and BLAS may give a > better performance for C++. (NOTE: the DLLs used Matlab lapack and blas > libraries.) > > Shall I nevertheless continue with the C++ version of DsgeLikelihood as > planned or try to revise the performance tests first? > > Best regards > > George > artilogica(a)btconnect.com > > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > To: "G. Perendia" <george(a)perendia.orangehome.co.uk> > Sent: Monday, June 01, 2009 10:32 AM > Subject: Re: [DynareDev] Kalman Filter > > > >> Thanks George, now it is clear. I'm also interested by a timing >> comparison. Could you time >> a) 10'000 calls to the Matlab filter routine >> b) 10'000 calls to the C++ filter routine >> c) add the 10'000 loop inside the DLL and time one call to this modified >> function. >> >> I guess after that we should pass to the coding in C++ of DsgeLikelihood.m >> >> All the best, >> >> Michel >> >> >> G. Perendia wrote: >> >>> Dear Michel >>> >>> No, I do not think there is additional problem (other than with Diffuse >>> Likelihood*.m files when if used) or that correction was not sufficient. >>> >>> Over the weekend I was just trying full 1st stage estimation with a >>> > model we > >>> use for Part Info, with and without the corrections, which just shows >>> > the > >>> scale of the effect of the log likelihood calculation error (now >>> > properly > >>> corrected for the SVN likelihood files) . As you can see in the .rtf >>> > file, > >>> in the few of the tests with SVN version (using >>> > /likelihood/kalman.filter.m) > >>> I used your new code too, but for comparing results in older versions of >>> Dynare- v4.0.2 and 4.0.3 - (using Diffuse Likelihood1.m), I used my >>> > quick > >>> fix applied to the 4.02 DiffuseLikelihod*.m that I also enclosed. >>> >>> Also, some additional tests I did over the weekend show that, after the >>> correction, Dynare now reports estimates more comparable to the new Part >>> Info method than before the correction, also proving that the correction >>> works ! >>> >>> As you suggested, I run a test of your new correction that can be >>> > compared > >>> to C++ and, for the test model I used, F converges at t=3 (start is 41), >>> > and > >>> your new kalman_filter.m code gives total >>> LIK=1640935.5855619563 >>> which is virtually the same as C++ and the quick fix for Diffuse >>> > Likelihood > >>> I tested last week (requoted from below): >>> >>> >>> >>>>>>> the reported likelihood for presample=40 from (quick >>>>>>> >>>>>>> >>> fix) Matlab KF >>> >>> >>>>>>> 1640935.5855267849 >>>>>>> which is nearly the same as that from C++ KF below: >>>>>>> 1640935.5854489324 >>>>>>> >>>>>>> >>> Let me know if you want me to run more tests or work on correcting any >>> > other > >>> Dynare Matlab code! >>> >>> Best regards >>> >>> George >>> >>> ----- Original Message ----- >>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>> To: "G. Perendia" <george(a)perendia.orangehome.co.uk> >>> Sent: Monday, June 01, 2009 9:08 AM >>> Subject: Re: [DynareDev] Kalman Filter >>> >>> >>> >>> >>>> Dear George, >>>> >>>> I'm a bit confused. We agreed that all the versions of Kalman >>>> likelihood, both in ./matlab and in ./matlab/kalman/likelihood were >>>> wrong when presample was larger than the period where the filter >>>> > reaches > >>>> its stationary state. So I don't see the point in still using this code >>>> for testing. >>>> >>>> the code in matlab/kalman/likelihood was corrected last Wednesday on >>>> > SVN > >>>> r2007 >>>> >>>> Do you have reasons to believe that the correction wasn't sufficient? >>>> > Or > >>>> is there another source of discrepancy between the Matlab code and the >>>> C++ code? >>>> Before comparing results of entire estimation procedures, we have to >>>> make sure that all the outputs from the Matlab and C++ functions with >>>> the same inputs are close enough. >>>> >>>> Note that we still have to correct DsgeLikelihood_hh.m that calls the >>>> obsolete DiffuseLikelihood*.m routines instead of the ones in >>>> kalman/likelihood >>>> >>>> Then we should just remove the DiffuseLikelihood*.m functions >>>> >>>> >>>> Best >>>> >>>> Michel >>>> >>>> >>>> G. Perendia wrote: >>>> >>>> >>>>> Dear Michel >>>>> >>>>> Problem and discrepancies with log likelihood calculation with >>>>> >>>>> >>> presampling >>> >>> >>>>> (start>1) is something I have been encountering for some time (i.e. >>>>> > few > >>>>> weeks) already , whilst doing some tests for the new partial >>>>> > information > >>>>> kalman estimation, but, expecting that the problem is in the new >>>>> > Partial > >>>>> info code, I did not get to the bottom of the cause until I >>>>> > encountered > >>>>> discrepancy between the C++ KF (modified to handle presampling >>>>> > properly) > >>> and >>> >>> >>>>> the Dynare KF too. It was that second discrepancy that led me to >>>>> > analyse > >>>>> Dynare KF handling of presampling and to find the problem there. >>>>> >>>>> I did some more preliminary tests yesterday with the older and the >>>>> >>>>> >>> latest >>> >>> >>>>> SVN versions of Dynare, without and with the correction (including >>>>> correction using the fix based around the min() function I initially >>>>> suggested and now applied as a quick fix to the older >>>>> >>>>> >>> DiffuseLikelihood1.m) >>> >>> >>>>> and the estimation results with and without correction are >>>>> > significantly > >>>>> different as can be seen from the enclosed rtf document which uses two >>>>> >>>>> >>> very >>> >>> >>>>> similar models, (i.e. one with gamma>0 the other with gamma=0), to >>>>> >>>>> >>> compare >>> >>> >>>>> the results (also very similar to the one I sent to Stephane). >>>>> >>>>> Although the initial calculations of log-likelihood ("Initial value of >>>>> >>>>> >>> the >>> >>> >>>>> log posterior (or likelihood): ") are same, (i.e. when reste is 0 or >>>>> >>>>> >>> very >>> >>> >>>>> small), it appears that the differences builds up during the >>>>> > estimation > >>> as >>> >>> >>>>> the kalman filter cov matrix start to converges earlier and earlier, >>>>> eventually before the given presample start is reached, as the later >>>>> > in > >>> the >>> >>> >>>>> ML and estimation error minimisation we are (and probably throughout >>>>> > the > >>>>> MCMC estimation process), when the parameters get close to the mode >>>>> (especially if the parameter priors were taken from previously >>>>> > estimated > >>>>> posteriors as is the case with the test models I used). >>>>> >>>>> (I did not yet run MCMC due to estimation failing with the chol() >>>>> non-positive definite error so I would need to change starting >>>>> > values.) > >>>>> Let me know if you would like me to develop and perform more >>>>> > structured > >>>>> tests on a large range of models. >>>>> >>>>> >>>>> Best regards >>>>> >>>>> George >>>>> >>>>> ----- Original Message ----- >>>>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>>>> To: "List for Dynare developers" <dev(a)dynare.org> >>>>> Cc: "G. Perendia" <george(a)perendia.orangehome.co.uk> >>>>> Sent: Wednesday, May 27, 2009 12:16 PM >>>>> Subject: Re: [DynareDev] Kalman Filter >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> In fact the discrepancy found by George occurs only for start ~= 1 >>>>>> So it is clearly related to the handling of the constants. >>>>>> >>>>>> Best >>>>>> >>>>>> Michel >>>>>> >>>>>> >>>>>> Stéphane Adjemian wrote: >>>>>> >>>>>> >>>>>> >>>>>>> George, Can you share your example. In my experience (with medium >>>>>>> scaled models) it takes much more iterations to get to the steady >>>>>>> state kalman filter... >>>>>>> >>>>>>> I didn't look at the cc codes, but one possible source of >>>>>>> > discrepancy > >>>>>>> between matlab and cc may be the steady state KF criterion. I test >>>>>>> > for > >>>>>>> the stationarity of KF considering the changes in the biggest >>>>>>> > elements > >>>>>>> of the Kalman Gain matrix, not the changes in the determinant of the >>>>>>> forecast errors covariance matrix... >>>>>>> >>>>>>> Best, >>>>>>> Stéphane. >>>>>>> >>>>>>> >>>>>>> 2009/5/27 G. Perendia <george(a)perendia.orangehome.co.uk >>>>>>> <mailto:george@perendia.orangehome.co.uk>> >>>>>>> >>>>>>> >>>>>>> 1. In my experience, the filter cov. determinants dF becomes >>>>>>> stationary after several steps (5-10), e.g. 7 >>>>>>> and number of periods the filter is stationary is >>>>>>> reste=smpl-7 >>>>>>> whilst, if start =41 (presample=40) than reste is then larger >>>>>>> > than > >>>>>>> smpl -start+1=smpl-40. >>>>>>> >>>>>>> 2. it is not problematic to add all the determinants at once in >>>>>>> the last period of the stationary filter >>>>>>> as long as we subtract from the total sum >>>>>>> >>>>>>> start-1-(smpl-reste) = 41 -1 -(smpl-smpl+7)=33 >>>>>>> determinants, or, add only >>>>>>> min(reste,(smpl-start+1)) >>>>>>> Best regards >>>>>>> >>>>>>> George >>>>>>> >>>>>>> ----- Original Message ----- >>>>>>> *From:* Stéphane Adjemian >>>>>>> > <mailto:stephane.adjemian@gmail.com> > >>>>>>> *To:* List for Dynare developers <mailto:dev@dynare.org> >>>>>>> *Cc:* G. Perendia <mailto:george@perendia.orangehome.co.uk> >>>>>>> *Sent:* Wednesday, May 27, 2009 11:04 AM >>>>>>> *Subject:* Re: [DynareDev] Kalman Filter >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> I agree, the matlab code is very unclear (even if I had fun >>>>>>> writting it this way ;-) and prone to errors if one uses the >>>>>>> vector lik (Marco is using it). I would rather prefer to add >>>>>>> the constants outside of the loop with a (sub)vector >>>>>>> operation, this should be more efficient. I will do it today >>>>>>> or tomorrow. >>>>>>> >>>>>>> Best, >>>>>>> Stéphane. >>>>>>> >>>>>>> 2009/5/27 Michel Juillard <michel.juillard(a)ens.fr >>>>>>> <mailto:michel.juillard@ens.fr>> >>>>>>> >>>>>>> On closer inspection, I don't think that the expression >>>>>>> pointed by George in kalman_filter.m is wrong: >>>>>>> >>>>>>> 1. reste = smpl-t or the number of periods during which >>>>>>> the filter is stationary. This shouldn't be larger than >>>>>>> T-start+1 >>>>>>> >>>>>>> 2. it is problematic (see below) but not wrong to add >>>>>>> > all > >>>>>>> the determinants at once in the last period of the >>>>>>> stationary filter >>>>>>> >>>>>>> 3. I don't think this explains the difference with the >>>>>>> > C++ > >>>>>>> version of the filter and we still have to look for it. >>>>>>> >>>>>>> 4. it remains that the current code is very unclear and >>>>>>> that if LIK is correct the vector lik doesn't have the >>>>>>> correct constants on each elements. >>>>>>> >>>>>>> 5. I would like to simplify the code and add the correct >>>>>>> constant to each element of the lik vector. It would be >>>>>>> > a > >>>>>>> little bit less efficient in Matlab than the current >>>>>>> > code, > >>>>>>> but I doubt it would be noticeable. >>>>>>> Stephane, what do you think? >>>>>>> >>>>>>> >>>>>>> Best >>>>>>> >>>>>>> Michel >>>>>>> >>>>>>> G. Perendia wrote: >>>>>>> >>>>>>> Dear Michel >>>>>>> >>>>>>> I think I found an error in Dynare Matlab >>>>>>> kalman_filter. suite of utilities >>>>>>> which affects the likelihood LIK results with >>>>>>> > start>1 > >>>>>>> (i.e. presampling>0): >>>>>>> >>>>>>> the calculation speed-up construct which relies on >>>>>>> converged covariance >>>>>>> matrix >>>>>>> >>>>>>> lik(t) = lik(t) + reste*log(dF); >>>>>>> >>>>>>> adds reste * log(dF) to the last-1 (i.e. the smpl) >>>>>>> member of lik >>>>>>> (the last, the lik(smpl+1) one contains >>>>>>> >>>>>>> >>>>>>> >>>>> smpl*pp*log(2*pi)) >>>>> >>>>> >>>>> >>>>>>> but reste is usually larger than T-start+1 so that >>>>>>> >>>>>>> LIK = >>>>>>> >>>>>>> >>>>>>> >>>>> .5*(sum(lik(start:end))-(start-1)*lik(smpl+1)/smpl) >>>>> >>>>> >>>>> >>>>>>> has much more log(dF)s added than required since >>>>>>> > they > >>>>>>> are all concentrated >>>>>>> in the last-1 (the T) member >>>>>>> >>>>>>> For example, if I change the above construct to >>>>>>> lik(t) = lik(t) + min(reste,(smpl-start+1))*log(dF); >>>>>>> >>>>>>> the reported likelihood for presample=40 from Matlab >>>>>>> >>>>>>> >>> KF >>> >>> >>>>> is >>>>> >>>>> >>>>> >>>>>>> 1640935.5855267849 >>>>>>> which is nearly the same as that from C++ KF below: >>>>>>> 1640935.5854489324 >>>>>>> >>>>>>> Shall I make changes to kalman/likelihood/ KFs and >>>>>>> upload the .m files? >>>>>>> This problem affects also the older versions of >>>>>>> DiffuseLikelihood**.m too. >>>>>>> >>>>>>> Best regards >>>>>>> >>>>>>> George >>>>>>> artilogica(a)btconnect.com >>>>>>> >>>>>>> >>>>>>> >>>>> <mailto:artilogica@btconnect.com> >>>>> >>>>> >>>>> >>>>>>> ----- Original Message ----- From: "Michel Juillard" >>>>>>> <michel.juillard(a)ens.fr >>>>>>> >>>>>>> >>> <mailto:michel.juillard@ens.fr>> >>> >>> >>>>>>> To: "G. Perendia" <george(a)perendia.orangehome.co.uk >>>>>>> <mailto:george@perendia.orangehome.co.uk>> >>>>>>> Sent: Tuesday, May 26, 2009 10:32 AM >>>>>>> Subject: Re: Kalman Filter+PS >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Hi George, >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Re 1) below: >>>>>>> I modified C++ KF so that it reports >>>>>>> log-likelihood for given >>>>>>> start/preampling in same/similar manner as >>>>>>> > the > >>>>>>> Matlab KFs do and I am >>>>>>> getting approximately close results, e.g. >>>>>>> >>>>>>> ll= -1640935.5854489324 for C++ and >>>>>>> (-) 1640482.4179242959 for Matlab KF (for >>>>>>> start=41, i.e. >>>>>>> >>>>>>> >>>>>>> presample=40). >>>>>>> >>>>>>> >>>>>>> whilst they appear same for presample=0 >>>>>>> (e.g.2.5906e+006), i.e. >>>>>>> -2590556.989730841 vs >>>>>>> 2590556.989778722 >>>>>>> >>>>>>> Are those results acceptably close or should >>>>>>> > I > >>>>>>> investigate further where >>>>>>> >>>>>>> >>>>>>> the >>>>>>> >>>>>>> >>>>>>> above difference may come form? >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> This indicates a problem . The difference should >>>>>>> be the same with and >>>>>>> without presample. It may come from the >>>>>>> computation of the likelihood >>>>>>> constant. This is done in a very obscure manner >>>>>>> > in > >>>>>>> Dynare Matlab. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> Dev(a)dynare.org <mailto:Dev@dynare.org> >>>>>>> http://www.dynare.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Stéphane Adjemian >>>>>>> CEPREMAP & Université du Maine >>>>>>> >>>>>>> Tel: (33)1-43-13-62-39 >>>>>>> >>>>>>> >>>>>>> >>>>>>> > ------------------------------------------------------------------------ > >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> Dev(a)dynare.org <mailto:Dev@dynare.org> >>>>>>> http://www.dynare.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> Dev(a)dynare.org <mailto:Dev@dynare.org> >>>>>>> http://www.dynare.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Stéphane Adjemian >>>>>>> CEPREMAP & Université du Maine >>>>>>> >>>>>>> Tel: (33)1-43-13-62-39 >>>>>>> >>>>>>> >>>>> ----------------------------------------------------------------------- >>>>> > - > >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> Dev(a)dynare.org >>>>>>> http://www.dynare.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> _______________________________________________ >>>>>> Dev mailing list >>>>>> Dev(a)dynare.org >>>>>> http://www.dynare.org/cgi-bin/mailman/listinfo/dev >>>>>> >>>>>> >>>>>> >>>>>> >>> >>> >>> >> > > > >

1 0

Re: [DynareDev] Kalman Filter
by Michel Juillard 29 May '09

29 May '09

On closer inspection, I don't think that the expression pointed by George in kalman_filter.m is wrong: 1. reste = smpl-t or the number of periods during which the filter is stationary. This shouldn't be larger than T-start+1 2. it is problematic (see below) but not wrong to add all the determinants at once in the last period of the stationary filter 3. I don't think this explains the difference with the C++ version of the filter and we still have to look for it. 4. it remains that the current code is very unclear and that if LIK is correct the vector lik doesn't have the correct constants on each elements. 5. I would like to simplify the code and add the correct constant to each element of the lik vector. It would be a little bit less efficient in Matlab than the current code, but I doubt it would be noticeable. Stephane, what do you think? Best Michel G. Perendia wrote: > Dear Michel > > I think I found an error in Dynare Matlab kalman_filter. suite of utilities > which affects the likelihood LIK results with start>1 (i.e. presampling>0): > > the calculation speed-up construct which relies on converged covariance > matrix > > lik(t) = lik(t) + reste*log(dF); > > adds reste * log(dF) to the last-1 (i.e. the smpl) member of lik > (the last, the lik(smpl+1) one contains smpl*pp*log(2*pi)) > but reste is usually larger than T-start+1 so that > > LIK = .5*(sum(lik(start:end))-(start-1)*lik(smpl+1)/smpl) > > has much more log(dF)s added than required since they are all concentrated > in the last-1 (the T) member > > For example, if I change the above construct to > lik(t) = lik(t) + min(reste,(smpl-start+1))*log(dF); > > the reported likelihood for presample=40 from Matlab KF is > 1640935.5855267849 > which is nearly the same as that from C++ KF below: > 1640935.5854489324 > > Shall I make changes to kalman/likelihood/ KFs and upload the .m files? > This problem affects also the older versions of DiffuseLikelihood**.m too. > > Best regards > > George > artilogica(a)btconnect.com > > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > To: "G. Perendia" <george(a)perendia.orangehome.co.uk> > Sent: Tuesday, May 26, 2009 10:32 AM > Subject: Re: Kalman Filter+PS > > > >> Hi George, >> >> >> >>> Re 1) below: >>> I modified C++ KF so that it reports log-likelihood for given >>> start/preampling in same/similar manner as the Matlab KFs do and I am >>> getting approximately close results, e.g. >>> >>> ll= -1640935.5854489324 for C++ and >>> (-) 1640482.4179242959 for Matlab KF (for start=41, i.e. >>> > presample=40). > >>> whilst they appear same for presample=0 (e.g.2.5906e+006), i.e. >>> -2590556.989730841 vs >>> 2590556.989778722 >>> >>> Are those results acceptably close or should I investigate further where >>> > the > >>> above difference may come form? >>> >>> >>> >> This indicates a problem . The difference should be the same with and >> without presample. It may come from the computation of the likelihood >> constant. This is done in a very obscure manner in Dynare Matlab. >> > > > >

4 18

Re: [DynareDev] Kalman Filter
by G. Perendia 28 May '09

28 May '09

PS: Aim is to emulate calling protocol for current matlab/kalman/likelihood .m files. I am currently adjusting the kalman_filter.cpp dll suite (kalman_filter.cpp to start with) to create internally H and Pinf 0 matrices of suitable size if empty or single 0 passed to them and so that non-diffuse on can take mf instead Z as C++ currently expects. (The matlab\*.cpp files have already been adjusted to return likelihood array as .m files do.) in meantime one may use .m file to prepare those on the lines of the kalmandll_test.m I just enclosed in the matlab subdirectory of sources/kalman repository. Best regards George ----- Original Message ----- From: G. Perendia To: List for Dynare developers Sent: Wednesday, May 27, 2009 2:56 PM Subject: Re: [DynareDev] Kalman Filter The filter routines there have names and should follow calling rules for the par matlab/kalman/likelihood ones but they do not yet take mf and still expect Z matrix of 1s to map system to observables instead and Pinf must not be empty but at least be initialised to a zeros matrix of Pstar size. The smoother is generic as it was, not suitable for running at all Best regards George ----- Original Message ----- From: Stéphane Adjemian To: List for Dynare developers Sent: Wednesday, May 27, 2009 2:46 PM Subject: Re: [DynareDev] Kalman Filter Hi George 2009/5/27 G. Perendia <george(a)perendia.orangehome.co.uk> Hi The C++ Kalman dll driver routines in /mex/kalman/matlab are still in development Sure... and the code there is still subject to developer testing ... But I'd like to play with it. and the Makefile there is not yet complete but it should use blas/lapack from Matlab libraries. Ok. I will adapt your Makefile for my platform. Best, Stéphane. Best regards George Perendia Tel.: 02072815392 Mob: 07951415480 ----- Original Message ----- From: Stéphane Adjemian To: List for Dynare developers Sent: Wednesday, May 27, 2009 2:18 PM Subject: Re: [DynareDev] Kalman Filter Oui, oui... I never use the presampling option... And with all the examples I have ever considered it takes much more than 10 iterations to get to the steady state kalman filter. By the way, is there somewhere linux a Makefile for the cc kalman routines? Also, why do we need Atlas (if we also use matlab's lapack/blas libraries)? Stéphane. 2009/5/27 Michel Juillard <michel.juillard(a)ens.fr> Si le filtre converge a l'etat stationnaire a une date avant start (la date a partir de laquelle on cumule la vraisemblance), alors additionner tous les determinants des iterations dans le filtre stationnaire a la derniere periode fausse le calcul de la constante. amicalement Michel Stéphane Adjemian wrote: Thanks Michel. Your commit is ok for me. I do not yet understand the problem raised by George. I need to go through his example... Stéphane. 2009/5/27 Michel Juillard <michel.juillard(a)ens.fr <mailto:michel.juillard@ens.fr>> Thanks Stephane, I just uploaded a new version of kalman_filter.m Tell me what you think Best Michel Stéphane Adjemian wrote: Hi all, I agree, the matlab code is very unclear (even if I had fun writting it this way ;-) and prone to errors if one uses the vector lik (Marco is using it). I would rather prefer to add the constants outside of the loop with a (sub)vector operation, this should be more efficient. I will do it today or tomorrow. Best, Stéphane. 2009/5/27 Michel Juillard <michel.juillard(a)ens.fr <mailto:michel.juillard@ens.fr> <mailto:michel.juillard@ens.fr <mailto:michel.juillard@ens.fr>>> On closer inspection, I don't think that the expression pointed by George in kalman_filter.m is wrong: 1. reste = smpl-t or the number of periods during which the filter is stationary. This shouldn't be larger than T-start+1 2. it is problematic (see below) but not wrong to add all the determinants at once in the last period of the stationary filter 3. I don't think this explains the difference with the C++ version of the filter and we still have to look for it. 4. it remains that the current code is very unclear and that if LIK is correct the vector lik doesn't have the correct constants on each elements. 5. I would like to simplify the code and add the correct constant to each element of the lik vector. It would be a little bit less efficient in Matlab than the current code, but I doubt it would be noticeable. Stephane, what do you think? Best Michel G. Perendia wrote: Dear Michel I think I found an error in Dynare Matlab kalman_filter. suite of utilities which affects the likelihood LIK results with start>1 (i.e. presampling>0): the calculation speed-up construct which relies on converged covariance matrix lik(t) = lik(t) + reste*log(dF); adds reste * log(dF) to the last-1 (i.e. the smpl) member of lik (the last, the lik(smpl+1) one contains smpl*pp*log(2*pi)) but reste is usually larger than T-start+1 so that LIK = .5*(sum(lik(start:end))-(start-1)*lik(smpl+1)/smpl) has much more log(dF)s added than required since they are all concentrated in the last-1 (the T) member For example, if I change the above construct to lik(t) = lik(t) + min(reste,(smpl-start+1))*log(dF); the reported likelihood for presample=40 from Matlab KF is 1640935.5855267849 which is nearly the same as that from C++ KF below: 1640935.5854489324 Shall I make changes to kalman/likelihood/ KFs and upload the .m files? This problem affects also the older versions of DiffuseLikelihood**.m too. Best regards George artilogica(a)btconnect.com <mailto:artilogica@btconnect.com> <mailto:artilogica@btconnect.com <mailto:artilogica@btconnect.com>> ----- Original Message ----- From: "Michel Juillard" <michel.juillard(a)ens.fr <mailto:michel.juillard@ens.fr> <mailto:michel.juillard@ens.fr <mailto:michel.juillard@ens.fr>>> To: "G. Perendia" <george(a)perendia.orangehome.co.uk <mailto:george@perendia.orangehome.co.uk> <mailto:george@perendia.orangehome.co.uk <mailto:george@perendia.orangehome.co.uk>>> Sent: Tuesday, May 26, 2009 10:32 AM Subject: Re: Kalman Filter+PS Hi George, Re 1) below: I modified C++ KF so that it reports log-likelihood for given start/preampling in same/similar manner as the Matlab KFs do and I am getting approximately close results, e.g. ll= -1640935.5854489324 for C++ and (-) 1640482.4179242959 for Matlab KF (for start=41, i.e. presample=40). whilst they appear same for presample=0 (e.g.2.5906e+006), i.e. -2590556.989730841 vs 2590556.989778722 Are those results acceptably close or should I investigate further where the above difference may come form? This indicates a problem . The difference should be the same with and without presample. It may come from the computation of the likelihood constant. This is done in a very obscure manner in Dynare Matlab. _______________________________________________ Dev mailing list Dev(a)dynare.org <mailto:Dev@dynare.org> <mailto:Dev@dynare.org <mailto:Dev@dynare.org>> http://www.dynare.org/cgi-bin/mailman/listinfo/dev -- Stéphane Adjemian CEPREMAP & Université du Maine Tel: (33)1-43-13-62-39 ------------------------------------------------------------------------ _______________________________________________ Dev mailing list Dev(a)dynare.org <mailto:Dev@dynare.org> http://www.dynare.org/cgi-bin/mailman/listinfo/dev _______________________________________________ Dev mailing list Dev(a)dynare.org <mailto:Dev@dynare.org> http://www.dynare.org/cgi-bin/mailman/listinfo/dev -- Stéphane Adjemian CEPREMAP & Université du Maine Tel: (33)1-43-13-62-39 ------------------------------------------------------------------------ _______________________________________________ Dev mailing list Dev(a)dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev _______________________________________________ Dev mailing list Dev(a)dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev -- Stéphane Adjemian CEPREMAP & Université du Maine Tel: (33)1-43-13-62-39 ------------------------------------------------------------------------ _______________________________________________ Dev mailing list Dev(a)dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev _______________________________________________ Dev mailing list Dev(a)dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev -- Stéphane Adjemian CEPREMAP & Université du Maine Tel: (33)1-43-13-62-39 ---------------------------------------------------------------------------- _______________________________________________ Dev mailing list Dev(a)dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev

2 4

Code beautification/indentation
by Sébastien Villemot 27 May '09

27 May '09

Hi everyone, I have found a great tool for indenting/beautifying C++ code: uncrustify I added a config file for it in the SVN, and some doc on the wiki (page CodingStandards). As an example, I have beautified korderpert code using that tool, and it works great. Best, -- Sébastien Villemot

2 1

Kordpert
by G. Perendia 27 May '09

27 May '09

Michel We should not use Dyn_pp subdirectory in korderpert directory of mex/sources for building dynarelib.a needed for korderpert but up-to-date dynare++. It remained there from the initial plan to separate two developments and may not be up-to-date.. If that is OK, I will delete Dyn_pp from SVN Best regards George artilogica(a)btconnect.com

2 1

Re: [DynareDev] Kalman Filter
by Michel Juillard 27 May '09

27 May '09

Thank you very much George. I will take a look at it. I think that the correction should be more direct than the min() construct Best Michel G. Perendia wrote: > Dear Michel > > I think I found an error in Dynare Matlab kalman_filter. suite of utilities > which affects the likelihood LIK results with start>1 (i.e. presampling>0): > > the calculation speed-up construct which relies on converged covariance > matrix > > lik(t) = lik(t) + reste*log(dF); > > adds reste * log(dF) to the last-1 (i.e. the smpl) member of lik > (the last, the lik(smpl+1) one contains smpl*pp*log(2*pi)) > but reste is usually larger than T-start+1 so that > > LIK = .5*(sum(lik(start:end))-(start-1)*lik(smpl+1)/smpl) > > has much more log(dF)s added than required since they are all concentrated > in the last-1 (the T) member > > For example, if I change the above construct to > lik(t) = lik(t) + min(reste,(smpl-start+1))*log(dF); > > the reported likelihood for presample=40 from Matlab KF is > 1640935.5855267849 > which is nearly the same as that from C++ KF below: > 1640935.5854489324 > > Shall I make changes to kalman/likelihood/ KFs and upload the .m files? > This problem affects also the older versions of DiffuseLikelihood**.m too. > > Best regards > > George > artilogica(a)btconnect.com > > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > To: "G. Perendia" <george(a)perendia.orangehome.co.uk> > Sent: Tuesday, May 26, 2009 10:32 AM > Subject: Re: Kalman Filter+PS > > > >> Hi George, >> >> >> >>> Re 1) below: >>> I modified C++ KF so that it reports log-likelihood for given >>> start/preampling in same/similar manner as the Matlab KFs do and I am >>> getting approximately close results, e.g. >>> >>> ll= -1640935.5854489324 for C++ and >>> (-) 1640482.4179242959 for Matlab KF (for start=41, i.e. >>> > presample=40). > >>> whilst they appear same for presample=0 (e.g.2.5906e+006), i.e. >>> -2590556.989730841 vs >>> 2590556.989778722 >>> >>> Are those results acceptably close or should I investigate further where >>> > the > >>> above difference may come form? >>> >>> >>> >> This indicates a problem . The difference should be the same with and >> without presample. It may come from the computation of the likelihood >> constant. This is done in a very obscure manner in Dynare Matlab. >> > > > >

1 0

Re: [DynareDev] quasi-triangular matrices
by Michel Juillard 26 May '09

26 May '09

Great! Then take contact directly with George about calling it from his Kalman filter code and general integration and testing. Please cc all messages to dev(a)dynare.org Many thanks and all the best, Michel andrea pagano wrote: > Hi Michel > > > I was about to send you the codes (which I wrote in fortran). > Actually I used a slight different strategy as one of DGEMM (since I > believe we can specify even more our situation). > > I am finishing to write a couple of extra files and the documentation, > and I will send you the entire package. > > Best > > Andrea > > > > > > > > > > On Tue, May 26, 2009 at 11:35 AM, Michel Juillard > <michel.juillard(a)ens.fr> wrote: > >> Dear Andrea, >> >> were you able to decipher the strategy used by ATLAS for DGEMM? Is there >> hope (and advantages) to specialize it for quasi-triangular matrices? >> >> Best >> >> Michel >> >> > > > >

3 3

Dynare in Ubuntu Karmic
by Sébastien Villemot 21 May '09

21 May '09

It looks like Dynare has now entered Ubuntu, and is in line for the next release in October 2009 (Karmic Koala). https://launchpad.net/ubuntu/+source/dynare This is a consequence of the automatic updates of Ubuntu from Debian. Recall that this first version of the package is an Octave-only version. I still need to add Matlab support (in the form of some build helper). Best, -- Sébastien Villemot

1 1

Dynare in Debian Unstable
by Sébastien Villemot 17 May '09

17 May '09

Hi all, Dynare has entered the official Debian Unstable distribution. At any time you can check the package status on the following page: http://packages.qa.debian.org/d/dynare.html Note that this page has an RSS feed which tracks changes. Best, -- Sébastien Villemot

1 0

parallel dynare
by Marco Ratto 15 May '09

15 May '09

Dear Michel, in order to be able to run the parallel computations under the windows environment, the attached executables must be unzipped and the folder where they are saved must be present in the windows path. I guess we have to find a way to `automatize' this procedure while installing dynare on windows machine? All the best Marco

1 0

stochastic steady state/Kalman filter
by Michel Juillard 11 May '09

11 May '09

Dear Andrea, I met with Pablo last week and we made some progress. He is going to go ahead with his algorithm taking approximation around the unconditional mean. He will present it in an intermediary report that is due at the end of this week. Please check it for typos and other mistakes when it is ready and, if possible, send him remarks by Thursday. I'm still thinking about an alternative approach around the steady sate, but I need to finish writing it before showing it to you. I would like you to start another project around the multiplication with quasi-triangular matrices. We are in the process of coding the Kalman filter in C++ hoping to accelerate this part of estimation. George is in charge of the overall project. However, when we estimate with non-stationary variables, we are forced to transform the transition equation of the state space representation and we have a transition matrix that is quasi-triangular. As it turns out, we could always used this transformation, even for models with only stationary variables. I expect some speed gain from taking into account the special structure of the transition matrix. I'm attaching a brief outline of the Kalman filte So I would like you to write a code (+, C or Fortran) for the two operations involving the transition matrix T. 1) T and P have the same dimensions. When they are large 100x100 and larger, I imagine there is a gain from using a block algorithm for T*P*T' as in BLAS/LAPACK dgemm and to optimize the size of a block so as to keep it in fast cache memory the same way as ATLAS is doing. 2) For smaller matrices, it may be better to simply inline the loops. The optimal size for both approaches should be determined empirically by testing. 3) Currently, in the Matlab code, for lage models, most of the time is spent in computing T*P*T' For this project, I would like you to consult with Marco. He has agreed to help and JRC has much experience with Kalman filter computations.

1 0

Re: [DynareDev] Extracting rules for deterministic exogenous vars
by Michel Juillard 11 May '09

11 May '09

One more thing. We realized that when we are estimating models with non-stationary variables (diffuse_filter option) it is necessary to transform the transition equation so that the transition matrix is quasi-upper diagonal. This transformation of the transition matrix is indeed available in all cases (even for models with stationary variables only) because it is a by-product of the algorithm for solving the Lyapunov equation. This equation is used for computing the unconditional variance of the endogenous variables used to initialize the matrix of forecast error variance in the Kalman filter (P matrix usually). Using a quasi-triangular transition matrix should permit speed gains in matrix multiplication where this matrix is involved. I'm planning to ask Andrea to look into writing a block algorithm for matrix multiplication specialized for the case where one of the two matrices is quasi-triangular. For the time being, you can use DGEMM for these matrix multiplications. Best, Michel G. Perendia wrote: > Deara Michel > > Thanks. > > As you already pointed out, the KF software from SVN is highly > over-modularised and difficult to grasp and modify. But we may not have to > modify much as it has some of the functionalities we expect already built-in > such as univariate and multivariate filtering, smoother and diffuse priors. > Also, as mentioned earlier, it should integrate well with other Dynare > applications around dynarelib (it uses sylvester code too). So far the > only two package that uses LAPACK/BLAS (either the optimised one , or, if > customised, the bespoke ports to various parallel hardware platforms) are > dynarelib/KF and a smaller of the two Bayesian filter libraries ( C++ > Bayesian Filtering Classes by Michael Stevens, see enclosed draft analysis). > > The most of other internet KF packages either rely on BOOST/uBLAS or their > own > matrix calculation implementations. And, contrary to my initial assumptions, > uBLAS, as the most of the BOOST libraries it is part of, is a self-contained > "header" library that implements most of important basic algebra > functionality in header (.hpp) files. It therefore does not use (or > interface) any external libraries such as LAPACK/BLAS and, all compiling and > optimisation is done at application compilation time. This is one of reasons > our SVN KF still scores high among the alternatives. > > Smoothing, however, is also present in one of the Bayesian filtering > packages (Orocos ), and though that one relies on BOOST/uBLAS header > libraries, the Orocos BFL is probably best (i.e. functionally richest) > alternative. > > Altogether it is still difficult to derive single recommendation and I am > still working on analysing the available packages - The latest draft > analysis is enclosed . > > Best regards > > George > > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > To: "G. Perendia" <george(a)perendia.orangehome.co.uk> > Sent: Sunday, May 10, 2009 1:28 PM > Subject: Re: Extracting rules for deterministic exogenous vars > > > >> There should be no problem if we want ot use it. This was developed for >> the Dynare project. What do you think of the code? >> >> best >> >> Michel >> >> G. Perendia wrote: >> >>> Dear Michel >>> I wandered also if we are allowed to use it. >>> Best regards >>> >>> George >>> >>> ----- Original Message ----- >>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>> To: "G. Perendia" <george(a)perendia.orangehome.co.uk> >>> Sent: Tuesday, May 05, 2009 1:49 PM >>> Subject: Re: Extracting rules for deterministic exogenous vars >>> >>> >>> >>> >>>> Hi George, >>>> >>>> my memory that it was too modular in conception and hard to understand, >>>> but I will fish it back and send it to you. >>>> >>>> All the best, >>>> >>>> Michel >>>> >>>> G. Perendia wrote: >>>> >>>> >>>>> Dear Michel >>>>> >>>>> I started looking into Kalman Filter DLL - Can we still re-use the >>>>> (existing?) Kalman Filter code that was/is (??) in SVN for the new >>>>> > KF? > >>>>> PS: I.e., I can not see, find or access the Kalman_Filter SVN repo any >>>>> more - i.e. since the move- - but browsing inside the current repo did >>>>> >>>>> >>> not >>> >>> >>>>> find Kalman-Filter." >>>>> >>>>> Best regards >>>>> >>>>> George >>>>> Mob. +44(0)7951415480 >>>>> artilogica(a)btconnect.com >>>>> >>>>> ----- Original Message ----- >>>>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>>>> To: "G. Perendia" <george(a)perendia.orangehome.co.uk>; "List for Dynare >>>>> developers" <dev(a)dynare.org> >>>>> Sent: Monday, April 27, 2009 8:33 PM >>>>> Subject: Re: Extracting rules for deterministic exogenous vars >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> Hi George, >>>>>> >>>>>> I have completely forgotten about dr.fuu, I don't see at the moment >>>>>> >>>>>> >>> what >>> >>> >>>>>> it is useful for. >>>>>> Porting the varexo_det stuff to C++ isn't a priority, far from it. >>>>>> I would rather that you clean-up existing k_order_perturbation code. >>>>>> When that is done, I would like you to turn to the Kalman filter, but >>>>>> >>>>>> >>> we >>> >>> >>>>>> will need to do a careful analysis before starting to code. >>>>>> One question will be which matrix implementation to use in the Kalman >>>>>> filter code? Shall we keep using Ondra's TwoD matrices? Shall we >>>>>> > build > >>>>>> our own (doubtful)? Should we use something like Boost? >>>>>> >>>>>> Best >>>>>> >>>>>> Michel >>>>>> >>>>>> >>>>>> G. Perendia wrote: >>>>>> >>>>>> >>>>>> >>>>>>> Dear Michel >>>>>>> >>>>>>> I was thinking of re-using some of the existing 2nd order dr1.m code >>>>>>> >>>>>>> >>> to >>> >>> >>>>>>> determine dr.fuu and the rules for, if any, deterministic exogenous >>>>>>> >>>>>>> >>> vars >>> >>> >>>>>>> (dr.gh??d). >>>>>>> >>>>>>> Looking at the code in dr1.m, for both dr.fuu and deterministic >>>>>>> >>>>>>> >>>>>>> >>>>> exogenous >>>>> >>>>> >>>>> >>>>>>> vars, we need hessian and, for the former, jacobian. Shall I >>>>>>> > retrieve > >>>>>>> >>>>> them >>>>> >>>>> >>>>> >>>>>>> from k-order-perturbation or is there a more elegant way to get >>>>>>> > those > >>>>>>> >>>>> rules >>>>> >>>>> >>>>> >>>>>>> that you can think of? Perhaps we can determine fuu from ghs2 (if >>>>>>> > not > >>> 0) >>> >>> >>>>>>> more elegantly but I can not see it as yet. >>>>>>> >>>>>>> I however could not find where dr.fuu is used other than internally >>>>>>> >>>>>>> >>>>>>> >>>>> within >>>>> >>>>> >>>>> >>>>>>> dr1.m (and dr11_sparse.m). That could mean that we only need hessian >>>>>>> >>>>>>> >>> and >>> >>> >>>>>>> only infrequently when if we have deterministic exogenous vars. >>>>>>> >>>>>>> Best regards >>>>>>> >>>>>>> George >>>>>>> Mob. +44(0)7951415480 >>>>>>> artilogica(a)btconnect.com >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> >>>> >>> >>> >>> >>

1 0

Re: [DynareDev] Extracting rules for deterministic exogenous vars
by Michel Juillard 11 May '09

11 May '09

Thanks George, this is very useful. I suggest the following. 1) Use matrix implementation in dynare++/tl/cc, and develop the interface to BLAS/LAPACK if additional such functions are needed 2) Start with the code developed by Ondra Kamenik, but try to redesign the code in a less complicated implementation and get rid of the cweb part. You will just mention the ancestry of this new code. Please, send me the classes and functions that you think would be optimal to implement before actually writing code. We have two preoccupations: * the code must be numerically robust for the type of models that we are handling (we will need tests ex-post) * speed because of the Metropolis algorithm that will call this function millions of times per run. 3) Look at OROCOS implementation when in doubt about algorithm details 4) For the moment, we are only concerned with filtering and log-likelihood computation, we will worry about the smoother (and which smoothers we need) later. All the best, Michel G. Perendia wrote: > Deara Michel > > Thanks. > > As you already pointed out, the KF software from SVN is highly > over-modularised and difficult to grasp and modify. But we may not have to > modify much as it has some of the functionalities we expect already built-in > such as univariate and multivariate filtering, smoother and diffuse priors. > Also, as mentioned earlier, it should integrate well with other Dynare > applications around dynarelib (it uses sylvester code too). So far the > only two package that uses LAPACK/BLAS (either the optimised one , or, if > customised, the bespoke ports to various parallel hardware platforms) are > dynarelib/KF and a smaller of the two Bayesian filter libraries ( C++ > Bayesian Filtering Classes by Michael Stevens, see enclosed draft analysis). > > The most of other internet KF packages either rely on BOOST/uBLAS or their > own > matrix calculation implementations. And, contrary to my initial assumptions, > uBLAS, as the most of the BOOST libraries it is part of, is a self-contained > "header" library that implements most of important basic algebra > functionality in header (.hpp) files. It therefore does not use (or > interface) any external libraries such as LAPACK/BLAS and, all compiling and > optimisation is done at application compilation time. This is one of reasons > our SVN KF still scores high among the alternatives. > > Smoothing, however, is also present in one of the Bayesian filtering > packages (Orocos ), and though that one relies on BOOST/uBLAS header > libraries, the Orocos BFL is probably best (i.e. functionally richest) > alternative. > > Altogether it is still difficult to derive single recommendation and I am > still working on analysing the available packages - The latest draft > analysis is enclosed . > > Best regards > > George > > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > To: "G. Perendia" <george(a)perendia.orangehome.co.uk> > Sent: Sunday, May 10, 2009 1:28 PM > Subject: Re: Extracting rules for deterministic exogenous vars > > > >> There should be no problem if we want ot use it. This was developed for >> the Dynare project. What do you think of the code? >> >> best >> >> Michel >> >> G. Perendia wrote: >> >>> Dear Michel >>> I wandered also if we are allowed to use it. >>> Best regards >>> >>> George >>> >>> ----- Original Message ----- >>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>> To: "G. Perendia" <george(a)perendia.orangehome.co.uk> >>> Sent: Tuesday, May 05, 2009 1:49 PM >>> Subject: Re: Extracting rules for deterministic exogenous vars >>> >>> >>> >>> >>>> Hi George, >>>> >>>> my memory that it was too modular in conception and hard to understand, >>>> but I will fish it back and send it to you. >>>> >>>> All the best, >>>> >>>> Michel >>>> >>>> G. Perendia wrote: >>>> >>>> >>>>> Dear Michel >>>>> >>>>> I started looking into Kalman Filter DLL - Can we still re-use the >>>>> (existing?) Kalman Filter code that was/is (??) in SVN for the new >>>>> > KF? > >>>>> PS: I.e., I can not see, find or access the Kalman_Filter SVN repo any >>>>> more - i.e. since the move- - but browsing inside the current repo did >>>>> >>>>> >>> not >>> >>> >>>>> find Kalman-Filter." >>>>> >>>>> Best regards >>>>> >>>>> George >>>>> Mob. +44(0)7951415480 >>>>> artilogica(a)btconnect.com >>>>> >>>>> ----- Original Message ----- >>>>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>>>> To: "G. Perendia" <george(a)perendia.orangehome.co.uk>; "List for Dynare >>>>> developers" <dev(a)dynare.org> >>>>> Sent: Monday, April 27, 2009 8:33 PM >>>>> Subject: Re: Extracting rules for deterministic exogenous vars >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> Hi George, >>>>>> >>>>>> I have completely forgotten about dr.fuu, I don't see at the moment >>>>>> >>>>>> >>> what >>> >>> >>>>>> it is useful for. >>>>>> Porting the varexo_det stuff to C++ isn't a priority, far from it. >>>>>> I would rather that you clean-up existing k_order_perturbation code. >>>>>> When that is done, I would like you to turn to the Kalman filter, but >>>>>> >>>>>> >>> we >>> >>> >>>>>> will need to do a careful analysis before starting to code. >>>>>> One question will be which matrix implementation to use in the Kalman >>>>>> filter code? Shall we keep using Ondra's TwoD matrices? Shall we >>>>>> > build > >>>>>> our own (doubtful)? Should we use something like Boost? >>>>>> >>>>>> Best >>>>>> >>>>>> Michel >>>>>> >>>>>> >>>>>> G. Perendia wrote: >>>>>> >>>>>> >>>>>> >>>>>>> Dear Michel >>>>>>> >>>>>>> I was thinking of re-using some of the existing 2nd order dr1.m code >>>>>>> >>>>>>> >>> to >>> >>> >>>>>>> determine dr.fuu and the rules for, if any, deterministic exogenous >>>>>>> >>>>>>> >>> vars >>> >>> >>>>>>> (dr.gh??d). >>>>>>> >>>>>>> Looking at the code in dr1.m, for both dr.fuu and deterministic >>>>>>> >>>>>>> >>>>>>> >>>>> exogenous >>>>> >>>>> >>>>> >>>>>>> vars, we need hessian and, for the former, jacobian. Shall I >>>>>>> > retrieve > >>>>>>> >>>>> them >>>>> >>>>> >>>>> >>>>>>> from k-order-perturbation or is there a more elegant way to get >>>>>>> > those > >>>>>>> >>>>> rules >>>>> >>>>> >>>>> >>>>>>> that you can think of? Perhaps we can determine fuu from ghs2 (if >>>>>>> > not > >>> 0) >>> >>> >>>>>>> more elegantly but I can not see it as yet. >>>>>>> >>>>>>> I however could not find where dr.fuu is used other than internally >>>>>>> >>>>>>> >>>>>>> >>>>> within >>>>> >>>>> >>>>> >>>>>>> dr1.m (and dr11_sparse.m). That could mean that we only need hessian >>>>>>> >>>>>>> >>> and >>> >>> >>>>>>> only infrequently when if we have deterministic exogenous vars. >>>>>>> >>>>>>> Best regards >>>>>>> >>>>>>> George >>>>>>> Mob. +44(0)7951415480 >>>>>>> artilogica(a)btconnect.com >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> >>>> >>> >>> >>> >>

1 0

Re: [DynareDev] cleaning up korderpert
by Michel Juillard 06 May '09

06 May '09

Thanks George. I guess that we will have to live with *.cpp and .hpp in some directories. Best Michel Quoting "G. Perendia" <george(a)perendia.orangehome.co.uk>: > Dear Michel > > I filtered out DOS ^Ms and replaced tabs with 2 spaces. My editors (MSVC and > Matlab) do not provide support for .cc and .hh file extensions but only the > more standard .cpp and .hpp. Shall then I also rename those files in the > SVN to fully match Dynare coding standards (though there is gensylv code > that does not follow that rule)?. > > Best regards > > George > > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > To: "G. Perendia" <george(a)perendia.orangehome.co.uk> > Sent: Tuesday, May 05, 2009 2:26 PM > Subject: cleaning up korderpert > > > > > > Hi George, > > > > when you clean-up the code of korderpert, could you please follow the > > coding standards in > > http://www.dynare.org/DynareWiki/CodingStandards, including an > > indentation of 2 spaces, > > and remove all the lines of code that were commented out because we > > preferred another alternative. > > Many thanks in advance > > All the best, > > > > Michel > > > > > > > > --

1 0

Boost libraries
by Sébastien Villemot 29 Apr '09

29 Apr '09

Hi all, Concerning the Boost libraries that are now needed to recompile the preprocessor: - the 1.33 version distributed with Cygwin is too old: it doesn't contain the boost/graph/max_cardinality_matching.hpp header which is needed for model normalization. Stéphane: could you tell me which version of Boost is present in Ubuntu Intrepid (in Debian we have several possible choices: 1.34, 1.37 and 1.38). Maybe the version in Ubuntu Intrepid contains that header, in which case it should be fine - installing the boost library normally consists of simply unpacking the archive. Since Boost is mainly made of C++ headers, there is normally no need to recompile anything Best, -- Sébastien Villemot

1 1

Re: [DynareDev] reordering algorithm
by Michel Juillard 28 Apr '09

28 Apr '09

Dear George, we need to rethink the organization of dr1.m and friends: we should factorize common code, move away initialization that should take place only once and probably split 1st and second order approximation (except for k_order_perturbation.DLL). We need also to plan for resolution by block along the line developed by Ferhat. We don't need to implement all functionalities in all versions of the solver. In my opinion, k_order_perturbation should be used for regular stoch_simul and for estimation of forward looking models only. For backward models, only checking eigenvalues and mixing with deterministic shocks (varexo_det) we can rely on existing code. As long as k_order_perturbation.DLL is in testing stage, we can leave the alternative dr1.m function in korderpert subdirectoy and call it for testing with appropriate addpath command I would suggest to modify resol only when the reorganization of dr1.m has taken place. For partial information, we need to pass additional information from the preprocessor and the exact requirements of that still need to be discussed with Joe and you. All the best, Michel Quoting "G. Perendia" <george(a)perendia.orangehome.co.uk>: > Dear Michel > > 1) I am not sure we should have two version of dr1.m. My current design for > k_order (and partial info I started working on for Joe) assumes existence of > two (or more) dr1_xx.m, all called from resol.m according to the option > settings, on the line of how dr1_sparse.m is called whilst dr1.m will still > be called if check_flag == 1, i.e.: > > if(options_.model_mode==1) > [dr,info,M_,options_,oo_] = dr1_sparse(dr,check_flag,M_,options_,oo_); > elseif(options_.usePartInfo==1)&& (check_flag == 0) > [dr,info,M_,options_,oo_] = dr1_PI(dr,check_flag,M_,options_,oo_); > elseif(options_.use_k_order==1)&& (check_flag == 0) > [dr,info,M_,options_,oo_] = dr1_k_order(dr,check_flag,M_,options_,oo_); > else > [dr,info,M_,options_,oo_] = dr1(dr,check_flag,M_,options_,oo_); > end > > dr1_k_order.m is not complete dr1 and by switching the option setting in the > code (i.e. in case of any problem with the primary solver) we can make > system switch from one solver to another in a smoother, more transparent way > (like I did for AIM). > > If you agree, I will upload resol.m which has switch for dr1_k_order on the > lines of above. > > PS: The new dr1_xx.m are prototypes and I should include check inside those > alternative dr1_xxx.m later too. > > 2) Re: Extracting rules for deterministic exogenous vars > I was not suggesting of porting of deterministic exogenous vars rule to C++ > but was suggesting using the existing dr1.m Matlab code to build that rule > inside dr1_kordp.m as it is done at the moment in dr1.m. > > However, I need hessian for those rules to be calculated and I will > initially call <model>_dynamic.m/.dll to calculate it again which is > straight forward to implement and does not require much more coding. Though > slower, it is used only rarely when we have to deal with deterministic > varexos, > > At a later stage, if performance become an issue, I may retrieve the already > pre-calculated hessian from the k_ordr_perturbation.dll which would then run > faster but requires more work on C++ coding (in part due to reordering > inside DLL). > > Best regards > > George > Mob. +44(0)7951415480 > artilogica(a)btconnect.com > > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > To: "G. Perendia" <george(a)perendia.orangehome.co.uk> > Sent: Monday, April 27, 2009 8:19 PM > Subject: Re: reordering algorithm > > > > > > Thanks George, > > > > I fixed a few bugs and renamed the function kordpert/dr1.m so we can > > just access it by adding kordpert to Matlab path > > > > I'm still not completely sure of what I have done. I still have problem > > with the DLL. For Judd example, it doesn't seem to return the right g_2 > > on my machine. > > > > Best > > > > Michel > > > > > > G. Perendia wrote: > > > Dear Michel > > > > > > Sorry, I should have given you more details rather than assumed you will > > > understand. It is giving wrong results - I think it is to do with the > > > symmetry handling in your algorithm. > > > > > > While my results for Judd model (judd_gpdr.mat) match Dynare Matlab > results, > > > yours (judd_mjdr.mat) do not. > > > > > > It is similar for the fs2000k... and portfolio_2 models - all enclosed. > > > > > > ( the enclosed <model>.mat files give you g_2) > > > > > > Best regards > > > > > > George > > > Mob. +44(0)7951415480 > > > artilogica(a)btconnect.com > > > > > > > > > > > > > > --

1 0

Re: [DynareDev] Extracting rules for deterministic exogenous vars
by Michel Juillard 27 Apr '09

27 Apr '09

Hi George, I have completely forgotten about dr.fuu, I don't see at the moment what it is useful for. Porting the varexo_det stuff to C++ isn't a priority, far from it. I would rather that you clean-up existing k_order_perturbation code. When that is done, I would like you to turn to the Kalman filter, but we will need to do a careful analysis before starting to code. One question will be which matrix implementation to use in the Kalman filter code? Shall we keep using Ondra's TwoD matrices? Shall we build our own (doubtful)? Should we use something like Boost? Best Michel G. Perendia wrote: > Dear Michel > > I was thinking of re-using some of the existing 2nd order dr1.m code to > determine dr.fuu and the rules for, if any, deterministic exogenous vars > (dr.gh??d). > > Looking at the code in dr1.m, for both dr.fuu and deterministic exogenous > vars, we need hessian and, for the former, jacobian. Shall I retrieve them > from k-order-perturbation or is there a more elegant way to get those rules > that you can think of? Perhaps we can determine fuu from ghs2 (if not 0) > more elegantly but I can not see it as yet. > > I however could not find where dr.fuu is used other than internally within > dr1.m (and dr11_sparse.m). That could mean that we only need hessian and > only infrequently when if we have deterministic exogenous vars. > > Best regards > > George > Mob. +44(0)7951415480 > artilogica(a)btconnect.com > > >

1 0

mex options for *_dynamic.c
by Michel Juillard 25 Apr '09

25 Apr '09

Following a suggestion from George, I changed the mex option LDFLAGS for *_dynamic.c to make the Dynamic function public. This was necessary as a temporary fix for kordpert. This most likely restricts the use fof USE_DLL to gcc compilers, but as nobody seems to use the option .... The ultimate solution will be to use Ferhat's bytecode solution and to dispense the users to have to use a compiler Best Michel

1 0

Dynare in Debian
by Sébastien Villemot 25 Apr '09

25 Apr '09

Hi all, The Dynare package for Debian has been uploaded to Debian servers: http://ftp-master.debian.org/new/dynare_4.0.3-1.html For the moment it is in the waiting queue for new packages. If everything goes well, it should enter unstable within a week. Best, -- Sébastien Villemot

2 1

Graphs and preprocessor
by Sébastien Villemot 23 Apr '09

23 Apr '09

Hi all, Since graph theory is becoming increasingly important in the preprocessor, Ferhat and myself have agreed on reusing a standard library for graph manipulation. A good choice seems to be the Boost Graph Library: http://www.boost.org/doc/libs/1_38_0/libs/graph/doc/index.html It is written as C++ generic headers, and is distributed under a BSD-style license (i.e. very permissive), compatible with the GPL. It contains various reusable structures for storing graphs (as adjacency list, adjacency matrix) and various algorithms (in particular strongly connected components). The whole set of Boost libraries is distributed as Debian and Cygwin packages. This leads me to another point: if we use the BGL (boost graph library), the build system of the preprocessor needs to know where the BGL headers are. Under Debian it is /usr/include, under Cygwin /usr/include/boost-1_33_1/, and it may be somewhere else for other systems (MacOS for example). So I think that we should now have a "configure" script at the top level of the Dynare directory, which would autodetect the location of the BGL, and other stuff (such as those that are currently tested for in the preprocessor Makefile). This "configure" script could also be extended to facilitate the building of MEX files from the command line (as it seems to be necessary for k-order perturbation DLL). It could even enable us to replace the build_matlab.m script by a Makefile. With a single "make" at the top-level, we could build the preprocessor, the documentation and the MEX files. The only drawback of adding this is that our build system will be tightly linked to Cygwin for Windows users, but I think that many people that recompile things on Windows already use Cygwin. Any thoughts or suggestions on the Boost Graph Library and the "configure" script ? Best, -- Sébastien Villemot

2 3

Re: [DynareDev] some news on monfispol
by Michel Juillard 20 Apr '09

20 Apr '09

Thanks Marco, the Monfispol paper look perfect. trunk/matlab/thread contains code written by Stephane for multithreaded Kronecker product DLL I imagine that your code is relying on starting multiple instances of Matlab a very different approach and use from above. So I wouldn't put your code in the same directory. For now, put it in trunk/matlab and we will move it later when we have a better view of common functions that would be logical to regroup. Best Michel Marco Ratto wrote: > Dear Michel, > > we have drafted two documents for the state-of-the-art report. We > kept them rather short, with the idea to just place MONFISPOL in the > literature, avoiding a full dissertation on the topics which would > make the report extremely long. Please have a look if they look OK to > you. > > We also have coded a generic MATLAB engine for parallelizing MATLAB > loops. I would like to begin committing it in official dynare. I > would like to know if you agree in committing those routines and > where should I commit them: in trunk\matlab? in trunk\matlab\threads. > > Concerning the new trunk\matlab\threads folder, I have a question: in > our generic engine we have to code checks (error traps) for the number > of processors actually present in the computer and so on. Do you have > already coded them or we proceed autonomously on that? > > Please let us know, > All the best > Marco >

3 10

Hessian
by G. Perendia 10 Apr '09

10 Apr '09

HI Sebastien Can you please confirm how is the 2D matrix Hessian that is coming from <model>_dynamic.c/dll created and organised (and, similarly the sparse hessian from the matlab .m version), e.g. Is it organised as a series of 2nd order jcols*jcols matrix blocks, concatenated together next to each other in order of the variables and shocks by which the 1st order was differentiated, and, all those in one of the standard Dynare orders of the variables (and in which standard order of the variables, order_var or as they are defined or....) and each block containing all of its own set of : gxx, gxu/gux and guu's, or is the hessian organised in some other order, e.g. as all gxx's stacked first (in order of variables, and which) to be followed by all of gxu's and then by all of the guu's Best regards George artilogica(a)btconnect.com

4 3

Re: [DynareDev] dynare++ with MATLAB 2009a on amd64
by Michel Juillard 10 Apr '09

10 Apr '09

Dear Ryan, Mathworks has changed types in the calling sequences of the BLAS and LAPACK library that they distribute with MATLAB v7.8 for 64 bit envirionments and that we use to link against. We will need to modify our code to reflect this change, but it will take a few weeks until we turn to it. All the best, Michel Ryan Lovett wrote: > Hello, > We are attempting to help one of our users utilize dynare_simul_ with > matlab on Linux amd64. His code worked fine with MATLAB 2007a, but not with > 2009a. > I have attached his .mod and the .jnl output. The test.mat will not run > properly as it appears dynare++ exits before finishing. > > Thanks for your time, > Ryan >

1 0

Re: [DynareDev] Bounds in Dynare (version 4)
by Michel Juillard 09 Apr '09

09 Apr '09

Hi Gianni, I changed prior_bounds.m so that if, in estimation, you set option prior_trunc=0, the boundaries are set to what is natural given the prior shape. I prefer that you set prior_trunc=0, than changing the default in Dynare. All the best, Michel giovanni.lombardo(a)ecb.int wrote: > > Hi Michel, > > some time ago you mentioned that the automatic correction of the > parameters bounds in estimation in Dynare was a controversial issue. > It is still giving me big problems in estimating the NAWM-FIN. (it > crashes even if the initial value is the mode!!) > > At every distribution update I have to replace dynare_estimation.m > with my version where I comment out your replacement of the bounds. > And I have to do so to all installations of Dynare (e.g. of my > co-authors etc.) > > Would it be possible for you in the next update to set it as an > optional thing (e.g. options_.correct_bounds)? > > Thanks and best regards > > Gianni > > PS: I have already an entry in the forum for this issue. > > > > > Any e-mail message from the European Central Bank (ECB) is sent in > good faith but shall neither be binding nor construed as constituting > a commitment by the ECB except where provided for in a written > agreement. This e-mail is intended only for the use of the > recipient(s) named above. Any unauthorised disclosure, use or > dissemination, either in whole or in part, is prohibited. If you have > received this e-mail in error, please notify the sender immediately > via e-mail and delete this e-mail from your system. > > >

1 0

Re: [DynareDev] Sparse Hessian
by G. Perendia 08 Apr '09

08 Apr '09

PS: The below sparse Hessian issue occurs when one uses the use_dll option - i.e. the hessian from compiled <mod>_dynamic.dll is not sparse but the one from the matlab version <mod>_dynamic.m is. However, the solution I suggested below works in both cases Best regards George ----- Original Message ----- From: "G. Perendia" <george(a)perendia.orangehome.co.uk> To: "List for Dynare developers" <dev(a)dynare.org> Sent: Tuesday, April 07, 2009 11:00 AM Subject: Re: [DynareDev] Sparse Hessian- poss solution > > I added a new line 546: > > hessian = sparse(hessian); > > to my dr1.m and that seems to work fine. > > Shall I commit it? > > Best regards > George > ----- Original Message ----- > From: "G. Perendia" <george(a)perendia.orangehome.co.uk> > To: "List for Dynare developers" <dev(a)dynare.org> > Sent: Tuesday, April 07, 2009 10:49 AM > Subject: [DynareDev] Sparse Hessian > > > > Sorry, I forgot to change the title > > > > PS: When I pass a sparsed hessian, the product works fine but I have the > > same error /problem if I run with 4.0.3 matlab set. > > > > ----- Original Message ----- > > From: "G. Perendia" <george(a)perendia.orangehome.co.uk> > > To: "List for Dynare developers" <dev(a)dynare.org> > > Sent: Tuesday, April 07, 2009 10:36 AM > > Subject: Re: [DynareDev] theoretical moments of non stationary variables > > > > > > > Hi > > > > > > When I try to run Dynare (stoch_simul) using binaries from 4.0.3 (and > > 4.0.2 > > > too) in combination with the latest matlab code and the latest dr1.m, I > > get > > > the following error on line 547 > > > > > > rhs = -sparse_hessian_times_B_kronecker_C(hessian,zx); > > > > > > ??? First input must be a sparse (dynare) hessian matrix. > > > > > > as it seems because the hessian is not sparse but flat 2D. > > > > > > Do I just need to rebuild the latest binaries (which I have few separate > > > problems with) or is there something else that needs to be done? > > > > > > Best regards > > > > > > George > > > > > > > > > > > > _______________________________________________ > > > Dev mailing list > > > Dev(a)dynare.org > > > http://www.dynare.org/cgi-bin/mailman/listinfo/dev > > > > > > > > _______________________________________________ > > Dev mailing list > > Dev(a)dynare.org > > http://www.dynare.org/cgi-bin/mailman/listinfo/dev >

4 9

Re: [DynareDev] some news on monfispol
by Michel Juillard 08 Apr '09

08 Apr '09

Marco, take also a look at http://www.dynare.org/DynareWiki/UsingMultithreadedDlls Best Michel Marco Ratto wrote: > Dear Michel, > > we have drafted two documents for the state-of-the-art report. We > kept them rather short, with the idea to just place MONFISPOL in the > literature, avoiding a full dissertation on the topics which would > make the report extremely long. Please have a look if they look OK to > you. > > We also have coded a generic MATLAB engine for parallelizing MATLAB > loops. I would like to begin committing it in official dynare. I > would like to know if you agree in committing those routines and > where should I commit them: in trunk\matlab? in trunk\matlab\threads. > > Concerning the new trunk\matlab\threads folder, I have a question: in > our generic engine we have to code checks (error traps) for the number > of processors actually present in the computer and so on. Do you have > already coded them or we proceed autonomously on that? > > Please let us know, > All the best > Marco >

1 0

theoretical moments of non stationary variables
by Stéphane Adjemian 07 Apr '09

07 Apr '09

Hi Michel, A question related to the last post by bigbigben on the forum. Why don't we report infinite second order moments for non stationary endogenous variables? Dynare crashes when all the endogenous variables listed after the stoch_simul command are non stationary. Best, Stéphane.

3 4

saved copy of bayestopt_
by Stéphane Adjemian 06 Apr '09

06 Apr '09

Hi Michel, Why is bayestopt_ saved in <MOD_FILE_NAME>_mode ? If we change the priors but load <MOD_FILE_NAME>_mode to initialize the optimization routine, the changes in the declared priors are not effective. I spent 20 minutes finding out that my problem was caused by this recent change ;-) I don't know where you need an old bayestopt_ but we could load only xparam1 and hh in dynare_estimation_1... Do you agree? Best, Stéphane.

2 2

mjdgges on karaba
by Stéphane Adjemian 06 Apr '09

06 Apr '09

Hi Michel, I tried to recompile the mex files on karaba and I obtained the following warning messages for mjdgges: mjdgges/mjdgges.c: In function 'mjdgges': mjdgges/mjdgges.c:62: warning: passing argument 4 of 'dgges_' from incompatible pointer type mjdgges/mjdgges.c:62: warning: passing argument 5 of 'dgges_' from incompatible pointer type mjdgges/mjdgges.c:62: warning: passing argument 7 of 'dgges_' from incompatible pointer type mjdgges/mjdgges.c:62: warning: passing argument 9 of 'dgges_' from incompatible pointer type mjdgges/mjdgges.c:62: warning: passing argument 10 of 'dgges_' from incompatible pointer type mjdgges/mjdgges.c:62: warning: passing argument 15 of 'dgges_' from incompatible pointer type mjdgges/mjdgges.c:62: warning: passing argument 17 of 'dgges_' from incompatible pointer type mjdgges/mjdgges.c:62: warning: passing argument 19 of 'dgges_' from incompatible pointer type mjdgges/mjdgges.c:62: warning: passing argument 20 of 'dgges_' from incompatible pointer type mjdgges/mjdgges.c:62: warning: passing argument 21 of 'dgges_' from incompatible pointer type Is this normal ? I used matlab 2008b. Best, Stéphane.

2 2

dlls for Matlab 7.8
by Michel Juillard 30 Mar '09

30 Mar '09

I fixed build_matlab.m and mjdgges.c for compatibility issue with Matlab 7.8 When we start releasing binaries for Windows 64bit, we will need to rethink the architecture of directories and distribution mode of the DLLs Best Michel

1 0

forum dynare v4
by Stéphane Adjemian 28 Mar '09

28 Mar '09

Hi Michel, How can we access to the (closed) forum of Dynare version 4? à+ Stéphane.

2 2

Matlab r2009a mexfiles BLAS LAPACK
by Michel Juillard 28 Mar '09

28 Mar '09

There is indeed a change in arguments to calls to Blas or Lapack functions on 64 bit platforms. See http://www.mathworks.com/access/helpdesk/help/techdoc/index.html?/access/he… "MEX-Files Calling BLAS or LAPACK Functions Must Be Updated On 64-Bit Platforms" I will change the mex sources expanding the mechanism we put in place for the pointers across different Matlab versions. Hopefully over the week-end. Best Michel

1 0

r2009a
by Michel Juillard 28 Mar '09

28 Mar '09

I have installed Matlab r2009a on my Linux Debian and the old DLLs work with it. I ran build_matlab.m and I don't have the link problem reported on the forum, but there are warnings on at the compilation of mjdgges.c and the DLL doesn't work. I will investigate the issue and let you know. Best Michel

1 0

Re: [DynareDev] AIM in Matlab (and DYNARE question)
by Michel Juillard 27 Mar '09

27 Mar '09

1) I can't either create pages on DynareWiki. The software was recently updated and this must have created the problem. 2) It makes good sense that problems with the solution method will trash estimation results. However, I prefer to test with stoch_simul, because the result is simpler and is supposed to be reproducible rather than with estimation that is a much longer chain of operations besides solution and may depend of random draws in the optimization step. Thanks again for your help on that. Best Michel G. Perendia wrote: > Dear Michel > > 1) I do not seem to be able to edit/create new Wiki page: when I add a new > title to my page (e.g MatlabEigsBug) and then click link to it after saving > my page, I get an unfamiliar page saying: > > This page does not exist yet. You can create a new empty page, or use one of > the page templates. > Action(edit,Create new empty page) > > Or choose a page template to create an already-formatted page: > Before creating the page, please check if a similar page already > exists. Here are some existing pages with similar names: > > EditTemplates > LikePages((none)) > > > > > but clicking on any of the given options makes no effect. > > > 2) Running estimation was useful after all: The eigs() problem was causing > very many AIM errors of type 4 - too few big roots.- to be reported during > the ML and MCMC estimation (see the ..AIM_truncated_run log I sent this > morning) but none of type 4 when I rerun the same estimation after the > change - only 2 of type 3 - too many big roots. Also, it affected the > results of the estimation : the likelihood was rather lower before than > that after the modification and the estimated parameters different. See new > log enclosed. > > Best regards > > George > Mob. +44(0)7951415480 > artilogica(a)btconnect.com > > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > To: "G. Perendia" <george(a)perendia.orangehome.co.uk> > Sent: Thursday, March 26, 2009 8:03 PM > Subject: Re: [DynareDev] AIM in Matlab (and DYNARE question) > > > >> Thank you very much George. I can't access the bug report, but it's OK. >> >> Could you please add a page to DynareWiki describing the problem a) in >> Matlab b) its consequence for AIM >> >> For the other test cases that you prepare, could you please remove the >> estimation part and do the same comparison as I did with and without >> AIM? I'm curious whether the bug is frequent or if we found a crazy case >> by chance. >> >> All the best, >> >> Michel >> >> >> G. Perendia wrote: >> >>> Hi Michel >>> >>> FYI: Mathworks eigs() bug report record (though I'm not sure if you can >>> access it this way): >>> >>> >>> > http://www.mathworks.com/support/service_requests/Service_Request_Detail.do… > >>> Best regards >>> >>> George >>> ----- Original Message ----- >>> From: "Michel Juillard" <michel.juillard(a)ens.fr> >>> To: <Gary.Anderson(a)frb.gov>; "List for Dynare developers" >>> > <dev(a)dynare.org> > >>> Sent: Thursday, March 26, 2009 5:27 PM >>> Subject: Re: [DynareDev] AIM in Matlab (and DYNARE question) >>> >>> >>> >>> >>>> Hi Gary, >>>> >>>> I will keep you posted on Mathworks answer concerning eigs(). >>>> >>>> Cocnerning the tests, I have been accumulating a series of tests over >>>> the years, that I'm planing to commit to SVN in the coming weeks. >>>> Why don't you start by seeing what I already have and then we discuss a >>>> strategy to be more systematic. >>>> >>>> All the best, >>>> >>>> Michel >>>> >>>> Gary.Anderson(a)frb.gov wrote: >>>> >>>> >>>>> michel, >>>>> that is odd. when i run eigs, i get one of two sets of 10 >>>>> > eigenvalues. > >>> i >>> >>> >>>>> didn't investigate the invariant space vectors. >>>>> >>>>> >>>>> sometime ago i was supposed to develop a suite of tests for dynare. >>>>> >>>>> >>> since >>> >>> >>>>> it has been so long i thought i should touch base with you >>>>> to see if you think it would still be useful, and if so, what areas i >>>>> >>>>> >>> might >>> >>> >>>>> focus my energies on. >>>>> >>>>> >>>>> gary >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Michel Juillard >>>>> <michel.juillard@ >>>>> ens.fr> >>>>> >>>>> >>> To >>> >>> >>>>> Gary.Anderson(a)frb.gov, "G. >>>>> 03/26/2009 12:29 Perendia" >>>>> PM >>>>> >>>>> >>> <george(a)perendia.orangehome.co.uk>, >>> >>> >>>>> giovanni.lombardo(a)ecb.int, >>>>> Daniela.Bragoli(a)ecb.int >>>>> >>>>> >>>>> >>> cc >>> >>> >>> Subject >>> >>> >>>>> AIM in Matlab >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Hi Gary, >>>>> >>>>> in experimenting with AIM inside Dynare, George came accross this very >>>>> weir behaviour of Matlab eigs() function. >>>>> >>>>> For a rather large matrix (101x101) >>>>> >>>>> [w,d]=eigs(a',10,'LM',opts);sort(diag(10)) >>>>> with opts containing disp:0, >>>>> >>>>> returns different eigenvalues when called repeatedly without changing >>>>> anything. >>>>> >>>>> I'm attaching a.mat that contains the a matrix (and opts, among other >>>>> things) >>>>> >>>>> For now, we are disabling the using of eigs() in Dynare and forcing >>>>> > the > >>>>> use of eig() >>>>> >>>>> We are going also to report the problem to Matlab. >>>>> >>>>> All the best, >>>>> >>>>> Michel >>>>> >>>>> (See attached file: a.mat) >>>>> >>>>> >>>> _______________________________________________ >>>> Dev mailing list >>>> Dev(a)dynare.org >>>> http://www.dynare.org/cgi-bin/mailman/listinfo/dev >>>> >>>> >>> >>> >>> >> > > > >

2 1

Re: [DynareDev] AIM in Matlab (and DYNARE question)
by Michel Juillard 26 Mar '09

26 Mar '09

Hi Gary, I will keep you posted on Mathworks answer concerning eigs(). Cocnerning the tests, I have been accumulating a series of tests over the years, that I'm planing to commit to SVN in the coming weeks. Why don't you start by seeing what I already have and then we discuss a strategy to be more systematic. All the best, Michel Gary.Anderson(a)frb.gov wrote: > michel, > that is odd. when i run eigs, i get one of two sets of 10 eigenvalues. i > didn't investigate the invariant space vectors. > > > sometime ago i was supposed to develop a suite of tests for dynare. since > it has been so long i thought i should touch base with you > to see if you think it would still be useful, and if so, what areas i might > focus my energies on. > > > gary > > > > > > Michel Juillard > <michel.juillard@ > ens.fr> To > Gary.Anderson(a)frb.gov, "G. > 03/26/2009 12:29 Perendia" > PM <george(a)perendia.orangehome.co.uk>, > giovanni.lombardo(a)ecb.int, > Daniela.Bragoli(a)ecb.int > cc > > Subject > AIM in Matlab > > > > > > > > > > > Hi Gary, > > in experimenting with AIM inside Dynare, George came accross this very > weir behaviour of Matlab eigs() function. > > For a rather large matrix (101x101) > > [w,d]=eigs(a',10,'LM',opts);sort(diag(10)) > with opts containing disp:0, > > returns different eigenvalues when called repeatedly without changing > anything. > > I'm attaching a.mat that contains the a matrix (and opts, among other > things) > > For now, we are disabling the using of eigs() in Dynare and forcing the > use of eig() > > We are going also to report the problem to Matlab. > > All the best, > > Michel > > (See attached file: a.mat)

1 0

Dynare 4.0.3
by Sébastien Villemot 25 Mar '09

25 Mar '09

Yesterday we released Dynare 4.0.3, which is a bug fixing release. The Windows and Linux packages are available as usual. Michel: it seems that you forgot to upload the source package. Best, Sébastien

1 0

Kirikou
by Sébastien Villemot 25 Mar '09

25 Mar '09

Hi all, Today I migrated our server "kirikou" to the latest Debian stable (codename "Lenny"). Everything went smoothly. The trickiest part was for the wikis: I had to upgrade the data for every wiki from version 1.5 to version 1.7 of Moinmoin. Please let me know if something appears to be broken. Best, Sébastien

1 0

linux mex files for dynare version 3
by Stéphane Adjemian 25 Mar '09

25 Mar '09

Hi Michel, I need to debug something on dynare version 3 (a recent post by glucke on the forum). Unfortunatly the mexglx files are not working on my linux... Where can I find the sources of the mex files? There is nothing on subversion (branches/3.xx). Stéphane.

2 5

Re: [DynareDev] AIM solver
by Michel Juillard 25 Mar '09

25 Mar '09

Hi George, 1) on which page of the Wiki is the faulty mexopt.bat? Chances are that I put it there, but at the moment I don't remember where it is. 2) What is the fix for spaces in the pathname? Where should be added quotes? Single quotes or double quotes? Thanks for you help Michel G. Perendia wrote: > Dear Michel > > I spent the afternoon trying to build the new release pre-processor and mex > files using build_matlab. but have to admit, without success. I will have to > use the pre-processor and dlls from the latest release 4..0.2 to test AIM > and I will deal with the build at some point later if that is OK. > > For start, MSVC would not work with .cc files and MinGW gcc fails in my > environment on Windows XP because paths with placements and spaces specified > in mexopt.bat from DynareWiki are not resolved properly, e.g > > gcc: C:DOCUME~1SculptorLOCALS~1Tempisopenmp.obj: No such file or directory > > In meantime, I resolved the issues with the spaces, i.e. > > gcc: C:\Program: No such file or directory > gcc: Files\MATLAB_SV71\extern\lib\win32\microsoft\libmex.lib: No such file > or directory > gcc: C:\Program: No such file or directory > > due to missing quotations but can not enclose modified mexopt.bat as is but > had to rename it as my server provider is refusing to accept a .bat file in > > Best regards > > George > Mob. +44(0)7951415480 > artilogica(a)btconnect.com > > ----- Original Message ----- > From: "G. Perendia" <george(a)perendia.orangehome.co.uk> > To: "Michel Juillard" <michel.juillard(a)ens.fr> > Sent: Tuesday, March 24, 2009 1:45 PM > Subject: Re: AIM solver > > > >> Hi Michel >> >> Thanks for fixing the problem with that elegant flipud/cumsum construct >> > and, > >> yes, I will test it . >> >> Best regards >> >> George >> artilogica(a)btconnect.com >> >> ----- Original Message ----- >> From: "Michel Juillard" <michel.juillard(a)ens.fr> >> To: "G. Perendia" <george(a)perendia.orangehome.co.uk>; "Sébastien Villemot" >> <sebastien.villemot(a)ens.fr> >> Sent: Tuesday, March 24, 2009 1:29 PM >> Subject: AIM solver >> >> >> >>> Hi George, >>> >>> as you saw I fixed a problem in the implementation of AIM solver when >>> some variabls have zero coefficients. The column in bb_ord is then >>> empty, but Dynare expect a column there. >>> Concerning AIM, could you please build 5 test cases: >>> 1) model with leads and lags on one period >>> 2) model with leads and lags on several periods >>> 3) model with leads and lags on several periods, but with holes: a >>> variable is present at t-1 and t-3, but not at t-2 >>> 4) model with only leads >>> 5) model with only lags >>> >>> One set of *.mod files should solve it with mjdgges, another set should >>> use AIM. Add please at the end of those a comparison for dr.ghx and >>> dr.ghu between the AIM solution and the mjdgges version. Please, create >>> a directory AIM in trunk/tests and upload these files. >>> >>> Thanks in advance and all the best, >>> >>> Michel >>> >>>

2 2

branches/4.0
by Michel Juillard 20 Mar '09

20 Mar '09

I have fixed the problems with dynare_estimation_1, but running various tests, I'm still finding bugs we may not have seen before. I will let you know when I'm done. Best Michel

1 0

Dynare 4.0.3 release candidate
by Sébastien Villemot 20 Mar '09

20 Mar '09

Hi all, We are about to release version 4.0.3, which fixes a lot of bugs present in 4.0.2. Please update your subversion repository, and test the code for the 4.0 branch, which is stored in the "branches/4.0" subdirectory. If nobody objects, we should release it next monday. All the best Sébastien

2 1

Re: [DynareDev] sensitivity toolbox
by Michel Juillard 19 Mar '09

19 Mar '09

Hi Marco, s sorry about that. This is an example of bad coordination of Dynare changes that we need to be more careful about in the future. 1) We are going to release (next week ?) version 4.0.3 that will be a bug fixing release of version 4.0.2 (Oct 2008) 2) We are going to release 4.1 in one or two months with new features and that will include everything that is now in the trunk of the SVN archive. So ideally, you should provide quickly a fix for version 4.0.2/3 as soon as possible and prepare a version working with current SVN in the coming weeks. I'm planing to incorporate the changes that you made in the last months to your functions on SVN to version 4.0.3, probably tomorrow. So maybe, you should wait that I have done it to work on your toolbox for version 4.0.3 What will become version 4.0.3 is kept in SVN as dynare/branches/4.0 What are the main changes in SVN trunk that broke your toolbox? Best Michel Marco Ratto wrote: > Dear Michel, > > sorry for tempesting you today ... > > I sometimes have bugs/questions related to my toolbox. > I noted most people downloads the dynare v4 installation of October > 2008 and uses this without any update. To comply with this version, > I have a couple of corrections to my routines. > > Moreover, to comply with the latest SVN version of dynare v4, I need > much more modification to the toolbox. > > So, I have two questions: > 1) shall I suggest user to update the latest SVN version, and only > care about the latest modifications and do a full update of my toolbox? > 2) shall I care of keeping my toolbox working with October 2008 > version and current version of dynare? > > Many thanks for your help > best wishes > Marco >

2 2

Dynare subversion server: migration complete
by Sébastien Villemot 19 Mar '09

19 Mar '09

Hi everybody, The migration of the Dynare subversion server is complete. Dynare subversion is now accessible at: https://www.dynare.org/svn/dynare We migrated the server to a secure connection (it is HTTPS and not HTTP !!); there should be no security warning, since we have bought an official digital certificate from Comodo. For committing changes, the user accounts and passwords are unchanged; however, for increased security, we should change every individual password in the future. We will do this in due time, when we find a way to securely exchange passwords. The new layout of the repository is the following: - trunk/: contains the version under development (formerly known as dynare_v4/) - branches/: containes the 3.x and 4.0 stable branches - tags/: contains official dynare releases You have to download a fresh working copy of the repository (for several technical reasons it is impossible to convert your already existing working copies; sorry for the inconvenience). With command-line subversion, you should issue the following command: svn checkout https://www.dynare.org/svn/dynare Please also note that Dynare++ is now in a separate repository, at address: https://www.dynare.org/svn/dynare++ The same remarks apply. Ondra: you can contact me if you want me to setup mail announcements for each changeset, as for the Dynare repository. Please also feel free to reorganize the layout of the Dynare++ repository, since for the moment it only contains a single dynare++ subdirectory. For the moment there is only a very rough web SVN browser, accessible at the same address than the repositories (type https://www.dynare.org/svn/dynare in your web browser). I will try to setup a more friendly web browser soon. All the best Sébastien

4 4

parser broken
by Michel Juillard 24 Jan '09

24 Jan '09

Hi Sebastien, since the last update, the parser doesn't accept empty expression as can be found in uniform priors when we only put something in p3 and p4, so we have something like sig,uniform_pdf,,,0,1; Best Michel

1 0

Estimated params
by Sébastien Villemot 22 Jan '09

22 Jan '09

Hi Stéphane (and everybody else), I just committed the change to allow arbitrary expressions in estimated_params block (and also in estimated_params_init and estimated_params_bounds). Previously only constants were allowed there. This involved more modifications than I initially thought, so please test it a little bit to make sure that I didn't break anything (I tested it on fs2000.mod only). Also note that by the way I fixed a bug when estimated_params_init and estimated_params_bounds were used in conjunction with correlations of endogenous variables; I also applied the fix to the 4.0 branch. Best Sébastien

1 0

Dynare development workshop
by Michel Juillard 14 Jan '09

14 Jan '09

Dear Friends, we would like to organize an internal workshop on Dynare development in Paris on February 19 and 20, 2009. In particular, we would like to discuss methods to assess identification of estimated parameters and issues around estimation of nonstationary variables. But, this would also be the occasion to review briefly all development projects. Please confirm as soon as possible that these dates are possible for you. Kind regards, Michel

2 1

Earleir reported Checkout Problem update
by G. Perendia 22 Dec '08

22 Dec '08

Hi According to http://ubuntuforums.org/showthread.php?t=552453 "the server computer had run out of space!" Best regards George Mob. +44(0)7951415480 artilogica(a)btconnect.com ----- Original Message ----- From: "G. Perendia" <george(a)perendia.orangehome.co.uk> To: "List for Dynare developers" <dev(a)dynare.org> Sent: Sunday, December 21, 2008 11:17 AM Subject: Re: [DynareDev] Dynare subversion server: migration complete > Hi > > I am having difficulties downloading new repository (which I need to do > before I can commit anything) > > I am getting the following error messages > > Error: REPORT request failed on '/svn/dynare/!svn/vcc/default' > Error: Can't find a temporary directory: Internal error > > Best regards > > George > > ----- Original Message ----- > From: "Sébastien Villemot" <sebastien.villemot(a)ens.fr> > To: "Tommaso Mancini" <tommaso.mancini(a)stanfordalumni.org>; > <marco.ratto(a)jrc.it>; "Junior Maih" <junior.maih(a)gmail.com>; "Heesun Kiem" > <HKiem(a)imf.org>; "Gianni Lombardo" <giovanni.lombardo(a)ecb.int>; > <dev(a)dynare.org>; "Ondra Kamenik" <ondra.kamenik(a)gmail.com>; "Assia > Ezzeroug" <Assia.Ezzeroug(a)ens.fr>; <Gary.Anderson(a)frb.gov> > Sent: Friday, December 19, 2008 5:04 PM > Subject: [DynareDev] Dynare subversion server: migration complete > > > > Hi everybody, > > > > The migration of the Dynare subversion server is complete. > > > > Dynare subversion is now accessible at: > > > > https://www.dynare.org/svn/dynare > > > > We migrated the server to a secure connection (it is HTTPS and not HTTP > !!); there should be no security warning, since we have bought an official > digital certificate from Comodo. > > > > For committing changes, the user accounts and passwords are unchanged; > however, for increased security, we should change every individual password > in the future. We will do this in due time, when we find a way to securely > exchange passwords. > > > > The new layout of the repository is the following: > > - trunk/: contains the version under development (formerly known as > dynare_v4/) > > - branches/: containes the 3.x and 4.0 stable branches > > - tags/: contains official dynare releases > > > > You have to download a fresh working copy of the repository (for several > technical reasons it is impossible to convert your already existing working > copies; sorry for the inconvenience). > > With command-line subversion, you should issue the following command: > > > > svn checkout https://www.dynare.org/svn/dynare > > > > > > Please also note that Dynare++ is now in a separate repository, at > address: > > > > https://www.dynare.org/svn/dynare++ > > > > The same remarks apply. > > > > Ondra: you can contact me if you want me to setup mail announcements for > each changeset, as for the Dynare repository. Please also feel free to > reorganize the layout of the Dynare++ repository, since for the moment it > only contains a single dynare++ subdirectory. > > > > > > For the moment there is only a very rough web SVN browser, accessible at > the same address than the repositories (type > https://www.dynare.org/svn/dynare in your web browser). I will try to setup > a more friendly web browser soon. > > > > All the best > > > > Sébastien > > > > > > > > _______________________________________________ > > Dev mailing list > > Dev(a)dynare.org > > http://www.dynare.org/cgi-bin/mailman/listinfo/dev > > >

3 4

Re: [DynareDev] Dynare subversion server Download problems Update
by G. Perendia 21 Dec '08

21 Dec '08

Hi, Just a note: using my 2nd machine which has the latest Tortoise SVN 1.5.5 to checkout, I get the following messages (and the same error): "Command: Checkout from https://www.dynare.org/svn/dynare/trunk, revision HEAD, Fully recursive, Externals included Error: Can't find a temporary directory: Internal error Finished!: " Best regards George ----- Original Message ----- From: "G. Perendia" <george(a)perendia.orangehome.co.uk> To: "Michel Juillard" <michel.juillard(a)ens.fr> Cc: "List for Dynare developers" <dev(a)dynare.org> Sent: Sunday, December 21, 2008 5:10 PM Subject: Re: [DynareDev] Dynare subversion server: migration complete] > Dear Michel > > I did and it is both the new site, > > https://www.dynare.org/svn/dynare > > and the trunk subdirectory that I have been trying to download from: > > https://www.dynare.org/svn/dynare/trunk > > but I still have no luck and I am repeatedly getting same error messages: > > Error: REPORT request failed on '/svn/dynare/!svn/vcc/default' > Error: Can't find a temporary directory: Internal error > > > Best regards > > George > > ----- Original Message ----- > From: "Michel Juillard" <michel.juillard(a)ens.fr> > To: "George Perendia" <artilogica(a)btconnect.com>; "G. Perendia" > <george(a)perendia.orangehome.co.uk> > Sent: Sunday, December 21, 2008 1:35 PM > Subject: [Fwd: [DynareDev] Dynare subversion server: migration complete] > > > > > > Hi George, > > > > you should have received this message via the Dynare developers email > list. > > > > Best > > > > Michel > > > > -------- Original Message -------- > > Subject: [DynareDev] Dynare subversion server: migration complete > > Date: Fri, 19 Dec 2008 18:04:54 +0100 > > From: Sébastien Villemot <sebastien.villemot(a)ens.fr> > > Reply-To: List for Dynare developers <dev(a)dynare.org> > > To: Tommaso Mancini <tommaso.mancini(a)stanfordalumni.org>, > > marco.ratto(a)jrc.it, Junior Maih <junior.maih(a)gmail.com>, Heesun Kiem > > <HKiem(a)imf.org>, Gianni Lombardo <giovanni.lombardo(a)ecb.int>, > > dev(a)dynare.org, Ondra Kamenik <ondra.kamenik(a)gmail.com>, Assia Ezzeroug > > <Assia.Ezzeroug(a)ens.fr>, Gary.Anderson(a)frb.gov > > > > > > > > Hi everybody, > > > > The migration of the Dynare subversion server is complete. > > > > Dynare subversion is now accessible at: > > > > https://www.dynare.org/svn/dynare > > > > We migrated the server to a secure connection (it is HTTPS and not HTTP > !!); there should be no security warning, since we have bought an official > digital certificate from Comodo. > > > > For committing changes, the user accounts and passwords are unchanged; > however, for increased security, we should change every individual password > in the future. We will do this in due time, when we find a way to securely > exchange passwords. > > > > The new layout of the repository is the following: > > - trunk/: contains the version under development (formerly known as > dynare_v4/) > > - branches/: containes the 3.x and 4.0 stable branches > > - tags/: contains official dynare releases > > > > You have to download a fresh working copy of the repository (for several > technical reasons it is impossible to convert your already existing working > copies; sorry for the inconvenience). > > With command-line subversion, you should issue the following command: > > > > svn checkout https://www.dynare.org/svn/dynare > > > > > > Please also note that Dynare++ is now in a separate repository, at > address: > > > > https://www.dynare.org/svn/dynare++ > > > > The same remarks apply. > > > > Ondra: you can contact me if you want me to setup mail announcements for > each changeset, as for the Dynare repository. Please also feel free to > reorganize the layout of the Dynare++ repository, since for the moment it > only contains a single dynare++ subdirectory. > > > > > > For the moment there is only a very rough web SVN browser, accessible at > the same address than the repositories (type > https://www.dynare.org/svn/dynare in your web browser). I will try to setup > a more friendly web browser soon. > > > > All the best > > > > Sébastien > > > > > > > > _______________________________________________ > > Dev mailing list > > Dev(a)dynare.org > > http://www.dynare.org/cgi-bin/mailman/listinfo/dev > > > > >

2 2

Re: [DynareDev] Dynare subversion server: migration complete]
by G. Perendia 21 Dec '08

21 Dec '08

Dear Michel I did and it is both the new site, https://www.dynare.org/svn/dynare and the trunk subdirectory that I have been trying to download from: https://www.dynare.org/svn/dynare/trunk but I still have no luck and I am repeatedly getting same error messages: Error: REPORT request failed on '/svn/dynare/!svn/vcc/default' Error: Can't find a temporary directory: Internal error Best regards George Mob. +44(0)7951415480 artilogica(a)btconnect.com ----- Original Message ----- From: "Michel Juillard" <michel.juillard(a)ens.fr> To: "George Perendia" <artilogica(a)btconnect.com>; "G. Perendia" <george(a)perendia.orangehome.co.uk> Sent: Sunday, December 21, 2008 1:35 PM Subject: [Fwd: [DynareDev] Dynare subversion server: migration complete] > > Hi George, > > you should have received this message via the Dynare developers email list. > > Best > > Michel > > -------- Original Message -------- > Subject: [DynareDev] Dynare subversion server: migration complete > Date: Fri, 19 Dec 2008 18:04:54 +0100 > From: Sébastien Villemot <sebastien.villemot(a)ens.fr> > Reply-To: List for Dynare developers <dev(a)dynare.org> > To: Tommaso Mancini <tommaso.mancini(a)stanfordalumni.org>, > marco.ratto(a)jrc.it, Junior Maih <junior.maih(a)gmail.com>, Heesun Kiem > <HKiem(a)imf.org>, Gianni Lombardo <giovanni.lombardo(a)ecb.int>, > dev(a)dynare.org, Ondra Kamenik <ondra.kamenik(a)gmail.com>, Assia Ezzeroug > <Assia.Ezzeroug(a)ens.fr>, Gary.Anderson(a)frb.gov > > > > Hi everybody, > > The migration of the Dynare subversion server is complete. > > Dynare subversion is now accessible at: > > https://www.dynare.org/svn/dynare > > We migrated the server to a secure connection (it is HTTPS and not HTTP !!); there should be no security warning, since we have bought an official digital certificate from Comodo. > > For committing changes, the user accounts and passwords are unchanged; however, for increased security, we should change every individual password in the future. We will do this in due time, when we find a way to securely exchange passwords. > > The new layout of the repository is the following: > - trunk/: contains the version under development (formerly known as dynare_v4/) > - branches/: containes the 3.x and 4.0 stable branches > - tags/: contains official dynare releases > > You have to download a fresh working copy of the repository (for several technical reasons it is impossible to convert your already existing working copies; sorry for the inconvenience). > With command-line subversion, you should issue the following command: > > svn checkout https://www.dynare.org/svn/dynare > > > Please also note that Dynare++ is now in a separate repository, at address: > > https://www.dynare.org/svn/dynare++ > > The same remarks apply. > > Ondra: you can contact me if you want me to setup mail announcements for each changeset, as for the Dynare repository. Please also feel free to reorganize the layout of the Dynare++ repository, since for the moment it only contains a single dynare++ subdirectory. > > > For the moment there is only a very rough web SVN browser, accessible at the same address than the repositories (type https://www.dynare.org/svn/dynare in your web browser). I will try to setup a more friendly web browser soon. > > All the best > > Sébastien > > > > _______________________________________________ > Dev mailing list > Dev(a)dynare.org > http://www.dynare.org/cgi-bin/mailman/listinfo/dev > >

1 0

Dynare subversion server: migration complete (bis)
by Sébastien Villemot 19 Dec '08

19 Dec '08

Two last remarks: - in normal times, you should only commit changes to the "trunk/" directory. No change should ever be made to the "tags/" subdirectory, which contains historical versions of Dynare. Changes to the "branches/" directories should only be bugfixes for inclusion in the future stable release. - the subversion no longer contains any binary. This means that you must recompile the preprocessor and the Matlab DLLs by yourself. Some help can be found on: http://www.dynare.org/DynareWiki/BuildingPreprocessor http://www.dynare.org/DynareWiki/BuildingDlls Merry Christmas and happy new year to all of you Sébastien

1 0

Dynare subversion server
by Sébastien Villemot 19 Dec '08

19 Dec '08

Hi everybody, Today we are changing the location of Dynare subversion server. Please no longer use http://www.cepremap.cnrs.fr/svn/ as the source address for Dynare subversion. I will send you the new address once it is set up, normally in a few hours. Sorry for informing you at the very last moment, we had initially not planned to do the move so quickly. All the best Sébastien

1 0

Maintaining 4.0 branch
by Sébastien Villemot 04 Dec '08

04 Dec '08

Hi all, I realize that I can't do myself the job of synchronizing the trunk and the 4.0 branch of Dynare. I don't have enough knowledge of some parts of the code to be able to decide whether to merge in the 4.0 the changes made to the trunk. I now think that the right solution is that the person who fixes a bug in the trunk is responsible for fixing it in the 4.0 branch (provided it also appears there, of course). Since the release of Dynare 4.0.2 (SVN revision 2224), there have been several changesets which fixed bugs (in the smoother, in OSR, in mult_elimination...). I prefer to leave you the work of merging the relevant ones in the 4.0 branch. The idea is to merge only bug fixes, but not new features or code restructuring (which could create new bugs), so that we can reasonably think that the number of bugs is stricly decreasing in the 4.0.x series. There are basically two ways of merging a changeset: - if the 4.0 branch and the trunk code don't differ much in the modified zones, you can use the "svn merge" command. For example, if changeset 3456 contains bugfixes in the trunk that you want to merge, go to "branches/4.0" directory, and type: svn merge -c 3456 http://www.cepremap.cnrs.fr/svn/dynare_v4 This will apply the changeset to your local copy of the 4.0 branch. If everything goes well, you can now commit it. If there is a conflict you need to resolve it by hand. Note that it is possible to give several changesets to the -c option, such as "-c 3456,4567,5678" - otherwise, if the two codes differ two much, just fix the code "by hand" in the 4.0 branch and commit In the changelog message, please indicate the number of the changeset that your merged, and a short description of what it did. Cheers Sébastien

1 0

Re: [DynareDev] integratio of Partial information and current expectations into Dynare
by Michel Juillard 03 Dec '08

03 Dec '08

Hi there, the guiding principle should be to do the integration right from the start and commit the necessary resources rather than having to do it over later. 1) to you have a write-up with the algebra that needs to be performed for partial information inside Dynare? 2) we have first to decide whether it is best to treat expectations as different (auxiliary) endogenous variables or to define a 4th type. - Currently, in full information, if we want E_{t-1} y_t, one needs to create an auxiliary variable and an auxiliary equation ey = y(+1) and use ey(-1) instead of E_{t-1} y_t. If we are to provide an explicit operator E{i}{y}, the preprocessor would need to create the auxiliary variable, add the equation to the model, and perform the substitution as needed. We would need to keep a table of these auxiliary variables, but it doesn't seem necessary or desirable to create a 4th type of variable because the rules for output would be the same as for regular endogenous variables [Currently, I have also doubt about the distinction endogenous/exogenous, but this is another story...] Are there reasons to believe this would be different with partial information? Best Michel Quoting "G. Perendia" <george(a)perendia.orangehome.co.uk>: > Dear Joe > > I have been thinking about the integration of the Partial information and > current expectations into Dynare as you asked me to and I have some ideas but > no easy solutions. > > As I mentioned, a shortcut solution may require additional proxy variables > for contemporary expectations such as ETYT_t = Et[y_t]. > > I expect an additional expectation error equations will be required that > should avoid determinacy issues with the additional variable and the possible > problems when solved, e,g, for steady state, poss on the lines of: > > y_t = ETYT_t + e_t > > Using then an additional structure (e.g. set inside the M_ or options_ > structures) that is mapping expectations to their bases on the lines of: > > ['YTT' 'Y_t'; ......] > > the Part info solver will be able to identify which of the relations are > "dummy" and then transform and solve the model for decision rule using its > own solver. > > This solution however may become rather tedious to model from the user > perspective, that is, if at all plausible - so, please let me know what you > think. It will also require some parsing of the model within PI module to > map-in expectations. > > > Alternatively, the proper integration would require changing start-to-end > Dynare pipeline, from the parser/prerpocessor to KF estimation. It may then > require adding 4th type of variable: expectation in addition to the existing: > predetermined, forward and static variables, and, thus, changes to the > numerous internal structures and functions. > > This will, however, probably introduce an additional burden on the design and > development resources, rather more than initially envisaged. > > > Best regards > > George > Mob. +44(0)7951415480 > artilogica(a)btconnect.com > --

5 11

4.0.2 release
by Sébastien Villemot 27 Oct '08

27 Oct '08

Hi all I have merged bug corrections from the trunk into the 4.0 branch, so we are close to release Dynare 4.0.2. The biggest changes since 4.0.1 are fixes in the Kalman filter and smoother; if you have some spare time, don't hesitate to test those parts. Otherwise, do you see anything that should be done before the release, or is it ok for you? Best Sébastien

1 0

Karaba
by Sébastien Villemot 23 Oct '08

23 Oct '08

Hi all Our new workstation "Karaba" is now up and running. I have dedicated a wiki page to it, with some configuration informations: http://www.dynare.org/DynareWiki/KarabaConfiguration Only the first hard drive is used for the moment. I shall format and mount the second one later, when we configure the backup system. There is a "dynare" user account, with the same password than on Hal. I can create other user accounts upon request. Michel: for the root password, we should change it when we see each other physically Best Sébastien

2 2

Wikis have moved
by Sébastien Villemot 20 Oct '08

20 Oct '08

Hi all I have just moved all wikis from Pythie to Kirikou. For example, the DynareWiki is now accessible at: http://www.dynare.org/DynareWiki I have also moved the following wikis: DsgeAdminWiki, GpmWiki, GimfWiki, CefWiki, JedcTestsuiteWiki A transparent redirection has been set up, so that if you type the old address you automatically get redirected to the new one. Note that since cepremap.cnrs.fr is not going to survive for a long time, we should advertise that change as much as possible. Please check that everything is working fine, and tell me if I should remove some useless wiki. Best Sébastien

1 0

Hal's hard drive is full
by Sébastien Villemot 20 Oct '08

20 Oct '08

Until I find the opportunity to install the new hard drive, is it possible to free some space on the drive ? Sébastien

1 0

Dynare 4.0.1
by Sébastien Villemot 23 Sep '08

23 Sep '08

Salut Stéphane Michel a décidé de release Dynare 4.0.1. J'ai fait les paquets Debian, ainsi que les paquets source et amd64 pour Ubuntu. Reste à faire le paquet Ubuntu/i386. J'ai changé le système de gestion des paquets, afin d'une part d'avoir la signature numérique (voir http://www.cepremap.cnrs.fr/DynareWiki/InstallOnDebianOrUbuntu), et d'autre part pour automatiser au maximum la procédure d'upload vers kirikou (via le programme dupload et un script maison sur kirikou). La nouvelle procédure (simplifiée!) est expliquée sur: http://www.cepremap.cnrs.fr/DynareWiki/KirikouConfiguration Pour cette fois tu peux te contenter d'un paquet "binary-only" (dernière section de la page wiki). Si tu n'as pas le temps de le faire, je m'en chargerai (je compte m'installer un Ubuntu/i386 dans une machine virtuelle). A+ Sébastien

1 0

Hal's hard drive
by Sébastien Villemot 18 Sep '08

18 Sep '08

I figured out that Hal's hard drive is not SATA, but SAS (Serial Attached SCSI, see: http://en.wikipedia.org/wiki/Serial_Attached_SCSI). The existing HDD is a Maxtor Atlas 10k5 073SAS. I didn't know this technology before. Since there has several flavours of connectors for SAS, I think we should ask Dell for the proper hardware to buy. Also note that the wiki page says that second generation SATA drives can be connected to SAS buses. What's exactly the model number of Hal ? Sébastien

2 1

[Fwd: Devis Dell n°17431959/3]
by Michel Juillard 18 Sep '08

18 Sep '08

Here the quote for Hal. I believe that the delivery was indeed that. Best Michel -------- Original Message -------- Subject: Devis Dell n°17431959/3 Date: Tue, 12 Sep 2006 14:05:19 +0100 From: <Gregory_Guibert(a)DELL.com> To: <michel.juillard(a)ens.fr> M. MICHEL JUILLARD, Veuillez trouver ci-joint votre devis faisant suite à notre entretien. Pour valider votre commande, je vous remercie de me retourner l'intégralité de ce document daté, signé, avec votre "bon pour accord". Je reste à votre disposition pour tout complément d'information. Cordialement GREGORY GUIBERT Phone: 04 99 75 46 98 Fax: 04 99 75 46 98

1 0

DLLs, one more thing
by Michel Juillard 18 Sep '08

18 Sep '08

Hi George, I thought of yet another thing concerning the 1st order DLL. We need to implement the same type of weighted penalty as in resol.m for models for which we can't find the steady state and for models that violate Blanchard and Kahn conditions. Best Michel

2 8

[Fwd: Re: license]
by Michel Juillard 17 Sep '08

17 Sep '08

-------- Original Message -------- Subject: Re: license Date: Wed, 17 Sep 2008 15:38:10 -0400 From: Gary.Anderson(a)frb.gov To: Michel Juillard <michel.juillard(a)ens.fr> michel, congratulations on the 4.0.0 release. gary

1 0

[Fwd: Dynare version 4.0.0]
by Michel Juillard 17 Sep '08

17 Sep '08

-------- Original Message -------- Subject: Dynare version 4.0.0 Date: Tue, 16 Sep 2008 19:27:19 -0700 From: Wesley_Phoa(a)capgroup.com To: Michel Juillard <michel.juillard(a)ens.fr> Michel: I just took a break from the financial crisis to install it. Congratulations to everyone on the team! Best regards, Wesley Phoa.

1 0

[Fwd: r2081 - dynare_v4/debian]
by Michel Juillard 17 Sep '08

17 Sep '08

How do we handle these bugs in packaging for one particular architecture? Do we replace the package silently ? Do we have different version numbers for different packages? Do we make a bug fixing release for all architectures, even if nothing changes in some? Best Michel -------- Original Message -------- Subject: r2081 - dynare_v4/debian Date: Wed, 17 Sep 2008 18:02:33 +0200 From: sebastien(a)pythie.cepremap.cnrs.fr To: subversion_users(a)cepremap.cnrs.fr Author: sebastien Date: 2008-09-17 18:02:33 +0200 (Wed, 17 Sep 2008) New Revision: 2081 Modified: dynare_v4/debian/dynare.install Log: trunk debian/ubuntu packaging: added missing matlab/distributions/ and matlab/distributions/toolbox/ directories Modified: dynare_v4/debian/dynare.install =================================================================== --- dynare_v4/debian/dynare.install 2008-09-17 16:00:41 UTC (rev 2080) +++ dynare_v4/debian/dynare.install 2008-09-17 16:02:33 UTC (rev 2081) @@ -1,7 +1,9 @@ -matlab/*.m /usr/lib/dynare/matlab -matlab/gensylv/*.m /usr/lib/dynare/matlab/gensylv -matlab/qz/*.m /usr/lib/dynare/matlab/qz -matlab/kronecker/*.m /usr/lib/dynare/matlab/kronecker -mex/octave/*.mex /usr/lib/dynare/mex/octave -mex/octave/rcond.m /usr/lib/dynare/mex/octave -preprocessor/dynare_m /usr/lib/dynare/matlab +matlab/*.m /usr/lib/dynare/matlab +matlab/gensylv/*.m /usr/lib/dynare/matlab/gensylv +matlab/qz/*.m /usr/lib/dynare/matlab/qz +matlab/kronecker/*.m /usr/lib/dynare/matlab/kronecker +matlab/distributions/*.m /usr/lib/dynare/matlab/distributions +matlab/distributions/toolbox/*.m /usr/lib/dynare/matlab/distributions/toolbox +mex/octave/*.mex /usr/lib/dynare/mex/octave +mex/octave/rcond.m /usr/lib/dynare/mex/octave +preprocessor/dynare_m /usr/lib/dynare/matlab

2 4

Virtualization on Hal
by Sébastien Villemot 17 Sep '08

17 Sep '08

Hi all, I am trying to run a virtual machine on Hal, so that I can compile packages for Ubuntu/AMD64 there (and it would probably be possible to also run a Windows/64). The software I use for that (VirtualBox) is very intuitive, but needs a hardware feature, called Intel VT. The feature is present in Hal, but is currently disabled in the BIOS, and therefore we need a manual intervention on Hal for enabling it. Does any of you plan to go to Jourdan in the near future ? Best Sébastien

2 2

Subversion migration
by Sébastien Villemot 16 Sep '08

16 Sep '08

Hi all I have written a wiki page about the project of migrating the SVN server (and other related stuff): http://www.cepremap.cnrs.fr/DynareWiki/SubversionMigration Please write your comments/ideas on the wiki page or on the list. Best Sébastien P.S. for George and Ferhat: you don't seem subscribed to changes on the Wiki. It would probably be a good thing if you did so, since you would then be informed by e-mail of the modifications on the Wiki. For doing that, log in into the Wiki with your personal account, click on "Preferences" and put ".*" (a dot followed by a star, i.e. the catch-all regular expression) in the last field of the preferences page.

1 0

v4.0
by Michel Juillard 15 Sep '08

15 Sep '08

Hi everybody, I'm ready to release v 4.0. It is still far from perfect, but I think that the main functions are working. What do you think? Best Michel

4 3

Broken functions
by Sébastien Villemot 10 Sep '08

10 Sep '08

The following functions are implemented in the preprocessor, but are currently broken: * dsample: the preprocessor sets options_.dsample when encountering this command, but this option is never used in the Matlab code (just initialized in global_initialization); however, there exists a dsample.m which sets options_.smpl. I suppose I should fix this by asking the preprocessor to call dsample.m (or maybe even incorporate the content of dsample.m into the preprocessor). Am I right ? * calib and calib_var: it is clear that the Matlab files (calib.m, calib_obj.m and calib_obj2.m) corresponding to these functions are currently broken (the function prototypes contain global variables, which makes no sense...). What should we do about these? * model_comparison: the Matlab file corresponding to this preprocessor command is actually missing (it has not been ported from v3). Stéphane is going to look at this. Best Sébastien

1 0

/branches/4.0/locked ?
by Stéphane Adjemian 08 Sep '08

08 Sep '08

Hi all, Did someone locked the svn repository (/branches/4.0) ? I cannot commit... Best, Stéphane. Stéphane Adjemian Université du Maine, GAINS & CEPREMAP http://www.cepremap.cnrs.fr/~adjemian

3 3

changes in Dynare v4
by Michel Juillard 06 Sep '08

06 Sep '08

Hi everybody, I just realized that with all the changes that we made to the function names (distributions) and the calling sequences, we broke all code written by Dynare users and using Dynare internals. We need to document these changes quickly on DynareWiki. We need to write a tool (perl, python, ...) that -compares two dynare versions (only /matlab/ and subdirectories) -which *.m f ile were added/deleted -extracts the function ..... line, compares the argument lists and reports differences. Who can start working on it? Best Michel

3 3

freezing version 4.0.0
by Michel Juillard 05 Sep '08

05 Sep '08

Hi everybody, on SVN, I have created branches/4.0 to freeze version 4.0.0 Most changes necessary to release version 4.0.0 have been done. There are a few keywords/options that need to be added to the preprocessor and more tests need to be done. Please keep adding new features to the trunk (don't change anything to your SVN setup). If you want to test version 4.0.0, please check out branches/4.0 in a separate working directory, then commit only bug corrections from this working directory. Best, Michel -------- Original Message -------- Subject: Re: SVN Date: Tue, 08 Jul 2008 17:40:56 +0200 From: Sébastien Villemot <sebastien.villemot(a)ens.fr> To: stephane.adjemian(a)ens.fr CC: Michel Juillard <michel.juillard(a)ens.fr>, George Perendia <artilogica(a)btconnect.com> References: <487370EC.4010500(a)ens.fr> <200807081638.05166.stephane.adjemian(a)ens.fr> <20080708165802.17365ydzqi1sg8sg(a)www.zelya.net> <200807081712.03561.stephane.adjemian(a)ens.fr> Here is the timeline: * we finish Dynare 4.0.0 in the trunk (dynare_v4/) * when we're done, we copy dynare_v4/ to branches/4.0 and tags/4.0.0 * we fix bugs in branches/4.0, creating new releases when we feel the need (creating a release involves copying branches/4.0 to tags/4.0.x where x is the minor release number) * meanwhile, we add new features or restructure the code in the trunk (dynare_v4/). Also note that many bug fixes done on the 4.0 branch will also apply to the trunk * when we think we can make a new major release (for example when we've completed September 30 goals), we release 4.1.0 by copying dynare_v4/ to branches/4.1 and tags/4.1.0 * we continue the parallel development of the trunk, the 4.1 branch, and maybe the 4.0 branch (or we can decide to drop the 4.0 branch at that point) Sébastien "Stéphane Adjemian" <stephane.adjemian(a)ens.fr> a écrit : > So we use branches/4.0 to fix the bugs and use dynare_v4 to develop > dynare. Am > I right ? But what is the purpose of branches/4.1 (before the release of > dynare 4.1.0) ? > > Stéphane. > > > Le Tuesday 08 July 2008 16:58:02, vous avez écrit : >> Under our current tree layout: >> * dynare_v4/ is the trunk >> * branches/ is for branches (for example branches/4.0/ or branches/4.1/) >> * tags/ is for releases (tags/4.0.0, tags/4.0.1, ...) >> >> Sébastien >> >> "Stéphane Adjemian" <stephane.adjemian(a)ens.fr> a écrit : >> > I also agree with Sébastien suggestion (if the trunk is the current >> > dynare_v4 directory under svn). >> > >> > Stéphane. >> > >> > Le Tuesday 08 July 2008 16:29:36 G. Perendia, vous avez écrit : >> >> I support Sebastian's suggestion which, as he pointed out, is in line >> >> with the SW industry standard practices. We may also have the >> >> "traditional" "alpha" and "beta" releases (but that is just a matter of >> >> naming the X.0.0 and X.0.1 releases respectively). >> >> >> >> Best regards >> >> >> >> George >> >> Mob. +44(0)7951415480 >> >> artilogica(a)btconnect.com >> >> >> >> ----- Original Message ----- >> >> From: "Sébastien Villemot" <sebastien.villemot(a)ens.fr> >> >> To: "Michel Juillard" <michel.juillard(a)ens.fr> >> >> Cc: "George Perendia" <artilogica(a)btconnect.com>; "StéphaneAdjemian" >> >> <stephane.adjemian(a)gmail.com> >> >> Sent: Tuesday, July 08, 2008 3:12 PM >> >> Subject: Re: SVN >> >> >> >> >> >> Concerning Dynare release schedule, one option would be the following: >> >> >> >> * finish Dynare 4.0, and release version 4.0.0 >> >> * create a 4.0 branch in the SVN, used only for bugfixes; in other >> >> words, versions 4.0.1, 4.0.2... would only incoporate bug fixes since >> >> 4.0.0, but no new feature nor code restructuring >> >> * add new features or code restructurings in the trunk; the trunk will >> >> later become the 4.1 branch (maybe for the September 30 deadline) >> >> * and so on at each major release/deadline... >> >> >> >> This model of development is used in many softwares. Its main interest >> >> is that it produces very stable software at the end of a development >> >> branch, since no new feature can destabilize the code along a branch. >> >> On the other hand, it involves some overhead related to the >> >> simultaneous management of a branch and a trunk (a bugfix needs often >> >> to be applied to both). >> >> >> >> Do you think it is a sensible development model for Dynare? >> >> >> >> Best >> >> >> >> Sébastien >> >> >> >> "Michel Juillard" <michel.juillard(a)ens.fr> a écrit : >> >> > I have added 'george' as username authorized to commit on the SVN >> >> > archive. The password is as mentioned at the Summer School. >> >> > George will also get the email messages after each commit. >> >> > >> >> > Stephane, I checked that these messages go for you at >> >> > stephane.adjemian(a)ens.fr. Is it what you want? >> >> > >> >> > best >> >> > >> >> > Michel > > >

2 2

options_.replic
by Stéphane Adjemian 04 Sep '08

04 Sep '08

Hi all, We have a problem with options_.replic which is used for three different purposes (number of prior simulations, irfs with a second order approximation and repeated stochastic simulations). In global_initialization.m this variable is defined twice! We have: * options_.replic =1 for prior simulations (?) , and after * options_.replic = 50 for irfs with a second order approximation ... So that the default value is 50. When the user asks for a stochastic simulation with a second order approximation, dynare simulates 50 times the model and displays the results associated to the last simulated time series. Shouldn't we create new fields in options_ ? If so, how should I call them (for the prior sampler and for the repeated stochastic simulations) ? Something will have to be fixed also in the preprocessor. Best, Stéphane. Stéphane Adjemian Université du Maine, GAINS & CEPREMAP.

2 1

AIM for Dynare copyrights
by Sébastien Villemot 02 Sep '08

02 Sep '08

Hi George, I intend to give proper license headers to all files under the AIM/ subdirectory (since Gary has said that their are in the public domain). By comparing to Gary's original files, I noticed that 3 files of the Dynare distribution are not in the ZIP file downloadable from the Fed's website: * SPEigQZ.m * SPQZ.m * SPSparse.m The first two seem to be never used in the code, while the third one is simply a call to sparse() and therefore could easily be removed. Is there any good reason to keep these files, and, if yes, who owns the copyright? Best Sébastien

2 1

Re: [DynareDev] varexo problem
by Sébastien Villemot 24 Aug '08

24 Aug '08

Salut Michel, Je comprends maintenant mieux le problème avec varexo. La classe VariableTable maintient la liste des variables dynamiques, c'est à dire soit des triplets (symbol_type, symbol_id, lead/lag) pour tous les types de variables (endo, exo, exodet) Elle attribue un numéro à ce triplet lors du parsing de la variable dans le modèle (numéro noté var_id), attribué dans l'ordre d'apparition lors du parsing. Ce numéro est utilisé en interne dans ModelTree pour faire les dérivations. Après avoir parsé et fait les dérivations, on appelle la méthode VariableTable::sort() qui va classer ces triplets par ordre lexicographique et donner les nouveaux numéros associés, qu'on appellera des sort_id (la méthode n'a en fait pas besoin de faire le tri, car la structure C++ qui stocke les triplets a déjà fait ce tri, donc il suffit juste de lire les triplets dans l'ordre dans lequel ils sont stockés). C'est le sort_id (qu'on récupère avec VariableTable::getSortID()) qui est utilisé pour déterminer la colonne du jacobien; la matrice d'incidence donne le mapping entre ce sort_id et le couple (symbol_id, lead/lag), mais *uniquement pour les endogènes*. Si je comprends bien, le code Matlab suppose qu'après les colonnes des endogènes, apparaissent les colonnes des exogènes puis des exogènes déterministes, en supposant que chaque symbole occupe exactement une colonne, et dans l'ordre de déclaration de varexo ou varexo_det Dans l'état actuel du code, le préprocesseur ne donnera pas ce qu'attend le code Matlab dans les deux situations suivantes: * si une variable exogène n'est pas déclarée, comme dans ton exemple; le code Matlab attend une colonne qu'il n'utilisera jamais (ou peut-être qu'il l'attend remplie de zéros), tandis que le préprocesseur va omettre cette colonne * si une variable exogène est utilisée avec un lead/lag, ce qui n'est pas interdit (cela va juste faire un warning); dans ce cas le préprocesseur va au contraire ajouter une colonne non désirée On peut facilement résoudre le premier problème avec la solution que tu as proposée, mais je la mettrais plutôt dans VariableTable::getSortID(). Je peux m'en occuper. Pour le cas d'une exogène avec lead/lag, la solution me semble moins évidente, car je comprends mal comment on doit rendre les dérivées dans ce cas. Amitiés Sébastien Le samedi 23 août 2008 à 19:28 +0200, Michel Juillard a écrit : > Salut Sébastien, > > dans le répertoire juillard4, il y a deux fichiers *.mod, > model_financial.mod et test1.mod. La seule différence est l'ordre des > variables dans l'instruction VAREXO. > > Dans ce modèle, il y a 125 variables dynamiques (en distinguant selon > le lag). Le Jacobien par rapport aux variables exogènes commence dans > la colonne 126. > La variable exogène gdx est en 3e position dans model_financial.mod et > en 17e position dans test1.mod. > gdx apparaît dans la 55e équation et la dérivée est placée correctement > dans g1(55,128) dans model_financial_dynamic.m > Dans test1_dynamic.m, elle apparaît de manière erronnée dans g1(55,140) > et non g1(55,142) > > Le problème apparaît lorsqu'il y a des variables exogènes déclarées, > mais qui ne sont pas utilisées dans le modèle. Dans ce cas elles ne sont > pas dans variable_table. > > Je t'écris ou t'appelle plus tard, car je crois que j'ai une solution. > > amitiés > > Michel

2 3

[Fwd: disque dur de hal]
by Michel Juillard 22 Aug '08

22 Aug '08

Thanks Stéphane for warning us. We will buy a big disk for Hal. I guess that there is no need to pay Dell prices and that we can install it ourselves. What do you think? I want also to buy a new computing workstation like Hal but with two quad-processors that we will install at Chevaleret. All the best, Michel -------- Original Message -------- Subject: disque dur de hal Date: Wed, 6 Aug 2008 09:51:09 +0200 From: Stéphane Adjemian <stephane.adjemian(a)ens.fr> Reply-To: stephane.adjemian(a)ens.fr Organization: Université du Maine & CEPREMAP To: Michel Juillard <michel.juillard(a)ens.fr> CC: Sébastien Villemot <sebastien.villemot(a)ens.fr> Salut, Un petit mail pour vous signaler que je viens de planter un gros métropolis sur hal parce qu'il n'y a plus assez d'espace mémoire pour enregistrer les fichiers mh. Le disque qui contient le /home (27G) est plein. Il est vrai que je consomme beaucoup d'espace sur le disque (là j'estimais récursivement un modèle sur les US pour comparer les prévisions avec une rêgle de Taylor et les prévisions avec une fonction de perte pour la banque centrale (politique monétaire optimale)). je pense donc qu'il faudrait prévoir d'acheter un autre disque dur à la rentrée. En attendant je vais virer mes fichiers pour que vous puissiez utiliser hal (qui est actuellement inutilisable) à+ Stéphane.

3 2

Re: [DynareDev] license
by Michel Juillard 19 Aug '08

19 Aug '08

Thanks Gary! Ben, is there some version of your code available somewhere on the web? Best Michel Gary.Anderson(a)frb.gov wrote: > michel ben, > > i should put the two of you in touch. ben has done a significant amount of > work improving the "C" version of sparseAim. > his goal, which he has nearly attained, is to provide a public domain/free > ware ansii "C" version. > i suspect that the code will be more durable and faster than any of the > matlab implementations and much easier to port than the existing "C" > versions. > > i hope this is timely and helpful. > > gary > > > >

1 0

Re: [DynareDev] [Fwd: Updated: mathomatic-14.1.4-1]
by Sébastien Villemot 17 Aug '08

17 Aug '08

Salut Michel, J'ai trouvé cette page de Wikipedia qui liste les différents logiciels de calcul symbolique: http://en.wikipedia.org/wiki/Comparison_of_computer_algebra_systems Il faudrait probablement prendre le temps de regarder tous ceux qui sont libres, pour voir ce qu'on pourrait réutiliser le plus aisément. Je me pencherai sur cette question à la rentrée. Amitiés Sébastien Le samedi 16 août 2008 à 10:01 +0200, Michel Juillard a écrit : > Salut Sébastien, > > est-ce que tu penses qu'il serait possible d'intégrer certaines > fonctions de mathomatic au pre-processor, pour simplifier les exressions? > > amitiés > > Michel > > -------- Original Message -------- > Subject: Updated: mathomatic-14.1.4-1 > Date: Sat, 16 Aug 2008 08:13:26 +0200 > From: Reini Urban <rurban(a)x-ray.at> > Reply-To: The Cygwin Mailing List <cygwin(a)cygwin.com> > To: cygwin-announce(a)cygwin.com > > > > Major algorithmical bugfixes in simplify. > > The calculate command now reports the number of solutions. > Made some big improvements and bug fixes to simplification, if only > someone had reported bugs earlier! > A major, long-standing simplify command malfunction has been fixed, but > it runs a little slower. The simplify command is now logically correct > and appears to work nicely now, please tell me if not. > > See http://mathomatic.orgserve.de/changes.txt > > Cygwin build changes: > * none > > About: > Mathomatic is a highly portable, general purpose symbolic math program > that can solve, simplify, combine, differentiate, integrate, and compare > algebraic equations. It can do standard, complex number, and polynomial > arithmetic. It is extremely easy to use and has pretty colored, easily > readable display of equations. > > See http://www.mathomatic.org/ > ======================================================================== > > To update your installation, click on the "Install Cygwin now" link on > the http://cygwin.com/ web page. This downloads setup.exe to your > system. Then, run setup and answer all of the questions. > > *** CYGWIN-ANNOUNCE UNSUBSCRIBE INFO *** > > If you want to unsubscribe from the cygwin-announce mailing list, look > at the "List-Unsubscribe: " tag in the email header of this message. > Send email to the address specified there. It will be in the format: > > cygwin-announce-unsubscribe-you=yourdomain.com(a)cygwin.com > > If you need more information on unsubscribing, start reading here: > > http://sources.redhat.com/lists.html#unsubscribe-simple > > Please read *all* of the information on unsubscribing that is available > starting at this URL. >

2 1

[Fwd: Re: [Fwd: Re: AIM in Dynare +PS]]
by Michel Juillard 16 Aug '08

16 Aug '08

Hi George, thank you for looking into Dynare++ and the resulting analysis. Given our objectives, I suggest that we go for the highest level of integration and use your solution 2.2, keeping in mind the following: 1) I don't understand fully what Ondra is doing for approximation around the stochastic steady state and I don't want to offer it yet in Dynare Matlab. So, for the time beeing we use only steps=0 2) After a first call to Approximation::walkStochSteady(), we need, for estimation, to be able to call repeatedly the first order approximation for different values of the parameters and, possibly different values of the deterministic steady state. As all the arrays (matrices, tensors ....) will have the same size in each interation, we need to factorize memory allocation is this case and only replace the values in existing arrays. This may require far reaching changes in the existing code. 3) For the deterministic steady state, we need to implement three variants: a) the exact steady state values are supplied by the user (then the steady state isn't affected by estimated parameters) b) the steady state is computed by a call to the nonlinear solver in Dynare++ (or we may substitute the nonlinear solver of GSL or yet another one) c) the steady state is obtained from a function supplied by the user. This function can be a *.m file (as currently the *_steadystate.m file) or a DLL when we integrate recursive steadystate computation in the *.mod file. 4) IMHO, the only change necessary to the DLLs created by the preprocessor with the USE_DLL option is to export as well the functions 'Dynamic', and respectively 'Static', and not only the function Mexfunction(). This requires an explicit compilation of the DLL rather than using the 'mex' shell function and a modification of the mex.def file (currently created and deleted by the mex shell function). 5) Currently, dynare++ uses the value 1.0 to discriminate between statble and unstable roots. This creates a problem for models with unit roots (models that are stable around a stochastic growth path). Because of small inaccuracies in the computation of (repeated) unit roots, we need to use a value slightly above one (controlled by option qz_criterium in Dynare Matlab). 6) This is a question. Could you please look whether and how dynare++ eliminates static variables when computing the first order approximation? I changed the way this is done in Dynare Matlab after having worked on dynare++. Currently, I use a QR decomposition of the part of the Jacobian relative to current variables. This only requires that the Jacobian relative to static variables be full column rank. This is a good occasion for me to write the algebra of this part. All the best, Michel -------- Original Message -------- Subject: Re: [DynareDev] [Fwd: Re: AIM in Dynare +PS] Date: Fri, 15 Aug 2008 12:10:58 +0100 From: G. Perendia <george(a)perendia.orangehome.co.uk> To: <michel.juillard(a)ens.fr> References: <48966576.9060700(a)ens.fr> Dear Michel I enclose zipped initial analysis notes (FirstOrderAnalysis.rtf) and UML documentation for the Kord package that may be of some help too. For a better understanding, I included some of Ondra's original, very useful explanations though you may be already familiar with them. I think that simple no 1.1 FirstOrder solution may be ready in a few weeks. This solution is equivalent to the option (b) discussed earlier: (b) preprocessor dynamically create a truly dynamically loadable module <mod>_dynamic.DLL callable from Dynare++.DLL in addition to the current <mod>_dynamic.m,, .... Best regards George ----- Original Message ----- From: "Michel Juillard" <michel.juillard(a)ens.fr> To: "List for Dynare developers" <dev(a)dynare.org> Sent: Monday, August 04, 2008 3:12 AM Subject: [DynareDev] [Fwd: Re: AIM in Dynare +PS] Dear Georges, thanks for looking into that. My responses are in upper case. 1) Do I understand well that we may be providing an optional alternative, or, poss. a full replacement for the current dr1.m? I.e., the current mjdgges strand solution for 1st order (i.e the 1st part) of dr1.m may optionally or fully be replaced with the new Dynare++ based 1st order solution DLL. Is this aimed to bring a benefit of improved performance and a better starting point for the k-order solution when if needed? YES, WE WANT TO OFFER THIS OPTION AND USE IT FOR LIKELIHOOD COMPUTATIONS As I understand so far, we can extend the current Dynare functionality substantially mainly if we extend: a) the current 2nd order with 2+ (i.e. k>1) order solution, thus, the 2nd part of dr1 by replacing it with the k-order solution DLL based on Dynare++, (either as optional alternative or as a full replacement) and, poss. AS AN OPTION b) replace/complement the Ramsey part of DR1 by similar action based on Dynare++ code. Will that change then also include replacing Ramsey planner part of dr1 with the Dynare++ Ramsey? NO, WE DON'T BRING IN DYNARE++ RAMSEY STUFF 2) Also, for 1st and k-order I think I see some broad-lines of possible integration solutions (but I could be wrong): A) , one is to run new Dynare++.DLL (Dynare++ without parser and simulation) callable in a flexible manner from Matlab (and Matlab Dynare), and its inputs may be the results of <mod>_dynamic.m run by a wrapping Matlab (.m) shell routine (e.g. a modified dr1.m) which calls both <mod>_dynamic.m, (at least to read the model,) and then Dynare++.DLL , or, WE DON'T WANT TO USE <MOD>_DYNAMIC.M. MY PREFERRED SOLUTION IS YOUR POINT b) B) Preprocessor needs to be changed and, either: a) new Dynare++.DLL becomes attached straight to the current Dynare preprocessor which now creates <mod>_dynamic.m in the first place, or, I DON'T THINK WE WANT THAT. WE LOSE MATLAB MODULAR APPROACH b) preprocessor dynamically create a truly dynamically loadable module <mod>_dynamic.DLL callable from Dynare++.DLL in addition to the current <mod>_dynamic.m, or, THIS EXISTS ALREADY AND IT IS WHAT WE WANT TO USE. IT IS TRIGGERED BY OPTION USE_DLL IN model KEYWORD. c) preprocessor dynamically create a modified (new) <mod>_dynamic.m which has a built-in call to dynamically loadable module Dynare++.DLL. The modified (new) <mod>_dynamic.m will be callable from Matlab Dynare alike the current <mod>_dynamic.m (e.g. from modified dr1.m) but with extra, k-order Taylor params being returned. THIS IS TOO COMPLICATED AND b) IS SIMPLER Please let me know what you think. PS, FYI, As far as _f and _ff outputs are concerned: "....two Matlab scripts, which are useful when calculating the deterministic steady state outside Dynare++. The scripts are created by Dynare++ as soon as an input file is parsed, that is before any calculations. The first Matlab script having a name modname f.m ...calculates a residual of the static system. ...The second script having a name modname ff.m calculates a (Jacobian) matrix: df(y,y,y,0)/dy ....". THANKS. ALL THE BEST MICHEL Best regards George ----- Original Message ----- From: "Michel Juillard" <michel.juillard(a)ens.fr> To: "G. Perendia" <artilogica(a)btconnect.com>; "Stéphane Adjemian" <stephane.adjemian(a)gmail.com>; "Sébastien Villemot" <sebastien.villemot(a)ens.fr>; "Ferhat MIHOUBI" <fmihoubi(a)univ-evry.fr> Sent: Saturday, August 02, 2008 1:45 PM Subject: Re: AIM in Dynare +PS > Hi George > > > 1) Without inspecting the code, is the recursive MEX Sylvester > > solution in Dynare 4 by Ondra the same as/or similar to / the one used > > in Dynare++ and, also, is it based on (and explained in) his 2005 > > paper "Solving SDGE..." on a solving k-order with a new recursive > > Sylvester solution method ? > yes to both questions. > > > 2) Is then your and Ondra's (05?) paper: "Solving SDGE Models: > > Approximation About The Stochastic Steady State" used for Dynare++ > > based on the same solution? - I could not find the article, only REPEC > > abstract. > > > Ondra later convinced me that my approach to approximation around > another point that the deterministic steady state was wrong. So we never > finish writing the paper. However, he implemented an iterative method > for the same thing, but, to my knowledge, he never wrote a description > of the algorithm. This is something I hope to be able to go back to with > him at some point. > > For the time being, I think that we should limit ourselves to provide > k-order approximation around the deterministic steady state in Matlab. > > I'm attaching unpublished notes concerning the k-order approximation. I > don't know how if it is very explicit in the text, but the order in > which we compute the derivatives in part 4 is important. I believe that > in this version it is in the right order. > > In this paper, the first order approximation is presented with tensor > notations, but it is exactly the same approach that I'm currently > presenting in slides with matrix notation. > > All the best, > > Michel > > > > > > > Best regards > > > > George > _______________________________________________ Dev mailing list Dev(a)dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev

1 0

Probability distributions
by Sébastien Villemot 12 Aug '08

12 Aug '08

Hi all, I have started the work of rewriting distribution-related functions, beginning by the easiest one (the normal). Concerning the others, a few points: 1) I discovered that prior_draw.m contains at the bottom a function which draws samples from a gamma. This would solve our copyright problem with gamm_rnd.m, if you confirm to me that you wrote it. 2) In some files, there is code for an inverse gamma of type 2 (represented by integer 6 in Dynare). This distribution is not handled by the preprocessor at this time. Should I add it ? Note that rndprior.m doesn't handle it, while prior_draw.m and draw_prior_density.m do. 3) Just for your information, note that the wikipedia page on the inverse gamma (http://en.wikipedia.org/wiki/Inverse-gamma_distribution) documents a distribution similar to your type-2-inv-gamma (up to a parameter rescaling). I could not find any page on your type-1-inv-gamma (represented by integer 4 in Dynare). This took me some time to understand, until I looked at the headers of lpdfig1.m and lpdfig2.m. Therefore it seems that the definition of what is inverse gamma is not really standardized. 4) There are some factorization opportunities which I would like to take advantage of: * in prior_draw.m: currently does not take advantage of beta_rnd.m (while rndprior.m does) * in draw_prior_density.m: we could use the new PDF functions instead of rewriting the PDF in an inline fashion * factorize code for the two inverse gammas Best Sébastien

4 11

Re: [DynareDev] license
by Michel Juillard 05 Aug '08

05 Aug '08

Thanks Gary, this is good, but we should add "author: Gary Anderson" before this text. All the best, Michel Gary.Anderson(a)frb.gov wrote: > michel, > > i had just copied some language used by someone in the same position. i > remember thinking the wording should be different too. > how about: > > > >> % Note: This program is in the public domain and may be used freely. >> % however the authors would appreaciate acknowledgement of the source by >> > citation of any > >> % of the following papers >> >> %Anderson, G and Moore, G >> %"A Linear Algebraic Procedure for Solving Linear Perfect Foresight >> Models." >> %Economics Letters, 17, 1985. >> >> % Anderson, G. >> %"Solving Linear Rational Expectations Models: A Horse Race" >> %Computational Economics, 2008 vol 31 issue 2 pages 95-113 >> >> %Anderson, G. >> %"A Reliable and Computationally Efficient Algorithm for Imposing the >> Saddle Point Property in Dynamic Models" >> %Journal of Economic Dynamics and Control, Forthcoming >> >> thanks, >> gary >> > > gary > > > > Michel Juillard > <michel.juillard@ > ens.fr> To > Gary.Anderson(a)frb.gov, Sébastien > 08/02/2008 09:04 Villemot > AM <sebastien.villemot(a)ens.fr> > cc > > Subject > Re: license > > > > > > > > > > > Hi GAry, > > Sébastien who is more attentive than me, remarked that it is > contradictory to claim a Copyright and put it in the public domain. > > We would suggest something like > > This program is in the public domain. Author: Gary Anderson > > All the best, > > Michel > > > > Gary.Anderson(a)frb.gov wrote: > >> michel, >> since i'm a US federal employee, all the code is actually freely >> > available > >> and in the public domain. >> but, if it's not to much trouble, it would be nice to have an >> > attribution > >> along the following lines: >> >> % (c) Gary S Anderson 2005 >> % >> % Note: This program (and associated subroutines) may be used freely >> % subject to acknowledgement of the source and citation of any >> % of the following papers >> >> %Anderson, G and Moore, G >> %"A Linear Algebraic Procedure for Solving Linear Perfect Foresight >> Models." >> %Economics Letters, 17, 1985. >> >> % Anderson, G. >> %"Solving Linear Rational Expectations Models: A Horse Race" >> %Computational Economics, 2008 vol 31 issue 2 pages 95-113 >> >> %Anderson, G. >> %"A Reliable and Computationally Efficient Algorithm for Imposing the >> Saddle Point Property in Dynamic Models" >> %Journal of Economic Dynamics and Control, Forthcoming >> >> thanks, >> gary >> >> >> >> > > >> Michel Juillard >> > > >> <michel.juillard@ >> > > >> ens.fr> >> > To > >> Gary.Anderson(a)frb.gov, Sébastien >> > > >> 07/30/2008 11:31 Villemot >> > > >> AM <sebastien.villemot(a)ens.fr>, >> > > >> Stéphane Adjemian >> > > >> <stephane.adjemian(a)gmail.com>, >> > > >> George Perendia >> > > >> <artilogica(a)btconnect.com> >> > > > cc > > > > Subject > >> license >> > > > > > > > > > > > > > > >> >> >> Dear Gary, >> >> George has finished the implementation of Aim in Dynare. It is available >> under SVN. >> As we are in the process to c leaning Dynare code, we want to add proper >> Copyrights and license agreement. >> As we are distributing Dynare under the GPL v3, we would like to add to >> your files >> Copyright Gary Andersen (2005) and distribute them as well under GPLv3. >> If your prefer you can choose to have them distributed under a less >> restrictive license such as BSD. >> >> All the best, >> >> Michel >> >> >> >> >> >> > > > > >

1 0

Kirikou
by Sébastien Villemot 04 Aug '08

04 Aug '08

Hi all, I set up a wiki page to describe the main points of the configuration of Kirikou: http://www.cepremap.cnrs.fr/DynareWiki/KirikouConfiguration We should expand this page as we migrate stuff from pythie to kirikou. Michel: I set up a wiki on http://www.monfispol.eu/MonFisPolWiki (I chose to use a more explicit name than InternalWiki, because it could become confusing when we migrate all our wikis to kirikou) For the moment everybody can write on this wiki without any restriction; please fix /etc/moin/monfispolwiki.py when you have decided who can write on this wiki Stéphane: * I moved the Ubuntu repository to /srv/d_kirikou/www.dynare.org (see the top of the wiki page for the reason why) * maybe you could write a few lines about the configuration of your private SVN Best Sébastien

3 4

[Fwd: Re: AIM in Dynare +PS]
by Michel Juillard 04 Aug '08

04 Aug '08

Dear Georges, thanks for looking into that. My responses are in upper case. 1) Do I understand well that we may be providing an optional alternative, or, poss. a full replacement for the current dr1.m? I.e., the current mjdgges strand solution for 1st order (i.e the 1st part) of dr1.m may optionally or fully be replaced with the new Dynare++ based 1st order solution DLL. Is this aimed to bring a benefit of improved performance and a better starting point for the k-order solution when if needed? YES, WE WANT TO OFFER THIS OPTION AND USE IT FOR LIKELIHOOD COMPUTATIONS As I understand so far, we can extend the current Dynare functionality substantially mainly if we extend: a) the current 2nd order with 2+ (i.e. k>1) order solution, thus, the 2nd part of dr1 by replacing it with the k-order solution DLL based on Dynare++, (either as optional alternative or as a full replacement) and, poss. AS AN OPTION b) replace/complement the Ramsey part of DR1 by similar action based on Dynare++ code. Will that change then also include replacing Ramsey planner part of dr1 with the Dynare++ Ramsey? NO, WE DON'T BRING IN DYNARE++ RAMSEY STUFF 2) Also, for 1st and k-order I think I see some broad-lines of possible integration solutions (but I could be wrong): A) , one is to run new Dynare++.DLL (Dynare++ without parser and simulation) callable in a flexible manner from Matlab (and Matlab Dynare), and its inputs may be the results of <mod>_dynamic.m run by a wrapping Matlab (.m) shell routine (e.g. a modified dr1.m) which calls both <mod>_dynamic.m, (at least to read the model,) and then Dynare++.DLL , or, WE DON'T WANT TO USE <MOD>_DYNAMIC.M. MY PREFERRED SOLUTION IS YOUR POINT b) B) Preprocessor needs to be changed and, either: a) new Dynare++.DLL becomes attached straight to the current Dynare preprocessor which now creates <mod>_dynamic.m in the first place, or, I DON'T THINK WE WANT THAT. WE LOSE MATLAB MODULAR APPROACH b) preprocessor dynamically create a truly dynamically loadable module <mod>_dynamic.DLL callable from Dynare++.DLL in addition to the current <mod>_dynamic.m, or, THIS EXISTS ALREADY AND IT IS WHAT WE WANT TO USE. IT IS TRIGGERED BY OPTION USE_DLL IN model KEYWORD. c) preprocessor dynamically create a modified (new) <mod>_dynamic.m which has a built-in call to dynamically loadable module Dynare++.DLL. The modified (new) <mod>_dynamic.m will be callable from Matlab Dynare alike the current <mod>_dynamic.m (e.g. from modified dr1.m) but with extra, k-order Taylor params being returned. THIS IS TOO COMPLICATED AND b) IS SIMPLER Please let me know what you think. PS, FYI, As far as _f and _ff outputs are concerned: "....two Matlab scripts, which are useful when calculating the deterministic steady state outside Dynare++. The scripts are created by Dynare++ as soon as an input file is parsed, that is before any calculations. The first Matlab script having a name modname f.m ...calculates a residual of the static system. ...The second script having a name modname ff.m calculates a (Jacobian) matrix: df(y,y,y,0)/dy ....". THANKS. ALL THE BEST MICHEL Best regards George ----- Original Message ----- From: "Michel Juillard" <michel.juillard(a)ens.fr> To: "G. Perendia" <artilogica(a)btconnect.com>; "Stéphane Adjemian" <stephane.adjemian(a)gmail.com>; "Sébastien Villemot" <sebastien.villemot(a)ens.fr>; "Ferhat MIHOUBI" <fmihoubi(a)univ-evry.fr> Sent: Saturday, August 02, 2008 1:45 PM Subject: Re: AIM in Dynare +PS > Hi George > > > 1) Without inspecting the code, is the recursive MEX Sylvester > > solution in Dynare 4 by Ondra the same as/or similar to / the one used > > in Dynare++ and, also, is it based on (and explained in) his 2005 > > paper "Solving SDGE..." on a solving k-order with a new recursive > > Sylvester solution method ? > yes to both questions. > > > 2) Is then your and Ondra's (05?) paper: "Solving SDGE Models: > > Approximation About The Stochastic Steady State" used for Dynare++ > > based on the same solution? - I could not find the article, only REPEC > > abstract. > > > Ondra later convinced me that my approach to approximation around > another point that the deterministic steady state was wrong. So we never > finish writing the paper. However, he implemented an iterative method > for the same thing, but, to my knowledge, he never wrote a description > of the algorithm. This is something I hope to be able to go back to with > him at some point. > > For the time being, I think that we should limit ourselves to provide > k-order approximation around the deterministic steady state in Matlab. > > I'm attaching unpublished notes concerning the k-order approximation. I > don't know how if it is very explicit in the text, but the order in > which we compute the derivatives in part 4 is important. I believe that > in this version it is in the right order. > > In this paper, the first order approximation is presented with tensor > notations, but it is exactly the same approach that I'm currently > presenting in slides with matrix notation. > > All the best, > > Michel > > > > > > > Best regards > > > > George >

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

Dev