Re: [DynareDev] 2nd and 3rd model derivatives + k-order
Le jeudi 25 juin 2009 à 12:59 +0100, G. Perendia a écrit :
However, I do not quite understand what do you mean with USE_DLL is roken - is it because sparse matrix is now returned and the k-order is broken?
USE_DLL is broken in the sense that the matrix returned by the MEX file at the place of g2 is no longer square: it has only 3 columns describing the hessian in a sparse way. So dr1.m is completely confused with USE_DLL option at order 2: it expects a square hessian and it gets a 3-columns matrix.
Can you remind me how is _dynamic MEX/dll file, used by k-order.dll, then produced - is USE_DLL going to stay as option e.g. for 2nd order or is it going to be discontinued?
We are going to drop USE_DLL option. We will then add an option like "korderpert" to stoch_simul, which will trigger the call to the k-order DLL. And ultimately we will drop the generation of MEX files containing the model, to replace it by a simulated bytecode. The interface to k-order DLL will then need to be changed.
Is jacobia_ still a full matrix?
Yes, nothing changed for the Jacobian -- Sébastien Villemot
Dear Sebastien Thanks, Just to recap, both dr1.m and k-order-perturbation.dll are affected but only k-order-perturbation would need to be fixed to cope with this change since dr1.m will not be using USE_DLL mex option anyway. Best regards George Perendia ----- Original Message ----- From: "Sébastien Villemot" <sebastien.villemot@ens.fr> To: <dev@dynare.org> Sent: Thursday, June 25, 2009 1:09 PM Subject: Re: [DynareDev] 2nd and 3rd model derivatives + k-order Le jeudi 25 juin 2009 à 12:59 +0100, G. Perendia a écrit :
However, I do not quite understand what do you mean with USE_DLL is roken - is it because sparse matrix is now returned and the k-order is broken?
USE_DLL is broken in the sense that the matrix returned by the MEX file at the place of g2 is no longer square: it has only 3 columns describing the hessian in a sparse way. So dr1.m is completely confused with USE_DLL option at order 2: it expects a square hessian and it gets a 3-columns matrix.
Can you remind me how is _dynamic MEX/dll file, used by k-order.dll, then produced - is USE_DLL going to stay as option e.g. for 2nd order or is it going to be discontinued?
We are going to drop USE_DLL option. We will then add an option like "korderpert" to stoch_simul, which will trigger the call to the k-order DLL. And ultimately we will drop the generation of MEX files containing the model, to replace it by a simulated bytecode. The interface to k-order DLL will then need to be changed.
Is jacobia_ still a full matrix?
Yes, nothing changed for the Jacobian -- Sébastien Villemot _______________________________________________ Dev mailing list Dev@dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev
Le jeudi 25 juin 2009 à 13:41 +0100, G. Perendia a écrit :
Just to recap, both dr1.m and k-order-perturbation.dll are affected but only k-order-perturbation would need to be fixed to cope with this change since dr1.m will not be using USE_DLL mex option anyway.
Yes, indeed. -- Sébastien Villemot
Dear Sebastien If USE_DLL is going to be discontinued why is MEX/DLL now going to return sparse matrix exclusively when its only future use is for k-ord-perturbation which requires full matrix. Can we than have PERTURB_DLL option now and return full hessian using the existing (old) mex/dll dynamic model (but renamed e.g. <model>_kordpert.mexXX/dll) when that option is used rather than fixing k-ordpert rather unnecessarily and poss. just temporary since we may revert to the full hessian returned form mex/dll once USE_DLL is abandoned. I think to remember we initially agreed this course of action anyway. In your email from 9th April 09 you mention: ..."For your own purposes, I have understood that you need a full hessian (instead of a sparse one). Can you confirm this? If it is the case, then I shall implement the "pertub_dll" option (as suggested by Michel), so that the preprocessor gives you a full hessian in that case." And, there are actually quite few issues to be resolved for sparse matrices being returned from dll. For a start, one problem with fixing k-ordpert is finding out what size matrix should be passed to dll to get hessian in: i.e. it may be as much as 3 times bigger than the flat hessian - which can be rather large!! Best regards George ----- Original Message ----- From: "Sébastien Villemot" <sebastien.villemot@ens.fr> To: "List for Dynare developers" <dev@dynare.org> Sent: Thursday, June 25, 2009 1:44 PM Subject: Re: [DynareDev] 2nd and 3rd model derivatives + k-order Le jeudi 25 juin 2009 à 13:41 +0100, G. Perendia a écrit :
Just to recap, both dr1.m and k-order-perturbation.dll are affected but only k-order-perturbation would need to be fixed to cope with this change since dr1.m will not be using USE_DLL mex option anyway.
Yes, indeed. -- Sébastien Villemot _______________________________________________ Dev mailing list Dev@dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev
Hi George, it is inefficient to pass full hessian from *_dynamic.m or *_dynamic.dll In k_ord_dynare, one uses only non-zero elements to populate the tensors. The details of implementation will change again in the future, but passing only non-zero elements is a step in the right direction. All the best, Michel G. Perendia wrote:
Dear Sebastien
If USE_DLL is going to be discontinued why is MEX/DLL now going to return sparse matrix exclusively when its only future use is for k-ord-perturbation which requires full matrix. Can we than have PERTURB_DLL option now and return full hessian using the existing (old) mex/dll dynamic model (but renamed e.g. <model>_kordpert.mexXX/dll) when that option is used rather than fixing k-ordpert rather unnecessarily and poss. just temporary since we may revert to the full hessian returned form mex/dll once USE_DLL is abandoned.
I think to remember we initially agreed this course of action anyway. In your email from 9th April 09 you mention:
..."For your own purposes, I have understood that you need a full hessian (instead of a sparse one). Can you confirm this? If it is the case, then I shall implement the "pertub_dll" option (as suggested by Michel), so that the preprocessor gives you a full hessian in that case."
And, there are actually quite few issues to be resolved for sparse matrices being returned from dll. For a start, one problem with fixing k-ordpert is finding out what size matrix should be passed to dll to get hessian in: i.e. it may be as much as 3 times bigger than the flat hessian - which can be rather large!!
Best regards
George ----- Original Message ----- From: "Sébastien Villemot" <sebastien.villemot@ens.fr> To: "List for Dynare developers" <dev@dynare.org> Sent: Thursday, June 25, 2009 1:44 PM Subject: Re: [DynareDev] 2nd and 3rd model derivatives + k-order
Le jeudi 25 juin 2009 à 13:41 +0100, G. Perendia a écrit :
Just to recap, both dr1.m and k-order-perturbation.dll are affected but
only
k-order-perturbation would need to be fixed to cope with this change since dr1.m will not be using USE_DLL mex option anyway.
Yes, indeed.
Thanks Michel I am well aware of the efficiency of sparse matrices being used for Hessian in general. However, k-order-perturbation dll has problem in finding out what size matrix should be passed to dll to get the Hessian back in.. Not knowing the number of non-zero elements in advance can result in having to pass between two DLLs as much as 3 times larger matrix than the flat Hessian (for both data and indices) which can be rather large - and mostly empty!! Best regards George ----- Original Message ----- From: "Michel Juillard" <michel.juillard@ens.fr> To: "List for Dynare developers" <dev@dynare.org> Sent: Friday, June 26, 2009 7:11 AM Subject: Re: [DynareDev] 2nd and 3rd model derivatives + k-order Hi George, it is inefficient to pass full hessian from *_dynamic.m or *_dynamic.dll In k_ord_dynare, one uses only non-zero elements to populate the tensors. The details of implementation will change again in the future, but passing only non-zero elements is a step in the right direction. All the best, Michel G. Perendia wrote:
Dear Sebastien
If USE_DLL is going to be discontinued why is MEX/DLL now going to return sparse matrix exclusively when its only future use is for k-ord-perturbation which requires full matrix. Can we than have PERTURB_DLL option now and return full hessian using the existing (old) mex/dll dynamic model (but renamed e.g. <model>_kordpert.mexXX/dll) when that option is used rather than fixing k-ordpert rather unnecessarily and poss. just temporary since we may revert to the full hessian returned form mex/dll once USE_DLL is abandoned.
I think to remember we initially agreed this course of action anyway. In your email from 9th April 09 you mention:
..."For your own purposes, I have understood that you need a full hessian (instead of a sparse one). Can you confirm this? If it is the case, then I shall implement the "pertub_dll" option (as suggested by Michel), so that the preprocessor gives you a full hessian in that case."
And, there are actually quite few issues to be resolved for sparse matrices being returned from dll. For a start, one problem with fixing k-ordpert is finding out what size matrix should be passed to dll to get hessian in: i.e. it may be as much as 3 times bigger than the flat hessian - which can be rather large!!
Best regards
George ----- Original Message ----- From: "Sébastien Villemot" <sebastien.villemot@ens.fr> To: "List for Dynare developers" <dev@dynare.org> Sent: Thursday, June 25, 2009 1:44 PM Subject: Re: [DynareDev] 2nd and 3rd model derivatives + k-order
Le jeudi 25 juin 2009 à 13:41 +0100, G. Perendia a écrit :
Just to recap, both dr1.m and k-order-perturbation.dll are affected but
only
k-order-perturbation would need to be fixed to cope with this change since dr1.m will not be using USE_DLL mex option anyway.
Yes, indeed.
_______________________________________________ Dev mailing list Dev@dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev
Good point! So the preprocessor needs to write in the main *.m file the number of non-zero elements (probably in M_.) and then k_order_perturbation can make the correct space available. Best, Michel G. Perendia wrote:
Thanks Michel
I am well aware of the efficiency of sparse matrices being used for Hessian in general.
However, k-order-perturbation dll has problem in finding out what size matrix should be passed to dll to get the Hessian back in.. Not knowing the number of non-zero elements in advance can result in having to pass between two DLLs as much as 3 times larger matrix than the flat Hessian (for both data and indices) which can be rather large - and mostly empty!!
Best regards
George
----- Original Message ----- From: "Michel Juillard" <michel.juillard@ens.fr> To: "List for Dynare developers" <dev@dynare.org> Sent: Friday, June 26, 2009 7:11 AM Subject: Re: [DynareDev] 2nd and 3rd model derivatives + k-order
Hi George,
it is inefficient to pass full hessian from *_dynamic.m or *_dynamic.dll
In k_ord_dynare, one uses only non-zero elements to populate the tensors.
The details of implementation will change again in the future, but passing only non-zero elements is a step in the right direction.
All the best,
Michel
G. Perendia wrote:
Dear Sebastien
If USE_DLL is going to be discontinued why is MEX/DLL now going to return sparse matrix exclusively when its only future use is for
k-ord-perturbation
which requires full matrix. Can we than have PERTURB_DLL option now and return full hessian using the existing (old) mex/dll dynamic model (but renamed e.g. <model>_kordpert.mexXX/dll) when that option is used rather than fixing k-ordpert rather unnecessarily and poss. just temporary since
we
may revert to the full hessian returned form mex/dll once USE_DLL is abandoned.
I think to remember we initially agreed this course of action anyway. In your email from 9th April 09 you mention:
..."For your own purposes, I have understood that you need a full hessian (instead of a sparse one). Can you confirm this? If it is the case, then I shall implement the "pertub_dll" option (as suggested by Michel), so that the preprocessor gives you a full hessian in that case."
And, there are actually quite few issues to be resolved for sparse
matrices
being returned from dll. For a start, one problem with fixing k-ordpert
is
finding out what size matrix should be passed to dll to get hessian in:
i.e.
it may be as much as 3 times bigger than the flat hessian - which can be rather large!!
Best regards
George ----- Original Message ----- From: "Sébastien Villemot" <sebastien.villemot@ens.fr> To: "List for Dynare developers" <dev@dynare.org> Sent: Thursday, June 25, 2009 1:44 PM Subject: Re: [DynareDev] 2nd and 3rd model derivatives + k-order
Le jeudi 25 juin 2009 à 13:41 +0100, G. Perendia a écrit :
Just to recap, both dr1.m and k-order-perturbation.dll are affected but
only
k-order-perturbation would need to be fixed to cope with this change
since
dr1.m will not be using USE_DLL mex option anyway.
Yes, indeed.
_______________________________________________ Dev mailing list Dev@dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev
_______________________________________________ Dev mailing list Dev@dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev
Yes, writing it in M_ would help greatly. Best regards George ----- Original Message ----- From: "Michel Juillard" <michel.juillard@ens.fr> To: "List for Dynare developers" <dev@dynare.org> Sent: Friday, June 26, 2009 7:30 AM Subject: Re: [DynareDev] 2nd and 3rd model derivatives + k-order Good point! So the preprocessor needs to write in the main *.m file the number of non-zero elements (probably in M_.) and then k_order_perturbation can make the correct space available. Best, Michel G. Perendia wrote:
Thanks Michel
I am well aware of the efficiency of sparse matrices being used for Hessian in general.
However, k-order-perturbation dll has problem in finding out what size matrix should be passed to dll to get the Hessian back in.. Not knowing the number of non-zero elements in advance can result in having to pass between two DLLs as much as 3 times larger matrix than the flat Hessian (for both data and indices) which can be rather large - and mostly empty!!
Best regards
George
----- Original Message ----- From: "Michel Juillard" <michel.juillard@ens.fr> To: "List for Dynare developers" <dev@dynare.org> Sent: Friday, June 26, 2009 7:11 AM Subject: Re: [DynareDev] 2nd and 3rd model derivatives + k-order
Hi George,
it is inefficient to pass full hessian from *_dynamic.m or *_dynamic.dll
In k_ord_dynare, one uses only non-zero elements to populate the tensors.
The details of implementation will change again in the future, but passing only non-zero elements is a step in the right direction.
All the best,
Michel
G. Perendia wrote:
Dear Sebastien
If USE_DLL is going to be discontinued why is MEX/DLL now going to return sparse matrix exclusively when its only future use is for
k-ord-perturbation
which requires full matrix. Can we than have PERTURB_DLL option now and return full hessian using the existing (old) mex/dll dynamic model (but renamed e.g. <model>_kordpert.mexXX/dll) when that option is used rather than fixing k-ordpert rather unnecessarily and poss. just temporary since
we
may revert to the full hessian returned form mex/dll once USE_DLL is abandoned.
I think to remember we initially agreed this course of action anyway. In your email from 9th April 09 you mention:
..."For your own purposes, I have understood that you need a full hessian (instead of a sparse one). Can you confirm this? If it is the case, then I shall implement the "pertub_dll" option (as suggested by Michel), so that the preprocessor gives you a full hessian in that case."
And, there are actually quite few issues to be resolved for sparse
matrices
being returned from dll. For a start, one problem with fixing k-ordpert
is
finding out what size matrix should be passed to dll to get hessian in:
i.e.
it may be as much as 3 times bigger than the flat hessian - which can be rather large!!
Best regards
George ----- Original Message ----- From: "Sébastien Villemot" <sebastien.villemot@ens.fr> To: "List for Dynare developers" <dev@dynare.org> Sent: Thursday, June 25, 2009 1:44 PM Subject: Re: [DynareDev] 2nd and 3rd model derivatives + k-order
Le jeudi 25 juin 2009 à 13:41 +0100, G. Perendia a écrit :
Just to recap, both dr1.m and k-order-perturbation.dll are affected but
only
k-order-perturbation would need to be fixed to cope with this change
since
dr1.m will not be using USE_DLL mex option anyway.
Yes, indeed.
_______________________________________________ Dev mailing list Dev@dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev
_______________________________________________ Dev mailing list Dev@dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev
_______________________________________________ Dev mailing list Dev@dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev
I uploaded a modification to k_dyn_order.cpp that should take care of the change of format of the Hessian passed *_dynamic.dll I didn't have time to test it or even to compile it, but it should give you the idea. George, could you please test it and report any problem Many thanks Michel G. Perendia wrote:
Dear Sebastien
Thanks,
Just to recap, both dr1.m and k-order-perturbation.dll are affected but only k-order-perturbation would need to be fixed to cope with this change since dr1.m will not be using USE_DLL mex option anyway.
Best regards
George Perendia
----- Original Message ----- From: "Sébastien Villemot" <sebastien.villemot@ens.fr> To: <dev@dynare.org> Sent: Thursday, June 25, 2009 1:09 PM Subject: Re: [DynareDev] 2nd and 3rd model derivatives + k-order
Le jeudi 25 juin 2009 à 12:59 +0100, G. Perendia a écrit :
However, I do not quite understand what do you mean with USE_DLL is roken - is it because sparse matrix is now returned and the k-order is broken?
USE_DLL is broken in the sense that the matrix returned by the MEX file at the place of g2 is no longer square: it has only 3 columns describing the hessian in a sparse way. So dr1.m is completely confused with USE_DLL option at order 2: it expects a square hessian and it gets a 3-columns matrix.
Can you remind me how is _dynamic MEX/dll file, used by k-order.dll, then produced - is USE_DLL going to stay as option e.g. for 2nd order or is
it
going to be discontinued?
We are going to drop USE_DLL option. We will then add an option like "korderpert" to stoch_simul, which will trigger the call to the k-order DLL. And ultimately we will drop the generation of MEX files containing the model, to replace it by a simulated bytecode. The interface to k-order DLL will then need to be changed.
Is jacobia_ still a full matrix?
Yes, nothing changed for the Jacobian
Hi Where is the best place to set path to matlab/kordpert when if needed, e.g. if options_.use_k_order==1 & exist('k_order_perturbation') == 3 addpath([dynareroot '/kordpert/']) end I thought dynare_config.m but it does not know about options_ Best regards George ----- Original Message ----- From: "Michel Juillard" <michel.juillard@ens.fr> To: "List for Dynare developers" <dev@dynare.org> Sent: Thursday, June 25, 2009 3:38 PM Subject: Re: [DynareDev] 2nd and 3rd model derivatives + k-order I uploaded a modification to k_dyn_order.cpp that should take care of the change of format of the Hessian passed *_dynamic.dll I didn't have time to test it or even to compile it, but it should give you the idea. George, could you please test it and report any problem Many thanks Michel G. Perendia wrote:
Dear Sebastien
Thanks,
Just to recap, both dr1.m and k-order-perturbation.dll are affected but only k-order-perturbation would need to be fixed to cope with this change since dr1.m will not be using USE_DLL mex option anyway.
Best regards
George Perendia
----- Original Message ----- From: "Sébastien Villemot" <sebastien.villemot@ens.fr> To: <dev@dynare.org> Sent: Thursday, June 25, 2009 1:09 PM Subject: Re: [DynareDev] 2nd and 3rd model derivatives + k-order
Le jeudi 25 juin 2009 à 12:59 +0100, G. Perendia a écrit :
However, I do not quite understand what do you mean with USE_DLL is roken - is it because sparse matrix is now returned and the k-order is broken?
USE_DLL is broken in the sense that the matrix returned by the MEX file at the place of g2 is no longer square: it has only 3 columns describing the hessian in a sparse way. So dr1.m is completely confused with USE_DLL option at order 2: it expects a square hessian and it gets a 3-columns matrix.
Can you remind me how is _dynamic MEX/dll file, used by k-order.dll, then produced - is USE_DLL going to stay as option e.g. for 2nd order or is
it
going to be discontinued?
We are going to drop USE_DLL option. We will then add an option like "korderpert" to stoch_simul, which will trigger the call to the k-order DLL. And ultimately we will drop the generation of MEX files containing the model, to replace it by a simulated bytecode. The interface to k-order DLL will then need to be changed.
Is jacobia_ still a full matrix?
Yes, nothing changed for the Jacobian
_______________________________________________ Dev mailing list Dev@dynare.org http://www.dynare.org/cgi-bin/mailman/listinfo/dev
participants (3)
-
G. Perendia -
Michel Juillard -
Sébastien Villemot