Thanks Sebastien. This will be very useful. However, I see a potential problem with including MINGW with Dynare. Institutions with security issues may not want to install MINGW. There is also a specific issue with the signature of the package. I don't remember if it covers only the binary of the preprossor or the entire package.
The easiest solution would be to keep distributing a minimal package without MINGW.
Best
Michel
Sébastien Villemot writes:
Hi all,
I have pushed several improvements to the "use_dll" option, which is the mechanism by which we tell Dynare to compile the model equations and derivatives using a C compiler, instead of using MATLAB output. This option is useful when the model is evaluated a large number of times (for example for estimation or extended path), since evaluation will then be much faster. This however comes at a (fixed) cost, corresponding to the compilation time, so there is a trade-off involved.
So far, this option was suffering from two limitations: – the compilation was very long on large models – under Windows, the option was only accessible after going through an extra (and possibly cumbersome) installation step (the user had first to install a compiler and then to tell MATLAB where to find it)
The changes that I have pushed address these two problems.
For the compilation speed issue, the solution consists in disabling certain compiler optimizations that take a long time while not being relevant for our use case (e.g. common sub-expression elimination, which is already done in the preprocessor under the name of “temporary terms”). By analyzing the time spent by the compiler in the various optimization phases (using the "-Q -v" options of GCC), I came up with a list of flags that give a very significant speed boost (as a data point, I can now compile the dynamic file for GIMF with 6 countries at 2nd derivation order in 40 sec on my machine). You can see the exact list of compilation flags in ModelTree::compileDll(). In the near future, I plan to provide an interface for customizing these flags.
For the other issue, the solution is in two steps. First, the compilation of the model MEX files is now done by the preprocessor (it used to be done by the mex command within MATLAB). Secondly, under Windows, the Dynare installer will now contain a full working copy of MinGW (i.e. GCC for Windows), so that everything now works out-of-the- box without any further manual step. This however comes at a space cost: the installer will now be 100Mb (instead of 20Mb previously), and the disk space used will now be 800Mb (instead of 100Mb). Note that both 32- and 64-bit compilers are included, and most users need only one of the two (depending on their version of MATLAB, and not on their version of Windows), so it is in theory possible to divide this space by two. One possibility is to unselect the version that is not needed during the installation process. Alternatively, we could produce two installers, one for 32-bit, one for 64-bit. But note that both solutions require that the user knows which flavor of MATLAB he has.
Also note that, under Octave for Windows, we use the compiler shipped with Octave instead of the one shipped with Dynare.
Finally, there is an issue concerning macOS. The optimization flags that are passed to the compiler are specific to GCC, while macOS people tend to work with Clang. We plan to solve this by distributing Dynare on macOS exclusively through homebrew (this was already the case for the Octave version under macOS, but not for the MATLAB version). This will provide an easy way of installing GCC (and by the way this will simplify things on macOS, since there will be a unified installation procedure across both MATLAB and Octave).
Please test the new setup (Windows snapshot installers should be ready by tomorrow) and give your feedback.
If everything works well, we could even consider making use_dll the default (but we're not yet there).
Best,