Hi Michel,
I ran dynare++ on the mentioned mod-file again under Windows++ and used the system monitoring tools for monitoring memory usage. The picture is attached. The blue line shows the percent of memory used of my 8GB RAM on my Laptop from the start until the crash. As you can see, there is not much movement. I do not get anywhere close to the 8GB of memory used for the task that you report for Linux. Rather, straight from the beginning the jnl-file reports 0 available memory and "out of memory". It simply seems as if Dynare++ is hardly using any memory straight from the start. This does not seem to be just a problem of measuring memory usage on Windows.
Best,
Johannes
------
Johannes Pfeifer
Department of Economics
University of Mannheim
L7, 3-5, Room 242
68131 Mannheim
Germany
+49 (0)621 181-3430
Johannes,
do you know for which order this example is working on your machine and for which order it doesn't?
When it is working does the jnl make sense?
It may be that 8GB is not enough to even store the derivatives of the model before starting computations but that for some reason the program doesn't crash until much later.
Best
Michel
Johannes Pfeifer writes:
Hi Michel,
I ran dynare++ on the mentioned mod-file again under Windows++ and used the system monitoring tools for monitoring memory usage. The picture is attached. The blue line shows the percent of memory used of my 8GB RAM on my Laptop from the start until the crash. As you can see, there is not much movement. I do not get anywhere close to the 8GB of memory used for the task that you report for Linux. Rather, straight from the beginning the jnl-file reports 0 available memory and "out of memory". It simply seems as if Dynare++ is hardly using any memory straight from the start. This does not seem to be just a problem of measuring memory usage on Windows.
Best,
Johannes
Johannes Pfeifer
Department of Economics
University of Mannheim
L7, 3-5, Room 242
68131 Mannheim
Germany
+49 (0)621 181-3430
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
Looking again the the journal, I believe now that the message "run out of memory" is not an error message but an information that triggers a change in the trade-off between speed and memory (i.e. decreasing number of the threads)
I don't think that dynare++ catches memory allocation failure and just crashes at that point.
Michel Juillard writes:
Johannes,
do you know for which order this example is working on your machine and for which order it doesn't?
When it is working does the jnl make sense?
It may be that 8GB is not enough to even store the derivatives of the model before starting computations but that for some reason the program doesn't crash until much later.
Best
Michel
Johannes Pfeifer writes:
Hi Michel,
I ran dynare++ on the mentioned mod-file again under Windows++ and used the system monitoring tools for monitoring memory usage. The picture is attached. The blue line shows the percent of memory used of my 8GB RAM on my Laptop from the start until the crash. As you can see, there is not much movement. I do not get anywhere close to the 8GB of memory used for the task that you report for Linux. Rather, straight from the beginning the jnl-file reports 0 available memory and "out of memory". It simply seems as if Dynare++ is hardly using any memory straight from the start. This does not seem to be just a problem of measuring memory usage on Windows.
Best,
Johannes
Johannes Pfeifer
Department of Economics
University of Mannheim
L7, 3-5, Room 242
68131 Mannheim
Germany
+49 (0)621 181-3430
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
Sébastien also indicated some time ago that this message "just" triggers a different algorithm. But this is still a very undesirable behavior. I get this exact same message in the jnl file that is created when I use this mod-file with Dynare under Matlab at order 3 with the k_order_pert.mex64. In this case, at order=3 memory should not be an issue. But the mex-file diagnoses negative memory and immediately switches to the different algorithm from the start.
-------- Johannes Pfeifer Friesenwall 104 50672 Köln Mobil: +49-(0)170-6936820 jpfeifer@gmx.de
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 15:28 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
Looking again the the journal, I believe now that the message "run out of memory" is not an error message but an information that triggers a change in the trade-off between speed and memory (i.e. decreasing number of the threads)
I don't think that dynare++ catches memory allocation failure and just crashes at that point.
Michel Juillard writes:
Johannes,
do you know for which order this example is working on your machine and for which order it doesn't?
When it is working does the jnl make sense?
It may be that 8GB is not enough to even store the derivatives of the model before starting computations but that for some reason the program doesn't crash until much later.
Best
Michel
Johannes Pfeifer writes:
Hi Michel,
I ran dynare++ on the mentioned mod-file again under Windows++ and used the system monitoring tools for monitoring memory usage. The picture is attached. The blue line shows the percent of memory used of my 8GB RAM on my Laptop from the start until the crash. As you can see, there is not much movement. I do not get anywhere close to the 8GB of memory used for the task that you report for Linux. Rather, straight from the beginning the jnl-file reports 0 available memory and "out of memory". It simply seems as if Dynare++ is hardly using any memory straight from the start. This Dynare++ does not seem to be just a problem of measuring memory usage on Windows.
Best,
Johannes
Johannes Pfeifer
Department of Economics
University of Mannheim
L7, 3-5, Room 242
68131 Mannheim
Germany
+49 (0)621 181-3430
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-- Michel Juillard _______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
Order=5 works, 6 does not. But the Journal file (attached) to me does not show much of a difference.
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 15:00 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
Johannes,
do you know for which order this example is working on your machine and for which order it doesn't?
When it is working does the jnl make sense?
It may be that 8GB is not enough to even store the derivatives of the model before starting computations but that for some reason the program doesn't crash until much later.
Best
Michel
Johannes Pfeifer writes:
Hi Michel,
I ran dynare++ on the mentioned mod-file again under Windows++ and used the system monitoring tools for monitoring memory usage. The picture is attached. The blue line shows the percent of memory used of my 8GB RAM on my Laptop from the start until the crash. As you can see, there is not much movement. I do not get anywhere close to the 8GB of memory used for the task that you report for Linux. Rather, straight from the beginning the jnl-file reports 0 available memory and "out of memory". It simply seems as if Dynare++ is hardly using any memory straight from the start. This does Dynare++ not seem to be just a problem of measuring memory usage on Windows.
Best,
Johannes
Johannes Pfeifer
Department of Economics
University of Mannheim
L7, 3-5, Room 242
68131 Mannheim
Germany
+49 (0)621 181-3430
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-- Michel Juillard _______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
Are you saying that under Windows, dynare++ always diagnoses negative memory?
Johannes Pfeifer writes:
Sébastien also indicated some time ago that this message "just" triggers a different algorithm. But this is still a very undesirable behavior. I get this exact same message in the jnl file that is created when I use this mod-file with Dynare under Matlab at order 3 with the k_order_pert.mex64. In this case, at order=3 memory should not be an issue. But the mex-file diagnoses negative memory and immediately switches to the different algorithm from the start.
Johannes Pfeifer Friesenwall 104 50672 Köln Mobil: +49-(0)170-6936820 jpfeifer@gmx.de
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 15:28 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
Looking again the the journal, I believe now that the message "run out of memory" is not an error message but an information that triggers a change in the trade-off between speed and memory (i.e. decreasing number of the threads)
I don't think that dynare++ catches memory allocation failure and just crashes at that point.
Michel Juillard writes:
Johannes,
do you know for which order this example is working on your machine and for which order it doesn't?
When it is working does the jnl make sense?
It may be that 8GB is not enough to even store the derivatives of the model before starting computations but that for some reason the program doesn't crash until much later.
Best
Michel
Johannes Pfeifer writes:
Hi Michel,
I ran dynare++ on the mentioned mod-file again under Windows++ and used the system monitoring tools for monitoring memory usage. The picture is attached. The blue line shows the percent of memory used of my 8GB RAM on my Laptop from the start until the crash. As you can see, there is not much movement. I do not get anywhere close to the 8GB of memory used for the task that you report for Linux. Rather, straight from the beginning the jnl-file reports 0 available memory and "out of memory". It simply seems as if Dynare++ is hardly using any memory straight from the start. This Dynare++ does not seem to be just a problem of measuring memory usage on Windows.
Best,
Johannes
Johannes Pfeifer
Department of Economics
University of Mannheim
L7, 3-5, Room 242
68131 Mannheim
Germany
+49 (0)621 181-3430
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
Dynare++ always diagnoses 0 memory and k_order_pert.mexw64 in Dynare unstable always negative memory.
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 17:29 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
Are you saying that under Windows, dynare++ always diagnoses negative memory?
Johannes Pfeifer writes:
Sébastien also indicated some time ago that this message "just" triggers a different algorithm. But this is still a very undesirable behavior. I get this exact same message in the jnl file that is created when I use this mod-file with Dynare under Matlab at order 3 with the k_order_pert.mex64. In this case, at order=3 memory should not be an issue. But the mex-file diagnoses negative memory and immediately switches to the different algorithm from the start.
Johannes Pfeifer Friesenwall 104 50672 Köln Mobil: +49-(0)170-6936820 jpfeifer@gmx.de
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 15:28 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
Looking again the the journal, I believe now that the message "run out of memory" is not an error message but an information that triggers a change in the trade-off between speed and memory (i.e. decreasing number of the threads)
I don't think that dynare++ catches memory allocation failure and just crashes at that point.
Michel Juillard writes:
Johannes,
do you know for which order this example is working on your machine and for which order it doesn't?
When it is working does the jnl make sense?
It may be that 8GB is not enough to even store the derivatives of the model before starting computations but that for some reason the program doesn't crash until much later.
Best
Michel
Johannes Pfeifer writes:
Hi Michel,
I ran dynare++ on the mentioned mod-file again under Windows++ and used the system monitoring tools for monitoring memory usage. The picture is attached. The blue line shows the percent of memory used of my 8GB RAM on my Laptop from the start until the crash. As you can see, there is not much movement. I do not get anywhere close to the 8GB of memory used for the task that you report for Linux. Rather, straight from the beginning the jnl-file reports 0 available memory and "out of memory". It simply seems as if Dynare++ is hardly using any memory straight from the start. This Dynare++ does not seem to be just a problem of measuring memory usage on Windows.
Best,
Johannes
Johannes Pfeifer
Department of Economics
University of Mannheim
L7, 3-5, Room 242
68131 Mannheim
Germany
+49 (0)621 181-3430
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-- Michel Juillard _______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
OK, but this is a problem different from the one encountered with Approx_10.mod
Johannes Pfeifer writes:
Dynare++ always diagnoses 0 memory and k_order_pert.mexw64 in Dynare unstable always negative memory.
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 17:29 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
Are you saying that under Windows, dynare++ always diagnoses negative memory?
Johannes Pfeifer writes:
Sébastien also indicated some time ago that this message "just" triggers a different algorithm. But this is still a very undesirable behavior. I get this exact same message in the jnl file that is created when I use this mod-file with Dynare under Matlab at order 3 with the k_order_pert.mex64. In this case, at order=3 memory should not be an issue. But the mex-file diagnoses negative memory and immediately switches to the different algorithm from the start.
Johannes Pfeifer Friesenwall 104 50672 Köln Mobil: +49-(0)170-6936820 jpfeifer@gmx.de
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 15:28 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
Looking again the the journal, I believe now that the message "run out of memory" is not an error message but an information that triggers a change in the trade-off between speed and memory (i.e. decreasing number of the threads)
I don't think that dynare++ catches memory allocation failure and just crashes at that point.
Michel Juillard writes:
Johannes,
do you know for which order this example is working on your machine and for which order it doesn't?
When it is working does the jnl make sense?
It may be that 8GB is not enough to even store the derivatives of the model before starting computations but that for some reason the program doesn't crash until much later.
Best
Michel
Johannes Pfeifer writes:
Hi Michel,
I ran dynare++ on the mentioned mod-file again under Windows++ and used the system monitoring tools for monitoring memory usage. The picture is attached. The blue line shows the percent of memory used of my 8GB RAM on my Laptop from the start until the crash. As you can see, there is not much movement. I do not get anywhere close to the 8GB of memory used for the task that you report for Linux. Rather, straight from the beginning the jnl-file reports 0 available memory and "out of memory". It simply seems as if Dynare++ is hardly using any memory straight from the start. This Dynare++ does not seem to be just a problem of measuring memory usage on Windows.
Best,
Johannes
Johannes Pfeifer
Department of Economics
University of Mannheim
L7, 3-5, Room 242
68131 Mannheim
Germany
+49 (0)621 181-3430
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
It's the same mod-file, Approx_10.mod, just used in the two programs with order=3. As far as I understood, they rely on the same underlying routines.
-------- Johannes Pfeifer Friesenwall 104 50672 Köln Mobil: +49-(0)170-6936820 jpfeifer@gmx.de
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 18:01 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
OK, but this is a problem different from the one encountered with Approx_10.mod
Johannes Pfeifer writes:
Dynare++ always diagnoses 0 memory and k_order_pert.mexw64 in Dynare unstable always negative memory.
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 17:29 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
Are you saying that under Windows, dynare++ always diagnoses negative memory?
Johannes Pfeifer writes:
Sébastien also indicated some time ago that this message "just" triggers a different algorithm. But this is still a very undesirable behavior. I get this exact same message in the jnl file that is created when I use this mod-file with Dynare under Matlab at order 3 with the k_order_pert.mex64. In this case, at order=3 memory should not be an issue. But the mex-file diagnoses negative memory and immediately switches to the different algorithm from the start.
Johannes Pfeifer Friesenwall 104 50672 Köln Mobil: +49-(0)170-6936820 jpfeifer@gmx.de
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 15:28 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
Looking again the the journal, I believe now that the message "run out of memory" is not an error message but an information that triggers a change in the trade-off between speed and memory (i.e. decreasing number of the threads)
I don't think that dynare++ catches memory allocation failure and just crashes at that point.
Michel Juillard writes:
Johannes,
do you know for which order this example is working on your machine and for which order it doesn't?
When it is working does the jnl make sense?
It may be that 8GB is not enough to even store the derivatives of the model before starting computations but that for some reason the program doesn't crash until much later.
Best
Michel
Johannes Pfeifer writes:
Hi Michel,
I ran dynare++ on the mentioned mod-file again under Windows++ and used the system monitoring tools for monitoring memory usage. The picture is attached. The blue line shows the percent of memory used of my 8GB RAM on my Laptop from the start until the crash. As you can see, there is not much movement. I do not get anywhere close to the 8GB of memory used for the task that you report for Linux. Rather, straight from the beginning the jnl-file reports 0 available memory and "out of memory". It simply seems as if Dynare++ is hardly using any memory straight from the start. This Dynare++ does not seem to be just a problem of measuring memory usage on Windows.
Best,
Johannes
Johannes Pfeifer
Department of Economics
University of Mannheim
L7, 3-5, Room 242
68131 Mannheim
Germany
+49 (0)621 181-3430
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-- Michel Juillard _______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
What I mean is that negative memory reported by Windows is not responsible for the crash in all OS for high orders
On August 12, 2015 6:02:52 PM CEST, Johannes Pfeifer jpfeifer@gmx.de wrote:
It's the same mod-file, Approx_10.mod, just used in the two programs with order=3. As far as I understood, they rely on the same underlying routines.
Johannes Pfeifer Friesenwall 104 50672 Köln Mobil: +49-(0)170-6936820 jpfeifer@gmx.de
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 18:01 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
OK, but this is a problem different from the one encountered with Approx_10.mod
Johannes Pfeifer writes:
Dynare++ always diagnoses 0 memory and k_order_pert.mexw64 in Dynare
unstable always negative memory.
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 17:29 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
Are you saying that under Windows, dynare++ always diagnoses negative
memory?
Johannes Pfeifer writes:
Sébastien also indicated some time ago that this message "just" triggers a different algorithm. But this is still a very undesirable
behavior. I get this exact same message in the jnl file that is created when I use this mod-file with Dynare under Matlab at order 3
with the k_order_pert.mex64. In this case, at order=3 memory should not be an issue. But the mex-file diagnoses negative memory and immediately switches to the different algorithm from the start.
Johannes Pfeifer Friesenwall 104 50672 Köln Mobil: +49-(0)170-6936820 jpfeifer@gmx.de
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 15:28 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
Looking again the the journal, I believe now that the message "run out of memory" is not an error message but an information that triggers a change in the trade-off between speed and memory (i.e. decreasing number of the threads)
I don't think that dynare++ catches memory allocation failure and just crashes at that point.
Michel Juillard writes:
Johannes,
do you know for which order this example is working on your machine
and for which order it doesn't?
When it is working does the jnl make sense?
It may be that 8GB is not enough to even store the derivatives of the model before starting computations but that for some reason the
program doesn't crash until much later.
Best
Michel
Johannes Pfeifer writes:
Hi Michel,
I ran dynare++ on the mentioned mod-file again under Windows++ and
used the system monitoring tools for monitoring memory usage. The picture is attached. The blue line shows the percent of memory
used
of my 8GB RAM on my Laptop from the start until the crash. As you can see, there is not much movement. I do not get anywhere close
to
the 8GB of memory used for the task that you report for Linux. Rather, straight from the beginning the jnl-file reports 0 available memory and "out of memory". It simply seems as if Dynare++ is hardly using any memory straight from the start. This Dynare++ does not seem to be just a problem of measuring memory usage on Windows.
Best,
Johannes
Johannes Pfeifer
Department of Economics
University of Mannheim
L7, 3-5, Room 242
68131 Mannheim
Germany
+49 (0)621 181-3430
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-- Michel Juillard _______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
Most probably not, but I am not sure if it’s not a symptom of the same underlying problem. The error messages/crashes seem to be different for order=6 on different OS, which may have to do with Dynare++ turning to a different algorithm depending on the detected memory.
--------
Johannes Pfeifer
Friesenwall 104
50672 Köln
Mobil: +49-(0)170-6936820
jpfeifer@gmx.de
Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 18:17 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
What I mean is that negative memory reported by Windows is not responsible for the crash in all OS for high orders
On August 12, 2015 6:02:52 PM CEST, Johannes Pfeifer jpfeifer@gmx.de wrote:
It's the same mod-file, Approx_10.mod, just used in the two programs with order=3. As far as I understood, they rely on the same underlying routines.
-------- Johannes Pfeifer Friesenwall 104 50672 Köln Mobil: +49-(0)170-6936820 jpfeifer@gmx.de
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 18:01 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
OK, but this is a problem different from the one encountered with Approx_10.mod
Johannes Pfeifer writes: Dynare++ always diagnoses 0 memory and k_order_pert.mexw64 in Dynare unstable always negative memory.
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 17:29 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
Are you saying that under Windows, dynare++ always diagnoses negative memory?
Johannes Pfeifer writes: Sébastien also indicated some time ago that this message "just" triggers a different algorithm. But this is still a very undesirable behavior. I get this exact same message in the jnl file that is created when I use this mod-file with Dynare under Matlab at order 3 with the k_order_pert.mex64. In this case, at order=3 memory should not be an issue. But the mex-file diagnoses negative memory and immediately switches to the different algorithm from the start.
--------
Johannes Pfeifer Friesenwall 104 50672 Köln Mobil: +49-(0)170-6936820 jpfeifer@gmx.de
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 15:28 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
Looking again the the journal, I believe now that the message "run out of memory" is not an error message but an information that triggers a change in the trade-off between speed and memory (i.e. decreasing number of the threads)
I don't think that dynare++ catches memory allocation failure and just crashes at that point.
Michel Juillard writes: Johannes,
do you know for which order this example is working on your machine and for which order it doesn't?
When it is working does the jnl make sense?
It may be that 8GB is not enough to even store the derivatives of the model before starting computations but that for some reason the program doesn't crash until much later.
Best
Michel
Johannes Pfeifer writes: Hi Michel,
I ran dynare++ on the mentioned mod-file again under Windows++ and used the system monitoring tools for monitoring memory usage. The picture is attached. The blue line shows the percent of memory used of my 8GB RAM on my Laptop from the start until the crash. As you can see, there is not much movement. I do not get anywhere close to the 8GB of memory used for the task that you report for Linux. Rather, straight from the beginning the jnl-file reports 0 available memory and "out of memory". It simply seems as if Dynare++ is hardly using any memory straight from the start. This Dynare++ does not seem to be just a problem of measuring memory usage on Windows.
Best,
Johannes
------
Johannes Pfeifer
Department of Economics
University of Mannheim
L7, 3-5, Room 242
68131 Mannheim
Germany
+49 (0)621 181-3430
_____
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-- Michel Juillard
_____
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
_____
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
The user who originally reported the issue has provided more info at http://www.dynare.org/phpBB3/viewtopic.php?f=1 http://www.dynare.org/phpBB3/viewtopic.php?f=1&t=7155 &t=7155
@Houtan: does the info on MacOS given there provide any clues?
Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Johannes Pfeifer Gesendet: Mittwoch, 12. August 2015 18:26 An: 'List for Dynare developers' Betreff: Re: [DynareDev] Dynare++ Memory
Most probably not, but I am not sure if it’s not a symptom of the same underlying problem. The error messages/crashes seem to be different for order=6 on different OS, which may have to do with Dynare++ turning to a different algorithm depending on the detected memory.
--------
Johannes Pfeifer
Friesenwall 104
50672 Köln
Mobil: +49-(0)170-6936820
jpfeifer@gmx.de
Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 18:17 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
What I mean is that negative memory reported by Windows is not responsible for the crash in all OS for high orders
On August 12, 2015 6:02:52 PM CEST, Johannes Pfeifer jpfeifer@gmx.de wrote:
It's the same mod-file, Approx_10.mod, just used in the two programs with order=3. As far as I understood, they rely on the same underlying routines.
-------- Johannes Pfeifer Friesenwall 104 50672 Köln Mobil: +49-(0)170-6936820 jpfeifer@gmx.de
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 18:01 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
OK, but this is a problem different from the one encountered with Approx_10.mod
Johannes Pfeifer writes: Dynare++ always diagnoses 0 memory and k_order_pert.mexw64 in Dynare unstable always negative memory.
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 17:29 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
Are you saying that under Windows, dynare++ always diagnoses negative memory?
Johannes Pfeifer writes: Sébastien also indicated some time ago that this message "just" triggers a different algorithm. But this is still a very undesirable behavior. I get this exact same message in the jnl file that is created when I use this mod-file with Dynare under Matlab at order 3 with the k_order_pert.mex64. In this case, at order=3 memory should not be an issue. But the mex-file diagnoses negative memory and immediately switches to the different algorithm from the start.
-------- Johannes Pfeifer Friesenwall 104 50672 Köln Mobil: +49-(0)170-6936820 jpfeifer@gmx.de
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 15:28 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
Looking again the the journal, I believe now that the message "run out of memory" is not an error message but an information that triggers a change in the trade-off between speed and memory (i.e. decreasing number of the threads)
I don't think that dynare++ catches memory allocation failure and just crashes at that point.
Michel Juillard writes: Johannes,
do you know for which order this example is working on your machine and for which order it doesn't?
When it is working does the jnl make sense?
It may be that 8GB is not enough to even store the derivatives of the model before starting computations but that for some reason the program doesn't crash until much later.
Best
Michel
Johannes Pfeifer writes: Hi Michel,
I ran dynare++ on the mentioned mod-file again under Windows++ and used the system monitoring tools for monitoring memory usage. The picture is attached. The blue line shows the percent of memory used of my 8GB RAM on my Laptop from the start until the crash. As you can see, there is not much movement. I do not get anywhere close to the 8GB of memory used for the task that you report for Linux. Rather, straight from the beginning the jnl-file reports 0 available memory and "out of memory". It simply seems as if Dynare++ is hardly using any memory straight from the start. This Dynare++ does not seem to be just a problem of measuring memory usage on Windows.
Best,
Johannes
------
Johannes Pfeifer
Department of Economics
University of Mannheim
L7, 3-5, Room 242
68131 Mannheim
Germany
+49 (0)621 181-3430
_____
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-- Michel Juillard
_____
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
_____
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
I don't have yet the answer from valgrind on this example. I'm still convinced that the problem has to do with size but not in the way I was thinking initially. If it was a problem of available memory, in Linux at least, we should get a "bad_alloc" error message, unless memory is allocated in a non standard manner. Now, I'm suspecting more an index or an offset that is too large for the type it is stored in. This will be much more difficult to debug. The problem could also be in a library called by dynare++. I wonder how much time should we spend on this problem
Johannes Pfeifer writes:
The user who originally reported the issue has provided more info at http://www.dynare.org/phpBB3/viewtopic.php?f=1 http://www.dynare.org/phpBB3/viewtopic.php?f=1&t=7155 &t=7155
@Houtan: does the info on MacOS given there provide any clues?
Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Johannes Pfeifer Gesendet: Mittwoch, 12. August 2015 18:26 An: 'List for Dynare developers' Betreff: Re: [DynareDev] Dynare++ Memory
Most probably not, but I am not sure if it’s not a symptom of the same underlying problem. The error messages/crashes seem to be different for order=6 on different OS, which may have to do with Dynare++ turning to a different algorithm depending on the detected memory.
Johannes Pfeifer
Friesenwall 104
50672 Köln
Mobil: +49-(0)170-6936820
jpfeifer@gmx.de
Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 18:17 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
What I mean is that negative memory reported by Windows is not responsible for the crash in all OS for high orders
On August 12, 2015 6:02:52 PM CEST, Johannes Pfeifer jpfeifer@gmx.de wrote:
It's the same mod-file, Approx_10.mod, just used in the two programs with order=3. As far as I understood, they rely on the same underlying routines.
Johannes Pfeifer Friesenwall 104 50672 Köln Mobil: +49-(0)170-6936820 jpfeifer@gmx.de
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 18:01 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
OK, but this is a problem different from the one encountered with Approx_10.mod
Johannes Pfeifer writes: Dynare++ always diagnoses 0 memory and k_order_pert.mexw64 in Dynare unstable always negative memory.
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 17:29 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
Are you saying that under Windows, dynare++ always diagnoses negative memory?
Johannes Pfeifer writes: Sébastien also indicated some time ago that this message "just" triggers a different algorithm. But this is still a very undesirable behavior. I get this exact same message in the jnl file that is created when I use this mod-file with Dynare under Matlab at order 3 with the k_order_pert.mex64. In this case, at order=3 memory should not be an issue. But the mex-file diagnoses negative memory and immediately switches to the different algorithm from the start.
Johannes Pfeifer Friesenwall 104 50672 Köln Mobil: +49-(0)170-6936820 jpfeifer@gmx.de
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Mittwoch, 12. August 2015 15:28 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
Looking again the the journal, I believe now that the message "run out of memory" is not an error message but an information that triggers a change in the trade-off between speed and memory (i.e. decreasing number of the threads)
I don't think that dynare++ catches memory allocation failure and just crashes at that point.
Michel Juillard writes: Johannes,
do you know for which order this example is working on your machine and for which order it doesn't?
When it is working does the jnl make sense?
It may be that 8GB is not enough to even store the derivatives of the model before starting computations but that for some reason the program doesn't crash until much later.
Best
Michel
Johannes Pfeifer writes: Hi Michel,
I ran dynare++ on the mentioned mod-file again under Windows++ and used the system monitoring tools for monitoring memory usage. The picture is attached. The blue line shows the percent of memory used of my 8GB RAM on my Laptop from the start until the crash. As you can see, there is not much movement. I do not get anywhere close to the 8GB of memory used for the task that you report for Linux. Rather, straight from the beginning the jnl-file reports 0 available memory and "out of memory". It simply seems as if Dynare++ is hardly using any memory straight from the start. This Dynare++ does not seem to be just a problem of measuring memory usage on Windows.
Best,
Johannes
Johannes Pfeifer
Department of Economics
University of Mannheim
L7, 3-5, Room 242
68131 Mannheim
Germany
+49 (0)621 181-3430
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-- Michel Juillard
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
I don't feel strongly about Dynare++ and would prefer to see it replaced by Dynare rather sooner than later (ticket 217 stands in the way). The reason I am interested in this issue are its consequences for higher order perturbation in Dynare. The k_order_pert.mex routine that is based on the codes in Dynare++ shows sometimes shows weird behavior on Windows. The last time we tried to investigate it, we did not find an answer. I was hoping that this model provides additional clues.
On a broader perspective, I agree that the model is most probably simply too large for order=6. What bothers me is the difference in algorithms one encounters in Linux vs. other OS. The algorithm Windows turns to comes with a large loss in performance that may make particle filtering and SMM at higher order too time-consuming.
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Donnerstag, 13. August 2015 13:18 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
I don't have yet the answer from valgrind on this example. I'm still convinced that the problem has to do with size but not in the way I was thinking initially. If it was a problem of available memory, in Linux at least, we should get a "bad_alloc" error message, unless memory is allocated in a non standard manner. Now, I'm suspecting more an index or an offset that is too large for the type it is stored in. This will be much more difficult to debug. The problem could also be in a library called by dynare++. I wonder how much time should we spend on this problem
I will try to look into the computation of available memory in Windows. This is a part well delimited in dynare++ (about 30 lines)
However, it will take me some time as I won't have access to a Windows comptuer next week
Best
Michel
Johannes Pfeifer writes:
I don't feel strongly about Dynare++ and would prefer to see it replaced by Dynare rather sooner than later (ticket 217 stands in the way). The reason I am interested in this issue are its consequences for higher order perturbation in Dynare. The k_order_pert.mex routine that is based on the codes in Dynare++ shows sometimes shows weird behavior on Windows. The last time we tried to investigate it, we did not find an answer. I was hoping that this model provides additional clues.
On a broader perspective, I agree that the model is most probably simply too large for order=6. What bothers me is the difference in algorithms one encounters in Linux vs. other OS. The algorithm Windows turns to comes with a large loss in performance that may make particle filtering and SMM at higher order too time-consuming.
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Donnerstag, 13. August 2015 13:18 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
I don't have yet the answer from valgrind on this example. I'm still convinced that the problem has to do with size but not in the way I was thinking initially. If it was a problem of available memory, in Linux at least, we should get a "bad_alloc" error message, unless memory is allocated in a non standard manner. Now, I'm suspecting more an index or an offset that is too large for the type it is stored in. This will be much more difficult to debug. The problem could also be in a library called by dynare++. I wonder how much time should we spend on this problem
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
Sure. It's not urgent.
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Donnerstag, 13. August 2015 15:24 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
I will try to look into the computation of available memory in Windows. This is a part well delimited in dynare++ (about 30 lines)
However, it will take me some time as I won't have access to a Windows comptuer next week
Best
Michel
Johannes Pfeifer writes:
I don't feel strongly about Dynare++ and would prefer to see it replaced by Dynare rather sooner than later (ticket 217 stands in the way). The reason I am interested in this issue are its consequences for higher order perturbation in Dynare. The k_order_pert.mex routine that is based on the codes in Dynare++ shows sometimes shows weird behavior on Windows. The last time we tried to investigate it, we did not find an answer. I was hoping that this model provides additional clues.
On a broader perspective, I agree that the model is most probably simply too large for order=6. What bothers me is the difference in algorithms one encounters in Linux vs. other OS. The algorithm Windows turns to comes with a large loss in performance that may make particle filtering and SMM at higher order too time-consuming.
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Donnerstag, 13. August 2015 13:18 An: List for Dynare developers Betreff: Re: [DynareDev] Dynare++ Memory
I don't have yet the answer from valgrind on this example. I'm still convinced that the problem has to do with size but not in the way I was thinking initially. If it was a problem of available memory, in Linux at least, we should get a "bad_alloc" error message, unless memory is allocated in a non standard manner. Now, I'm suspecting more an index or an offset that is too large for the type it is stored in. This will be much more difficult to debug. The problem could also be in a library called by dynare++. I wonder how much time should we spend on this problem
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-- Michel Juillard _______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-
Johannes Pfeifer -
Johannes Pfeifer -
Michel Juillard