Increase the size of simple maze example

[mohsen-ghaffari1992]

I made an small change in simple maze example. If you test SarsaExperiments, your will face with memory issues for 10000 episodes.
I set the TimeHorizon = 1200. If you increase it, your heap will be full even faster because I added time into steps that if it is greater than or equal to TimeHorizon, the episode is finished (it is a final condition).
The evaluation process is now commented out that shows the issue appears in learning process.

[wasowski]

Is it possible to have the opposite? I prefer the issue to be demonstrated for evaluation rather than for learning. Learning is quite a bit more complex code, so it is better to get the prototype on evaluation (also less files need to be modified to get evaluation working).

…

On Tue, Jan 9, 2024, 20:08 Mohsen Ghaffari ***@***.***> wrote: I made an small change in simple maze example. If you test SarsaExperiments, your will face with memory issues for 10000 episodes. I set the TimeHorizon = 1200. If you increase it, your heap will be full even faster because I added time into steps that if it is greater than or equal to TimeHorizon, the episode is finished (it is a final condition). The evaluation process is now commented out that shows the issue appears in learning process. ------------------------------ You can view, comment on, or merge this pull request online at: #229 Commit Summary - 339bad8 <339bad8> Increase the size of simple maze example File Changes (3 files <https://github.com/itu-square/symsim/pull/229/files>) - *A* simplemaze.csv <https://github.com/itu-square/symsim/pull/229/files#diff-de4858ef176cdeea90042a38757aad309c2a49cab0bb6835965f8f2b3741f2af> (2) - *M* src/main/scala/symsim/examples/concrete/simplemaze/Maze.scala <https://github.com/itu-square/symsim/pull/229/files#diff-b4e92078972f23590ce2778410d896baf927844340081c298bb72f0f3b69f153> (30) - *M* src/test/scala/symsim/examples/concrete/simplemaze/SarsaExperiments.scala <https://github.com/itu-square/symsim/pull/229/files#diff-8e1862bcf468b9d6f3add15cfa3aa941f44e955d9a2891f4b55a274364f7e5e0> (66) Patch Links: - https://github.com/itu-square/symsim/pull/229.patch - https://github.com/itu-square/symsim/pull/229.diff — Reply to this email directly, view it on GitHub <#229>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANMEHHJEQQ6ERMZJ43O5BLYNWITBAVCNFSM6AAAAABBTTEL32VHI2DSMVQWIX3LMV43ASLTON2WKOZSGA3TEOJZGMZDCOA> . You are receiving this because your review was requested.Message ID: ***@***.***>

[mohsen-ghaffari1992]

I see how you mean but I am afraid to say that this we cannot observe until learning process finishes. However, I can delete the timeout, and this will put us in the situation that you want to see. I am not sure whether you are looking for a case like this because we already solved it by adding timeout. Should I change the code as I explained? Best, Mohsen From: Andrzej Wąsowski ***@***.***> Date: Tuesday, 9 January 2024 at 22.00 To: itu-square/symsim ***@***.***> Cc: Mohsen Ghaffari ***@***.***>, Author ***@***.***> Subject: Re: [itu-square/symsim] Increase the size of simple maze example (PR #229) Is it possible to have the opposite? I prefer the issue to be demonstrated for evaluation rather than for learning. Learning is quite a bit more complex code, so it is better to get the prototype on evaluation (also less files need to be modified to get evaluation working).

On Tue, Jan 9, 2024, 20:08 Mohsen Ghaffari ***@***.***> wrote: I made an small change in simple maze example. If you test SarsaExperiments, your will face with memory issues for 10000 episodes. I set the TimeHorizon = 1200. If you increase it, your heap will be full even faster because I added time into steps that if it is greater than or equal to TimeHorizon, the episode is finished (it is a final condition). The evaluation process is now commented out that shows the issue appears in learning process. ------------------------------ You can view, comment on, or merge this pull request online at: #229 Commit Summary - 339bad8 <339bad8> Increase the size of simple maze example File Changes (3 files <https://github.com/itu-square/symsim/pull/229/files>) - *A* simplemaze.csv <https://github.com/itu-square/symsim/pull/229/files#diff-de4858ef176cdeea90042a38757aad309c2a49cab0bb6835965f8f2b3741f2af> (2) - *M* src/main/scala/symsim/examples/concrete/simplemaze/Maze.scala <https://github.com/itu-square/symsim/pull/229/files#diff-b4e92078972f23590ce2778410d896baf927844340081c298bb72f0f3b69f153> (30) - *M* src/test/scala/symsim/examples/concrete/simplemaze/SarsaExperiments.scala <https://github.com/itu-square/symsim/pull/229/files#diff-8e1862bcf468b9d6f3add15cfa3aa941f44e955d9a2891f4b55a274364f7e5e0> (66) Patch Links: - https://github.com/itu-square/symsim/pull/229.patch - https://github.com/itu-square/symsim/pull/229.diff — Reply to this email directly, view it on GitHub <#229>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANMEHHJEQQ6ERMZJ43O5BLYNWITBAVCNFSM6AAAAABBTTEL32VHI2DSMVQWIX3LMV43ASLTON2WKOZSGA3TEOJZGMZDCOA> . You are receiving this because your review was requested.Message ID: ***@***.***>

— Reply to this email directly, view it on GitHub<#229 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ASQKWROI5YHMQY3XDA3XB3LYNWVWRAVCNFSM6AAAAABBTTEL32VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBTG44DIMJXGU>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[mohsen-ghaffari1992]

I pushed the new branch that you can observe heap issue in evaluation. Please read the commit comment and let me know if you had any questions. Best, Mohsen From: Mohsen Ghaffari ***@***.***> Date: Wednesday, 10 January 2024 at 08.33 To: itu-square/symsim ***@***.***>, itu-square/symsim ***@***.***> Cc: Author ***@***.***> Subject: Re: [itu-square/symsim] Increase the size of simple maze example (PR #229) I see how you mean but I am afraid to say that this we cannot observe until learning process finishes. However, I can delete the timeout, and this will put us in the situation that you want to see. I am not sure whether you are looking for a case like this because we already solved it by adding timeout. Should I change the code as I explained? Best, Mohsen From: Andrzej Wąsowski ***@***.***> Date: Tuesday, 9 January 2024 at 22.00 To: itu-square/symsim ***@***.***> Cc: Mohsen Ghaffari ***@***.***>, Author ***@***.***> Subject: Re: [itu-square/symsim] Increase the size of simple maze example (PR #229) Is it possible to have the opposite? I prefer the issue to be demonstrated for evaluation rather than for learning. Learning is quite a bit more complex code, so it is better to get the prototype on evaluation (also less files need to be modified to get evaluation working).

On Tue, Jan 9, 2024, 20:08 Mohsen Ghaffari ***@***.***> wrote: I made an small change in simple maze example. If you test SarsaExperiments, your will face with memory issues for 10000 episodes. I set the TimeHorizon = 1200. If you increase it, your heap will be full even faster because I added time into steps that if it is greater than or equal to TimeHorizon, the episode is finished (it is a final condition). The evaluation process is now commented out that shows the issue appears in learning process. ------------------------------ You can view, comment on, or merge this pull request online at: #229 Commit Summary - 339bad8 <339bad8> Increase the size of simple maze example File Changes (3 files <https://github.com/itu-square/symsim/pull/229/files>) - *A* simplemaze.csv <https://github.com/itu-square/symsim/pull/229/files#diff-de4858ef176cdeea90042a38757aad309c2a49cab0bb6835965f8f2b3741f2af> (2) - *M* src/main/scala/symsim/examples/concrete/simplemaze/Maze.scala <https://github.com/itu-square/symsim/pull/229/files#diff-b4e92078972f23590ce2778410d896baf927844340081c298bb72f0f3b69f153> (30) - *M* src/test/scala/symsim/examples/concrete/simplemaze/SarsaExperiments.scala <https://github.com/itu-square/symsim/pull/229/files#diff-8e1862bcf468b9d6f3add15cfa3aa941f44e955d9a2891f4b55a274364f7e5e0> (66) Patch Links: - https://github.com/itu-square/symsim/pull/229.patch - https://github.com/itu-square/symsim/pull/229.diff — Reply to this email directly, view it on GitHub <#229>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANMEHHJEQQ6ERMZJ43O5BLYNWITBAVCNFSM6AAAAABBTTEL32VHI2DSMVQWIX3LMV43ASLTON2WKOZSGA3TEOJZGMZDCOA> . You are receiving this because your review was requested.Message ID: ***@***.***>

— Reply to this email directly, view it on GitHub<#229 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ASQKWROI5YHMQY3XDA3XB3LYNWVWRAVCNFSM6AAAAABBTTEL32VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBTG44DIMJXGU>. You are receiving this because you authored the thread.Message ID: ***@***.***>