How to explain the result when measuring stack size of REPL

counter · April 21, 2020, 12:27pm

For the following code

def stack(n: Int): Int = {
  try stack(n+1)
  catch {
    case e: StackOverflowError => {
      val inMB = 1024 * 1024
      val totalMemory = Runtime.getRuntime().totalMemory() / inMB
      val maxMemory = Runtime.getRuntime().maxMemory() / inMB
      val freeMemory = Runtime.getRuntime().freeMemory() / inMB
      println(n, totalMemory, maxMemory, freeMemory)
      n
    }
  }
}
stack(0)

stack function should get the max size of stack and print runtime memory usage infomation.

But the result is not as except.

For example

$ scala
Welcome to Scala 2.13.1 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_181).
Type in expressions for evaluation. Or try :help.

scala> def stack(n: Int): Int = {
     |   try stack(n+1)
     |   catch {
     |     case e: StackOverflowError => {
     |       val inMB = 1024 * 1024
     |       val totalMemory = Runtime.getRuntime().totalMemory() / inMB
     |       val maxMemory = Runtime.getRuntime().maxMemory() / inMB
     |       val freeMemory = Runtime.getRuntime().freeMemory() / inMB
     |       println(n, totalMemory, maxMemory, freeMemory)
     |       n
     |     }
     |   }
     | }
stack: (n: Int)Int

scala> stack(0)
(5845,94,228,52)
res0: Int = 5845

scala> stack(0)
(3673,94,228,43)(3672,94,228,43)(3671,94,228,43)(3670,94,228,43)
res1: Int = 3670

scala> stack(0)
(14695,94,228,33)(14694,94,228,33)(14693,94,228,32)(14692,94,228,32)(14691,94,228,32)(14690,94,228,32)(14689,94,228,32)(14688,94,228,32)(14687,94,228,32)(14686,94,228,32)(14685,94,228,32)(14684,94,228,32)(14683,94,228,32)(14682,94,228,32)(14681,94,228,32)(14680,94,228,32)(14679,94,228,32)
res2: Int = 14679

...

scala> stack(0)
(14701,115,228,44)(14700,115,228,44)(14699,115,228,44)(14698,115,228,44)(14697,115,228,44)(14696,115,228,44)(14695,115,228,44)(14694,115,228,44)(14693,115,228,44)(14692,115,228,44)(14691,115,228,44)
res36: Int = 14691

...

scala> stack(0)
(14701,108,228,54)(14700,108,228,54)(14699,108,228,54)(14698,108,228,54)(14697,108,228,54)(14696,108,228,54)(14695,108,228,54)(14694,108,228,54)(14693,108,228,54)(14692,108,228,54)(14691,108,228,54)(14690,108,228,54)(14689,108,228,54)
res84: Int = 14689

As the number of function calls increases, so does the result changes.

I have some questions here

Why the first result of stack(0) output one line as except while call to function after print output result multiple times?
Why the stack size will change from 5748 down to 3670(It sems not happen every time), then increase as the function invokes much times?

curoli · April 21, 2020, 2:00pm

The methods you are calling report heap size, not stack size.

There used to be a time when a typical JVM had a fixed heap size over its lifetime, but modern JVMs typically expand according to need as long as there is sufficient system memory. Monitoring the heap space was meaningful and straight-forward back then, but now it is less clear what those number even mean, unless you configured the JVM to keep the heap space fixed.

BalmungSan · April 21, 2020, 2:19pm

Also, AFAIK, catching a SOE is meaningless, if you do not have stack, you can not allocate the space for the catch block. So, unless I am mistaken, I am not sure how this is even working.

curoli · April 21, 2020, 2:35pm

In which case a new SOE is thrown and it tries to catch it, and the process may repeat itself a few times before enough stack is unwound to allow the catch block to succeed.

counter · April 21, 2020, 2:47pm

Thanks for reply.

I misunderstand the heap and stack size , Runtime.getRuntime().xxMemroy just get information about heap. So the code can be simplified.

scala> def stack(n: Int): Int = {
     |   try stack(n+1)
     |   catch {
     |     case e: StackOverflowError => {
     |       println(n+",")
     |       n
     |     }
     |   }
     | }
             println(n+",")
                      ^
On line 5: warning: method + in class Int is deprecated (since 2.13.0): Adding a number and a String is deprecated. Use the string interpolation `s"$num$str"`
stack: (n: Int)Int

scala> stack(0)
9000,
res0: Int = 9000

scala> stack(0)
9797,9796,9795,9794,9793,9792,9791,9790,9789,9788,9787,9786,
res1: Int = 9786

scala> stack(0)
5344,5343,5342,5341,5340,5339,5338,
res2: Int = 5338

scala> stack(0)
23515,23514,23513,23512,23511,23510,23509,23508,23507,23506,23505,23504,23503,23502,23501,23500,23499,23498,23497,23496,23495,23494,23493,23492,23491,23490,23489,
res3: Int = 23489

It seems the println doesn’t behave as usually, it looks like that even the code trigger SOE, println still print result.

In which case a new SOE is thrown and it tries to catch it, and the process may repeat itself a few times before enough stack is unwound to allow the catch block to succeed.

If in this case, why println is invoded,

doesn’t it have enough space before the stack is unwound?
And why repeat it? I’m really confused.

curoli · April 21, 2020, 5:46pm

Try to print the stack trace to see where the last SOE is coming from.

So, at some point, the stack is full, let’s say it is full when we enter stack(50000). Trying to call stack(50001) will throw a SOE, which we catch, but the stack is still full, so trying to call println will fail with another SOE. So, we are exiting stack(50000) and are now back to stack(49999), where we catch the SOE and again try println, and the stack is now not completely full, but still not sufficient to do println, so that will throw another SOE, which will be caught in stack(49998) and so on, until there is enough space on the stack left to successfully complete the println, and then we complete normally.

counter · April 23, 2020, 3:32am

Thanks for detailed explanation.
There is still one question, why stack size changed.
I guess is relative to JIT, but I’m not familiar with JIT

AFAIK, the function run in REPL, after many times invoke, the hotspot arrange it to compile.
When the background compiler finish its task , it will replace the fucntion entrance with compiled one.

The compiled version of the function is optimized, so the stack space observed is larger

For example

scala -Djava.compiler=NONE 
Welcome to Scala 2.13.1 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_181).
Type in expressions for evaluation. Or try :help.

scala> def stack(n: Int): Unit = {
     |   try stack(n+1)
     |   catch {
     |     case e: StackOverflowError => {
     |       println(n+",")
     |     }
     |   }
     | }
             println(n+",")
                      ^
On line 5: warning: method + in class Int is deprecated (since 2.13.0): Adding a number and a String is deprecated. Use the string interpolation `s"$num$str"`
stack: (n: Int)Unit

scala> stack(0)
8997,

scala> stack(0)
9042,9041,9040,9039,9038,9037,9036,9035,

scala> stack(0)
9042,9041,9040,9039,9038,9037,9036,9035,

scala> stack(0)
9042,9041,9040,9039,9038,9037,9036,9035,

This theory can partly explain the observed phenomena, but there are still several questions

When enable JIT, why the print stack size(a number, not real stack size) will first down and then up?
When disbale JIT, why the print stack size will change? Shouldn’t it be a fixed value？