How to figure out which test failed

I’m running my test suites in a gitlab pipeline whenever my branch gets pushed.
I’m using an sbt command line such as the following
sbt -Dsbt.log.noformat=true "set parallelExecution in Test := false" test |& tee $tmpfile

This command line is supposed to prevent parallel execution for the purpose of easing debugging.
However, the test fails because of java.lang.OutOfMemoryError: Java heap space. I don’t get this error when I run the tests interactively.

The info message seems to indicate that the error is in the test named coloring, but I’m sure that’s not the case because I have printed the debug message on line 191 finished test coloring.

Can someone give me a clue about how to figure out which is the offending test?


179 [info] - read dimacs benchmark files
180 [info] BddFiddleTestSuite:
181 [info] - genSample
182 [info] - genDnfFromBitMask
183 [info] - extractBits
184 [info] - genKthBdd
185 [info] - gen truth table range
186 4 (4) states: List(a, b, c, d)
187 plot 1.0 1.0
188 plot 2.0 19.0
189 plot 3.0 19.0
190 plot 4.0 57.0
191 finished test coloring
192 30 (30) states: List(Russia, Estonia, Belarus, Poland, Ukraine, Romania, Serbia, Croatia, Slovenia, Slovakia, Czech Republic, Finland, Kosovo, Albania, Greece, Lithuania, Sweden, Denmark, North Macedonia, Austria, Liechtenstein, Latvia, Italy, Moldova, Bulgaria, Norway, Montenegro, France, Luxembourg, Spain)
193 [info] MapColoringTestSuite:
194 [info] - coloring
195 [info] bdd.MapColoringTestSuite *** ABORTED ***
196 [info]   java.lang.OutOfMemoryError: Java heap space
197 [info]   at scala.collection.mutable.HashTable.resize(HashTable.scala:258)
198 [info]   at scala.collection.mutable.HashTable.addEntry0(HashTable.scala:158)
199 [info]   at scala.collection.mutable.HashTable.findOrAddEntry(HashTable.scala:170)
200 [info]   at scala.collection.mutable.HashTable.findOrAddEntry$(HashTable.scala:167)
201 [info]   at scala.collection.mutable.HashMap.findOrAddEntry(HashMap.scala:44)
202 [info]   at scala.collection.mutable.HashMap.put(HashMap.scala:126)
203 [info]   at scala.collection.mutable.HashMap.update(HashMap.scala:131)
204 [info]   at bdd.BinaryOperation.memoize(BinaryOperation.scala:74)
205 [info]   at bdd.And$.apply(BinaryOperation.scala:117)
206 [info]   at bdd.BinaryOperation.bddOp(BinaryOperation.scala:100)
207 [info]   ...
208 java.lang.OutOfMemoryError: Java heap space
209 	at scala.collection.mutable.HashTable.resize(HashTable.scala:258)
210 	at scala.collection.mutable.HashTable.addEntry0(HashTable.scala:158)
211 	at scala.collection.mutable.HashTable.findOrAddEntry(HashTable.scala:170)
212 	at scala.collection.mutable.HashTable.findOrAddEntry$(HashTable.scala:167)
213 	at scala.collection.mutable.HashMap.findOrAddEntry(HashMap.scala:44)
214 	at scala.collection.mutable.HashMap.put(HashMap.scala:126)
215 	at scala.collection.mutable.HashMap.update(HashMap.scala:131)
216 	at bdd.BinaryOperation.memoize(BinaryOperation.scala:74)
217 	at bdd.And$.apply(BinaryOperation.scala:117)
218 	at bdd.BinaryOperation.bddOp(BinaryOperation.scala:100)
219 	at bdd.And$.$anonfun$apply$3(BinaryOperation.scala:117)
220 	at bdd.And$$$Lambda$7440/0x0000000842154440.apply(Unknown Source)
221 	at bdd.BinaryOperation.memoize(BinaryOperation.scala:74)
222 	at bdd.And$.apply(BinaryOperation.scala:117)
223 	at bdd.BinaryOperation.bddOp(BinaryOperation.scala:100)
224 	at bdd.And$.$anonfun$apply$3(BinaryOperation.scala:117)
225 	at bdd.And$$$Lambda$7440/0x0000000842154440.apply(Unknown Source)
226 	at bdd.BinaryOperation.memoize(BinaryOperation.scala:74)
227 	at bdd.And$.apply(BinaryOperation.scala:117)
228 	at bdd.BinaryOperation.bddOp(BinaryOperation.scala:100)
229 	at bdd.And$.$anonfun$apply$3(BinaryOperation.scala:117)
230 	at bdd.And$$$Lambda$7440/0x0000000842154440.apply(Unknown Source)
231 	at bdd.BinaryOperation.memoize(BinaryOperation.scala:74)
232 	at bdd.And$.apply(BinaryOperation.scala:117)
233 	at bdd.BinaryOperation.bddOp(BinaryOperation.scala:100)
234 	at bdd.And$.$anonfun$apply$3(BinaryOperation.scala:117)
235 	at bdd.And$$$Lambda$7440/0x0000000842154440.apply(Unknown Source)
236 	at bdd.BinaryOperation.memoize(BinaryOperation.scala:74)
237 	at bdd.And$.apply(BinaryOperation.scala:117)
238 	at bdd.BinaryOperation.bddOp(BinaryOperation.scala:98)
239 	at bdd.And$.$anonfun$apply$3(BinaryOperation.scala:117)
240 	at bdd.And$$$Lambda$7440/0x0000000842154440.apply(Unknown Source)
241 [error] [launcher] error during sbt launcher: java.lang.OutOfMemoryError: Java heap space
242 [error] [launcher] error during sbt launcher: java.lang.OutOfMemoryError: Java heap space
243 error running tests
244 ERROR: Job failed: exit code 1

Here is the text of my tests.

Do I really need to go into every test and call println("running test so-and-so") ?

Test case names are logged after completion, so it should be the test case following “coloring”, i.e. “europe”.

Any custom “epilogue” output would not survive the OutOfMemoryError, either, unless you explicitly catch Errors, which is rarely a good idea. I’d rather add proper detailed logging to the code under test.

no, the epilog would not survive, but the prolog would. Right? It is weird to me that such a stack trace does not tell me which test failed. OK OK OK, the test didn’t technical fail in the the sense of the unit test framework, but in a semantic sense my test failed.

BTW, here is a recipe for making tests print prolog/epilog.