Americium - Test cases galore! Automatic case shrinkage! Bring your own test style. For Scala and Java

Introducing Americium: https://github.com/sageserpent-open/americium

  1. Offers automatic shrinkage to a minimal or nearly-minimal test case. Yes, invariants are preserved on test case data.
  2. Shrinks efficiently.
  3. Offers direct reproduction of a failing, minimised test case.
  4. Covers finite combinations of atomic cases without duplication when building composite cases.
  5. Gets out of the way of testing style - doesn’t care about whether the test are pure functional or imperative, doesn’t offer a DSL or try to structure your test suite.
  6. Supports Scala and Java as first class citizens.
  7. Supports covariance of test case generation in Scala, so cases for a subclass can be substituted for cases for a supertrait/superclass.
  8. Supports covariance of test case generation in Java, so cases for a subclass can be substituted for cases for a superinterface/superclass.
  9. Allows automatic derivation of test case generation for sum/product types (aka case class hierarchies) in the spirit of Scalacheck Shapeless.

It’s in early stages - one look at the introductory documentation will confirm that, but it’s reached the point where I’m going to attempt to use it seriously on my own work.

If others would like to give it a whirl, I’d love to hear their feedback.

The latest artifact is mentioned here: https://mvnrepository.com/artifact/com.sageserpent/americium_2.13/0.1.14-trials-rc

  • which reminds me to add a nice version widget to the GitHub front page!

If you’re a Scalacheck user, then where you would start with a Gen.const, Gen.oneOf, Gen.frequency, you cutover to:

a) Getting a TrialsApi via Trials.api on the Trials companion object: val api = Trials.api

b) … then you hit that instance for your Trials … these are analogous to the Gen monad instances, so
api.only("One string and that's it").
api.choose(1, 3, -4),
api.alternateWithWeights(1 -> oneSortOfTrials, 5 -> aMorePopularSortOfTrials)

c) … then you apply a limit on how many cases you want and hit your own favourite test lambda with it:
myGrandTrialsInstance.withLimit(500).supplyTo{testCase => ........... }

What you put in that lambda is up to you - the point is that it either executes successfully, or throws an exception.

You can see a simple example of usage at the top here:

https://github.com/sageserpent-open/americium/blob/master/src/test/scala/com/sageserpent/americium/Examples.scala

There are some Java smoke tests here too: https://github.com/sageserpent-open/americium/blob/master/src/test/scala/com/sageserpent/americium/java/TrialsApiTests.java

Hope you have fun with it!

3 Likes

I forgot to mention - there are practically infinite streamed cases too, these are analogues to the Arbitrary.arbitrary* from Scalacheck:

api.integers // Arbitrary.arbInt
api.strings // Arbitrary.arbString
api.instants // There's one out there somewhere, or you rolled your own by mapping....

api.stream(myTransformThatTakesALongAndYieldsSomeCase) // For when you want to write one from scratch and control the definition of shrinking for your type.

A brief final update for now:

  1. I have added some more documentation, I hope this makes it clearer what this is about and how to use it.
  2. I did some more dog-food testing of the Java API and have streamlined it a little so that it is more on par with the Scala API.
  3. Improved the test case shrinkage after making the unit tests for this more aggressive.

The latest version is here: https://mvnrepository.com/artifact/com.sageserpent/americium_2.13/0.1.17

May your failing test cases be minimal, and quick to reproduce…

1 Like