Context
Consider a sealed trait that has further sealed traits as inheritors. For example:
package example
sealed trait Number
sealed trait OddNumber extends Number
case object Zero extends Number
case object One extends OddNumber
When implementing typeclass derivation in Scala 2 using shapeless, the base trait (Number) would be “flattened” to all of its concrete inheritors (ignoring all abstract inheritors) because that’s how the shapeless macros were implemented - see One and Zero in the output below:
scala> import shapeless._
import shapeless._
scala> Generic[example.Number]
val res0: shapeless.Generic[example.Number]{type Repr = example.One.type :+: example.Zero.type :+: shapeless.CNil} = shapeless.Generic$$anon$1@ff1a5a0
In Scala 3 using the scala.derivation package, such traits are not “flattened” automatically - the MirroredElemTypes for the base trait will only contain the direct inheritors - see OddNumber and Zero below:
scala> import scala.deriving.*
scala> summon[Mirror.SumOf[example.Number]]
val res0:
scala.deriving.Mirror.Sum{
type MirroredMonoType = example.Number; type MirroredType = example.Number;
type MirroredLabel = "Number";
type MirroredElemTypes = (example.OddNumber, example.Zero.type);
type MirroredElemLabels = ("OddNumber", "Zero")
} = anon$1@d70dbeb
If one is not aware of that difference, this can lead to bugs when porting typeclass derivation mechanisms from Scala 2 to 3 (e.g. derived JSON codecs changing their encoding).
Question:
Is there a generic / general solution for Scala 3 (e.g. a library or a language feature) supporting derivation in the “flattened” manner?
(Of course, there are various ways how to mitigate this for individual derivation mechanisms and I implemented such mitigations before - what I’m looking for is a standard solution, e.g. an alternative Mirror for Sum with “flattened” members similar to the behavior in Shapeless 2.)
1 Like
If I were in your position, I would push this problem off Mirror and on to the recursive decomposition of the type hierarchy that you would have to implement yourself, as per: Type Class Derivation.
So your typeclass deriving thingummajig that uses the mirror to get the first level of sum types has to recursively summon the typeclass for OddNumber, in which case it’s invocation will presumably use a new mirror that will dig out One, because it should jump from one sealed hierarchy to another - or so I naively hope.
Did you try that already, or are you just thinking about how to approach the problem?
I’m curious, does Magnolia already automate this enough to avoid having to hand-roll the solution via scala.deriving?
EDIT: If you’re ultimately trying to write JSON en/decoders, are Circe et al not an option?
Thanks for your response!
Did you try that already, or are you just thinking about how to approach the problem?
I did that already for several and succeeded for the most part - that’s what I meant with “ways how to mitigate this”. But for me, doing that was non-trivial enough to think “someone should come up with a general solution”. So my question is not so much “how can I solve this problem?” but rather “could it make sense for me to try to come up with a general solution or has someone (probably more clever than me) done that already?”
Some of the reasons why I’d prefer to have such a “flattened” Mirror:
- there are edge cases that make this less trivial than one might think. For example, diamond inheritance can lead to the same inheritor occurring multiple times while recusively resolving the
Mirror.SumOf of the base trait (which, depending on the nature of your typeclass, might raise the need to filter out the duplicates).
I’d rather have those covered once and then not worry about them again.
- if I had something like a
type FlattenedElemTypes = (example.One.type, example.Zero.type), I could use certain optimizations to reduce the amount of recursive inline calls
- I’d expect my derivation code to become much cleaner
I’m curious, does Magnolia already automate this enough to avoid having to hand-roll the solution via scala.deriving?
Magnolia should be fine for most use cases, but I’m not sure about the cases where the difference in handling nested sealed traits matters. I will check that.
I did check shapeless 3 a while ago, but that uses “non-flattened” mirrors just like vanilla Scala 3.
EDIT: If you’re ultimately trying to write JSON en/decoders, are Circe et al not an option?
JSON was just an example and circe is fine (there, the bugs related to the problem above have already been fixed a good while ago). What I’m actually working on is derivation of scalacheck typeclasses and that is mostly solved, but there’s still an open bug for diamond inheritance and I’d like to optimize the inlining behavior some more.
1 Like
I think I understand your use case now - the problem is not that you can’t recurse into the next level of hierarchy (because the classic decomposition of a product type will summon the typeclass for each variant, and that will implicitly cause recursive derivation over the next level via the same mechanism that decomposed the top-level product type).
What you seem to want based on that bug report is to jump ahead of the implicit recursion and preemptively flatten all levels of the hierarchy to avoid repetition of the leaf types in the composed generators, so that the tree of types is turned into a DAG. In other words, it’s specific to your problem domain (and expectations) with Scalacheck.
Could you recursively summon the mirrors for the product element types and bundle up the leaves into a set to avoid duplication?
Anyway, you have a bigger problem than that - you are working with Scalacheck and not Americium. That will only lead to pain and misery. Sorry, couldn’t resist the chance to advertise. 
Here’s how Americium uses Magnolia, if that’s of any use: Scala 2.13, Scala 3.
I’m not so fussed about the DAG versus tree thing regarding frequencies of the leaf type generators, but for all I know Magnolia may be preemptive in the way you are looking for…
Now you’ve got me curious: how would Americium fare?
import com.sageserpent.americium.Trials.api
import com.sageserpent.americium.{Factory, Trials}
sealed trait SealedDiamond
object SealedDiamond:
sealed trait SubtraitA extends SealedDiamond
sealed trait SubtraitB extends SealedDiamond
case class FooLaLa(value: Int) extends SealedDiamond
case class BwaHaHa(value: Int) extends SubtraitA with SubtraitB
case object Foo extends SealedDiamond
case object Bar extends SubtraitA with SubtraitB
end SealedDiamond
val gems: Trials[SealedDiamond] = summon[Factory[SealedDiamond]].trials
import SealedDiamond.*
given Factory[Foo.type] = Factory.lift(api.only(Foo))
given Factory[Bar.type] = Factory.lift(api.only(Bar))
gems.withLimit(20).supplyTo(println)
As Americium already deduplicates the generated test cases, I extended the original example from the bug report with some case classes alongside the original case objects.
The output is:
FooLaLa(-54163312)
Bar
BwaHaHa(2128104420)
Foo
FooLaLa(637157859)
BwaHaHa(1652254930)
FooLaLa(1987104774)
BwaHaHa(1265385001)
FooLaLa(203172046)
BwaHaHa(-2145422663)
BwaHaHa(1658077352)
FooLaLa(-2071825701)
BwaHaHa(-1428546316)
FooLaLa(-1545019583)
BwaHaHa(362036375)
FooLaLa(423603859)
FooLaLa(-1859034913)
BwaHaHa(-104195808)
BwaHaHa(-1292735007)
FooLaLa(251974104)
So equal frequencies of FooLaLa versus BwaHaHa, likewise both case objects occur just once.
1 Like
In other words, it’s specific to your problem domain (and expectations) with Scalacheck.
Well, there’s two sides: there is the specific problem I’m currently trying to solve (which is indeed Scalacheck-specific). But from my experience, it is just one of many (often minor) challenges one needs to solve when using scala.derivation related to nested sealed traits that one would not need to solve if there was a “flattening” variant of Mirror.SumOf or something similar.
Could you recursively summon the mirrors for the product element types and bundle up the leaves into a set to avoid duplication?
That seems like a good direction for solving this. However:
- if I put the mirrors into a set, I lose the compile-time information about the mirrored types and then cannot summon my typeclass instances for the products
- there’s no proper equality defined for Mirrors - two instances of a Mirror for the same type are not considered equal (at least according to a quick repl experiment)
What I could try is: summon the typeclass instances of the product element types and summon some runtime value representing the corresponding type that has proper equality defined and then use those values to filter out the duplicates (maybe scala.reflect.ClassTag could do the trick).
I’m curious, does Magnolia already automate this enough to avoid having to hand-roll the solution via scala.deriving?
Magnolia should be fine for most use cases, but I’m not sure about the cases where the difference in handling nested sealed traits matters. I will check that.
Turns out Magnolia does indeed “flatten” nested sealed trait (and does not produce duplicates for inheritance diamonds) as this combination of an example type class from the Magnolia examples and your SealedDiamond shows: Scastie - An interactive playground for Scala.
So here’s a “Yes, there is” as answer to my original question (“Is there a generic / general solution for Scala 3 (e.g. a library or a language feature) supporting derivation in the “flattened” manner?”): Magnolia.
2 Likes
In the end, for my current problem I went down that route because it was much simpler than migrating to Magnolia.
Turns out that scala.reflect.ClassTag is unsafe for that purpose (for simple enums, one can get equal ClassTags for different enum values), so I ended up with a simple macro to use the full name of the type (via scala.quoted.Type.show) to filter out the duplicates.
1 Like