Can type erasure cause run-time type error?

Question
1

The code example has a run-time type error on the line I’ve indicated.

Do I understand correctly that type erasure allows code to compile which is not type-safe?

// this is a type error pairList(li) returns List[((B,B),(B,B))] not List[(B,B)]

The run-time error exhibits when I run it in IntelliJ.

mList=List((1,2), (3,4), (5,6), (7,8), (9,10), (11,12), (13,14), (15,16), (17,18), (19,20), (21,22), (23,24), (25,26), (27,28), (29,30), (31,32), (33,34), (35,36), (37,38), (39,40), (41,42), (43,44), (45,46), (47,48), (49,50), (51,52), (53,54), (55,56), (57,58), (59,60), (61,62), (63,64), (65,66), (67,68), (69,70), (71,72), (73,74), (75,76), (77,78), (79,80), (81,82), (83,84), (85,86), (87,88), (89,90), (91,92), (93,94), (95,96), (97,98), (99,100))   leftover=None
li=List((1,2), (3,4), (5,6), (7,8), (9,10), (11,12), (13,14), (15,16), (17,18), (19,20), (21,22), (23,24), (25,26), (27,28), (29,30), (31,32), (33,34), (35,36), (37,38), (39,40), (41,42), (43,44), (45,46), (47,48), (49,50), (51,52), (53,54), (55,56), (57,58), (59,60), (61,62), (63,64), (65,66), (67,68), (69,70), (71,72), (73,74), (75,76), (77,78), (79,80), (81,82), (83,84), (85,86), (87,88), (89,90), (91,92), (93,94), (95,96), (97,98), (99,100))   maybeB=None
li=List(((3,4),(1,2)), ((7,8),(5,6)), ((11,12),(9,10)), ((15,16),(13,14)), ((19,20),(17,18)), ((23,24),(21,22)), ((27,28),(25,26)), ((31,32),(29,30)), ((35,36),(33,34)), ((39,40),(37,38)), ((43,44),(41,42)), ((47,48),(45,46)), ((51,52),(49,50)), ((55,56),(53,54)), ((59,60),(57,58)), ((63,64),(61,62)), ((67,68),(65,66)), ((71,72),(69,70)), ((75,76),(73,74)), ((79,80),(77,78)), ((83,84),(81,82)), ((87,88),(85,86)), ((91,92),(89,90)), ((95,96),(93,94)), ((99,100),(97,98)))   maybeB=None
Exception in thread "main" java.lang.ClassCastException: scala.Tuple2 cannot be cast to java.lang.Integer
	at scala.runtime.BoxesRunTime.unboxToInt(BoxesRunTime.java:103)
	at scala.runtime.java8.JFunction2$mcIII$sp.apply(JFunction2$mcIII$sp.java:23)
	at treereduce.TreeParallelReduce$.$anonfun$pairMapReduce$1(TreeParallelReduce.scala:95)
	at scala.collection.immutable.List.map(List.scala:286)
	at treereduce.TreeParallelReduce$.recur$1(TreeParallelReduce.scala:95)
	at treereduce.TreeParallelReduce$.pairMapReduce(TreeParallelReduce.scala:113)
	at treereduce.TreeParallelReduce$.main(TreeParallelReduce.scala:123)
	at treereduce.TreeParallelReduce.main(TreeParallelReduce.scala)

Process finished with exit code 1

In Scastie, I get the following warnings, but not errors, which I don’t understand the meaning of.

non-variable type argument (B, B) in type pattern List[(B, B)] (the underlying of List[(B, B)]) is unchecked since it is eliminated by erasure

abstract type B in type pattern Option[B] is unchecked since it is eliminated by erasure
2

When I replace pairList(li) with pairList(reduced) the code runs as expected. And some (but not all) the warnings go away.

3

Yes – this is the heart of erasure, and is why it’s important to pay attention to those warnings.

The summary is that, at runtime, there is no difference to the code between List[((B, B), (B, B))] and List[(B, B)] – the type parameters get lost. At runtime, they are basically both just List. So any code that depends on the difference at runtime is inherently very dangerous, and more often than not broken.

This shows up most often in pattern matches like this:

foo match {
  case List[Int] => // do first thing
  case List[String] => // do second thing
}

Those are the same thing at runtime, so the match is broken – it will always match on the first, even if the value is actually a List[String]: the system can’t tell the difference. It will compile, but generate a warning like the ones you are seeing.

That’s what “erasure” means: the type parameters get erased from existence during compilation, and aren’t there at runtime. (And this is why typeclasses are more powerful than doing things at runtime – since typeclasses are computed at compile time, when we still have full information, they let you avoid these problems.)

1 Like
4

Yes, I understand that they are the same at run-time? But are they the same also at compile time? Isn’t it possible to statically determine in this case that the code is not correct in terms of types?

5

Basically, when you have

case bList: List[B] => …

this will match any List, because pattern matching is a runtime check, and at runtime, there is only List. That the List is actually a List[B] is an assertion that the code makes that the compiler cannot verify (On the other hand, the compiler will complain if the matchee clearly cannot be a List[B]). If it turns out to be false, you usually get a runtime error once you try access an element. This is a hole in Scala’s type safety that is hard to close without making things awkward.

1 Like
6

Depends on what you’re doing. A lot of stuff does get checked at compile time, but some operations (like match) are inherently runtime-centric.

Mind, if you think that the compiler should be able to reject this with an error, and it isn’t doing so, it can be worth examining your code – sometimes that can be an indication that your types are looser than you think, and you aren’t giving the compiler enough proof to work from. For example, the compiler will generally reject matches that are clearly impossible. But you have to provide it with enough to go on for that, and there are limits to how far it can statically reason…

1 Like
7

For example, the compiler will generally reject matches that are clearly impossible .

Generally, a compiler should reject anything for which is room to fail. Otherwise we would deal with dynamically typed language where there exists a case which would not fail and that’s enough for us to assume it will work.

1 Like
8

What would you propose? Making such matches illegal? Any workaround?

9

Great, I’ll take a harder look at my code.

10

Use -Yfatal-warnings compiler flag. Problem solved :slight_smile:

2 Likes
11

Given below code:

def method[B](s: List[B]): Unit = {
  val s0: List[String] = s // error
  val (s1: List[String], o1) = s -> None // warning
  val (s2 @ List(_*), o2) = s -> None // ok
  val (s3 @ List(_*), o3) = s.toVector -> None // warning + error
}

it’s a little weird that line val s0 is rejected, while line val (s1, o1) just produces a warning.

However, if you want to just ensure you have a List you can use the third option val (s2, o2). This keeps inferred generic parameter type, but compiler will reject your code in more situations, e.g. as in val (s3, o3) line.

12

I’ve seen language newcomers ignore these erasure warnings too many times — and I think it sometimes happens to more experienced Scala coders, too.

The warnings are easy enough to spot in an otherwise warning-free build, but in practice, there are a lot of builds out there that are noisy with many warnings, and then it’s easy to fail to notice when you’ve added one more.

Personally, I think those erasure warnings should be errors. And without having to enable -Xfatal-warnings, either. The user should have to explicitly add an @unchecked annotation to force compilation to succeed.

Perhaps a pull request that made this change to the compiler would be accepted.

6 Likes
13

Here is a case where I get erasure warnings from the compiler. Am I wrong when I simply ignore these warnings? Or should I remove the declarations from my code which the compiler is complaining about? In my opinion the declarations which the compiler is unable to check, helps me understand the code.

  trait Pairable[F[_]] {

    def map[A, B](p: F[A], f: A => B): F[B]

    def foldLeft[A, B](p: F[A], z: B)(f: (B, A) => B): B

    def paired[A, B](tree: F[A], f: A => B): (List[(B, B)], Option[B]) = {
      val none: Option[B] = None
      val nil: List[(B, B)] = Nil
      foldLeft(tree, (nil, none)) {
        case ((stack: List[(B, B)], Some(b:B)), a: A) => (((b, f(a)) :: stack), none)  // line 44, compiler warns about b:B and a:A
        case ((stack: List[(B, B)], None), a: A) => (stack, Some(f(a))) // line 45, compiler warns about a:A
      } match {
        case (stack, leftover) => (stack.reverse, leftover)
      }
    }
  }

Warning:(44, 44) abstract type pattern B is unchecked since it is eliminated by erasure
        case ((stack: List[(B, B)], Some(b:B)), a: A) => (((b, f(a)) :: stack), none)
Warning:(44, 52) abstract type pattern A is unchecked since it is eliminated by erasure
        case ((stack: List[(B, B)], Some(b:B)), a: A) => (((b, f(a)) :: stack), none)
Warning:(45, 47) abstract type pattern A is unchecked since it is eliminated by erasure
        case ((stack: List[(B, B)], None), a: A) => (stack, Some(f(a)))
14

Note that you can match against these types correctly when requiring a ClassTag. In your case, it doesn’t actually change the match, as the types are always the ones given, and in my opinion the type ascriptions don’t help readability in this case. It’s a reasonably short method and the variable names are the same as the type names, which is so common in such generic methods, that I’d assume a to be of type A anyways.

1 Like
15

The problem is that when you do type ascription in destructuring in pattern match you’re effectively trying to check for those types - that’s by design of Scala. Since the types are erased, compiler warns you that checking is impossible.

In other words, type ascriptions:

  • in destructuring are converted to dynamic isInstanceOf checks (and then asInstanceOf casts) at runtime which are unsafe in case of type erasure (lack of generic type information at runtime)
  • outside of destructuring are not converted to isInstanceOf checks and instead are safely checked statically

Below code:

x match {
  case (fst: FstType, snd: SndType) => consume(fst, snd)
}

is mostly equivalent to:

x match {
  case tmp$pair1: Tuple2[FstType, SndType] =>
    val (fst, snd) = tmp$pair1
    consume(fst, snd)
}

and since generic types are erased this is equivalent to:

x match {
  case tmp$pair1: Tuple2[_, _] =>
    val fst = tmp$pair1._1.asInstanceOf[FstType]
    val snd = tmp$pair1._2.asInstanceOf[SndType]
    consume(fst, snd)
}

Generally if you want to add explicit types to help you understand the code but don’t want the “unchecked” compiler warnings, then you must move the type ascriptions out of destructuring statements in case patterns to a piece of code with no destructuring.

For example you can change your code:

      foldLeft(tree, (nil, none)) {
        case ((stack: List[(B, B)], Some(b:B)), a: A) => (((b, f(a)) :: stack), none)  // line 44, compiler warns about b:B and a:A
        case ((stack: List[(B, B)], None), a: A) => (stack, Some(f(a))) // line 45, compiler warns about a:A
      }

to:

      foldLeft[A, (List[(B, B)], Option[B])](tree, (nil, none)) {
        case ((stack @ List(_*), Some(b)), a) => (((b, f(a)) :: stack), none)
        case ((stack @ List(_*), None), a) => (stack, Some(f(a)))
      }

or:

      foldLeft(tree, (nil, none)) {
        case ((stack, Some(b)), a) => (((b: B, f(a: A)) :: (stack: List[(B, B)])), none)
        case ((stack, None), a) => (stack: List[(B, B)], Some(f(a: A)))
      }

or you can even do both changes at the same time.

1 Like