OK, this is just my non-expert impression, take it for what it is worth.
The danger is that the language allows me to do lots of things with case classes which conventional wisdom, but not the compiler, discourage. This gives me the impression that case classes form a brittle abstraction. Some things are fine to do with case classes, and some things are dangerous, and nothing inherent in the system indicates which are which. For example, I’ve been overriding equal and hashCode in case classes for a while now, without ever encountering a problem. And lo and behold, it turns out to be a dubious behavior which is predicted to bite me at some unexpected point in the future.
Another danger with case classes is programmers can use them blindly without understanding how they work. For example, I have never written an unapply method and in particular never have I written a var-args unapply method and I don’t even know how to do it. The case class just magically does it for me. This is fine except that once the programmer accidentally or intentionally ventures beyond the standard/supported use case, things apparently break with no warning. The only solution seems to be that in order to use case class you must intimately understand how they work.