Stale symbol defined in Period(2.1-7), is referred to in run Period(3.12)

Question
1

In my fork of airframe surface, I have implemented a change in which I store type arguments for type aliases (both normal and opaque). The intended use is to be able to inspect type arguments of opaque type NativeArrayImpl[T] = org.mozilla.javascript.NativeArray.

The whole thing works fine in tests, but when I publish the library and use it from my application, I get error about stale symbols:

dotty.tools.dotc.core.Denotations$StaleSymbolException: stale symbol; class xxxx#7613 in module class xxx, defined in Period(2.1-7), is referred to in run Period(3.12)

What does this mean and how can I prevent it?

The code triggering the issue is:

    private def extractSymbol(t: TypeRepr) =
      val dealiased = t.dealias

      //println(s"dealiased ${dealiased.show} of ${t.show}")
      //println(s"  maybeOwner.declarations ${t.typeSymbol.maybeOwner.declarations.map(_.name).mkString(",")}")

      if t != dealiased then
        //println(s"dealiased as $dealiased")
        surfaceOf(dealiased)
      else
        // t.dealias does not dealias for path dependent types, so try to find a matching inner type
        val symbolInOwner = t.typeSymbol.maybeOwner.declarations.find(_.name.toString == t.typeSymbol.name.toString)
        symbolInOwner match
          case Some(sym) =>
            //println(s"Match 1 $sym in $symbolInOwner as ${sym.typeRef.show}")
            surfaceOf(sym.typeRef.dealias)
          case _ =>
            //println(s"Match 2 $symbolInOwner as ${t.simplified.show}")
            surfaceOf(t.simplified)

    private def extractTypeArgs(t: TypeRepr): Expr[Seq[Surface]] = {
      //println(s"extractTypeArgs of ${t.show}: ${typeArgsOf(t).map(_.show).mkString("[", ",", "]")}")
      Expr.ofSeq(typeArgsOf(t).map(s => extractSymbol(s)))
    }

More complete callstack follows:

dotty.tools.dotc.core.Denotations$StaleSymbolException: stale symbol; class DynamicWalkParameters#7613 in module class anim, defined in Period(2.1-7), is referred to in run Period(3.12)
	at dotty.tools.dotc.core.Denotations$SingleDenotation.staleSymbolError(Denotations.scala:963)
	at dotty.tools.dotc.core.Denotations$SingleDenotation.bringForward(Denotations.scala:761)
	at dotty.tools.dotc.core.Denotations$SingleDenotation.toNewRun$1(Denotations.scala:808)
	at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:879)
	at dotty.tools.dotc.core.Symbols$Symbol.recomputeDenot(Symbols.scala:124)
	at dotty.tools.dotc.core.Symbols$Symbol.computeDenot(Symbols.scala:118)
	at dotty.tools.dotc.core.Symbols$Symbol.denot(Symbols.scala:109)
	at dotty.tools.dotc.core.Symbols$.toDenot(Symbols.scala:556)
	at dotty.tools.dotc.core.Substituters$.substThis(Substituters.scala:112)
	at dotty.tools.dotc.core.Types$Type.substThis(Types.scala:1921)
	at scala.quoted.runtime.impl.QuotesImpl$reflect$TypeReprMethods$.memberType(QuotesImpl.scala:1871)
	at scala.quoted.runtime.impl.QuotesImpl$reflect$TypeReprMethods$.memberType(QuotesImpl.scala:1867)
	at org.opengrabeso.airframe.surface.CompileTimeSurfaceFactory$Session.methodArgsOf$$anonfun$2$$anonfun$1(CompileTimeSurfaceFactory.scala:520)
	at scala.collection.immutable.List.map(List.scala:251)
	at org.opengrabeso.airframe.surface.CompileTimeSurfaceFactory$Session.methodArgsOf$$anonfun$2(CompileTimeSurfaceFactory.scala:520)
	at scala.collection.immutable.List.map(List.scala:247)
	at org.opengrabeso.airframe.surface.CompileTimeSurfaceFactory$Session.methodArgsOf(CompileTimeSurfaceFactory.scala:518)
	at org.opengrabeso.airframe.surface.CompileTimeSurfaceFactory$Session.org$opengrabeso$airframe$surface$CompileTimeSurfaceFactory$Session$$methodParametersOf(CompileTimeSurfaceFactory.scala:553)
2

This kind of issues typically occur in some combinations of lazy-vals + macros/inlines (given is also a lazy val) and sometimes opaque types. Unfortunately these are typically are to find, but when having the source code it should be possible to minimize them.
I can try to check it out later if I’ll have a free time.

I can see that you’re using Scala 3.3.6 LTS in the mentioned project, are you also using the same version when testing the published library?
Have you tried it as well with Scala Next versions (e.g. 3.6.4 or 3.7.1)? AFAIR most of fixes for stale symbols were backported to the LTS, but maybe some fixes could not be backported without braking the compatibility.

3

It occurred to me it might perhaps be an issue which was fixed in later compiler version, but compiling the library with 3.7.1 did not change anything. The application using this is compiled by 3.7.1

I am not sure: is this a bug in the library, or in the compiler?

I am not sure how difficult would it be to reproduce this in a open source project. While the library itself is open-source, the application is not and I am not sure how much work would it be to extract the repro or create a fresh one.

4

I am not sure: is this a bug in the library, or in the compiler?

Probably in compiler, but it would be hard to reproduce it without your help on minimising the closed source part. It must be using some special combination of features that’s crashing the compiler. We can deal with the public part on our side later.

I assume that mentioned class DynamicWalkParameters is defined in the closed source application. I’m not sure if the crash is triggered when compiling this class or when referring to it , possibly from different compilation unit (from test or different module).
You can start with eliminating different parts of the code base (or replacing with ???) to see when it successfully compiles again. Based on that it you should be able to at some point find some minimal code that triggers the crash, on which we can work on the compiler side.
You can also track which classes reference this class (possibly transitivelly), you can comment these usages out to limit the scope. As soon as you’ll have only 1 path leading to triggering error it should be easier to follow.

5

I have tried reducing my 500 files project in such 150 I hoped will be relevant, but the issue disappeared. A more careful and gradual approach will probably be needed for the repro, which I am not sure when I will be able to do.

It seems the classes mentioned in the error message (DynamicWalkParameters and Turn) are not much relevant to the issue, as they were almost untouched by the reduction.

Those classes themselves are actually pretty straightforward, they more or less contain just primitives and other simple types. They are more like random bystanders to an issue which is caused by something else.

6

Sadly I can’t offer any help, other than perhaps this as inspiration: Jane Street Blog - A Higgs-bugson in the Linux Kernel.

Sometimes you just have to keep trying…