Updated to scala 3.4.0 and now getting compiler crash

The error is

error when pickling tree 
  io.qt.core.QObject#Signal1[
    <error The match type contains an illegal case:
    case a | Null => a
(this error can be ignored for now with `-source:3.3`)>
    ]

It seems with 3.4.0, using -Yexplicit-nulls makes the compiler really angry. Has anyone encountered this?

EDIT: actually, taking away that compiler setting still triggers this :frowning:

The compilation error is probably due to changed match types specifications for the SIP-56, see SIP-56: Better foundations for match types by sjrd · Pull Request #18262 · lampepfl/dotty · GitHub, however it should have been a regular compilation error instead of a crash. If you’d be able to provide some more context, are a minimimization it would really help us to fix the underlying issue.

If you don’t need to upgrade to 3.4.x for some new important feature, it’s recommended to stay on 3.3.x LTS line for the best stability - most of fixes and improvements are back-ported there.

1 Like

Minimization is what I’d love to do, but the project is quite large and has too many parts. I wouldn’t even know where to start.

There are only 2 match types in the entire codebase. One being

  type VarToSignalType[T] = T match {
    case Int => java.lang.Integer
    case Long => java.lang.Long
    case Float => java.lang.Float
    case Double => java.lang.Double
    case Boolean => java.lang.Boolean
    case a | Null => a
    case _ => T
  }

(because I was using explicit-nulls)
and

    type AndThen[t <: Tuple, u <: Tuple] <: Tuple = (t, u) match {
      case (t1, Tuple1[u1]) => Tuple.Concat[t, u]
      case _ => Tuple.Concat[t, Tuple1[u]]
    }

the project is split in several submodules, the first type match I listed is in core, which compiles fine and allows the javax.swing submodule to compile as well, so I don’t think it’s that one, and the tuples one is… well it looks fairly trivial to me.

Is there any chance it’s the bytecode from the qt library that’s crashing the compiler?

The error points to first match type. You can try to compile it with following scalac options for more context: -Ydebug -Ydebug-error -Ydebug-unpickling. My initial though is that maybe it fails during macros, but I’m not sure.

@sjrd I wasn’t aware that case a | Null => or case Nullable[a] => where type Nullable[T] = T | Null is illegal match type. Can we somehow workaround this?

I ran it with those settings, the only difference with the previous output is just stack traces. There are no macros involved, unless simple inlines methods count, there’s only one:

/** Unsafe not null. Casts away nullity without a check */
extension [T](e: T | Null) inline def unn: T = e.asInstanceOf[T]

In the end though, I think the explicit-null experiment died, it failed as it is but provided insight for future revisions, but it doesn’t feel like the compiler team is maintaining it so I’ll move away from it. It’s a pity because I was working with these null-heavy APIs that are Qt and Swing.

If possible you can publish the stacktraces (maybe as gist/file), I assume they don’t contain any project related data, instead it should probably only contain compiler stacktraces. These would be really useful to track the issue and find the couse.

This shouldn’t compile in the first place, because of the case a | Null => which is indeed illegal under SIP-56. It’s likely that the compiler crashes later because this passed without an error.

Match type is not the correct way to remove | Null from a type. You have to use trick in your reply.

Explicit nulls is an experiment feature still under development and improving. Currently we are forcusing on refining the migration tool (unsafe nulls) and interoperation layer (flexible types).

You can check the latest progress at add flexible types to deal with Java-defined signatures under -Yexplicit-nulls by olhotak · Pull Request #18112 · lampepfl/dotty · GitHub .

type IllegalMatch1[T] = T match {
  case Int => java.lang.Integer
  case a | Null => a
  case _ => T
}

type IllegalMatch2[T] = T match {
  case a | Null => a
  case _ => T
}

I find only IllegalMatch2 will report the error at definition site. Is this intended behaviour?

No, both should error at definition site.

Oh, thanks. A follow up question, Scala defines

def nn[T](x: T | Null): x.type & T

and that works (?), that’s where I based that match case on. I guess match type behaves differently to a value case. I have a hard time developing an intuition for how match type work. Today I was trying to do

type BufferType[Buff] = Buff match {
  case StructBuffer[a, ?] => a
} 

and pass BufferType[org.lwjgl.vulkan.VkSurfaceFormatKHR.Buffer], where
VkSurfaceFormatKHR.Buffer is defined like

class Buffer extends StructBuffer[VkSurfaceFormatKHR, Buffer]

but this also doesn’t reduce (can’t prove it’s disjoint either).

What trick?

That works because it’s based on type inference, which chooses one T at each call site, and stores it once and for all in TASTy. So if a later version of the compiler would choose a different T, it’s not an issue. For match types, all compilers must compute the same captures forever.

I don’t really have enough context here to determine what is happening.

1 Like

That works because it’s based on type inference, which chooses one T at each call site, and stores it once and for all in TASTy

Ah, gotcha, thanks.

Would the following scala-cli script help?

//> using scala 3.2.2
//> using dep org.lwjgl:lwjgl-vulkan:3.3.3

type BufferType[Buff] = Buff match {
  case org.lwjgl.system.StructBuffer[a, ?] => a
}

summon[BufferType[org.lwjgl.vulkan.VkSurfaceFormatKHR.Buffer] =:= org.lwjgl.vulkan.VkSurfaceFormatKHR]

That outputs

[error] Cannot prove that testMatchType$_.this.BufferType[org.lwjgl.vulkan.VkSurfaceFormatKHR.Buffer] =:= org.lwjgl.vulkan.VkSurfaceFormatKHR.
[error] 
[error] Note: a match type could not be fully reduced:
[error] 
[error]   trying to reduce  testMatchType$_.this.BufferType[org.lwjgl.vulkan.VkSurfaceFormatKHR.Buffer]
[error]   failed since selector  org.lwjgl.vulkan.VkSurfaceFormatKHR.Buffer
[error]   does not match  case org.lwjgl.system.StructBuffer[a, _] => a
[error]   and cannot be shown to be disjoint from it either.
[error] summon[BufferType[org.lwjgl.vulkan.VkSurfaceFormatKHR.Buffer] =:= org.lwjgl.vulkan.VkSurfaceFormatKHR]

I think it goes back to what you said first. I’m not sure how to build an intuition of how to reason about match types :sweat_smile: . I Guess I’ll go read that SIP.

Thanks again.