Allow Experimental 0.1.0 - implementation-scoped access to Scala 3 `@experimental` APIs

Announce
1

I’ve released the first version of Allow Experimental, a small Scala 3 compiler plugin:

The motivation is to separate two ideas that Scala’s normal @experimental mechanism deliberately couples:

  • an API is experimental;

  • a consumer intentionally accepts the risk of using that API internally.

For example:

import scala.annotation.experimental
import io.github.dmytromitin.allowexperimental.allowExperimental

@experimental
def provider(): Int = 1

@allowExperimental
def allowed(): Int =
  provider()

def ordinaryCaller(): Int =
  allowed()

allowed may use the experimental API in its implementation, but callers of allowed do not themselves become experimental. A direct unmarked call to provider() still fails with the normal Scala experimental-use diagnostic.

The release currently supports exactly Scala 3.3.8, 3.8.4, and 3.9.0. The compiler plugin is full-crossed because it depends on compiler internals.

ThisBuild / scalaVersion := "3.9.0" // 3.3.8, 3.8.4

libraryDependencies ++= Seq(
  "com.github.dmytromitin" %% "allow-experimental-annotation" % "0.1.0" % Provided,
  compilerPlugin(("com.github.dmytromitin" % "allow-experimental-plugin" % "0.1.0").cross(CrossVersion.full)),
)

One practical use case is macro implementations: a public inline macro frontend can delegate to a private non-inline @allowExperimental implementation that uses an experimental compiler/reflection API, without leaking that requirement to downstream users.

The scope is intentionally conservative rather than a general replacement for scalacOptions += "-experimental". Public inline permission owners, experimental signatures/types, constructors, and several other placements remain unsupported.

0.1.0 is available from Maven Central and the release is at Github.

Feedback on the semantics, implementation approach, and useful real-world cases would be very welcome.

Cross-posted at Reddit

2

Hello, while a cool project, I would highly recommend against using it !

As you hinted at, there is a very good reason these things are coupled, it goes as follows:
Assume a library uses experimental feature F to do X

  • If your library requires F to implement X, then it must be experimental as well, since X will need to evolve as F evolves !
  • If your library can implement X without F, then you refactor it, experimental features are bound to change, and you should not rely on them for your stable library

This was a hard-learned lesson from Scala 2’s macros, which were experimental, but could be used by regular libraries, and so spread over the whole ecosystem.
Since some things were only possible using them, even changing them could require vast amount of code to need to be updated, and removing them entirely was impossible.
AFAIK the most difficult part in translating code from Scala 2 to Scala 3 is the macros.
(If I made any mistake in the above, feel free to correct me)

11 Likes
3

@Sporarum

Thanks for your opinion.

I share your concerns, and I agree that @experimental exists for a good reason. Abusing an escape hatch like this could recreate exactly the sort of ecosystem problem you describe.

At the same time, I think there are legitimate cases.

First, as context: even a widely used non-experimental library can make breaking changes, and migration can still be painful independently of @experimental as a language feature. Of course, stable libraries are expected to avoid that, while experimental APIs explicitly give weaker guarantees.

The distinction I have in mind is between an experimental dependency that becomes part of the effective public contract and one that is confined to an implementation detail.

For example, if an experimental method is used in one isolated place in a library, propagating @experimental through the public API can be painful for users, while making the whole library experimental may be impractical.

With @allowExperimental, the library author is explicitly taking responsibility for that particular implementation dependency: track changes to the experimental API, adapt that implementation when necessary, and verify it against every supported Scala version.

If the experimental thing leaks through public types, inline code/TASTy, inheritance, generated public API, etc., then I agree that hiding its experimentality would be a bad idea. The plugin intentionally does not support several such cases.

Cross-building is another example. A compiler API can be experimental in one supported Scala version and stable, changed, or replaced in another. Making the public API experimental merely because one implementation for one compiler lane needs that method may also be unnecessarily restrictive.

This is not only hypothetical: I currently use allow-experimental in Quasiquotes for Scala 3 for the implementation of tqr"...", which is cross-built for the supported Scala versions.

There are also already ways to bypass the restriction. For example, structural types can do it:

object Library:
  @experimental
  def provider(): Int = 1

object Bypass:
  def allowed(): Int =
    val structuralLibrary =
      Library.asInstanceOf[{ def provider(): Int }]
    structuralLibrary.provider()

object Consumer:
  def ordinaryCaller(): Int =
    Bypass.allowed()

So instead of writing a wrapper like that, one can state the intention explicitly:

@experimental
def provider(): Int = 1

@allowExperimental
def allowed(): Int =
  provider()

def ordinaryCaller(): Int =
  allowed()

This is not an argument that the safety mechanism is useless—just an observation that when a language restriction becomes sufficiently inconvenient, people tend to find ways around it.

There is a similar pattern with checked exceptions in Java: they were designed as a protective feature, but wrapping checked exceptions in unchecked exceptions became common when the propagation requirement was considered too intrusive.

So I don’t see allow-experimental as saying “experimental APIs are actually safe”. I see it as an explicit escape hatch: I know this implementation dependency is experimental, and I take responsibility for maintaining this particular use without forcing that status onto my callers.

I’m also going to add a warning to the README making this clearer: it should be used carefully and not as a general way to erase experimentality.

2 Likes
4

Thank you for your thorough reply !

Note that I specifically mentioned this case (“if your library can implement X without F”)
And I think it is not a valid argument: if it is truly an implementation detail, meaning there is 0 observable difference compared to a stable implementation, then there are strong reasons to use that stable implementation instead.
(And I suspect most cases would instead be something like “it has the same interface, but F actually makes it faster”, which we don’t want, because it means removing F will lead to potentially widespread performance regression !)

I can however 100% get behind the following:

Given this is one-way (a stable feature cannot become experimental[1]!), we know that if F becomes stable in one version, we don’t really need to stop people from using it in previous versions !
(But we should still warn in the case that the API/syntax changed between the current and stable version.)

And maybe we could add something to that effect to the compiler directly ?

Here is a very rough draft of how that would work:
If experimental feature F becomes stable in version X, then every actively maintained version < X gets a patch (c.f. SemVer) where F is either not experimental, or “soft-experimental” which is like experimental, but not contagious (name TBD)

  1. Or at least that would be equivalent to removing that feature, a big breaking change ↩︎

1 Like