Code sharing between Scala compilers' backends and their standard libraries


#1

AFAIR Scala from the start didn’t have it’s own VM or it’s own complete building toolchain and instead relies on compilation to .NET bytecode, Java bytecode, JavaScript or LLVM bitcode. Most of the time there we multiple active Scala compilers - at the beginning there were .NET and Java backends, then .NET backend died and after that Scala.js and Scala-Native were born.

Questions are:

  • how much of compiler code is shared between different backends: Scala.NET, Scala.JVM, Scala.js, Scala-Native?
  • how much of standard library code is shared between different backends: Scala.NET, Scala.JVM, Scala.js, Scala-Native?

Let’s assume a standard library means everything that is available out-of-the-box when starting with empty project. That would be ambiguous in JavaScript environment because it depends if we run inside of browser or Node.js, so consider only the overlapping part.

The main question (for which the answers for previous questions are also needed) is: how sure the Scala programmer can be that his code will compile on different backends and work correctly (have identical outcomes) without change? If there were multiple Scala language parsers in different Scala compilers then most probably they would differ in e.g. syntax parsing bugs, number of implemented features, pattern matching exhaustiveness checks, etc

P.S. I’m mixing “Scala compiler” with “compiler backend” because I’m not sure how to uniformly call elements of set ( Scala.NET, Scala.JVM, Scala.js, Scala-Native ).


#2

The alternate back ends are compiler plugins that run at a very late stage of compilation, so nearly all of the compilation pipeline is the same regardless of target. That’s how it’s possible for Scala.js and Scala Native to support the entire Scala language, including macros and other compiler plugins and so forth, without any compatibility worries. (But see https://www.scala-js.org/doc/semantics.html and http://www.scala-native.org/en/v0.3.8/user/lang.html for minor caveats.)

I haven’t answered all of your questions. Perhaps someone else can fill in with more.


#3

Thanks.

I’ve searched for comparison between compilation phases of Scala.JVM, Scala.js and Scala-Native and found that: https://stackoverflow.com/q/4527902

It seems that Scala.js and Scala-Native add some early compiler phases, namely jspretyper and jsinterop in Scala.js case and nativeinterop in Scala-Native case. Are they affecting any of the standard (i.e. not added by Scala.js or Scala-Native) compilation phases that execute later?

Also I recall that Scala.js has some extra compiler support for type unions in Scala 2.12. How it’s implemented? Which compilation phase takes care of them?


#4

I think the scala.js support for union types is pure library.


#5

That’s my understanding as well.

As a heavy Scala JVM/JS user, I’ve never in practice hit any differences in the language per se, and I rely on that. Scala is Scala.

OTOH, I’ve always taken for granted that you have to be quite careful about the libraries – some of the JVM types have been translated to JS, and the list is steadily growing, but there are still many missing. Much of it needs serious rewriting to work in the different environment.

So basically: no, you can’t take arbitrary JVM code, recompile it on JS and count on it working correctly. In some cases, it’s just plain impossible, since the environments are very different. (The most obvious example being anything that counts on being multi-threading such as Await – those cannot exist in the single-threaded JS environment.) And in a few cases there are slight differences due to pragmatic compromises. (Eg, the JS version of regex is slightly different from the JVM version.)

Mind, it’s pretty common for a well-written library, that isn’t doing anything weird, to cross-compile just fine. But you do have to do your homework about what library calls exist where…