Supporting cross-compilation and repository organization pros and cons

Design
1

The most typical way of managing many release lines for projects I worked on was to keep seperate branches in the same project. This means that backports and forward-ports are relatively easy. The downside is that, in order to publish several versions at once requires manually switching branches and executing separate builds.

However, in terms of supporting both Scala 2 and Scala 3, most libraries I have seen seemed to decide on an approach with three source roots: classes source compatible with both, plus one for each Scala epoch version.

I am an sbt noob, but I guess it should be pretty easy to evolve the former, traditional approach by having custom code in build.sbt which does all the branch juggling magic. If not sbt, then Bazel most likely could.

For projects where the differences are expected to be very minor, there is also a third path, which currently seems very attractive to me: use some sort of source generation tool to generate both Scala 2 and Scala 3 files for the compilator from a single source file (or a single one, if they turn out to be equal). I have an idea for such a preprocessor which I started to implement, but I am still not sure how a repo should be organized to make the most of this approach, as source code file edited by the programmer is not the source file actually used by the compiler in that case. In particular, I am concerned about IDE integration. Unless I write specific plugins for each IDE, and that is much larger effort in terms of barrier of empty than the preprocessor, what IDE should consider as source file in terms of navigation should not be the actual source file for the build tool, so I don’t know if importing a project from build configuration files would work at all.

However, this problem seems very similar to what projects with separate source roots for different versions face, as they have to be prepared to have two versions of the same class. How do you manage it for qol?

I’d like to hear your insight on the matter, the reasons you chose the approach that you did, and your experience with it.

2
  1. Releases are often tied to tagging - if one were to tag several branches to release the same version but for different Scala it makes the proces more complicated
  2. Before the release one has to somehow test that code for each Scala for the same version behaves the same way - if any such change would require a separate merge between branches it would quickly get very messy
  3. The development where each Scala version would live on a separate branch would require constant swapping between branches, recompiling everything from scratch every time the version would be changed
  4. Meanwhile, differences between Scala versions and even JVM/JS/native can usually be extracted to a few mix-in traits Dat jas to differ - everything else stays the same

So cross-compilation within the same build is just simpler to maintain.

sbt-cross-project was the old solution relying on using “++ command” to run the same command for several versions of Scala, but it still needed to reload build on each version change.

But nowadays there is sbt-projectmatrix (built-in sbt 2.0) which generates separate project for each Scala version and platform.

Both allow sharing the code and just add some directories which are considered only by one Scala version/one platform. When running CI one has a lot pf freedom how one would aggregate the test and releasing a new version on a git tag - with multiple modules for multiple Scala versions on multiple platforms - is rather trivial. (Or at least it’s not much harder than releasing them on just one Scala/platform.)

1 Like