Streamlining the Scala installation procedure

Question
1

One of our goals this year at the Scala Center is to streamline the distribution of Scala. I am opening this thread to get a picture as wide and accurate as possible about the current state of the installation procedure of Scala.

There are currently several ways to set up a Scala development environment, and I wonder if the one that we recommend on the getting started instructions is the best choice and, even if that’s the case, if there is anything we can do to make it even easier.

What I expect from the installation procedure is to:

  • work everywhere (Windows, Linux, macOS)
  • be simple (one step)
  • be complete (after the setup, I should be able to build any Scala project — that can use any version of Scala)

The current recommendation is to use the Coursier setup script, which addresses all those requirements. It installs a JVM if needed, as well as the most common command-line tools (notably, scala-cli — soon to be renamed to scala —, sbt, cs, and scalafmt), it works on all the operating systems, and it is just one command to invoke in a terminal (or one .exe to run, on Windows).

After the installation, you can use cs to manage the JVM to use, and switch to a more recent one or an older one, if needed by your projects.

Overall, I think this solution works very well, although I noticed a few issues that would be nice to fix:

  • on Linux and macOS you need to be careful about picking the right launcher for your architecture although it would be technically possible to have a single setup script that works for any *nix-like system (just like the sbt launcher works on any *nix-like system). The instructions would then be simplified to invoking curl https://scala-lang.org/setup.sh | sh (see coursier#2647)
  • updating the JVM does not work perfectly on Linux: e.g. in some cases you may still have the previous one on the PATH until the next reboot.

And you, what is your experience? How do you manage the installed JVM? What works well, and what does not work well, when a new colleague tries to set up their machine? What type of installer works best on Windows (currently, we distribute a .zip that contains a .exe)? What type of installer works best on macOS (e.g. Brew formula vs ad-hoc script)?

I am equally interested in hearing from people who do follow the recommended instructions, and those who don’t (in such a case, please explain why).

4 Likes
2

Currently, I have scala, scala-cli and sbt installed via Homebrew.

brew install \
  openjdk@17 \
  scala \
  sbt \
  Virtuslab/scala-cli/scala-cli

This is imperfect because as JVM developers, we end up needing multiple JDK versions, so SDKMan finds its way onto the system. Alternatively, you install the JDK from wherever, then use jenv. For getting started, however, installing an OpenJDK via Homebrew is good enough.

I never used the “Coursier setup script”. Someday I may use it; however, I consider executing scripts downloaded from the Internet, via curl, to be a security risk. And we do what we must, but limiting such instances is a good idea, IMO. Furthermore, I’m updating all my Homebrew installed packages via brew upgrade, which isn’t something I remember doing for any other channel.

Homebrew is surprisingly up-to-date. I am uncertain if I’d have the same preferences on top of an Ubuntu with outdated packages; however, I’d look for a Debian package repository first. And lacking that, I’d give Nix another shot, before going for Coursier scripts.

I may be too conservative, though. One of these days I’ll give cs another honest shot.

2 Likes
3

I use the Coursier setup script (using it on WSL2). Getting started is a breeze (especially if no JVM is installed, thus letting Coursier take care of everything), trying out different JVMs is very easy thanks to eval "$(cs java --jvm <VERSION> --env)" (though trying to set up the default JVM just doesn’t really seem to work), and all the Scala executables “just work” :tm: . It is very convenient for me to use when testing out apps in cloud environments where I have a “blank slate” Debian/Ubuntu and need to get up and running quickly. On my personal computer, I already enabled the “Scala-CLI as scala” feature and it’s very nice to work with.

All in all, cs setup suits my needs well enough to get up and running with Scala in a fast and painless way.

2 Likes
4

cs installs everything and does a good job managing JVMs and getting the environment set up correctly, which is really hard and took a long time to get right. It has made a huge difference for me personally because I use it to distribute command-line Scala apps at work, and I have spent zero hours debugging install issues over the past few years.

It has been suggested that managing JVMs shouldn’t be our problem since there are other solutions out there and it’s always nice to simplify, but I think that puts us on the hook for the last-mile wiring between Scala tools and a JVM that could be literally anywhere on the user’s system (with a good chance the user also doesn’t know where it is). So I would urge caution there.

Anyway I think the situation is pretty good. :slight_smile:

6 Likes
5

One issue I noticed on Coursera discussion forums is that cs defaults to AdoptOpenJDK 8 if there is no JVM present; and this seems to be too old for the programming assignments of the online courses. Some learners get weird sbt or JVM errors (which are undecipherable) and believe their code is wrong. Should default to at least Java 11 in my opinion.

1 Like
6

I lean near @alexelcu o this matter. There are general-purpose solutions for installing or setting up JVMs and development, so developing a Scala-bespoke one has some costs.

It has been suggested that managing JVMs shouldn’t be our problem since there are other solutions out there and it’s always nice to simplify.

There are several points for this. First, having a solution to manage JVMs takes time and effort, which Scala open-source contributors may have better uses for. Second, general-purpose solutions, like brew or sdkman have more people using it, contributing to it, and debugging it; thus the maintenance or debugging is shared with others. Third, those general-purpose tools are likely known by some people we should be concerned about: developers who use Java or Kotlin and may want to pick up Scala. Scala-bespoke solutions put extra hurdles for them, instead of putting it within reach. So, it may be better to start moving out of Scala-bespoke solutions good as it is, and moving to where more developers may be.

Launched without options, setup checks that a JVM and the standard Scala CLI tools are installed on your system, and updates your profile files. Installation · Coursier

Edit: As a first step, is there perhaps a way that the scala-cli and cs command line utility can be unified, so as to make the coursier commands a subset of scala-cli interface? One less CLI program to watch for.

7

Third and main reason, those general-purpose tools are very likely already known by the people we should be concerned about: developers who use Java or Kotlin and may want to pick up Scala.

Ah, well if this is the case then I agree with you. Java programmers know how to do this. But my understanding is that we want to appeal to Python programmers (among others) who have no JVM experience at all and might become frustrated by the two-part installation. So I suppose it’s worthwhile to identify the target audience.

1 Like
8

I have to question this. Why is this the only audience we should be concerned about? I’ve been a Scala engineer for a decade, and cs makes my life far easier every time I have to set up a new machine; it also makes it miles easier for me to get new folks started on my teams. I don’t think those audiences should be discounted.

I can believe that alternatives might exist, but cs is less than a quarter the effort of any other instructions I’ve tried, not least in that it usually Just Works. That’s not a small thing.

1 Like
9

Not the only people, but rather “some people” we should be concerned about. Corrected the text.

The advantages and ease of use of cs cannot be denied. How does it compare with brew (for Mac mostly) or sdkman, for managing those setups?

10

If someone comes up with a complete brew formula, a proper one-stop-shop, that might work well for Mac. I haven’t personally come across one that is anywhere near as comprehensive as cs – just individual bits and pieces that are, in practice, a headache to put together correctly. brew isn’t a magic bullet: it would also require a bunch of work and maintenance in order to work well.

And of course, that’s just a Mac solution – it fails the “work everywhere” criterion that Julien is (reasonably) looking for.

11

If someone comes up with a complete brew formula, a proper one-stop-shop, that might work well for Mac. I haven’t personally come across one that is anywhere near as comprehensive as cs – just individual bits and pieces that are, in practice, a headache to put together correctly. brew isn’t a magic bullet: it would also require a bunch of work and maintenance in order to work well.

This is true, however, a vast majority of developers working on macOS already use Homebrew, just like a vast majority of developers on Linux already use whatever native package manager is installed. And seniors tend to use multiple programming languages, which makes things complicated, since we tend to search for unifying solutions.

On the other hand, there’s nothing wrong with having a bootstrapping solution like cs. Other languages have it as well, e.g., Rust has rustup and I think people appreciate it.

I’m glad that multiple solutions exist, and I think that @julienrf was asking for feedback on what we are doing, so we should keep it on-topic before the discussion drowns in a sea of opinions :slightly_smiling_face:

12

I do use cs to manage everything on both Mac and Windows (with SDKMan on Mac as well), but as Alex pointed out, I think <packagemanager> install scala should at least give the users an acceptable experience - namely, at least install scala-cli once that becomes the default scala runner.

I mention this because the current experience on Windows is pretty bad, IMO. I think there are 3 ways a person new to Scala would probably install it:

  • Go to the Scala website and follow the instructions - This installs coursier and works well, no problems here
  • choco install scala, which will install Scala 2.11.4
  • winget install scala, which is a bit more updated, but not ideal (I think right now it would install Scala 2.13.8)

From what I can tell, scala-cli already has an official release in chocolatey, so it would be nice if the Scala package on the major package managers was also officially maintained.

Also, it would be great if coursier was available on those package managers. I was searching for it and I think it’s not the case (even though that was discussed in the past)

2 Likes
13

(I previously posted a version of this at SIP-46 - Scala CLI as default Scala command - Tooling - Scala Contributors , but it’s even more on-topic here…)

I doubt our ability as an organization to maintain an entire package manager. Coursier is a one-person project that encompasses both a dependency manager and a package manager. The dependency management aspect is a big and undermaintained project unto itself…!

On MacOS and Linux, I see little value in having yet another package manager. I personally have stopped using cs at all on these OSs. I’d much rather use a standard, robust package manager that I’m already familiar with, such as Homebrew, to install Scala CLI and sbt. And I’d much rather manage my JDKs some standard way, too.

Now that scala-cli can do things like run scalafmt and scalafix, the package management aspect of Coursier no longer provides any value for me. (It certainly doesn’t provide value in proportion to the amount of development effort needed to maintain it, going forward.)

On Windows, sadly, the question is more complicated. cs setup has value there. Perhaps we should retire the cross-platform package management aspect of Coursier and concentrate on providing a Windows-only tool that does the equivalent of cs setup. (Or perhaps scratch “Windows-only” from tha? The core of the suggestion is to handle initial setup only, rather than ongoing package management.)

2 Likes
14

I don’t know about MacOS, homebrew seems standard there, but I don’t personally know people using homebrew on Linux. To not have another package manager would mean packaging for the distro package manager, which there are plenty of, so it is a lot of maintenance work to provide packages for various flavors of linux. Looking at the existing packages, Ubuntu is still at Scala 2.11 (no scala-cli package), Fedora has 2.13 (but also no scala-cli or dotty), Arch has scala-cli in the user repositories and not as an official package. My personal distro of choice (void) has neither and only packages sbt.
So I’m not sure that would be significantly less effort.

In my experience, management of multiple JVMs via the package manager is either completely different for each distribution or not even existent (ETA: and anything besides OpenJDK is also rarely packaged). At work, I don’t always have the choice of distro, so coursier is helpful in having a consistent setup everywhere.

If scala-cli replaces coursier in the future, that’s fine by me. But if scala-cli downloads things like scalafmt and scalafix, isn’t that basically the same as package management, just without adding stuff to the $PATH? Maybe I’m missing some steps that coursier does.

1 Like
15

I use sdkman on MacOS for managing java. I find it provides the best experience for switching between different distributions. And at the same time it provides a good experience for installing sbt and scala-cli as well.

1 Like
16

I’ve been a long time user of cs on Mac and I like it very much for the following reasons:

  • Quick and efficient installation of cs itself
  • Very efficient at installing and updating a lot of applications (scalac, scala, scalafmt, …)
  • Installing of different JVMs
  • Bootstrapping binaries (via cs bootstrap).

Of course, there’s always room for improvement. Something that was annoying until recently was the lack of the stable releases (new versions were stuck in milestone or release candidates for too long). With the release of 2.1.0 on March 10 and subsequent patch releases, that seems to have been rectified.

De-emphasizing cs would not be a good decision if you ask me. I use it on a regular basis and it suits my needs very well. As for alternatives, I have a love and hate relation wrt. brew. It’s orders of magnitude slower than cs which is quite annoying. I’ve used SDKMAN just for managing JVMs. Pretty good tool, but no specific preference from my side between cs and SDKMAN.

If I read some of the replies in this thread, I sense that some people would like to de-emphasise cs. I think that would be a bad idea. My 2 cents.

1 Like
17

I’m in the same camp. On the other hand:

  • There is praise for cs here, it is easy to use, it fills a gap and provides value.
  • We haven’t been able to provide / maintain packages for system package managers in the past. Fixing that might be just as hard as maintaining coursier, there are many systems (brew, dep, rpm, windows installer, winget (?), choco (?), sdkman (?)). We’d need to make sure at least scala(-cli) and sbt are available and up to date, and we’d need to provide instructions for JVM setup.

A few mixed notes

  • There’s brew install coursier that gives the coursier tool, and brew install coursier/formulas/coursier that gives cs, that’s weird.
  • I also think curl ... | bash (or alike) can throw some people off, so it’s good to provide an alternative (brew install, download cs executable)
  • The macOS setup should ideally be the same for both arm and intel; brew install ... should work for arm.
  • I think a short section on the download page (e.g., one that unfolds like “Testing your setup” currently does) about “what does cs setup do” would go a long way. Things like: what does it install, where are the files stored, what files does it modify, how to manage JVMs.
  • The “Other ways to install Scala” download page could give a summary about installing JVM, scala-cli and sbt without cs setup, provide some links (adoptium, sdkman, scala-cli install page, sbt install page).
  • The Getting Started page should maybe not duplicate the instructions from the download page?
  • Will the download page would continue recommending cs setup also after scala-cli becomes the official default scala command?
1 Like
18

looking at the user statistics, both pages are visited equally as much, which are both majority entry points from search engines

19

A note that I think hasn’t been addressed yet: installing in non-interactive (e.g. CI) environments.

If installing everything through coursier is the intended approach there too, the documentation doesn’t really make that clear. Keeping CI images up to date is also quite different from keeping local machines up to date.

1 Like
20

Since sbt is a launcher type of application (ie, it launches a proper sbt program with the correct version), and scala is going to also be a launcher type of application (with its replacement with Scala CLI), I agree that there is less value in having cs managing them for us (e.g., instead of a cs launch scala:2.12.17 we can do a scala console --scala-version=2.12.17). Also, sbt is widely available in common package managers (I think), and scala is soon going to be widely available as well (with the update to Scala CLI).

That being said I think the bootstrap and install commands of cs are cool (for instance, to use scalap), and they are already implemented and don’t require much maintenance effort. I agree that most of the things beyond using scala or sbt are for advanced users, but if cs setup does cover the advanced needs as well as the basic needs, I think it’s good to have.

The current Getting Started instructions tell you to download and run cs setup. What you get is a JDK, and the sbt and scala commands (and cs and sbtn). If we removed those instructions, we would replace them with:

  1. Set up a JDK using the preferred way for your OS
  2. Install scala using your preferred package manager

(assuming scala would be Scala CLI)

The instructions are pretty simple and they are enough for the basic needs, but the users are likely to also need to install sbt at some point at least, which is one more task (a simple one, though).

For me, it is not clear that it’s a win.

I am not really sure we can do that because the setup does rely on the package management capabilities of cs (otherwise, we would have to install scala, sbt, etc. through another Windows package manager, but which one?).

1 Like