Question about scala-cli packaging

I am a new user to Scala (started using almost half a year ago) and I’ve started exploring scala-cli. I like the tool but I’m not a fan of the behavior of the package argument.

By default, the package argument will build a lightweight JAR with only the source of my project, and no Scala runtime, but the behavior isn’t the same as building directly with scalac and the -d argument.
scala-cli will instead build a bash script with the jar code so it’s self-executable, but I don’t want this. I just want the jar as it’s created with scalac with the -d argument.

The jar created by scalac also works out of the box without issue, but the script generated by scala-cli fails to execute and throws an error that it was compiled with class file version 63.0, and “this java runtime only recognizes classes up to 61.0”. This is probably fixed just by specifying the JVM version to compile to.

Here’s some information about my setup:

[space@CirnOS:~/Software/yakumo]$ java --version
openjdk 17.0.7 2023-04-18
OpenJDK Runtime Environment (build 17.0.7+7-nixos)
OpenJDK 64-Bit Server VM (build 17.0.7+7-nixos, mixed mode, sharing)
[space@CirnOS:~/Software/yakumo]$ scala --version
Scala code runner version 3.3.1 -- Copyright 2002-2023, LAMP/EPFL
[space@CirnOS:~/Software/yakumo]$ scala-cli --version
Scala CLI version: 1.0.5
Scala version (default): 3.3.1
Your Scala CLI version is outdated. The newest version is 1.1.0
It is recommended that you update Scala CLI through the same tool or method you used for its initial installation for avoiding the creation of outdated duplicates.

My scala-cli is outdated since I’m on NixOS stable as of now and I favor the system’s package manager.

I also tried just compiling my project with scala-cli and then with java I packaged the directory that has all the classes with into a jar, specifying that the main class is “main”. Didn’t work either.

Is it possible to build the jar similarly to scalac -d, with scala-cli?

Package ⚡️ | Scala CLI seems relevant?

1 Like

This is the page I based myself on, but it doesn’t seem to have any information on building a jar that isn’t a self-executable script on top of the actual jar.

Edit: I noticed you linked specifically to the library section, I’m going to try it out. I think I bypassed this section, thinking the class wouldn’t be able to run on its own, in the context of a library.

Suddenly scala-cli throws this error:

Compiling project (Scala 3.3.1, JVM (19))
Error compiling project (Scala 3.3.1, JVM (19))
Error: Unexpected error when compiling src_103be31561: 'xsbt.CompilerInterface'
Compilation failed

It’s not exclusive to the --library argument, I’ll try to fix this and then check out packaging my jar as a library.

Fixed it, I just had multiple processes of scala-cli hanging in the background, interesting.

You might want to update Scala-cli, you’ll probably receive some bugfixes; they jumped from 1.0.6 to 1.1.0 (minor release number change) which suggests something significant; also no offense to nixos or anything but repository packages can have issues (I suffer from this on Ubuntu sometimes).

1 Like

Thanks very much, this is exactly what I was thinking of.

I just have another question: scala-cli can run this resulting jar just fine, but running it with “scala” throws a null pointer exception. On the other hand, scala-cli can run both this jar and also the resulting jars from compiling with scalac. Just a curiosity how one goes both ways but the other doesn’t.

Yeah, I personally think all Linuxes should copy the packaging model of FreeBSD, where the core system base updates every 1-3 years + bugfixes, but your software is either updated in quarterly or rolling-release fashion.

Thankfully NixOS lets me mix both the stable and unstable branches. I’ll check out the latest scala-cli.

1 Like

Not sure about the details, but the old scala code runner is somewhat limited, as very few people used it (most used a build tool like SBT) so it didn’t get much development / polish. I sometimes ran into bugs / NPE with the old runner myself (I remember reporting some bugs with command line arguments being parsed incorrectly). In 3.4 (few months from now) scala-cli will replace the old runner to become the default scala command.

1 Like

This is really nice, and so far I really like scala-cli. I should try compiling to a native binary.

1 Like

I would need to see a full stack trace to comment on a possible cause.

Here it is:

java.lang.NullPointerException
        at java.base/java.lang.Class.forName0(Native Method)
        at java.base/java.lang.Class.forName(Class.java:495)
        at java.base/java.lang.Class.forName(Class.java:474)
        at dotty.tools.runner.RichClassLoader$.tryClass$extension$$anonfun$1(ScalaClassLoader.scala:28)
        at scala.util.control.Exception$Catch.$anonfun$opt$1(Exception.scala:245)
        at scala.util.control.Exception$Catch.apply(Exception.scala:227)
        at scala.util.control.Exception$Catch.opt(Exception.scala:245)
        at dotty.tools.runner.RichClassLoader$.dotty$tools$runner$RichClassLoader$$$tryClass$extension(ScalaClassLoader.scala:28)
        at dotty.tools.runner.RichClassLoader$.tryToInitializeClass$extension(ScalaClassLoader.scala:24)
        at dotty.tools.runner.RichClassLoader$.run$extension(ScalaClassLoader.scala:32)
        at dotty.tools.runner.CommonRunner.run(ObjectRunner.scala:23)
        at dotty.tools.runner.CommonRunner.run$(ObjectRunner.scala:13)
        at dotty.tools.runner.ObjectRunner$.run(ObjectRunner.scala:48)
        at dotty.tools.runner.CommonRunner.runAndCatch(ObjectRunner.scala:30)
        at dotty.tools.runner.CommonRunner.runAndCatch$(ObjectRunner.scala:13)
        at dotty.tools.runner.ObjectRunner$.runAndCatch(ObjectRunner.scala:48)
        at dotty.tools.MainGenericRunner$.run$1$$anonfun$1(MainGenericRunner.scala:220)
        at scala.Option.flatMap(Option.scala:283)
        at dotty.tools.MainGenericRunner$.run$1(MainGenericRunner.scala:224)
        at dotty.tools.MainGenericRunner$.process(MainGenericRunner.scala:270)
        at dotty.tools.MainGenericRunner$.main(MainGenericRunner.scala:281)
        at dotty.tools.MainGenericRunner.main(MainGenericRunner.scala)

Hmm. I think we would also need to see the full scala command line that you’re running, and for comparison, also the full scala-cli command line that works.

The best thing we would be if you could share complete instructions and/or code that would allow us to reproduce the problem on our own computers. But if we can’t have that, we’re going to need all the clues we can get.

Here are the full steps:

git clone https://github.com/spacebanana420/yakumo.git
cd yakumo

This is the jar that always works:

scalac src/*.scala src/*/*.scala -d yakumo.jar
scala yakumo.jar # Works
scala-cli yakumo.jar # Also works

This is the jar that only works with scala-cli:

scala-cli -f --power package src --library -o yakumo.jar
scala-cli yakumo.jar # Works
scala yakumo.jar # Doesn't work

I also noticed that scala-cli crates the “.bsp” and “.scala-build” directories even if you just use it for running software, which creates a bit of clutter even when not developing/building.

I had to alter your instructions from scala-cli -f --power package to scala-cli --power package -f, or I got “Unrecognized argument: -f”.

With that change, I’m able to reproduce the problem on my own computer.

When I unpack the two JARs and diff the contents with diff -u -r, I see that the META-INF/MANIFEST.MF file included by scala-cli doesn’t have “Main-Class: yakumo.main”, so that’s why it doesn’t run with the normal scala or java commands. (I guess it works with scala-cli because it does automatic main class detection?)

How or whether scala-cli can be convinced to include the Main-Class entry in the manifest, I don’t know. But hopefully this is enough to get you unstuck.

1 Like

P.S. oh, this seems relevant: Allow to fill in the jar manifest when a `--library`/`--assembly` jar is packaged · Issue #1699 · VirtusLab/scala-cli · GitHub

1 Like

Thank you very much

1 Like