xgboost: Caused by: java.lang.ClassNotFoundException: scala.Serializable

Question
1

Tried to switch my project on java11.

I use xgboost (https://github.com/dmlc/xgboost) and scala inside, however
it failed with

        at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.lang.ClassNotFoundException: scala.Serializable
        at jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581) ~[?:?]
        at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) ~[?:?]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:521) ~[?:?]

Is this scala’s bug or I need to add some configuration parameters for JVM args ?
i use scala 2.13.1

1 Like
2

there are some further details at https://github.com/scala/bug/issues/11832

3

As Seth already pointed out, it appears as though something was compiled with an earlier Scala version.

@toktarev, on the GitHub issue you said:

—So this error you are getting appears to be caused by using Scala 2.13 to run code that was compiled by some earlier Scala version. Is that possible?

I compile xgboost source code together with all project.
I don’t use maven xgboost

How are you compiling xgboost with Scala 2.13?

Can you provide a sample project that reproduces this problem?

I don’t know anything about xgboost but it looks as though it might use Apache Spark which does not support Scala 2.13.

4

I compile xgboost using gradle and scala plugin:

sourceSets {
apply plugin: ‘java’
apply plugin: ‘scala’
main {
scala {
srcDirs = [
‘jvm-packages/xgboost4j/src/main/java’,
‘jvm-packages/xgboost4j/src/main/scala’
]
}
}
}

5

Please provide a complete example.

6

I opened Scala plugin documentation

https://docs.gradle.org/current/userguide/scala_plugin.html

Configuring the Zinc compiler

The Scala plugin uses a configuration named zinc to resolve the Zinc compiler and its dependencies. Gradle will provide a default version of Zinc, but if you need to use a particular Zinc version, you can change it. Gradle supports version 1.2.0 of Zinc and above.

It says that for Scala 2.10.x through 2.13.x can be compiled I should use gradle 6.

I will try gradle 6 and if it will not help will create complete example.

Thanks.

7

You said

It stopped working after switching to JDK11
After that I changed scala version on 2.13 and obtained corresponding error.

The error you are getting now is probably just because your project is mixing different Scala versions. If you depend on Spark it will be non-trivial to upgrade everything to 2.13 in the near future.

It might be more interesting to share your original error you got when upgrading to JDK 11. Though AFAICT Spark doesn’t support JDK 11 either for the time being.

8

Which scala version should I use ?

<spark.version>2.3.1</spark.version>

JDK 11

9

Next error after switching to the same scala version:

error: cannot access Serializable

during compilation phase

10

Scala version 2.12.10

11

XGBFormatTest.java:82: error: cannot access Serializable
assertEquals(aFeatures.size(), aPoint.values().length);
^
class file for scala.Serializable not found

12

So now it fails compiling the tests? Have you checked the test compilation classpath?

13

Yes, I checked all dependencies and found many scala versions

I revisioned all versions and now I have only one scala version.
No more described error - but I see some tests failures and working on them …

Once I will make everything work - I will close this topic.

Thanks everyone for good advices.

1 Like