Google cloud dataflow (scio) and cats-effect 3, cannot inherit from final class

Syntax
1

Hi all,
i’m trying to setup a simple project to use the google cloud hyperscaler (GCP) using the scio library, and the “cats-effect 3” framework.
This is my problem:
My build.sbt:

lazy val commonSettings: Seq[Setting[_]] = Seq(
  ThisBuild / version := "1.0",
  ThisBuild / scalaVersion := "2.12.18",
  ThisBuild / organization := "Organizzazione LC"
)

scalaHome := Some(file("/usr/share/scala/"));

lazy val root = (project in file(".")).
  settings(commonSettings).  
  settings(
    sbtPlugin := true,
    autoCompilerPlugins := true,
    name := "StimaAccessiFraudolenti",
    description := "Descrizione stima accessi fraudolenti",
    scalacOptions := Seq("-feature",
  			 "-deprecation",
  			 "-unchecked",
  			 "-language:postfixOps",
  			 "-language:higherKinds",
  			 "-Ypartial-unification",
  			 "-Yresolve-term-conflict:package")
	  )

addCommandAlias("scioRun",
"runMain src.main.scala.StimaAccessiEffectful.StimaAccessi " + 
"--project=project " +
"--runner=DataflowRunner " +
"--zone=europe-west2-b " + 
"--input=gs://path " +
"--output=gs://path ");

addCommandAlias("stimaAssembly",
"clean; " +
"reload; " +
"clean; " +
"reload; " +
"compile; " + 
"scioRun; " + 
"package");

Compile / compileIncremental / run / mainClass := Some("src.main.scala.StimaAccessiEffectful.StimaAccessi");

addCompilerPlugin("com.olegpy" %% "better-monadic-for" % "0.3.1")

libraryDependencies += "org.slf4j" % "slf4j-api" % "2.0.7";	//Aggiunto per eliminare l'errore slf4j
libraryDependencies += "org.slf4j" % "slf4j-simple" % "2.0.7"; 	//Aggiunto per eliminare l'errore slf4j
//libraryDependencies += "org.apache.avro" % "avro" % "1.11.0";
libraryDependencies += "org.typelevel" %% "cats-effect-kernel" % "3.4.5";
libraryDependencies += "org.typelevel" %% "cats-effect-std" % "3.4.5";
libraryDependencies += "org.typelevel" %% "cats-effect" % "3.4.5";
libraryDependencies += "com.spotify" %% "scio-core" % "0.13.1";
libraryDependencies += "com.spotify" %% "scio-test" % "0.13.1";
libraryDependencies += "org.apache.beam" % "beam-runners-direct-java" % "2.6.0";
libraryDependencies += "org.apache.beam" % "beam-runners-google-cloud-dataflow-java" % "2.6.0";
libraryDependencies += "com.google.api.client" % "google-api-client" % "1.2.0-alpha";

dependencyOverrides += "org.scala-lang.modules" % "scala-xml_2.12" % "2.1.0";

And this is my only scala source:

package src.main.scala.StimaAccessiEffectful

import com.spotify.scio._

import cats.effect.unsafe.implicits._
import cats.implicits._  
import cats.effect.{IO, IOApp, Fiber, ExitCode} 

object StimaAccessi {

  def main(cmdLineArgs: Array[String]) : Unit = {

    val pathSQLdump : String = ("/path/auth.log");

    import src.main.scala.getIOSQL.GetIOSQL;
    val SQLout = for {
      getIOSQL <- GetIOSQL(pathSQLdump).GetIOSQLtext.start
      SQLresult <- getIOSQL.join
                     } 
    yield SQLresult;

    val (sc, args) = com.
                     spotify.
                     scio.
                     ContextAndArgs(cmdLineArgs);

    val authGStorage = "gs://path";
    val desinationGStorageDirectory = "gs://path";

    val input : String = args.getOrElse("input", authGStorage);                 
    val output : String = args("output");                 

//    sc.textFile(authGStorage).
    sc.textFile(input).
       map(_.trim).
       flatMap(_.split("[^a-zA-Z']+").
       filter(_.nonEmpty)).
       countByValue.
       map(t => t._1 + ": " + t._2).
       saveAsTextFile(output);

    val result = sc.run().waitUntilFinish();



    IO.println("");

  }.as(ExitCode.Success); //Chiude la main

println("Sono arrivato all'esterno della funzione main");

}//Chiude l'object

once sbt build has been completed, it raise this error:
(run-main-0) java.lang.VerifyError: Cannot inherit from final class
I found this error is caused by the ContextAndArgs method, but I can’t figure out how I can fix this issue.

I hope you can point me to the correct way.

Thank you

2

Probably some library version inconsistency. Does this run inside some GCP service? Then there could be a version mismatch between the classpath inside GCP and one of your (possibly transitive) dependencies.

Also something slightly suspicious is that you seem to want google-api-client 1.2.0-alpha while scio depends on 2.0.0 (https://mvnrepository.com/artifact/com.spotify/scio-core_2.12/0.13.1). Same for apache beam 2.6.0 while scio depends on 2.49.0, but at least the major versions still match.

2 Likes