How to test scala native code interop with C

Question
1

Hello everyone, I’m trying to run a test on a scala native code that calls a C function from within it but got this error

[[146/147] bar.test.nativeLink
[146] [info] Linking (multithreadingEnabled=true, disable if not used) (2252 ms)
[146] [info] Discovered 1944 classes and 14162 methods after classloading
[146] [info] Checking intermediate code (quick) (105 ms)
[146] [info] Discovered 1904 classes and 10865 methods after optimization
[146] [info] Optimizing (debug mode) (4012 ms)
[146] [info] Produced 5 LLVM IR files
[146] [info] Generating intermediate code (5389 ms)
[146] [info] Compiling to native code (5663 ms)
[146] [info] Linking with [pthread, dl, HelloWorldBar]
[146] [error] /usr/bin/ld: cannot find -lHelloWorldBar: No such file or directory
[146] [error] clang: error: linker command failed with exit code 1 (use -v to see invocation)
[146] [info] Total (15529 ms)
[147/147] ========================================================= bar.test ============================================================ 15s
1 tasks failed
bar.test.nativeLink scala.scalanative.build.BuildException: Failed to link /workspaces/mill-codespace/mill/example/scalalib/native/3-multi-module/out/bar/test/nativeWorkdir.dest/native/scala.scalanative.testinterface.TestMain

Here is my project structure

 build.mill
 src/foo/
   HelloWorld.scala
 native-src/
HelloWorld.c
 test/
     src/
         foo/
             HelloWorldTests.scala

I’m using mill build tool.
What I find strange is that mill run compiles and work successfully but when I try to run mill test, for the test I got the linking error

HelloWorld.scala

package foo
import scala.scalanative.libc._
import scala.scalanative.unsafe._

object Main {
  def main(args: Array[String]): Unit = {
    println("Running HelloWorld function")
    stdio.printf(c"Reversed: %s\n", HelloWorld.reverseString(c"Hello, World!"))
    println("Done...")
  }
}

// Define the external module, the C library containing our function "reverseString"
@extern
@link("HelloWorld")
// Arbitrary object name
object HelloWorld {
  // Name and signature of C function
  def reverseString(str: CString): CString = extern
}

HelloWorld.c

#include <stdio.h>
#include <string.h>
#include <stdlib.h>

char* reverseString(const char *str) {
  int length = strlen(str);
  char *reversed = (char*) malloc((length + 1) * sizeof(char));

  for (int i = 0; i < length; i++) {
    reversed[i] = str[length - i - 1];
  }
  reversed[length] = '\0'; // Null-terminate the string

  return reversed;
}

HelloWorldTests.scala

package foo

import utest._
import scala.scalanative.unsafe._

object HelloWorldTest extends TestSuite {
  val tests = Tests {
    test("reverseString should reverse a C string correctly") {
      val expected = "!dlrow olleH"

      val result = HelloWorld.reverseString(c"Hello World!")

      // Check if the reversed string matches the expected result
      assert(fromCString(result) == expected)
      fromCString(result)
    }
  }
}

and lastly my config file

build.mill

package build
import mill._, scalalib._, scalanativelib._

object `package` extends RootModule with ScalaNativeModule {
  def scalaVersion = "2.13.11"
  def scalaNativeVersion = "0.5.5"

  object test extends ScalaNativeTests {
    def ivyDeps = Agg(ivy"com.lihaoyi::utest::0.8.4")
    def testFramework = "utest.runner.Framework"
  }

  def nativeLinkingOptions = Seq("-L" + millSourcePath.toString + "/target")

  // Compiled C
  def nativeCompiled = T {
    os.makeDir.all(millSourcePath / "target")

    os.proc("gcc", "-m64", "-shared",
      "-c", millSourcePath.toString + "/native-src/HelloWorld.c",
      "-o", millSourcePath.toString + "/target/libHelloWorld.so"
    ).call(stdout = os.Inherit)

    PathRef(T.dest / "target/libHelloWorld.so")    
  }
}

the build file change changes the nativeLinkingOptions to a custom path named target

2 Likes
2

This is the corect error I got and not the one above. I couldn’t edit it cause I think I must have reach my limit for editing the post

[147/147] =========================================================== test ============================================================== 10s
1 tasks failed
test.nativeLink scala.scalanative.build.BuildException: Failed to link /workspaces/mill-codespace/mill/example/scalalib/native/2-interop/out/test/nativeWorkdir.dest/native/scala.scalanative.testinterface.TestMain
3

Probably it fails because it cannot find HelloWorld.so, every extern function annotated (directly or having an annotated owner) is automatically added when linking as -l <name>, in the case of this code it would be -l HelloWorld.
If you only require a simple glue code, then you might consider placing C sources directly in resources/scala-native directory (see Native Code in your Application or Library — Scala Native 0.5.5 documentation although it might require adoption to the Mill build tool.)
If you require C code to be compiled outside Scala Native then remove @link annotation and pass .so path directly to linking options or ensure to inform linker where to find linked libraries (although the nativeLinkingOptions seems correct at first glance)

Can’t really help more without decent logs, probably most of these are hidden by mill. What we’re missing are stderr logs from clang that might give more context.

I’ve just seen these:

146] [info] Linking with [pthread, dl, HelloWorldBar]
[146] [error] /usr/bin/ld: cannot find -lHelloWorldBar: No such file or directory

There’s no HelloWorldBar defined anywhere in published snippets. Consider cleaning mill caches

2 Likes
4

You are so right!

Removing the @link("HelloWorld") and keeping C source files in resources/scala-native are the fixes to the problem, since I already set a custom path in nativeLinkingOptions to find the so file so no need for the @link in the code.

Thank you once again.

4 Likes