Overload not found when using Java library with float[]: how to circumvent?

Question
#1

I have the following code:

    val X: Array[Float] = x.toFloatArray()
    val Y = y.toFloatArray()

    val data = Table.create("Data").addColumns(
        FloatColumn.create("X", X),
        FloatColumn.create("relu(x)", Y)
    )

which results in:

[error] -- [E134] Type Mismatch Error: /home/hmf/VSCodeProjects/pdm_toyadmos/dcase2020/src/examples/djl/Chap4MLP.scala:46:20 
[error] 46 |        FloatColumn.create("X", X),
[error]    |        ^^^^^^^^^^^^^^^^^^
[error]    |None of the overloaded alternatives of method create in object FloatColumn with types
[error]    | (x$0: String, x$1: java.util.stream.Stream[Float]): 
[error]    |  tech.tablesaw.api.FloatColumn
[error]    | (x$0: String, x$1: Array[Float]): tech.tablesaw.api.FloatColumn
[error]    | (x$0: String, x$1: Int): tech.tablesaw.api.FloatColumn
[error]    | (x$0: String, x$1: Float*): tech.tablesaw.api.FloatColumn
[error]    | (x$0: String): tech.tablesaw.api.FloatColumn
[error]    |match arguments (("X" : String), (X : Array[Float]))

But the first option seems to match perfectly. Any way I can force this?

TIA

#2

Hi,

First of all, it would be good to mention from where do those functions came from.
Second, are you sure the error is real (meaning it was produced by a real compiler) or it is just in your IDE; if you are sure it is real then adding the compiler version, as well as the libraries and their versions, would help in reproduce and fix the problem.

1 Like
#3

@BalmungSan Thanks for taking the time to look at this.

In regards to the functions, these are Java libraries. But I can simply further just using:

    val cx: FloatColumn = FloatColumn.create("X", Array(1.0f, 2.0f))

which produces the same error. The FloatColumn.create is a call to the tablesaw Java library.

As for the compiler: I am using Mill to continuously compile in Dotty (3.0.0-M3). So the error is real.

I may be wrong but this seems like a generic interop issue. I was hoping someone already had such an issue and new the fix.

1 Like
#4

Uhm that is weird, it feels like a bug but then it has something to do with Java interop since I tried to reproduce it, but it worked as expected.

Hope someone with more knowledge on Dotty can help.
I would suggest also asking in the gitter channel and maybe also in SO.

1 Like
#5

I noticed something weird with these 2 methods:

(x$0: String, x$1: Array[Float]): tech.tablesaw.api.FloatColumn
(x$0: String, x$1: Float*): tech.tablesaw.api.FloatColumn

If that comes from java then those methods should erase to exactly the same thing. Which makes me wonder how that bytecode is generated and accepted.

So I looked for the corresponding Java code and found this:

public static FloatColumn create(String name, Float[] arr)

So that’s an Array[java.lang.Float], not an array of primitive floats.

I guess you have to either convert to an Array[java.lang.Float] or call the varargs method with create("X", X :_*). I would expect the second one to be faster.

2 Likes
#6

Thanks to @Jasper-M findings I am able to reproduce and fix the error: https://scastie.scala-lang.org/BalmungSan/XYNKJahKR2WGX4SoyerXCA/5

(But being honest, what is even the point of an array of boxed floats? “Oh well, whatever, nevermind”)

#7

@Jasper-M and @BalmungSan thank you.

I can report this as an issue to the Tablesaw developers and find out why they use a boxed float. Maybe its needed to match on columns of different types?

I do have one more question though for @Jasper-M: why is the varargs call faster? That’s a conversion to a sequence, correct?

TIA

#8

Not really in this case, since it is a vargars from Java which is a plain array, so it would just pass the variable as it is (I believe).

Jasper says it is faster because that, it won’t create a new array like my map approach and box each float. However, if the underlying algorithm boxes them in a new array then both are the same at the end :stuck_out_tongue: - in any case, for small arrays you shouldn’t notice any difference, and for bigger ones then benchmarking in the only valid answer.

1 Like
#9

Ok. Thanks.