Scala map function fails when transforming a list of tuples to a list of class

Let define a simple case class

case class Dummy(a: Int)

I try to transform List[(Int, Int)] to List[Dummy]using List.map.

Passing an anonymous function to map like this works fine:

val as = List((1, 1), (2, 4), (3, 9))
as.map(a => Dummy(a._2))
/* Output */
val res2: List[Dummy] = List(Dummy(1), Dummy(4), Dummy(9))

But when using _ it fails:

as.map(Dummy(_._2))
/* Output */
1 |as.map(Dummy(_._2))
  |             ^
  |     Missing parameter type
  |
  |     I could not infer the type of the parameter _$1 of expanded function:
  |     _$1 => _$1._2.

Why?

FYI, it seems that _._2 can be used if it is not inside the constructor Dummy:

as.map(_._2)
/* Output */
val res3: List[Int] = List(1, 4, 9)

Moreover, when accessor _2 is not used, there is no problems:

val as =  List(1,2,3,4)
as.map(Dummy(_))
/* Output */
val res1: List[Dummy] = List(Dummy(1), Dummy(2), Dummy(3), Dummy(4))

see Scala FAQ | Scala Documentation

1 Like

Also asked and “answered” on SO: Scala map function fails when transforming a list of tuples to a list of objects - Stack Overflow

After seeing a comment on SO, I did a little bit more experiment and found that as.map(Dummy(_)) works. Because of this, I was convinced that as.map(Dummy(_)) does not expand to as.map(Dummy(x => x)) and thought that the answer in SO is wrong.

To seek a new answer, I appended that finding to my question and post it here. Anyway, the FAQ link given by SethTisue confirms that the answer in SO is right.

Really, the best part of the SO answer is that the _ was a mistake.

As time passes I found myself defending it less and less.

I’ve increasingly come to the rule of thumb that, when _ is in play and I get a mysterious error, the first step is to remove it. It works well in simple cases, and I’d be loathe to remove it, but it is very easy to misuse.

(And as this example makes clear: when you have nested parentheses, it’s no longer a “simple” situation.)

2 Likes

That is exactly what I do, too.

It’s because the Scala compiler is terribly bad at producing useful error output. The output from the Scala compiler strongly reminds me of the absolutely mind boggling bad errors of the early C compilers.

I’ve been using Scala for +11 years. And for the first 3 years, I spent so many hours on completely unproductive tangents trying to figure out what incantations I had wrong about getting the Scala compiler to compile my code. And +90% of those tangents were related to the error the Scala compiler emitted (which typically was secondary to a prior error in a deeper scope).

Scala is already hard enough to learn in the first iterations of interacting with the language, just as a language. The Scala compiler errors just magnify difficulty of those initial iterations significantly.

If Scala adoption is a primary goal, improving errors like this one are critical to reduce drag on a person persevering through the learning process.

I would take issue with that characterization, even if it explains my sense of nostalgia when I first learned about Scala.

The error messaging has been improved over many years, and now it can be improved all over again with Scala 3.

I see that one of my first contributions was an improved message about package objects. People still ask about package objects. I also see retronym later fixed the behavior it was explaining.

Arguably, the greatest frustration is when “common knowledge” is not available to the user.

I remember a conversation with Paul Phillips about having error messages index a documentation site with deep dives, especially about “feature interactions”. I think that is an ongoing conversation on Scala 3, where the effort to improve messages did not quite tap a vein of rich information.

For the current example, I remember another Paul Phillips conversation in which the idea was floated to try creating functions that the user might have intended and present the ones that compile, much like the compiler will ask “Did you mean …?” about simple spelling mistakes.

Probably there is already a scalafix rewrite to “convert underscore to parameter” and a similar quick fix in Intellij.

It’s also worth advertising that // print<tab> in REPL shows the expansion

scala> as.map(Dummy(_._2)) // print
                                      as.map[B](Dummy(((x$1) => x$1._2)))

Only since 2.13.7 has it indented the expansion text like that. Quick, somebody write a ticket to improve the tooling output!

I feel sure that “How is placeholder syntax expanded” has been explained in a few places.

The rule is deceptively simple: the underscore expands at the enclosing Expr, where the Expr is not just _ by itself. The useful distinction is between Expr expressions and the subexpressions called “simple expressions” or “infix expressions”.

scala> List(42).map(_ + 1)    // infix operands are infix exprs
val res0: List[Int] = List(43)

scala> List("hi").map(_.length)  // selection is from simple expr
val res1: List[Int] = List(2)

scala> case class C(c: Int)
class C

scala> List(42).map(C(_))    // argument is Expr but here it is just `_`, it does not properly contain it
val res2: List[C] = List(C(42))

This distinction between simple expressions and the “top-level” expression is useful in other contexts. For example, if is an expression, so why can’t I just write:

scala> 42 + if (true) 1 else 0
            ^
       error: illegal start of simple expression

An Expr is turned into a simple expression by enclosing it in parens:

scala> 42 + (if (true) 1 else 0)
val res3: Int = 43

An Expr is where you can write a function literal, so it makes sense that this is where placeholders become function parameters. You can tell if a syntactic element is an Expr by writing a function. For example, the condition to if is an Expr, so this is good syntax:

scala> if ((i: Int) => i + 1) 42
                    ^
       error: type mismatch;
        found   : Int => Int
        required: Boolean

Obviously, it doesn’t compile because it doesn’t typecheck.

This compiles for the same reason as res2, where it doesn’t expand at the bare _ but at the enclosing Expr:

scala> def f: Boolean => Int = if (_) 42 else -1
def f: Boolean => Int

and this does not, because the expansion is at the Expr condition, not outside the if:

scala> def f: String => Int = if (_.toBoolean) 42 else -1
                                  ^
       error: missing parameter type for expanded function ((<x$1: error>) => x$1.toBoolean)
                                               ^
       error: type mismatch;
        found   : Int(42)
        required: String => Int
                                                       ^
       error: type mismatch;
        found   : Int(-1)
        required: String => Int

And that is the same reason this does not compile:

scala> List(42).map(C(_ + 1))
                      ^
       error: missing parameter type for expanded function ((<x$1: error>) => x$1.$plus(1))

or more precisely, that is why the placeholder is expanded inside the inner parens.

Each arg in an application is an Expr, which is why placeholder syntax works when there are multiple args:

scala> Option(42).map(List(17, 27, _))
val res7: Option[List[Int]] = Some(List(17, 27, 42))

Here is an example where one might hope to receive tutorial assistance:

scala> case class C(s: String) { def length = if (s.length < 3) ??? else s.length }
class C

scala> List(C("hi")).map(try _.length catch (_ => 42))
                                                  ^
       error: type mismatch;
        found   : Int(42)
        required: C => ?
                                               ^
       warning: This catches all Throwables. If this is really intended, use `case _ : Throwable` to clear this warning.

       did you mean...?
       List(C("hi")).map(x => try x.length catch { case _: Throwable => 42 })

where in this hypothetical example, it applies all the scalafixes it knows.

Possibly this more absurd version makes it more obvious that the body of the function is not evaluated in a try expression:

scala> List(C("bye")).map(try { case x @ C(_) if x.length >= 3 => "ok" })
                          ^
       warning: A try without a catch or finally is equivalent to putting its body in a block; no exceptions are handled.
                              ^
       warning: match may not be exhaustive.
       It would fail on the following input: C(_)
val res10: List[String] = List(ok)

An improved message would say that there are no exceptions to handle.

3 Likes

I got as far as

placeholder-help.scala:3: error: missing parameter type for expanded function ((<x$1: error>) => x$1.$plus(1))
  did you mean ...? ((x$1) => Option(x$1.$plus(1)))

  def f(x: Option[Int]) = x.flatMap(Option(_ + 1))    // intended x => Option(x + 1)
                                           ^
1 error
1 Like