How to understand this syntax

I come across the following syntax. I am aware of method calls, closures. but couldnt understand what this syntax mean
What does this expression means and how it works . I went through the documentation , I am also aware of closures. groupByKey is actually method, but in this context , hope it is invoked as method but i am assuming the () should present ( perhaps with some params if needed) but this is invoked with {}

val negativeData = positiveData.select("user", "artist").as[(Int,Int)].
      **groupByKey { case (user, _) => user }.**
      flatMapGroups { case (userID, userIDAndPosArtistIDs)
    
    }

Also asked in SO.

In any case, in Scala single argument methods can be called without the dot . and the parenthesis (). This allow us to do things like:

1 + 1 // instead of 1.+(1)
list map { x => x + 1 } // instead of list.map({ x => x + 1})

In any case, those are basic syntax rules that any tutorial / course / book about the language would have covered.
I would recommend you to study a little bit of the language before using an advanced framework for distributed computation.

At a high level, it’s probably enough to think of groupByKey as a
method that’s bring called with the function literal (closure, whatever)
case (user, _) => user as its only argument. That gets at the gist of
what’s going on.

More details if you’re curious:

The above is almost right. In Scala you can (usually?) replace parens
with curly braces:

> def inc(i: Int): Int = i + 1
inc: (i: Int)Int

> inc { 42 }
res16: Int = 43

(No one does that with simple calls like that, but you get the idea).

And you can also omit parens/braces in some cases, notably like your
groupByKey example, which leads to what’s actually going on:

groupByKey is a method being called with one argument, but the
braces/parens are omitted, and the argument is the partial function
literal { case (user, _) => user }, which works because partial
functions are functions (but not the other way around).

Thanks for detailed explanation with lot of patience and without being rude :-).
The point where I was disconnected initially was replace parens
with curly braces:. This is one thing I am NOT aware of . And, to my surprise, not many tutorials didn’t cover this except simplifying the expression ( Perphaps, I would have overlooked them). The only point which I know so far, specifying arguments or parenthesis is optional is the method is invoked with one param as in

val x = List.range(1, 10)
All are same which I am aware of so far
x.foreach((i:Int) => println(i))
x.foreach((i) => println(i))
x.foreach(i => println(i))
x.foreach(println(_))
x.foreach(println)

From this discussion, I also understand, these also similar
x.foreach{println}
x.foreach{(i:Int) => println(i)}

In case you want to dive more, here are some relevant links:

Almost! This is only a partial function if the expected type is a partial function. Here, there the expected type is a Function, not a PartialFunction, so this is just a function literal, not a partial function literal. I can proof it too!

def test(f: Int => Int) =
  f match {
    case _: PartialFunction[_, _] => println("this is actually a partial function cast to a normal function")
    case _ => println("no, this isn't a partial function")
  }

test { case i => i * 2 }
1 Like

What’s going on is a bunch of different interactions of different features. In scala, a block expression is an expression returning the value of its last expression. Examples:

val x = {
  val initial = 1
  doSomethingWith(initial)
}

or

val x = {
     1
 }

or on a single line

val x = { 1 }

or twice on a single line:

val x = {{ 1 }}

That implies that if you can write x foreach f you can also write x foreach {f} or x foreach {{f}}. So if you can write x foreach println you can also write x foreach {println} or x foreach {{println}}. Just like the expression {1} is the same as 1, the expression println is the same as the expression {println}

You can infinitely put {} - or () - around an expression without changing the meaning of the expression. You’re easily lulled into thinking that has anything to do with the (x) in foo(x) for function application, but in many of these cases it’s not, it’s just dropping optional parentheses, and putting the value in a block.

Note that while you can infinitely add more {} or (), you can’t remove them infinitely. i => println(i) can’t be naked on the right side of List(1, 2, 3) foreach i => println(i) – that doesn’t compile. You need to put that expression in parentheses or in a block to be parsed correctly.

Also, function in cases includes {} around it. You can’t omit the {} in

val f: Int => String = {
  case 1 => "one"
  case n if n < 5 => "a few"
  case _ => "many"
}

not even when it’s a single case.

1 Like

You can infinitely put {} - or () - around an expression without changing the meaning of the expression. You’re easily lulled into thinking that has anything to do with the (x) in foo(x) for function application, but in many of these cases it’s not, it’s just dropping optional parentheses, and putting the value in a block.

No, sometimes curlies can replace parens that are not optional:

Welcome to Scala 2.13.2 (OpenJDK 64-Bit Server VM, Java 11.0.8).
Type in expressions for evaluation. Or try :help.

> def sum2(i: Int)(j: Int): Int = i + j
def sum2(i: Int)(j: Int): Int

> sum2(3)(4)
val res0: Int = 7

> sum2(3){4}
val res1: Int = 7

> sum2(3) 4
^
error: ';' expected but integer literal found.

That’s still the block expression {4} following the simple expression sum2(3).

You demonstrate this by the following example being allowed:

def sum2(i: Int)(j: Int): Int = i + j

sum2(3) {
  val i = 1
  val j = 3
  i + j
}

A block expression is always a valid argument list when the argumentlist has one parameter, even if a naked expression isn’t.

1 Like

I’m merely saying it is not a case of dropping optional parentheses.

1 Like

hmm, i am now feeling more confused than it is clear before :frowning: . In summary, “it always depends on the context” , isnt there a documentation I can take a look especially about these specific rules ?

PS : there is a lot of documentation available in the net , most of them sounds too generic to get lost .

The most variation in your question in converting methods to function values. There are a few different ways to convert a method into a function value. Let’s say you have the following method:

def showInt(i: Int): Unit = println(s"the value is $i")

you can convert that in to a function value by following the name with _:

val shower = showInt _

you can also use partial application syntax, and leave the only parameter blank:

val shower = showInt(_)

you could also use a function literal

val shower = (i: Int) => showInt(i)

If the expected type is fully known and doesn’t need to be inferred, you have some additional options and shorthands. The shortest is that you can just give the name of the method

val shower: Int => Unit = showInt

If there is an expected type and you use a literal, you can also leave out the parameter type since it’s already known from the expected type:

val shower: Int => Unit = (i: Int) => showInt(i)
//instead of this, you already know that i: Int
//so you can write it shorter as follows
val shower: Int => Unit = i => showInt(i)

and you can also define it in cases:

val shower: Int => Unit = { case i => showInt(i)}

the i in that last one can be any pattern, so you can do things like this:

val shower: Int => Unit = {
   case 1 => println("one")
   case n if n < 5 => println("a few")
   case n => println("more than a few")
}

Then you can pass shower to foreach –

List(1, 2, 3).foreach(shower)

or in infix notation

List(1, 2, 3) foreach shower

Like I said earlier, you can also put as many extra braces and brackets around an expression as you want, and it won’t change a thing: 1 == (1) == {1} == {({{(1)}})}

So the following is also legal:

List(1, 2, 3) foreach {({shower})}

From there, you can do all variations you want. You can always do the full literal syntax, but you may prefer a less verbose alternative if it isn’t needed.

1 Like