Macros in scala 3

I just watched a video by Odersky Countdown to Scala 3. He mentioned that the meta programming interface is changing significantly, and also that the documentation of the system is undergoing significant revisiting. Where can I find the documentation for macros in scala 3?

I have a project in scala 2.13, and I was thinking of looking into macros. Should I wait until scala 3 or should I dive in already?

Background: I’ve been writing Lisp macros for 20 years. I’m hoping to understand how they are similar and different from Scala macros. I assume some of the power of Lisp macros has been expressly forbidden in Scala macros. For example, a lisp macro is basically a function which takes an s-expression as input and returns an s-expression. The OUTPUT s-expression should be valid code to input to the compiler, however, being valid code means that it might be another macro call, or it might be code which contains macro calls in any evaluation position. However, the INPUT s-expression (to a Lisp macro) need not at all be valid code. The semantics of the input s-expression is 100% up to the interpretation of the macro.

As I understand, in Scala macro input must be a valid object which the compiler has already successfully build into an AST. Is this correct?

Anyway, I’d love to take a look at the documentation.

1 Like

http://dotty.epfl.ch/docs/reference/metaprogramming/toc.html



2 Likes

What is the difference between Lisp macros and scala macros?

1 Like

Doc is in progress, but check out https://lampepfl.github.io/scala3-macro-tutorial/.

2 Likes

Listening to the presentation I would say the RCs will be delayed. There have been some changes from M1 to M2 in the macros but they were minimal and were documented. So in my opinion you can dive in with the caveat that some changes may still occur. Two downside though: lack of documentation and the need to adapt existing code. My 2cts.

It’s actually much stronger than that: the old system (which was always marked experimental, never really endorsed for production work, although in practice it was used widely) has been largely scrapped and replaced by a completely new, officially-sanctioned framework.

The two systems overlap in that the popular “quasi-quotes” mechanism that was built on top of the old system inspired the quotation/splicing layer of the new metaprogramming framework. But beyond that, the two systems are mostly disjoint.

So at this point, I would probably recommend just focusing on the new Scala 3 metaprogramming framework.

2 Likes

Yes, the poster of that message seems to be asking the same question as I am, but nobody seems to be able to really answer it.

@jimka Well, actually there is an answer there. I just didn’t copy the text from the thesis (p. 206), kept it as a link. Check it.

I haven’t read the entire 226 page PhD thesis, but I did scan it for the word Lisp, in particular page 206. (If you see an interesting passage that I missed, please refer me to it, I’d love to take a second, third look.) He mentions the existance of macros in lisp, but I don’t find a treatment of what lisp macros can do, and whether (or how) these scenarios have counterparts in Scala macros. He seems more concerned with issues of hygiene than issues of usability.

1 Like

Let me give an example, I hope it’s not too complicated nor too application specific to understand for the non-lisper. The example is found in a Clojure project which I’m converting to Scala as a research project. I have introduced a macro named typecase. The docs can be found here. As operands typecase takes an expression to be evaluated a maximum of once (or never in case it can be optimized away), and a series of pairs (even number of arguments) of the form type-designator, expression (with no delimiting characters other than spaces), ending with an optional default value.

The type designators are not valid evaluatable expressions, rather just s-expressions (raw parsed but not yet compiled data) which the macro code examines; however the other expressions, (+ 1 2 3), 42, 43, 44, and 45, are valid, compilable expressions in the Clojure language. They are both available to the macro programmer, because parenthesisized lists are read by the parser as lists of lists of lists of symbols, strings, and numbers. This is treated by the macro as raw data, but it is the exact same data structure which is otherwise sent to the compiler–homoiconicity.

(typecase (+ 1 2 3)
 (or String Double) 42
 (and (satisfies int?) (satisfies odd?)) 43
 (and (satisfies int?) (not (satisfies odd?)) (not (satisfies neg?))) 44
 45)

The clojure language does not know what to do with (or String Double), (and (satisfies int?) (satisfies odd?)), nor (and (satisfies int?) (not (satisfies odd?)) (not (satisfies neg?))), but the macro understands their expected syntax, and examines them to output another piece of code, which is either valid clojure code or another macro call which the compiler then expands.

When the macro looks at code like (and (satisfies int?) (satisfies odd?)) it understands it as the intersection of two type checks, and tries to figure out whether one type is a subtype of the other, or whether the types are disjoint, or whether a previous type check higher up in the type case has an effect on whether this type check can fail or must pass. Furthermore, when it sees (satisfies int?) it interprets as the set of all values which when passed to the function named int? would return true. Thus it looks up the function int?, decompiles it to try to figure out whether any of the type checks in that function relate to other types being checked at this point of the typecase.

The macro basically expands to an if then else, the then part assumes the object has type Double and the else part assumes does not have type Double, and reduces the logic of the remaining type checks two different ways for the two halves of the if.

Since each half of the if both contain a call to the typecase macro, the compiler happily expands them during the process. Note also that the macro expansion outputs the macro calls which themselves include other type designators which the Clojure compiler will not understand as AST, but rather the recursive macro call will reduce, eventually to code which Clojure can convert to its AST.

Also note that this kind of macro, even if non-trivial, it not considered exotic. (OK, I admit the it is exotic that the macro decompiles the predicate like int?) It is one of the normal kinds of things that Lisp macros do.

(let [value__20493__auto__ (+ 1 2 3)]
  (if (sut/optimized-typep value__20493__auto__ Double)
    (sut/typecase
     value__20493__auto__
     :sigma     42
     :empty-set     43
     :empty-set     44
     :sigma     45)
    (sut/typecase
     value__20493__auto__
     String     42
     (and (or Long Integer Short Byte) (satisfies odd?))
     43
     (and
      (or Long Integer Short Byte)
      (not (satisfies odd?))
      (not (satisfies neg?)))
     44
     :sigma     45)))

After all the macros are expanded, the final code looks like the following. That code is achieved by fine interaction between the compiler and the macro facility, a macro being a user hook into the compiler.

(let [v1 (+ 1 2 3)]
  (if (gns/typep v1 'Double)
    42
    (if (gns/typep v1 'String)
      42
      (if (gns/typep v1 'Byte)
        (if (odd? v1)
          43
          (if (neg? v1)
              45
              44))
        (if (gns/typep v1 'Short)
          (if (odd? v1)
            43
            (if (neg? v1)
                45
                44))
          (if (gns/typep v1 'Integer)
            (if (odd? v1)
              43
              (if (neg? v1)
                  45
                  44))
            (if (odd? v1)
              (if (gns/typep v1 'Long)
                  43
                  45)
              (if (gns/typep v1 '(and Long (not (satisfies neg?))))
                  44
                  45))))))))
1 Like

@jimka

As I understand, in Scala macro input must be a valid object which the compiler has already successfully build into an AST. Is this correct?

It’s correct for def macros because parameters of a def macro are typechecked before the macro is expanded. But for macro annotations this is different since a macro annotation transforms untyped trees.

import scala.annotation.{StaticAnnotation, compileTimeOnly}
import scala.language.experimental.macros
import scala.reflect.macros.blackbox

@compileTimeOnly("Enable macro annotations")
class default extends StaticAnnotation {
  def macroTransform(annottees: Any*): Any = macro DefaultMacro.impl
}

object DefaultMacro {
  def impl(c: blackbox.Context)(annottees: c.Tree*): c.Tree = {
    import c.universe._

    implicit class ModifiersOps(left: Modifiers) {
      def & (right: FlagSet): Modifiers = left match {
        case Modifiers(flags, privateWithin, annots) => Modifiers(flags & right, privateWithin, annots)
      }
    }

    implicit class FlagSetOps(left: FlagSet) {
      def & (right: FlagSet): FlagSet = (left.asInstanceOf[Long] & right.asInstanceOf[Long]).asInstanceOf[FlagSet]
      def unary_~ : FlagSet = (~ left.asInstanceOf[Long]).asInstanceOf[FlagSet]
    }

    annottees match {
      case q"$mods object $tname extends { ..$earlydefns } with ..$parents { $self => ..$body }" :: Nil =>
        val body1 = body.map {
          case q"$mods val $tname: Int = $EmptyTree" =>
            val mods1 = mods & ~Flag.DEFERRED
            q"$mods1 val $tname: Int = 0"
          case q"$mods val $tname: String = $EmptyTree" =>
            val mods1 = mods & ~Flag.DEFERRED
            q"""$mods1 val $tname: String = "" """
          case t => t
        }

        q"$mods object $tname extends { ..$earlydefns } with ..$parents { $self => ..$body1 }"
    }
  }
}
@default
object X {
  val i: Int
  val s: String
}
//scalac: object X extends scala.AnyRef {
//  def <init>() = {
//    super.<init>();
//    ()
//  };
//  val i: Int = 0;
//  val s: String = ""
//}

Notice that

object X {
  val i: Int
  val s: String
}

is illegal.

@jimka

http://www.scala-archive.org/Expand-macros-before-typechecking-its-arguments-trees-td4641188.html

Seems like the suggestion is to pass a string at the macro call site, and within the macro, parse the string. That sounds less than ideal.

1 Like