How to run Scala code at compile time?

Question
#1

This is my small utility method to fetch an Image from internet

object HttpClient {
  private val connectTimeout = 15000
  private val readTimeout    = 15000
  private val requestMethod  = "GET"
 
  def getBytes(inline url: String) = Try {
    println("fetching data...")
    val connection =
      (new URL(url)).openConnection.asInstanceOf[HttpURLConnection]
    connection.setConnectTimeout(connectTimeout)
    connection.setReadTimeout(readTimeout)
    connection.setRequestMethod(requestMethod)
    val inputStream = connection.getInputStream
    val bytes       = inputStream.readAllBytes()
    println("DONE")
    if (inputStream != null) inputStream.close
    bytes
  }.toOption
}

And below is the code consuming it to fetch an image convert to string and print it.

object Test extends App {
  val url = "<some random http url of an image>"
  val bytes = HttpClient.getBytes(url).getOrElse(Array.empty[Byte])
  val img = java.util.Base64.getEncoder.encodeToString(bytes)
  println(img)
}

It works correctly, but I am trying to fetch the image at compile time and print it on runtime.
Is this possible in Scala 3?
If yes, how?

1 Like
#2

You should be able to simply run that code in a macro.

It should look a little something like this.

import scala.quoted._

inline def fetchImage() = ${fetchMacro()}

def fetchMacro()(using Quotes): Expr[String] = {
  val url = "<some random http url of an image>"
  val bytes = HttpClient.getBytes(url).getOrElse(Array.empty[Byte])
  val img = java.util.Base64.getEncoder.encodeToString(bytes)
  Expr(img)
}
#3

Thanks @Jasper-M

This is downloading the image at compile time now.
However, compilation is failing with error scala.tools.asm.MethodTooLargeException

#4

Hmm that’s interesting. My guess is that the size of the generated code is somehow proportional to the size of the string. Which is probably pretty huge in this case. I wonder if it’s some kind of inefficiency in how macro code gets generated or how Expr lifts strings into expressions, or if there’s a limitation in how big a string literal can get.

#5

Yeah you are right.
When I do inputStream.readAllBytes().take(100), it works

#6

Isn’t an inline method enough here ?

#7

An inline method without quotes/macro won’t actually run the code at compile time.

#8

Java has a limit of 64k for method sizes: Method Size Limit in Java - DZone Java

Is the image you are generating a string for > 64k?

#9

I would expect this macro to generate a string literal which is part of the constant pool of the class file and shouldn’t contribute that much to the actual method size. But apparently there is a similar limitation to those constants: Java with String length limit - Stack Overflow

#10

I’m surprised the constant string literal limit is also 64k. Unfortunate! For small images that would be fine, and would still be useful. For larger images downloading to the resources dir and loading seems the only option.

#11

A possible alternative if you really want a string may be cutting the big string into smaller pieces which you then emit as literals and then paste them back together in a StringBuilder.

2 Likes
#12

We use that encoding in dotty/TastyString.scala at main · lampepfl/dotty · GitHub. You can copy that code if you need it.

1 Like