Announcing Dsl.scala 1.3.0, an alternative to `for` comprehension, Scala Async and Scala Continuations

#1

Hi, all,

Recently I released Dsl.scala 1.3.0, a framework to create embedded DSLs (Domain-Specific Languages) in Scala.

Dsl.scala can be considered as an alternative syntax to for comprehension, Scala Async and Scala Continuations. It unifies monads, generators, asynchronous functions, coroutines and continuations to a single universal syntax, and can be easily integrate to Scalaz, Cats, Scala collections, Scala Futures, Akka HTTP, Java NIO, or your custom domains.

A DSL author is able to create language keywords in few lines of code. No knowledge about Scala compiler or AST macros is required. DSLs written in Dsl.scala are collaborative with others DSLs and Scala control flows. A DSL user can create functions that contains interleaved DSLs implemented by different vendors, along with ordinary Scala control flows.

New features

There are 68 commits between Dsl.scala 1.0.0 and 1.3.0. The highlighted new feature is the for comprehension support. Other features include asynchronous IO for files and exception handling in Future domains.

Asynchronous file IO

Two keywords, AsynchronousIo.ReadFile and AsynchronousIo.WriteFile, are added in Dsl.scala 1.3, for performing Asynchronous file IO on Java NIO file channels. The two new keywords are similar to existing AsynchronousIo.Read and AsynchronousIo.Write, except they accept an additional parameter for the position in file.

Asynchronous file IO keywords can be used in functions that return Task or other continuations. The following example create a function to create an asynchronous loop to read entire file content into memory:

def readAll(channel: AsynchronousFileChannel, bufferSize: Int = 4096): Task[ArrayBuffer[CharBuffer]] = Task {
  val charBuffers = ArrayBuffer.empty[CharBuffer]
  val decoder = Codec.UTF8.decoder
  val byteBuffer = ByteBuffer.allocate(bufferSize)
  var position = 0L
  while (!AsynchronousIo.ReadFile(channel, byteBuffer, position) != -1) {
    position += byteBuffer.position()
    byteBuffer.flip()
    charBuffers += decoder.decode(byteBuffer)
    byteBuffer.clear()
  }
  charBuffers
}

for comprehension support

Dsl.scala 1.2+ supports for comprehension. Now you can create a single for block to extract and compose values from different types of keywords. For example, the following cat function contains a single for block to concatenate file contents. It asynchronously iterates elements Seq, ArrayBuffer and String with the help of Each keyword, managed native resources with the help of Using, performs previously created readAll task with the help of a Shift keyword, and finally converts the return type as an asynchronous task to produce a vector.

def cat(paths: Path*) = {
  for {
    path <- Each(paths)
    channel <- Using(AsynchronousFileChannel.open(path))
    charBuffers <- Shift(readAll(channel))
    charBuffer <- Each(charBuffers)
    char <- Each(charBuffer.toString)
  } yield char
}.as[Task[Vector[Char]]]

The above for-comprehension expression has the same functionality as the following code in !-notation.

def cat(paths: Path*): Task[Vector[Char]] = {
  val path = !Each(paths)
  val channel = !Using(AsynchronousFileChannel.open(path))
  val charBuffers = !Shift(readAll(channel))
  val charBuffer = !Each(charBuffers)
  val char = !Each(charBuffer.toString)
  !Return(char)
}

If you only use for-comprehension and never use !-notation, you don’t need compilerplugins-bangnotation nor compilerplugins-reseteverywhere mentioned in Getting Started.

Out-of-the-box exception handling for Futures

In Dsl.scala 1.0.x, the ability of exception handling for Scala Future is only achieved by MonadError type classes in Cats or Scalaz. In order to use try / catch / finally expressions with !-notation in Dsl.scala 1.0.x, you need the dependencies of Scalaz or Cats.

// Need the following imports in Dsl.scala 1.0:
// 
// import com.thoughtworks.dsl.domains.scalaz._
// import scalaz.std.scalaFuture._
// 
// Or:
//
// import com.thoughtworks.dsl.domains.cats._
// import cats.instances.future._
//
def failedFuture = Future { 0 / 0 }
def recovered = Future {
  try {
    100 + !Await(failedFuture)
  } catch {
    case _: ArithmeticException =>
      42
  }
}

In Dsl.scala 1.1+, exception handling for Futures is now built-in. You can use try / catch / finally out of the box. No dependency to Scalaz or Cats is required. This makes Dsl.scala a clean replacement to scala.async.

Note that you still need MonadError instances for the exception handling feature for Monix or Cats Effect.

Other utilities

I also published Dsl.scala-akka-http recently, the Akka HTTP integration for Dsl.scala.

Backward and forward compatibility

All Dsl.scala versions conform to semantic versioning for binary compatibility. Dsl.scala 1.3.0 is binary backward compatible with all Dsl.scala 1.x versions, and all 1.3.x versions will be forward compatible with other 1.3.x version.

Source level backward compatible is broken in Dsl.scala 1.3. The keyword Each used in Unit domain does not compile any more. Use Foreach instead if you want to iterate a collection without mapping it to other values.

Author information

Dsl.scala is founded and maintained by me when I was in ThoughtWorks, and I will keep maintaining this library in my part time, though I have resigned from ThoughtWorks recently. Since I am back to the job market, you can contact me or give me a job referral if you recognize my expertise in my open source contributions. I am currently seeking for career opportunity in California, especially in the San Francisco bay area or San Diego.

Links

1 Like
#2

I have recently discovered this library and find it quite handy. I wonder if it will work on Dotty. Probably it’s too early to ask but maybe you have already looked at planned macro support there.

#3

Hi @yangbo,
Your project looks brilliant, but it is not clear to me why we need a framework to create embedded DSLs. It is also not clear to me what are the benefits of unifying monads, generators, async functions, and continuations. Do you mind expanding a little bit on the engineering problem that this library addresses?

3 Likes
#4

@julienrf As a user I don’t need to know the framework details. What I want is a better Scala Async, which supports exception handling, resource management, collection comprehension, etc, for any asynchronous task types.

As a DSL author, I need the framework to make the above stuffs work.

1 Like
#5

I did not test but I think it may be possible to use Dsl.scala’s for-comprehension syntax in Dotty with only few changes (if any).

!-notation is implemented in Scala Compiler plugin, which is not available in Dotty. Fortunately the compiler plugin is a relatively small parts (only 529 LOC) of the code base of Dsl.scala. I think it can be re-implemented in Dotty and Scala 2.14.x with the help of Tasty.

#6

Thanks for sharing your library.

What are the differences to the Monadless library? I think both look very similar.

#7

Not similar at all.
Monad type classes require you to always use the same type for your operator and the return value.

In contrast, Dsl.scala allows arbitrary combination of keywords and domains. You can consider Monad as a (very limited) special case of Dsl.