Runtime reflection in Scala

First I’d think that e.g. JsValue shouldn’t require a huge amount of studying. Second, having a restricted set of types is a feature. With a JsValue, there’s exactly 8 different forms it can take. A Map[String, Any] can literally contain anything. I’ll have to guess which types are used to encode JSON arrays/numbers/…, I can’t rely on the compiler to tell me when I’ve forgotten to cover one of the cases, and so on.

I think you are conflating two things here. The JsValue structure is completely equivalent to the Map[String, Any] structure, and you could do the same check for array nesting depth in both. The convention of adding polymorphic subtype information as JSON attributes just makes this distinction easier, and it covers more ambiguous cases - imagine a list that can contain both absolute and relative coordinate pairs, both consisting of two numbers.

That’s the part you have completely omitted on the Clojure side. The equivalent of json/read-str (slurp ...) with spray-json would simply be JsonParser("/tmp/small-example.json"). That’s the format-level part, and the resulting structure is fully equivalent. Converting to the domain level (i.e. Map[String,List[Perimeter]]) is the interesting part. (And I think I recall that I had given some suggestions at the time how this could be approached a bit more conveniently.)

And again, if you don’t like JsObject and friends, a conversion to Map[String, Any] should be feasible in a single, recursive pattern-matching function, if I’m not missing anything.

1 Like

I would not characterize the philosophy of dynamically typed languages as “data is unsafe/hostile and needs to be checked all the time”, but rather as “it’s up to the user to make sure the data has the correct type”

In many dynamically typed languages, providing an argument of the wrong type will rarely result in a type error and more likely result in unexpected behavior. JavaScript is particularly notorious for that.

In a statically typed language, if you have input of unknown type, the first thing is usually to project it to some expected type and flag it as an error if it is not. Therefore, Scala libraries that deal with querying a database or parsing JSON allow you to define precisely what type you expect and will deliver you an object of that type or an error.

1 Like

(Context: I’ve been a full-time Scala teacher, and still spend a fair bit of my time tutoring new folks in the language, including completely new programmers as part of ScalaBridge. I get the “put yourself in the student’s shoes” thing.)

Right – my point is, that’s all optional. You need that in order to get the strongest possible types from Circe. You’re saying that you don’t want strong types, which is fine – don’t use them. Instead, use a JSON parser designed to support a less strongly-typed version, like play-json, uPickle or weePickle.

I mean, the total parse code for play-json is:


That’s it – and that gives you an AST that is structurally identical to the one you say you’re looking for. Yes, you have to explain that JsObject is a specialized version of Map that specifically means that this is a JSON Map, but that’s not a hard concept, and it helps get across the Scala way of thinking, that Types Are Good.

(The actual functions and types are different for uPickle and weePickle, but otherwise they’re identical to this.)

All the effort is in producing strong types; those are much easier to work with in nearly every business case, but they are entirely optional. If what you want is a simple AST, just use a simple AST.


I should note an important corollary: you’re thinking about this as “this is how Scala does it”. That’s usually wrong – it’s all about how a specific library does it. If you don’t like the way that library works, there tends to be another one that is more like what you want…

1 Like

I’ve just revisited that thread and your code, and I still cannot really understand your complaints. I still like my spray-json/lenses approach a bit better, and I probably would do a few things different than you with circe, but I don’t think that the HCursor API is that terrible. I’m really wondering how an Any-based implementation should fare significantly better.

In case you’re curious to try it out, here’s a naive conversion from circe Json to Map[String, Any]:

def toUntyped(json: Json): Option[Map[String, Any]] = {
  def cnv(j: Json): Any =
  def cnvObj(jo: JsonObject): Map[String, Any] =

Another thing is that your use case is somewhat special, though not completely unusual. You are reading the JSON data while transforming/extracting at the same time. This is perfectly fine, but the more common case is that one really wants to map JSON data to an equivalent representation in the code. If we take this route, the picture is somewhat different.


sealed trait Geometry
case class Polygon(coordinates: List[List[List[Double]]]) extends Geometry
case class MultiPolygon(coordinates: List[List[List[List[Double]]]]) extends Geometry
case class Properties(name: String)
case class Feature(properties: Properties, geometry: Geometry)
case class FeatureCollection(features: List[Feature])

implicit val decodeGeometry: Decoder[Geometry] =
  (c: HCursor) =>
    c.downField("type").as[String].flatMap {
      case "Polygon" =>[Polygon]
      case "MultiPolygon" =>[MultiPolygon]

val features = parse(jsonStr).flatMap([FeatureCollection])

circe in particular is pretty much opiniated in that regard, as expressed in the design document:

You generally shouldn’t need or want to work with JSON ASTs directly.

Once you have this direct representation of the JSON model, it should be much easier to transform it into the representation your application actually wants, i.e. Map[String, List[List[Location]]]. This is certainly less efficient than the direct transformation upon parsing - on the other hand now you have the full content of the JSON data available in a nicely typed fashion for whatever else you may want to repurpose it to.

HI Sangamon, is what you copied into that message correct? Maybe I’m blind, but I don’t see the difference between the first two expressions which evaluate differently.
123.getClass.getClass vs 123.getClass.getClass

The first 123.getClass.getClass should have just been 123.getClass


Yeah, sorry, Murphy’s law - I think I missed the first line when copying the REPL session and then somehow managed to grab the wrong line when trying to fix it up. :roll_eyes: I’ll edit it for posteriority.

1 Like

Not really intending to resurrect this thread, but I just came across a blog post that reminded me of this discussion: Parse, don’t validate.

1 Like