Trying to parse JSON

I’m trying to parse a json file. I have written a really ugly expression which parses it correctly. And I have extractor-based solution which silently parses to an empty list.
I found the extractor-based approach in a discussion on stack overflow which at first looked straighforward until I actually tried to use it on a real example.

I have several issues.

  1. I’m importing import scala.util.parsing.json.JSON which works for me, but when I try to copy the code to scastie I get an error that object parsing is not a member of package util. Should I avoid using scala.util.parsing.json?

  2. The ugly version of my code filled with lots of .asInstanceOf[List[List[List[Double]]]] works. But the version using for comprehension silently fails. Unfortunately inserting _ = println(this and that does nothing. Is this related to this previous issue where the for comprehension is not optimized for debugging? How can I convince the println to actually print the value rather than being lazily ignored?

  3. Using the style of JSON parsing suggested by shahjapan, how can I iterate two different ways depending on a condition? e.g., depending on the value of geometryType I need to interpret the type of coordinates differently, so I still fill out the for comprehension with lots of .asInstanceOf[thisandthat]

    val try1 = for {
      // L(features) <- geo.asInstanceOf[Map[String,Any]]("features")
      M(countryFeature) <- features.asInstanceOf[List[Any]]
      _ = println(countryFeature)
      S(countryName) = countryFeature("name")
      M(geometry) = countryFeature("geometry")
      S(geometryType) = geometry("type")
      L(coordinates) = geometry("coordinates")
      perimeters <- geometryType match {
        case "Polygon" => extractPolygons(coordinates.asInstanceOf[List[List[List[Double]]]])
        case "MultiPolygon" => coordinates.asInstanceOf[List[List[List[List[Double]]]]].flatMap(extractPolygons)
        case _ => sys.error("not handled type="+geometry("type"))
      }
    } yield (countryName,perimeters)

Well, curiously enough when I type in the example from the stack overflow post, it also simply parses to empty list. So perhaps the problem isn’t my interpretation of the example, the example itself seems wrong.

Don’t reinvent the wheel. Use spray-json, play-json, µPickle,…

Not really re-inventing. I searched for JSON in Scala and got lots and lots of different solutions. So I picked one from the scala standard library scala.util.parsing.json.

I took a look, just now at micro pickle, the documentation talks a lot about how to write JSON, but not so much about reading it.

Thanks for the recommendations.

In current versions of Scala, the scala.util.parsing package was moved to a separately developed and versioned package, so it still can be used with the dependency added. But the json part of it was actually deprecated and removed.

I personally use circe for JSON handling, with which the parsing of one of your feature objects would have similar structure to your current code. I don’t think that you can reduce specifying of types much, as the types of JSON are not known until runtime and resolving the extractPolygons method requires knowing that it gets passed a 3-dimensional list.

case class Feature(id: String, properties: Map[String, Any], geometry: List[List[Location]])

import io.circe._
import io.circe.parser._

val json = parse("""
      {"type":"Feature","id":"ARE",
       "properties":{"name":"United Arab Emirates"},
       "geometry":{"type":"Polygon",
       "coordinates":[[[51.579519,24.245497],[51.757441,24.294073],[51.579519,24.245497]]]}}""")

val featureDecoder = new Decoder[Feature] {
  override def apply(c: HCursor): Decoder.Result[Feature] =
    for {
      id <- c.downField("id").as[String]
      properties <- c.downField("properties").as[Map[String, String]]
      geometryType <- c.downField("geometry").downField("type").as[String]
      coords = c.downField("geometry").downField("coordinates")
      perimeters <- geometryType match {
        case "Polygon" => coords.as[List[List[List[Double]]]].map(extractPolygons)
        case "MultiPolygon" => coords.as[List[List[List[List[Double]]]]].map(_.flatMap(extractPolygons))
        case _ => sys.error("not handled type=" + geometryType)
      }
    } yield Feature(id, properties, perimeters)
}

println(json.map(featureDecoder.decodeJson))

I’m not sure, how you would decode properties to a Map[String, Any], you’d probably have to specify some implicits explicitly.

1 Like

To put a sharper point on it: do not use scala.util.parsing.json – it’s basically abandonware, and is way behind the times.

Instead, I would strongly recommend one of:

  • uPickle
  • Circe
  • Possibly play-json, particularly if you are already using Play

(There are other valid alternatives, but AFAIK these are the most popular.)

They’re all conceptually similar: for a given type, you declare typeclass instances of a Reader typeclass and a Writer typeclass. In all cases, there is a macro that makes this into a one-liner for regular data structures. (Tuples, case classes and ADTs.)

You may be expecting it to be more complicated than it really is. It’s nothing more than declaring the typeclass instance (which, like I said, is a one-liner call to macroRW if you have a straightforward case class), and then calling .read[MyClass]. There’s not all that much to say.

Yes, that’s not so different than what I already have. Thanks.
One problem with the extractor method from stackoverflow, is that if it fails it just fails, with no clue, and inserting println into the for comprehension apparently has no effect. Does circle do a better job of helping you figure how what’s going wrong?

I’ve been using lift-json for 9 years and am quite happy with it.

Circe’s parse method returns an Either with a message describing the position (line, char, and if applicable field name) and error description.

1 Like