Flattening arbitrarily nested tuples in Iterator


#1

I would like to map an Iterator that contains arbitrarily nested tuples and flatten those.
My search lead me to this Shapeless example

I copied the code:

  import shapeless._
  import ops.tuple.FlatMapper
  import syntax.std.tuple._

  trait LowPriorityFlatten extends Poly1 {
    implicit def default[T] = at[T](Tuple1(_))
  }
  object flatten extends LowPriorityFlatten {
    implicit def caseTuple[P <: Product](implicit lfm: Lazy[FlatMapper[P, flatten.type]]) =
      at[P](lfm.value(_))
  }

And the following test works:


  def flattenList[P1,P2,P3](l:List[((P1,P2),P3)]): List[(P1,P2,P3)] = {
    l.map( flatten(_) )
  }

    val test1 = List(((1,2),3), ((4,5),6))
    val ftest1: List[(Int,Int,Int)] = test1.map( flatten(_) )
    println(ftest1.mkString(","))
    val ftest1b: List[(Int,Int,Int)] = flattenList(test1)
    println(ftest1b.mkString(","))

which compiles and executes as expected. However neither this:

    val test2 = List(((1,2),3), ((4,5),6)).toIterator
    val ftest2: Iterator[(Int,Int,Int)] = test2.map( flatten(_) )
    println(ftest2.mkString(","))

nor this compiles:

    val test2 = List(((1,2),3), ((4,5),6)).toIterator
    val ftest2: TraversableOnce[(Int,Int,Int)] = test2.map( flatten(_) )
    println(ftest2.mkString(","))

with the error:

[error]  found   : x$6.type (with underlying type ((Int, Int), Int))
[error]  required: flatten.ProductCase.Aux[shapeless.HNil,?]
[error]     (which expands to)  shapeless.poly.Case[flatten.type,shapeless.HNil]{type Result = ?}
[error]     val ftest2: TraversableOnce[(Int,Int,Int)] = test2.map( flatten(_) )

This seems to be the result of some interference with the canBuild thingy. Anyone see a
how this should be done correctly?

In addition to this, say I want to define my own iterator so:

  def lazyMapP3[P1,P2,P3](i:Iterable[((P1,P2),P3)]) = {
    val convertable = new Iterable[(P1,P2,P3)] {

      override def iterator: AbstractIterator[(P1,P2,P3)] = {

        val converter = new AbstractIterator[(P1,P2,P3)] {
          private val iter1 = i.iterator

          override def hasNext: Boolean = iter1.hasNext
          override def next() = {
            val in: ((P1,P2),P3) = iter1.next()
            PipeSearch.flatten(in)
          }
        } // iterator
        converter
      }
    }
    convertable
  }

The above does not compile due to the same error. But my question is: can I code this
up so that it can accept iterators of any type or arbitrarily nested tuples? If so, what type should
I use for the parameter and return?

TIA


#2

I can’t tell you why, but adding an explicit type for flatten fixes it.

val ftest2: Iterator[(Int,Int,Int)] = test2.map( flatten[((Int, Int), Int)](_) )

#3

@shawjef3 Indeed this does work. I think inference may not be playing well with the macro (after all map2’s type is well defined). Now this still leaves the question open: how can I define a call that generalizes to any arbitrarily nested tuple?

Thanks


#4

Some additional weirdness. This compiles and runs correctly:

    val test3 = List(((1,2),3), ((4,5),6)).toIterable
    val ftest3: Iterable[(Int,Int,Int)] = test3.map( flatten(_) )
    println(ftest3.mkString(","))

and so does this

    val ftest4 = test3.map( flatten(_) )
    println(ftest4.mkString(","))

#5

For anyone that may be interested, I found a working solution in 2.12.6
for the lazyMap. Cannot seem to get a cleaner solution and I am limited to
Iterables. Here it is:


  type InTuple[T] = shapeless.poly.Case[flatten.type, T::HNil]
  type OutTuple[T] = PolyDefns.Case[flatten.type, T :: HNil]#Result

  def lazyMapP3[T](i:Iterable[T])(implicit cse : InTuple[T]) : Iterable[OutTuple[T]] = {
    val convertable = new Iterable[OutTuple[T]] {

      override def iterator: AbstractIterator[OutTuple[T]] = {

        val converter = new AbstractIterator[OutTuple[T]] {
          private val iter1 = i.iterator

          override def hasNext: Boolean = iter1.hasNext
          override def next(): OutTuple[T] = {
            val in = iter1.next()
            PipeSearch.flatten(in)
          }
        } // iterator
        converter
      }
    }
    convertable
  }

And this compiles:

    val test3 = List(((1,2),3), ((4,5),6)).toIterable
    val ftest7 = lazyMapP3(test3)
    println(ftest7.mkString(","))

If anyone has a simpler solution or can explain why I could
not use the Iterator, please tell.

TIA

P.S: PipeSearch is just the module name that contains the Shapeless flatten function.


#6

For the record, the solution indicated above has an error.
Output type should be inferred via implicit otherwise it won’t
carry to other implicits resulting in type errors.

  type InTuple[T] = shapeless.poly.Case[Flatten.type, T::HNil]

  def lazyMapP3[T](i:Iterable[T])(implicit cse : InTuple[T]) : Iterable[cse.Result] = {
    val convertable: Iterable[cse.Result] = new Iterable[cse.Result] {

      override def iterator: AbstractIterator[cse.Result] = {

        val converter: AbstractIterator[cse.Result] = new AbstractIterator[cse.Result] {
          private val iter1 = i.iterator

          override def hasNext: Boolean = iter1.hasNext
          override def next(): cse.Result = {
            val in = iter1.next()
            Ops.Flatten(in)
          }
        } // iterator
        converter
      }
    }
    convertable
  }