I am working with infinite sequences that represent infinite time-series data. So I need to have all the operations perform in a lazy fashion and avoid copying. I have taken the route of using Iterable
.
To get a sense of what I am trying to achieve consider the following code:
val q1 = Iterator.continually('A')
val q2 = q1.drop(5)
println("q2")
val q3 = q2.take(2)
println(q3.mkString(","))
The last println
will execute correctly with no problem. Now if I use an Iterable
instead, things don’t work as I expected. Consider the following code:
val q10 = Iterable.from(Iterator.continually('B'))
println("q20 init")
val q20 = q10.drop(5)
println("q20")
val q30 = q20.take(2)
println(q30.mkString(","))
The code above will not terminate. More specifically it seems like the from
is iterating through the infinite sequence. The following will also not work:
val q10 = Iterator.continually('B').to(Iterable)
Looking at the code I see that:
Factory[-A, +C].fromSpecific(it: IterableOnce[A]): C
is use to create the Iterable
. After some searching I found documentation on implementing Custom Collections (Scala 2.13). I basically replicated the code using the final 3rd version of the code (test here done without extending StrictOptimizedIterableOps
).
Using this code, if I do the following:
val q12 = TimeSeriesC.continually('D')
println("q12 init")
val q22 = q12.drop(5)
println("q22")
val q32 = q22.take(2)
println(q32.mkString(","))
Its a little better. Now its the drop
that does not terminate but I cheated somewhat. More concretely I use the following code to create the initial Iterable
:
def continually[A](elem: => A): TimeSeriesC[A] = {
//Iterable.from(Iterator.continually(elem)) // not lazy
//Iterator.continually(elem).to(Iterable) // not lazy
//Iterator.continually(elem).to(new TimeSeriesCFactory()) // not lazy
// not lazy on drop
val iterable = new Iterable[A] {
override def iterator: Iterator[A] = {
val converter: AbstractIterator[A] = new AbstractIterator[A] {
private val iter = Iterator.continually(elem)
override def hasNext: Boolean = iter.hasNext
override def next(): A = iter.next()
} // iterator
converter
}
}
TimeSeriesC[A](iterable)
}
Notice that if I used any of the library’s ‘from’ or ‘to’ methods, initializing the infinite Iterable
will still hang. For completeness sake I also list below the factory - note how the use of to(Iterable)
there still causes issues there.
Currently, the only way I can circumvent this is by using:
val q12 = TimeSeriesC.continually('D').iterator
So my question is, how does one ensure laziness? I looked up LazyOptimizedIterableOps and
LazyIterableOps` but did not find additional clues.
TIA
class TimeSeriesCFactory() extends IterableFactory[TimeSeriesC] {
def from[A](source: IterableOnce[A]): TimeSeriesC[A] = TimeSeriesC[A](source.iterator.to(Iterable))
def empty[A]: TimeSeriesC[A] = TimeSeriesC.empty[A]()
// used by with StrictOptimizedIterableOps[A, TimeSeriesC, TimeSeriesC[A]]
def newBuilder[A]: mutable.Builder[A, TimeSeriesC[A]] =
new mutable.ImmutableBuilder[A, TimeSeriesC[A]](empty) {
def addOne(elem: A): this.type = { ??? ; this }
}
}