Calling methods on Iterator does not mutate its state in 2.12?

Question
1

Hi everyone, I just discovered a weird discrepancy for Iterator between scala 2.12 and 2.13. And wondering to know if this is a bug of 2.12, or a behavior change in 2.13?

In scala 2.12.11 REPL:

scala> val it = Iterator(1,2,3)
it: Iterator[Int] = <iterator>

scala> it.take(1).toList
res0: List[Int] = List(1)

scala> it.take(1).toList
res1: List[Int] = List(1)

scala> it.take(1).toList
res2: List[Int] = List(1)

In scala 2.13.3 REPL:

scala> val it = Iterator(1,2,3)
val it: Iterator[Int] = <iterator>

scala> it.take(1).toList
val res14: List[Int] = List(1)

scala> it.take(1).toList
val res15: List[Int] = List(2)

scala> it.take(1).toList
val res16: List[Int] = List(3)

As you can see, in 2.12, the underlying state is not shared between source and derivative iterators. So take() does not move current element forward, while in 2.13 the opposite is true.

So is this expected in 2.12 that derived iterator not sharing state with parent iterator, or is this a bug?

1 Like
2

Hi.

From the docs (Scala Standard Library 2.12.11 - scala.collection.Iterator):

An iterator is mutable: most operations on it change its state.

You seem to assume the opposite, which is just not true in general.

edit: I see no bug here, it states that it is mutable and that means it can change, or not, but that is an implementation detail.

3

Also, note the clear warning for the take method (Scala Standard Library 2.12.11 - scala.collection.Iterator):

Reuse: After calling this method, one should discard the iterator it was called on, and use only the iterator that was returned. Using the old iterator is undefined, subject to change, and may result in changes to the new iterator as well.

4

Hi, thanks for replying. I do assume .take() or other method will change its state. So I thought 2.13’s behavior was what I expected. i.e., calling .take() will change its current element.

I get your point that mutation is not guaranteed. If so, I have another question:

How to easily iterate an iterator one batch of elements at a time? For example, I wanna take the first 10 elements from iterator, then the next 10 elements, until it’s exhausted. I thought .take() would do that. It seems it’s not guaranteed.

5

Sorry I didn’t see this, because I was referencing the latest doc (2.13), which doesn’t have this warning.

6

You probably want to use sliding:

it.sliding(10, 10)
2 Likes
7

Thank you! Exactly what I needed. :smile:

8

The 2.13 docs say this:

It is of particular importance to note that, unless stated otherwise, one should never use an iterator after calling a method on it . The two most important exceptions are also the sole abstract methods: next and hasNext .

Behavior when reusing such an Iterator is undefined.
So basically when working with iterators you can either use the high-level API in a pipeline fashion without reusing intermediary iterators (e.g. it.map(...).take(...).filter(...).drop(...).toList). Or you can implement some custom low-level logic with next and hasNext.

2 Likes
9

sliding is perfect for taking a series of chunks. To take just one chunk, use splitAt:

scala 2.13.4> val i = Iterator.range(1, 10)
val i: Iterator[Int] = <iterator>

scala 2.13.4> val (some, more) = res0.splitAt(3)
val some: Iterator[Int] = <iterator>
val more: Iterator[Int] = <iterator>

scala 2.13.4> some.toList
val res2: List[Int] = List(1, 2, 3)

scala 2.13.4> more.toList
val res3: List[Int] = List(4, 5, 6, 7, 8, 9)
10

Probably more grouped than sliding, given the question (“the first 10 elements from iterator, then the next 10 elements, …”).

1 Like
11

But be careful with using splitAt in a loop or a recursive function. Last time I tried, this created long chains of iterators that blew in my face on large iterations.