Can you give examples for what's lazy computing and their advantages?

I know there are a lot of instance methods in spark which have the lazy computing capability. But for the theory in scala, can you help giving the examples to show what’s the lazy computing and why need them?

Thank you.

What do you mean with “lazy computing”? You mean lazy vals? By name parameters, i.e. => Foo? Or effect types like IO which are sometimes referred as lazy futures?

such as Iterator is a lazy object which can be evaluated by needed.

In Spark, it is used to reduce network operations to avoid latency as much as possible (the biggest bottleneck when your huge dataset is distributed across multiple computers over a network). I highly recommend taking the Big Data Spark course https://www.coursera.org/learn/scala-spark-big-data/

Here is a slide from the course:


“Transformations” are like map, filter, flatMap, groupBy etc. They return collections from collections.
“Actions” are like reduce, fold, aggregate etc. They return single values from collections.

1 Like

Here is an example:


1 Like

I am skilled at spark. what I want to learn is the native lazy implementation in scala language. thank you.

In that case you should take this course https://www.coursera.org/learn/scala-functional-program-design which covers Lazy Evaluation (native Scala) in detail (week 2), it even has a programming assignment that uses infinite lazy streams.

2 Likes

Uhm, I am still not sure what is the answer you want.

But, use cases for the laziness of Iterator we have:

  1. Processing a big file which you don’t want to load completely on memory.
  2. Creating and manipulating infinite collections.
  3. Melding multiple operations on a collection into a single loop.
2 Likes