# Parallel chain of operations

If I have a sequence of objects, and a sequence of transformations of object to object, Is there an idiom I can use to run the transformations in parallel?

More specifically suppose I have a `seq:Seq[A1]`, and functions `f1:A1=>A2`, `f2:A2=>A3`, … `fk:Ak=>Ak+1`,

E.g., How can I do the following:

``````val result:Seq[T6] = seq.map(f1).map(f2).map(f3).map(f4).map(f5)
``````

in a way so that the functions `f1``fk` have a chance to run in parallel/concurrently if resources permit?

I suspect you’re looking for the parallel collections library, which was split out from stdlib in 2.13. Specifically, I suspect you want ParSeq.

(NB: I haven’t used this stuff myself. But what you’re asking seems to match those docs.)

@jducoeur, as I understand a `ParSeq` allows one operation such as 'foldLeft` to partition the sequence and perform on it in parallel. What I want is different, at least I think it is different. I want a sequence of non-parallelizable operations to be done concurrently.

``````val result:Seq[T6] = seq.map(f1).map(f2).map(f3).map(f4).map(f5)
``````

should be semantically the same as `seq.map(f5.compose(f4.compose(f3.compose(f2.compose(f1)))))`

Is there some sort of conversion guide that explains how to convert from parallel collections < 2.13 to newer? Is it just changing the include path? As I recall it is more than that. Right?

ParSeq means that if you do `map(f1)`, `f1` can be applied to multiple objects in parallel to build a new (parallel) collection.

If you do `map(f1).map(f2)`, it will first apply `f1` to all objects and build a new collection before it starts applying `f2`.

If you want to begin applying `f2` before all it is finished applying `f1` to all objects, you could compose `f1` and `f2`, or use views, or use some library. I think ScalaZ and cats support that. Spark, of course, supports it, too, but would be an overkill unless you want to do distributed computation.

1 Like

Yes, composing would be one possibility. Although, unless I’m confusing myself, composing doesn’t work if we replace `map` with `flatMap` as each stage would be a one-to-many mapping.

I’ll take a look at views. Not sure what that is yet.