Concurrent Computation - List/Array

Question
1

Hi all, sorry if this has been asked before, or I’m just not searching for the right thing on google, but I can’t seem to find a good answer.
Say we have a function
def query(x : String) : Seq[T] = { ... }
and a list of strings we want to process
a,b,......,z
Doing
Query(A), Query(B), .... , Query(Z)
sequentially takes a while to run, (and each query is independent of eachother) so I’d like to run the queries in parallel, and then return the results in one big seq. However, when searching online, it seems like all the examples involving futures are of the form
val af = future{}
val bf = future{}
....
val zf = future{}
for {
a <- af
b <- bf
...
z <- zf
} yield (a ++ b ++ ... ++ z)

But in my case, I’m not always going to have the same string inputs/same number of inputs. So I was wondering if say I had an array or list of the inputs
say Array(a,b,c,…) or List(a,b,c,…,d)
is there a way to have each element computed in parallel, and then joined back together in a yield like statement. Thanks in advance!

2

“seqOfQueryStrings.par.map(performTheQuery(_))” is the simplest way. It uses a hidden thread pool that should have as many thread as you have CPU processors.

Using Akka Streaming you’d use something like “Source(seqOfQueryStrings).mapAsynch(numberOfParallelThreads, performTheQuery(_))”

Brian Maso

3

Future.sequence is your friend here. It lets you take bunch of Futures
of things, and turn it into /one/ Future of the bunch of things that’s
the results from all the input Futures. Or, for example,
Seq[Future[A]] => Future[Seq[A]].

So in your case,

val stuff: Seq[A] = ...

def query(thing: A): B = ...

val futures: Seq[Future[B]] = stuff.map(a => Future(query(a)))

val futureBs: Future[Seq[B]] = Future.sequence(futures)