I’m creating engineering software where analysis of multiple loading conditions gets completed sequentially. Each of these analysis cases involve thousands to tens of thousands of calculations. The end result comes down to a boolean - if the item fails get a stronger one and start again.
Of course this is ideal for parallel processing. Unlike talking to web or a db, this solution has to block until the final state of things are known.
My idea to do this was cobbled together and looks like following test code. I used this code to work out the framework of my solution:
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Await
import scala.concurrent.duration.Duration
import scala.util.{Failure, Success}
import scala.collection.mutable.ListBuffer
object FutureTester extends App {
var maxTime: Int = 10; // milliseconds
def method(x: Int): Int = { // Normal method with out a future
Thread.sleep(scala.util.Random.nextInt(maxTime)); // randomly sleep to maxTime
x * x
}
def doParallel(x: Int, y: Int, z: Int): Future[List[Int]] = {
val result1 = Future(method(x)) // Note application of Future here.
val result2 = Future(method(y))
val result3 = Future(method(z))
val result = for {
r1 <- result1
r2 <- result2
r3 <- result3
} yield List(r1, r2, r3)
result
}
for (i <- 0 to 6) { // do 6 runs doubling the maximum permitted sleep time each time
val rList = new ListBuffer[Int]
val result = Await.ready(doParallel(2, 3, 4), Duration.Inf) // do the parallel stuff while waiting
result onComplete {
case Success(result) => {
for (rslt <- result)
rList += rslt
}
case Failure(e) => println(s"Failed with ${e.printStackTrace()}")
}
val finalArray = rList.toArray // will use the results as an array
print(s"Max Time: ${maxTime} Final Array: ")
for (e <- finalArray)
print(s"$e ")
println()
maxTime *= 2 // double the maximum time to wait
}
}
As you can see it just calculates squares of numbers going through a loop with a different maximum sleep time, which doubles through each loop.
It works, mostly! For the longer square calculations with the times near a second the algorithm
silently fails frequently.
Typical runs of the program look like this:
Max Time: 10 Final Array: 4 9 16
Max Time: 20 Final Array:
Max Time: 40 Final Array:
Max Time: 80 Final Array:
Max Time: 160 Final Array: 4 9 16
Max Time: 320 Final Array: 4 9 16
Max Time: 640 Final Array:
Or
Max Time: 10 Final Array: 4 9 16
Max Time: 20 Final Array: 4 9 16
Max Time: 40 Final Array: 4 9 16
Max Time: 80 Final Array: 4 9 16
Max Time: 160 Final Array:
Max Time: 320 Final Array:
Max Time: 640 Final Array:
The results come back in a random pattern each time.
I need the final algorithm to always return a result and sometimes that will take well over a second on my slower machine.
What have I done wrong here? My current web server is a single threaded but the machine at my office has twelve threads, and neither returns the full set of proper answers.
