Converting List[Future[Int]] to Future[List[Int]] with failed futures

Question
#1

Tried asking this on stack overflow but could not get an answer. Trying here

Say I have the following snippet

def testFailure2() = {
    val f1 = Future.failed(new Exception("ex1"))
    val f2 = Future.successful(2);
    val f3 = Future.successful((5));
    val f4 = Future.failed(new Exception("ex4"))
    val lst = List(f1, f2, f3, f4)
    lst
  }

The return type is List[Future[Int]]. In a normal way, I can just do Future.sequence and get List[Future[Int]]. But in this scenario it won’t work as I have a failed Future. So I want to convert this to List[Future[Int]] by ignoring the failed Futures. How do I do that?

Second Q on similar topic I have is, I understand filter, collect, partition, etc on a List. In this scenario, say I wanted to filter/partition the list into two lists - Failed Futures in one - Successfully done Futures in another.

How do I do that?

#2

It loos like recover coupled with Try is probably what you are looking for: https://stackoverflow.com/a/20874404/3096687

That said, you may also want to look at a library that has more control over the execution model, like Monix. I haven’t used monix, but it looks like it would have a lot of flexibility if you are working with things like Futures a lot, and in more sophisticated ways.

1 Like
#3

I’d recommend Monix as well, but if you need to use Future then the recover and recoverWith methods let you essentially write a catch block for a Future, e.g.

import scala.util.control.NonFatal

def testFailure2() = {
    val f1 = Future.failed(new Exception("ex1"))
    val f2 = Future.successful(2);
    val f3 = Future.successful((5));
    val f4 = Future.failed(new Exception("ex4"))
    val lst = List(f1, f2, f3, f4)

    val handleErrors = lst.map(_.recover {
      case NonFatal(e) => 0
    })
    
    Future.sequence(handleErrors)
  }
#4

You can use the same reasoning to separate for different cases and do something a bit more complex

import scala.util.control.NonFatal

val errorReplace = Future.successful(None)
def wrapSome[A] = (_: Future[A]).map(Some(_): Option[A])
def ignoreErrors[A] = (_: Future[Option[A]]).fallbackTo(errorReplace)

val skipErrors = {
  /* skipping f1, f2, etc. definitions here... */
  val lst = List(f1, f2, f3, f4)

  val ignoring = lst map (wrapSome andThen ignoreErrors)
  Future.sequence(ignoring).map(_.flatten)
}

def errorReplace(t: Throwable) = Future.successful(Left(t))
def wrapRight[A] = (_: Future[A]).map(Right(_): Either[Throwable, A])
def selectErrors[A] = (_: Future[Either[Throwable, A]])
  .recoverWith {case NonFatal(t) => errorReplace(t)}

val separateErrors = {
  /* skipping f1, f2, etc. definitions here... */
  val lst = List(f1, f2, f3, f4)

  val select = lst map (wrapRight andThen selectErrors)
  Future.sequence(select)
}
1 Like
#5

Thanks. Very creative way of doing it. Scala coding is art :slight_smile:

#6

I like to do that with foldLeft:

lst
  .map(_.map(Some(_)).recover { case _ => None }) // recover things
  .foldLeft(Future.successful(List.empty[Int])) { // combine it together
    case (accFuture, nextFuture) =>
      nextFuture.zip(accFuture)
        .map {
          case (Some(next), acc) => next :: acc
          case (_, acc) => acc
        }
  }
#7

A simple (but stack-unsafe) way is to use transformWith and general recursion.

def filterSuccess[A](tasks: List[Future[A]])(implicit ec: ExecutionContext): Future[List[A]] = {
  tasks match {
    case Nil => Future(Nil)
    case fa :: tailf => fa.transformWith {
      case Success(a) => filterSuccess(tailf).map(tail => a :: tail)
      case Failure(_) => filterSuccess(tailf)
    }
  }
}
#8

Thanks @ivanopagano, very clean and compassable solution.