Advice on interface design

Question
1

Hi.

Our system’s original designers did the RIGHT THING™, developed some traits, coded an implementation, and the worked against the traits.

Basically

trait Repository[A] {
   def upsert(uuid: UUID, item: A): Boolean
   def save(uuid: UUID, item: A, lastUpdated): Boolean
   def load(uuid: UUID): Option[A]
   def delete(uuid: UUID): Boolean
}

trait JobRequestRepository
  extends Repository[JobRequest] {
    def find(jobState: JobState): List[JobRequest]
}

object JobRequestRepository {
  def apply(): JobRequestRepository = new MongoJobRequestRepository(MongoUtils.defaultClient)
}

class MongoJobRequestRepository(client: MongoClient)
  extends JobRequestRepository {
    ...
}

class SomewhereInSystem(jobReqRepo: JobRequestRepository) {
  my reqs = jobReqRepo.find(JobState.RUNNING)
}

Great stuff and when we were told, “Mongo is out. Use postgres.”, then we coded up a PgJobRequestRepository, modified JobRequestRepository#apply and Bob’s-your-uncle.

For the easy Repositories. Along the way someone needed transactions and started using MongoThingyRepository directly so they could get at ClientSession#startTransaction.

Mongo has to stick around while we move the data to Postgres. We won’t be allowed production downtime as 75k jobs/day. I’ve created a class CutoverRepository[A, R <: Repository[A](mongo: R, pgrepo: R) which can write to the PG side and on reads, if needed lift data over from mongo as a side-effect. We’re hoping we can remove Mongo in about 90 days after this goes live. Oh. And I’ve got about 2 weeks to finish this up. :melting_face:

I’m looking for advice. It doesn’t seem I can reconcile MongoDB and Postgres (via quill-jdbc) transactions easily. Mongo wants the clientSession the transaction is running in so new method signatures. Quill doesn’t change signatures but wants to wrap the transaction so I’d need some method like def inTransaction[A](body: =>A): A to indicate what to put in a transaction.

Given the short time frame I’m thinking of finding the places SomeWhereInSystem that the transactions are getting used and making bespoke classes that can handle the correct transaction logic and wiring it together from startup config info. Ugly, but fast and would go away in a few months. (Such code always gets cleaned up, right? /shiver)

Raising this in case someone has faced something similar and found something a bit cleaner. Also raising it because I find it an interesting problem and thought I’d share.

2

Perhaps you can abstract your way out of this.

Java style…

  1. Add a definition of .inTransaction to JobRequestRepository that takes a lambda accepting a JobRequestRepository. The lambda performs operations on the supplied repository. It will presumably have implied side-effects in it; you don’t care and will revel in it anyway.
  2. Implement this method for Mongo so that it builds a forwarding implementation of JobRequestRepository that supplies that pesky client session to the original repository. It has to call ClientSession.startTransaction and the corresponding end-of-session thing around the call to the lambda.
  3. Something similar for Quill, but with direct use of the original repository by the lambda.

Scala style…

Come up with some free monad / tagless-final-object-algebra abstraction for running your transactions that abstracts over your Mongo / Quill choice, and makes you feel better about side-effect management.

Either way, you end up running your operations in some monadic flow.

I have a feeling that the object algebra approach will give you an algebra that looks a bit like your existing repository trait but with more flexible signatures.

I’m unsure as to whether transactions go into the object algebra analogously to the Java approach - or perhaps the object algebra should model operations within a transaction. Try writing against stub implementations to get a feel for what it looks like.

The free monad approach means you have to lift the repository operations according to the free monad cookbook. Again, I’m unsure as to whether that should be done for the original repository API, or just for operations in a transaction. Try it out - at least with a free monad approach, you get to write against stub code straightaway and can worry about the interpreters for Mongo and Quill later.

  1. Someone else can chime in with a simpler, idiomatic Scala solution.

I’d go for the Java approach first, then once the dust has settled, consider doing something more high-church. Or take a holiday and enjoy the outcome of the performance review. May the latter be a good one.:grin:

3

Nice. And thank you.

Was working this over the weekend and because of my lack of experience with Free/Tagless and the time crush I went the Java direction you laid out. I had one little wart in that I was hanging .transaction(...) off the repositories directly but when I looked at the code I was implementing it kept looking like one repo or the other was somehow special and able to run transactions. I introduced a case class wrapper around the respective clients so now the call sites look like myRepo.transactorInstance.transaction(...) so no question why one repo over another was chosen to run the transactions.

Another clean-up is introducing a case class named ${Subsystem}TransactionGroup which carries around the repositories that will be participating in transactions for the given subsystem. You can get at the (optional) implementation repositories and will have to figure out how to run your transactions depending on Mongo or Postgres being active but I’ve exposed methods that return the trait instances of the repository (doing the right thing for what’s active) so where you don’t need transactions you will feel you are just working with a normal repo.

When the dust settles I’m definitely taking on a side-project to figure out a Tagless solution to get that experience I’m currently lacking.

Thanks again for the reply. Gives me confidence that, if I’m not on the right track, then at least I’m not too far off the beaten path.

1 Like