Golang developer trying to migrate to Scala

Question
1

Hello! I’m a Go developer that has been programming on it for many years. I’m very comfortable with the language, including the bad parts. Is a tradeoff that I accepted and I understood the guidelines that I have to follow to build robust software. I did programmed in other languages before Go, but it was with Go that I had for the first time the felling that I actually “cracked the code” and I was able to do anything I wanted with it, specially concurrent applications. It’s such a joy to work with non blocking IO by default, goroutines, select, and the race detector.

But, the company that I’m working for has decided to migrate to Scala. Long story short, it was a decision coming from the CTO and from this moment on, everything related to backend should be in Scala. At first I thought that it was ok, but I never had to learn anything more difficult than Scala in my life. The cognitive load is unbearable. I understand the point of being “scalable” but there is just way too many ways to do the same operation, which is the absolute opposite of Go.
The fragmentation of the stack doesn’t help as well. We have services in Play, others in Akka HTTP, then some in http4s. We also have mixed with these services cats and scalaz.

I’m trying to get some feedback from folks that have worked with Go and Scala. These are the questions that I have:

  • Did you migrated from Go to Scala? Or work directly with Go. If yes, how it was your experience? Do you miss Go?
  • Should I go with Scala 3 or 2? The future is 3 for sure, but most of the services are on 2. If I learn 3, can I still work on 2?
  • What is your suggestion to get the language as quick as possible? I’m reading Odersky book right now. I’m on 1/3 of it. Any other suggestions? Books, trainings, videos?
  • How do you feel about the future of Scala? Scala is such a niche language right now and the constant flux of changes in the language is not helping (from my point of view). I don’t want to put so much effort to a language that if I quit my current position I’ll not find another one to work on.

For a moment I thought that the FP concepts were the difficult part, but I’ve been doing Elixir for some time and it’s easy. It’s way closer to Go than it is to Scala in terms of easy of usage. Maybe the issue is the FP + very strong type system.

I don’t know if I’m just too dumb to learn Scala, if Go broke my brain, or if this is normal and others have been through the same situation. But at the same time I feel blessed because many would want to be in a position that I’m right now and don’t have the opportunity. I just don’t want to give up too quickly on Scala, even though this is my deepest desire right now.

Thanks for the help!

2

Haven’t worked in Go myself, but to a few of the other points:

If most of the services are on Scala 2, it may be easier to just learn 2 for the time being. 3 has a bunch of nice improvements, but if you can’t use them it’s just going to be a bit annoying.

I don’t recommend Programming in Scala as a starting point – it’s not terrible, but it’s the language bible, and is pretty heavy and over-detailed.

I usually start folks out with Essential Scala, which is tight, focused, and clear. Do all the exercises as you go. Mostly ignore section 5.6 (Variance), which will break your brain for little benefit.

I disagree, but you’re obviously in a biased space here. I use Scala for everything – front end, back end, and scripting – and it works nicely for all of those so long as you pick the right library and compiler stack. And while the language is constantly shifting, it tends to be pretty good about back-compatibility.

Try out Essential Scala, work your way through it, and have a bit of patience. I usually have folks spend about three weeks on that part-time, with questions and a bit of tutoring to get up to speed. And do ask questions!

3 Likes
3

I don’t have much experience with Go but I may try to help with some of the Scala parts.

Sadly yeah that is true, and I also agree it would be better if that was not the case for a lot of minor things.
My advice is try to reduce your options, like

  • Pick a consistent formatting style and enforce it with Scalafmt, even if you sometimes dislike it.
  • Always be immutable even if you have stuff like var and while
  • Always handle errors in the same way
  • etc

This is really a bad sign.
Maybe I misunderstood the way you related the story but I had the impression your company just started migrating. As such, there should not be a mix of ecosystems.
If this is something that could be tackled I would say pick just one (like my previous advice). But, if there is some legacy coming from somewhere else then that is harder to achieve. If anything, everything new should be in a single ecosystem; or at least you focus on only one for now.

My personal advice would be to focus on either typelevel or ZIO. I prefer typelevel for a bunch of both technical and personal reasons, but the two are great technologies and similar enough so that learning one means switching to the other is easy.

This is a FAQ: Scala FAQ | Scala Documentation

I would say:
First, get the basics of the language. The Odersky book is fantastic but is the bible, is very long and goes very deep. You may try some lighter resources like Scala for the impatient, Creative Scala, Scala Programming, Essential Scala, etc. Check Books | Scala Documentation

Then, when you are comfortable with the basic language syntax, the stdlib, ADTS & pattern matching, and immutability you can focus on the ecosystem you picked. Sure, you are probably skipping a lot of things like variance, kinds, typeclasses, implicits, macros, etc. I know all that can feel intimidating, but most of those are only useful for library authors and you can pick them as you go. Let’s focus on being practical.
If you end up picking either typelevel or ZIO the main mental switch is that they both are based in a different programming paradigm. I like to call it “Programs as Values” and I have some resources about it here: GitHub - BalmungSan/programs-as-values: Source code of the programs as values presentation (feel free to ping if you have questions about this).

Also, skip anything Category Theory related, that is a fun topic but you don’t need to learn it to do FP. Even libraries that are based on those concepts like cats are far from the theory. Also, speaking of cats, you will sooner than later use it, I would advice seeing it as a utils library, you only need to add import cats.syntax.all.* and that is it, you get a bunch of cool methods for free. No need to worry with what the fuck a Monad is.

If you haven’t see them already both DevInsideYou and RockTheJVM are great YouTube channels about Scala.

I would say the future of Scala is pretty stable.
Sure, nobody expects everyone and their friends will migrate to Scala in the near future, but the job market is there and AFAIK growing.
Also, if you ask me, Scala is the best language to write concurrent systems thanks to the already mentioned “Programs as Values” paradigm a nothing is close to beat that. The closer would be “Effect Capabilities” which is pretty much still in research now and funny enough Scala will get its own flavor.

It may also help if you show concrete stuff that you in particular have found hard.
We may try to help / explain.

I would say is quite normal.
Scala tends to break a lot of stuff that is “normal” and telling people to re-learn programming 101 tends to be frustrating.
All I can say is that it will definitively pay off, even if you end up quitting and hating the language (which I hope doesn’t happen) I am sure you will end up being a better dev just for being exposed to different mindsets and learning from those.

Just give it an honest try with an open mind.

BTW, final advice, please join the Discord server: Scala chatting is a great way to ask questions and solve issues. Also, we usually have interesting discussions and you may learn a lot just from reading that.

Welcome to the community!
Good luck and happy coding :slight_smile:

2 Likes
4

You’re not dumb, and it’s quite normal yes.
(Although Go is quite the opposite of Scala yes.)
Unfortunately Scala takes a while to learn properly.
It takes even longer to appreciate.
It’s a “big investment with big rewards” kind of language.
Not because of FP but its unique blending of OOP, FP, and imperative.
People’s tastes tend to be in one camp.
I was the opposite as you, coming from the purely FP camp (Haskell) and initially hated Scala, but now I appreciate it way more than any other language.

Writing Scala code often requires constantly switching gears in your mind, jumping from paradigm to paradigm, connecting different mental models.
Yes it’s a lot initially, but once you get through this phase it becomes a superpower.
If you are patient, you’ll see that it’s a great language and you’ll feel that “cracked the code” feeling much more strongly than you ever did.
(Although having to do this at your job must be painful, I feel for you :smiley: )

Yes this will require a mentality change from you.
I’d say, drop by the Scala Discord often, and ask for quick feedback like “is this the common / idiomatic way?” and others will give you quick responses.

Hortsmann’s Scala for the Impatient is probably the best option right now as others pointed out.

The most “baby” book is probably Daniela’s Get Programming with Scala. The best part is that the code is Scala 2/3 interchangeable, with a few Scala 3 parts clearly marked and optional:
Captura de pantalla de 2024-10-10 09-45-38

It also starts you off on http4s basics fairly early on:
Captura de pantalla de 2024-10-10 09-49-50

I was going to say, maybe it would be too easy for you, but coming from Go, it might be a good match. The lessons are fairly short and quick.

Other advice would be to continue your Scala education even after you get basic competence enough to do your job. The online courses will show you the language more deeply.

Best of luck :smiley:

3 Likes
5

@BalmungSan made the same comment. Indeed. I’m spending a ton of time on that book trying to get every single detail, of every single feature, but still missing the bigger picture.

Sorry about that. This is me skipping a part of the story. We have a bunch of Scala engineers at the company and we’ve been using Scala alongside other languages. The decision was to stick with just a single language at the backend, and given we have more Scala engineers and projects than anything else, Scala was the choose language.
The mix of frameworks and paradigms comes from the long history of Scala here.

I would say that my biggest barrier right now is the for comprehensions and the myriad of ways to compose computations there using FP. .liftT.leftMap[Err] these kind of things like unpacking and packing the information to fulfil the type system checks. I found it very complex and even using things like ChatGPT to try to explain the code, it is a hit and miss. The other parts of Scala like objects, trait, class, etc… it’s okish because I can relate to the other languages that I’ve used before.

Right now I’m trying to gather facts to evaluate the value proposition of the language based on its complexity. Something can be very complex, but if there is real value behind it, that’s ok. But if the same could be done with something way simpler, then why go to the complex path? I may be over simplifying this but right now my comparison point is Go, a language that I did amazing things that are running stable in production for years without ever having any kind of runtime exceptions, and with a very very low bug rate.
My comparison point with Scala are the services that others have built here in the company. Most can’t be easily updated, actually it’s quite hard to update, they always complain that they can’t understand the code, it’s full of bugs, uses 10x more resources than the Go services, etc. Maybe I’m just getting a bad counterpart and this left the negative feeling of Scala in me.

Sometimes I think about this. If Haskell would be simpler. How can I stand by the rules of FP when consuming Java libraries given that internally it can be muttating and throwing exceptions? A closer example for me is Elixir. There is no way to, easily, mutate data. You’re forced to follow more closely the FP paradigm, more than in Scala I would say.

Looking forward to it :sweat_smile:

1 Like
6

Maybe make this argument to your boss? :sweat_smile: Indeed Go might be a better fit.

Again, have you told your boss about this? If people at the company are bad at the language, then the language itself doesn’t matter… as BalmungSan said, it’s a bad sign there is such a mixture.

As for memory usage, it’s probably mostly a JVM thing, it was never designed for tiny microservice type things. Scala now has Scala Native which can massively cut down on resource usage if needed. All the major frameworks (Cats, ZIO etc.) support Scala Native. When properly used, these frameworks have very good performance and low resource usage (probably not as low as Go though, but close enough).

Maybe look for another Go job? :laughing:
Scala is amazing once you learn it well, but I’ll admit it’s not for everybody / every environment.

Don’t hold your breath, it takes a while! :smiley:

1 Like
7

I don’t know whether you have recently joined the company, having worked wih Go elsewhere, or in your own time. Perhaps you are a consultant, or a new project manager? Maybe switched teams in the organisation?

However I do feel you owe it to your employers / client / new team to give things a fair shake. If you don’t put in the effort, don’t be too surprised if your path to a new position gets suddenly, ahem, ‘expedited’.

OK, so if you’ve got a whole array of code in production that presumably isn’t completely broken, is it an easy path to completely rewrite it in Go? Will those bugs all magically disapper, or will they be replaced with new ones to call your own? I assume the existing Scala engineers will have to be retrained in Go. They may not approach that with relish. Or is there perhaps some other team that is making a case to do this?

Which ‘they’ are we talking about here? I assume the people who wrote the services do at least some extent understand their own products’ code. Again, is this some other team who are unfamiliar with the code making an assessment? Or has there been massive churn at the organisation? :grin:

Comparisons between actual code and hypothetical code using some other technology are cheap to hypothesize, but has anyone got an example of a production Go service and a production Scala service doing roughly the same thing? Do they show that 10x resource disparity? In CPU? In memory usage? In wall-clock time? Over multiple services? Using the same algorithms under the hood?

I’m probably barking up the wrong tree here, but maybe there are issues with the team looking at the codebase. Perhaps some skills need to be learned. Or maybe some code needs an overhaul, and a bit of “hard-assed bug fixing”, to quote that Joel Spolsky character.

If that’s the case, would said team (or a replacement team) really be able to do substantially better in Go?

These are all rhetorical questions, no need to respond (and perhaps not a good idea to go into too much detail), just something I’d be thinking about if I were in your shoes…

1 Like
8

Ah – yeah, that can be a pain. More communication can help over the long run, but it can be confusing.

My previous employer, Rally Health, was somewhat the same way – fairly big company, built mainly on Scala, with tons of microservice teams using different stacks like that. We found that talking about the tech and stacks regularly, both formally and informally, and building out a rich ecosystem of internal shared libraries, helped keep us all moving in the same direction despite that. Despite having many projects based on “Lightbend” style, some in cats-effect, and some in ZIO, we found that we could all share lots of code so long as we worked together.

But it absolutely required strong leadership per-project, to decide which stack to focus on (and why), and to make sure everyone on that project understood how to use it well.

Well, one bit of value is that Scala, used well, is generally one of the lowest-boilerplate, lowest-code-duplication languages around – you can generally factor your code better than in most languages, and keep things more concise. That’s not a small matter: better factoring generally results in lower bug counts and easier maintenance, which is critical for keeping a company moving in the long run.

Complexity isn’t a single dimension. Scala as a language is somewhat more complex than Go, yes. But the resulting code is generally tighter and less complex if you’re doing it right.

(This all depends on decent training, though. I’m a little concerned that your company may not be putting in the effort to teach the language and share knowledge around, and that’s important for any project’s success.)

That bit isn’t generally a problem – you just have to make sure that you’re encapsulating the Java layers. I usually write a pure Scala layer around the impure Java one, which typically is fairly straightforward and frequently already exists. (Although it obviously depends on what you’re wrapping.)

Yes, there’s impurity under the hood, but there’s nothing odd there: when you delve deep enough under the surface of any pure-FP engine, there’s usually some impurity needed for optimization. The key is to make sure that the impure layer is itself deeply tested, and the mutations are hidden away from the pure business logic.

3 Likes
9

Makes sense.
As @jducoeur said, it would be good to start a standardization initiative.
That doesn’t mean migrating everything immediately. But, at least, new projects should be done using a single stack, creating shared libraries, gradually migrating active stuff, etc.

This also relates to another point Justin made.
Given the decision if the company, it would be expected to provide training to everyone.

What exactly you find confusing of for?
BTW, for the record, I think is best starting writing the flatMap and map methods manually. I even still avoid for now, but for other reasons.

Okay yeah.
This relates to the “Programs as Values” paradigm that I mentioned.
Take a look to the resources I shared, they may help.

But yeah, it seems you are struggling to think with the types.
This may help as well: Playing "Type Tetris" - Underscore

So, again, if you ask me.
For concurrent and complex back end systems I would say Scala is simple the best language there is for that.
Effect systems make managing concurrency easier, and the strong type system makes modelling complex domains simple.

So yeah, as other said.
It is true that Scala as a language is very complex. But, usually, good Scala programs are simpler even when dealing with complex problems.

Well, this is weird to hear.
Usually, we see the opposite, but I guess is also a mater of opinion and perspective.
It may also be that those were written by bad Scala devs. And, TBH, Scala does have an issue that it indeed requires you to be proficient in both the language and programming in general.

Like a bad dev can write normal Java / Python / JS code, because there is not much to do wrong at the basic level. But a bad dev will indeed create a mess in Scala.
That is why training and pairing should be a priority for your company right now.

About resources, yeah that is fair and is mostly the JVM.
But also could mean a bad deployment strategy. You want fewer but bigger replicas, which is the opposite of other languages like JS or Python where you want smaller but multiple replicas.
That is because those languages and their runtimes are terrible at managing more than one thread.

1 Like
10

Well, the decision was already taken. I don’t think anyone will revert it and there are others very happy with the decision to stay on Scala. I don’t want to be the special snowflake that is not programming on the favourite language. But from my current knowledge or view of the language I would not pick it for personal projects, but I’ll do my best to be good at it for my day job. Maybe this feeling will revert at some point.
In my opinion Scala is the type of language that is very extreme because, if the code is good, it indeed really good, but if the language is abused, damn, it can produce some nightmares like in C++.

You would be surprised by some JVM flags that I see here in the company. Crazy things like requiring a ton of memory, and then limiting the heap to 0.2% of the requested memory, or sometimes just booting the application with the heap locked over the required memory limit which causes the service autoscale to the max. These things for sure contributed to my poor impression of the language and it’s not even the language’s fault.

Poor market conditions right now, but I would jump to another Go position for sure. US has tons of positions, here in Europe is harder to find good ones.

The company has over 10 years and never had a strong technical lead, so teams were just doing and experimenting with whatever they want. Eventually more teams went to the Scala route, and then later, Go. But, we still have way more code and engineers doing Scala. It’s not an easy decision because both Go and Scala have very different paradigms. There is no way to make a choice that would make everyone happy because. The folks doing Scala doesn’t want to program in Go, and the folks doing Go doesn’t want to program in Scala.

I don’t have enough knowledge to evaluate the Scala understanding of the folks here at the company, but I would say that they’re not good.
We had a few core services that were flagged as “deprecated” because they, the core Scala engineers, could not add simple features and maintain the services. Those services eventually got transferred to my team and we’re being doing all the changes that management is asking for. It’s freaking hard? Yes. But it’s still code.

That is a fair point. I would say that if they were built using the same techniques they would probably have similar statistics. But, being fair with Go, doing the naive approach in Go is usually the fastest and more robust way. There is no need to worry about thread pools, blocking IO, and concurrency. What I usually see on the Scala side is that operations that could have been done in parallel are being done sequentially for the sake of simplicity. Again, not Scala’s fault and more of a skill issue with the team.

With the for, nothing. But it gets very complicated when you have a 50 loc for comprehension mixed with scalaz and akka. The lifts, pures, maps, flatmaps, the conversion from one side to another is what is confusing me right now. Not even the editor presents the types correctly :sweat_smile:

I really hope to stay in Scala to get to this point. :crossed_fingers:

Not really, this is a design choice. All Go code written since 1.0 still compiles today without modification. This is a guarantee that the language did. When a new version is released, I can simply deploy to production at the same time without worring if my dependencies got updated or not.

I think this is pretty much what happened here. My expectation was that a code from a senior dev in Scala would be really simple given the expressiveness of the language, but here it’s the opposite, the more senior devs are trying to “be smart” and the code looks like a math PhD dissertation with custom operators all over the place. Reminds me Clojure :sweat_smile:

I decided to start a side project in Scala. It is an idea that I had for quite some time and decided to implement it. This is the way I learn things, by doing it. If all goes well, I’ll make a lot of money and quit my job, if not, I’ll have learned Scala and keep the job. Not a bad trade hehe

1 Like
11

100% true, although the community has gotten pretty good at discouraging those anti-patterns. (And the language is gradually evolving in ways that discourage them more inherently.)

Yeah, that sounds like some naivete in using the environment. Relatively little I ever do in Scala is sequential – indeed, the reason Play took off in the first place is that it’s relatively easy to do good parallelism, which was unusual at the time. And with the modern FP frameworks (cats-effect and ZIO), it’s common to have massive parallelism that is relatively fire-and-forget: you think heavily in terms of streams of data (fs2 and ZStream, respectively), and the concurrency is largely handled by the engine itself.

That’s a rather unusual combination, yeah.

If the code is decently pure, then the first thing I would do is refactor it, breaking that huge for comprehension down into more-comprehensible functions. But that depends on the code being written appropriately in the first place. (Scalaz would tend to drive you in that direction, but traditional Akka and for comprehensions tend not to mix as well – indeed, I wrote a whole library specifically to make them work together a little better.)

Yeah, that’s an old habit that the Scala community has to a fair degree moved away from. But Scalaz was always relatively prone to it.

1 Like
12

Excellent - wishing you success, be it in either making a living off your works, or an accelerated path to grappling with the day job.

Two things from my side:

  1. Regarding your slogging through the Scala Handbuch, I found that was alright back in the day, but I had a fair bit of exposure to F# (and some prior OCAML) beforehand. So how did I get to learn that?

A book that helped me immensely was Expert F#, mostly because it didn’t set out to preach the virtues of functional programming, rather it walked the reader through lots of pithy examples that were convincingly related to real world problems - although the reader was subtly guided in the direction of functional programming, but not all the time; it used impure code too, and that’s fine by me.

There is a similar book, Real World Haskell that takes a similar approach, although this time it’s functional programming all the way, with programs-as-values, Haskell-style.

I recall a discussion on a corporate Discord server a while back with a chap @nrinaudo and others where somebody suggested the equivalent book for Scala. I think this might have been Scala for the Impatient, perhaps someone would confirm?

  1. When you’re rich and famous and appearing on talk-shows, don’t forget us little people. :grin:
13

No, almost the opposite – Scala for the Impatient is written for the experienced OO programmer who wants to get into Scala. It’s quite efficient at that, but it doesn’t do a lot with the FP idioms. It’s a very efficient starting point for a Java or C# engineer (and I often recommend it in that context), but I wouldn’t use it to teach FP.

Nowadays, I start most folks on Essential Scala (a bit out of date by now, but still great if you’re focused on 2.13), then move them on to the first half of Scala with Cats to get the grounding in the most important typeclasses (the second half tends to matter less), and then Adam Rosien’s Essential Effects to get the pure-FP programming. (Even if you’re doing ZIO, the ideas are still largely the same, so the book is still relevant, and it’s well-written.)

4 Likes