Is Scala Too Complex?

Design
1

Years ago, I raised this topic elsewhere, and it became a big stink for many reasons… Not long after that, I left that forum because I was getting trolled too much, and shortly after that, they established long-overdue guidelines on discussions.

Recently I watched KEYNOTE Simply Scala Martin Odersky, which inspired further thought…

  1. When we use terms like “scala is complex” or “scale is simple,” we must be clear on what we are discussing.
  2. I agree with Martin that the Scala language is relatively simple in some ways, such as language syntax. And I also agree with Martin that some things, like SBT, are too complex.

Overall, it is clear that Martin strives for simplicity and has demonstrated outstanding leadership. However, sometimes the levels of abstraction going on make Scala feel more complex. I think I favour Scala becoming more opinionated, as Martin claims, as when I read other people’s Scala code, there are just too many ways to skin a cat, so to speak.

While Scala is my favourite programming language, I rarely recommend it to others because it takes a lot more discipline than Java and Kotlin not to abuse the power of Scala.

On another tangent, I look forward to Java 21 when Project Loom becomes more standard, and I will be watching closely to see how that affects both Scala and Kotlin. Already, I have used Project Loom APIs in both Kotlin and Scala with few problems. In particular, while Project Loom is impressive, I feel there are still lessons to learn from Scala, particularly wrt Capabilities, Effects, Resources, etc. But Scala needs a stable foundation of Virtual Threads, Structured Concurrency, Scoped Variables, etc., to experiment with.

2

Scala Center have commissioned a study to track complexity in Scala 3. (By a study, I mean a work-study student.) So far, they have been able to determine that Scala 3 was quite simple for a period of about 8 days after greenfield work was begun to rewrite the compiler. (Odersky rested on the 7th day.)

In the beginning, the compiler was totally formless and void. People often ask why does such-and-such a transform wind up in a given phase of the compiler? Well, on the first day, Odersky wrote the parser, which takes about a day. Then he wished to name those things which populated this new language, so he wrote the namer. Then came the third day, Wednesday, or “Hump Day”, which did not receive that nickname because it is a day for releasing stress. No, it is the day he created the typer, and since that turned out pretty good, he went on to create all the remaining phases according to their order. And, as already noted, after working the weekend, he had a Weißbier.

I hope this bit of cultural anthropology clarifies how Scala turned out to be as simple as designed but not simpler.

I’m sorry to hear you had a bad forum experience, Eric. I think the community leaders have done an important and commendable job at crafting the current environment.

The issue you raise here is still pertinent. I think it is expressed mostly as a question of which language features to prioritize. Everyone just wants their tooling to work.

My opinion is that Scala is not an academic language, but a working language that challenges working programmers. The chore of industry is to invent the jobs that need those workers.

4 Likes
3

Scala is a language that is difficult to appreciate and to see its simplicity. I also reacted the same way “way too complex, everything and the kitchen sink” when I first learned it. Also at first I really disliked the fusion of object-oriented and functional (I was in the functional camp). I had to stick to it for 3 years to appreciate it, and start teaching with it to see how simple it really is. I ported a lot of learning materials from other languages (Python, Ruby, Racket / Scheme, Standard ML, Java) and Scala is always much simpler.

So I think the best way to make someone “get” Scala is to teach them while they’re young and un-opinionated (and also not push to “sell FP” to them), and avoid showing the super-powerful advanced features / libraries at first. When presented as a neutral tool without any ideology / baggage / paradigm, people react quite positively to Scala 3 (because it’s not that different than Python in that form).

When it comes to more experienced people (who already have many firm opinions and expectations), I think it’s best not to keep insisting “hey Scala is SIMPLE! OK?” (which leads to a lot of knee-jerk rejections and makes people even more defensive) and instead be honest about it: it takes a long time to appreciate, and as you said, a lot of discipline not to abuse it. It’s the kind of language where you can refactor your code multiple times and find better, safer, simpler ways to achieve your goal every time.

One more universal pattern I see when it comes to Scala / Haskell is that, most people do not know ANY math. NONE whatsoever. Without solid discrete mathematics fundamentals on sets, relations, functions, logic, proofs, FP is too difficult to appreciate. The FP world tries to teach their languages by skipping this fundamental education, but it leads to more problems. I am over at Coursera discussion forums every day helping, and this is a very frequent issue with the programming assignments of the online courses. The language designers / course creators expect students of today to have received the same education as they did in the past. For example, Wadler says here that “the talk will explain… why you already learned the most important stuff in high school.” But THEY DIDN’T. I certainly did, in high school, but not people of today. Math education is in a much, much worse state today, worldwide. Even at college level.

The complex stuff that the community does with it is unfortunately natural. When a simple language puts powerful abstractions in the hands of clever, creative, opinionated people this is the natural result. (It’s a bit similar to what happened to Lisp). From a different perspective this could also be seen as a huge success. Have you heard of Chisel? RISCV was designed with it.

When you start a broad language discussion that sounds like “language XYZ is bad” (I know that’s not the intention, but…), unfortunately it will attract the trolls. Hopefully the community is better this time.

4 Likes
4

This gets said from time to time, but I passionately disagree. Indeed, I’m a little disappointed that Martin has been pushing this line, because it suggests he has forgotten one of Scala’s greatest strengths.

Restating the above: Scala doesn’t have a single clear idiom for coding. And that’s absolutely true. IMO, that’s the point, and it’s a good thing.

What Scala has always done better than any other language I know is to empower libraries to define the idioms, plural. And they’ve done so, quite nicely. We have Typelevel and ZIO, Lightbend and Haoyi, and more besides – library ecosystems that are optimized for different sorts of problems, each of which does have a distinct idiom and style.

There have been complaints lately the the Scala Center is too academic, disconnected from the real world, and I mostly consider that to be nonsense. But one way in which I think there is a grain of truth to it is that folks there seem to be drifting to the notion that Scala should be taught as a tabula rasa – a language in isolation. That is out of touch: it’s not how I’ve taught it for many years, and it’s not how people actually build serious systems in it. The real-world reality is that folks choose an ecosystem, and adopt their idiom from that. You shouldn’t build anything larger than a toy in raw Scala; increasingly, people take their very first steps inside one of the ecosystems.

Yes, Haoyi-verse centric applications look very different from Typelevel ones. That’s intentional, and that’s good – those ecosystems are typically chosen for very different problems with very different needs, and the different styles reflect that.

Let’s please not forget our roots as the SCAlable LAnguage. Scala covers a wide breadth because of how it empowers ecosystems; its flexibility is what makes that possible. Scala Center should be leaning into that flexibility, supporting those various ecosystems, not pulling away from them.

6 Likes
5

LOL…

I have been trying to help a colleague deal with some Scala code he has inherited. As a Python ML Engineer, he has many problems, and I often cannot give him simple answers. He faces a steep learning curve, and my boss would like to see all our Scala code replaced by some other language.

The original Scala coder has left our company; a Data Scientist who used Scala but was not a Scala expert. I think he was just learning it because we use SPARK a lot but are now using Beam APIs with Google Dataflow.

While I am competent in using Scala, I think I am alone, so I will not advocate for it, especially when my boss has negative feelings about it.

These days, my Scala coding is just for recreational purposes.

I really feel Scala 3 is much better, and Scala is going in the right direction. However, Kotlin has really taken a lot of the wind out of the sails of Scala, and I understand why.

My opinion is that Scala is an academic language, but with care and discipline, it can be a very effective general-purpose language for commercial projects and products.

I also think Scala should be used more as a teaching language, as it has many critical academic principles helpful in teaching Computer Science. I first learned programming in 1970 at age 12, and I wish I had learned Scala 3 instead of BASIC, FORTRAN, etc. I think children younger than 12 could also learn Scala…

2 Likes
6

I cannot say I disagree with you… indeed the unopinionated nature of Scala has many benefits as well…

I see many things, such as ZIO, CATS, etc., that provide more specific opinions on how to use Scala, and maybe when we talk about Scala, we need to talk more about such ‘opinions.’

  1. Scala/agnostic
  2. Scala/CATS
  3. Scala/ZIO

Maybe Scala needs a setting, such as a way to state in the code the opinion(s) the code should conform to. I am unsure how serious I am here; just thinking out loud.

The problem I am trying to address is that working in a particular team, we need the team to agree on some coding styles, some opinions, and some idioms, so can Scala help support this in code? Maveric coders can cause a lot of problems…

I don’t want to give up on experimentation in Scala, but for good teamwork, there is often too much flexibility for effective collaboration.

Years ago, I worked on a team where one coder used Haskell calling conventions unilaterally. At the time, I did not know what they were, and they added no value to our project. One reason I feel Scala can be too complex is that some people love the complexity and choose to abuse Scala.

The feature I am searching for is ‘Opinion Management’ – is that a feature or nonsense?

7

Something like this maybe? Community rules · Scalafix

Although I’d say that programmers have a tendency to look for external mechanical solutions to everything. Sometimes problems need to be solved internally with “soft” skills instead, like meetings and talking. Team issues are probably better solved at a human level. I guess programmers prefer more configuration, settings, libraries instead of… just talking to each other. :laughing:

2 Likes
8

Speaking from decades of experience, “just talking to each other” does not always work, and in more recent times, it works less often.

  1. There is higher turnover on teams, so new people are coming in more frequently, unaware of conventions.
  2. There is less and poorer quality onboarding, documentation, guidance, etc.
  3. While code reviews should solve such problems, I have rarely been somewhere with quality code reviews.

Anyway, I tend to disagree with most of what you say. I tried reading the Scalafix reference but could not get the gist of it. Like too much documentation, it lacks context. But thanks for referencing it, I will study it more later.

I don’t know if there is a solution to the opinion problem; I was just thinking out loud, maybe not defining the situation well.

  1. I agree with Martin that Scala would benefit from being more opinionated.
  2. I also agree with @jducoeur that sometimes, we need a more agnostic environment in which to experiment until we can form opinions.
  3. Can we have both?

I love that Scala supports both high-quality imperative programming and high-quality functional programming.

Maybe the solution is in tools like https://errorprone.info that hook into the compiler… this works for Java, but I am not sure if it could work with Scala… technically it could work if scalac has the right API…

Anyway, I imagine a team could write their own rules to detect opinions the team does not endorse and flag these at compile time. For example, it could catch Haskell calling conventions and flag it as a non-conforming team style.

9

Really, I think a lot of the solution is to agree which idiom this project is going to go by, as early as possible, and stick to that. My last company was Typelevel-based – we followed TL idiom, and gradually helped each other level up in that. My current project is ZIO-based, and it’s much the same way: while I don’t agree with every design choice in ZIO, it’s the world we’re working in so I follow its idioms.

That doesn’t solve every problem, mind. Every project I’ve ever been on has arguments about naming and project structure. But there’s nothing unique to Scala there: if you’re building enterprise software, there are always a lot of challenging decisions. The heart of dealing with that is always about agreeing some ground rules, and living by them.

Granted, I’ve only worked at places with a strong code-review culture for the past six years, and that does help immensely, as does having a team that is good at having honest debates without people getting excessively stubborn or personal. Frankly, I find it hard to produce good software at scale in any environment without those.

5 Likes
10

Granted, I’ve only worked at places with a strong code-review culture for the past six years

I am envious. Good words of wisdom. Thanks.