CLOS constructors

Scala’s rules on constructors are very irritating. At least I think they are. I’m a beginning Scala user and could be very wrong.

Using Java classes with constructors is very irritating. Classes like RuntimeException have to primary constructors. The null constructor, “()” and the detail constructor “(message: String)”. Making an exception class in Java is a common pattern. However, I can’t find a way to call the super constructors. I do understand the difficulty in languages with multiple inheritance (classes and traits). It is less than straightforward.

Using “extends” does signal the main inheritance class, so those could be used for identifying the proper “super”. Both traits and classes may be used for the main line. That creates a discord between using “extends” and “with”. A trait constructor might be available or might not be available. As with method calls, the actual method being executed is extremely complex.

CLOS (Common Lisp Object System) does allow a solution. CLOS is fundamentally different than most Smalltalk based languages (C++, C#, Java, Scala). In those languages there are few, if any functions. There are only methods. The functional languages have true functions, but the only can be created withing classes or traits.

CLOS classes have no methods. There are only functions. Structural multiple inheritance exists at the class level, but that only defines structure. Traits in scala is kind of like that, but there are implied accessors as well as methods. Even structure is difficult to manage with traits.

The primary difference between scala’s multiple inheritance and CLOS’s is CLOS only has functions. There are no methods. That means that multiple inheritance only exists at the function level. A single define function, bound to a symbol, takes zero or more arguments. Each argument can be typed and the type can be a class graph. Again, the class is only structural. The method can specialize on the class graph.

Here comes the big difference. The function may also be overloaded (using the scala definition). The parameter can any type graph. Class graphs and overloading class graphs are identical as far as the language is concerned. In scala, the equivalent would be a single virtual type that is a class that is a subclass of all of the class and overloaded types. The closest that scala gets to this are traits inherited via “with.” This gives scala all of the complicated type graphs of CLOS, but few of the benefits.

In Smalltalk derived languages, every function has an implicit first argument (self, this). But the function is a method, not a function. It is bound to a class. The true functions may not implement true multiple inheritance. They are limited to a single class graph. The parameter is a hamstrung type. CLOS has no methods, and must handle the type graph differently than scala. It is more general. It is equivalent to types that can be multiple, optional +T.

To deal with the issue, it has the call-next function call. It solves the same type calculus as the class/trait calculus. As we all know, with both +T and -T type graphs. In the large, this might not stop. In reality, it does. The developer’s brain would blow up. Programmers quickly learn not to approach the problem. CLOS only has +T type graphs [last I used it]. The type graph does halt. This allows call-next to determine what to call next. In scala, type calculus is declarative. That makes it run-time cheap. A dynamic function call graph never happens at run time.

Mostly - monads complicate the problem with higher level functions. Scala does do a good enough job. It punts when it doesn’t. Not halting means there will always be some times where there is no solution. Either the compiler will terminate, hopefully with an error, or hang trying to compute non halting type calculus. At runtime, it is an infinite loop.

So the problem exists in scala with methods. Scala doesn’t allow this with constructors as a special case. It gets close to pretending it does “def this” but doesn’t really handle the problem. It treats constructors as second class methods and not at all like functions. The problem isn’t addressed with constructors, somewhat handled with methods, almost handled with higher level functions. Calling (or blocking) the +T graph super methods is just skipped. With high level functions, -T can cause problems. But (I think) not at run time. Where it can’t handle it, the problem is just punted. Punting makes the language simpler, but irritating.

What is really irritating is that constructors are handled differently than every other method. Functions can be overloaded as methods, they can’t be overloaded as constructors. That irritation can be handled by making constructors no different than any other method.

The proposal is

  1. Treat constructors no different than any other method. Limit call-next to “extends” and disallow “with”. Call-next is limited to super calls. Traits still get treated differently depending on how they are declared.
  2. Treat all methods the same, don’t treat “extends” and “with” differently. Use “super-next”.
    2.5) Treat overloading no different for methods and constructors.
  3. Treat all functions the same.

The options are ordered by relatively simple to difficult implementation. Constructors are no different than methods. The limits and generosity of both is annoying. The really are identical. Both constructor and method functions just have an implied first argument.

@tyohDeveloper, I don’t completely understand your proposal, but I echo your frustration about constructors being handled differently than other methods. In CLOS, the constructor is just a method. responsible for populating the content of a newly allocated (or newly changed-class) object. I’ve always found it bizarre that languages like C++ (and I assume also Java) make a special concept call constructor with special rules, but don’t allow the same flexibility to other methods.

The result in Scala (and C++ etc) is that there is some rule for multiple inheritance, but it is too complicated and scary for people to use. So people avoid it. I can’t count the times people have told me, to avoid multiple inheritance because of the diamond problem. It’s as if the problem doesn’t even exist in CLOS.

I recently encountered this in Scala, and was surprised at the results. I wanted to write a toString method which wrapped the previously available toString functionality for a particular type. What I found was that it is really not possible, for a reason more complicated that I was able to understand.

That’s unfortunate :frowning:

This is the norm for object-oriented languages – for better or worse, CLOS is the outlier here.

That’s odd – it certainly works in the general case. What were you trying to do?

In Scala, every class has one and only one primary constructor. All other constructors are auxiliary constructors that call the primary constructor. This allows Scala to fuse the definitions of the class and the primary constructor, which usually saves a lot of boilerplate.

Without this fusion, you would write a class something like this (not legal Scala):

class User {

** val name: String**

** val id: Int**

** val dir: String**

** def User(name: String, id: Int) {**

** this.name = name**

** this.id = id**

** dir = “/user/home/” + name**

** }**

}

Fusing class definition and primary constructor definition allows you to write this as:

class User(val name: String, val id: Int) {

** val dir = “/user/home/” + name**

}

Or, as a case class:

case class User(name: String, id: Int) {

** val dir = “/user/home/” + name**

}

The well-known limitation is that only the primary constructor can call a super-class constructor, and therefore only one super-class constructor can be called per class. I also ran into this when I wanted to extend (Runtime-)Exception. If you really want to call more than one super-class constructors, you would need to create two sub-classes (which may, if that helps, extend some common trait).

if you find yourself in this situation, see In Scala, how can I subclass a Java class with multiple constructors? - Stack Overflow

I agree that it has become normal in the statistical sense, but it is fundamentally broken in my opinion. I shouldn’t have to know which classes my superclass inherits from, or which classes in the class precedence list define which methods in order simply to call the method.

@jducoeur, I also find that it seems to work in the simple case. But as I understand, this is only an illusion, and in fact it does not work in the most general case. If I’m wrong, I’ll be happy to know it. Can you explain what super means?

This question was discussed in another post, and the result I came to (perhaps I misunderstood) was that because Scala class lineraization does not guarantee that the topological constraints be maintained, you cannot depend on the semantics of super.

I’m not sure I understand the question. It means what it sounds like – call this function in my immediate super-type. The formal definition is here, but I’m not sure that’s what you’re looking for.

On a technical level I kind of get where you’re coming from here, but honestly – in 11 years of working in Scala, I’ve never actually experienced this as a problem, nor heard of it being one. I’m not sure if the difference is that you expect to do much more complex inheritance than is typical in Scala or something, but I’ve never actually seen this come up as an issue in practice.

1 Like

Linearization rules should be seen as a way for the compiler to handle more cases, not as something people should rely on.

In general, if your type inherits multiple implementations of the same method, that’s a red flag that the implementations are too high up in the type hierarchy or that inheritance has been overused.

If it happens anyway, you should provide your own implementation, even if it is just a forward to a super implementation, which you invoke by explicitly stating the supertype ( super[TheSupertype].method).

e.g.

trait A { def greet: Unit }

trait B extends A { override def greet: Unit = println(“Hello!”) }

trait C extends A with B { override def greet: Unit = println(“Hi”) }

trait D extends C with B { override def greet: Unit = println(“Yo!”) }

class E extends C with A

class F extends E with C with D { override def greet: Unit = super[D].greet }

For the most part I agree. Scala’s special treatment of constructors, and primary constructors in particular make the problem much worse. Using vals (finals in Java) make it problematic in the simple case. Particularly when wrapping Java classes, all Exceptions being the most irritating.

In general, I believe, all nominally structural members should be “def” in traits and abstract classes. The implementations can then use vals or override the def.

If I use val message=“” in a vase constructor, it creates a new structural item rather than reuse the message instance variable from the Exception class. I can’t use the null constructor or the message constructor of the super class. There is no way to avoid hiding the structural member of the superclass. The semantics can’t be represented.

Using a call-next pattern and treating both methods and constructors as functions rather than weak methods and weaker constructors would solve this problem. The simple, weaker case of being able to call “super” methods/constructors of the extends would solve the greatest irritation w/o changing the semantics of scala too much (or have a high implementation cost).

CLOS has the around, before, primary, and after function pattern built into the language itself. That addresses 90% of the problem w/o complicating the design too much. I don’t suggest adding that to scala. Treating constructors and methods as functions would be a v4 change. Doing it in dotty would be too drastic. Even adding super for the direct inheritance chain would be possible in 3.1+ without too much effort. I think. Any change to the language needs that much research to jump to conclusion.

For the non-structural problem, higher level functions and monads can solve most of the issue without changes to the language. There are lots of issues with the structural issue that can’t be addressed.

Thanks for the reference, that really helps. Here is how I understand the formal definition. super does not really have a meaning. Only super.m has a meaning. And it means method m in the next most specific superclass which defines the method, but with the restriction that that can be determined statically. I.e., super IS NOT the super class, but rather super.m is the next most specific statically determined method. And the syntax super.t has an analogous meaning in the case t is a type rather than a method.

That’s how I understand the description. However, I don’t understand the designation least proper supertype. What is a proper supertype, and how does such differ from any other genre of supertypes?

@curoli, I’m not sure how to interpret this statement. There seem to be two philosophies held by different people. 1) depend that the language work as specified, vs 2) don’t use elegant features.
In my opinion, a feature is added to a language so that programmers should use it.

Also you comment that

What (respectfully) bothers me about this comment is that it might not by MY class—I’d like to inherit from a class without having to understand how it is implemented. I should not have to reimplement features already implemented in the class I’m inheriting from. And conversely, when I implement a class, I should do so in a way that other applications can inherit from it and it will work according to its documentation.

That all sounds correct to me.

I don’t think this is correct – I don’t think super.t means anything in this sense. (But please keep in mind that I’m at the edge of my expertise here.) What I get from reading the spec (which I hadn’t known about at all – I don’t know if I’ve ever seen it used in the wild) is that super[T].m means that you want to call the definition of m on the specific ancestor type T.

Honestly, I’m not sure. You can find the definition in here, but it’s a tad opaque. The spec is definitive, but sometimes (in the name of being precise) makes up jargon that is not otherwise in common use.

1 Like

I second you claim that the definition is opaque.

The least proper supertype of a template is the class type or compound type consisting of all its parent class types.

Keep in mind, at least some of us don’t think of linearization as a feature per se – rather, I think of it as simply a rule to tame diamond inheritance. I mean, the problem of diamond inheritance is non-determinism: not being able to predict which supertype is being called under some circumstances. Linearization makes that explicit and deterministic – which is lovely, but I think of it more as a bugfix than a feature.

That makes sense, but I’m not clear on how it is different from the current state of things. What would you like to be able to do that doesn’t work as you would expect?

@jducoeur, that’s an excellent question. Actually, I don’t have an example. I was afraid to use super.m and expect the same semantics as CL call-next-method. But according to the discussion, it seems to be intended to work the same way as I expect. My hesitation came from discussion I had earlier saying WARNING don’t depend on multiple inheritance or linearization.

In CLOS linearization is indeed seen as a dependable feature, and its consequences and limitations are all understood and accepted by most intermediate and advanced programmers, so much so that nobody even cares about or mentions the diamond problem. Moreover, the CLOS metaobject protocol, allows users to create a subclass of standard-object which overrides the class linearization method on a per-application basis.

Since in CL, there is no concept of final class or final method, this causes programmers who implement libraries to be very conscious to allow classes to still work when people extend them.

Another part of my hesitation of using super.m was because of comments in this post. Which led me to believe that delegating to the next method was a bad idea, and I should refactor my code when I discover a need for it, and for example use composition a chain-of-responsibility pattern rather than inheritance.

Nevertheless, since it seems super.m works as expected, perhaps I should embrace it and just start using it, and see what happens.

Ah – that was either mis-stated or exaggerated. While I’ve found OO approaches to be slightly limited (compared to the FP typeclass-focused style), and have gradually backed off to using them more modestly myself, they’re generally well-defined and consistent in Scala, and have been for a long time.

Hmm – not sure I grok the use case, but okay…

That’s a definite difference. While final can sometimes be a bit controversial, it’s used in a fair number of libraries to indicate that this class or member is not designed to be extensible.

Yeah, that sounds reasonable…

1 Like

A class A is one of its own supertypes. A proper supertype of A is a supertype of A that is not A itself.

The terminology comes from set theory, because types are essentially sets of possible instances.

1 Like

Yes! that makes perfect sense.

Yes, CLOS has no methods. There is a pattern that fills the same purpose as constructors. But there are only functions. It might help to understand ancient history.

Classes were just lists with a fixed structure. All instance of the class the same set of lists, with different CDR.* Classes normally used the same (eq) lists unless the element was different than the class definition. Instances just had an eq link to the class object at the top of the list structure. A constant reference for every instance. Classes had a List of types/classes/mix-ins at the same place. There is very little difference between a class and an instance. When the instance had many of default instance it used the class’s list structure.

Anonymous super/mix-in classes are possible. As everything is a list, the links have a guaranteed linearity. The class itself is a type. -T are no different than +T.

Structural inheritance is having the same list structure(s).

Functions are a list. There really isn’t a difference between a data list and a function list. The functions are just executed rather than data.

By the time of The Little Lisper with objects added, functions could be affected by the make-specializable function. That changed the definition of the function. The function would execute the type (named or anonymous) specific function based on it’s parameter’s type.

Modern CLOS systems have optimized the entire implementation, not everything is as simple as the Little Lisper interpreter. Think of it as Hotspot for classes & instances along with functions.

Classes can have +T or -T or more complex composition. The class definition is static, structural. Functions are pattern matched/specialized based on a class. The class itself may be anonymous. In all cases, the classes are pre-linearized. The function is specialized. The class itself doesn’t own the function. The function controls its own specialization. Call-next is owned by the function.

The closest thing to methods are just functions that expect the class to be part of the parameter. In this, one doesn’t know details about the implementation of the classes and other functions. One know and can depend on the semantics of the other functions. In Java, the need is caused by needing the implementation, but not knowing the implementation.

In a proper type system, +T and -T calculus can not be guaranteed to halt. In CLOS, the operations are on physical data structures. In that, -T isn’t really supported. [or wasn’t, things might have changed]. The result of type calculus is really applying a function on the type representation. It’s possible to construct a function that doesn’t halt, in practice it’s not possible to create a function given the combination rules.

It is more possible that the function create a pattern match that doesn’t halt. In general, the developer’s mind will implode before that kind of pattern matching problem will blow up the compiler or run-time system.

To make it short

  1. There are only functions
  2. Constructors are functions that manipulate the creation of the data structure of an instance.
  3. Structural inheritance is owned by the class.
  4. Function pattern matching is owned by the function itself. Both the function and the class matched may be anonymous.
  5. -T will destroy the developer’s mind before it will realistically destroy the compiler. The downside is -T isn’t really supported.

A) Changing the schematics of scala to hand the same general case would be very difficult and require a lot of hard analysis to define.
B) The primary need in scala is from needing the implementation’s behavior, but not having it.
C) Limiting it to the direct extends would require less thinking and much less implementation (I think).

  • All lists were made of Tupple2. CDR being the ._2. CAR is ._1. Head/tail might make more sense to people.