Physical Quantities Library (dimensions, units)

Bersier · July 11, 2022, 3:39pm

Here is a small Scala 3 project I just made that uses opaque types to provide type-safe, zero-overhead computation with physical quantities

Russ · July 11, 2022, 5:46pm

Congratulations on a nice piece of work. If your implementation is really “zero overhead”, then it is very significant. But is it? A few years ago, someone posted an implementation similar to yours, but it had a high run-time overhead when I tested it. I’m not saying yours does, but have you tested actual performance compared to using standard so-called “Doubles” with no explicit units?

FYI, I developed an method of representing physical scalars with near zero run-time overhead, but the unit compatibility checks are done at run time. Actually, it can be compiled in two different modes, one of which has near zero runtime overhead, and the other that checks unit compatibility. You can find it here:

Here is the overview:

An open-source scalar package and associated software tools have been developed in the Scala programming language, including plotting tools based on the free GRACE plotting package. The scalar package represents physical scalars and can help to prevent errors involving physical units in engineering and scientific computation. The scalar package includes a complete implementation of the standard SI metric system of units and many common non-metric units. The design also allows users to easily define a specialized or reduced set of physical units for any particular application or domain. The scalar package can be used in two different modes: one mode provides unit compatibility checking but is slower, and the other mode bypasses the compatibility checks but is much faster and still prevents the most common type of unit error. Switching between the two modes requires no changes in the user’s code, making it convenient and usable with no significant performance penalty for even the most computationally intensive applications.

jducoeur · July 11, 2022, 8:24pm

Sure, but was it Scala 3? Things have changed a lot in the past few years, and there is a world of difference between opaque and, say, classic value types. @Bersier’s library appears to be entirely based on opaque types and pushing the type system fairly hard – at least at a quick glance, it looks like it should largely compile down to raw Doubles…

Bersier · July 11, 2022, 10:59pm

I believe it really is zero runtime overhead; the compiler does a significant amount of work when type checking (addition and multiplication, for example).

Russ · July 12, 2022, 6:44am

Well, If you can do unit compatibility checks with no runtime overhead, you may have found the Holy Grail of scientific and engineering computing.

As I explained in my user guide, there are two basic kinds of unit errors: scaling errors and unit compatibility errors. A scaling error is using the wrong scaling for the right physical quantity, such as using meters when you meant kilometers or seconds when you meant milliseconds. A unit compatibility error, on the other hand, is trying to add, subtract, or compare two quantities of different physical types, such as adding a length to a velocity.

I believe that scaling errors are more common, and my scheme avoids them even when used in the fast mode with unit compatibility checks disabled. But it sure would be nice to be able to get the unit compatibility checks without the huge runtime overhead, which you apparently have done.

Your code is only a couple hundred lines, but I must admit I don’t understand much of it. It would be nice to have a version with extensive commenting to explain the code in detail. In the meantime, I will think about how its usage and convenience actually compares with my scalar package.

One of the most common unit scaling errors is passing an angle in degrees to a trig function that takes radians. In my field of aviation and air traffic control, this error is almost guaranteed to occur somewhere in any large code base. It bothered me so much that I came up with a scheme to force the user to explicitly specify radians or degrees.

@home > calculator
[info] welcome to sbt 1.5.0 (Private Build Java 1.8.0_292)
[info] loading project definition from /home/rpaielli/scalar/project
[info] loading settings for project scalar from build.sbt …
[info] set current project to physical-scalar (in build file:/home/rpaielli/scalar/)

scala> sin(30)
java.lang.RuntimeException: WARNING: trig arg should be scalar (use rad or deg)

at scalar_.Scalar$package$.sin(Scalar.scala:231)
… 38 elided

scala> sin(30*deg)
val res1: types_.Real = 0.49999999999999994

I am wondering if your scheme can do that.

Bersier · July 12, 2022, 10:37am

My code deals with both scaling and unit errors as well. One limitation is that each dimension is hard-coded. So it would require a library code change to add a new dimension for angles, as well as degree and radiant units for it. However, that code change is straightforward boilerplate.

Here are a few things I used from Scala 3:

opaque types
match types
type lambdas
implicit conversions
higher-kinded type polymorphism

jducoeur · July 12, 2022, 12:26pm

The library is very clever – really, a delightful illustration of the power of Scala 3 – and I think you’ve got the beginnings of something quite useful here.

Just one request: types is an awfully generic package name, and I worry about the likelihood of name conflicts. I’d recommend changing the name to something more precise (eg quantities), and consider giving it a standard package prefix (at least bersier.) – that’s likely to make it easier to use for serious work.

devlaam · July 12, 2022, 1:32pm

As a physicist i am both exited and surprised. As soon as the Opague Types came out i realised that such a (zero overhead) library now became a possibility, and wanted to have such a thing. But, as usual, other activities came in the way. Looking at your code, i think it is for the better, for i would have not done it in such a clean way probably.

I am also a bit surprised, why you choose for:

type ElectricCharge  = Dim[_0, _0, _0, _0, _1, _0]

and not for the base SI Unit electric current. I understand charge feels more fundamental, but it is not what we all decided it to be. (A bit like with the kilogram, which has the ‘kilo’ in its name, making it non coherent).

And, of course, you are still missing an entry for luminous intensity, but that can be added lateron.

dwalend · July 19, 2022, 3:36am

Thanks, Bersier. Your example fantastic - easy to follow, and I learned a lot.

Adding degrees was a breeze. (As was changing it to use floats - I’ve only got a limited 32-bit processor and less than 10 bits of accuracy on my best day. )

I’d like to override the toString() to provide some units - like “12.8 meters” . When I went searching for a way to do it with opaque types I found consistent answers saying it wouldn’t work.

My attempts to add a toString() to Dimensions’ extension methods were a flop. I was able to add an extension method that makes a string, but not with the right type signature to override def toString():String . I poked around with value types but didn’t find any way to connect a value type to an opaque type.

Is there a way to tell the compiler “when you box an opaque type to use a method please unbox to this other type - or use this other method - for toString()” ?

Jasper-M · July 19, 2022, 7:51am

I don’t think it’s possible. The problem is that Any has a toString method. You would need a parametric top type that doesn’t have any methods. Then you could have this:

opaque type Foo <: Top = Float

(a: Foo).toString() // error: value toString is not a member of Foo

Which allows you to add your own toString extension method.

They considered adding such a top type, but eventually went with Matchable which solves a related problem, but not yours.

The best you can do is adding your own show method, and/or using something like cats.Show.

dwalend · July 19, 2022, 2:03pm

Thanks Jasper-M.

That’s why I’d gone with value classes (instead of opaque types) in my own code.

Would it be possible to follow Bersier’s pattern with value classes? (Please tell me if it’s hopeless before I tilt my lance at that windmill.) That seems like it could give the best of all worlds - strong types, zero-overhead optimization where possible, and control over boxing.

Bersier · July 19, 2022, 2:41pm

I’m thinking that transparent inline macros might perhaps be able to print the dimension of a quantity.

dwalend · July 19, 2022, 5:01pm

When I compile Bersier/physical without a clean - it runs until I give up and kill the process. (With a clean it’s successful in <10 seconds.)

Is that a bug I should report? Where?

Thanks,

David

12:39 $ sbt run
[info] welcome to sbt 1.6.2 (AdoptOpenJDK Java 11.0.9.1)
[info] loading global plugins from /Users/dwalend/.sbt/1.0/plugins
[info] loading project definition from /Users/dwalend/projects/scala-ev3/physical/project
[info] loading settings for project root from build.sbt ...
[info] set current project to physical (in build file:/Users/dwalend/projects/scala-ev3/physical/)
[info] compiling 1 Scala source to /Users/dwalend/projects/scala-ev3/physical/target/scala-3.1.3/classes ...
^C
[warn] Canceling execution...
^C
[warn] Canceling execution...
^C
[warn] Canceling execution...
✘-TERM ~/projects/scala-ev3/physical [master|✚ 1…143] 
12:59 $ oot / Compile / compileIncremental 1030s

dwalend · July 19, 2022, 5:14pm

I threw down a first attempt over lunch. No luck. I think the challenge is getting the type signature to really be just

transparent inline override def toString():String

. It’s not clear to me how to get there from Dimensions’ extension . I think the generics are getting underfoot. (Also -explain is pretty great.)

[error] -- [E038] Declaration Error: /Users/dwalend/projects/scala-ev3/physical/src/main/scala/types/Dimensions.scala:90:36 
[error] 90 |    transparent inline override def toString():String = s"$x from extension"
[error]    |                                    ^
[error]    |method toString has a different signature than the overridden declaration
[error]    |----------------------------------------------------------------------------
[error]    | Explanation (enabled by `-explain`)
[error]    |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[error]    | There must be a non-final field or method with the name toString and the
[error]    | same parameter list in a super class of object Dimensions to override it.
[error]    |
[error]    |   override inline transparent def toString[L <: types.IntType.IntT, T <: 
[error]    |   types.IntType.IntT
[error]    | , P <: types.IntType.IntT, M <: types.IntType.IntT, Q <: types.IntType.IntT, C
[error]    |    <: 
[error]    | types.IntType.IntT](x: types.Dimensions.Dim[L, T, P, M, Q, C])(): String
[error]    |
[error]    | The super classes of object Dimensions contain the following members
[error]    | named toString:
[error]    |   def toString(): String
[error]     ----------------------------------------------------------------------------
[error] Explanation
[error] ===========
[error] There must be a non-final field or method with the name toString and the
[error] same parameter list in a super class of object Dimensions to override it.
[error] 
[error]   override inline transparent def toString[L <: types.IntType.IntT, T <: 
[error]   types.IntType.IntT
[error] , P <: types.IntType.IntT, M <: types.IntType.IntT, Q <: types.IntType.IntT, C
[error]    <: 
[error] types.IntType.IntT](x: types.Dimensions.Dim[L, T, P, M, Q, C])(): String
[error] 
[error] The super classes of object Dimensions contain the following members
[error] named toString:
[error]   def toString(): String

Russ · July 19, 2022, 5:58pm

I have the same problem. I got it to compile and run, but then if I do so much as add a blank line at the end of a source file and recompile, the compiler goes into an infinite loop, and I don’t know how to stop it other than killing the xterm tab. Needless to say, that makes it very difficult to experiment with.

dwalend · July 20, 2022, 1:20am

I’ve tried value classes with some success, but have gotten past the boundary of helpful messages from the compiler while overriding toString() . I started with Bersier’s example, replacing the opaque types with a value class. That’s mostly worked well, but I’m stuck.

I want to preserve the type in that match/case (via inline), and eventually call a method on a companion of the units if it exists to get a nice postfix on the string. Am I doing something wrong with inline ?

object Dimensions:

  //noinspection ScalaUnusedSymbol
  class Dim[
    L <: IntT,
    T <: IntT,
    P <: IntT,
    M <: IntT,
    Q <: IntT,
    C <: IntT,
  ](val d:Double) extends AnyVal:
//    override def toString():String = s"$d override" //this works!
//.   inline override def toString():String = s"$d override" //this does not work - same compiler crash

    transparent inline override def toString():String = inline this match {
      case m:Mass => s"${m.d} kilograms"
      case o => s"${o.d} not kilograms"
    } // this causes compiler crash

While compiling I get

[info] exception occurred while compiling /Users/dwalend/projects/physical/src/main/scala/types/AdditionalUnits.scala, /Users/dwalend/projects/physical/src/main/scala/types/Dimensions.scala, /Users/dwalend/projects/physical/src/main/scala/types/Example.scala, /Users/dwalend/projects/physical/src/main/scala/types/IntType.scala, /Users/dwalend/projects/physical/src/main/scala/types/package.scal
java.lang.AssertionError: assertion failed: no extension method found for:

  method toString$retainedBody:(): String with signature Signature(List(),java.lang.String) in object Dim

 Candidates:

 Candidates (signatures normalized):

  while compiling /Users/dwalend/projects/physical/src/main/scala/types/AdditionalUnits.scala, /Users/dwalend/projects/physical/src/main/scala/types/Dimensions.scala, /Users/dwalend/projects/physical/src/main/scala/types/Example.scala, /Users/dwalend/projects/physical/src/main/scala/types/IntType.scala, /Users/dwalend/projects/physical/src/main/scala/types/package.scala

followed by a towering stack trace. The complaining part of the compiler maybe expected a previous stage to create a toString$retainedBody:(): String method.

What’s the right next step?

dwalend · July 20, 2022, 12:10pm

This works:

object Dimensions:

  //noinspection ScalaUnusedSymbol
  class Dim[
    L <: IntT,
    T <: IntT,
    P <: IntT,
    M <: IntT,
    Q <: IntT,
    C <: IntT,
  ](val d:Double) extends AnyVal:
    override def toString():String = s"$d override" //this works!

    transparent inline def pretty:String = inline this match {
      case m:Mass => s"${m.d} kilograms"
      case o => s"${o.d} not kilograms"
    }

Can you just not inline override ? That’s pretty reasonable.

Jasper-M · July 20, 2022, 12:46pm

No it seems to be a bug when you mix inline override with extends AnyVal.

Yes probably. I would start in the dotty issue tracker.

It may be possible to replace the opaque type with a value class (if we ignore possible compiler bugs ). However you will potentially have a lot more boxing and unboxing.

dwalend · August 1, 2022, 9:59pm

It should still only box the doubles when it needs to call a method, just with some opportunity to box to call my own methods (toString() - not interesting for performance), right?

Bugs are

https://github.com/lampepfl/dotty/issues/15724 and
https://github.com/lampepfl/dotty/issues/15725

Bersier · August 9, 2022, 9:10pm

I made several improvements:

conversion of quantities to strings with units
taking roots of quantities (units allowing)
additional dimensions (angles, information, etc)
additional units
abstract charge and potential dimensions that can be mapped onto concrete dimensions
more readable code, with documentation