Functional Tensorflow


#1

Hello Scalas,

I have been away for a while doing Deep Learning with Tensorflow in Python.

Python makes me miss type-safe functional programming, so I want to create a Scala wrapper for Tensorflow

At it’s core, Tensorflow is functional: Graphs transform Tensors of one shape into another shape. For example, one might represent an image as a 256x256x3 tensor (pixels x pixels x rgb). An image classifier graph might transform that into a 1000x1 tensor (scores for 1000 classes). One can make more complicated graphs by combining simple graphs.

I want to represent graphs as functions with a signature like f(T1): T2, where T1 and T2 encode the shape of the Tensor. One could then do Deep Learning by combining these functions.

Can anyone suggest a nice way to represent shape (e.g. 256x256x3) as a Type? Such a type needs to be parameterized with an seq of integers because each application has its own unique forms of data and network architecture.

I am looking for something like

def foo( Tensor( [256,256,3] )) : Tensor( [1000,1] )

Thanks in advance
P


#2

I think your point is you want to check whether the shape is mismatched during compiling time instead of runtime, right?

I think Scala is able to do it but I’m also new to it. Maybe Trait can do it? Not an expert, sorry. :frowning:


#3

Strictly speaking, scala is not a dependently typed language, so type parameters must be types, not objects. But there are ways to capture quite a lot. Details depend on the specific goals but just as an indication:

  • You cannot have Box[3, 12, 15] as a type, as 3, 12 and 15 are objects.
  • The most precise way is to use singleton types such as 3.type - this is a type with just one object. For this, it may be best to look at the documentation of the shapeless project.
  • In practice, one can just use parameters which are later ignored, e.g.
class Box[L <: Int, B <: Int, H <: Int](length: L, breadth: B, height: H){
   def stack[H2 <: Int](that: Box[L, B, H2]) 

There is some level of type checking here even though eventually L, B and H will be inferred to be Int

I am myself interested in using tensor flow from scala and would be happy to contribute… It may be best to wrap the Java API.

regards,
Siddhartha


#4

Thank you Czeng and Siddhartha,

Scala is my favorite language, and I have been using it for a while, but I couldn’t figure out how to do this

I am very interested in developing a better way to program Tensorflow and Deep Learning. There are already Java wrappers such as JavaCPP, but they don’t seem to offer any advantage over the Python. Code is full of magic, pretty impossible to read, and debugging is a nightmare!

What I think is needed is a new programming language. Scala, which is both functional and flexible, would be a great base for this DSL


#5

I wonder if Refined might be useful for something like this; I haven’t used it myself as yet, but hope to do so.


#6

Just to clarify, what I was suggesting is targeting the Java API. This is usable from Java but almost completely untyped. It seems that one can make a scala facade for this, and successively refine the types.

regards,
Siddhartha


#7

A bit late for the answer. But the Nexus project may be what you’re looking for.
Nexus. There’s also a paper on that page that is worth reading.