@eyalroth, Yes, I realize that the people reading this forum are not necessarily aware of particular research of people chatting here. However, I don’t understand why such would be necessary. That’s why I tried to reduce it to a pretty self contained use case.
Nevertheless, if anyone cares to look at the actual code, here is an excerpt.
The problem is that the use of the
WeakHashMap (line 68) requires me to have a key which pretty much contains the same information as the value. I.e., the case class object has a
negative and the hash table key (lines 79 and 84) is a triple of
(label, positive,negative). That pretty much doubles the memory usage right there. Additionally, it seems from some profiling that I’ve done, the hash implement has yet another copy of the objects internally. Or at least lots of overhead per entry.
If I had a way to just store the values once, without having to have a redundant key, that would decrease the memory. So I thought about using a weak-key-hash which just mapped the objects themselves to some ignore-me value, like
true. But then I have the problem of trying to retrieve a key, which seems to be omitted from the scala API.