I’m unable to retrieve a value from a `MutableHash...

# announcements

[Benjamin Dobell]

I’m unable to retrieve a value from a MutableHashMap, using the keys retrieved from the map itself. Does anyone know how that could be possible?

Attached is a screenshot of the seemingly contradictory debugger outputs.

I should note, this isn’t just an issue with the debugger. In practice I’m inserting a key into a map, then almost immediately retrieving (further up the same call stack) using the exact same key and getting back null for the value corresponding to that key. There’s no multi-threaded access to this map etc.

hashCode and equals are stable for said key.

This is using Jetbrain’s runtime.

[Michael de Kaste]

can you show us more of the insertion and extraction code?

[Benjamin Dobell]

Unfortunately, not easily.

Values are inserted in a recursive function, and I’ve already worked around the issue in the codebase by replacing the keys (which were an interface ITy) with strings. As the ITy effectively had a unique toString() method. It’s just slower than it needs to be.

The insertion was essentially just:

tyProblems[someITy] = mutableListOf<Problem>()

and retrieval

tyProblems[someITy]

Where someITy are not simply equivalent (equal) values, but rather the exact same value (reference).

I must admit, just based on the screenshot above, ignoring any possible code. I’m unclear how retrieving a value from a map using its own key could ever fail (assuming hashCode and equals are stable).

To add insult to injury, cloning the map (last inspection) then retrieving with that exact same key works just fine, 100% of the time 😐

[Benjamin Dobell]

I did try debug getNode i.e. the LinkedHashMap implementation, however it would seem the source-code Jetbrains are distributing doesn’t match the binaries they’re distributing 😐

[Michael de Kaste]

very interesting, as an outsider I can only make a remark about hashcode/equality stability but you assured me that it's correct (also with strings as keys) so I'm afraid my knowledge stops here too.

[Michael de Kaste]

Maybe something with lazyness and multithreading?

[Benjamin Dobell]

No multithreading involved unfortunately.

I’m admittedly not sure how laziness would come into play here. However, I’m also not at all sure what’s going on, so definitely wouldn’t rule it out! 😛

[Michael de Kaste]

what if, instead of [] you just use .get()?

[David Eriksson]

What does ITy look like?

[Benjamin Dobell]

@David Eriksson https://github.com/Benjamin-Dobell/IntelliJ-EmmyLua/blob/type-safety/src/main/java/com/tang/intellij/lua/ty/Ty.kt#L69-L123

Here’s a commit where I revert from my functional (string key) work-around to using ITy keys again:
https://github.com/Benjamin-Dobell/IntelliJ-EmmyLua/commit/67f6ac58e57ac335c0c55ef00d66f4f6c184dc7e

Realistically, I don’t expect anyone to go pouring through my code. I know non-stable hashCode and/or equals could lead to something like the above; basically was hoping someone may generally know of something else that could lead to a similar outcome.

I was thinking perhaps something to do with Kotlin type safety/casting rejecting my ArrayList value, however that theory was somewhat shot down by my last inspection where I clone the collection using the exact same type definition 🤷‍♂️

[Benjamin Dobell]

Well, and I managed to partially debug (as mentioned, for some reason Jetbrains aren’t distributing the correct source) the LinkedHashMap internals, and it’s seemingly getting a null back from getNode i.e. before Kotlin is even involved.

[Benjamin Dobell]

Oh, the ITy definition isn’t all that useful. The offending concrete type is TyUnion (https://github.com/Benjamin-Dobell/IntelliJ-EmmyLua/blob/type-safety/src/main/java/com/tang/intellij/lua/ty/TyUnion.kt#L102-L110). Also, no I didn’t implement that equals method. It’s stable, it’s just subject to “aliasing” (false positives) - I wouldn’t have written it like that 😉

[David Eriksson]

How about adding a default implementation of hashCode and equals to your abstract Ty class, that throws a NotImplementedError or similar?

[Benjamin Dobell]

Not going to pretend I knew this, however your receive an An interface may not implement a method of 'Any' compilation error if you try implement hashCode or equals in an interface.

There is however a concrete superclass called Ty that in practice all classes that implement ITy extend, so I added the implementations there.

Several types do indeed lead to an exception being raised. They’re (correctly) reliant on the default hashCode and equals implementations. I replaced it with a call to super’s implementation so I could more easily debug. The offending code does not call into this method, as expected `TyUnion`’s implementations are being hit instead, both when putting and attempting to get from the linked hash map.

[David Eriksson]

I don't think you can rely on default implementation of hashCode anywhere. If I remember correctly it will be unique for the instance. And the Union is using the hashCode of its children.

[Benjamin Dobell]

Well, the default implementation is valid for references, not functionally equivalent instances. In this case, that’s what is desired (there is only one instance of said types, they’re constants). However, irrespective of whether it’s a good idea per se, hashCode is demonstrably stable (inspections show this also). The hashCode returned when setting and getting is identical (verified not just based on the inspections, but when debugging), yet no result is returned from the hash map. Honestly, it looks like a runtime bug to me. However, I can’t confirm that as I’ve no idea what code my computer is running seems as Jetbrains aren’t distributing the correct source for their forked OpenJDK JRE 😐

[Benjamin Dobell]

Hmm, you know what. I’m wondering whether these `hashCode`s are as stable as they ought to be. I have a sneaking suspicion that my debugging is altering the hashCode of one of these more complex types. It certainly seems a lot more feasible than a runtime bug in a class that is no doubt used a lot.

By that I mean, I think it’s conceivable an object is inserted into the map whilst its hashCode is one value, then its hashCode is changing there-after. However, I think when debugging my inspections are triggering that change in hashCode, so it seems stable to me. I did debug hashCode used during the put process i.e. inside `LinkedHashMap`’s internals, but I think I must have looked at it just after it’s own internal call to hashCode .

Nothing verified, but that’s my running theory. As far as I’m concerned it’d make significantly more sense if these classes were immutable, but I didn’t write them, only this particular piece of code utilising them.

[Benjamin Dobell]

Actually, that’s almost certainly the issue. Because it explains my final inspection i.e. Why cloning the map and then using the key works just fine, because the 2nd map is created using the latest hashCode.

I’m going to see what I can do about making these types immutable. Not my project, so might have some resistance, will see.

Thanks for your help everyone 🙂