Fascinating! Is it really exploitation? It seems to me that the node should optimize to the minimal inputs (weights) in the happy path. I.e. it's optimal for all non-operand inputs to be weighted to zero.

I'm curious how reproducible this is. I'm also curious how networks form abstractions and if we can verify those abstractions in isolation (or have other networks verify them, like this) then the building blocks become far less opaque.