The only way to preclude jury nullification is to infringe on the rights of the jury as the sole determiners of fact, as you put it.
Without adding another non-jury body that can overrule the jury on what actually happened, how can you ever get rid of the possibility of jury nullification?
In the US, judges can overturn jury verdicts. That will be appealed to a higher court (which means they need a REALLY good reason to do it).
The most common place for this to happen is in civil court. A jury can come back with "11ty billion dollars" and the judge can decide "Ok, they got a little overzealous with that, $100".
The judge can effectively overturn a "guilty" verdict but cannot overturn a "not guilty" because of double-jeopardy (though I suppose a judge could force/rule a mistrial).
juries don't determine damages, only that damages occurred as evidenced. Similarly, jurors don't determine guilt or innocence, but determines whether the presented evidence from both parties are convincing. Judges determine innocence or guilt, and also hands down sentences.
The only way to preclude jury nullification is to infringe on the rights of the jury as the sole determiners of fact, as you put it.
Without adding another non-jury body that can overrule the jury on what actually happened, how can you ever get rid of the possibility of jury nullification?