For whatever reason X11 shoving pixbufs over the network seems to have orders of magnitude better performance than actual RDP over a high speed network.