Well the byte stream includes everything you throw in (TCP/UDP/L2), so while the tunnel will not suffer from any signaling issues in the payload flow, the opposite is not true - the internal traffic is affected by any TCP problems on the SSH tunnel so a short blip on the tunnel can cause a cascade of blips depending on whats in there.