That draft you feel is money going out the (proverbial) window. It's better to stop the air going in/out through walls, doors, and windows, and bring it in in a controlled fashion:
The problem is that many retrofits don't bring in outside air into the mix and just recirculate what's already in the house. This is not current best practice.
Any kind of air exchange will cause heat loss or gain, decreasing insulation. Does not matter if it's through wall or HVAC.
If there is no draft, it means there is no air circulation.
The only difference is if the draft goes through HVAC vents or through unsealed doors and windows, or through walls.
If your local heaters are effective and seal controllable, there should be really no difference in many municipal settings as the heating is waste heat from electricity generation anyway. (Those cooling towers "greens" love to hate.)
Certain baseline air flow is strictly required and it is quite a lot, especially in small volumes.
Now, if you have to burn something to heat, then by all means do install a heatpipe recovery system.
* https://www.youtube.com/watch?v=CIcrXut_EFA
The problem is that many retrofits don't bring in outside air into the mix and just recirculate what's already in the house. This is not current best practice.