Except the exact opposite thing is happening in the regular housing market. Small houses (e.g., townhomes) were intended to maximize land and reduce cost, except they’re now not even affordable.
It's the same problem -- you need more. Zoning regulations continue to prevent enough new construction, and the conversion of large lots/units into smaller ones.
I honestly don't know what you're proposing instead. But for some reason you're pessimistic about what all traditional economists agree on what is the solution -- remove more zoning regulations that constrict what can be built.
It's literally just supply and demand. Prices come down when you increase supply to meet demand. Just because increasing supply a little bit doesn't work when demand is increasing even faster, doesn't mean the basic principle is falsified. It's just that you need to increase supply much, much more.
Demand for floor space in these places is very large, no doubt, but it is not infinite.
If you don’t believe we can or should allow private landowners to build as much floor space as they want for everyone who wants to live in these places, then somehow you’re going to have to have a mechanism to control who is allowed to live there. What will it be and who will get to decide who is worthy?
This will happen with these too.