In the places people want to live, yeah. It would be great if Manhattan were twice as wide, for example. There is the artificial scarcity from the zoning stuff, but then there is also just the geographic boundaries and human preferences for the "cool part of town".

Couldn't we just build higher?

Absolutely you can, and we should! not saying manhattan is maxed out

Manhattan already builds higher. It just so happens that that new real estate is being sold as multi-million dollar condos instead of affordable housing (for certain definitions of "affordable")

That's fine. Affordable housing depresses building and raises rent

You could make the case that some of the luxury condos (e.g. the skinny buildings by central park) are not dense enough. It only has 60 condos and its 84 stories. Its kind of analogous to the McMansion issue in suburbs. I agree w/ your point broadly though.