Tony, I was just thinking the same thing recently. I think you're on the right track.
I see two reasons to keep the rectangular sensor, though...
(1) "Seeing" a rectangular format in the viewfinder aids composition, assuming that a rectangular crop is eventually going to happen. Of course, this view could easily be toggled by a button on the camera (without having to rotate, as you said).
(2) Under the assumption that a rectangular crop is eventually desired (which, I realize, is an assumption), a rectangular sensor saves the manufacturers money. A landscape and portrait rotation covers almost the same area as a square sensor. So, for your inconvenience of having to rotate the camera, they save money. I think that this reason alone will keep our sensor oblong for a long time.