Hi Stormtech, I have more or less completely moved from micro four thirds to a K5. I was chasing the immediacy of an Optical VF and the serious performance of K5 sensor in low light.
I used an E-P1, which doesn't have an Electronic VF, and I didn't enjoy the shooting style. I haven't used the EVF's alot, but they are ok for most things, but action is a bit slow.
Ultimately with a good exposure the image difference is not very different. There are some lovely lenses made by each manufacturer, and Olympus has SR like Pentax, where Panasonic uses in-lense SR. I was using the Panasonic 14-140mm as my telephoto and it's image quality was actually pretty good, where the cheap Olympus telephoto lenses had great IQ.
I recommend
Four Thirds | Micro Four Thirds | Products(Lenses) , it's agreat tool to get a feel for the system, the rest of the site explains things very well too.
I think that the weight and volume of the systems compared is proportionate to the sensor size. So you can expect a saving there. As for image quality, there will be a slight drop in micro four thirds compared to K20D, but it may be pretty similar. Ultimately there is no 300/2.8 equivalent, and a lense with that spec will always be big. Where you could use any manual focus 200/2.8 lense adapted on micro four thirds, and save some weight and size.
Hope this helps.
T