@ledohung55
It strikes me that you're managing portraiture just fine with the 40mm. Maybe you have to get a bit closer in than you might otherwise want to, but the 40 is not a big, intimidating piece of glass.
I have just acquired the DA* 55mm. I've not done any serious shooting with it, just a couple of shots in my living room of things close up and far away. It looks pretty decent at F1.4 as far as I can tell. However, I do have an issue with it: it's kinda big for a prime. It's not too bad without the hood, but stick that enormous, bulbous, unnecessarily-large lens hood on the front and it suddenly becomes a rather aggressive looking thing that shouts "I AM TAKING A PHOTOGRAPH". I'm not altogether certain that I like it.
I will also point out that the 55mm is the slowest focusing lens that I own. Doesn't matter overmuch to me since I'm not planning on using it for continuous action shots. It may matter to you.
You have two highly prized lenses: the 21mm and the 40mm. Together they can handle most situations. I wouldn't trade them for a single lens, even if it does go up to F1.8 or F1.4. Have you tried using the 50-200 for initimate, head-and-shoulders style portraiture? I don't have that particular lens but I do have the 55-300 and it is excellent: