I had an FA31 and the DA40 Limiteds. Both great lenses but I ended up finding that I wasn't using either of them enough, and having decided that I need to consolidate my lens collection I bit the bullet and with some regret sold them both, to be replaced with the DA 20-40 Limited.
The 20-40 is much more convenient, and covers the wider end of the range meaning I don't need to get the 21. I find I'm using it more than either the 31 or 40 that I sold, and the image quality is plenty good enough for me. Like all the limiteds it's a joy to use, and certainly better than the 18-135, but unsurprisingly not as nice as the 31.
If I look at images I've taken with the 31 they are better, but the 20-40 is still just fine. Having said that, I do miss the 31 and could be tempted by another one if I can justify the cost one day! In the meantime, I have no problems with the 20-40.