I have both. They produce very similar results at 20mm, even with the difference in maximum aperture.
But I use them very differently. The FA 20-35 is my landscape lens, companion to my Sigma 8-16. The FA 20 is companion to my FA 43, part of my small prime kit. The difficulty with the FA 20 is its large 67mm thread and finding a decent hood that doesn't look like a giant saucer or doesn't vignette. I eventually found a short black metal hood that looks very cool and is ideal for my K-5/ASP-C.
EDIT: Oh and the reason, beside their superb imaging qualities, that I hang on to them: full frame - on my ME super and the mystical "K-1" digital.
If I really had to choose between the two: I'd keep the FA 20 because I love using primes.
Here they are side by side to give you an idea of size.