I guess the question is what is important. Lenses can be smallest when they are close in focal length to the registration distance. For Fuji, that is 17.7mm, while for Pentax that is 45.46mm. So, wide angle lenses will tend to be smaller with Fuji (tough to compare totally due to the fact that Fuji makes their lenses a lot faster). On the telephoto side, Pentax lenses will tend to be smaller, although Fuji hasn't released many long primes. Even their macro lens is on the short side at just 60mm.
On the other hand, Fuji's 50-140mm f2.8 is 995 grams and 1600 dollars on Amazon, while the Pentax DA *50-135 is 685 grams and costs 845 on Amazon right now.
Pentax has tended to make lenses a little slower over time in order to limit their size. This may or may not be a problem. I, for instance, have never had a situation that I can recall where I wanted a 16mm f1.4 lens. I mainly shoot landscapes with such a wide angle and while such a lens will probably be sharper, it is probably more prone to flare than the DA 15 f4 is.
Anyway, I would just say that there are two different approaches. I would highly recommend, for example, the FA limiteds for their sharpness, small size and awesome build, but they aren't as fast as Fuji's primes (at least until we get the full frame camera to mount them on).
(prerequisite DA 15 limited shot)