Does your lens have a hood?
When in direct sunlight, the lens hood is a necessity to reduce flare.
Since the VF views through the lens, I would suspect that you see some flares in the VF, reducing the 'visibility'.
If your lens(es) does(do) not have a hood, you can make a simple one easily in paper (like a truncated cone around your lens) and user a rubberband to fix it to the lens.
"
Will I get noticeably more VF light with an f/2.8 or faster prime instead of the kit lens?"
With my fast prime lenses (Nokton 58mm, FA31mm, and even Tamron 90mm), I do not have any VF issue. I would not expect any VF issue different from any other lenses.
Hope that the comments will help.