There isn't a perfect camera bag; but there are perfect camera bags for specific shooting needs / situations. Most of us amateurs may have 3 camera bags. 1) Gear intense all-around bag loaded with lenses and creative lighting options, 2) the light, travel & multi purpose bag, 3) the adventure, sports and wildlife backpack that holds longer glass. Based on your post I believe #2 is your focus.
A couple of years ago I purchased a Timbuk2 snoop messenger. Great bag, protective and functional. Just over a year ago my wife gave me a Domke F-4AF bag. It's now my primary camera bag. I even cannibalized the Timbuk2 for extra dividers for the Dome bag! In one word the Domke bags are "superb" - they're not quite as luxurious as the Billingham Hadley bags but the Domke line is reasonably priced and available in a weatherproof wax wear finish. Most of all the Domke bags look very casual and blend in like an old pair of jeans.
The Dome F-5XC or F-5XB (the XB is smaller) are ideal for a small kit Camera with lens + a lens or two. The Dome F-832 medium photo courier bag in a weatherproof wax wear coating is also worth looking at.
If you wish to keep the budget down a little more; Vanguard manufactures excellent equipment and has a "Havana" line that would meet your needs as well.
http://www.vanguardworld.ca/photo_video_ca/products/camera-bags.html?series=279