Ah, 103F ...barely warming up! Wait till you get to 43-44C (110-112F).
Seriously though, just don't leave camera or lens in direct sunlight for any great length of time where they can get a real baking. It you slip the camera out of its bag for a few shots and then pop it back in while you walk to the next point of interest, you'll be fine.
Also try to avoid leaving gear in a locked car in full sunshine for long periods. The interior of a car can get to 60C or more. A camera bag out of direct sunshine wrapped in a towel will stay reasonably cool for a hour or more, but eventually will start moving towards the interior temperature of the car.
And as Adam suggests, the worse part is the effect on the actual photos. Middle of the day when 100F+ means intense sun, lots of glare and flat photos. Beginning (up to say 10am) and end of the day (5pm+) are best when the light is softer. Birds are also likely to be quiet and out of sight in the hottest part of the day - they're not stupid when it comes to self preservation. First couple of hours after dawn are probably best for birds, or towards dusk when they come in numbers to watercourses or waterholes to drink. If you must have the 150-500 out in the midday sun for prolonged periods, one of those wrap around camo covers would keep direct sun off the lens and keep its temperature reasonable - you would want one of those whitish ones intended for snow rather than a dark colour.
A polariser filter can really help if you have to shoot in the middle part of the day. Polarised lens sunnies over your own eyes also greatly improve your own viewing when you aren't using a camera - much better colour saturation and contrast without any unnatural tinting that other types of sunglasses create.
More importantly, protect yourself with a wide brimmed hat and a quality sunscreen lotion - bad sunburn really does hurt (and not a pretty look to boot). And carry water with you - dehydration sets in reasonably quickly, even mild dehydration equals a headache and fatigue which I'm sure you can do without. Guzzling can after can of soft drink in those conditions isn't necessarily the best.
It was just starting to cool down from 45C (113F) when I took these photos at the end of the day a couple of years ago:
A warm day in Adelaide | Photo Morsels