In the winter I often go to our local conservatory to take macros in their butterfly wing. Going from a dry 70 degrees at home, to a dry 0 degrees outside, and then to a humid 80 degrees at the conservatory, all within about 10 minutes, would probably cause my camera to explode.
At home I attach the lens I will use, and then keep the camera in the bag until I have been at the conservatory for a few minutes.
And I make sure never to remove the lens. I did that once about a week after getting my K100 and everything fogged up, inside and out on the camera. Luckily there was no harm done!