Unless you are planning to make money off of your photography, try to keep in perspective that it's not a challenge/competition (except with yourself ;-)) and should be free of the accompanying stress from the need to "produce".
One thing I've found is you have to have thick skin; there will be people who like the way you shoot and people who don't. Having said that, it's good to listen to those who don't for the sake of learning why and if there are things you can take away to improve your skills.
Another is to be bold and not be afraid to show your work; it goes along with my first point that that is the only way you will improve. Thanks to the likes of Facebook, Twitter, Flickr, and other social sharing sites, I've gained quite a few fans at work who regularly look forward to my new posts so that they can download them as desktop backgrounds. Alas, maybe not the Louvre, but at least I know my stuff is being viewed and enjoyed. In fact, I just had a woman at work ask if I would consider doing some candid family portraits for her in return for some good Singaporean BBQ.
Lastly, as a hobby, unless you get bitten by serious LBA, it's actually one of the cheaper hobbies you can pick up. The initial investment of several hundred dollars or more might hurt, but over 3-5 years, you've easily provided yourself with hours of entertainment for very little cost (or at least that's what I keep telling myself...
)
Cheers.