I'd have no qualms about photographing the barn from the road, I'm pretty sure you're on solid legal ground there. As for general politeness, I'm sure that if someone came along and asked what the heck you were doing and you explained that you were an amateur and would they like a print, they'd be happy.
The scenario that has me baffled - although I'll admit I haven't really looked into this, so please excuse me if the answer is right there in another thread or otherwise accessible - is photographing private property that has some sort of restrictions, from a public space. For example, there's a candy store in Halifax, NS with "The Rules" painted on the wall right beside the entrance, including the stipulation "NO PHOTOGRAPHS". Okay, fine - anyone famous who comes to Halifax goes to the candy store (it's that kind of place...) so I can see that they wouldn't want people snapping away. Besides, it's really colourful and wacky, so I can imagine it'd be a fun place to shoot, and they don't want hordes of photographers in there not buying stuff.
But - what if I were on the sidewalk, which is city property, shooting towards the store? I could be doing anything from taking a picture of the general architecture, which includes other stores, to attempting to shoot into the store and photograph Kiefer Sutherland buying pixie stix. At what point, if any, am I breaking their rules?
This is probably why I stick to photographing bugs and plants - they rarely sue!
Julie