PNGs are very widely supported. Any modern browser can display them and any halfway competent image program can read/write them.
The difference is the compression. PNG uses lossless compression (all data is preserved), while JPEG uses lossy compression (throws away data you hopefully won't notice is missing) designed for photographs. (The acronym, in fact, expands to Joint Photographic Experts Group.) The tradeoff is fidelity to the original image vs. file size.
Rule of thumb: if you have an image which is a photo or something similar, and all editing is complete, and you want a small file that is easily shared, JPEG is appropriate. Otherwise, JPEG is the wrong choice.
In particular, JPEG is inappropriate for images which aren't photos or similar. You'll get visible compression artifacts and look stupid.
PNGs are also easily shared, but the PNG compression algorithm doesn't work very well on photos, so the files are big. JPEGs are generally much smaller (5-10x) and the compression artifacts are rarely a problem. For web stuff, don't worry about it; if I were submitting a good photo for a good print, I might send it as a PNG.
PNG is more like TIFF than JPEG. However, TIFF is an extraordinarily complex format with all sorts of weird extensions, so it's not as portable. If two programs "support" TIFF, you never know what they really mean. No one fully supports TIFF.
Here's Wikipedia's
take on the comparison. There's a good illustration.
HTH,
Reid