I had the same issue some years back, I had to fix it with a freshly built new PC...
What are the specs on the PC (MOBO, ram, processor etc...)
Few things to try though no guarantees,
1. unseat and re-seat the CPU (you will probably need thermal grease to go between the fan and cpu as you have to remove the fan to do this)
2. Unseat and re-seat your ram
3. Verify that the new power supply is good by unplugging it from the mobo and peripherals and jumping the power wire as shown here
(though unlikely you may have received a DOA new power supply)
4. remove all PCI cards or agp cards and try and start without them. If it does start, add them back one at a time until the issue occurs a again.