My basic views are,
If you have an iPhone 3GS upgrade to iOS 6, do it for the screen.
The retina display introduced on the iPhone 4 has been further refined for the iPhone 5 and in side-by-side comparisons with all screens at full brightness, the colors do look more vivid on the iPhone 5.
If you have an iPhone 4, the decision is harder. Consider what new features you really need. If you would get a lot of use out of the Siri voice-assistant, for example, this is a good time to upgrade. If you are a hardcore mobile gamer or serious iPhone photographer, the speed is worth it. But if you find that your two-year-old, well-crafted iPhone is holding up nicely, perhaps stick it out for another year.
IPhone 4S owners should just relax and enjoy their current phones. Wait a year for the next iteration, when your carrier will offer you the upgrade at the subsidized price and all the iPhone 5 owners will suddenly be stuck with old technology.