Here's an interesting write-up that summarizes several sources of information about Amazon and high scalability systems. I thought some of the bullet points were pretty insightful...
- Between 100-150 services are accessed to build a page.
- Take it for granted stuff fails, that's reality, embrace it. For example, go more with a fast reboot and fast recover approach.
- Work from the customer backward. Focus on value you want to deliver
for the customer. Start with a press release of what features the user will see and work backwards to check that you are building something valuable.
- Use measurement and objective debate to separate the good from the bad. Referred to as getting rid of the influence of the HiPPO's, the highest paid people in the room. This is done with techniques like A/B testing and Web Analytics. If you have a question about what you should do, code it up, let people use it, and see which alternative gives you the results you want.