Why do CDs and most good quality stereo MP3s have a sample rate of 44,100 Hz?

The human ear has a theoretical frequency range of 20,000 Hz. You need at least two sample points to represent the frequency (for any given sine wave), i.e. if you digitally sample a continuous signal at a sampling rate of n, you cannot capture frequencies above n / 2. This is called the Nyquist frequency (written fs / 2).

A sine wave represented by 2 sample points per cycle (Hz)

So, to represent the full theoretical maximum range of the human ear (20,000 Hz) you need at least twice the samples, 40,000 Hz (40,000 measurements per second). For safety's sake a little headroom was deemed sensible (in fact 2050 Hz of headroom).

Thus you have: (Theoretical Range + Headroom) x 2, or:

2 (20,000 Hz + 2,050 Hz) = 44,100 Hz.