He was trying to offer three explanations for the disparity in women in high end science and engineering positions.
It didn't go well and the RW jumped all over it. There were other reasons that he left Harvard but this was a major factor (I may just be echoing what the MSM drilled into my brain here so forgive me).
Sexism allegations
In January 2005, at a Conference on Diversifying the Science & Engineering Workforce sponsored by the National Bureau of Economic Research, Summers described three hypotheses for explaining the higher number of males in high-end science and engineering positions. Summers claimed that he was adopting an "entirely positive, rather than normative approach" and that his remarks were intended to be an "attempt at provocation."<14> It was the second hypothesis, termed "different availability of aptitude at the high end" by Summers, that drew allegations of sexism.
According to Summers, the
preponderance of males in high-end science and engineering positions could be explained by the following: first, that more men than women were willing to make the commitment in terms of time and flexibility demanded by high-powered jobs; second, and controversially,
that there were differences in the intrinsic abilities of men and women at the extremes (3.5 to 4 standard deviations above the mean), as shown by higher variance in aptitude, abilities, or preferences relevant to science and engineering among men (see Gender differences); and third, that the discrepancy was due to discrimination or socialization.<14> He also stated his view that the order given reflected the relative importance of each of the three hypotheses.<14>
An attendee made Summers' remarks public, and an intense response followed in the national news media and on Harvard's campus.<15> Nancy Hopkins, a biologist at MIT, walked out during the talk in disgust.<15> It was claimed that this incident cost him the job of Treasury Secretary in Obama's Administration.<16>
http://en.wikipedia.org/wiki/Lawrence_Summers#Sexism_allegations