DSM-5 Field Trials Discredit the American Psychiatric Association

[Jackpine Radical]

http://www.huffingtonpost.com/allen-frances/dsm-5-field-trials-discre_b_2047621.html

The $3 million DSM-5 Field Trials have been a pure disaster from start to finish. First, there was the poor choice of design. The study restricted itself to reliability -- the measurement of diagnostic agreement among different raters. Unaccountably, it failed to address two much more crucial questions -- DSM-5's potential impact on who would be diagnosed and on how much its dramatic lowering of diagnostic thresholds would increase the rates of mental disorder in the general population. There was no possible excuse for not asking these simple-to-answer and vitally important questions. We have a right to know how much DSM-5 will contribute to the already rampant diagnostic inflation in psychiatry, especially since this risks even greater overuse of psychotropic drugs.

Second problem -- the design of the DSM-5 field trial had a byzantine complexity that could be dreamed up only by people with no experience in real-life field testing. One look made clear that there would be serious implementation problems and that it would be impossible to complete within the time allotted. The first stage of the field trial limped in eighteen months late, having taken twice as long as was scheduled. APA then had to choose between delaying the publication of DSM-5 or canceling its planned second stage of field testing that was meant to provide for desperately needed quality control. APA decided to cancel the trial and instead is rushing ahead with the premature publication of DSM-5 next May -- publishing profits clearly trumped concern for the quality and integrity of the product. Fiduciary responsibility was thrown out the window.

Now, we have strike three. The DSM-5 leadership has reported the results of its field trial in a distressingly misleading paper.

According to the authors, 14 of the 23 disorders had "very good" or "good" reliability; six had questionable, but "acceptable" levels; and just three had "unacceptable" rates. Sounds okay until you look at the actual data and discover that the cheerful words used by the DSM-5 leaders simply don't fit their extremely disappointing results. The paper is a classic example of Orwellian "newspeak." When DSM-5 failed to achieve acceptable reliability by historical standards, the DSM-5 leadership arbitrarily decided to move the goal posts in and lower the bar in defining what is "acceptable." In fact, only the five of the 23 DSM-5 diagnoses that achieved kappa levels of agreement between 0.60-0.79 would have been considered "good" in the past. DSM-5 cheapens the coinage of reliability by hyping these merely okay levels as "very good." Then it gets much worse. The nine DSM-5 disorders in the kappa range of 0.40-0.59 previously would have been considered just plain poor, but DSM-5 puffs these up as "good." Then DSM-5 has the chutzpah to call acceptable the six disorders that achieved lousy, absolutely unacceptable reliabilities with kappas of 0.20-0.39. DSM-5 finally finds unacceptable the three diagnoses that were below <0.20 (which is barely better than chance).
…
The DSM-5 field trial fiasco and its attempted cover-up is more proof (if any were needed) that APA has lost its competence and credibility as custodian for DSM. A diagnostic system that affects so many crucial decisions in our society cannot be left to a small professional association whose work is profit driven, lacking in scientific integrity, and insensitive to public weal.

[Shrike47]

1. Do you know which 5 disorders had good reliability under the earlier standards?

[Jackpine Radical]

2. No, & the Am J Psychiat is pay-only access, so I can't get the answer easily.

http://psychiatryonline.org/article.aspx?articleid=1387935