By John W. Henson, MD, FAAN, AAN.com Science Editor
As of January 15, all Neurology® articles dealing with therapeutic intervention will include level of evidence (LOE) classification. LOE evaluations will range from Class 1 (randomized clinical trials with particular populations, etc., that indicate high level of reliability for practice) to Class IV (observational studies with no controls that have less reliability for practice). AAN.com asked Robert A. Gross, MD, PhD, FAAN, Deputy Editor of Neurology, to comment on this new initiative. John W. Henson, MD, FAAN, AAN.com Science Editor conducted the interview.
AAN.com: This certainly looks like an important editorial initiative. What percent of papers submitted to Neurology will be affected by the first phase?
Gross: Initially we will include all articles relating to therapeutic trials, representing about two out of the 10 of the papers we publish each week. We hope to eventually publish designations of level of evidence for all scientific articles.
AAN.com: How many other medical journals are planning this step or have already taken it? Have there been any early impressions about the impact of publishing LOEs?
Robert A. Gross, MD, PhD, FAAN
Gross: There are few other journals that publish levels of evidence. Since we have yet to institute this policy in print, and the editorial announcing it has only just appeared, we have not heard from other editors.
AAN.com: At what stage of the editorial process will the adjudication occur, and will the peer reviewers be aware of the LOE class?
Gross: The authors will provide their own assessment of the level of evidence for each clinical question the study was designed to address. For example, a secondary analysis may carry a lower level of evidence since the study may not have been designed with that analysis in mind. Our reviewers therefore will be aware of the authors' designations. For the papers that are selected for revision, an adjudication team, drawn from the members of the Quality Standards Subcommittee (QSS) of the AAN, will determine whether the authors' assessment is correct. If not, then revision of the designation will occur during the process of revising the manuscript. By the time of acceptance, we anticipate the designation of the clinical question(s) and the associated level(s) of evidence will have been completed.
AAN.com: In the development of a practice parameter, an LOE is assigned to multiple related studies and a treatment recommendation is synthesized. Is there a risk of readers mistakenly equating the value of an LOE designation for a single study with the more powerful conclusion synthesized from several studies?
Gross: We hope not, and our editorial addressed this explicitly to help avoid confusion. The LOE for a single article is a statement about that particular study as directed toward a specific clinical question. Practice parameters and guidelines, which assess the strength of evidence (or lack thereof) based on the analysis of many similar studies, represent stronger recommendations.
The LOE policy should assist in the development of practice parameters and guidelines, because the work of assigning a level of evidence for each article (more precisely, for the clinical question studied) will already have been completed. Rather than leading to an over-emphasis, it will assist in the meta-analysis. In this respect, we are collaborating with the QSS to help develop a database of articles, each assigned with appropriate LOEs for each clinical question, which we hope will be, once completed, available to clinicians for topic searches. Knowing the LOE will help them determine how to use the data to inform their practices.
AAN.com: Your January 6 editorial noted that the policy could "spur an increase in the overall quality of work submitted to Neurology." Do you think it could increase the chance that some important articles would no longer be submitted to Neurology? Could it short-circuit the importance of peer review in deciding what studies members of the AAN publish in Neurology?
Gross: In instituting any new policy, we are concerned with unintended consequences. In our own discussions, we were concerned that publishing levels of evidence might discourage authors of (for example) case series from submitting to Neurology, because they could be concerned about having a lower level of evidence designation. However, we took pains in the editorial to point out that a lower level of evidence refers not to the scientific merit of the article—we publish only those articles that are scientifically valid—but rather to the strength of that evidence for determining its value as applied to clinical care recommendations. It is possible that we could see a decline in papers submitted of studies of lower class evidence. On the other hand, we may see more high quality work, especially of Class I studies. We will be reviewing our progress periodically to assess changes in submission rates. Since we currently publish only 20 to 30 percent of the papers submitted to us, any effect on submissions may not be evident in the "portfolio" of published papers.
Neurology is on the relative cutting edge of this practice and we anticipate that many journals will be assigning LOE in the next several years as the clinical community and governing bodies are much more interested in evidence-based medicine than in the past.
Peer review remains the bedrock of our publishing process in that it provides an assessment of the quality of the science and informs the revision process so that the paper is strengthened. Assigning LOEs should not affect the review process, which concerns itself primarily with the adequacy of the science, the novelty of the question(s) and the value of the information for researchers or clinicians. LOE provides a short summation of the strength of the evidence for particular clinical questions. As such, it provides added value to the reader by placing the presented evidence in the context of clinical care.
AAN.com: Thank you very much for addressing these questions for the readers of AAN.com.
Dr. Henson has received personal compensation for consulting with GlaxoSmithKline within the past year. He receives compensation for serving as Associate Editor for Science, AAN.com.