How to select studies for a Cochrane Oral Health systematic review

Screening for eligibility
After the searches have taken place, the next step is for authors to screen the results for eligibility for inclusion in the review. Cochrane Oral Health strongly advises using a reference management software package to manage and keep track of results. For example, Endnote is a useful tool which allows import of references directly into RevMan.

Many of these software packages (including EndNote) require a subscription but may be available at authors’ institutions. There are some free packages available (e.g. Zotero, designed to work with the Firefox browser) but these have more limited functionality than subscription versions.

References should be imported in the reference management software and merged to remove duplicates; Anne will screen for duplicates but it is possible that some might be missed.

According to the Cochrane Handbook (Section 7.2.3), the process should then be as follows:

  • Examine titles and abstracts to remove obviously irrelevant reports (authors should generally be over-inclusive at this stage);
  • Retrieve full-text of the potentially relevant reports;
  • Link together multiple reports of the same study;
  • Examine full-text reports for compliance of studies with inclusion criteria of the review (please note however, that for most reviews, the absence of one or more relevant outcomes, or outcome data not in the desired format, is NOT a valid reason for excluding a study);
  • Correspond with investigators, where appropriate, to clarify study eligibility (it may be appropriate to request further information, such as missing results, at the same time);
  • Make final decisions on study inclusion and proceed to data collection.

It is most important that the final selection of studies into the review is undertaken by more than one author (Cochrane Handbook, Section 7.2.4).

Disagreements about whether a study should be included can generally be resolved by discussion. Often the cause of a disagreement is a simple oversight on the part of one of the review authors. When the disagreement is due to a difference in interpretation, this may require arbitration by another person (Cochrane Handbook, Section 7.2.4).

Occasionally, it will not be possible to resolve disagreements about whether to include a study without additional information. In these cases, authors may choose to categorise the study in their review as one that is awaiting assessment until the additional information is obtained from the study authors.

The methods section of both the protocol and the review should detail (Cochrane HandbookSection 7.2.4):

  • which author(s) examine each title and abstract to discard obviously irrelevant reports;
  • whether those who examine each full-text report to determine eligibility will do so independently (this should be done by at least two people);
  • whether the decisions on the above are made by content area experts, methodologists, or both;
  • whether the people assessing the relevance of studies know the names of the authors, institutions, journal of publication and results when they apply the eligibility criteria; and
  • how disagreements are handled.

Janet Lear at the Editorial Base can be contacted to obtain full-text papers, but only if the particular journal is not already available to the authors.

PRISMA Flowchart
All new Cochrane Reviews (not updated reviews) are required to have a PRISMA Flowchart. This can be generated by the RevMan software (see p52 of user guide). It is a diagram to demonstrate how many studies have been assessed for eligibility at each stage of the screening process.

Sections include:

  • number of records found through database searching;
  • number of additional records found through other means;
  • number of records after duplicates removed;
  • number of records screened;
  • number of records discarded after assessment of title and abstract;
  • number of full-text articles assessed for eligibility
  • number of studies included;
  • number of studies excluded (listed in Excluded studies).

ONLY studies which, based on assessment of the full-text report, do not meet the inclusion criteria for the review, should be listed in the Excluded Studies section.

The list of excluded studies should be as brief as possible. It should not list all of the reports that were identified by a comprehensive search. It should not list studies that obviously do not fulfil the entry criteria for the review as listed under ‘Types of studies’, ‘Types of participants’, and ‘Types of interventions’, and in particular should not list studies that are obviously not randomised if the review includes only randomised trials.” (Cochrane Handbook, Section 7.2.5)

The list of excluded studies should not include in vitro studies, CCTs or any other study designs.

Authors should therefore document the screening process carefully, and take note of the number of references and studies that they are dealing with at each stage.

If you have any questions about searching, or screening for eligibility, please contact the Editorial Base.