Response to: Critique of the review of ‘Water fluoridation for the prevention of dental caries' published by the Cochrane Collaboration in 20241
Authors of response: Anne-Marie Glenny, Dwayne Boyers, Janet E Clarkson, Zipporah Iheozor-Ejiofor, Sharon R Lewis, Lucy O’Malley, Philip Riley, Tanya Walsh, Helen V Worthington
Walmsley et al1 present a critique of the 2024 Cochrane review on the initiation or cessation of community water fluoridation (CWF).2
The authors of the critique form part of the author team who "heavily criticised" the 2015 Cochrane review with the same flawed interpretation. In preparing to update the 2024 Cochrane review, we engaged with clinicians and researchers to discuss appropriate changes to the methods. For example, these discussions led to the inclusion of single time point studies in an appendix in order to provide wider context. We believe the critique, written by representatives from the British Fluoridation Society (BFS), continues to misrepresent our review to the wider audience.
We reached out to the authors of the critique to discuss their concerns with our review; they chose not to meet with us prior to publication of their critique. We have submitted a letter to the BDJ in response to the critique. Journal guidelines allow us only 500 words to address the critique. Here, we present a more detailed response.
1. “The core flaw is conceptual: the review applies evidence hierarchies that privilege RCTs…” We are uncertain why Walmsley and colleagues focus so frequently on randomised controlled trials (RCTs) throughout their critique. The Cochrane review included no RCTs. In fact, we mention this study design only once in the review to state that they are “logistically unfeasible” for this public health intervention.
2. Study designs. Walmsley and colleagues state that “An evaluation of the effectiveness of CWF would benefit from a combination of evidence from experimental and observational studies.” We agree with the authors. In fact, we developed our protocol to allow for this. Our criteria included non‐randomised studies of interventions (NRSI) (encompassing experimental, quasi-experimental and observational studies). Because we aimed to assess the initiation or cessation of CWF, we included prospective studies with a concurrent control comparing a fluoridated water community with a non‐fluoridated water community. We applied no “hierarchy of study designs” in the review criteria.
3. Risk of bias assessment. The critique incorrectly states that the review included the “misapplication of bias assessment tools designed for clinical interventions”. To assess risk of bias of included studies, we used ROBINS-I. This tool is specifically designed for assessing risk of bias in NRSI. Those developing the tool recognise that for many types of organisational or public health interventions, NRSIs are the main source of evidence.3 Because the ROBINS-I tool was being updated at time of our review, we discussed the application of the tool to CWF studies directly with the key development team prior to use.
Walmsley and colleagues state that the “ROBINS-I tool is applied to try to control for biases”. This is a fundamental misunderstanding of risk of bias assessment tools. ROBINS-I does not control for bias. It is a structured tool to assess the risk of bias.
4. Older versus newer study designs. Walmsley and colleagues indicate that we applied “inconsistent treatment of older versus newer studies”. We do not recognise this in the conduct of our review. We stratified our analyses into studies conducted before or after the widespread use of fluoride toothpaste in 1975. However, the analytical approach, risk of bias, and reporting was the same across all studies, including in the summary of findings tables, irrespective of the study year.
5. “…most research into fluoridation has necessarily been observational….While these designs are more vulnerable to bias and confounding than RCTs, they remain entirely appropriate and necessary for evaluating public health interventions.” We agree with Walmsley and colleagues on this point. The designs of included studies in our review reflects this statement, and the increased risk of bias in these study designs is reflected in our ROBINS-I assessments.
6. “Such a restrictive use of a hierarchy of study designs, as a measure of validity, is considered by Rothman to be a ‘persistent research misconception’.” We do not use a hierarchical approach to study selection. Nor did we assume certain study designs automatically provided more valid results than others. As outlined above, we used a structured tool (ROBINS-I) to assess the risk of bias within each study, focusing more on the features of each study rather than the overarching study design itself. We do not believe our approach contradicts Rothman’s ‘research misconceptions’. Indeed, the persistent misconception appears to be that we focused on RCTs within the review at the expense of observational studies designed to assess public health interventions.
7. Exclusion of “97% of the available evidence.” We believe the figure of 97% has been calculated from the original PRISMA flow diagram that presents the flow of search results during our screening and selection process. As anyone involved in searching for studies in systematic reviews will know, highly sensitive searches may introduce a lot of ‘noise’, retrieving studies that are not related to the topic area at all. The first stage of screening is to exclude the obvious ‘noise’ before looking at the full texts of potentially relevant studies. However, if the authors of the critique truly believe we have missed 97% of the available evidence (we include 22 studies on the initiation/cessation of CWF meeting our eligibility criteria; excluding 97% would equate to over 700 missing studies) we are happy to review any relevant studies they provide.
8. “This has been explicitly acknowledged by the UK Chief Medical Officers, who have stated that water fluoridation is an effective public health intervention for improving dental health equality across the UK.” We do not believe this statement is underpinned by evidence that demonstrates a reduction in dental health inequalities.
9. CATFISH study.4 Walmsley and colleagues state that “CATFISH has drawn criticism due to complications arising from the early cessation of the fluoridation scheme under study.” This statement is misleading. Fluoridation wasn’t stopped early within the study. The dose delivery of fluoride across both treatment plants was inconsistent and some months average fluoride dosing was suboptimal. In addition, severe flooding led to a temporary halt in fluoride dosing at one plant for 10 months. BFS state that “Fluoride levels in fluoridated water are checked daily by water companies and independently regulated by the Drinking Water Inspectorate. If levels deviate from the target, automatic systems trigger safety controls to correct them” (https://fluoridation.co.uk/faq/). Clearly this did not occur in this instance, and previous research has shown that variability and suboptimal fluoridation is not uncommon.5 CATFISH was a pragmatic study reflecting what happens in practice. Careful monitoring of water fluoridation plants is of critical importance but appears to get little mention when discussing the benefits of water fluoridation.
10. Fluorosis data. The Walmsley critique dedicates a section for dental fluorosis with the subheading - “Fluorosis: Reanalysis Without New Evidence”. This subheading is misleading. We explicitly state that we did not update the evidence for fluorosis, nor did we reanalyse the included data. In the review, we state clearly that “For the association of fluoridated water with dental fluorosis, the review evidence is current to February 2015”.
11. Interpretation of the effect estimates. Walmsley and colleagues state, “…the authors conclude that fluoridation “may lead to a slightly greater reduction in deft” (decayed, extracted, and filled teeth in primary dentition). The choice of language, “slightly greater”, is vague and is not accompanied by numerical estimates such as PCR or PF. This imprecision undermines the utility of the finding for policymakers, who require clearer quantification to inform decisions.” Firstly, we do not present any statement on deft in the conclusion; we comment on decayed missing and filled teeth (dmft). Secondly, we provide numerical effect estimates in the abstract, the summary of findings tables, the results section and figures 1.1 to 1.12. All effect estimates are presented with their accompanying confidence intervals to give an indication of precision. We do not envisage policy makers relying simply on statements within the conclusion of an article. Indeed, the summary of findings table are perhaps more typically used by guideline developers and policy makers; these present a quantification of all critical outcomes (where available) and a description of the certainty of the evidence.
12. Unclear use of the word bias in the critique. Authors state that “the fundamental framing of the review remains unchanged from the 2015 iteration…. As a result, the same biases persist.” We are unclear what biases are referred to here. Hence, we have been unable to respond to this claim.
13. Comparison with other systematic reviews. We are unclear why the authors have presented another critique for the update of this Cochrane review. We have set out clear objectives and used methods that are appropriate to meet these objectives. Multiple systematic reviews have explored the effectiveness of CWF or the association between fluoridated water and caries.6,7 The scope of the reviews and their methods vary, but findings are broadly consistent. Older studies show that water fluoridation reduced the incidence of dental caries and increased the proportion of caries‐free children; more recent studies show similar, but effects are smaller.
14. Using the data from our review for modelling. We note that some authors of the critique have recently published a modelling paper in the BDJ.8 Interestingly, the authors have used the effect estimates from our Cochrane review. A decision to use the data from our review for modelling is at odds with the same authors' critique of its limitations.
There are other points raised by the authors of the critique that we have not addressed here. However, we are open to discussion of these issues as required.
We encourage stakeholders to engage with the evidence on CWF, focusing attention on where and how best to implement CWF programmes to ensure optimal benefit, including maintaining optimum delivery/dosage with robust monitoring processes in place.
1. Walmsley, A., Lowry, R., Al Rasheed, A. et al. Critique of the review of ‘Water fluoridation for the prevention of dental caries' published by the Cochrane Collaboration in 2024. Br Dent J 241, 26–30 (2026). https://doi.org/10.1038/s41415-026 9613-y
2. Iheozor-Ejiofor, Z., Walsh, T., Lewis, S.R., et al. Water fluoridation for the prevention of dental caries. Cochrane Database Syst Rev. 2024 Oct 4;10(10):CD010856. https://doi.org/10.1002/14651858.CD010856.pub3
3. Sterne, J.A., Hernán, M.A., Reeves, B.C., et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016 Oct 12;355:i4919. DOI: 10.1136/bmj.i4919.
4. Goodwin, M., Emsley, R., Kelly, M.P., et al. Evaluation of water fluoridation scheme in Cumbria: the CATFISH prospective longitudinal cohort study. Public Health Res 2022;10(11). DOI: 10.3310/SHMX1584
5. Moore, D., Goodwin, M., Pretty, I.A.. Long-term variability in artificially and naturally fluoridated water supplies in England. Community Dent Oral Epidemiol 2020;48:49 55. https://doi.org/10.1111/cdoe.12502
6. McDonagh, M., Whiting, P., Bradley, M., et al. A systematic review of public water fluoridation. York: NHS Centre for Reviews and Dissemination. University of York, 2000. (Report 18.)
7. National Health and Medical Research Council (NHMRC). Information paper – Water fluoridation: dental and other human health outcomes, report prepared by the Clinical Trials Centre at University of Sydney, NHMRC; Canberra. www.nhmrc.gov.au/sites/default/files/documents/reports/fluoridation-info-paper.pdf (accessed January 2025)
8. Al Rasheed, A., Nanjappa, S., Mossey, P. et al. Modelling the potential impact of water fluoridation on dental caries in Scotland: a pilot study. Br Dent J (2026). https://doi.org/10.1038/s41415-025-9109-1