A reviewer wants me to drop one high-risk study from my six-study meta-analysis. Is that reasonable?

Your flow diagram would suddenly go from six to five in the last box without giving a reason for exclusion. Even if that satisfies this reviewer, other reviewers, or readers after publication, may find it strange, may object, and may even write a letter to the editor to express their confusion.

So if you really want five studies, you need a clear rule that excludes that study at an earlier step, written into the inclusion and exclusion criteria in your Materials and Methods. That is the standard way. If that rule was not in your original protocol, adding it now is a change to the protocol, and it has to be reported as one.

Think of an admissions process with a written exam and an interview. Six candidates were admitted. We cannot announce, after the list is posted, "I think this one is not good enough," and remove one.

If you want to do it, you give a low score at the interview, which means stating the exclusion criteria in advance. Otherwise it is a protocol violation.

Also, we usually do not exclude a study just because its quality assessment is poor. A study of poor quality is not meaningless. It is still a complete study; it may just carry some bias. If you are really concerned, you can pool only the low-risk studies to see the direction. Or consider one-study removal: remove the high-risk study and see whether the direction and conclusion change. Or do a subgroup analysis by level of risk of bias.

I asked the author to run it. Even without that study, the overall conclusion and the direction of the effect did not change, which shows that removing it does not lead to a different result.

I would consider keeping the original six-study analysis and calculating a five-study analysis to put in the reply to the reviewer. Explain that the result is the same without that study, describe the study's strengths and the specific reason it was rated high risk, and note that it still has real reference value and its results are not clearly different from those of the other studies.

Clear figures and text, a careful explanation, references, and an analysis run especially for the reviewer: with that much good faith, the reply should be accepted.

Of course, you know your whole study best. If you still think the reviewer has a point, that this study really is of too low quality and singling it out is reasonable, you can also change how you select studies: include only those rated low risk of bias or some concerns, exclude those at high risk, then redo the figures and tables and rewrite the paper. In that case, state in the Methods that this is a change from your protocol or registration and explain why, as PRISMA 2020 asks (item 24c), and report the analysis of all six studies as a sensitivity analysis.

References

• Boutron I, Page MJ, Higgins JPT, Altman DG, Lundh A, Hróbjartsson A. Chapter 7: Considering bias and conflicts of interest among the included studies (last updated August 2022), Section 7.6.2. In: Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane, 2024.
• Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.