New Paper (Therapeutic): Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial about covid19-review HOT 9 OPEN

Is there a good way to credit other reviewers, e.g. on PubPeer?

Maybe we could cite the PubPeer review? It has an author, date, and permalink. We would have to add a manual reference in Manubot. I tried a URL reference to the comment permalink but Manubot extracts metadata from the manuscript, not the review

$ manubot cite --render url:https://pubpeer.com/publications/B4044A446F35DF81789F6F20F8E0EE#2
1. Hydroxychloroquine and Azithromycin as a treatment of COVID-19: preliminary results of an open-label non-randomized clinical trial
Philippe Gautret, Jean Christophe Lagier, Philippe Parola, Van Thuan Hoang, Line Medded, Morgan Mailhe, Barbara Doudier, Johan Courjon, Valerie Giordanengo, Vera ESTEVES Vieira, … Didier Raoult
(2020-03-20) https://pubpeer.com/publications/B4044A446F35DF81789F6F20F8E0EE#2

from covid19-review.

I did not mean to close it. Still navigating GitHub :S

from covid19-review.

Retraction Watch reports Hydroxychloroquine-COVID-19 study did not meet publishing society’s “expected standard” based on a statement by the International Society of Antimicrobial Chemotherapy.

from covid19-review.

Good job @nafisajadavji for getting us started on looking at this paper that has been making the rounds!

There's a discussion here on this as well https://pubpeer.com/publications/3B1F9EAD4982C64445A60F5E83CCFE
@cgreene @rando2: Is there a good way to credit other reviewers, e.g. on PubPeer?

In any case, we should make sure we note the loss to follow-up issue that is highlighted on PubPeer (bold face is added by me):
"Six hydroxychloroquine-treated patients were lost in follow-up during the survey because of early cessation of treatment. Reasons are as follows: three patients were transferred to intensive care unit, including one transferred on day2 post-inclusion who was PCR-positive on day1, one transferred on day3 post-inclusion who was PCR-positive on days1-2 and one transferred on day4 post-inclusion who was PCR-positive on day1 and day3; one patient died on day3 post inclusion and was PCR-negative on day2; one patient decided to leave the hospital on day3 post-inclusion and was PCR-negative on days1-2; finally, one patient stopped the treatment on day3 post-inclusion because of nausea and was PCR-positive on days1-2-3."

from covid19-review.

Note: I am a biostatistics MD-PhD student at Penn who is currently studying clinical trial methodology. I believe I have expertise in this area, but want to disclose I am still a student. Answers from above are duplicated and additional contributions/edits from me are included.

Title: Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial

Please paste a link to the paper or a citation here:

Link: https://www.sciencedirect.com/science/article/pii/S0924857920300996

What is the paper's [Manubot-style citation]?

Citation: @doi:10.1016/j.ijantimicag.2020.105949

Please list some keywords (3-10) that help identify the relevance of this paper to COVID-19

• hydroxychloroquine
• azithomycin
• clinical trial
• therapeutics

Which areas of expertise are particularly relevant to the paper?

• virology
• epidemiology
• biostatistics
• immunology
• pharmacology

Questions to answer about each paper:

Please provide 1-2 sentences introducing the study and its main findings

The study investigated the impact of hydroxychloroquine combined with azithromycin on viral loads in SARS-CoV-2-infected patients. The study argues hydroxychloroquine with or without azithromycin is an effective therapeutic for reducing viral load and hospital stay.

Study question(s) being investigated:

How many/what drugs/combinations are being considered?

The trial arms were: hydroxychloroquine, hydroxychloroquine + azithromycin, and standard of care

What are the main hypotheses being tested?

Of the treatments hydroxychloroquine and standard of care, which is most effective for treating COVID-19 patients?

Is there a synergistic effect of azithromycin on treatment with hydroxychloroquine?

Study population:

What is the model system (e.g., human study, animal model, cell line study)?

Human

What is the sample size? If multiple groups are considered, give sample size for each group (including controls).

• 26 treated with hydroxychloroquine (6 dropped from analysis, 6 additionally treated with azithromycin)
• 16 treated with standard of care

For human studies:

What countries/regions are considered?

Various regions in France

What is the age range, gender, other relevant characteristics?

• Hydroxychloroquine arm: age (51.2, sd = 18.7), male gender (n = 9, 45%), asymptomatic (n = 2, 10%), time from symptoms to trial inclusion (4.1 days, sd = 2.6)
• Standard of care arm: age (37.3, sd = 24.0), male gender (n = 6, 37.5%), asymptomatic (n = 4, 25%), time from symptoms to trial inclusion (3.9 days, sd = 2.8)

What is the setting of the study (random sample of school children, inpatient, outpatient, etc)?

Hospitalized patients at Marseille hospital and various, unnamed hospitals.

What other specific inclusion-exclusion criteria are considered?

Inclusion criteria: age >12 years, PCR documented SARS-CoV-2 carriage in nasopharyngeal sample at admission whatever their clinical status.

Patients were excluded if they had a known allergy to hydroxychloroquine or chloroquine or had another known contraindication to treatment with the study drug, including retinopathy, G6PD deficiency and QT prolongation. Breastfeeding and pregnant patients were excluded based on their declaration and pregnancy test results when required.

Treatment assignment:

How are treatments assigned?

Interventional study

Is the study randomized?

Study is non-randomized, non-placebo-controlled, non-blinded. (Non-placebo-controlled here to mean the control arm is standard of care, not standard of care + placebo pill.)

Provide other relevant details about the design.

Treatment arm is entirely confounded by hospital. All treatment arm patients were recruited from a single center while control arm patients could come from other centers. Patients who refused hydroxychloroquine were placed in control arm.

Six treatment arm patients were dropped due to discontinuation of drug because of ICU placement, death, or leaving the hospital. No Intention-To-Treat analysis was provided.

Bias in treatment could be present due to lack of blinding. Decisions made for treatment arm patients could differ from those made for control arm patients.

Outcome Assessment:

Describe the outcome that is assessed and whether it is appropriate.

Outcome is defined as negative nasopharyngeal PCR by day post-study inclusion. Outcome is assessed by a clinician. Outcome seems appropriate; however, there are discrepancies in results, detailed below.

Are outcome measurements complete?

Six individuals are lost to follow up and not included in analysis. All six patients are from treatment arm - five of the six patients had negative effects.

Data on negative nasopharyngeal PCR is incomplete and imputed for some control patients. There are discrepancies in this reporting: on day 4 post-inclusion, four control patients are virus-free but on day 5, only three are, and on day 6, only 2 are. The data suggests two patients had false negative results.

Are outcome measurements subject to various kinds of bias?

Bias in treatment could be present due to lack of blinding. Decisions made for treatment arm patients could differ from those made for control arm patients.

Statistical Methods Assessment:

What methods are used for inference?

Fisher's exact test and chi-squared test are used for binomial outcomes. Use is not specified, but assumption of "large enough n" for each outcome is likely violated. Multiple comparisons (due to testing sequential data) are not corrected for.

Are the methods appropriate for the study?

Treatment arm is entirely confounded by hospital. This issue cannot be accounted for by analysis method. No Intention-To-Treat analysis was conducted. This is inappropriate.

Are adjustments made for possible confounders?

No adjustments are made, despite imbalance in known confounder of age.

Results Summary:

What is the estimated association?

Estimated association is difference in binary outcome.

What measures of confidence or statistical significance are provided?

Day 3: hydroxychloroquine (10/20 with negative titer), control (1/16 with negative titer). P = 0.005
Day 4: hydroxychloroquine (12/20 with negative titer), control (4/16 with negative titer). P = 0.04
Day 5: hydroxychloroquine (13/20 with negative titer), control (3/16 with negative titer). P = 0.006
Day 6: hydroxychloroquine (14/20 with negative titer), control (2/16 with negative titer). P = 0.001

Note: If Intention-to-Treat analysis is conducted (which is a more appropriate analysis), p-values change to 0.030, 0.21, 0.056, and 0.010, respectively. If p-values are corrected for using Bonferroni correction (which is also appropriate), only the Day 6 results are significant at alpha = 0.05.

Interpretation of results for study population:

Can we make a causal interpretation for the individuals in the study of drug -> outcome, such as "taking drug A improves likelihood of survival twofold over taking drug B."

No causal interpretation can be made.

Are there identified side effects or interactions with other drugs?

Hydroxychloroquine side effects in non-COVID-19 patients is well known. Side effects or interactions in COVID-19 patients are not well characterized.

Are there specific subgroups with different findings?

No subgroup testing was done.

Extrapolation of conclusions to other groups of individuals not specifically included in the study:

If the study is an animal study, which animal and how relevant is that model?

N/A

If it is a human study, what characteristics of the study population may support/limit extrapolation?

• Poor study design, confounding by hospital, lack of randomization, lack of blinding, and lack of placebo limit extrapolation.
• Further study by RCT is necessary.

Summary of reliability

Study provides promising exploratory data on use of hydroxychloroquine to treat COVID-19. Results are not further interpretable due to study design issues. An RCT is necessary to study whether this hydroxychloroquine is useful in this setting.

Progress

Check off the components as they are completed. If the component is not applicable, check the box as well.

• 1-2 sentences introducing the study and its main findings
• Describe How many/what drugs/combinations are being considered
• Describe the model system
• What is the sample size?
• What countries/regions are considered
• What is the age range, gender, other relevant characteristics
• Describe study setting
• Describe other specific inclusion-exclusion criteria
• Describe how treatments are assigned
• Describe randomization (or not) and other relavent details about the design
• Describe the outcome that is assessed and whether it is appropriate.
• Describe whether the outcome measurements are complete
• Are outcome measurements subject to various kinds of bias?
• Describe methods used for inference
• Describe whether the methods are appropriate for the study
• Are adjustments made for possible confounders?
• Describe the estimated association
• What measures of confidence or statistical significance are provided?
• Describe whether a causal interpretation can be made
• Are there identified side effects or interactions with other drugs?
• Are there specific subgroups with different findings?
• If the study is an animal study, which animal and how relevant is that model?
• If it is a human study, what characteristics of the study population may support/limit extrapolation?
• Summary of reliability

from covid19-review.

@hufengling thanks for contributing! You made some good additions!

from covid19-review.

Thank you both!

@hufengling: Great job filling in so many details! Can you include numerical results under Results Summary? For example, you say that only the p-values are given, but can you give the actual p-value? (Same thing here #123) I should maybe have been a bit clearer in the template... I meant to say that you should include what the measures are + their estimates. Eg if the estimate is of an odds ratio and the 95% CI is also include, state that both the odds ratio and the 95% CI are estimated and give their estimates as well. If the p-value is also included, also give that (even though it can be obtained just from inverting the CI).

Also: I don't think that a study is necessarily problematic just because it's not placebo-controlled or that this is even a weakness. In general it's often not possible or it may even be unethical to run a placebo-controlled study, so comparing to standard-of-care is fine (indeed, this is what you want to do for a new treatment, you want to compare it to the care that patients with that illness will generally get).

Again, thank you so much for your great contributions!

from covid19-review.

Sorry, I misunderstood! I've updated my comment (and added in some comments on how results would change given alternate, more appropriate clinical trial analysis.)

Regarding your point about placebo, I guess I should have been more clear. I meant that trial arms should be: 1) hydroxychloroquine + supportive care and 2) placebo + supportive care. The latter is standard of care, but giving control patients an additional placebo pill would make the argument stronger. I've clarified this in my comment above.

Patients can do better if they think they are receiving the "new and better" drug. Physicians may also treat patients differently if they know they are on hydroxychloroquine or not - perhaps they pay closer attention to hydroxychloroquine patients.

from covid19-review.

Thanks for the update and got what you're saying now about placebo-controlled trials!

from covid19-review.