Inability to reproduce DMPfold result reported in the bioRxiv paper about dconstruct HOT 4 CLOSED

We ensured that all competing methods are run with default parameter setting according to their published papers. For DMPfold, we used the target sequences to generate final models as done in the original paper. Not sure of your benchmark, but we have made all input and output data openly available to fully reproduce DConStruct results. And one can always download the competing methods from their published papers to run in their original form with default parameter setting for reproducibility.

from dconstruct.

Based on what you describe, different MSAs were used for DMPfold and for DConStruct. DMPfold benchmark uses its default MSA (hhblits+uniclust30). DConStruct benchmark uses DeepMSA. This is not clearly explained in the manuscript, which explicitly states that DMPfold was run with DeepMSA ("We install and run DMPfold locally to predict distance histogram maps directly from the multiple sequence alignments (MSA) [34]").

If that is the case, the performance advantage of DConStruct compared to DMPFold shown in the manuscript is likely caused by better MSA used by DConStruct. This does not indicate that compare to CNS (used by DMPFold), DConStruct has a better ability to fold the protein given the same distance prediction.

from dconstruct.

We never "explicitly" state that "DMPfold was run with DeepMSA" in our manuscript. It is interesting to note your incorrect presumptions and the resulting unfounded conclusions, indicating an apparent bias for DeepMSA [1].

Reference:

[1] Chengxin Zhang, Wei Zheng, S. M. Mortuza, Yang Li, Yang Zhang. “DeepMSA: constructing deep multiple sequence alignment to improve contact prediction and fold-recognition for distant-homology proteins.” Bioinformatics, 36 (7): 2105-2112 (2020).

from dconstruct.

The sentence I quoted is from section "DConStruct: hybridized distance- and contact-based hierarchical structure modeling" of the DConStruct manuscript.

We install and run DMPfold locally to predict distance histogram maps directly from the multiple sequence alignments (MSA) [34]

[34] Zhang C, Zheng W, Mortuza SM, Li Y, Zhang Y. DeepMSA: constructing deep multiple sequence alignment to improve contact prediction and fold-recognition for distant-homology proteins. Bioinformatics. 2020;36: 2105–2112. doi:10.1093/bioinformatics/btz863

A similar sentence is from section "Test datasets and programs to compare" of the DConsStruct manuscript.

We feed the multiple sequence alignments (MSAs) [34] to DMPfold and obtain the predicted initial distance histograms (rawdispred.current files) containing 20 distance bins with their associated likelihoods

In any case, even if my understanding is incorrect, could you explain the following questions:

1. What is the MSA used by "DConsStruct" method in Table 3? Please kindly provide the command line, which is not included in the github repo of DConsStruct.
2. What is the MSA used by "DMPFold" method in Table 3? Again, please kindly provide the command line, as DMPfold can accept arbitrary user input MSA despite having its built-in MSA generation pipeline.
3. Could you provide the link to the MSA files used herein? If the files are too big for all dataset, you can just provide the MSAs for the 40 CASP FM targets.

from dconstruct.