by Zihao Zhao*, Sheng Wang*, Jinchen Gu*, Yitao Zhu*, Lanzhuju Mei, Zixu Zhuang, Zhiming Cui, Qian Wang, Dinggang Shen

This repository provides the official implementation of some components of ChatCAD+:

• Modality identification
• Chinese version Interactive CAD of Chest X-rays
• LLM-based knowledge retrieval
• An easy-deploy local web ui based on Gradio
• The online demo will be available soon

• We would like to thank Merck Manual Professional who make all these medical knowledge public, we sorted their website for easier usage: here
• A BART-based model that has the capability to translate chest X-ray reports into Chinese well [link]

• R2GenCMN: r2gcmn_mimic-cxr.pth and annotation.json
• PCAM weights: JFchexpert.pth
• Place annotation.json under ./r2g/ and pre-trained weights under ./weights/
• For template retrieval system, please download MIMIC-CXR reports from official website and organize them into a dictionary, save as report_en_dict.json under the ./

You can either find them from original repository or dowload from Google Drive

• pip install -r requirements.txt
• implement web.py and load your openai api-key

• Migrate the project to Gradio
• Online demo with available dental and Knee MRI network

@article{wang2023chatcad,
  title={Chatcad: Interactive computer-aided diagnosis on medical image using large language models},
  author={Wang, Sheng and Zhao, Zihao and Ouyang, Xi and Wang, Qian and Shen, Dinggang},
  journal={arXiv preprint arXiv:2302.07257},
  year={2023}
}

@article{zhao2023chatcad,
      title={ChatCAD+: Towards a Universal and Reliable Interactive CAD using LLMs},
      author={Zihao Zhao and Sheng Wang and Jinchen Gu and Yitao Zhu and Lanzhuju Mei and Zixu Zhuang and Zhiming Cui and Qian Wang and Dinggang Shen},
      journal={arXiv preprint arXiv:2305.15964},
      year={2023},
}

Our implementation (including coming version) is based on the following codebases. We gratefully thank the authors for their wonderful works.

R2GenCMN, PCAM, CSNet.