This is an actively updated list of practical guide resources of healthcare LLMs. It's based on our survey paper: A Survey of Large Language Models for Healthcare: Progress, Application, and Challenge and efforts from @simon599.

These sources aim to help practitioners navigate the vast landscape of healthcare-specific large language models (LLMs) and their applications in medical natural language processing (NLP) applications. If you find any resources in our repository helpful, please feel free to use them (don't forget to cite our paper! 😃).

**bibtex to be changed

    @article{yang2023harnessing,
        title={Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond}, 
        author={Jingfeng Yang and Hongye Jin and Ruixiang Tang and Xiaotian Han and Qizhang Feng and Haoming Jiang and Bing Yin and Xia Hu},
        year={2023},
        eprint={2304.13712},
        archivePrefix={arXiv},
        primaryClass={cs.CL}
    }

• Version 1 live!

• Medical-domain LLMs
  • Pre-training
  • Fine-tuning
  • Prompting
• Clinical Applications
  • Medical Diagnosis
  • Formatting and ICD Coding
  • Clinical Report Generation
  • Medical Education
  • Medical Robotics
  • Medical Language Translation
  • Mental Health Support
• Challenges
  • Hallucination
  • Lack of Evaluation Benchmarks and Metrics
  • Domain Data Limitations
  • New Knowledge Adaptation
  • Behavior Alignment
  • Ethical, Legal, and Safety Concerns
• Future Directions
  • Introduction of New Benchmarks
  • Interdisciplinary Collaborations
  • Multi-modal LLM
  • LLMs in Less Established Fields of Healthcare

• BioBERT
  • BioBERT: a pre-trained biomedical language representation model for biomedical text mining. 2020. paper
  • Saama research at mediqa 2019: Pre-trained biobert with attention visualisation for medical natural language inference. 2019. paper
• PubMedBERT
  • Domain-specific language model pretraining for biomedical natural language processing. 2021. paper
• SciBERT
  • SciBERT: A pretrained language model for scientific text. 2019. paper
• ClinicalBERT
  • Publicly available clinical BERT embeddings. 2019. paper
• BlueBERT
  • Transfer learning in biomedical natural language processing: an evaluation of BERT and ELMo on ten benchmarking datasets. 2019. paper
  • Detecting redundancy in electronic medical records using clinical bert. 2020. paper
  • Identification of semantically similar sentences in clinical notes: Iterative intermediate training using multi-task learning. 2020. paper
• BioCPT
  • Biocpt: Contrastive pre-trained transformers with large-scale pubmed search logs for zero-shot biomedical information retrieval. 2023. paper
• BioGPT
  • BioGPT: generative pre-trained transformer for biomedical text generation and mining. 2022. paper
• OphGLM
  • OphGLM: Training an Ophthalmology Large Language-and-Vision Assistant based on Instructions and Dialogue. 2023. paper
• GatorTron
  • Gatortron: A large clinical language model to unlock patient information from unstructured electronic health records. 2022. paper
  • A large language model for electronic health records. 2022. paper
• GatorTronGPT
  • A Study of Generative Large Language Model for Medical Research and Healthcare. 2023. paper

• DoctorGLM
  • Doctorglm: Fine-tuning your chinese doctor is not a herculean task. 2023. paper
• BianQue
  • BianQue: Balancing the Questioning and Suggestion Ability of Health LLMs with Multi-turn Health Conversations Polished by ChatGPT. 2023. paper
• ClinicalGPT
  • ClinicalGPT: Large Language Models Finetuned with Diverse Medical Data and Comprehensive Evaluation. 2023. paper
• Qilin-Med
  • Qilin-Med: Multi-stage Knowledge Injection Advanced Medical Large Language Model. 2023. paper
• Qilin-Med-VL
  • Qilin-Med-VL: Towards Chinese Large Vision-Language Model for General Healthcare. 2023. paper
• ChatDoctor
  • ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge. 2023. paper
• BenTsao
  • Huatuo: Tuning llama model with chinese medical knowledge. 2023. paper
• HuatuoGPT
  • HuatuoGPT, towards Taming Language Model to Be a Doctor. 2023. paper
• LLaVA-Med
  • Llava-med: Training a large language-and-vision assistant for biomedicine in one day. 2023. paper
• Baize-healthcare
  • ?
• Visual Med-Alpeca
  • Visual med-alpaca: A parameter-efficient biomedical llm with visual capabilities. 2023. Repo
• PMC-LLaMA
  • Pmc-llama: Further finetuning llama on medical papers. 2023. paper
• Clinical Camel
  • Clinical Camel: An Open-Source Expert-Level Medical Language Model with Dialogue-Based Knowledge Encoding. 2023. paper
• MedPaLM 2
  • Towards expert-level medical question answering with large language models. 2023. paper
• MedPaLM M
  • Towards generalist biomedical ai. 2023. paper

• DelD-GPT
  • Deid-gpt: Zero-shot medical text de-identification by gpt-4. 2023. paper
• ChatCAD
  • Chatcad: Interactive computer-aided diagnosis on medical image using large language models. 2023. paper
• Dr. Knows
  • Leveraging a medical knowledge graph into large language models for diagnosis prediction. 2023. paper
• MedPaLM
  • Large language models encode clinical knowledge. 2022. paper

• 108
• 12
• 68
• 44
• 26
• 26

• 118
• 148
• 27
• 156
• 186
• 65
• 181
• 90
• 4
• 2
• 22
• 185
• 188
• 5
• 132
• 105
• 86
• 51
• 98
• 166
• 3
• 108
• 65
• 136
• 150

• Designing a Deep Learning-Driven Resource-Efficient Diagnostic System for Metastatic Breast Cancer: Reducing Long Delays of Clinical Diagnosis and Improving Patient Survival in Developing Countries. 2023. paper
• AI in health and medicine. 2022. paper
• Large language models in medicine. 2023. paper
• Leveraging a medical knowledge graph into large language models for diagnosis prediction. 2023. paper
• Chatcad: Interactive computer-aided diagnosis on medical image using large language models. 2023. paper

• Applying large language model artificial intelligence for retina International Classification of Diseases (ICD) coding. 2023. paper
• PLM-ICD: automatic ICD coding with pretrained language models. 2022. paper
• MIMIC-III, a freely accessible critical care database. 2016. paper
• Generative ai text classification using ensemble llm approaches. 2023. paper

• Reporting guidelines for clinical trials evaluating artificial intelligence interventions are needed. 2019. paper
• Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. 2010. paper
• Retrieve, reason, and refine: Generating accurate and faithful patient instructions. 2022. paper
• Chatcad: Interactive computer-aided diagnosis on medical image using large language models. 2023. paper
• Can GPT-4V (ision) Serve Medical Applications? Case Studies on GPT-4V for Multimodal Medical Diagnosis. 2023. paper
• Qilin-Med-VL: Towards Chinese Large Vision-Language Model for General Healthcare. 2023. paper
• Customizing General-Purpose Foundation Models for Medical Report Generation. 2023. paper
• Towards generalist foundation model for radiology. 2023. paper
• Pmc-vqa: Visual instruction tuning for medical visual question answering. 2023. paper
• Clinical Text Summarization: Adapting Large Language Models Can Outperform Human Experts. 2023. paper

• Large Language Models in Medical Education: Opportunities, Challenges, and Future Directions. 2023. paper
• Large ai models in health informatics: Applications, challenges, and the future. 2023. paper
• The Advent of Generative Language Models in Medical Education. 2023. paper
• The impending impacts of large language models on medical education. 2023. paper

• A Nested U-Structure for Instrument Segmentation in Robotic Surgery. 2023. paper
• The multi-trip autonomous mobile robot scheduling problem with time windows in a stochastic environment at smart hospitals. 2023. paper
• Advanced robotics for medical rehabilitation. 2016. paper
• GRID: Scene-Graph-based Instruction-driven Robotic Task Planning. 2023. paper
• Large ai models in health informatics: Applications, challenges, and the future. 2023. paper
• Trust in Construction AI-Powered Collaborative Robots: A Qualitative Empirical Analysis. 2023. paper

• Machine translation of standardised medical terminology using natural language processing: A Scoping Review. 2023. paper
• The Advent of Generative Language Models in Medical Education. 2023. paper
• The impending impacts of large language models on medical education. 2023. paper

• ChatCounselor: A Large Language Models for Mental Health Support. 2023. paper
• An overview of the features of chatbots in mental health: A scoping review. 2019. paper
• Tell me, what are you most afraid of? Exploring the Effects of Agent Representation on Information Disclosure in Human-Chatbot Interaction, 2023, paper
• A Brief Wellbeing Training Session Delivered by a Humanoid Social Robot: A Pilot Randomized Controlled Trial. 2023. paper
• Real conversations with artificial intelligence: A comparison between human–human online conversations and human–chatbot conversations. 2015. paper

• Survey of hallucination in natural language generation. 2023. paper
• Med-halt: Medical domain hallucination test for large language models. 2023. paper
• A survey of hallucination in large foundation models. 2023. paper
• Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback. 2023. paper
• Improving Factuality of Abstractive Summarization via Contrastive Reward Learning. 2023. paper
• Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. 2023. paper
• Retrieval augmentation reduces hallucination in conversation. 2021. paper
• Chain-of-verification reduces hallucination in large language models. 2023. paper

• What disease does this patient have? a large-scale open domain question answering dataset from medical exams. 2021. paper
• Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. 2022. paper
• Large language models encode clinical knowledge. 2023. paper
• Truthfulqa: Measuring how models mimic human falsehoods. 2021. paper
• HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models. 2023. paper

• Large language models encode clinical knowledge. 2023. paper
• Towards expert-level medical question answering with large language models. 2023. paper
• ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge. 2023. paper
• Textbooks Are All You Need. 2023. paper
• Model Dementia: Generated Data Makes Models Forget. 2023. paper

• Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark. 2023. paper
• Editing Large Language Models: Problems, Methods, and Opportunities. 2023. paper
• Retrieval-augmented generation for knowledge-intensive nlp tasks. 2020. paper

• Training language models to follow instructions with human feedback. 2022. paper
• Aligning ai with shared human values. 2020. [https://arxiv.org/abs/2008.02275]
• Training a helpful and harmless assistant with reinforcement learning from human feedback. 2022. paper
• The power of scale for parameter-efficient prompt tuning. 2021. paper
• P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. 2021. paper
• Finetuned language models are zero-shot learners. 2021. paper
• Improving alignment of dialogue agents via targeted human judgements. 2022. paper
• Webgpt: Browser-assisted question-answering with human feedback. 2021. paper
• Languages are rewards: Hindsight finetuning using human feedback. 2023. paper

• ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. 2023. paper
• ChatGPT listed as author on research papers: many scientists disapprove. 2023. paper
• A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics. 2023. paper
• Multi-step jailbreaking privacy attacks on chatgpt. 2023. paper
• Jailbroken: How does llm safety training fail?. 2023. paper

• A comprehensive benchmark study on biomedical text generation and mining with ChatGPT. 2023. paper
• Creation and adoption of large language models in medicine. 2023. paper
• Large language models encode clinical knowledge. 2023. paper

• Creation and adoption of large language models in medicine. 2023. paper
• ChatGPT and Physicians' Malpractice Risk. 2023. paper

• A Survey on Multimodal Large Language Models. 2023. paper
• Mm-react: Prompting chatgpt for multimodal reasoning and action. 2023. paper
• Minigpt-4: Enhancing vision-language understanding with advanced large language models. 2023. paper
• ChatGPT for shaping the future of dentistry: the potential of multi-modal large language model. 2023. paper
• Frozen Language Model Helps ECG Zero-Shot Learning. 2023. paper
• Exploring and Characterizing Large Language Models For Embedded System Development and Debugging. 2023. paper
• MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models. 2023. paper

• Large language models encode clinical knowledge. 2023. paper
• Towards expert-level medical question answering with large language models. 2023. paper
• A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics. 2023. [paper]https://arxiv.org/abs/2310.05694()
• Large Language Models in Sport Science & Medicine: Opportunities, Risks and Considerations. 2023. paper

Adapted from https://github.com/Mooler0410/LLMsPracticalGuide