Thanks to the impressive planning, reasoning, and tool-calling capabilities of Large Language Models (LLMs), people are actively studying and developing LLM-powered agents. These agents are possible to autonomously (and collaboratively) solve complex tasks, or simulate human interactions.
Our goal with this project is to build an exhaustive collection of awesome resources relevant to LLM-powered agents encompassing papers, repositories, and more. We strive to keep these updated regularly and continuously. We greatly appreciate any contributions via PRs, issues, emails, or other methods.
New! Call for Papers until Aug 25 Sep 10, 2023 - Personalized Generative AI @ CIKM'23 - Submissions relating to LLM-powered agents are highly welcomed and valued!
- Papers
- Open-Source Projects
- Perspectives
- Other Related Sources
- Acknowledgement
π₯ for papers with >50 citations or repositories with >200 stars.
π for papers accepted by reputed conferences/journals.
- [Sep 2023] "Self-driven Grounding: Large Language Model Agents with Automatical Language-aligned Skill Learning." Shaohui Peng (CAS) et al. arXiv. [paper]
- [Aug 2023] "ExpeL: LLM Agents Are Experiential Learners." Andrew Zhao (THU) et al. arXiv. [paper]
- [Aug 2023] "Synergistic Integration of Large Language Models and Cognitive Architectures for Robust AI: An Exploratory Analysis." Oscar J. Romero (CMU) et al. arXiv. [paper]
- [Aug 2023] "Dynamic Planning with a LLM." Gautier Dagan (U of Edinburgh) et al. arXiv. [paper] [code]
- [Aug 2023] "Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization." Weiran Yao (Salesforce) et al. arXiv. [paper]
- π₯ [May 2023] "ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models." Binfeng Xu et al. arXiv. [paper] [code]
- [May 2023] "SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks." Bill Yuchen Lin (AI2) et al. arXiv. [paper] [code] [project page]
- π₯ [May 2022] "Reasoning with Language Model is Planning with World Model." Shibo Hao (UCSD) et al. arXiv. [paper] [code] [project page]
- [May 2023] "ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models." Zhipeng Chen (RUC) et al. arXiv. [paper] [code]
- π₯ [Apr 2023] "LLM+P: Empowering Large Language Models with Optimal Planning Proficiency." Bo Liu (UT Austin) et al. arXiv. [paper] [code]
- π₯ [Mar 2023] "Reflexion: Language Agents with Verbal Reinforcement Learning." Noah Shinn (Northeastern) et al. arXiv. [paper] [code]
- π₯π [Oct 2022] "ReAct: Synergizing Reasoning and Acting in Language Models." Shunyu Yao (Princeton & Google Brain) et al. ICLR 2023. [paper] [code] [project page]
- [Aug 2023] "ProAgent: Building Proactive Cooperative AI with Large Language Models." Ceyao Zhang (CUHK & PKU) et al. arXiv. [paper] [project page]
- π₯ [Aug 2023] "AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors in Agents." Weize Chen (THU) et al. arXiv. [paper] [code]
- [Aug 2023] "GPT-in-the-Loop: Adaptive Decision-Making for Multiagent Systems." Nathalia Nascimento (U of Waterloo) et al. arXiv. [paper]
- [Aug 2023] "How susceptible are LLMs to Logical Fallacies?" Amirreza Payandeh (GMU & Vail Systems) et al. arXiv. [paper] [code]
- [Aug 2023] "AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework." Qingyun Wu et al. arXiv. [paper] [project page]
- [Aug 2023] "ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate." Chi-Min Chan (THU) et al. arXiv. [paper] [code]
- [Aug 2023] "LLM As DBA." Xuanhe Zhou (THU) et al. arXiv. [paper] [code]
- [Aug 2023] "Gentopia: A Collaborative Platform for Tool-Augmented LLMs." Binfeng Xu et al. arXiv. [paper] [code] [project page]
- π₯ [Aug 2023] "MetaGPT: Meta Programming for Multi-Agent Collaborative Framework." Sirui Hong (DeepWisdom) et al. arXiv. [paper] [code]
- [Jul 2023] "PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations." Ruosen Li (UT Dallas) et al. arXiv. [paper] [project page][code]
- [Jul 2023] "Unleashing Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration." Zhenhailong Wang (UIUC & MSRA) et al. arXiv. [paper] [code]
- [Jul 2023] "RoCo: Dialectic Multi-Robot Collaboration with Large Language Models." Mandi Zhao (Columbia) et al. arXiv. [paper] [code] [project page]
- [Jul 2023] "Wireless Multi-Agent Generative AI: From Connected Intelligence to Collective Intelligence." Hang Zou (Technology Innovation Institute, UAE) et al. arXiv. [paper]
- [Jul 2023] "Building Cooperative Embodied Agents Modularly with Large Language Models." Hongxin Zhang (UMass) et al. arXiv. [paper] [code] [project page]
- [Jun 2023] "RestGPT: Connecting Large Language Models with Real-World Applications via RESTful APIs." Yifan Song (PKU) et al. arXiv. [paper] [project page]
- [Jun 2023] "Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents." Yashar Talebirad (UAlberta) et al. arXiv. [paper]
- [May 2023] "Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate." Tian Liang (THU & Tencent) et al. arXiv. [paper] [code]
- π₯ [May 2023] "Large Language Models as Tool Makers." Tianle Cai (Deepmind & Princeton) et al. arXiv. [paper] [code]
- [May 2023] "Improving Factuality and Reasoning in Language Models through Multiagent Debate." Yilun Du (MIT) et al. arXiv. [paper] [code] [project page]
- [May 2023] "Agreement and Statistical Efficiency in Bayesian Perception Models." Yash Deshpande (MIT) et al. arXiv. [paper]
- [May 2023] "Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback." Yao Fu (U of Edinburgh) et al. arXiv. [paper] [code]
- π₯ [Sep 2023] "Agents: An Open-source Framework for Autonomous Language Agents." Wangchunshu Zhou (AI Waves) et al. arXiv. [paper] [code] [project page]
- π₯ [Sep 2023] "Cognitive Architectures for Language Agents." Theodore Sumers (Princeton) et al. arXiv. [paper] [repo]
- π₯ [Jul 2023] "WebArena: A Realistic Web Environment for Building Autonomous Agents." Shuyan Zhou (CMU) et al. arXiv. [paper] [code] [project page]
- [Jul 2023] "A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis." Izzeddin Gur (DeepMind) et al. arXiv. [paper]
- π₯ [Jun 2023] "Mind2Web: Towards a Generalist Agent for the Web." Xiang Deng (OSU) et al. arXiv. [paper] [code] [project page]
- [May 2023] "Augmenting Autotelic Agents with Large Language Models." CΓ©dric Colas (MIT & Inria) et al. arXiv. [paper]
- [May 2023] "Mobile-Env: An Evaluation Platform and Benchmark for Interactive Agents in LLM Era." Danyang Zhang (SJTU) et al. arXiv. [paper] [code]
- π [Apr 2023] "Emergent autonomous scientific research capabilities of large language models." Daniil A. Boiko (CMU) et al. arXiv. [paper]
- [Mar 2023] "Language Models can Solve Computer Tasks." Geunwoo Kim (UCI) et al. arXiv. [paper] [code] [project page]
- π [Jul 2022] "WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents." Shunyu Yao (Princeton) et al. NeurIPS 2022. [paper] [code] [project page]
- [Aug 2023] "LaGR-SEQ: Language-Guided Reinforcement Learning with Sample-Efficient Querying." Thommen George Karimpanal (Deakin University) et al. arXiv. [paper] [code]
- [Jul 2023] "Dialogue Shaping: Empowering Agents through NPC Interaction." Wei Zhou (GaTech) et al. arXiv. [paper]
- [Jul 2023] "Towards A Unified Agent with Foundation Models." Norman Di Palo (ICL & DeepMind) et al. Reincarnating RL @ ICLR 2023. [paper]
- [Jun 2023] "Large Language Model Is Semi-Parametric Reinforcement Learning Agent." Danyang Zhang (SJTU) et al. arXiv. [paper]
- [May 2023] "Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learning." Wenhao Li (CUHK) et al. arXiv. [paper]
- [Sep 2023] "SayNav: Grounding Large Language Models for Dynamic Planning to Navigation in New Environments." Abhinav Rajvanshi (SRI International) et al. arXiv. [paper]
- [Sep 2023] "Developmental Scaffolding with Large Language Models." M. Batuhan Celik (Bogazici University) et al. arXiv. [paper]
- [Jul 2023] "March in Chat: Interactive Prompting for Remote Embodied Referring Expression." Yanyuan Qiao (Adelaide University) et al. arXiv. [paper] [code]
- [Aug 2023] "A^2Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Models." Peihao Chen (SCUT) et al. arXiv. [paper]
- [Jul 2023] "Embodied Task Planning with Large Language Models." Zhenyu Wu (BUPT) et al. arXiv. [paper] [code] [project page]
- [Jun 2023] "Enabling Intelligent Interactions between an Agent and an LLM: A Reinforcement Learning Approach." Bin Hu (Zhejiang Lab) et al. arXiv. [paper] [code]
- π₯ [May 2023] "Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory." Xizhou Zhu (THU & SenseTim) et al. arXiv. [paper] [code]
- π₯ [May 2023] "Voyager: An Open-Ended Embodied Agent with Large Language Models." Guanzhi Wang (NVIDIA & Caltech) et al. arXiv. [paper] [code] [project page]
- [May 2023] "Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agents." Yue Wu (CMU) et al. arXiv. [paper]
- [Feb 2023] "Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents." Zihao Wang (PKU) et al. arXiv. [paper] [code]
- [Feb 2023] "Collaborating with language models for embodied reasoning." Ishita Dasgupta (DeepMind) et al. LaReL @ NeurIPS 2022. [paper]
- [Jan 2023] "Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling." Kolby Nottingham (UCI) et al. ICML 2023. [paper] [code] [project page]
- π [Dec 2022] "LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models." Chan Hee Song (OSU) et al. ICCV 2023. [paper] [project page]
- [Aug 2023] "Ambient Adventures: Teaching ChatGPT on Developing Complex Stories." Zexin Chen (GaTech) et al. arXiv. [paper]
- [Jul 2023] "Tachikuma: Understading Complex Interactions with Multi-Character and Novel Objects by Large Language Models." Yuanzhi Liang (UTS) et al. arXiv. [paper]
- [May 2023] "Role-Play with Large Language Models." Murray Shanahan (DeepMind & ICL) et al. arXiv. [paper]
- [May 2023] "clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents." Kranti Chalamalasetti (University of Potsdam) et al. arXiv. [paper] [code]
- [Apr 2023] "Towards autonomous system: flexible modular production system enhanced with large language model agents." Yuchen Xia (University of Stuttgart) et al. arXiv. [paper] [code]
- π₯ [Mar 2023] "CAMEL: Communicative Agents for "Mind" Exploration of Large Scale Language Model Society." Guohao Li (KAUST) et al. arXiv. [paper] [code] [project page]
- [Sep 2023] "An Appraisal-Based Chain-Of-Emotion Architecture for Affective Language Model Game Agents." Maximilian Croissant (UOY) et al. arXiv. [paper]
- [Sep 2023] "Unleashing the Power of Graph Learning through LLM-based Autonomous Agents." Lanning Wei (CAS & 4Paradigm) et al. arXiv. [paper]
- [Sep 2023] "TradingGPT: Multi-Agent System with Layered Memory and Distinct Characters for Enhanced Financial Trading Performance." Yang Li (SIT) et al. arXiv. [paper]
- π₯ [Sep 2023] "ModelScope-Agent: Building Your Customizable Agent System with Open-source Large Language Models." Chenliang Li (Alibaba) et al. arXiv. [paper] [code] [demo]
- [Aug 2023] "Recommender AI Agent: Integrating Large Language Models for Interactive Recommendations." Xu Huang (USTC) et al. arXiv. [paper] [code]
- [Aug 2023] "RecMind: Large Language Model Powered Agent For Recommendation." Yancheng Wang (ASU) et al. arXiv. [paper]
- [Aug 2023] "LLM Powered Sim-to-real Transfer for Traffic Signal Control." Longchao Da (ASU) et al. arXiv. [paper]
- [Aug 2023] "Out of the Cage: How Stochastic Parrots Win in Cyber Security Environments." Maria Rigaki (ΔVUT) et al. arXiv. [paper] [code]
- [Aug 2023] "Is There Any Social Principle for LLM-Based Agents?" Jitao Bai (TJU) et al. arXiv. [paper]
- [Aug 2023] "ChatEDA: A Large Language Model Powered Autonomous Agent for EDA." Zhuolun He (CUHK & Shanghai AI Lab) et al. arXiv. [paper]
- [Aug 2023] "The Hitchhiker's Guide to Program Analysis: A Journey with Large Language Models." Haonan Li (UCR) et al. arXiv. [paper]
- [Jun 2023] "Towards Autonomous Testing Agents via Conversational Large Language Models." Robert Feldt (Chalmers University of Technology) et al. arXiv. [paper]
- π₯ [Mar 2023] "HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face." Yongliang Shen (ZJU & MSRA) et al. arXiv. [paper] [code]
- [Aug 2023] "Enhancing Trust in LLM-Based AI Automation Agents: New Considerations and Future Challenges." Sivan Schwartz (IBM Research) et al. AutoMate @ IJCAI 2023. [paper]
- [Aug 2023] "CGMI: Configurable General Multi-Agent Interaction Framework." Jinxin Shi (ECNU) et al. arXiv. [paper]
- [Aug 2023] "Exploring the Intersection of Large Language Models and Agent-Based Modeling via Prompt Engineering." Edward Junprung (UC Berkeley) et al. arXiv. [paper] [code]
- π₯ [Aug 2023] "AgentSims: An Open-Source Sandbox for Large Language Model Evaluation." Jiaju Lin (PTA Studio & PSU) et al. arXiv. [paper] [code] [project page]
- [Jul 2023] "S^3: Social-network Simulation System with Large Language Model-Empowered Agents." Chen Gao (THU) et al. arXiv. [paper]
- [Jul 2023] "Are you in a Masquerade? Exploring the Behavior and Impact of Large Language Model Driven Social Bots in Online Social Networks." Siyu Li (SCU) et al. arXiv. [paper] [dataset]
- [Jul 2023] "Communicative Agents for Software Development." Chen Qian (THU) et al. arXiv. [paper]
- [Jul 2023] "Epidemic Modeling with Generative Agents." Ross Williams (Virginia Tech) et al. arXiv. [paper] [code]
- [Jul 2023] "To Infinity and Beyond: SHOW-1 and Showrunner Agents in Multi-Agent Simulations." Philipp Maas (Fable Studio) et al. preprint. [paper] [project page]
- [Jun 2023] "RecAgent: A Novel Simulation Paradigm for Recommender Systems." Lei Wang (RUC) et al. arXiv. [paper] [code]
- [May 2023] "Playing repeated games with Large Language Models." Elif Akata (U of TΓΌbingen) et al. arXiv. [paper]
- [May 2023] "The Role of Summarization in Generative Agents: A Preliminary Perspective." Xiachong Feng (HIT) et al. arXiv. [paper]
- [Apr 2023] "Multi-Party Chat: Conversational Agents in Group Settings with Humans and Models." Jimmy Wei (Cornell & Meta) et al. arXiv. [paper] [dataset] [code]
- π₯ [Apr 2023] "Generative Agents: Interactive Simulacra of Human Behavior." Joon Sung Park (Stanford) et al. arXiv. [paper] [code]
- [Aug 2023] "Quantifying the Impact of Large Language Models on Collective Opinion Dynamics." Chao Li (ZJU) et al. arXiv. [paper]
- [Aug 2023] "SAPIEN: Affective Virtual Agents Powered by Large Language Models." Masum Hasan (U of Rochester) et al. arXiv. [paper]
- [Jul 2023] "Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Support." Zilin Ma (Harvard) et al. arXiv. [paper]
- π₯ [May 2023] "Training Socially Aligned Language Models in Simulated Human Society." Ruibo Liu (Dartmouth) et al. arXiv. [paper] [code]
- [May 2023] "Language Models Meet World Models: Embodied Experiences Enhance Language Models." Jiannan Xiang (UCSD) et al. arXiv. [paper] [code]
- [Aug 2023] "BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents." Zhiwei Liu (Salesforce) et al. arXiv. [paper] [code]
- π₯ [Aug 2023] "AgentBench: Evaluating LLMs as Agents." Xiao Liu (THU) et al. arXiv. [paper] [code] [project page]
- [Aug 2023] "TPTU: Task Planning and Tool Usage of Large Language Model-based AI Agents." Jingqing Ruan (SenseTime) et al. arXiv. [paper]
- [Sep 2023] "The Rise and Potential of Large Language Model Based Agents: A Survey." Zhiheng Xi (FDU) et al. arXiv. [paper] [GitHub]
- π₯ [Aug 2023] "A Survey on Large Language Model based Autonomous Agents." Lei Wang (RUC) et al. arXiv. [paper] [GitHub]
- π₯ [Mar 2023] "A Survey of Large Language Models (Sec. 6.3 - Planning for Complex Task Solving)." Wayne Xin Zhao (RUC) et al. arXiv. [paper] [GitHub]
Auto-GPT - An experimental open-source attempt to make GPT-4 fully autonomous.
π¦οΈπ LangChain - Building applications with LLMs through composability.
GPT Engineer - Specify what you want it to build, the AI asks for clarification, and then builds it.
MetaGPT - π The Multi-Agent Framework: Given one line Requirement, return PRD, Design, Tasks, Repo.
BabyAGI - An AI-powered task management system.
AI Town π π»π - A deployable starter kit for building and customizing your own version of AI town - a virtual town where AI characters live, chat and socialize.
GPTeam - An open-source multi-agent simulation.
π ChatArena - Multi-agent language game environments for LLMs.
π€ AgentVerse πͺ - A flexible framework that simplifies the process of building custom multi-agent environments for large language models (LLMs).
- Language agents: a critical evolutionary step of artificial intelligence - Yu Su (OSU), Sep 5, 2023.
- Introducing XLang: An Open-Source Framework for Building Language Model Agents via Executable Language Grounding - XLANG Lab, Aug 9, 2023.
- What are GPT Agents? A deep dive into the AI interface of the future - Learn why Agents are a core part of the future of AI, Logan Kilpatrick (OpenAI), Jul 25, 2023.
- Language Agents in the Digital World: Opportunities and Risks - Shunyu Yao (Princeton) et al., Jul 24, 2023.
- KokoMind: Can LLMs Understand Social Interactions? - Imagine an AI π€ at a cocktail party π», Weiyan Shi (Columbia) et al., Jul, 2023
- LLM Powered Autonomous Agents - Amazing blog by Lilian Weng (OpenAI), Jun 23, 2023.
- Personalized Generative AI @ CIKM'23
- LLM-Agents-Papers - A repo lists papers about LLM role playing, memory mechanism and LLM game playing.
- LLMAgentPapers - Must-read papers on multiagents of LLMs.
- awesome-llm-agents - A curated list of awesome LLM agents.
We greatly appreciate any contributions via PRs, issues, emails, or other methods. Thanks Tianle Cai (@ctlllll), Yifan Song (@Yifan-Song793), Xinya Du (@xinyadu), Binfeng Xu (@billxbf), Xuanhe Zhou (@zhouxh19), Boyuan Zheng (@boyuanzheng010) and many others for their kind suggestions and contributions. β€οΈ
The repository is initially built and maintained by Yupeng Hou ([email protected]).