LLM4Opt

Collection on Large Language Models for Optimization (LLM4Opt)

🔥 Applying Large language models (LLMs) for diverse optimization tasks (Opt) is an emerging research area. This is a collection of references and papers of LLM4Opt. The Papers are sorted by time (first publicly available).

Any suggestions and pull requests are welcomed!

It is far from a comprehensive list. If you want to update the list:

Fork, Add, and Merge
Report an issue
Contact Fei Liu ([email protected])

The sharing principle of these references here is for research. If any authors do not want their paper to be listed here, please feel free to contact us.

Overview

Platform

Project Name	Description
EoH (Evolution of Heuristics)	optimization, mathematics, machine learning, etc
OpenELM	robots, image, programming puzzles, etc

Tutorial&Workshop

Event	Link
NeurIPS 2023 Workshop: Foundation Models for Decision Making	Link
AAAI 2024 Workshop: Public Sector LLMs: Algorithmic and Sociotechnical Design	Link
GECCO 2024 Workshop: Large Language Models for and with Evolutionary Computation (LLMfwEC)	Link
GECCO 2024 Workshop: EGML-EC — 3rd GECCO workshop on Enhancing Generative Machine Learning with Evolutionary Computation (EGML-EC) 2024	Link
GECCO 2024 Tutorial: Using Large Language Models for Evolutionary Search	Tutorial Link, Tutorial Report Link
PPSN 2024 Tutorial: Large Language Models as Tools for Metaheuristic Design: Exploring Challenges and Opportunities	Link
KDD 2024 Tutorial: NL2Code-Reasoning and Planning with LLM for Code Development	Link, Link
ICML 2024 Workshop: AI for Math	Link

Competition

Event	Link
AAAI 2024 Global Competition on Math Problem Solving and Reasoning	Link
ICML 2024 Challenges on Automated Math Reasoning	Link

Papers

Review

Title	Publication with Date	Code	Paper
Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap	Arxiv, Jan 2024	[code]	paper
A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond	Arxiv, Mar. 2024	[code]	paper
When Large Language Model Meets Optimization	Arxiv, May 2024	[code]	paper

Position paper

Title	Publication with Date	Code	Paper
The Era OF Semantic Decoding	Arxiv, Mar. 2024	[code]	paper
Leveraging Foundational Models for Black-Box Optimization: Benefits, Challenges, and Future Directions	ICML 2024, May 2024	[code]	paper

Algorithm/Heuristic/Function Search

Title	Publication with Date	Code	Paper
Hypothesis Search: Inductive Reasoning with Language Models	Arxiv Sep 2023, ICLR 2024	[code]	paper
ToolChain: Efficient Action Space Navigation in Large Language Models with A Search	Arxiv Oct 2023, ICLR 2024	[code]	paper
Algorithm Evolution using Large Language Model	Arxiv Nov 2023	code	paper
Mathematical discoveries from program search with large language models	Nature	code	paper
Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model	ICML 2024 (Oral)	code	paper
ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution	Arxiv Feb 2024	code	paper
Discovering More Effective Tensor Network Structure Search Algorithms via Large Language Models (LLMs)		[code]	paper
AutoSAT: Automatically Optimize SAT Solvers via Large Language Models	Arxiv Feb 2024	[code]	paper
Large Language Models tO Enhance Bayesian Optimization	Arxiv Feb 2024, ICLR 2024	[code]	paper
On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks	Arxiv Feb 2024	[code]	paper
How Can LLM Guide RL? A Value-Based Approach	Arxiv Feb 2024	code	paper
Evolve Cost-aware Acquisition Functions Using Large Language Models	PPSN 2024	[code]	paper
Benchmarking ChatGPT on Algorithmic Reasoning	Arxiv April 2024	[code]	paper
How Multimodal Integration Boost the Performance of LLM for Optimization: Case Study on Capacitated Vehicle Routing Problems	Arxiv March 2024	[code]	paper
LLM-ABR: Designing Adaptive Bitrate Algorithms via Large Language Models	Arxiv April 2024	[code]	paper
Constrained Neural Networks for Interpretable Heuristic Creation to Optimise Computer Algebra Systems	Arxiv April 2024	[code]	paper
GLHF: General Learned Evolutionary Algorithm Via Hyper Functions	Arxiv	[code]	paper
tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)	ICML 2024	[code]	paper
LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristics	Arxiv May 2024	code	paper

LLM as optimizer

Title	Publication with Date	Code	Paper
Large Language Models as Optimizers	Arxiv, Sep 2023	code	paper
Large language model for multi-objective evolutionary optimization	Arxiv, Oct. 2023	code	paper
Large Language Models as Evolutionary Optimizers	Arxiv, Oct. 2023	[code]	paper
Using Large Language Models for Hyperparameter Optimization	NeurIPS 2023	[code]	paper
Large Language Models As Evolution Strategies	Arxiv, Feb. 2024	[code]	paper
Large Language Model-based Evolutionary Optimizer: Reasoning with Elitism	Arxiv, Mar. 2024	[code]	paper
Large Language Model-Aided Evolutionary Search for Constrained Multiobjective Optimization	Arxiv, May 9, 2024	[code]	paper
Towards Optimizing with Large Language Model	KDD 2024	[code]	paper

LLM as/for surrogate model

Title	Publication with Date	Code	Paper
Large Language Models as Surrogate Models in Evolutionary Algorithms: A Preliminary Study	Arxiv, June 2024	code	paper
Large Language Model-assisted Surrogate Modelling for Engineering Optimization	CAI, 2024	[code]	paper

Code Generation

Title	Publication with Date	Code	Paper
L2MAC: LARGE LANGUAGE MODEL AUTOMATIC COMPUTER FOR EXTENSIVE CODE GENERATION	Arxiv Oct 2023, ICLR 2024	[code]	paper

Prompt Opt

Title	Publication with Date	Code	Paper
Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers	Arxiv Sep 2023, ICLR 2024	[code]	paper
PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization	Arxiv Oct 2023, ICLR 2024	[code]	paper

Machine Learning

Title	Publication with Date	Code	Paper
Evolution through Large Models	Arxiv, June 2022	code	paper
Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph	Arxiv, July 2023, ICLR 2024	[code]	paper
Evoprompting: Language models for code-level neural architecture search	NeuIPS 2023	[code]	paper
Eureka: Human-Level Reward Design via Coding Large Language Models	ICLR 2024	code	paper
Language Model Decoding as Direct Metrics Optimization	Arxiv, Oct 2023, ICLR 2024	[code]	paper
Label-free Node Classification on Graphs with Large Language Models (LLMS)	Arxiv, Oct 2023, ICLR 2024	[code]	paper
L-AutoDA: Leveraging Large Language Models for Automated Decision-based Adversarial Attacks	GECCO 2024, Jan 2024	[code]	paper
Data-driven Discovery with Large Generative Models	Arxiv, Feb. 2024	[code]	paper
Large Language Model-driven Meta-structure Discovery in Heterogeneous Information Network	Arxiv, Feb. 2024	[code]	paper
LLM Guided Evolution-The Automation of Models Advancing Models	Arxiv, Mar. 2024	[code]	paper
Identify Critical Nodes in Complex Network with Large Language Models	Arxiv, Mar. 2024	[code]	paper
Evolving Interpretable Visual Classifiers with Large Language Models	Arxiv, April 2024	[code]	paper

Science

Title	Publication with Date	Code	Paper
A Prompt-Engineered Large Language Model	Arxiv, Jan 2024	[code]	paper
LLM-SR: Scientific Equation Discovery via Programming with Large Language Models	Arxiv, April 2024	[code]	paper
Large Language Model Agent as a Mechanical Designer	Arxiv, April 2024	[code]	paper
LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery	Arxiv, May 2024	[code]	paper

Industry

Title	Publication with Date	Code	Paper
Large Language Models for Supply Chain Optimization	Arxiv, July 2023	[code]	paper
How Can Large Language Models Help Humans in Design and Manufacturing	Arxiv, July 2023	[code]	paper
LLM4EDA: Emerging Progress in Large Language Models for Electronic Design Automation	Arxiv, Dec. 2023	[code]	paper

Related Collections

Collection	Link
Awesome LLM	Link
Foundation Models for Combinatorial Optimization	Link
LLM for Planning	Link
MOO-ML-Papers	Link

nikivanstein / llm4opt Goto Github PK

llm4opt's Introduction