ai4co / llm-as-hh Goto Github PK
View Code? Open in Web Editor NEWLarge Language Models as Hyper-Heuristics for Combinatorial Optimization (CO)
License: MIT License
Large Language Models as Hyper-Heuristics for Combinatorial Optimization (CO)
License: MIT License
Hello ReEvo team!
I've been trying to use ziphuAi instead of Gpt4 to run the prompts. I've provided the api key for ziphuAi, and I've done some changes in the config.yaml to enable that.
# LLM parameters
model: GLM-4 # LLM model (other options include gpt-4-turbo-preview, GLM-3-Turbo, GLM-4)
temperature: 1 # temperature for chat completion
suffix: GLM # suffix for generated files (indicates LLM model)
but when I run the code an exception happens due to seed function. Can you please help with that?
Here is the full error:
python main.py problem=tsp_constructive algorithm=reevo
[2024-04-08 13:05:18,818][root][INFO] - Workspace: C:\Users\kjkj0\LLM-as-HH\outputs\main\2024-04-08_13-05-18
[2024-04-08 13:05:18,819][root][INFO] - Project Root: C:\Users\kjkj0\LLM-as-HH
[2024-04-08 13:05:18,819][root][INFO] - Using LLM: GLM-4
[2024-04-08 13:05:18,819][root][INFO] - Using Algorithm: reevo
[2024-04-08 13:05:19,378][root][INFO] - Problem: tsp_constructive
[2024-04-08 13:05:19,378][root][INFO] - Problem description: Solving Traveling Salesman Problem (TSP) with constructive heuristics. TSP requires finding the shortest path that visits all given nodes and returns to the starting node.
[2024-04-08 13:05:19,378][root][INFO] - Function name: select_next_node
[2024-04-08 13:05:19,379][root][INFO] - Evaluating seed function...
[2024-04-08 13:05:19,389][root][INFO] - Seed function code:
import numpy as np
def select_next_node_v2(current_node: int, destination_node: int, unvisited_nodes: set, distance_matrix: np.ndarray) -> int:
"""Select the next node to visit from the unvisited nodes."""
threshold = 0.7
c1, c2, c3, c4 = 0.4, 0.3, 0.2, 0.1
scores = {}
for node in unvisited_nodes:
all_distances = [distance_matrix[node][i] for i in unvisited_nodes if i != node]
average_distance_to_unvisited = np.mean(all_distances)
std_dev_distance_to_unvisited = np.std(all_distances)
score = c1 * distance_matrix[current_node][node] - c2 * average_distance_to_unvisited + c3 * std_dev_distance_to_unvisited - c4 * distance_matrix[destination_node][node]
scores[node] = score
next_node = min(scores, key=scores.get)
return next_node
[2024-04-08 13:05:19,389][root][INFO] - Iteration 0: Running Code 0
[2024-04-08 13:05:19,553][root][INFO] - Iteration 0: Code Run 0 execution error!
[2024-04-08 13:05:19,572][root][INFO] - Iteration 0, response_id 0: Objective value: inf
Error executing job with overrides: ['problem=tsp_constructive', 'algorithm=reevo']
Traceback (most recent call last):
File "C:\Users\kjkj0\LLM-as-HH\main.py", line 52, in <module>
main()
File "C:\Users\kjkj0\LLM-as-HH\revo-venv-ziphu\Lib\site-packages\hydra\main.py", line 94, in decorated_main
_run_hydra(
File "C:\Users\kjkj0\LLM-as-HH\revo-venv-ziphu\Lib\site-packages\hydra\_internal\utils.py", line 394, in _run_hydra
_run_app(
File "C:\Users\kjkj0\LLM-as-HH\revo-venv-ziphu\Lib\site-packages\hydra\_internal\utils.py", line 457, in _run_app
run_and_report(
File "C:\Users\kjkj0\LLM-as-HH\revo-venv-ziphu\Lib\site-packages\hydra\_internal\utils.py", line 223, in run_and_report
raise ex
File "C:\Users\kjkj0\LLM-as-HH\revo-venv-ziphu\Lib\site-packages\hydra\_internal\utils.py", line 220, in run_and_report
return func()
^^^^^^
File "C:\Users\kjkj0\LLM-as-HH\revo-venv-ziphu\Lib\site-packages\hydra\_internal\utils.py", line 458, in <lambda>
lambda: hydra.run(
^^^^^^^^^^
File "C:\Users\kjkj0\LLM-as-HH\revo-venv-ziphu\Lib\site-packages\hydra\_internal\hydra.py", line 132, in run
_ = ret.return_value
^^^^^^^^^^^^^^^^
File "C:\Users\kjkj0\LLM-as-HH\revo-venv-ziphu\Lib\site-packages\hydra\core\utils.py", line 260, in return_value
raise self._return_value
File "C:\Users\kjkj0\LLM-as-HH\revo-venv-ziphu\Lib\site-packages\hydra\core\utils.py", line 186, in run_job
ret.return_value = task_function(task_cfg)
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\kjkj0\LLM-as-HH\main.py", line 31, in main
reevo = ga(cfg, ROOT_DIR)
^^^^^^^^^^^^^^^^^
File "C:\Users\kjkj0\LLM-as-HH\reevo.py", line 26, in __init__
self.init_population()
File "C:\Users\kjkj0\LLM-as-HH\reevo.py", line 98, in init_population
raise RuntimeError(f"Seed function is invalid. Please check the stdout file in {os.getcwd()}.")
RuntimeError: Seed function is invalid. Please check the stdout file in C:\Users\kjkj0\LLM-as-HH\outputs\main\2024-04-08_13-05-18.
1、想请问这个项目api_key具体如何传入,试过很多方式始终无法顺利运行
2、关于适配api问题,能否像eoh一样支持传入endpoint(baseurl),方便使用一些非openai官方的key
希望版主有空能够解答疑惑,十分感谢。
我运行的结果发现,seed_func对于结果的影响很大。reevo所生成的算法,很多程度上受到seed_func的影响,会偏向于生成相似的函数结构和变量,在探索能力上有所欠缺。不知道是否存在避免这种影响的设计,让框架尽可能多的探索
How to solve this problem?
[2024-05-27 11:03:36,897][root][INFO] - Using LLM: gpt-3.5-turbo
[2024-05-27 11:03:36,899][root][INFO] - Using Algorithm: reevo
[2024-05-27 11:03:38,178][root][INFO] - Problem: tsp_aco
[2024-05-27 11:03:38,178][root][INFO] - Problem description: Solving Traveling Salesman Problem (TSP) via stochastic solution sampling following "heuristics". TSP requires finding the shortest path that visits all given nodes and returns to the starting node.
[2024-05-27 11:03:38,178][root][INFO] - Function name: heuristics
[2024-05-27 11:03:38,191][root][INFO] - Evaluating seed function...
[2024-05-27 11:03:38,191][root][INFO] - Seed function code:
import numpy as np
def heuristics_v2(distance_matrix: np.ndarray) -> np.ndarray:
return 1 / distance_matrix
[2024-05-27 11:03:38,191][root][INFO] - Iteration 0: Running Code 0
[2024-05-27 11:03:40,944][root][INFO] - Iteration 0: Code Run 0 successful!
[2024-05-27 11:03:49,394][root][INFO] - Iteration 0, response_id 0: Objective value: 6.737054565194424
[2024-05-27 11:03:49,394][root][INFO] - Iteration 0: Elitist: 6.737054565194424
[2024-05-27 11:03:49,394][root][INFO] - Iteration 0 finished...
[2024-05-27 11:03:49,396][root][INFO] - Best obj: 6.737054565194424, Best Code Path: problem_iter0_code0.py
[2024-05-27 11:03:49,396][root][INFO] - Function Evals: 1
[2024-05-27 11:03:49,396][root][INFO] - Initial Population Prompt:
System Prompt:
You are an expert in the domain of optimization heuristics. Your task is to design heuristics that can effectively solve optimization problems.
Your response outputs Python code and nothing else. Format your code as a Python code string: "python ...
".
User Prompt:
Write a heuristics function for Solving Traveling Salesman Problem (TSP) via stochastic solution sampling following "heuristics". TSP requires finding the shortest path that visits all given nodes and returns to the starting node.
The heuristics
function takes as input a distance matrix, and returns prior indicators of how promising it is to include each edge in a solution. The return is of the same shape as the input.
def heuristics_v1(distance_matrix: np.ndarray) -> np.ndarray:
return 1 / distance_matrix
Refer to the format of a trivial design above. Be very creative and give heuristics_v2
. Output code only and enclose your code with Python code block: python ...
.
When running ReEvo with the command python main.py problem=tsp_aco
, an error occurs during the evaluation of the seed function, leading to a RuntimeError
. The error message suggests that the seed function is invalid, but it doesn't provide specific details about what caused the invalid evaluation.
python main.py problem=tsp_aco
.RuntimeError
related to the invalid seed function evaluation.[2024-05-16 12:38:34,259][root][INFO] - Workspace: /remote-home/iot_liuguangyi/llm/LLM-as-HH/outputs/main/2024-05-16_12-38-34
[2024-05-16 12:38:34,259][root][INFO] - Project Root: /remote-home/iot_liuguangyi/llm/LLM-as-HH
[2024-05-16 12:38:34,259][root][INFO] - Using LLM: gpt-3.5-turbo
[2024-05-16 12:38:34,259][root][INFO] - Using Algorithm: reevo
[2024-05-16 12:38:34,668][root][INFO] - Problem: tsp_aco
[2024-05-16 12:38:34,668][root][INFO] - Problem description: Solving Traveling Salesman Problem (TSP) via stochastic solution sampling following "heuristics". TSP requires finding the shortest path that visits all given nodes and returns to the starting node.
[2024-05-16 12:38:34,668][root][INFO] - Function name: heuristics
[2024-05-16 12:38:34,668][root][INFO] - Evaluating seed function...
[2024-05-16 12:38:34,669][root][INFO] - Seed function code:
import numpy as np
def heuristics_v2(distance_matrix: np.ndarray) -> np.ndarray:
return 1 / distance_matrix
[2024-05-16 12:38:34,669][root][INFO] - Iteration 0: Running Code 0
[2024-05-16 12:38:36,349][root][INFO] - Iteration 0: Code Run 0 successful!
[2024-05-16 12:38:56,351][root][INFO] - Error for response_id 0: Command '['python', '-u', '/remote-home/iot_liuguangyi/llm/LLM-as-HH/problems/tsp_aco/eval.py', '50', '/remote-home/iot_liuguangyi/llm/LLM-as-HH', 'train']' timed out after 19.99986495007761 seconds
Please let us know if you need further information or assistance in resolving this issue. Thank you!
[2024-07-22 19:57:06,333][root][INFO] - Workspace: D:\deeplearning\LLM-as-HH\outputs\main\2024-07-22_19-57-06
[2024-07-22 19:57:06,333][root][INFO] - Project Root: D:\deeplearning\LLM-as-HH
[2024-07-22 19:57:06,333][root][INFO] - Using LLM: llama3-70b
[2024-07-22 19:57:06,333][root][INFO] - Using Algorithm: reevo
[2024-07-22 19:57:07,393][root][INFO] - Problem: tsp_aco
[2024-07-22 19:57:07,393][root][INFO] - Problem description: Solving Traveling Salesman Problem (TSP) via stochastic solution sampling following "heuristics". TSP requires finding the shortest path that visits all given nodes and returns to the starting node.
[2024-07-22 19:57:07,393][root][INFO] - Function name: heuristics
[2024-07-22 19:57:07,394][root][INFO] - Evaluating seed function...
[2024-07-22 19:57:07,395][root][INFO] - Seed function code:
import numpy as np
def heuristics_v2(distance_matrix: np.ndarray) -> np.ndarray:
return 1 / distance_matrix
[2024-07-22 19:57:07,395][root][INFO] - Iteration 0: Running Code 0
[2024-07-22 19:57:09,359][root][INFO] - Iteration 0: Code Run 0 successful!
[2024-07-22 19:57:16,735][root][INFO] - Iteration 0, response_id 0: Objective value: 6.469335357675092
[2024-07-22 19:57:16,735][root][INFO] - Iteration 0: Elitist: 6.469335357675092
[2024-07-22 19:57:16,735][root][INFO] - Iteration 0 finished...
[2024-07-22 19:57:16,735][root][INFO] - Best obj: 6.469335357675092, Best Code Path: problem_iter0_code0.py
[2024-07-22 19:57:16,735][root][INFO] - Function Evals: 1
[2024-07-22 19:57:16,735][root][INFO] - Initial Population Prompt:
System Prompt:
You are an expert in the domain of optimization heuristics. Your task is to design heuristics that can effectively solve optimization problems.
Your response outputs Python code and nothing else. Format your code as a Python code string: "python ...
".
User Prompt:
Write a heuristics function for Solving Traveling Salesman Problem (TSP) via stochastic solution sampling following "heuristics". TSP requires finding the shortest path that visits all given nodes and returns to the starting node.
The heuristics
function takes as input a distance matrix, and returns prior indicators of how promising it is to include each edge in a solution. The return is of the same shape as the input.
def heuristics_v1(distance_matrix: np.ndarray) -> np.ndarray:
return 1 / distance_matrix
Refer to the format of a trivial design above. Be very creative and give heuristics_v2
. Output code only and enclose your code with Python code block: python ...
.
How to solve the problem?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.