llama2 如何接入 about lagent HOT 4 CLOSED

See examples here https://github.com/InternLM/lagent#run-a-react-agent-with-internlm.
You can simply choose to use llama2 by replacing the model name of internlm to llama2 through Huggingface Transformer.

from lagent.

See examples here https://github.com/InternLM/lagent#run-a-react-agent-with-internlm. You can simply choose to use llama2 by replacing the model name of internlm to llama2 through Huggingface Transformer.

我用Llama-2-7b-chat-hf 模型，报如下错误
Cell In[5], line 16
8 python_interpreter = PythonInterpreter()
10 chatbot = ReAct(
11 llm=llm,
12 action_executor=ActionExecutor(
13 actions=[search_tool, python_interpreter]),
14 )
---> 16 response = chatbot.chat('若$z=-1+\sqrt{3}i$,则$\frac{z}{{z\overline{z}-1}}=\left(\ \ \right)$')
17 print(response.response)

File ~/lagent/lagent/agents/react.py:222, in ReAct.chat(self, message)
216 for turn in range(self.max_turn):
217 prompt = self._protocol.format(
218 chat_history=self.session_history,
219 inner_step=self._inner_history,
220 action_executor=self._action_executor,
221 force_stop=force_stop)
--> 222 response = self._llm.generate_from_template(prompt, 512)
223 self._inner_history.append(
224 dict(role='assistant', content=response))
225 thought, action, action_input = self._protocol.parse(
226 response, self._action_executor)

File ~/lagent/lagent/llms/huggingface.py:128, in HFTransformer.generate_from_template(self, templates, max_out_len, **kwargs)
121 """Generate completion from a list of templates.
122
123 Args:
124 templates (List[PromptType]): A list of templates.
125 max_out_len (int): The maximum length of the output.
126 """
127 inputs = self.parse_template(templates)
--> 128 response = self.generate(inputs, max_out_len=max_out_len, **kwargs)
129 return response.replace(
130 self.template_parser.roles['assistant']['end'].strip(),
131 '').strip()

File ~/lagent/lagent/llms/huggingface.py:105, in HFTransformer.generate(self, inputs, max_out_len, **kwargs)
101 input_ids = self.tokenizer(
102 inputs, truncation=True,
103 max_length=self.max_seq_len - max_out_len)['input_ids']
104 input_ids = torch.tensor(input_ids, device=self.model.device)
--> 105 outputs = self.model.generate(
106 input_ids, max_new_tokens=max_out_len, **kwargs)
108 if not self.extract_pred_after_decode:
109 outputs = outputs[:, input_ids.shape[1]:]

File ~/anaconda3/envs/langchain/lib/python3.11/site-packages/torch/utils/_contextlib.py:115, in context_decorator..decorate_context(*args, **kwargs)
112 @functools.wraps(func)
113 def decorate_context(*args, **kwargs):
114 with ctx_factory():
--> 115 return func(*args, **kwargs)

File ~/anaconda3/envs/langchain/lib/python3.11/site-packages/transformers/generation/utils.py:1261, in GenerationMixin.generate(self, inputs, generation_config, logits_processor, stopping_criteria, prefix_allowed_tokens_fn, synced_gpus, assistant_model, streamer, **kwargs)
1258 synced_gpus = False
1260 # 1. Handle generation_config and kwargs that might update it, and validate the .generate() call
-> 1261 self._validate_model_class()
1263 # priority: generation_config argument > model.generation_config (the default generation config)
1264 if generation_config is None:
1265 # legacy: users may modify the model configuration to control generation -- update the generation config
1266 # model attribute accordingly, if it was created from the model config

File ~/anaconda3/envs/langchain/lib/python3.11/site-packages/transformers/generation/utils.py:1135, in GenerationMixin._validate_model_class(self)
1133 if generate_compatible_classes:
1134 exception_message += f" Please use one of the following classes instead: {generate_compatible_classes}"
-> 1135 raise TypeError(exception_message)

TypeError: The current model class (LlamaModel) is not compatible with .generate(), as it doesn't have a language model head. Please use one of the following classes instead: {'LlamaForCausalLM'}

from lagent.

As suggested in the error log, you should change the LLaMaModel to LlamaForCauslLM. For Lagent, you need to change the LLM from HFTransformer to HFTransformerCausalLM.

from lagent.

As suggested in the error log, you should change the LLaMaModel to LlamaForCauslLM. For Lagent, you need to change the LLM from HFTransformer to HFTransformerCausalLM.
agent 代理的一点都不准

from lagent.