declare-lab / flacuna Goto Github PK

Anthropic changed their python sdk - making this code line outdated.

Would love to know if this might help - https://github.com/BerriAI/litellm

~Simple I/O library, that standardizes all the llm api calls to the OpenAI call

from litellm import completion

## set ENV variables
# ENV variables can be set in .env file, too. Example in .env.example
os.environ["OPENAI_API_KEY"] = "openai key"
os.environ["ANTHROPIC_API_KEY"] = "anthropic key"

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

# anthropic call
response = completion("claude-v-2", messages)

Hi,

I am trying to run the model on Sagemaker, but I am getting the following warning:

"Some weights of the model checkpoint at declare-lab/flacuna-13b-v1.0 were not used when initializing LlamaForCausalLM"

I tried two ways, but both of them gave same issue.(hf and github source code)
How can I solve this? Can you help me please?
Thanks

import torch
from transformers import LlamaForCausalLM, LlamaTokenizer
base_model = "declare-lab/flacuna-13b-v1.0"
tokenizer = LlamaTokenizer.from_pretrained(base_model)
model = LlamaForCausalLM.from_pretrained(base_model,torch_dtype=torch.float16, device_map="auto")

from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("declare-lab/flacuna-13b-v1.0")
model = AutoModelForCausalLM.from_pretrained("declare-lab/flacuna-13b-v1.0",torch_dtype=torch.float16, device_map="auto")

Hi, I am running on my local machine with 1 A100 and A6000.
I am facing the following problem:

[Errno 1] Operation not permitted: '.../python3.8/site-packages/pandas/core/groupby/numba_.py'

I think this problem happens due to the conflict of the packages, and I can not identify the correct version.
Can I request a requirement.txt file for the correct version of the packages?

Thank you for your support.

Thanks for your great work. Could you share or help explain the part of how you evaluate on these datasets?