The banana-python-sdk from bananaml

banana-python-sdk's People

Contributors

Stargazers

Watchers

banana-python-sdk's Issues

Loosen dependency versions?

I'm running into a conflict with banana-dev depending on requests==2.26.0. I need requests>=2.27.0 and was wondering if the was a reason for such a narrow constraint.

Exception: inference server error: taskID does not exist: task_619077b8-05b3-4dfc-a0eb-a02c6a9411d0

Problem:

After I got the model key, I tried and waited for a long time. I got this error "Exception: inference server error: taskID does not exist: task_619077b8-05b3-4dfc-a0eb-a02c6a9411d0". How do solve that problem?

My Code:

import banana_dev as banana
import time

api_key = "api_key" # "YOUR_API_KEY"
model_key = "model_key" # "YOUR_MODEL_KEY"
model_inputs = {'prompt': 'Hello I am a [MASK] model.'}

startTime = time.time()
out = banana.run(api_key, model_key, model_inputs)
print(out)
endTime =  time.time()
print("Time: ", endTime - startTime)

Code app.py :

from transformers import pipeline
import torch

# Init is ran on server startup
# Load your model to GPU as a global variable here using the variable name "model"
def init():
    global model
    
    device = 0 if torch.cuda.is_available() else -1
    model = pipeline('fill-mask', model='bert-base-uncased', device=device)

# Inference is ran for every server call
# Reference your preloaded global model variable here.
def inference(model_inputs:dict) -> dict:
    global model

    # Parse out your arguments
    prompt = model_inputs.get('prompt', None)
    if prompt == None:
        return {'message': "No prompt provided"}
    
    # Run the model
    result = model(prompt)

    # Return the results as a dictionary
    return result

Code download.py :

# In this file, we define download_model
# It runs during container build time to get model weights built into the container

# In this example: A Huggingface BERT model

from transformers import pipeline
import torch

def download_model():
    # do a dry run of loading the huggingface model, which will download weights
    pipeline('fill-mask', model='bert-base-uncased')

if __name__ == "__main__":
    download_model()