Getting 'undefined' response in nodejs implementation of the sdk about ai HOT 6 CLOSED

I'm trying to debug an issue with the example you've shared which is documented. At first my implementation was working but after a few interactions I am no longer getting results.

My investigation brought me to this issue in which I'd thought the source was with the API reponse, however I have been unable to debug where the problem occurs.

First thought was perhaps there is an issue with the streams not being closed which is exacerbated when NextJS restarts the dev server on code change. I'm currently unable to confirm this and as there are so many moving parts, I fear this may take some time to debug.

In the meantime, I have fallen back to using the OpenAI integration which has been easier to setup.

Absolutely love the project and the work that has gone into streamlining the integrations ❤️ I'm excited to see where this project goes.

from ai.

Thank you! Feel free to open an issue or discussion whenever you're running into a problem again. I'll close this for now as there's nothing to track at the moment.

from ai.

Can you check what response is? There might be some error message attached.

from ai.

Running the following:

const h = require('@huggingface/inference');

const Hf = new h.HfInference(process.env.HUGGINGFACE_API_KEY);

// Request the HuggingFace API for the response
async function start() {
  const response = Hf.textGenerationStream({
    model: 'oasst-sft-4-pythia-12b-epoch-3.5',
    inputs: 'Hello',
    parameters: {
      max_new_tokens: 200,
      typical_p: 0.2,
      repetition_penalty: 1,
      truncate: 1000,
    },
  });

  console.log('Response: ', response);
  const completion = response.generated_text;
  console.log('Completion: ', completion);
}

start();

Outputs:

Response:  Object [AsyncGenerator] {}
Completion:  undefined

from ai.

As you can see, response.generated_text is not defined. Where did you find the code of response.generated_text in the docs?

We have an example here here that shows how the SDK should be used:

from ai.

@karlbateman did you find what was the issue? I encounter the same situation in which it just stops working and I can't error handle it. Code change hot reloading came to my mind too but can't confirm it

from ai.