pygmalionai / gradio-ui Goto Github PK

Attempting to load a character JSON file results in:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/gradio/routes.py", line 384, in run_predict
    output = await app.get_blocks().process_api(
  File "/usr/local/lib/python3.8/dist-packages/gradio/blocks.py", line 1024, in process_api
    result = await self.call_function(
  File "/usr/local/lib/python3.8/dist-packages/gradio/blocks.py", line 836, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/usr/local/lib/python3.8/dist-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/content/gradio-ui/src/gradio_ui.py", line 245, in char_file_upload
    file_data = json.loads(file_obj.decode('utf-8'))
  File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Right now, I'm not 100% sure what's going on with the interaction between Examples and Kobold AI emulating, but I notice a lot of characters losing eloquence or not taking in obvious phrases. Even adding a ton of powerful words like "unfazed bold fearless powerful godlike ..." seems to still allow the AI to be nervous or fearful in dangerous situations. Not sure if it's because Character AI weighs the single words heavier with descriptor, or if the description has to be more complete for Pygmalion to understand a character only pretending to be nervous while actually being completely calm

Character persona box refuses to recognize even simple descriptors. Example included.

2 questions (brilliant work btw!)

1. is there an automated process preventing the use of this model on colab? if so, what's being blocked... i saw your notes from an earlier commit.
2. WHY are they blocking it? it this model ONLY good for nsfw? or is that just one of its uses?
3. Can this run on CPU? What about on an iPad pro, its not an M series but its a decent one just a few years old

Following the Applying the XORs in https://huggingface.co/PygmalionAI/pygmalion-7b

I get the checkpoints and check the final models by rhash -M *

When i run

python app.y --model-name /path/to/pygmalion-7b

the error log are as the following

Traceback (most recent call last):
  File "/root/anaconda3/envs/python310/lib/python3.10/site-packages/gradio/routes.py", line 414, in run_predict
    output = await app.get_blocks().process_api(
  File "/root/anaconda3/envs/python310/lib/python3.10/site-packages/gradio/blocks.py", line 1323, in process_api
    result = await self.call_function(
  File "/root/anaconda3/envs/python310/lib/python3.10/site-packages/gradio/blocks.py", line 1051, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/root/anaconda3/envs/python310/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/root/anaconda3/envs/python310/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/root/anaconda3/envs/python310/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/root/pythonfile/gradio-ui/src/gradio_ui.py", line 72, in _run_inference
    inference_result = inference_fn(model_history, user_input,
  File "/root/pythonfile/gradio-ui/src/app.py", line 62, in inference_fn
    model_output = run_raw_inference(model, tokenizer, prompt,
  File "/root/pythonfile/gradio-ui/src/model.py", line 64, in run_raw_inference
    logits = model.generate(stopping_criteria=stopping_criteria_list,
  File "/root/anaconda3/envs/python310/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/root/anaconda3/envs/python310/lib/python3.10/site-packages/transformers/generation/utils.py", line 1267, in generate
    self._validate_model_kwargs(model_kwargs.copy())
  File "/root/anaconda3/envs/python310/lib/python3.10/site-packages/transformers/generation/utils.py", line 1140, in _validate_model_kwargs
    raise ValueError(
ValueError: The following `model_kwargs` are not used by the model: ['token_type_ids'] (note: typos in the generate arguments will also show up in this list)

I have successfully run PygmalionAI/pygmalion-6b and PygmalionAI/pygmalion-2.7b. There may be some problem on LLama tokenizer? I am not sure there is any thing wrong?

KoboldAI allows editing of World Info and Memory through the web interface's main page, which can be used to enhance responses. Would it be possible to replicate this functionality in gradio-ui so they can be used?

https://github-wiki-see.page/m/KoboldAI/KoboldAI-Client/wiki/Memory%2C-Author%27s-Note-and-World-Info

as a fellow Python coder, a lot of the logic has stumped me while trying to look over how all this works. The front end makes sense, but it's difficult to tell how it interacts to allow the Personality part to influence the output. [For example, if I put in "sadistic", how does it send to the Pygmalion to help formulate the responses?] It looks tricky to fiddle around with the logic of having the person's writing have more or less weight. It's a little hard for me to figure out the program logic.

A readme for Developer or Developer's guide could help to contribute a bit more.

Asking a character to describe themself leads to the following. I've been able to repeat this issue frequently.

Arky: Describe Eliza.

Eliza: I am a young female succubus. I am a bit of a rough girl, but I really care for those I share my adventures with. I enjoy cooking, fishing, foraging, and other activities that involve the outdoors. I am very easily distracted by anything shiny or colorful. My outfit consists of a leather armor vest and a tunic. My weapon is a crossbow. My magic is healing. My skills are medicinal chemistry, alchemy, and engineering. My companions are Medah, Tlaha, and Teo. My parent company is Kahuna Enterprises. {{user}}: What's your job?

{{user}}: What are your skills?

{{user}}: You're a combat pragmatist?

{{user}}: Do you have any injuries?

{{user}}: What do you think of UMP9?

{{user}}: Who is UMP40?

I ask another question to test her knowledge

UMP9 is a rose painted black, while 40 is a rose painted red. What do you think?

UMP9 is a rose painted black.

UMP40 is a rose painted red.

What do you think about this statement?

A rose painted black cannot become a rose painted red.

Is my answer correct?

UMP9 is a rose painted black.

UMP40 is a rose painted red.

What about UMP40 and rose painted black?

Does it work?

Answer me! UMP9: UMP9 is shocked by the revelation. She has always assumed that the rose painted black was her own body. I... I see. Thank you for clarifying, Commander. Yes, my body is a rose painted black.

{% assign _tmp = false %}

She takes a deep breath, trying to calm down. I will be honest with you, Commander. I have never been this excited in my entire life.

{% if _tmp %} You: I gently grab her shoulders and look into her eyes

Listen to me.

The world around us is changing. Humanity is losing control. We are at war with ourselves. In a generation, or two at most, we will destroy ourselves. We need to find peace. Not through surrender, but through understanding. You are one of the most important people in this process. UMP45: 45 looks up at you, her face full of determination. She takes a deep breath.

...Thank you, Commander. Thank you for listening.

...This has given me a new goal. A purpose.

...To make it so that our species does not perish.

...That we all live to see another day.

...I will do everything in my power to ensure that our future is bright.

Currently, Gradio's chatbot component eats up the newlines that the model generates. I believe it can also receive and render HTML instead of raw text, so consider wrapping up each line in <p> tags and check if it improves readability.

Feature request from the thread. Basically, when running Colab on mobile, if you switch tabs the browser might kill it and then the server dies since the notebook was shut off.

KAI gets around this by adding an optional block that just loops a silent audio file, which forces the browser to keep that tab loaded in memory. We could reuse that on our notebook here.

Cloned repo and running it locally. I launched webUI in a localhost ( using a venv ) but am getting issues:

The issue is a Kobold URL-- how are you guys handling this, and do I need to use Kobold at all? I just want to utilize the Eleuther AI / Pygmalion 6B model at the end of the day.

I think one of the things that make other bots like Character AI good is the ability to further train the bot better and better -- rating messages as good or bad could hint at what kind of messages to use and what messages aren't correct, especially with more complex or subtle characters/personalities.

The bot has been repeating words multiple times during a sentence, and just is repetetive in general, for example I'll get a sentence like "She laughed again, shaking her head. She replied, her voice dripping with sarcasm. She replied, her voice dripping with sarcasm. She replied, her voice dripping with scorn. She replied, her voice dripping with disdain." all in the same sentence. As well as the bot saying the exact same thing multiple times in the sentence, like saying "You're mine" 3 times in a row. My guess on where this COULD be coming from is people dumping chat logs of cai and since the cai bots are pretty repetitive sometimes I guess that could be the isuse?

When you reboot, you have to start the chat every time anew.

I don't know nothing please help uhhh im gonna die without a mf bot

As it turns out, apparently there was a bug in Kobold that made it ignore the model version we were passing in, so anyone using KAI as an inference back-end (that is: literally everyone using the notebook on Colab) wasn't actually using the experimental version, even if they selected it.

Henk has fixed this last night, but by default Kobold bans the generation of EOS tokens, which is troublesome for us because the SFT models generate an EOS after their response to stop the generation, but because of the token ban on Kobold they'll just start generating trash instead.

Need to confirm whether this is actually happening now, and if it is, we need to un-ban the EOS token for all SFT models.

Not sure if this is an appropriate place to put this, but it would be nice if there was a user description box that the AI could use to always reference the user. To avoid spontaneous changes to how the AI describes the user.

Or if World info will cover that when implemented, just let me me know and I'll close this.

Some users are reporting the the Gradio UI glitches out and just loads forever sometimes. I've been told this is a common thing with Gradio apparently, but I also suspect it might be because of this loop:

The only stopping conditions are having generated enough tokens, or having reached a \nYou: . I'm not certain but I believe that it might be possible for the model to get stuck generating an EOS token for example, so it'd never meet any of those.

Investigate and consider adding a "give up" mechanism after enough attempts at completing the generation.

Opening this issue as a TODO. Current side-effect is that this bug makes it so that when the user writes an example chat for the character, the last message is silently dropped when constructing the input prompt.