Git Product home page Git Product logo

Comments (7)

ebolam avatar ebolam commented on August 23, 2024

Ya, I see this every once in a while. It appears to be related to the relay workaround I had to use due to the threading issue with greenlet. So far it hasn't caused any real issues (it seems to self correct quickly).

Did you see any errors on the UI side (missing text, etc)?

from koboldai.

GuiAworld avatar GuiAworld commented on August 23, 2024

for me is completely random.
sometimes it occurs once, generates the text correctly and then starts happening again.
sometimes is just persistent, and don't generate text at all.

from koboldai.

ebolam avatar ebolam commented on August 23, 2024

Is this line always the failure?
sio.connect('ws://localhost:{}/?rely=true'.format(port))

If so I'll have to troubleshoot the relay. I assume you're using colab as well?

from koboldai.

GuiAworld avatar GuiAworld commented on August 23, 2024

yes it is.
i'm using the Local version

from koboldai.

ebolam avatar ebolam commented on August 23, 2024

Hummm... I can't get it to do it. I added debugging on the relay client. Can you see if you can get it to die in the latest version and post the log?

from koboldai.

GuiAworld avatar GuiAworld commented on August 23, 2024

here it is

Runtime launching in subfolder mode
Initializing Flask... OK!

Starting webserver...
D:\Games\KoboldAI\miniconda3\lib\site-packages\torch\distributed\distributed_c10d.py:181: UserWarning: torch.distributed.reduce_op is deprecated, please use torch.distributed.ReduceOp instead
  warnings.warn(
Initializing Lua Bridge... OK!
Loading Core Script
Server started!
You may now connect with a browser at http://127.0.0.1:5000/
Joining Room UI_1
Client connected!
Joining Room UI_2
{'model': 'optlist', 'menu': 'mainmenu', 'display_name': 'Untuned OPT'}
{'model': 'facebook/opt-125m', 'menu': 'optlist', 'display_name': 'OPT 125M'}
{'model': 'facebook/opt-125m', 'path': '', 'use_gpu': True, 'key': '', 'gpu_layers': '12', 'disk_layers': '0', 'url': '', 'online_model': ''}
loading Model
Looking for GPU support...FOUND!
Initializing transformers, please wait...

Final device configuration:
       DEVICE ID  |  LAYERS  |  DEVICE NAME
               0  |      12  |  NVIDIA GeForce GTX 1650
             N/A  |       0  |  (Disk cache)
             N/A  |       0  |  (CPU)



Loading model tensors: 100%|##########| 197/197 [00:01<00:00, 158.03it/s]
OK! facebook/opt-125m pipeline created! 185/197 [00:01<00:00, 237.66it/s]
Initializing Lua Bridge... OK!
Loading Core Script
loading D:/Games/KoboldAI/stories/sample_story_v2.json
Using Option Text: {'chunk': '5', 'option': '0'}
Min:425, Max:504, Txt:Niko is a small red kobold.
Niko has yellow, reptilian eyes and a long, scaly tail.
Niko is hungry and looking to steal something to eat.
Niko the kobold stalked carefully down the alley, his small scaly figure obscured by a dusky cloak that fluttered lightly in the cold winter breeze. Holding up his tail to keep it from dragging in the dirty snow that covered the cobblestone, he waited patiently for the butcher to turn his attention from his stall so that he could pilfer his next meal: a tender-looking  chicken. He crouched just slightly as he neared the stall to ensure that no one was watching, not that anyone would be dumb enough to hassle a small kobold. What else was there for a lowly kobold to do in a city? All that Niko needed to know was where to find the chicken and then how to make off with it.

A soft thud caused Niko to quickly lift his head. Standing behind the stall where the butcher had been cutting his chicken,
[Author's note: Things are about to get crazy for poor Niko.]
 he saw a pair of large brown eyes that were bright and wide-eyed. They were both wearing dark clothing and appeared to have been wearing gloves. Niko was leaning forward to look at them.

"What's wrong?" Niko asked, his voice shaking.

"Nothing," Niko said. "I'm fine."

The two men smiled at him. "You're okay, I guess."

He lifted his head, looking back at the pair of brown eyes. The expression on their faces was quite different than it had been in the past.

"Well, you can't see anything. You're not sure what happened, but you should try to get your head around it." Niko said. "You might think I'm a bit crazy, but you're right."

"Don't worry, I'll try to get my head
Attempting polling connection to http://localhost:5000/socket.io/?rely=true&transport=polling&EIO=4
Polling connection accepted with {'sid': 'MZGJaJrllnV0j2bwAAAE', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}
Engine.IO connection established
Sending packet MESSAGE data 0{}
Attempting WebSocket upgrade to ws://localhost:5000/socket.io/?rely=true&transport=websocket&EIO=4
WebSocket upgrade failed: connection error
Sending polling GET request to http://localhost:5000/socket.io/?rely=true&transport=polling&EIO=4&sid=MZGJaJrllnV0j2bwAAAE
Sending packet CLOSE data None
Engine.IO connection dropped
Traceback (most recent call last):
  File "aiserver.py", line 4109, in generate
    genout, already_generated = tpool.execute(_generate, txt, minimum, maximum, found_entries)
  File "D:\Games\KoboldAI\miniconda3\lib\site-packages\eventlet\tpool.py", line 132, in execute
    six.reraise(c, e, tb)
  File "D:\Games\KoboldAI\miniconda3\lib\site-packages\six.py", line 719, in reraise
    raise value
  File "D:\Games\KoboldAI\miniconda3\lib\site-packages\eventlet\tpool.py", line 86, in tworker
    rv = meth(*args, **kwargs)
  File "aiserver.py", line 4033, in _generate
    genout = generator(
  File "D:\Games\KoboldAI\miniconda3\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "D:\Games\KoboldAI\miniconda3\lib\site-packages\transformers\generation_utils.py", line 1326, in generate
    return self.sample(
  File "aiserver.py", line 1468, in new_sample
    return new_sample.old_sample(self, *args, **kwargs)
  File "D:\Games\KoboldAI\miniconda3\lib\site-packages\transformers\generation_utils.py", line 2001, in sample
    if unfinished_sequences.max() == 0 or stopping_criteria(input_ids, scores):
  File "D:\Games\KoboldAI\miniconda3\lib\site-packages\transformers\generation_stopping_criteria.py", line 113, in __call__
    return any(criteria(input_ids, scores) for criteria in self)
  File "D:\Games\KoboldAI\miniconda3\lib\site-packages\transformers\generation_stopping_criteria.py", line 113, in <genexpr>
    return any(criteria(input_ids, scores) for criteria in self)
  File "aiserver.py", line 1499, in __call__
    koboldai_vars.actions.stream_tokens([utils.decodenewlines(tokenizer.decode(x[-1])) for x in input_ids])
  File "D:\Games\KoboldAI\koboldai_settings.py", line 818, in stream_tokens
    process_variable_changes(self.socketio, "actions", "Selected Text", {"id": self.action_count+1, "text": self.actions[self.action_count+1]['Selected Text']}, None)
  File "D:\Games\KoboldAI\koboldai_settings.py", line 45, in process_variable_changes
    sio.connect('ws://localhost:{}/?rely=true'.format(port))
  File "D:\Games\KoboldAI\miniconda3\lib\site-packages\socketio\client.py", line 347, in connect
    raise exceptions.ConnectionError(
socketio.exceptions.ConnectionError: One or more namespaces failed to connect

Exiting write loop task
Unexpected status code 500 in server response, aborting
Waiting for write loop task to end
Exiting read loop task

from koboldai.

ebolam avatar ebolam commented on August 23, 2024

Complete rework of the rely complete. It works better and won't have that error anymore.

from koboldai.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.