way to easily stop/reload about gradio HOT 15 CLOSED

Good point. This happens because gradio spawns a non-daemon child thread to run the server, which blocks the main thread from exiting until the server ends, as documented here: https://stackoverflow.com/questions/50486083/ending-non-daemon-threads-when-shutting-down-an-interactive-python-session

We could fix this by taking the approach mentioned in the Stack Overflow question. If you want to make the change, please go ahead and make a PR. The threading happens here:

from gradio.

Interesting, I'm trying to understand ThreadingMixIn. Isn't the purpose of the class seems to be to allow the main thread to terminate once the children (server threads) have terminated? But we want the reverse. Maybe I'm misunderstanding, but I'm happy to test out the code to see if it works on my end!

from gradio.

Hi Russ, any luck with this? Let us know if you have something we can integrate

from gradio.

@abidlabs ok, I forked and started playing around with it. I should be able to get something back to you guys Thurs or Fri evening for review.

What would you think about a threading mixin, like in https://stackoverflow.com/questions/14088294/multithreaded-web-server-in-python ? (minus the multi-threading implementation)

class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
    """Handle requests in a separate thread."""

from gradio.

It just looked like it might be easier, but I could also be wrong. I don't usually enter python threading land if I can avoid it 👻 .

I had to work a little late tonight, but I starting making a set of tests 🧪 we could run through together at https://github.com/rcox771/threading_tests . So far, I've tested with and without a thread running the httpd, and tomorrow night I'll try the mixin 🥣 idea. I might still be understanding you basskwargs, so I did this to bridge some distance.

-Russ

from gradio.

hey @abidlabs I did. Sorry got absorbed in work and forgot to get back to you. I'll shoot you something this weekend

from gradio.

from gradio.

@abidlabs so the way I got things to work, step-by-step, was:

1. instantiating a custom HTTPServer within a context manager,
2. which then passes itself into a custom thread,
3. this thread then runs the server within its own context manager and listens for a KeyboardInterrupt and passes the signal down to the server

When integrating my tests into networking.py, I realized that I'd probably need at least one more hoisting to get the whole context-threaded-spaghetti up into interface.py where networking.start_simple_server is called. I could be completely off-base here with my hoisting ... 😅. I tried your suggestion, tried the mixin, and the only other thing I could think of was maybe having the server send a tailor-made request to some endpoint like GET /kill to circumvent this whole process.

I'll pause here and wait for feedback before I press on.

Here's the custom HTTPServer, thread on top of the server (TopThread), and context management (inside serve_thread_over_httpserver) for you to look at:

from http.server import HTTPServer as BaseHTTPServer, SimpleHTTPRequestHandler
import threading
import sys
import time


class HTTPHandler(SimpleHTTPRequestHandler):
    """This handler uses server.base_path instead of always
    using os.getcwd()"""


class HTTPServer(BaseHTTPServer):
    """The main server, you pass in base_path which is the
    path you want to serve requests from"""

    def __init__(self, base_path, server_address,
                 RequestHandlerClass=HTTPHandler):
        self.base_path = base_path
        self.host, self.port = server_address
        BaseHTTPServer.__init__(self, server_address, RequestHandlerClass)
        # self.keep_going = True

    def run(self):
        print(f'serving {self.base_path} on {self.host}:{self.port}...')
        try:
            sys.stdout.flush()
            self.serve_forever()
        except (KeyboardInterrupt, OSError):
            print('>>closing HTTPServer...')
            # self.keep_going = False
            pass
        finally:
            print('>>server_close')
            self.server_close()

        print('>>leaving run')


class TopThread(threading.Thread):
    def __init__(self, *args, server='blah', **kwargs):
        super().__init__(*args, **kwargs)
        self.keep_going = True
        self.server = server

    def run(self):
        print(self.server)
        self.server.run()
        while self.keep_going:
            try:
                time.sleep(.21)
            except KeyboardInterrupt:
                print('caught keyboard interrupt at top level thread')
                self.server.close()
                print('>>server close')
                self.keep_going = False
            finally:
                print('>top level thread close')
                return
        print('>top level thread closed.')


def serve_thread_over_httpserver(directory_to_serve='data',
                                 server_name='localhost', port=9001):
    with HTTPServer(directory_to_serve, (server_name, port)) as server:
        server_thread = TopThread(server=server, target=server.run)
        server_thread.start()
        while server_thread.keep_going:
            time.sleep(.2)
        server.shutdown()
        server_thread.join()


if __name__ == "__main__":
    serve_thread_over_httpserver()  # <- this works

from gradio.

Hi @rcox771 thanks I’ll take a look at it! Just to get a sense of things, does this solve the issue? Or are you still working on it? Wasn’t sure by what you meant by hoisting

from gradio.

@abidlabs If I catch and stop things at my level, I have no way to propagate that all the way up to stop your other threads elsewhere. This is my current understanding, with the main thread starting other things like your ngrok service. Green is as far as I've gotten in networking.py. If I continue down my current course, I'd probably either need to start having each thread self-managed through a pubsub, or refactor context management top-down from interfaces.py? Not sure really.

from gradio.

maybe we could flatten the hierarchy for everything that uses a thread, and given it one common launch point? A threadpool for example?

from gradio.

I'm working with your code now @rcox771. Just to clarify, the only non-daemon thread is the httpserver. (You can verify this by setting daemon=True in that thread, and then the program is terminated automatically when the main thread ends). So we shouldn't have to worry about anything else, e.g. the tunneling

from gradio.

I just pushed a fix with the the latest version of gradio (1.1.0). Can you see if this works for you?

from gradio.

@abidlabs works great! Yeah I guess I was confused about how the daemon was working behind the scenes in your tunneling.py

Thanks for the lesson and threading experience. I'll close this out and look for other ways I can help pack you back!

🤜
-Russ

from gradio.

Glad to hear! Thanks for helping :)

from gradio.