Comments (15)
Good point. This happens because gradio
spawns a non-daemon child thread to run the server, which blocks the main thread from exiting until the server ends, as documented here: https://stackoverflow.com/questions/50486083/ending-non-daemon-threads-when-shutting-down-an-interactive-python-session
We could fix this by taking the approach mentioned in the Stack Overflow question. If you want to make the change, please go ahead and make a PR. The threading happens here:
Line 224 in bdbdc3d
from gradio.
Interesting, I'm trying to understand ThreadingMixIn
. Isn't the purpose of the class seems to be to allow the main thread to terminate once the children (server threads) have terminated? But we want the reverse. Maybe I'm misunderstanding, but I'm happy to test out the code to see if it works on my end!
from gradio.
Hi Russ, any luck with this? Let us know if you have something we can integrate
from gradio.
@abidlabs ok, I forked and started playing around with it. I should be able to get something back to you guys Thurs or Fri evening for review.
What would you think about a threading mixin, like in https://stackoverflow.com/questions/14088294/multithreaded-web-server-in-python ? (minus the multi-threading implementation)
class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
"""Handle requests in a separate thread."""
from gradio.
It just looked like it might be easier, but I could also be wrong. I don't usually enter python threading land if I can avoid it ๐ป .
I had to work a little late tonight, but I starting making a set of tests ๐งช we could run through together at https://github.com/rcox771/threading_tests . So far, I've tested with and without a thread running the httpd, and tomorrow night I'll try the mixin ๐ฅฃ idea. I might still be understanding you basskwargs, so I did this to bridge some distance.
-Russ
from gradio.
hey @abidlabs I did. Sorry got absorbed in work and forgot to get back to you. I'll shoot you something this weekend
from gradio.
from gradio.
@abidlabs so the way I got things to work, step-by-step, was:
- instantiating a custom
HTTPServer
within a context manager, - which then passes itself into a custom thread,
- this thread then runs the server within its own context manager and listens for a
KeyboardInterrupt
and passes the signal down to the server
When integrating my tests into networking.py, I realized that I'd probably need at least one more hoisting to get the whole context-threaded-spaghetti up into interface.py where networking.start_simple_server
is called. I could be completely off-base here with my hoisting ... ๐
. I tried your suggestion, tried the mixin, and the only other thing I could think of was maybe having the server send a tailor-made request to some endpoint like GET /kill to circumvent this whole process.
I'll pause here and wait for feedback before I press on.
Here's the custom HTTPServer
, thread on top of the server (TopThread
), and context management (inside serve_thread_over_httpserver
) for you to look at:
from http.server import HTTPServer as BaseHTTPServer, SimpleHTTPRequestHandler
import threading
import sys
import time
class HTTPHandler(SimpleHTTPRequestHandler):
"""This handler uses server.base_path instead of always
using os.getcwd()"""
class HTTPServer(BaseHTTPServer):
"""The main server, you pass in base_path which is the
path you want to serve requests from"""
def __init__(self, base_path, server_address,
RequestHandlerClass=HTTPHandler):
self.base_path = base_path
self.host, self.port = server_address
BaseHTTPServer.__init__(self, server_address, RequestHandlerClass)
# self.keep_going = True
def run(self):
print(f'serving {self.base_path} on {self.host}:{self.port}...')
try:
sys.stdout.flush()
self.serve_forever()
except (KeyboardInterrupt, OSError):
print('>>closing HTTPServer...')
# self.keep_going = False
pass
finally:
print('>>server_close')
self.server_close()
print('>>leaving run')
class TopThread(threading.Thread):
def __init__(self, *args, server='blah', **kwargs):
super().__init__(*args, **kwargs)
self.keep_going = True
self.server = server
def run(self):
print(self.server)
self.server.run()
while self.keep_going:
try:
time.sleep(.21)
except KeyboardInterrupt:
print('caught keyboard interrupt at top level thread')
self.server.close()
print('>>server close')
self.keep_going = False
finally:
print('>top level thread close')
return
print('>top level thread closed.')
def serve_thread_over_httpserver(directory_to_serve='data',
server_name='localhost', port=9001):
with HTTPServer(directory_to_serve, (server_name, port)) as server:
server_thread = TopThread(server=server, target=server.run)
server_thread.start()
while server_thread.keep_going:
time.sleep(.2)
server.shutdown()
server_thread.join()
if __name__ == "__main__":
serve_thread_over_httpserver() # <- this works
from gradio.
Hi @rcox771 thanks Iโll take a look at it! Just to get a sense of things, does this solve the issue? Or are you still working on it? Wasnโt sure by what you meant by hoisting
from gradio.
@abidlabs If I catch and stop things at my level, I have no way to propagate that all the way up to stop your other threads elsewhere. This is my current understanding, with the main thread starting other things like your ngrok service. Green is as far as I've gotten in networking.py. If I continue down my current course, I'd probably either need to start having each thread self-managed through a pubsub, or refactor context management top-down from interfaces.py? Not sure really.
from gradio.
maybe we could flatten the hierarchy for everything that uses a thread, and given it one common launch point? A threadpool for example?
from gradio.
I'm working with your code now @rcox771. Just to clarify, the only non-daemon thread is the httpserver. (You can verify this by setting daemon=True in that thread, and then the program is terminated automatically when the main thread ends). So we shouldn't have to worry about anything else, e.g. the tunneling
from gradio.
I just pushed a fix with the the latest version of gradio (1.1.0). Can you see if this works for you?
from gradio.
@abidlabs works great! Yeah I guess I was confused about how the daemon was working behind the scenes in your tunneling.py
Thanks for the lesson and threading experience. I'll close this out and look for other ways I can help pack you back!
๐ค
-Russ
from gradio.
Glad to hear! Thanks for helping :)
from gradio.
Related Issues (20)
- TypeError: Object of type JsonData is not JSON serializable When Using gr.JSON in HTTP API HOT 1
- `every` not working for subsequent events.
- Downloaded file is empty HOT 8
- Markdown formatting for gr.JSON() HOT 2
- Max height / scrolling for `gr.Code`, `gr.Accordion`, and `gr.Tabs` HOT 3
- Allow for markdown in `info=`
- PDF Component change Event Fires Twice HOT 1
- Using both gr.Interface with examples and gr.Audio could cause serious memory leak and continuously increasing CPU usage
- Add flagging functionalities to ChatInterface
- Vertical Scrollbar for dynamic apps HOT 4
- If the decorator contains yield, the decorator will not be able to use ordinary functions.
- Disable autofocus in chatbot if embedded in another page/docs
- [Dynamic components] - Problem with update values on client after component is removed and added again.
- [chatbox, ChatInterface]gradio user input markdown code block shows out of page
- Add autofocus for gr.Chatbot HOT 1
- Error: Could not resolve app config when connecting to model using Gradio client in Node.js
- Chatbot image does not have download button HOT 1
- Add a submit icon to `gr.Textbox`, particularly for `gr.ChatInterface(multimodal=False)`
- gradio imageeditor to generate output in 'background', 'layers', 'composite' HOT 3
- State variable changes aren't monitored for custom class instances
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gradio.