Comments (5)
I'm looking at the multiprocess module and I may want to move over to that to simplify the codebase and reduce performance overheads. Will need to investigate further
from salve.
Yes, it seems that the multiprocess library is just a builtin form of IPC specifically for python :D
from salve.
The send is near instant with even 140k char strings. I will definitely look into rewriting the basic example and then the main library piece by piece
from salve.
Should also restore full windows compatability
from salve.
Got this working:
from multiprocessing import Process, Pipe, Queue
from multiprocessing.connection import Connection
from time import sleep, time
from typing import Any, NotRequired, TypedDict
from random import randint
from sys import exit
class Message(TypedDict):
"""Base class for messages in and out of the server"""
id: int
class Request(Message):
"""Request results/output from the server"""
index: int
class Response(Message):
"""Server responses to requests, notifications, and pings"""
cancelled: bool
result: NotRequired[str]
class Client:
def __init__(self) -> None:
self.used_ids: list[int] = []
self.current_id: int = 0
self.client_end: Connection
self.response_queue: Queue[Response] = Queue()
self.requests_queue: Queue[Request] = Queue()
self.create_server()
def create_server(self) -> None:
self.client_end, server_end = Pipe()
self.proccess = Process(
target=Server,
args=(server_end, self.response_queue, self.requests_queue),
)
self.proccess.start()
self.response_queue.get() # Wait until server has been made
def check_server(self) -> None:
if not self.proccess.is_alive():
self.create_server()
def make_request(self, request: int) -> None:
self.check_server()
id = randint(0, 5000)
while id in self.used_ids:
id = randint(0, 5000)
self.used_ids.append(id)
self.current_id = id
final_request: Request = {"id": id, "index": request}
self.requests_queue.put(final_request)
def check_output(self) -> Any:
items: list[Response] = []
while not self.response_queue.empty():
items.append(self.response_queue.get())
wanted_response: Response = {"id": 0, "cancelled": True}
for item in items:
id: int = item["id"]
self.used_ids.remove(id)
if id == self.current_id:
wanted_response = item
return wanted_response
def kill_server(self) -> None:
self.proccess.kill()
class Server:
def __init__(
self,
server_end: Connection,
response_queue: Queue,
requests_queue: Queue,
) -> None:
self.internal_list: list[str] = ["Yuh", "anyways", "soooo"]
self.server_end: Connection = server_end
self.response_queue: Queue[Response] = response_queue
self.requests_queue: Queue[Request] = requests_queue
self.response_queue.put({"id": 0, "cancelled": False, "result": ""})
self.old_time = time()
while True:
self.run_tasks()
def run_tasks(self) -> None:
current_time = time()
if current_time - self.old_time > 5:
exit(0)
while not self.requests_queue.empty():
self.old_time = current_time
request: Request = self.requests_queue.get()
cancelled: bool = not self.response_queue.empty()
output = ""
if not cancelled:
try:
output = self.internal_list[request["index"]]
except IndexError:
output = "Bad Index"
final_response: Response = {
"id": request["id"],
"cancelled": cancelled,
"result": output,
}
self.response_queue.put(final_response)
sleep(0.0025)
if __name__ == "__main__":
st = time()
c = Client()
c.make_request(3)
while (x := c.check_output()) is None or x == {"id": 0, "cancelled": True}:
continue
print(x)
c.kill_server()
print(time() - st, c.__sizeof__())
from salve.
Related Issues (20)
- Multi line comments and docstrings aren't highlighted right HOT 3
- Feature Idea: reduce tokens by overwriting old ones HOT 6
- Feature Idea: make and use caches to improve speed HOT 1
- Bug Report: Minimum Python Version should be 3.11 HOT 1
- Feature Idea: buffer tokens HOT 2
- Feature Idea: Major refactor of highlight file HOT 1
- Feature Idea: Make module 3.10 compatible HOT 2
- Feature Idea: Add windows tests HOT 1
- Feature Idea: Explain python version plans
- Bug Report: Remove tokens outside of text tange HOT 1
- Bug Report:
- Feature Idea: Add optional logging
- Recommendation: Use salve-dependency-hub throughout the project
- Feature Idea: custom language parser
- Feature Idea: Make link finding and hidden char finding a separate function HOT 1
- Feature report: Use `Token Tools` throughout the project
- Feature Idea: Add To do section to PR template
- Feature Idea: make formatting jobs go in same workflow
- Feature Idea: Make IPC/Server backend a secondary tool HOT 1
- Code Change: remove the wrappers
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from salve.