monarch-initiative / agent-smith-ai Goto Github PK

this has been mostly tested with simple endpoints and query params; need to spruce up the ApiWrapper code a bit and make sure we can handle other param types (esp. body and path)

Currently pretty much all errors are caught and returned within a message to the user. This is robust for interactive dependent use, but not great for example when running automated tests, where we want to raise errors so the caller can try again. Probably there should be an option in .chat() for "error_handle", for capture and return in message, raise to parent caller, or maybe retry (but this depends on the nature of the error). Related to #17 .

Newer models have context sizes up to 128K, current behavior defaults to 4k for unknown models.

It seems there's no good way to do this: https://community.openai.com/t/easy-way-to-get-a-context-window-for-a-model/552099/5

There's a regex they check for: {'session_id': '195022ac-6041-4a65-a98b-cf38f02888f0', 'message': {'role': 'assistant', 'author': 'System', 'intended_recipient': 'User', 'is_function_call': False, 'content': "Error in sending function or method call result to model: 'go.get_term_by_name' does not match '^[a-zA-Z0-9_-]{1,64}$' - 'messages.4.function_call.name'", 'func_name': None, 'func_arguments': None, 'finish_reason': None}, 'agent': 'GO Assistant (GPT-3.5)'}

currently I'm using whatever fastapi defines the endpoint operation id as (right?), which might be too long/convoluted

Currently devs can register callable methods (only) by defining the method as part of the agent class and registering it by name (string). May as well allow any callable, but this will be a breaking change.

These are getting really redundant in code and probably we could just do with a generic .process_message() or something to handle both cases. This would be a breaking change.

rarely the model will try to pass a function parameter that doesn't exist; e.g.

Error in message processing: SemanticScholarAgent.search() got an unexpected keyword argument 'sort'. Full Traceback: Traceback (most recent call last): File "/Users/oneils/Library/Caches/pypoetry/virtualenvs/oneilsh-agent-explorations-2v5X1dTk-py3.11/lib/python3.11/site-packages/agent_smith_ai/utility_agent.py", line 159, in chat for message in self._process_model_response(response_raw, intended_recipient = author): File "/Users/oneils/Library/Caches/pypoetry/virtualenvs/oneilsh-agent-explorations-2v5X1dTk-py3.11/lib/python3.11/site-packages/agent_smith_ai/utility_agent.py", line 393, in _process_model_response for potential_message in func_result: File "/Users/oneils/Library/Caches/pypoetry/virtualenvs/oneilsh-agent-explorations-2v5X1dTk-py3.11/lib/python3.11/site-packages/agent_smith_ai/utility_agent.py", line 230, in _call_function result = func(**params) ^^^^^^^^^^^^^^ TypeError: SemanticScholarAgent.search() got an unexpected keyword argument 'sort'

I like the look of https://github.com/BerriAI/litellm (which also supports streaming, though currently agent smith is not setup to handle that)

early attempts at building one from scratch (agentserver webapp) lacked security, especially since it consisted of a server-side component callable by nefarious clients

this repo contains a streamlit app that can wrap these agents, it should be rolled into here to make it super duper easy to deploy a web-based chat smart agent, replacing the current attempts at an 'agent server'

https://platform.openai.com/docs/guides/moderation

This rather cryptic error comes from not being able to parse the type hint info to generate docs for functions; need to provide a better error message to let the dev know they are missing a type hint or something.

NotImplementedError: Unsupported type: <class 'inspect._empty'>
...
File "/Users/oneils/Library/Caches/pypoetry/virtualenvs/oneilsh-agent-explorations-2v5X1dTk-py3.11/lib/python3.11/site-packages/agent_smith_ai/utility_agent.py", line 511, in _python_type_to_json_schema
    raise NotImplementedError(f'Unsupported type: {py_type}')