Comments (2)
What is more, we should require all unit tests that call LLM to provide text generated by LLM to simulate its output, so we can see in CI/CD whether the generated text is correctly formatted. Finally, before deciding to merge PR, we run a real output of LLM once.
from camel.
Do we really need the real output of LLM? As I checking several famous agent projects, all of them just use mock outputs. A possible reson is most of tests do not require the output contents but only require their formats. Also they do not have so many tests as we have. I wonder if our strict tests limited our development progress?
Can we remove some tests that are actually useless?
from camel.
Related Issues (20)
- [Feature] Add `llama.cpp` local backend model support HOT 1
- [Roadmap] Action and ActionAgent
- [BUG] `get_usage_dict` may not work as expected for open source models
- [Feature Request] Discussion on the implementation of open source model type HOT 1
- [Feature Request] add retrieval function for agent to use
- [BUG] About using role_playing_with_open_source_model.py base on local LLM(fastChat vicuna-7b-v1.5) HOT 2
- [Feature Request] Implementing Structured Output Support in AI Role-Playing Module
- [BUG] AugAssign is not supported in `PythonInterpreter` HOT 1
- [BUG] Multi-agent Compatibility Score of Role Assignment HOT 3
- [BUG] Subtasks Decomposition with Dependencies in Cycle HOT 4
- [BUG] fix duplicate system message
- [Feature Request] Integrate `Read the Docs` to CAMEL for better documentation
- [Feature Request] Deprecate `functions` and use `tools` to align with the latest openai api
- [BUG] Dependency conflict between fasctchat and pydantic.alias_generator in openai_function.py HOT 4
- [Feature Request] Load .env file environment variables before instantiating OpenAI clients
- [Feature Request] Context template for better user experience
- [Feature Request] Interface converting string into `element` supported by Unstructured IO
- [Feature Request] Integrate rerank model HOT 1
- [Feature Request] Integrate key word search method BM25 HOT 1
- [Feature Request] Multi-modal RAG(Retrieval-Augmented Generation)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from camel.