Comments (3)
Yep, it's possible.
For the search engine part, check out e.g. the search_with_bing() function, and the photon's init() function. We currently support bing, google and https://serper.dev/. It's probably easy to swap in your own search engine.
For the LLM model, you can replace the openai client to connect to other openai-compatible servers. The related question part requires a bit care, as your llm server need to support function calling / structured output. All lepton LLM endpoints support this (with custom models too) out of the box. With others, you might need a bit of adjustment, and you can also choose to simply turn off related questions.
from search_with_lepton.
For local deployment, you just need to do (in commandline):
pip install -U leptonai
lep login
and make sure you log in to your workspace.
For the other non-Lepton hosted models, see above - essentially it is this line
https://github.com/leptonai/search_with_lepton/blob/db27467/search_with_lepton.py#L257
You might want to start with environmental variable RELATED_QUESTIONS=False with other api endpoints.
from search_with_lepton.
I have successfully deployed on lepton.ai, and its lightning-fast response has left a deep impression on me : )
For local deployment, I guess we need to modify the following two parts, right?
search_with_lepton/search_with_lepton.py
Lines 256 to 260 in a6ac6da
search_with_lepton/search_with_lepton.py
Line 215 in a6ac6da
For online deployment, is it not possible to switch to other non-Lepton Hosted models (even those I deployed on Lepton)?"
LLM_MODEL: the LLM model to run. We recommend using mixtral-8x7b, but if you want to experiment other models, you can try the ones hosted on LeptonAI, for example, llama2-70b, llama2-13b, llama2-7b. Note that small models won't work that well.
https://dashboard.lepton.ai/workspace/olcdfyso/explore/detail/search-by-lepton
from search_with_lepton.
Related Issues (20)
- Start server fail saying Directory 'ui' does not exist HOT 7
- failure in query HOT 2
- 能否接入国内搜索引擎吗 HOT 1
- pip install -U leptonai && lep login之后显示command not found: lep
- client-side exception has occurred HOT 4
- 如何使用部署自己本地的模型用于问答生成? HOT 1
- KV error HOT 2
- "Application error: a client-side exception has occurred" when rendering the answer HOT 8
- 搜索重定向问题 HOT 2
- Add brief & detail mode HOT 2
- Can not delete a "blank" kv namespace HOT 5
- Local deployment failed HOT 1
- predict user next related search, or predict next question, to create a complete context chain HOT 1
- can not delete kv namespaces HOT 3
- 431 (Request Header Fields Too Large) when trying to search a query HOT 1
- How does the routing in the app work? When I add a new route, it renders plain text.
- Why need <INST>? HOT 1
- 希望在搜索结果处增加复制/copy按钮
- 怎么接入自己的模型呢 HOT 5
- backend cannot be SEARCHAPI HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from search_with_lepton.