different-ai / embedbase Goto Github PK
View Code? Open in Web Editor NEWA dead-simple API to build LLM-powered apps
Home Page: https://docs.embedbase.xyz
License: MIT License
A dead-simple API to build LLM-powered apps
Home Page: https://docs.embedbase.xyz
License: MIT License
feat
fix
chore
only from the matching project, example:
Bob commit "feat(dashboard): add gpt5"
John commit "fix(hosted): fix bug"
Alice commit "chore(js): add something"
js-1.0.1
Alice commit "chore(js): add something"
dashboard-1.6.7
Bob commit "feat(dashboard): add gpt5"
etc
.
.
I find it a bit hard to parse curl requests and I think the readme would benefit from an example with fetch
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 375, in run_asgi
result = await app(self.scope, self.receive, self.send)
File "/usr/local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
return await self.app(scope, receive, send)
File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 276, in __call__
await super().__call__(scope, receive, send)
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 333, in _sentry_patched_asgi_app
return await middleware(scope, receive, send)
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py", line 139, in _run_asgi3
return await self._run_app(scope, lambda: self.app(scope, receive, send))
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py", line 188, in _run_app
raise exc from None
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py", line 183, in _run_app
return await callback()
File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 122, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 130, in _create_span_call
return await old_call(app, scope, new_receive, new_send, **kwargs)
File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
raise exc
File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 130, in _create_span_call
return await old_call(app, scope, new_receive, new_send, **kwargs)
File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 84, in __call__
await self.app(scope, receive, send)
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 130, in _create_span_call
return await old_call(app, scope, new_receive, new_send, **kwargs)
File "/usr/local/lib/python3.10/site-packages/starlette/middleware/base.py", line 108, in __call__
response = await self.dispatch_func(request, call_next)
File "/middlewares/auth_api_key/auth_api_key.py", line 182, in dispatch
response = await call_next(request)
File "/usr/local/lib/python3.10/site-packages/starlette/middleware/base.py", line 84, in call_next
raise app_exc
File "/usr/local/lib/python3.10/site-packages/starlette/middleware/base.py", line 70, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 227, in _sentry_exceptionmiddleware_call
await old_call(self, scope, receive, send)
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 130, in _create_span_call
return await old_call(app, scope, new_receive, new_send, **kwargs)
File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
raise exc
File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
await self.app(scope, receive, sender)
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 130, in _create_span_call
return await old_call(app, scope, new_receive, new_send, **kwargs)
File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
raise e
File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await self.app(scope, receive, send)
File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 66, in app
response = await func(request)
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/fastapi.py", line 130, in _sentry_app
return await old_app(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 237, in app
raw_response = await run_endpoint_function(
File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 163, in run_endpoint_function
return await dependant.call(**values)
File "/usr/local/lib/python3.10/site-packages/embedbase/app.py", line 213, in add
await self.db.update(
File "/usr/local/lib/python3.10/site-packages/embedbase/database/supabase_db.py", line 125, in update
await asyncio.gather(*[_insert(batch_df) for batch_df in batches])
File "/usr/local/lib/python3.10/site-packages/embedbase/database/supabase_db.py", line 122, in _insert
.execute()
File "/usr/local/lib/python3.10/site-packages/postgrest/_sync/request_builder.py", line 68, in execute
raise APIError(r.json())
postgrest.exceptions.APIError: {'code': '22P05', 'details': '\\u0000 cannot be converted to text.', 'hint': None, 'message': 'unsupported Unicode escape sequence'}
Steps to reproduce the behavior:
python --version
ChatGPT like user feedback to open opportunities to improve user experience
I'm imagining a middleware that save search queries and a new end point that accept feedback on search result with the ID of the result
from typing import Callable
import uuid
from fastapi import Request
from fastapi.responses import JSONResponse
from embedbase.database.base import VectorDatabase
from embedbase.embedding.base import Embedder
async def save_search(
request: Request, call_next: Callable, db: VectorDatabase, embedder: Embedder
):
"""
Upon search request, save the request to a database.
"""
# todo overlap with add on "search" dataset
if request.method != "POST" or "/v1/search" not in request.url.path:
return await call_next(request)
request_body = await request.json()
new_id = str(uuid.uuid4())
request_body["id"] = new_id
response = await db.save("search", request_body)
return await call_next(request)
(almost) pseudo code for feedback endpoint:
app = (
get_app()
.use_embedder(...)
.use_db(...)
.run()
)
# An endpoint that let you rate search results
@app.post("/feedback")
async def human_feedback(req, cb, db, embedder):
# here would save to a table feedback
# the request body looks like "searchid: vrevrwrew, feedback: 0 or 1"
db.save("feedback", req.body)
return 200
Iโm using your embedbase library on python. I wonder how can I use the createContext api like on the embedbase.xyz playground menu. By the way embedbasd is very helpful to me. Thank you very much.
Local model and storage for data privacy concerned people
I believe we should store only vectors in vector databases and everything else in another storage (firestore, polybase, mongo, postgres, whatever) (+ eventually the vectors)
Getting started quickly without having to install anything locally (using in memory db, no docker stuff)
It would be so helpful in my use-case
results = await client.dataset(recipe_id, farm_id, user_id, locaton_id).search(question, max_token=3000, ratio=[.7, .1, .1, .1)
max_token of 2100, 300, 300, 300 are applied to each dataset.
Originally posted by @ccomkhj in #71 (comment)
Probably similar to https://github.com/louis030195/cloud-computer-vision
In embedbase cloud, to view the data table I am supposed to go to this url (dev as my datasets name), https://app.embedbase.xyz/dashboard/explorer/dev, however it fails to show the data.
I need to click the arrow and lead to this https://app.embedbase.xyz/dashboard/explorer/dev?page=0 to show the data table. In addition, further clicking of left arrow will lead to
https://app.embedbase.xyz/dashboard/explorer/dev?page=-1
https://app.embedbase.xyz/dashboard/explorer/dev?page=-2
First of all, that functionality looks very promising.
I updated the plugin, logged out and in, restarted, cleared cache, disabled enabled... but the plugin doesn't work. I don't know if I have to wait for a long time...
crash sometimes (chain of issues, backend crashing default timeout supabase is 5 seconds)
Steps to reproduce the behavior:
Dashboard/playground
TypeError: Cannot read properties of undefined (reading 'data')
File "app:///_next/server/pages/api/createContext.js", line 139, col 18, in <anonymous>
return r.data;
File "<anonymous>", in Array.map
File "app:///_next/server/pages/api/createContext.js", line 132, col 30, in createContext
const datas = topResults.map((r)=>{
File "node:internal/process/task_queues", line 95, col 5, in process.processTicksAndRejections
File "app:///_next/server/pages/api/createContext.js", line 170, col 21, in buildPrompt
const context = await createContext(prompt, datasetIds, apiKey);
File "/var/task/dashboard/node_modules/@sentry/nextjs/cjs/server/wrapApiHandlerWithSentry.js", line 143, col 33, in <anonymous>
const handlerResult = await wrappingTarget.apply(thisArg, args);
File "/var/task/dashboard/node_modules/next/dist/server/api-utils/node.js", line 372, col 9, in Object.apiResolver
await resolver(req, res);
File "/var/task/dashboard/node_modules/next/dist/server/next-server.js", line 513, col 9, in NextNodeServer.runApi
await (0, _node1).apiResolver(req.originalRequest, res.originalResponse, query, pageModule, {
File "/var/task/dashboard/node_modules/next/dist/server/next-server.js", line 815, col 35, in Object.fn
handled = await this.handleApiRequest(req, res, query, // TODO: see if we can add a runtime check for this
File "/var/task/dashboard/node_modules/next/dist/server/router.js", line 243, col 32, in Router.execute
const result = await route.fn(req, res, params, parsedUrlUpdated, upgradeHead);
Backend
different-ai/embedbase-hosted#3
Solutions:
hosted
add with empty data maybe?
ValueError: Out of range float values are not JSON compliant
at .iterencode ( /usr/local/lib/python3.10/json/encoder.py:257 )
at .encode ( /usr/local/lib/python3.10/json/encoder.py:199 )
at .dumps ( /usr/local/lib/python3.10/json/init.py:238 )
at .render ( /usr/local/lib/python3.10/site-packages/starlette/responses.py:199 )
at .init ( /usr/local/lib/python3.10/site-packages/starlette/responses.py:55 )
at .init ( /usr/local/lib/python3.10/site-packages/starlette/responses.py:196 )
at .add ( /usr/local/lib/python3.10/site-packages/embedbase/app.py:164 )
at .run_endpoint_function ( /usr/local/lib/python3.10/site-packages/fastapi/routing.py:163 )
at .app ( /usr/local/lib/python3.10/site-packages/fastapi/routing.py:237 )
at ._sentry_app ( /usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/fastapi.py:130 )
at .app ( /usr/local/lib/python3.10/site-packages/starlette/routing.py:66 )
at .handle ( /usr/local/lib/python3.10/site-packages/starlette/routing.py:276 )
at .call ( /usr/local/lib/python3.10/site-packages/starlette/routing.py:718 )
at .call ( /usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py:18 )
at .call ( /usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py:21 )
at ._create_span_call ( /usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py:130 )
at .call ( /usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py:68 )
at .call ( /usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py:79 )
at ._create_span_call ( /usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py:130 )
at ._sentry_exceptionmiddleware_call ( /usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py:229 )
at .coro ( /usr/local/lib/python3.10/site-packages/starlette/middleware/base.py:70 )
at .call_next ( /usr/local/lib/python3.10/site-packages/starlette/middleware/base.py:84 )
at .dispatch ( /middlewares/auth_api_key/auth_api_key.py:182 )
at .call ( /usr/local/lib/python3.10/site-packages/starlette/middleware/base.py:108 )
at ._create_span_call ( /usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py:130 )
at .simple_response ( /usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py:146 )
at .call ( /usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py:91 )
at ._create_span_call ( /usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py:130 )
at .call ( /usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py:162 )
at .call ( /usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py:184 )
at ._create_span_call ( /usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py:130 )
at .call ( /usr/local/lib/python3.10/site-packages/starlette/applications.py:122 )
at ._run_app ( /usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py:183 )
at ._run_app ( /usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py:188 )
at ._run_asgi3 ( /usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py:139 )
at ._sentry_patched_asgi_app ( /usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py:335 )
at .call ( /usr/local/lib/python3.10/site-packages/fastapi/applications.py:276 )
at .call ( /usr/local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py:78 )
at .run_asgi ( /usr/local/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py:435 )
na
messages stay here (need to clear react state)
new chat everytime
embedbase-client: 0.1.2
ubuntu 22.04
Python 3.8.16
client = EmbedbaseAsyncClient(embedbase_url, embedbase_key)
if farm_id is None:
# this works!
[results] = await asyncio.gather(
*[client.dataset(recipe_id).search(question, limit=7).get()]
)
elif recipe_id is None:
# this does not work!
recipe_id = getRecipeId(farm_id)
[results_recipe, results_farm] = await asyncio.gather(
*[
client.dataset(recipe_id).search(question, limit=6).get(),
client.dataset(farm_id).search(question, limit=1).get(),
]
)
two dataset combined and return
async function call not working (uvicorn, fastapi)
from embedbase_client.client import EmbedbaseClient
import yaml
from fastapi import FastAPI
with open("credential/embedbase.yaml", "r") as f:
API = yaml.safe_load(f)
embedbase_key = API['EMBEDBASE_KEY']
embedbase_url = API['EMBEDBASE_URL']
app = FastAPI()
@app.get("/")
async def root():
client = EmbedbaseClient(embedbase_url, embedbase_key)
client.dataset('basil_greenhouse').search(
'how to grow basil?', limit=5)
return {"message": "Hello World"}
uvicorn bug_test:app
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Return list of datasets
Using FastAPI
Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/tenacity/init.py", line 382, in call result = fn(*args, **kwargs) File "/app/embedbase/api.py", line 197, in embed return openai.Embedding.create(input=input, model=model)["data"] File "/usr/local/lib/python3.10/site-packages/openai/api_resources/embedding.py", line 33, in create response = super().create(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 153, in create response, _, api_key = requestor.request( File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 226, in request resp, got_stream = self._interpret_response(result, stream) File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 619, in _interpret_response self._interpret_response_line( File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 679, in _interpret_response_line raise self.handle_error_response( openai.error.InvalidRequestError: [''] is not valid under any of the given schemas - 'input'
Great project. Would be nice to get support for a local DB, without having to rely on a third party. Maybe a local PostgreSQL or local SQLite?
weaviate.io
Hi,
I have tried to to npm run dev
the dashboard part yet it shows
You have added a custom /_error page without a custom /404 page. This prevents the 404 page from being auto statically optimized. See here for info: https://nextjs.org/docs/messages/custom-error-no-custom-404
Line 159 in d494ec0
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 436, in run_asgi
result = await app( # type: ignore[func-returns-value]
File "/usr/local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
return await self.app(scope, receive, send)
File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 276, in __call__
await super().__call__(scope, receive, send)
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 333, in _sentry_patched_asgi_app
return await middleware(scope, receive, send)
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py", line 139, in _run_asgi3
return await self._run_app(scope, lambda: self.app(scope, receive, send))
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py", line 188, in _run_app
raise exc from None
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py", line 183, in _run_app
return await callback()
File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 122, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 130, in _create_span_call
return await old_call(app, scope, new_receive, new_send, **kwargs)
File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
raise exc
File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 130, in _create_span_call
return await old_call(app, scope, new_receive, new_send, **kwargs)
File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 84, in __call__
await self.app(scope, receive, send)
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 130, in _create_span_call
return await old_call(app, scope, new_receive, new_send, **kwargs)
File "/usr/local/lib/python3.10/site-packages/starlette/middleware/base.py", line 108, in __call__
response = await self.dispatch_func(request, call_next)
File "/app/middlewares/auth_api_key/auth_api_key.py", line 182, in dispatch
response = await call_next(request)
File "/usr/local/lib/python3.10/site-packages/starlette/middleware/base.py", line 84, in call_next
raise app_exc
File "/usr/local/lib/python3.10/site-packages/starlette/middleware/base.py", line 70, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 227, in _sentry_exceptionmiddleware_call
await old_call(self, scope, receive, send)
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 130, in _create_span_call
return await old_call(app, scope, new_receive, new_send, **kwargs)
File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
raise exc
File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
await self.app(scope, receive, sender)
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 130, in _create_span_call
return await old_call(app, scope, new_receive, new_send, **kwargs)
File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
raise e
File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await self.app(scope, receive, send)
File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 66, in app
response = await func(request)
File "/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/fastapi.py", line 130, in _sentry_app
return await old_app(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 237, in app
raw_response = await run_endpoint_function(
File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 163, in run_endpoint_function
return await dependant.call(**values)
File "/app/embedbase/app.py", line 159, in add
return JSONResponse(
File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 196, in __init__
super().__init__(content, status_code, headers, media_type, background)
File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 55, in __init__
self.body = self.render(content)
File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 199, in render
return json.dumps(
File "/usr/local/lib/python3.10/json/__init__.py", line 238, in dumps
**kw).encode(obj)
File "/usr/local/lib/python3.10/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/local/lib/python3.10/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
ValueError: Out of range float values are not JSON compliant
ci
run local test in the same time than ci or multiple projects integration tests running in parallel
pass
run tests on different datasets according to environment/context
i.e. multiple tags
Especially firebase_admin seems a huge dependency
https://github.com/another-ai/embedbase/blob/main/docker/Dockerfile#L12
add response model to endpoints
hard to parse
.
get footnotes ref with github and pdf uploader in playground or search responses
.
.
copy pasta docs script code or other stuff just throw urls in metadata path and at generation time include metadata
.
.
When follow https://docs.embedbase.xyz/examples/local-embedder. I got error below.
INFO: 127.0.0.1:58500 - "POST /v1/animals HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/uvicorn/protocols/http/httptools_impl.py", line 436, in run_asgi
result = await app( # type: ignore[func-returns-value]
File "/usr/local/lib/python3.8/dist-packages/uvicorn/middleware/proxy_headers.py", line 78, in call
return await self.app(scope, receive, send)
File "/usr/local/lib/python3.8/dist-packages/fastapi/applications.py", line 276, in call
await super().call(scope, receive, send)
File "/usr/local/lib/python3.8/dist-packages/starlette/applications.py", line 122, in call
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/errors.py", line 184, in call
raise exc
File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/errors.py", line 162, in call
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/exceptions.py", line 79, in call
raise exc
File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/exceptions.py", line 68, in call
await self.app(scope, receive, sender)
File "/usr/local/lib/python3.8/dist-packages/fastapi/middleware/asyncexitstack.py", line 21, in call
raise e
File "/usr/local/lib/python3.8/dist-packages/fastapi/middleware/asyncexitstack.py", line 18, in call
await self.app(scope, receive, send)
File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 718, in call
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 276, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 66, in app
response = await func(request)
File "/usr/local/lib/python3.8/dist-packages/fastapi/routing.py", line 237, in app
raw_response = await run_endpoint_function(
File "/usr/local/lib/python3.8/dist-packages/fastapi/routing.py", line 163, in run_endpoint_function
return await dependant.call(**values)
File "/usr/local/lib/python3.8/dist-packages/embedbase/app.py", line 230, in add
await self.db.update(
File "/usr/local/lib/python3.8/dist-packages/embedbase/database/postgres_db.py", line 197, in update
cur.execute(q, flat_values)
File "/usr/local/lib/python3.8/dist-packages/psycopg/cursor.py", line 723, in execute
raise ex.with_traceback(None)
psycopg.errors.DataException: expected 1536 dimensions, not 384
https://docs.embedbase.xyz/tutorials/nextra-qa-docs is only setup for a cloudhosted embedbase - is there a way to get this to run with a local deployment?
.
send 1000000000000000000000000 length string to add
pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err` value: StackOverflow
at .encode ( [/usr/local/lib/python3.10/site-packages/tiktoken/core.py:120](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Ftiktoken%2Fcore.py&line=120&project=embedbase) )
at .is_too_big ( [/usr/local/lib/python3.10/site-packages/embedbase/embedding/openai.py:72](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fembedbase%2Fembedding%2Fopenai.py&line=72&project=embedbase) )
at .add ( [/usr/local/lib/python3.10/site-packages/embedbase/app.py:140](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fembedbase%2Fapp.py&line=140&project=embedbase) )
at .run_endpoint_function ( [/usr/local/lib/python3.10/site-packages/fastapi/routing.py:163](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Ffastapi%2Frouting.py&line=163&project=embedbase) )
at .app ( [/usr/local/lib/python3.10/site-packages/fastapi/routing.py:237](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Ffastapi%2Frouting.py&line=237&project=embedbase) )
at ._sentry_app ( [/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/fastapi.py:130](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fsentry_sdk%2Fintegrations%2Ffastapi.py&line=130&project=embedbase) )
at .app ( [/usr/local/lib/python3.10/site-packages/starlette/routing.py:66](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fstarlette%2Frouting.py&line=66&project=embedbase) )
at .handle ( [/usr/local/lib/python3.10/site-packages/starlette/routing.py:276](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fstarlette%2Frouting.py&line=276&project=embedbase) )
at .__call__ ( [/usr/local/lib/python3.10/site-packages/starlette/routing.py:718](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fstarlette%2Frouting.py&line=718&project=embedbase) )
at .__call__ ( [/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py:18](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Ffastapi%2Fmiddleware%2Fasyncexitstack.py&line=18&project=embedbase) )
at ._create_span_call ( [/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py:130](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fsentry_sdk%2Fintegrations%2Fstarlette.py&line=130&project=embedbase) )
at .__call__ ( [/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py:68](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fstarlette%2Fmiddleware%2Fexceptions.py&line=68&project=embedbase) )
at ._create_span_call ( [/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py:130](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fsentry_sdk%2Fintegrations%2Fstarlette.py&line=130&project=embedbase) )
at ._sentry_exceptionmiddleware_call ( [/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py:229](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fsentry_sdk%2Fintegrations%2Fstarlette.py&line=229&project=embedbase) )
at .coro ( [/usr/local/lib/python3.10/site-packages/starlette/middleware/base.py:70](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fstarlette%2Fmiddleware%2Fbase.py&line=70&project=embedbase) )
at .__aexit__ ( [/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py:662](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fanyio%2F_backends%2F_asyncio.py&line=662&project=embedbase) )
at .__call__ ( [/usr/local/lib/python3.10/site-packages/starlette/middleware/base.py:106](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fstarlette%2Fmiddleware%2Fbase.py&line=106&project=embedbase) )
at ._create_span_call ( [/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py:130](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fsentry_sdk%2Fintegrations%2Fstarlette.py&line=130&project=embedbase) )
at .__call__ ( [/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py:83](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fstarlette%2Fmiddleware%2Fcors.py&line=83&project=embedbase) )
at ._create_span_call ( [/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py:130](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fsentry_sdk%2Fintegrations%2Fstarlette.py&line=130&project=embedbase) )
at .__call__ ( [/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py:162](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fstarlette%2Fmiddleware%2Ferrors.py&line=162&project=embedbase) )
at ._create_span_call ( [/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py:130](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fsentry_sdk%2Fintegrations%2Fstarlette.py&line=130&project=embedbase) )
at .__call__ ( [/usr/local/lib/python3.10/site-packages/starlette/applications.py:122](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fstarlette%2Fapplications.py&line=122&project=embedbase) )
at ._run_app ( [/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py:183](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fsentry_sdk%2Fintegrations%2Fasgi.py&line=183&project=embedbase) )
at ._run_asgi3 ( [/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py:139](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fsentry_sdk%2Fintegrations%2Fasgi.py&line=139&project=embedbase) )
at ._sentry_patched_asgi_app ( [/usr/local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py:335](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fsentry_sdk%2Fintegrations%2Fstarlette.py&line=335&project=embedbase) )
at .__call__ ( [/usr/local/lib/python3.10/site-packages/fastapi/applications.py:276](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Ffastapi%2Fapplications.py&line=276&project=embedbase) )
at .__call__ ( [/usr/local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py:78](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fuvicorn%2Fmiddleware%2Fproxy_headers.py&line=78&project=embedbase) )
at .run_asgi ( [/usr/local/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py:435](https://console.cloud.google.com/debug?referrer=fromlog&file=%2Fusr%2Flocal%2Flib%2Fpython3.10%2Fsite-packages%2Fuvicorn%2Fprotocols%2Fhttp%2Fhttptools_impl.py&line=435&project=embedbase) )
simple fix: if len(s) > 10000 > then check tokens (no need to count tokens if the string is incredibly long already)
Would be cool to see a few extra examples.
How would embedbase work in various use case. Here's a couple cool repos for inspo:
https://github.com/dandv/shower-thoughts-weaviate
https://github.com/gil--/shoppy-gpt
https://github.com/gmchad/embed-vtt
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.