infiniflow / ragflow Goto Github PK
View Code? Open in Web Editor NEWRAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.
Home Page: https://ragflow.io
License: Apache License 2.0
RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.
Home Page: https://ragflow.io
License: Apache License 2.0
main
Collecting accelerate==0.27.2 (from -r requirements.txt (line 1))
Downloading accelerate-0.27.2-py3-none-any.whl (279 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 280.0/280.0 kB 6.4 MB/s eta 0:00:00
Requirement already satisfied: aiohttp==3.9.3 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 2)) (3.9.3)
Requirement already satisfied: aiosignal==1.3.1 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 3)) (1.3.1)
Requirement already satisfied: annotated-types==0.6.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 4)) (0.6.0)
Collecting anyio==4.3.0 (from -r requirements.txt (line 5))
Downloading anyio-4.3.0-py3-none-any.whl (85 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 85.6/85.6 kB 13.8 MB/s eta 0:00:00
Requirement already satisfied: argon2-cffi==23.1.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 6)) (23.1.0)
Requirement already satisfied: argon2-cffi-bindings==21.2.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 7)) (21.2.0)
Collecting Aspose.Slides==24.2.0 (from -r requirements.txt (line 8))
Downloading Aspose.Slides-24.2.0-py3-none-manylinux1_x86_64.whl (88.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 88.7/88.7 MB 2.6 MB/s eta 0:00:00
Requirement already satisfied: attrs==23.2.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 9)) (23.2.0)
Collecting blinker==1.7.0 (from -r requirements.txt (line 10))
Downloading blinker-1.7.0-py3-none-any.whl (13 kB)
Collecting cachelib==0.12.0 (from -r requirements.txt (line 11))
Downloading cachelib-0.12.0-py3-none-any.whl (20 kB)
Requirement already satisfied: cachetools==5.3.3 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 12)) (5.3.3)
Requirement already satisfied: certifi==2024.2.2 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 13)) (2024.2.2)
Requirement already satisfied: cffi==1.16.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 14)) (1.16.0)
Requirement already satisfied: charset-normalizer==3.3.2 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 15)) (3.3.2)
Requirement already satisfied: click==8.1.7 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 16)) (8.1.7)
Collecting coloredlogs==15.0.1 (from -r requirements.txt (line 17))
Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 46.0/46.0 kB 7.2 MB/s eta 0:00:00
Requirement already satisfied: cryptography==42.0.5 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 18)) (42.0.5)
Collecting dashscope==1.14.1 (from -r requirements.txt (line 19))
Downloading dashscope-1.14.1-py3-none-any.whl (1.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 86.1 MB/s eta 0:00:00
Collecting datasets==2.17.1 (from -r requirements.txt (line 20))
Downloading datasets-2.17.1-py3-none-any.whl (536 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.7/536.7 kB 57.4 MB/s eta 0:00:00
Collecting datrie==0.8.2 (from -r requirements.txt (line 21))
Downloading datrie-0.8.2.tar.gz (63 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.3/63.3 kB 9.7 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Collecting demjson==2.2.4 (from -r requirements.txt (line 22))
Downloading demjson-2.2.4.tar.gz (131 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 131.5/131.5 kB 17.2 MB/s eta 0:00:00
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
Preparing metadata (setup.py) ... error
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
pip install -r -r requirements.txt
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
Preparing metadata (setup.py) ... error
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
pip install -r -r requirements.txt will be successful
1. pip install -r requirements.txt
2. Error happened again.
Collecting accelerate==0.27.2 (from -r requirements.txt (line 1))
Using cached accelerate-0.27.2-py3-none-any.whl (279 kB)
Requirement already satisfied: aiohttp==3.9.3 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 2)) (3.9.3)
Requirement already satisfied: aiosignal==1.3.1 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 3)) (1.3.1)
Requirement already satisfied: annotated-types==0.6.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 4)) (0.6.0)
Collecting anyio==4.3.0 (from -r requirements.txt (line 5))
Using cached anyio-4.3.0-py3-none-any.whl (85 kB)
Requirement already satisfied: argon2-cffi==23.1.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 6)) (23.1.0)
Requirement already satisfied: argon2-cffi-bindings==21.2.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 7)) (21.2.0)
Collecting Aspose.Slides==24.2.0 (from -r requirements.txt (line 8))
Using cached Aspose.Slides-24.2.0-py3-none-manylinux1_x86_64.whl (88.7 MB)
Requirement already satisfied: attrs==23.2.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 9)) (23.2.0)
Collecting blinker==1.7.0 (from -r requirements.txt (line 10))
Using cached blinker-1.7.0-py3-none-any.whl (13 kB)
Collecting cachelib==0.12.0 (from -r requirements.txt (line 11))
Using cached cachelib-0.12.0-py3-none-any.whl (20 kB)
Requirement already satisfied: cachetools==5.3.3 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 12)) (5.3.3)
Requirement already satisfied: certifi==2024.2.2 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 13)) (2024.2.2)
Requirement already satisfied: cffi==1.16.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 14)) (1.16.0)
Requirement already satisfied: charset-normalizer==3.3.2 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 15)) (3.3.2)
Requirement already satisfied: click==8.1.7 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 16)) (8.1.7)
Collecting coloredlogs==15.0.1 (from -r requirements.txt (line 17))
Using cached coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)
Requirement already satisfied: cryptography==42.0.5 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 18)) (42.0.5)
Collecting dashscope==1.14.1 (from -r requirements.txt (line 19))
Using cached dashscope-1.14.1-py3-none-any.whl (1.2 MB)
Collecting datasets==2.17.1 (from -r requirements.txt (line 20))
Using cached datasets-2.17.1-py3-none-any.whl (536 kB)
Collecting datrie==0.8.2 (from -r requirements.txt (line 21))
Using cached datrie-0.8.2.tar.gz (63 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Collecting demjson==2.2.4 (from -r requirements.txt (line 22))
Using cached demjson-2.2.4.tar.gz (131 kB)
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
Preparing metadata (setup.py) ... error
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
main
c3b2a1
No response
Continually print these warning
No response
The first time you startup the system by:
docker compose up
No response
demo website create knowledge:hint : 102 Tenant not found.
No response
change language
No response
No response
No response
main
No response
None
No response
import an empty excel file into knowledge base.
No response
main
No response
Documents stop processing after uploading a PDF on demo.ragflow.io
No response
Upload a PDF on demo.ragflow.io
No response
pymysql.err.ProgrammingError: (1146, "Table 'rag_flow.knowledgebase' doesn't exist")
main
main
No response
I signed up on the demo site and uploaded pdf and docx files. They are both stuck at 0.62% for over 10 minutes now and not moving.
I would expect parsing to finish, I guess.
1. Create an account on https://demo.ragflow.io/
2. Upload a document
No response
The command "docker compose -f docker-compose-CN.yml up -d" can run normally, but when I execute the command " docker logs -f ragflow-server". The exception occurred. Has anyone encountered a similar situation before?
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/ragflow/rag/svr/task_broker.py", line 180, in
dispatch()
File "/ragflow/rag/svr/task_broker.py", line 64, in dispatch
rows = collect(tm)
^^^^^^^^^^^
File "/ragflow/rag/svr/task_broker.py", line 38, in collect
docs = DocumentService.get_newly_uploaded(tm)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3128, in inner
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/ragflow/api/db/services/document_service.py", line 101, in get_newly_uploaded
return list(docs.dicts())
^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 7243, in iter
self.execute()
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 2011, in inner
return method(self, database, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 2082, in execute
return self._execute(database)
^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 2255, in _execute
cursor = database.execute(self)
^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3299, in execute
return self.execute_sql(sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3289, in execute_sql
with exception_wrapper:
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3059, in exit
reraise(new_type, new_type(exc_value, *exc_args), traceback)
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 192, in reraise
raise value.with_traceback(tb)
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3291, in execute_sql
cursor.execute(sql, params or ())
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/cursors.py", line 153, in execute
result = self._query(query)
^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/cursors.py", line 322, in _query
conn.query(q)
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 558, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 822, in _read_query_result
result.read()
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 1200, in read
first_packet = self.connection._read_packet()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 772, in _read_packet
packet.raise_for_error()
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
peewee.ProgrammingError: (1146, "Table 'rag_flow.document' doesn't exist")
[WARNING] Load term.freq FAIL!
[WARNING] Load term.freq FAIL!
Fetching 9 files: 100%|██████████| 9/9 [00:00<00:00, 114044.52it/s]
Fetching 9 files: 100%|██████████| 9/9 [00:00<00:00, 26564.91it/s]
Traceback (most recent call last):
Traceback (most recent call last):
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3291, in execute_sql
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3291, in execute_sql
cursor.execute(sql, params or ())
cursor.execute(sql, params or ())
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/cursors.py", line 153, in execute
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/cursors.py", line 153, in execute
result = self._query(query)
result = self._query(query)
^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/cursors.py", line 322, in _query
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/cursors.py", line 322, in _query
conn.query(q)
conn.query(q)
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 558, in query
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 558, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 822, in _read_query_result
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 822, in _read_query_result
result.read()
result.read()
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 1200, in read
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 1200, in read
first_packet = self.connection._read_packet()
first_packet = self.connection._read_packet()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 772, in _read_packet
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 772, in _read_packet
packet.raise_for_error()
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/protocol.py", line 221, in raise_for_error
packet.raise_for_error()
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
err.raise_mysql_exception(self._data)
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1146, "Table 'rag_flow.task' doesn't exist")
main
No response
tail -f -n 100 logs/rag/es.log
Elasticsearch version: (8, 12, 1)
Fail to connect to es: Connection error caused by: ConnectionError(Connection error caused by: NameResolutionError(<urllib3.connection.HTTPConnection object at 0x7ffa7bbce100>: Failed to resolve 'es01' ([Errno -3] Temporary failure in name resolution)))
Fail to connect to es: Connection error caused by: ConnectionError(Connection error caused by: NameResolutionError(<urllib3.connection.HTTPConnection object at 0x7ffa7bbce790>: Failed to resolve 'es01' ([Errno -3] Temporary failure in name resolution)))
No response
After running the docker
No response
如题,可以自己自定义接入openAI或其他LLM的能力嘛
After deployed with the pre-built docker images and started up the ragflow server, I could successfully access the ragflow web page, but failed to parse any pdf file in the knowlege base.
All configurations follow the official configuration, except for the service port of Minio in the ./docker/docker-compose.yml
minio:
image: quay.io/minio/minio:RELEASE.2023-12-20T01-00-02Z
container_name: ragflow-minio
command: server --console-address ":9001" /data
ports:
- 19000:9000
- 19011:9001
environment:
- MINIO_ROOT_USER=${MINIO_USER}
ERROR msg:
ES updateByQuery deleteByQuery: NotFoundError(404, 'index_not_found_exception', 'no such index [ragflow_0eea0066f16411eeadae0242ac150006]', ragflow_0eea0066f16411eeadae0242ac150006, index_or_alias)【Q】:{'match': {'doc_id': '9f5aac32f18611eeb9eb0242ac150006'}}
Fail put 55d6b0f0f16411ee90d40242ac150006/xxxxxx_my_test_file.pdf: S3 operation failed; code: NoSuchKey, message: Object does not exist, resource: /55d6b0f0f16411ee90d40242ac150006/26-Tesla%20Model%20X%E8%AF%8A%E6%96%AD%E5%AF%B9%E6%A0%87%E6%8A%A5%E5%91%8A20171013.pdf, request_id: 17C2B3E1E8FCF95A, host_id: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8, bucket_name: 55d6b0f0f16411ee90d40242ac150006, object_name: -xxxxxx_my_test_file.pdf
main
This happens on both demo site and a local deployment instance.
on the page: https://demo.ragflow.io/knowledge/dataset?id=<...>
After added a dataset, and try to add text chunks to the dataset via the UI interface, the following error message is encoutered:
Possible issue is that the field 'create_time' in your index ragflow_15b4f374f2e011eeae1b0242ac180006 is a text field, and operations like sorting or aggregating require field data. However, field data is disabled by default on text fields to optimize performance.
BadRequestError(
"search_phase_execution_exception",
meta=ApiResponseMeta(
status=400,
http_version="1.1",
headers={
"X-elastic-product": "Elasticsearch",
"content-type": "application/vnd.elasticsearch+json;compatible-with=8",
"content-length": "2231",
},
duration=0.0018017292022705078,
node=NodeConfig(
scheme="http",
host="es01",
port=9200,
path_prefix="",
headers={
"user-agent": "elasticsearch-py/8.12.1 (Python/3.11.0; elastic-transport/8.12.0)"
},
connections_per_node=10,
request_timeout=10.0,
http_compress=False,
verify_certs=True,
ca_certs=None,
client_cert=None,
client_key=None,
ssl_assert_hostname=None,
ssl_assert_fingerprint=None,
ssl_version=None,
ssl_context=None,
ssl_show_warn=True,
_extras={},
),
),
body={
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "Fielddata is disabled on [create_time] in [ragflow_15b4f374f2e011eeae1b0242ac180006]. Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [create_time] in order to load field data by uninverting the inverted index. Note that this can use significant memory.",
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": True,
"failed_shards": [
{
"shard": 0,
"index": "ragflow_15b4f374f2e011eeae1b0242ac180006",
"node": "90aM0LzhTSqdYA-X6yX5mg",
"reason": {
"type": "illegal_argument_exception",
"reason": "Fielddata is disabled on [create_time] in [ragflow_15b4f374f2e011eeae1b0242ac180006]. Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [create_time] in order to load field data by uninverting the inverted index. Note that this can use significant memory.",
},
}
],
"caused_by": {
"type": "illegal_argument_exception",
"reason": "Fielddata is disabled on [create_time] in [ragflow_15b4f374f2e011eeae1b0242ac180006]. Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [create_time] in order to load field data by uninverting the inverted index. Note that this can use significant memory.",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "Fielddata is disabled on [create_time] in [ragflow_15b4f374f2e011eeae1b0242ac180006]. Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [create_time] in order to load field data by uninverting the inverted index. Note that this can use significant memory.",
},
},
},
"status": 400,
},
)
No response
Add a new dataset via the WebUI (successful)
Add a new chunk to the newly created dataset (error).
This happens on both official demo site and a local deployment testing environment.
No response
main
newest
No response
I have pulled the images successfully and do docker compose -f docker-compose-CN.yml up -d.
No response
[+] Running 6/8
⠿ Network docker_ragflow Created 0.1s
⠿ Container ragflow-es-01 Healthy 21.2s
⠿ Container ragflow-mysql Healthy 11.2s
⠿ Container ragflow-minio Started 1.7s
⠇ es01 Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap. 0.0s
⠿ Container ragflow-kibana Started 21.6s
⠿ Container ragflow-server Started 21.8s
⠇ kibana Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap. 0.0s
(base) lk@lk:/media/lk/disk1/lk_git/6_NLPandCNN/LLM/ragflow/docker$ docker logs -f ragflow-server
[HUQIE]:Build default trie
[HUQIE]:Build default trie
[HUQIE]:Build default trie
[HUQIE]:Build trie /ragflow/rag/res/huqie.txt
[HUQIE]:Build trie /ragflow/rag/res/huqie.txt
[HUQIE]:Build trie /ragflow/rag/res/huqie.txt
WARNING:root:Realtime synonym is disabled, since no redis connection.
WARNING:root:Realtime synonym is disabled, since no redis connection.
WARNING:root:Realtime synonym is disabled, since no redis connection.
[WARNING] Load term.freq FAIL!
pytorch_model.bin: 7%|▋ | 94.4M/1.30G [00:29<06:09, 3.27MB/s]WARNING:root:Realtime synonym is disabled, since no redis connection.
[WARNING] Load term.freq FAIL!
Traceback (most recent call last):
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3291, in execute_sql
cursor.execute(sql, params or ())
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/cursors.py", line 153, in execute
result = self._query(query)
^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/cursors.py", line 322, in _query
Anyone can helps ? Thanks!
### Additional information
![screenshot1](https://github.com/infiniflow/ragflow/assets/20237650/246876fb-4737-4066-bae1-57605561a678)
It shows {"data":null,"retcode":100,"retmsg":"<NotFound '404: Not Found'>"} in the website.
main
I've uploaded docs to the dataset, parsed and chunked successfully but testing the retrieval fails consistently. Using OpenAI model.
Error in top right - 'Index Not Found'
It should produce output from the LLM.
Test anything in retrieval testing.
No response
main
No response
No response
![image](https://github.com/infiniflow/ragflow/assets/8089971/3b77d8ce-fc78-4006-8c4c-1e25f7d29fa6)
No response
main
No response
For any type of file, if the parsing method is general, the chunk token number needs to be displayed.
![image](https://github.com/infiniflow/ragflow/assets/8089971/640577f4-a7ad-4394-a22c-4ab4db336491)
No response
Building the project truly from source would involve building all resources, including base image. Any chance ragflow-base
dockerfile could be included in repo?
local docker with latest image, the document process is blocked at 80%. LLM is ChatGLM and the API Key is set in the web ui.
Error log is:
Can't update token usage for 0196ca84f5a111ee80170242ac150006/EMBEDDING
Can't update token usage for 0196ca84f5a111ee80170242ac150006/EMBEDDING
ES create index error ragflow_0196ca84f5a111ee80170242ac150006 ----BadRequestError(400, 'resource_already_exists_exception', 'index [ragflow_0196ca84f5a111ee80170242ac150006/gHToUXxJSNSqdLG9Yo0mNA] already exists')
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '367e6c34f5a111ee9a840242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '367e6c34f5a111ee9a840242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '367e6c34f5a111ee9a840242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '367e6c34f5a111ee9a840242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '367e6c34f5a111ee9a840242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '367e6c34f5a111ee9a840242ac150006'}}
Fail put 0beab266f5a111eeab0c0242ac150006/附件1:《好烤漆金牌造》销售工具话术.pdf: S3 operation failed; code: NoSuchKey, message: Object does not exist, resource: /0beab266f5a111eeab0c0242ac150006/%E9%99%84%E4%BB%B61%EF%BC%9A%E3%80%8A%E5%A5%BD%E7%83%A4%E6%BC%86%E9%87%91%E7%89%8C%E9%80%A0%E3%80%8B%E9%94%80%E5%94%AE%E5%B7%A5%E5%85%B7%E8%AF%9D%E6%9C%AF.pdf, request_id: 17C44D75219FE3C5, host_id: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8, bucket_name: 0beab266f5a111eeab0c0242ac150006, object_name: 附件1:《好烤漆金牌造》销售工具话术.pdf
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '9b962552f5a211eea4b10242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '9b962552f5a211eea4b10242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '9b962552f5a211eea4b10242ac150005'}}
Can't update token usage for 0196ca84f5a111ee80170242ac150006/EMBEDDING
Can't update token usage for 0196ca84f5a111ee80170242ac150006/EMBEDDING
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '9b962552f5a211eea4b10242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '9b962552f5a211eea4b10242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '9b962552f5a211eea4b10242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '9b962552f5a211eea4b10242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '9b962552f5a211eea4b10242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '9b962552f5a211eea4b10242ac150005'}}
Fail put 8568867ef5a411eebc050242ac150005/附件1:《好烤漆金牌造》销售工具话术.pdf: S3 operation failed; code: NoSuchBucket, message: The specified bucket does not exist, resource: /8568867ef5a411eebc050242ac150005, request_id: 17C44E3BAF64631C, host_id: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8, bucket_name: 8568867ef5a411eebc050242ac150005
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
Can't update token usage for 0196ca84f5a111ee80170242ac150006/EMBEDDING
Can't update token usage for 0196ca84f5a111ee80170242ac150006/EMBEDDING
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】:{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
Can't update token usage for 0196ca84f5a111ee80170242ac150006/EMBEDDING
Can't update token usage for 0196ca84f5a111ee80170242ac150006/EMBEDDING
What might be the error?
main
No response
Configure the LLM as ChatGLM and chat, got
ERROR: Completions.create() got an unexpected keyword argument 'presence_penalty'
No response
Deployed local docker environment.
Create a knowledge base, uploading some docs.
Config ChatGLM as LLM and config the ApiKey.
Then create an assistant with ChatGLM, chat with it, the error will happen.
No response
ragflow is integrating with the OCR model of InfiniFlow/deepdoc. what's the performance of the text extraction and table structure extraction compare with the commercial OCR tools such as the text extraction of Azure and Aws.
Couldn't find any swagger api on first glance to use this locally with an external code base and just as a RAG engine.
main
pop_os 22.04
docker 26.0
Intel i7-12800h
32gb
(base) hitesh@whiskey:~/ragflow/docker$ docker logs -f ragflow-server
[HUQIE]:Build default trie
[HUQIE]:Build trie /ragflow/rag/res/huqie.txt
WARNING:root:Realtime synonym is disabled, since no redis connection.
[WARNING] Load term.freq FAIL!
____ ______ __
/ __ \ ____ _ ____ _ / // / _ __
/ // // __ // __
// / / // __ | | /| / /
/ , // // // // // / / // // /| |/ |/ /
// || _,/ _, /// // _/ |/|_/
/____/
ERROR:dashscope:Request: https://dashscope.aliyuncs.com/api/v1/services/aigc/text-generation/generation failed, status: 401, message: Invalid API-key provided.
INFO:werkzeug:WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
____ ______ __
/ __ \ ____ _ ____ _ / // / _ __
/ // // __ // __
// / / // __ | | /| / /
/ , // // // // // / / // // /| |/ |/ /
// || _,/ _, /// // _/ |/|_/
/____/
I followed the instructions for docker:
$ git clone https://github.com/infiniflow/ragflow.git
$ cd ragflow/docker
$ docker compose up -d
No response
main
c3b2d1
No response
I test on https://demo.ragflow.io/, upload a pdf file. Index failure every time.
Page(13~25): [ERROR]Index failure!
No response
I test on https://demo.ragflow.io/, upload a pdf file. Index failure every time.
Page(13~25): [ERROR]Index failure!
No response
Can somebody tell us how we can load a json(l) or csv file into the system?
main
newest
No response
No response
Just click here
https://github.com/infiniflow/ragflow?tab=readme-ov-file#-community
The Discord link not work.
No response
main
No response
All documents in the knowledge base cannot be selected if they have not been parsed.
![image](https://github.com/infiniflow/ragflow/assets/8089971/e580162d-149d-42ed-881d-7123beb35458)
![image](https://github.com/infiniflow/ragflow/assets/8089971/cea5d535-8613-4f3f-8cd7-b5b19d43ecea)
No response
请问如果精召回了3个chunk,都丢给大模型,最终是如何确认答案是基于哪个chunk回答的呢?
Local LLM, especially for LLAMA families should be easily integrated
Support ollama
No response
No response
No response
Hi.
It looks like the docker image and instructions are for Linux. I tried to run the docker compose on my M2 Mac, but I do get errors related to MySQL.
! mysql The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
runtime: failed to create new OS thread (have 2 already; errno=22)
### Describe the feature you'd like
Apple Silicone support and clear instructions how to install in Mac
### Describe implementation you've considered
I tried to add `platform: linux/amd64` to MySQL image deofinition in docker compose, but this didn't help.
### Documentation, adoption, use case
_No response_
### Additional information
_No response_
docker compose down -v docker compose up
above command doesn't work
We've observed that the size of the database.log file increases rapidly, reaching gigabytes in a very short span of time. By using tail -f to monitor the file, we noticed it generates numerous entries similar to the ones below. Is there a way to suppress these logs?
Returning 140199553734352 to pool.
('SELECT t1
.id
, t1
.doc_id
, t1
.from_page
, t1
.to_page
, t2
.kb_id
, t2
.parser_id
, t2
.parser_config
, t2
.name
, t2
.type
, t2
.location
, t2
.size
, t3
.tenant_id
, t3
.language
, t4
.embd_id
, t4
.img2txt_id
, t4
.asr_id
, t1
.update_time
FROM task
AS t1
INNER JOIN document
AS t2
ON (t1
.doc_id
= t2
.id
) INNER JOIN knowledgebase
AS t3
ON (t2
.kb_id
= t3
.id
) INNER JOIN tenant
AS t4
ON (t3
.tenant_id
= t4
.id
) WHERE ((((((t2
.status
= %s) AND (t2
.run
= %s)) AND NOT (t2
.type
= %s)) AND (t1
.progress
= %s)) AND (t1
.update_time
>= %s)) AND ((t1
.create_time
%% %s) = %s)) ORDER BY t1
.update_time
ASC LIMIT %s OFFSET %s', ['1', '1', 'virtual', 0.0, 1712592641735, 2, 1, 64, 0])
('SELECT t1
.id
, t1
.doc_id
, t1
.from_page
, t1
.to_page
, t2
.kb_id
, t2
.parser_id
, t2
.parser_config
, t2
.name
, t2
.type
, t2
.location
, t2
.size
, t3
.tenant_id
, t3
.language
, t4
.embd_id
, t4
.img2txt_id
, t4
.asr_id
, t1
.update_time
FROM task
AS t1
INNER JOIN document
AS t2
ON (t1
.doc_id
= t2
.id
) INNER JOIN knowledgebase
AS t3
ON (t2
.kb_id
= t3
.id
) INNER JOIN tenant
AS t4
ON (t3
.tenant_id
= t4
.id
) WHERE ((((((t2
.status
= %s) AND (t2
.run
= %s)) AND NOT (t2
.type
= %s)) AND (t1
.progress
= %s)) AND (t1
.update_time
>= %s)) AND ((t1
.create_time
%% %s) = %s)) ORDER BY t1
.update_time
ASC LIMIT %s OFFSET %s', ['1', '1', 'virtual', 0.0, 0, 2, 0, 64, 0])
Returning 139777349632080 to pool.
Returning 140199553734352 to pool.
('SELECT t1
.id
, t1
.doc_id
, t1
.from_page
, t1
.to_page
, t2
.kb_id
, t2
.parser_id
, t2
.parser_config
, t2
.name
, t2
.type
, t2
.location
, t2
.size
, t3
.tenant_id
, t3
.language
, t4
.embd_id
, t4
.img2txt_id
, t4
.asr_id
, t1
.update_time
FROM task
AS t1
INNER JOIN document
AS t2
ON (t1
.doc_id
= t2
.id
) INNER JOIN knowledgebase
AS t3
ON (t2
.kb_id
= t3
.id
) INNER JOIN tenant
AS t4
ON (t3
.tenant_id
= t4
.id
) WHERE ((((((t2
.status
= %s) AND (t2
.run
= %s)) AND NOT (t2
.type
= %s)) AND (t1
.progress
= %s)) AND (t1
.update_time
>= %s)) AND ((t1
.create_time
%% %s) = %s)) ORDER BY t1
.update_time
ASC LIMIT %s OFFSET %s', ['1', '1', 'virtual', 0.0, 1712592641735, 2, 1, 64, 0])
('SELECT t1
.id
, t1
.doc_id
, t1
.from_page
, t1
.to_page
, t2
.kb_id
, t2
.parser_id
, t2
.parser_config
, t2
.name
, t2
.type
, t2
.location
, t2
.size
, t3
.tenant_id
, t3
.language
, t4
.embd_id
, t4
.img2txt_id
, t4
.asr_id
, t1
.update_time
FROM task
AS t1
INNER JOIN document
AS t2
ON (t1
.doc_id
= t2
.id
) INNER JOIN knowledgebase
AS t3
ON (t2
.kb_id
= t3
.id
) INNER JOIN tenant
AS t4
ON (t3
.tenant_id
= t4
.id
) WHERE ((((((t2
.status
= %s) AND (t2
.run
= %s)) AND NOT (t2
.type
= %s)) AND (t1
.progress
= %s)) AND (t1
.update_time
>= %s)) AND ((t1
.create_time
%% %s) = %s)) ORDER BY t1
.update_time
ASC LIMIT %s OFFSET %s', ['1', '1', 'virtual', 0.0, 0, 2, 0, 64, 0])
Returning 140199553734352 to pool.
Returning 139777349632080 to pool.
No response
Add support for ollama
No response
No response
No response
After pip install -r requrement.txt, I just use python deepdoc/vision/t_ocr.py -h, it returned error messages below:
Traceback (most recent call last):
File "/deepdoc/vision/t_ocr.py", line 14, in
from deepdoc.vision.seeit import draw_box
ModuleNotFoundError: No module named 'deepdoc'
How i can use the deepdoc by cli?
DeepDoc对中文的支持怎么样,我看文档都是用的英文文档
system requirements:
hardward, operating system.
How to get ragflow from dockerhub
How to config ragflow
Community
Roadmap
License
If there is a pdf with 2 columns with headings and tables. I want to extract the text/OCR result separately for individual layout segments. How can I do it directly just by using deepdoc?
Fail put 29f4f2dcf21b11ee97630242c0a80006/AcademicGPT.pdf: S3 operation failed; code: NoSuchBucket, message: The specified bucket does not exist, resource: /29f4f2dcf21b11ee97630242c0a80006, request_id: 17C2EC907DD3BC46, host_id: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8, bucket_name: 29f4f2dcf21b11ee97630242c0a80006
Can't update token usage for d11309c4f0c111eea3da0242ac150005/EMBEDDING
Object of type ndarray is not JSON serializable
Traceback (most recent call last):
File "/ragflow/api/apps/conversation_app.py", line 172, in completion
ans = chat(dia, msg, **req)
^^^^^^^^^^^^^^^^^^^^^
File "/ragflow/api/apps/conversation_app.py", line 215, in chat
kbinfos = retrievaler.retrieval(" ".join(questions), embd_mdl, dialog.tenant_id, dialog.kb_ids, 1, dialog.top_n,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ragflow/rag/nlp/search.py", line 314, in retrieval
sres = self.search(req, index_name(tenant_id), embd_mdl)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ragflow/rag/nlp/search.py", line 115, in search
es_logger.info("【Q】: {}".format(json.dumps(s)))
^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/json/__init__.py", line 238, in dumps
**kw).encode(obj)
^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/json/encoder.py", line 200, in encode
chunks = self.iterencode(o, _one_shot=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/json/encoder.py", line 258, in iterencode
return _iterencode(o, 0)
^^^^^^^^^^^^^^^^^
File "/ragflow/api/utils/__init__.py", line 128, in default
return json.JSONEncoder.default(self, obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/json/encoder.py", line 180, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type ndarray is not JSON serializable
Can't update token usage for d11309c4f0c111eea3da0242ac150005/EMBEDDING
Object of type ndarray is not JSON serializable
How can I solved this?
When a user asks a question and there's no relevant content in the knowledge base, we can reply with a custom message.
For example:
Knowledge base: 1. Professional knowledge
User input: Hello, xxxxxx?
[No relevant content found]
Assistant output: Sorry, I'm unable to answer your question. You can submit a ticket at https://xxx.com.
When a user asks a question and there's no relevant content in the knowledge base, we can reply with a custom message.
No response
No response
No response
The README.md
show the way to set vm.max_map_count
in Linux. But how to set it in Windows?
main
No response
Refresh the login page and the language setting becomes invalid.
No response
Refresh the login page and the language setting becomes invalid.
No response
main
No response
As title describe
No response
Save knowledgebase configuration.
Load it again.
Embedding configuration dismissed.
No response
main
No response
README.md mentions:
RAGFlow flourishes via open-source collaboration. In this spirit, we embrace diverse contributions from the community. If you would like to be a part, review our Contribution Guidelines first.
But the main branch of ragflow does not contain CONTRIBUTING.md.
CONTRIBUTING.md should exist.
NA
No response
main
No response
I registered user yh, and have a chat as shown in pic1. After that, I registered a new user yh01 and found the history chat of user yh(pic2). I think it's a function bug.
No response
1.start the ragflow
2.config apikeys
3.registered user A and config model and start to chat
4.registered user B and start to chat
By the way, there still some error message in ERROR.log as follows,
Fail put 77dd584cf57a11eebdea0242ac190005/LOMO.pdf: S3 operation failed; code: NoSuchBucket, message: The specified bucket does not exist, resource: /77dd584cf57a11eebdea0242ac190005, request_id: 17C43DDAB4AB4F18, host_id: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8, bucket_name: 77dd584cf57a11eebdea0242ac190005
Can't update token usage for c4c74360f54e11ee863b0242ac190005/EMBEDDING
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.