[Bug]: docker-compose failed! about ragflow HOT 23 CLOSED

Btw, what is the docker / docker-compose version is expected?
In my environment, cannot understand the

include:
  - path: ./docker-compose-base.yml
    env_file: ./.env

inside docker-compose.yml
I have to merge the yml files manually.
I'm on Mac.

from ragflow.

Same error here, and I noticed no tables are created in mysql. And from the log: raise Exception("Can't connect to ES cluster") Exception: Can't connect to ES cluster

looks like connection to ES is not setup too.

Did you set sysctl -w vm.max_map_count=262144 in the docker host ?

from ragflow.

+1

from ragflow.

1+
Not getting it in the logs but getting the same outcome in the browser when I hit the url

from ragflow.

[HUQIE]:Build trie /ragflow/rag/res/huqie.txt
[HUQIE]:Build trie /ragflow/rag/res/huqie.txt
[HUQIE]:Build trie /ragflow/rag/res/huqie.txt
WARNING:root:Realtime synonym is disabled, since no redis connection.
WARNING:root:Realtime synonym is disabled, since no redis connection.
WARNING:root:Realtime synonym is disabled, since no redis connection.
[WARNING] Load term.freq FAIL!
pytorch_model.bin: 34%|███▍ | 440M/1.30G [00:22<00:36, 23.9MB/s]WARNING:root:Realtime synonym is disabled, since no redis connection.
pytorch_model.bin: 100%|██████████| 1.30G/1.30G [01:06<00:00, 19.7MB/s]
[WARNING] Load term.freq FAIL!
[WARNING] Load term.freq FAIL!

from ragflow.

Is there an existing issue for the same bug?

• I have checked the existing issues.

Branch name

main

Commit ID

newest

Other environment information

No response

Actual behavior

I have pulled the images successfully and do docker compose -f docker-compose-CN.yml up -d.

Expected behavior

No response

Steps to reproduce

[+] Running 6/8
 ⠿ Network docker_ragflow                                                                                                 Created                                                                       0.1s
 ⠿ Container ragflow-es-01                                                                                                Healthy                                                                      21.2s
 ⠿ Container ragflow-mysql                                                                                                Healthy                                                                      11.2s
 ⠿ Container ragflow-minio                                                                                                Started                                                                       1.7s
 ⠇ es01 Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.                                                                                 0.0s
 ⠿ Container ragflow-kibana                                                                                               Started                                                                      21.6s
 ⠿ Container ragflow-server                                                                                               Started                                                                      21.8s
 ⠇ kibana Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.                                                                               0.0s
(base) lk@lk:/media/lk/disk1/lk_git/6_NLPandCNN/LLM/ragflow/docker$ docker logs -f ragflow-server
[HUQIE]:Build default trie
[HUQIE]:Build default trie
[HUQIE]:Build default trie
[HUQIE]:Build trie /ragflow/rag/res/huqie.txt
[HUQIE]:Build trie /ragflow/rag/res/huqie.txt
[HUQIE]:Build trie /ragflow/rag/res/huqie.txt



WARNING:root:Realtime synonym is disabled, since no redis connection.
WARNING:root:Realtime synonym is disabled, since no redis connection.
WARNING:root:Realtime synonym is disabled, since no redis connection.
[WARNING] Load term.freq FAIL!
pytorch_model.bin:   7%|▋         | 94.4M/1.30G [00:29<06:09, 3.27MB/s]WARNING:root:Realtime synonym is disabled, since no redis connection.
[WARNING] Load term.freq FAIL!
Traceback (most recent call last):
  File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3291, in execute_sql
    cursor.execute(sql, params or ())
  File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/cursors.py", line 153, in execute
    result = self._query(query)
             ^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/cursors.py", line 322, in _query

Anyone can helps ? Thanks!

### Additional information

![screenshot1](https://github.com/infiniflow/ragflow/assets/20237650/246876fb-4737-4066-bae1-57605561a678)

It shows {"data":null,"retcode":100,"retmsg":"<NotFound '404: Not Found'>"} in the website.

Try the solution this issue used.
#226

from ragflow.

1+ Not getting it in the logs but getting the same outcome in the browser when I hit the url

Figure out the true IP your machine use.

For windows:
ipconfig

For linux:
ifconfig

from ragflow.

I guess there is something else

from ragflow.

+1

from ragflow.

me too

from ragflow.

Same error here, and I noticed no tables are created in mysql.
And from the log:
raise Exception("Can't connect to ES cluster")
Exception: Can't connect to ES cluster

looks like connection to ES is not setup too.

from ragflow.

When I run the command "docker logs -f ragflow-server
", the next exception occurred, how to solve it?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/ragflow/rag/svr/task_broker.py", line 180, in
dispatch()
File "/ragflow/rag/svr/task_broker.py", line 64, in dispatch
rows = collect(tm)
^^^^^^^^^^^
File "/ragflow/rag/svr/task_broker.py", line 38, in collect
docs = DocumentService.get_newly_uploaded(tm)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3128, in inner
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/ragflow/api/db/services/document_service.py", line 101, in get_newly_uploaded
return list(docs.dicts())
^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 7243, in iter
self.execute()
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 2011, in inner
return method(self, database, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 2082, in execute
return self._execute(database)
^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 2255, in _execute
cursor = database.execute(self)
^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3299, in execute
return self.execute_sql(sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3289, in execute_sql
with exception_wrapper:
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3059, in exit
reraise(new_type, new_type(exc_value, *exc_args), traceback)
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 192, in reraise
raise value.with_traceback(tb)
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3291, in execute_sql
cursor.execute(sql, params or ())
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/cursors.py", line 153, in execute
result = self._query(query)
^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/cursors.py", line 322, in _query
conn.query(q)
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 558, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 822, in _read_query_result
result.read()
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 1200, in read
first_packet = self.connection._read_packet()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 772, in _read_packet
packet.raise_for_error()
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
peewee.ProgrammingError: (1146, "Table 'rag_flow.document' doesn't exist")
pytorch_model.bin: 27%|██▋ | 346M/1.30G [03:20<12:38, 1.26MB/s]WARNING:root:Realtime synonym is disabled, since no redis connection.
Traceback (most recent call last):
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3291, in execute_sql
[WARNING] Load term.freq FAIL!
cursor.execute(sql, params or ())
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/cursors.py", line 153, in execute
result = self._query(query)
^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/cursors.py", line 322, in _query
conn.query(q)
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 558, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 822, in _read_query_result
result.read()
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 1200, in read
first_packet = self.connection._read_packet()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 772, in _read_packet
packet.raise_for_error()
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1146, "Table 'rag_flow.document' doesn't exist")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/ragflow/rag/svr/task_broker.py", line 180, in
dispatch()
File "/ragflow/rag/svr/task_broker.py", line 64, in dispatch
rows = collect(tm)
^^^^^^^^^^^
File "/ragflow/rag/svr/task_broker.py", line 38, in collect
docs = DocumentService.get_newly_uploaded(tm)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3128, in inner
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/ragflow/api/db/services/document_service.py", line 101, in get_newly_uploaded
return list(docs.dicts())
^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 7243, in iter
self.execute()
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 2011, in inner
return method(self, database, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 2082, in execute
return self._execute(database)
^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 2255, in _execute
cursor = database.execute(self)
^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3299, in execute
return self.execute_sql(sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3289, in execute_sql
with exception_wrapper:
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3059, in exit
reraise(new_type, new_type(exc_value, *exc_args), traceback)
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 192, in reraise
raise value.with_traceback(tb)
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3291, in execute_sql
cursor.execute(sql, params or ())
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/cursors.py", line 153, in execute
result = self._query(query)
^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/cursors.py", line 322, in _query
conn.query(q)
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 558, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 822, in _read_query_result
result.read()
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 1200, in read
first_packet = self.connection._read_packet()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 772, in _read_packet
packet.raise_for_error()
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
peewee.ProgrammingError: (1146, "Table 'rag_flow.document' doesn't exist")
WARNING:root:Realtime synonym is disabled, since no redis connection.
[WARNING] Load term.freq FAIL!
Traceback (most recent call last):
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/peewee.py", line 3291, in execute_sql
cursor.execute(sql, params or ())
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/cursors.py", line 153, in execute
result = self._query(query)
^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/cursors.py", line 322, in _query
conn.query(q)
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 558, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 822, in _read_query_result
result.read()
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 1200, in read
first_packet = self.connection._read_packet()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/connections.py", line 772, in _read_packet
packet.raise_for_error()
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/root/miniconda3/envs/py11/lib/python3.11/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1146, "Table 'rag_flow.document' doesn't exist")

from ragflow.

in my case ,it keeps complaint :

[WARNING] Load term.freq FAIL!
    from rag.app import laws, paper, presentation, manual, qa, table, book, resume, picture, naive, one
  File "/ragflow/rag/app/laws.py", line 20, in <module>
    from deepdoc.parser import PdfParser, DocxParser, PlainParser
  File "/ragflow/deepdoc/parser/__init__.py", line 3, in <module>
    from .pdf_parser import HuParser as PdfParser, PlainParser
  File "/ragflow/deepdoc/parser/pdf_parser.py", line 15, in <module>
    from PyPDF2 import PdfReader as pdf2_read
ModuleNotFoundError: No module named 'PyPDF2'

even thought I have install it manally :

docker exec  -it  ragflow-server  /bin/bash
pip install PyPDF2
docker restart   ragflow-server  

and fail to sign up

from ragflow.

Same error here, and I noticed no tables are created in mysql. And from the log: raise Exception("Can't connect to ES cluster") Exception: Can't connect to ES cluster
looks like connection to ES is not setup too.

Did you set sysctl -w vm.max_map_count=262144 in the docker host ?

Thanks, the platform is Winfows and it does not support the command set sysctl -w vm.max_map_count=262144, what should I do to solve it

from ragflow.

Btw, what is the docker / docker-compose version is expected? In my environment, cannot understand the

include:
  - path: ./docker-compose-base.yml
    env_file: ./.env

inside docker-compose.yml I have to merge the yml files manually. I'm on Mac.

Version: 25.0.3

from ragflow.

in my case ,it keeps complaint :

[WARNING] Load term.freq FAIL!
    from rag.app import laws, paper, presentation, manual, qa, table, book, resume, picture, naive, one
  File "/ragflow/rag/app/laws.py", line 20, in <module>
    from deepdoc.parser import PdfParser, DocxParser, PlainParser
  File "/ragflow/deepdoc/parser/__init__.py", line 3, in <module>
    from .pdf_parser import HuParser as PdfParser, PlainParser
  File "/ragflow/deepdoc/parser/pdf_parser.py", line 15, in <module>
    from PyPDF2 import PdfReader as pdf2_read
ModuleNotFoundError: No module named 'PyPDF2'

even thought I have install it manally :

docker exec  -it  ragflow-server  /bin/bash
pip install PyPDF2
docker restart   ragflow-server  

and fail to sign up

pull the latest images

from ragflow.

Same error here, and I noticed no tables are created in mysql. And from the log: raise Exception("Can't connect to ES cluster") Exception: Can't connect to ES cluster

looks like connection to ES is not setup too.

chang ES_PORT=1200 to ES_PORT=9200 can solve this problem

you can see it curl the url in docker-compose-base.yml
healthcheck:
test: ["CMD-SHELL", "curl http://localhost:9200"]
so 9200 is the right port rather than 1200

from ragflow.

This problem still exists.

MacOS 13.5.2 (22G91)
Apple M2 Pro

from ragflow.

I have changed ES_PORT=1200 to ES_PORT=9200, but it still occurs the above exception. The configuration of ES_PORT is as follows:

from ragflow.

[HUQIE]:Build trie /ragflow/rag/res/huqie.txt [HUQIE]:Build trie /ragflow/rag/res/huqie.txt [HUQIE]:Build trie /ragflow/rag/res/huqie.txt WARNING:root:Realtime synonym is disabled, since no redis connection. WARNING:root:Realtime synonym is disabled, since no redis connection. WARNING:root:Realtime synonym is disabled, since no redis connection. [WARNING] Load term.freq FAIL! pytorch_model.bin: 34%|███▍ | 440M/1.30G [00:22<00:36, 23.9MB/s]WARNING:root:Realtime synonym is disabled, since no redis connection. pytorch_model.bin: 100%|██████████| 1.30G/1.30G [01:06<00:00, 19.7MB/s] [WARNING] Load term.freq FAIL! [WARNING] Load term.freq FAIL!

Hello, bro, has this problem been solved? I have the same problem as you. I hope you can give me some helpful advice. Thank you！

from ragflow.

同样的问题，

from ragflow.

为什么作者将这个bug关闭了，这显然是不负责任的开源项目

from ragflow.

problem too:

[1191261511eb:00023] [[INVALID],INVALID] ORTE_ERROR_LOG: The system limit on number of children a process can have was reached in file ../../../../../../orte/mca/ess/singleton/ess_singleton_module.c at line 633
[1191261511eb:00023] [[INVALID],INVALID] ORTE_ERROR_LOG: The system limit on number of children a process can have was reached in file ../../../../../../orte/mca/ess/singleton/ess_singleton_module.c at line 172
[1191261511eb:00022] [[INVALID],INVALID] ORTE_ERROR_LOG: The system limit on number of children a process can have was reached in file ../../../../../../orte/mca/ess/singleton/ess_singleton_module.c at line 633
[1191261511eb:00022] [[INVALID],INVALID] ORTE_ERROR_LOG: The system limit on number of children a process can have was reached in file ../../../../../../orte/mca/ess/singleton/ess_singleton_module.c at line 172
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  orte_ess_init failed
  --> Returned value The system limit on number of children a process can have was reached (-119) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  orte_ess_init failed
  --> Returned value The system limit on number of children a process can have was reached (-119) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems.  This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):

  ompi_mpi_init: ompi_rte_init failed
  --> Returned "The system limit on number of children a process can have was reached" (-119) instead of "Success" (0)
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems.  This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):

  ompi_mpi_init: ompi_rte_init failed
  --> Returned "The system limit on number of children a process can have was reached" (-119) instead of "Success" (0)
--------------------------------------------------------------------------
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)

from ragflow.