Yatai memory leak about yatai HOT 7 OPEN

I've the same problem, even though I asked in the slack channel I couldn't get an answer.

I’m using image: quay.io/bentoml/yatai:1.1.13 but its keep getting oom killed in my cluster.

from yatai.

Hi, can you provide the yatai version, also 1.1.13? @a-pichard

from yatai.

Yes i am running yatai 1.1.13

from yatai.

It's been 3 weeks, is there anything that you can give us to understand the reason and possibly how to fix ?

from yatai.

Sorry for the late reply. According to the release notes, I don't think it was introduced in 1.1.13, probably an older version, since the 1.1.13 version only includes a minor fix in helm chart.
If you can provide the version without the memory leak problem will be helpful to find the root cause.

from yatai.

I downgraded to 1.1.11 and memory leak still continues. I checked the processes running inside of the container and this was the only one "/app/api-server serve -c /conf/config.yaml" where it consistently reaches the 3Gi memory limit and gets OOM killed, without any significant increase in workload. The application configuration and Kubernetes setup are standard, with memory limits set as expected. Could you please help identify what might be causing this memory usage spike?

from yatai.

/app/api-server serve is actually the entrypoint of yatai backend.
cc @yetone

from yatai.