Comments (7)
You can use JSON editor to add resource limits to image builder Containers. If this is not possible, consider switching to a different image build engine, such as buildkit
How to switch image build engine:
helm -n yatai-image-builder get values yatai-image-builder > ./values.yaml
helm -n yatai-image-builder upgrade yatai-image-builder --values ./values.yaml --set bentoImageBuildEngine=buildkit
from yatai.
Somehow the JSON Editor is not showing anything. Is it because we are in a non external internet access environment?
Anyways, we have switched the image build engine to buildkit, still the same OOM issue.
Besides, if we delete the deployment through the Web UI, and try to deploy the same model, it will give us image build failed immediately. And I checked the log, it was the last deployment error. Not sure if this is a bug or what. We have to delete the BentoRequest for the bento in order to create a new deployment for the same model version.
from yatai.
After changing the default limitrange of memory, we can control the resource of the image building container. However, the next step was block by an image named "quay.io/bentoml/bentoml-proxy:0.0.1". As we do not have external internet access in the developing environment, we pulled the image outside the developing environment. But it seems the image repository of this bentoml-proxy image is fixed. Not sure how to change the repository to our internal repository. It may helpful if you guys can show us how. Thank you!
from yatai.
@herunyu Thanks for your feedback, I just updated and released yatai-deployment and its helm chart, now you can specify a custom proxy image with this value, you can now update the helm repo and then update the yatai-deployment helm release to set this image
from yatai.
@herunyu I can demonstrate how to do this update.
First, update the helm repo:
helm repo update bentoml
Then save the previous values:
helm get values yatai-deployment -n yatai-deployment > /tmp/yatai-deployment-values.yaml
Final update on release:
helm -n yatai-deployment upgrade yatai-deployment bentoml/yatai-deployment --values /tmp/yatai-deployment-values.yaml --set internalImages.proxy=${your proxy image here}
from yatai.
@yetone Thank you! We will try this update and see if the problem is solved.
from yatai.
You can use JSON editor to add resource limits to image builder Containers. If this is not possible, consider switching to a different image build engine, such as
buildkit
![]()
![]()
How to switch image build engine:
helm -n yatai-image-builder get values yatai-image-builder > ./values.yaml helm -n yatai-image-builder upgrade yatai-image-builder --values ./values.yaml --set bentoImageBuildEngine=buildkit
Hello!
I am running to a similar issue and I didn't find the JSON editor in the webUI.
Could you please enlighten me? Thank you very much.
It's actually quite easy to get OOM for this setup especially with Transformers models.
I passed several hours on the documentation but I didn't find any thing about builder memory except for this page.
from yatai.
Related Issues (20)
- fix trans in BentoCloud HOT 1
- Error: kaniko(bentoImageBuildEngine) apt-get install error HOT 1
- Error: get Bento CR bentoes.resources.yatai.ai not found HOT 1
- Will yatai login ldap be supported? HOT 1
- Deploy a "custom Bentoml image" using "yatai-deployment".
- Support labels from values.yaml
- Error: YAML parse error on yatai/templates/deployment.yaml HOT 4
- Installation of yatai-deployment failed to [ERROR registerYataiComponent...] HOT 5
- Provide a way to disable home page cards
- Cannot access web ui HOT 1
- can not start dashboard when running yarn start HOT 3
- Autoscaling LLM HOT 2
- how to deploy with cutom-runner HOT 1
- Does yatai deployment require saving the model to the bentos model store? HOT 2
- Yatai 2.0 Proposal HOT 2
- Support BentoML 1.2 in Yatai HOT 1
- How can I deploy bentoDeployment with both remote runner and embedded runner?
- How can I host the yatai image in a private registry HOT 3
- Yatai memory leak HOT 7
- Request to set the root path of the Dashboard
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from yatai.