Git Product home page Git Product logo

Comments (7)

janljd avatar janljd commented on July 3, 2024 1

it works ,thx

from yatai.

yetone avatar yetone commented on July 3, 2024

Thanks for your report! Can you show me what the helm release values are for your yatai-image-builder?

helm -n ${namespace} get values yatai-image-builder

from yatai.

janljd avatar janljd commented on July 3, 2024

helm -n hk-ai-test get values yatai-image-builder

WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /Users/houfei/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /Users/houfei/.kube/config
USER-SUPPLIED VALUES:
aws:
  accessKeyID: ""
  secretAccessKeyExistingSecretKey: ""
  secretAccessKeyExistingSecretName: ""
dockerRegistry:
  bentoRepositoryName: yatai-bentos
  inClusterServer: docker-registry.hk-ai-test.svc.cluster.local:5000
  password: ""
  secure: false
  server: 127.0.0.1:5000
  username: ""
yatai:
  endpoint: http://${yatai-host}
yataiSystem:
  namespace: hk-ai-test

helm -n hk-ai-test get values yatai-deployment

WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /Users/houfei/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /Users/houfei/.kube/config
USER-SUPPLIED VALUES:
bentoDeploymentNamespaces:
- hk-ai-test
enableRestrictedSecurityContext: true
layers:
  network:
    automaticDomainSuffixGeneration: false
    domainSuffix: ""
    ingressClass: nginx-aws-private
yatai:
  endpoint: http://${yatai-host}
yataiServiceAccount:
  namespace: hk-ai-test
yataiSystem:
  namespace: hk-ai-test

from yatai.

yetone avatar yetone commented on July 3, 2024

Can you show me the actual env in yatai-image-buider and yatai-deployment?

kubectl -n ${yatai-image-builder-namespace} get secret yatai-image-builder-env -o jsonpath='{.data.YATAI_SYSTEM_NAMESPACE}' | base64 -d

kubectl -n ${yatai-deployment-namespace} get secret yatai-deployment-env -o jsonpath='{.data.YATAI_SYSTEM_NAMESPACE}' | base64 -d

If the env is hk-ai-test, you should restart yatai-image-builder and yatai-deployment:

kubectl -n ${yatai-image-builder-namespace} rollout restart deploy/yatai-image-builder

kubectl -n ${yatai-deployment-namespace} rollout restart deploy/yatai-deployment

from yatai.

janljd avatar janljd commented on July 3, 2024

kubectl -n hk-ai-test get secret yatai-image-builder-env -o jsonpath='{.data.YATAI_SYSTEM_NAMESPACE}' | base64 -d
hk-ai-test

kubectl -n hk-ai-test get secret yatai-deployment-env -o jsonpath='{.data.YATAI_SYSTEM_NAMESPACE}' | base64 -d
hk-ai-test

kubectl -n hk-ai-test rollout restart deploy/yatai-image-builder
deployment.apps/yatai-image-builder restarted

kubectl -n hk-ai-test rollout restart deploy/yatai-deployment
deployment.apps/yatai-deployment restarted

yatai-image-builder error log

Version: 1.1.3
GitCommit: ea82e2b
BuildDate: 2023-01-17T10:02:40Z
1.6777302369828238e+09	INFO	setup	starting manager
1.6777302369829607e+09	INFO	getting yatai client	{"func": "doRegisterYataiComponent"}
1.6777302369836986e+09	INFO	Starting server	{"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"}
1.677730236983708e+09	INFO	Starting server	{"kind": "health probe", "addr": "[::]:8081"}
I0302 04:10:36.983933       1 leaderelection.go:248] attempting to acquire leader lease hk-ai-test/58b75536.yatai.ai...
time="2023-03-02T04:10:37Z" level=error msg="POST http://${yatai-host}/api/v1/clusters/bento/yatai_components status=403, {\"message\":\"get api token: failed to get yatai config in cluster bento in organization default: secret yatai-common-env not found in namespace yatai-system: secrets \\\"yatai-common-env\\\" not found\"}"
1.6777302370496786e+09	ERROR	registerYataiComponent	{"func": "registerYataiComponent", "error": "register yatai component: DoJsonRequest Error: [POST]http://${yatai-host}/api/v1/clusters/bento/yatai_components: POST http://${yatai-host}/api/v1/clusters/bento/yatai_components status=403, {\"message\":\"get api token: failed to get yatai config in cluster bento in organization default: secret yatai-common-env not found in namespace yatai-system: secrets \\\"yatai-common-env\\\" not found\"}", "errorVerbose": "POST http://${yatai-host}/api/v1/clusters/bento/yatai_components status=403, {\"message\":\"get api token: failed to get yatai config in cluster bento in organization default: secret yatai-common-env not found in namespace yatai-system: secrets \\\"yatai-common-env\\\" not found\"}\ngithub.com/bentoml/yatai-common/reqcli.(*JsonRequestBuilder).Do\n\t/go/pkg/mod/github.com/bentoml/[email protected]/reqcli/http.go:193\ngithub.com/bentoml/yatai-image-builder/yatai-client.(*YataiClient).RegisterYataiComponent\n\t/workspace/yatai-client/client.go:152\ngithub.com/bentoml/yatai-image-builder/controllers/resources.(*BentoRequestReconciler).doRegisterYataiComponent\n\t/workspace/controllers/resources/bentorequest_controller.go:1861\ngithub.com/bentoml/yatai-image-builder/controllers/resources.(*BentoRequestReconciler).registerYataiComponent\n\t/workspace/controllers/resources/bentorequest_controller.go:1882\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594\nDoJsonRequest Error: [POST]http://${yatai-host}/api/v1/clusters/bento/yatai_components\ngithub.com/bentoml/yatai-common/reqcli.(*JsonRequestBuilder).Do.func1\n\t/go/pkg/mod/github.com/bentoml/[email protected]/reqcli/http.go:128\ngithub.com/bentoml/yatai-common/reqcli.(*JsonRequestBuilder).Do\n\t/go/pkg/mod/github.com/bentoml/[email protected]/reqcli/http.go:194\ngithub.com/bentoml/yatai-image-builder/yatai-client.(*YataiClient).RegisterYataiComponent\n\t/workspace/yatai-client/client.go:152\ngithub.com/bentoml/yatai-image-builder/controllers/resources.(*BentoRequestReconciler).doRegisterYataiComponent\n\t/workspace/controllers/resources/bentorequest_controller.go:1861\ngithub.com/bentoml/yatai-image-builder/controllers/resources.(*BentoRequestReconciler).registerYataiComponent\n\t/workspace/controllers/resources/bentorequest_controller.go:1882\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594\nregister yatai component\ngithub.com/bentoml/yatai-image-builder/controllers/resources.(*BentoRequestReconciler).doRegisterYataiComponent\n\t/workspace/controllers/resources/bentorequest_controller.go:1876\ngithub.com/bentoml/yatai-image-builder/controllers/resources.(*BentoRequestReconciler).registerYataiComponent\n\t/workspace/controllers/resources/bentorequest_controller.go:1882\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594"}
github.com/bentoml/yatai-image-builder/controllers/resources.(*BentoRequestReconciler).registerYataiComponent
	/workspace/controllers/resources/bentorequest_controller.go:1884
1.6777302616634731e+09	DEBUG	events	yatai-image-builder-59f757d555-cjltk_b486d0ca-cfdf-4a48-9114-1c588498a195 became leader	{"type": "Normal", "object": {"kind":"Lease","namespace":"hk-ai-test","name":"58b75536.yatai.ai","uid":"b203a26b-240d-4bfa-a0da-114e7ecee122","apiVersion":"coordination.k8s.io/v1","resourceVersion":"585834952"}, "reason": "LeaderElection"}
I0302 04:11:01.664956       1 leaderelection.go:258] successfully acquired lease hk-ai-test/58b75536.yatai.ai

yatai-deployment error log

Version: 1.1.8
GitCommit: 0bd8ecf
BuildDate: 2023-02-14T08:16:37Z
1.6777302431243255e+09	INFO	controller-runtime.builder	skip registering a mutating webhook, object does not implement admission.Defaulter or WithDefaulter wasn't called	{"GVK": "serving.yatai.ai/v1alpha2, Kind=BentoDeployment"}
1.6777302431243482e+09	INFO	controller-runtime.builder	skip registering a validating webhook, object does not implement admission.Validator or WithValidator wasn't called	{"GVK": "serving.yatai.ai/v1alpha2, Kind=BentoDeployment"}
1.6777302431244867e+09	INFO	start cleaning up abandoned runner services	{"func": "doCleanUpAbandonedRunnerServices"}
1.6777302431245127e+09	INFO	getting yatai client	{"func": "doRegisterYataiComponent"}
1.6777302431249635e+09	INFO	controller-runtime.webhook	Registering webhook	{"path": "/convert"}
1.6777302431250212e+09	INFO	controller-runtime.builder	Conversion webhook enabled	{"GVK": "serving.yatai.ai/v1alpha2, Kind=BentoDeployment"}
1.6777302431250324e+09	INFO	setup	starting manager
1.6777302431252067e+09	INFO	controller-runtime.webhook.webhooks	Starting webhook server
1.6777302431252685e+09	INFO	Starting server	{"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"}
1.6777302431253283e+09	INFO	Starting server	{"kind": "health probe", "addr": "[::]:8081"}
I0302 04:10:43.125364       1 leaderelection.go:248] attempting to acquire leader lease hk-ai-test/b292d523.yatai.ai...
1.6777302431254594e+09	INFO	controller-runtime.certwatcher	Updated current TLS certificate
1.677730243125541e+09	INFO	controller-runtime.webhook	Serving webhook server	{"host": "", "port": 9443}
1.6777302431255946e+09	INFO	controller-runtime.certwatcher	Starting certificate watcher
time="2023-03-02T04:10:43Z" level=error msg="POST http://${yatai-host}/api/v1/clusters/bento/yatai_components status=403, {\"message\":\"get api token: failed to get yatai config in cluster bento in organization default: secret yatai-common-env not found in namespace yatai-system: secrets \\\"yatai-common-env\\\" not found\"}"
1.677730243200576e+09	ERROR	registerYataiComponent	{"func": "registerYataiComponent", "error": "DoJsonRequest Error: [POST]http://${yatai-host}/api/v1/clusters/bento/yatai_components: POST http://${yatai-host}/api/v1/clusters/bento/yatai_components status=403, {\"message\":\"get api token: failed to get yatai config in cluster bento in organization default: secret yatai-common-env not found in namespace yatai-system: secrets \\\"yatai-common-env\\\" not found\"}", "errorVerbose": "POST http://${yatai-host}/api/v1/clusters/bento/yatai_components status=403, {\"message\":\"get api token: failed to get yatai config in cluster bento in organization default: secret yatai-common-env not found in namespace yatai-system: secrets \\\"yatai-common-env\\\" not found\"}\ngithub.com/bentoml/yatai-common/reqcli.(*JsonRequestBuilder).Do\n\t/go/pkg/mod/github.com/bentoml/[email protected]/reqcli/http.go:193\ngithub.com/bentoml/yatai-deployment/yatai-client.(*YataiClient).RegisterYataiComponent\n\t/workspace/yatai-client/client.go:145\ngithub.com/bentoml/yatai-deployment/controllers.(*BentoDeploymentReconciler).doRegisterYataiComponent\n\t/workspace/controllers/bentodeployment_controller.go:2979\ngithub.com/bentoml/yatai-deployment/controllers.(*BentoDeploymentReconciler).registerYataiComponent\n\t/workspace/controllers/bentodeployment_controller.go:2999\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594\nDoJsonRequest Error: [POST]http://${yatai-host}/api/v1/clusters/bento/yatai_components\ngithub.com/bentoml/yatai-common/reqcli.(*JsonRequestBuilder).Do.func1\n\t/go/pkg/mod/github.com/bentoml/[email protected]/reqcli/http.go:128\ngithub.com/bentoml/yatai-common/reqcli.(*JsonRequestBuilder).Do\n\t/go/pkg/mod/github.com/bentoml/[email protected]/reqcli/http.go:194\ngithub.com/bentoml/yatai-deployment/yatai-client.(*YataiClient).RegisterYataiComponent\n\t/workspace/yatai-client/client.go:145\ngithub.com/bentoml/yatai-deployment/controllers.(*BentoDeploymentReconciler).doRegisterYataiComponent\n\t/workspace/controllers/bentodeployment_controller.go:2979\ngithub.com/bentoml/yatai-deployment/controllers.(*BentoDeploymentReconciler).registerYataiComponent\n\t/workspace/controllers/bentodeployment_controller.go:2999\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594"}
github.com/bentoml/yatai-deployment/controllers.(*BentoDeploymentReconciler).registerYataiComponent
	/workspace/controllers/bentodeployment_controller.go:3001

from yatai.

yetone avatar yetone commented on July 3, 2024

@janljd Thank you very much for reporting the bug. I have fixed it and released a new version. You need to update yatai to version 1.1.7.

from yatai.

linyqh avatar linyqh commented on July 3, 2024

The latest version of yatai is 1.1.13, and the same error has occurred again.

from yatai.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.