deployKF builds machine learning platforms on Kubernetes.
We combine the best of
Kubeflow,
Airflow†, and
MLflow†
into a complete platform that is easy to deploy and maintain.
deployKF combines the ease of a managed service with the flexibility of a self-hosted solution.
Our goal is that any Kubernetes user can build a machine learning platform for their organization,
without needing specialized MLOps knowledge, or a team of experts to maintain it.
deployKF is a new and growing project.
If you like what we are doing, please help others discover us by sharing the project with your colleagues and/or the wider community.
We greatly appreciate GitHub Stars ⭐ on the deployKF/deployKF repository:
Other Resources
Commercial Support
To discuss commercial support options for deployKF, please connect with Aranui Solutions, the company started by the creators of deployKF.
Learn more on the Aranui Solutions Website.
Community
The deployKF community uses the Kubeflow Slack for informal discussions among users and contributors.
deployKF was originally created and is maintained by Mathew Wicks (GitHub: @thesuperzapper), a Kubeflow lead and maintainer of the popular Apache Airflow Helm Chart.
deployKF is a community-led project that welcomes contributions from anyone who wants to help.
Could you please provide an example of how to provide a dex connector for oidc?
for example, the code has the following comments
## dex connectors
## - dex connectors which allow bridging trust to external identity providers
## https://dexidp.io/docs/connectors/
## - not all connector types support refresh tokens, notably "SAML 2.0" and "OAUTH 2.0" do not
## however, most providers support "OpenID Connect" which does support refresh tokens
## without refresh tokens, users will be forced to re-authenticate every `expiry.idToken` period
## - each element is a map with keys `type`, `id`, `name`, and `config` (which are the same aas upstream dex)
## additionally, `configExistingSecret` and `configExistingSecretKey` allow you to set `config`
## from a YAML-formatted string in a kubernetes secret
## - in most cases `config.redirectURI` will be set to "https://{DEPLOYKF_HOST}/dex/callback" (if port is 443)
##
connectors: []
I am very interesting to migrate some homemade CI/CD pipeline to use deployKF, and would like to know there is an example of values.yaml (the file provided to the cli tool using the --values command) available ?
I was able to get everything working if I put everything in the value file as plain text, however, I'm having a hard time using the secrets. I apologize if this is a basic question, but I'm having a hard time here.
Documentation has the following,
## NOTE:
## - the `configExistingSecretKey` key in the secret must
## contain a string of YAML that is formatted the same
## as the CONTENTS of the `config` map key above
#configExistingSecret: "my-dex-connector-secret"
#configExistingSecretKey: "google-config"
Add userNameKey: cognito:username to the AWS Cognito provider example here otherwise you cannot login. Here is someone's explanation argoproj/argo-cd#12185 (comment).
Also maybe remove the offline_access scope? I don't know what that is and it errors with "invalid scope" when I add it.
This is a documentation enhancement request asking to add the following information into the official deployKF docs to help other users. The content is written below, inline using markdown as I could not find a way to contribute to the docs directly and this information is not otherwise available to my knowledge.
I wasted alot of time figuring this out and getting it working. It should be written somewhere.
Configuring DeployKF Plugin on ArgoCD for Self Signed or Custom SSL Certificates
This Guide is for DevOps admins who may be deploying in an On Prem datacenter or potentially those with custom SSL configurations in a public cloud VPC
The deployKF plugin downloads dependencies using a sidecar container that runs inside the argocd-repo-server pod. Both the argocd server and sidecar container need to have SSL certs added into their truststore chains to properly authenticate with a self signed/custom new certificate when executing the sync process with argocd. This method has been tested using the argocd sync script.
1.) Store downloaded pem certs as configmap
Using the below command we can load the contents of the certificate we need to add into Kubernetes in a persistent way:
2.) Patch the argocd repo server deployment to include the cert in all it's container images as a volume mount into the container level default truststore (/etc/ssl)
Fill in the yaml spec below to match your configmap name/key and set desired mount paths/attributes:
Convert to JSON, minify and use kubectl patch to make the deployment patch persist through the lifecycle of the pods ensuring your SSL cert is always in the chain when a resource needs to be synced
Validate by using curl from within the deploykf plugin sidecar container to reach github or custom repo/resource. Default ssl store is /etc/ssl in both containers but only the plugin sidecar has curl installed. Sync your apps as normal after this change.
Proxy variables can be added to all containers in the repo server pod using the following syntax if needed, though be sure to set this BEFORE applying the app of apps yaml file to prevent sync inconsistencies:
kubectl -n argocd set env deployment/argocd-repo-server HTTP_PROXY=http://10.0.0.0:8080 HTTPS_PROXY=http://10.0.0.0:8080 NO_PROXY=argocd-repo-server,argocd-application-controller,argocd-metrics,argocd-server,argocd-server-metrics,argocd-redis,argocd-dex-server,mydomain.com
Kubeflow,
Airflow†, and
MLflow† into a complete platform that is easy to deploy and maintain.
Istio,
cert-manager,
Kyverno,
S3, and
MySQL
Title: deployKF: A better way to deploy Kubeflow (and more)
Aranui Solutions, the company started by the creators of deployKF. Learn more on the Aranui Solutions Website.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent