Comments (4)
Forced to use Chart version 1.2.7 to make it work
Fun fact that this is the only working version (1.2.5 also has timeout)
Do you have any problems with 1.2.7? For example, I don't see driver-pods creating while running spark-pi example, maybe because this is the first k8s touch from my side)
#
# Copyright 2017 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: "sparkoperator.k8s.io/v1beta2"
kind: SparkApplication
metadata:
name: spark-pi
namespace: default
spec:
type: Scala
mode: cluster
image: "spark:3.5.0"
imagePullPolicy: Always
mainClass: org.apache.spark.examples.SparkPi
mainApplicationFile: "local:///opt/spark/examples/jars/spark-examples_2.12-3.5.0.jar"
sparkVersion: "3.5.0"
sparkUIOptions:
serviceLabels:
test-label/v1: 'true'
restartPolicy:
type: Never
volumes:
- name: "test-volume"
hostPath:
path: "/tmp"
type: Directory
driver:
cores: 1
coreLimit: "1200m"
memory: "512m"
labels:
version: 3.5.0
serviceAccount: spark-operator-spark
volumeMounts:
- name: "test-volume"
mountPath: "/tmp"
executor:
cores: 1
instances: 1
memory: "512m"
labels:
version: 3.5.0
volumeMounts:
- name: "test-volume"
mountPath: "/tmp"
from spark-operator.
Forced to use Chart version 1.2.7 to make it work
from spark-operator.
Forced to use Chart version 1.2.7 to make it work
Can you please specify here the helm install command?
I have the similar problem (timeout here, mac m2)
helm install spark-operator/spark-operator --namespace spark-operator --set sparkJobNamespace=default --set webhook.enable=true --generate-name --debug
UPD:
seems
helm install eee spark-operator/spark-operator --namespace spark-operator --set sparkJobNamespace=default --set webhook.enable=true --debug --version 1.2.7
from spark-operator.
Related Issues (20)
- [QUESTION] spark-submit not called, and other questions HOT 6
- [BUG] Failed to pull image "gcr.io/spark-operator/spark:v3.1.1" HOT 5
- [ERROR] Error initializing SparkContext
- [QUESTION] configmap not found HOT 1
- Custom environment variables provided in Kubernetes spark job is not getting picked up HOT 2
- [BUG] spark.eventLog.enable and spark.eventLog.dir not working HOT 2
- [FEATURE] integrate this operator into training-operator
- [FEATURE] Upgrade default image to Spark 3.5.1
- [BUG] Executor state map becoming too large can result in failures to write the CR to etcd
- how works cache or persist with Spark Operator
- [FEATURE] Manage Spark History Server as a deployment via the Spark Operator helm chart. HOT 8
- [BUG] Spark operator not create enough ingressURL when create batch sparkapplication
- [BUG] Unable to assign environment variables HOT 4
- [QUESTION] Error related with webhook HOT 3
- Helm chart references non-existent version `v1beta2-1.4.2-3.5.0` HOT 3
- Add support for livenessProbe and ReadinessProbe of driver and executor pods HOT 4
- [FEATURE] Custom log patterns
- [BUG] Not able to open additional port on executor HOT 1
- [FEATURE] Metrics for pod update, pod delete, pod added
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spark-operator.