Comments (10)
I have a PR pending with a fix for certain uses of case classes in Spark executor run functions. I'm not sure if this covers the problem you're seeing. Can you provide an example of the code and a stack trace? Or did you mean to close this issue as you found a fix?
from spylon-kernel.
@parente
Hello! Thanks so much for asking. Actually I did not find a fix. I closed the issue simply because I did not find a good way to upload my code. They look so ugly. Wait a minute please
from spylon-kernel.
sealed trait List[+A]
case object Nil extends List[Nothing]
case class Cons[+A](head: A, tail: List[A]) extends List[A]
object List {
def sum(ints: List[Int]): Int = ints match {
case Nil => 0
case Cons(x,xs) => x + sum(xs)
}
def product(ds: List[Double]): Double = ds match {
case Nil => 1.0
case Cons(0.0, _) => 0.0
case Cons(x,xs) => x * product(xs)
}
def apply[A](as: A*): List[A] =
if (as.isEmpty) Nil
else Cons(as.head, apply(as.tail: _*))
}
def sum(ints: List[Int]): Int = ints match {
case Nil => 0
case Cons(x,xs) => x + sum(xs)
}
val x = List(1,2,3,4,5) match {
case Cons(x, Cons(2, Cons(4, _))) => x
case Nil => 42
case Cons(x, Cons(y, Cons(3, Cons(4, _)))) => x + y
case Cons(h, t) => h + sum(t)
case _ => 101
}
from spylon-kernel.
Above is all the coding I put into one cell and it worked well with the outputs:
defined trait List
defined object Nil
defined class Cons
defined object List
sum: (ints: List[Int])Int
x: Int = 3
from spylon-kernel.
However when I put exactly the same last part of the code in the next cell, it cannot work
from spylon-kernel.
val x = List(1,2,3,4,5) match {
case Cons(x, Cons(2, Cons(4, _))) => x
case Nil => 42
case Cons(x, Cons(y, Cons(3, Cons(4, _)))) => x + y
case Cons(h, t) => h + sum(t)
case _ => 101
}
Output:
<console>:40: error: reference to List is ambiguous;
it is imported twice in the same scope by
import $line16$read.List
and import INSTANCE.List
val x = List(1,2,3,4,5) match {
^
<console>:43: error: type mismatch;
found : Any
required: String
case Cons(x, Cons(y, Cons(3, Cons(4, _)))) => x + y
^
<console>:44: error: type mismatch;
found : List[Any]
required: List[Int]
case Cons(h, t) => h + sum(t)
^
from spylon-kernel.
Updated the code. Please have a look if possible
from spylon-kernel.
did you redefine List
? Pretty sure that the scala repl is going to get somewhat angry at that since as part of the repl everything in scala.Predef gets pulled into the class that gets built for evaluating a cell.
from spylon-kernel.
@mariusvniekerk Thanks! That should be the problem.
from spylon-kernel.
@mariusvniekerk I tried to run the code in the original Scala REPL but I need not redefine the class every time. Whenever I run the last part of code it can work smoothly. I had expected the same thing in Jupyter Notebook but it seemed to be hard
from spylon-kernel.
Related Issues (20)
- ExecutorClassLoader error in Spylon notebook
- How add additional jar files to SparkContext HOT 5
- [Request] provide example of Spark Yarn cluster conectivity (EMR) HOT 1
- Write NULL File to HDFS
- Graph Frames modules are missing
- Not able of import external packages HOT 6
- Question: How to gracefully stop execution in a cell?
- Does spylon-kernel support Spark 3.0? HOT 2
- spylon-kernel error : compilation: disabled (not enough contiguous free space left) HOT 1
- Unable to install spylon kernel HOT 2
- spylon launcher.packages inside kernel.json args
- s3 filesystem not found
- [BUG]: Spark submit fails: No such file or directory: '/opt/spark/python/pyspark/./bin/spark-submit'
- Cannot get Hive data HOT 1
- Run Scala cell on Jupyter notebook
- Cannot install spylon-kernel on Ubuntu 22 HOT 1
- Failed running `python -m spylon_kernel install`
- Unable to use existing spark server with spylon-kernel
- Using spylon-kernel with java?
- Outdated versioneer.py broken for Python 3.12
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spylon-kernel.