The Any class has a toString method that Java classes can override. Right now, toString is not marked as nullable, because it's synthetic and not coming from Java. However, this can lead to unsoundness. Consider:

// J.java
class C {
  String toString() { return null; }
}

// S.scala
class S {
  val c = new C()
  val s: String = c.toString()
}

The assignment to s won't throw: instead, s will silently be null until it's used at a later point.

We'd like to check that c.toString() returns a non-null value early on.

The idea is to add a phase after typer where every time we see c.toString(), if the static type of c is a Java class, we'll replace it by denullify(c.toString()), where denullify is a compiler-generated method that roughly looks like:

def denullify[T](x: T|Null): T = {
  x match {
    case null => throw new NullPointerException("TODO MESSAGE")
    case _ => x.asInstanceOf[T]
  }
}

When we write var s: String, s is "initialized" to null. Do we want to force the user to declare s as var s: String|Null in these cases?

  1 class Foo {
  2   val s: String = ???
  3   s match {
  4     case x: String => 1
  5     // Test that this isn't marked as unreachable (because pattern matching) type tests
  6     // use isInstanceOf and null.isInstanceOf[T] returns false for all T.
  7     case null => 2
  8   }
  9 }

-- [E122] Syntax Warning: tests/pos/explicit-null-pattern-matching2.scala:4:9 --
4 |    case x: String => 1
  |         ^^^^^^^^^
  |The highlighted type test will always succeed since the scrutinee type (class String) is t

This is inaccurate because case x: String won't match null.

Possibly (probably) an upstream bug.

Liftable.scala has a bunch of liftable instances for primitive types, including String (not shown below):

object Liftable {

  implicit def BooleanIsLiftable: Liftable[Boolean] = new Liftable[Boolean] {
    def toExpr(x: Boolean): Expr[Boolean] = liftedExpr(x)
  }

  implicit def ByteIsLiftable: Liftable[Byte] = new Liftable[Byte] {
    def toExpr(x: Byte): Expr[Byte] = liftedExpr(x)
  }
...

Do we need to provide an instance for nullable strings?

Reference: https://biboudis.github.io/papers/pcp-gpce18.pdf

Consider what, if any, changes are required to CheckRealizable in light of this comment

// check fields only under strict mode for now.
 // Reason: An embedded field could well be nullable, which means it
// should not be part of a path and need not be checked; but we cannot recognize
// this situation until we have a typesystem that tracks nullability.

https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/core/CheckRealizable.scala#L178

If we have Java code

class Foo {
  public static<T> void foo(T[] arr) {}
}

we can no longer call it from Scala with an argument that's an array of a value type

class Bar { // in Scala
  val arr: Array[Int] = ???
  Foo.foo(arr) // error: expected Array[Int|JavaNull]|JavaNull, but got Array[Int]
}

Instead, we need to make arr nullable: val arr: Array[Int|Null] = ???, which is bad for performance. Pre explicit nulls, this used to work because the array was automatically copied to an Array[Object] at the callsite.

See 59671f7#diff-459714fae8d4a9d0faf4dee5e40a5b2dR1 for the test that motivated this.

As per our last meeting, we need to support the following kinds of idioms:
(easier)

// s: String|Null
if (s != null) {
  // s: String
}

and (harder)

if (s != null && s.length > 0 /*s: String in second test. Need to use the fact that && is short-circuit*/) {
}

The following should typecheck, but currently doesn't:

 class Foo {
   val x: String|Null = "hello"
   val res: String = x match {
     case null      => "it's null"
     case s          => s
  } 
} 

The current workaround is to annotate s:

 class Foo {
   val x: String|Null = "hello"
   val res: String = x match {
     case null => "it's null"
     case s: String => s
  } 
} 

This doesn't typecheck

object Test {

  def main(args: Array[String]): Unit = { 
    val f = new Foo 
    println(1)
    println(f.foo)
    println(2)
    println(f.foo)

    // TODO: Erase
    // Currently not erasing fields for lazy vals
    assert(f.getClass.getDeclaredFields.exists(_.getName.startsWith("foo")), "Field foo erased. Optimized accidentally?")
  }


}

class Foo {
  lazy val foo: Null = { 
    println("foo")
    null
  }
}

because

-- [E008] Member Not Found Error: tests/run/null-lazy-val.scala:13:49 ----------
13 |    assert(f.getClass.getDeclaredFields.exists(_.getName.startsWith("foo")), "Field foo erased. Optimized accidentally?")
   |                                               ^^^^^^^^^
   |       value `getName` is not a member of java.lang.reflect.Field | Null

The argument type for the lambda is inferred to be Field|Null, when it should be Field|JavaNull.

After frontend

        assert(
          refArrayOps[(java.lang.reflect.Field | Null)](
            f.getClass().getDeclaredFields().asInstanceOf[
              Array[java.lang.reflect.Field | JavaNull]
            ]
          ).exists(
            {
              def $anonfun(_$1: java.lang.reflect.Field | Null): Boolean = 
                _$1.getName.startsWith()
              closure($anonfun)
            }
          )

So the implicit conversion is applied, but the argument type is incorrectly inferred. This is because annotations are dropped when subtyping constraints are added to the set of current constraints.

The current workaround is to say .nn.getName.startsWith. I discussed this with Ondrej and we're gonna let it be for now.

Opening this to track the status.

Related to #9

trait Demo {
  def m: String
}

val d: Demo = Mockito.mock(classOf[Demo])
val str = d.m  // method m is unmocked, so Mockito returns null for it
println(str.length)  // compile error or runtime null pointer exception?

What would happen if a Scala trait defines a method that returns a non-nullable type, then someone provides an implementation in another language (like Java) that returns null?

Scala:

trait Demo {
  def m: String
}

Java:

public class JavaImpl {
  public static Demo demo() {
    return new Demo() {
      @Override String m() { return null; }
    }
  }
}

Scala again:

val d: Demo = JavaImpl.demo
val str = d.m
println(str.length)  // compile error or runtime null pointer exception?

The pattern matcher can't distinguish between

??? match {
  case a: Array[String|Null] => ???
  case a: Array[String] => ???
}

Should we warn the user when they try to do this?

Recognize the @nonnull annotation (in its multiple forms) in the Java interop layer, so we can do a more precise translation of types coming from Java.

Right now

val l: java.util.ArrayList<String> = new java.util.ArrayList<String>()
val e = l.get(0)

infers Object as the type of e. This probably has to do with scala#4867

A Java functional interface ("SAM type" in dotty parlance) is an interface with a single abstract method. We can then pass lambdas in contexts where a functional interface is required (see https://docs.oracle.com/javase/8/docs/api/java/util/function/package-summary.html).

For the nullability project, we want the following to continue to work:

 // Assignment context
val p: Predicate[String]|Null = (x) => true

// Method invocation context
val s: Stream[String] = ???
s.filter((x) => true) // filter takes a Predicate[String]|Null

// Method overloading context
trait MyFun {
  def apply(x: Int): String
} 
def foo(m: MyFun|Null) = {}
def foo(m: Int) = {}
foo((x) => "hello")

Might need to wait until Dotty drops support of scalac.

Smaller repro of the failure:

 class Foo extends scala.util.control.NoStackTrace {
 }

The problem is that https://docs.oracle.com/javase/7/docs/api/java/lang/Throwable.html#fillInStackTrace() returns a Throwable|JavaNull, but since the standard library is non-boostrapped, there's no type error in NoStackTrace, where there should be one:

https://github.com/scala/scala/blob/v2.12.6/src/library/scala/util/control/NoStackTrace.scala#L24

As a result, when we Ycheck resolveSuper, there's a type error dotc -Ycheck:resolveSuper -Xprint:resolveSuper kk.scala

Exception in thread "main" java.lang.AssertionError: assertion failed: found:    Throwable | JavaNull
required: Throwable
tree = super[Throwable].fillInStackTrace()
java.lang.AssertionError: assertion failed: found:    Throwable | JavaNull
required: Throwable
tree = super[Throwable].fillInStackTrace() while compiling kk.scala
	at scala.Predef$.assert(Predef.scala:219)
	at dotty.tools.dotc.transform.TreeChecker$Checker.adapt(TreeChecker.sca

@liufengyun writes

Currently, there are 3 well-known NonNull annotations in the Java world:

• JSR 305 : javax.annotation.Nonnull
• Android: @nonnull
• CheckerFramework: @nonnull

A quick github search suggests that Android @nonnull seems to be widely adopted among Android developers.

Kotlin currently supports all known annotations (source):

JetBrains (@Nullable and @NotNull from the org.jetbrains.annotations package)
Android (com.android.annotations and android.support.annotations)
JSR-305 (javax.annotation)
CheckerFramework
FindBugs (edu.umd.cs.findbugs.annotations)
Eclipse (org.eclipse.jdt.annotation)
Lombok (lombok.NonNull).

If the Java source code has no @nonnull annotations, e.g. JDK, can we do better?

Yes. CheckerFramework recently published standard format about
external annotation database (.jaif), on August 30th, 2018:

1. https://checkerframework.org/annotation-file-utilities/
2. https://checkerframework.org/annotation-file-utilities/annotation-file-format.html
3. https://github.com/typetools/annotated-libraries

A natural step is to (1) use CheckerFramework inference tools automatically generate annotation databases for Java libraries; (2) grow the databases via crowdsourcing (a more friendly DB interface required). This can be useful for JDK and widely used libraries like Guava, as the APIs are very stable.

Java enums get a bunch of synthesized methods: https://docs.oracle.com/javase/7/docs/api/java/lang/Enum.html

Some of them (e.g. valueOf and values) never return null. Adjust JavaNullMap to account for that.

This is trying to solve the following issue:

1. library A is compiled with dotty pre null change
2. program P uses library A via tasty (for typechecking) and is compiled with dotty post null change
3. library A defines

def foo(): String = ???

4. P should interpret foo as having a nullable return type

We'd like to re-use the Java null transform when loading A from tasty: https://github.com/abeln/dotty/blob/explicit-null/compiler/src/dotty/tools/dotc/core/JavaNull.scala

cc @olhotak

We should be able to extend the Java class

abstract class J {
  String foo(String x)
}

with the Scala class

class S extends J {
  override def foo(x: String): String = ???
}

Notice that the method in J has type String|Null => String|Null and the one in S is String => String. But we're considering allowing the (unsound) override to go through for easier interop.