python / mypy Goto Github PK

Add support for global variables to the C back end.

Currently the implementation does not use packages, because the Python translator does not support them yet (#24). Refactor the implementation to use packages.

The first step could be to translate all modules m into modules mypy.m (e.g. mypy.build). Later on, we could use a deeper hierarchy (e.g. mypy.checker.expressions, mypy.transform.functions).

Also rename mtypes to mypy.types (the name mtypes was used to avoid name clash with the types module).

An assertion such as

assert (c, s)

is always true, even though the programmer probably intended

assert c, s

Give a warning if using the first form (like Python does).

This code

void f(int x, str y='x'): print(1)
void f(str x): print(2)

f(1)
f('x')

is translated into

def f(x, *args):
    def f1(x, y='x'): print(1)
    def f2(x): print(2)

    if len(args) == 0 and isinstance(x, int) and isinstance(args[0], str):   # INVALID
        return f1(x, args[0])
    elif len(args) == 0 and isinstance(x, str):
        return f2(x)
    else:
        raise TypeError("Invalid argument types")

f(1)
f('x')

Note the # INVALID comment: the tuple index is out of range.

For optional arguments, there should 2 or more if cases in the dispatcher, for example:

...
    if len(args) == 0 and isinstance(x, int):
        return f1(x)
    elif len(args) == 1 and isinstance(x, int) and isinstance(args[0], str):
        return f1(x, args[0])
...

Type checking this code

def f(): pass
def g(): pass
def g(a): pass
f.x
g.x

produces these errors

t.py, line 4: function has no member "x"
t.py, line 5: object has no member "x"

Note that the error messages are different, even though both refer to functions (the first message refers to an ordinary function, the second to an overloaded function).

It would be better if the error message were consistent.

Support statically typed __init__ methods in the C back end.

it's very cool! yes, it is!

It would be nice to be able to "self reference" local methods and variables/fields so that you can write the type definition once and refer to it elsewhere in the class definition.

If that's not easily doable, an informative error message would be preferable over a stack trace :-)

class A:
int b
void init(A self, A.b b):
print(b + 2)

Expected output:
b is defined as int, the print statement outputs the sum of b + 2.

Received output:

Traceback (most recent call last):
File "../../mypy-py/mypy.py", line 138, in
main()
File "../../mypy-py/mypy.py", line 52, in main
True)
File "/home/toor/mypydev/mypy-py/build.py", line 99, in build
return manager.process(UnprocessedFile(info, program_text))
File "/home/toor/mypydev/mypy-py/build.py", line 180, in process
next.process()
File "/home/toor/mypydev/mypy-py/build.py", line 516, in process
self.sem_analyzer().visit_file(self.tree, self.tree.path)
File "/home/toor/mypydev/mypy-py/semanal.py", line 170, in visit_file
d.accept(self)
File "/home/toor/mypydev/mypy-py/nodes.py", line 321, in accept
return visitor.visit_type_def(self)
File "/home/toor/mypydev/mypy-py/semanal.py", line 281, in visit_type_def
defn.defs.accept(self)
File "/home/toor/mypydev/mypy-py/nodes.py", line 386, in accept
return visitor.visit_block(self)
File "/home/toor/mypydev/mypy-py/semanal.py", line 351, in visit_block
s.accept(self)
File "/home/toor/mypydev/mypy-py/nodes.py", line 255, in accept
return visitor.visit_func_def(self)
File "/home/toor/mypydev/mypy-py/semanal.py", line 188, in visit_func_def
self.analyse_function(defn)
File "/home/toor/mypydev/mypy-py/semanal.py", line 209, in analyse_function
defn.typ.accept(self)
File "/home/toor/mypydev/mypy-py/nodes.py", line 907, in accept
return visitor.visit_annotation(self)
File "/home/toor/mypydev/mypy-py/semanal.py", line 300, in visit_annotation
ann.typ = self.anal_type(ann.typ)
File "/home/toor/mypydev/mypy-py/semanal.py", line 382, in anal_type
return t.accept(a)
File "/home/toor/mypydev/mypy-py/mtypes.py", line 208, in accept
return visitor.visit_callable(self)
File "/home/toor/mypydev/mypy-py/typeanal.py", line 81, in visit_callable
res = Callable(self.anal_array(t.arg_types),
File "/home/toor/mypydev/mypy-py/typeanal.py", line 98, in anal_array
res.append(t.accept(self))
File "/home/toor/mypydev/mypy-py/mtypes.py", line 38, in accept
return visitor.visit_unbound_type(self)
File "/home/toor/mypydev/mypy-py/typeanal.py", line 20, in visit_unbound_type
sym = self.lookup(t.name, t)
File "/home/toor/mypydev/mypy-py/semanal.py", line 704, in lookup_qualified
n = (n.node).names.get(parts[i], None)
AttributeError: 'TypeInfo' object has no attribute 'names'

Currently interfaces with default method implementations are not supported. For example,
this code is rejected:

interface A:
    def f(self): return 1
class B(A): pass

Also, it should be an error if multiple implemented interfaces have a default implementation
for a method, but the class and its base classes don't defined the method.

Add support for statically typed generic types in the C back end. In particular, add a simple, unoptimized list implementation with a few methods.

Currently there's no documentation about supported Python modules. Users have to manually inspect the stubsdirectory (and that's not documented either).

I don't think it's completely clear what the following example should print:

class A: pass
class B(A): pass

def f(A a): print('A')
def f(B b): print('B')

f(B())

One might intuitively expect the 'best match' to be chosen, which would print 'B', but the current implementation actually chooses the 'first match', which would print 'A'. This isn't wrong per se but I think it's at least worth documenting.

The translator to Python should support packages (nested modules).

The type checker already supports packages.

Consider this program

import m
m.f([])

assuming this module m:

void f(any[] x): print('list')
void f(int y): print('int')

When running the program, Python raises an exception:

File "...m.py", line 7, in f
    if isinstance(x, __builtins__.list):
AttributeError: 'dict' object has no attribute 'list'

Apparently we can't use __builtins__ to refer to the type object list. However, the above function f works if it is in the main file of a program instead of a module (strange).

Relevant discussion (Python 2 specific, though):

http://stackoverflow.com/questions/1184016/why-builtins-is-both-module-and-dict

There are no type check errors reported for the following code:

import re
re()        # Should be an error
int x = re  # Should be an error

The type checker cannot infer the type of the list in this example:

str[] x
x = [None] * 5   # Cannot infer correct type (str[]) for [None]

Instead, the type has to be spelled out explicitly, which is inconvenient:

x = <str> [None] * 5

The type checker should infer the type correctly based on context.

Currently function types are invariant with respect to argument types. This makes it impossible to use some useful and common idioms, for example:

print(list(map(str, [1, 2, 3])))

Function subtyping (and type inference, if needed) should be modified so that the above code works.

However, argument types should perhaps still be invariant in overriding to avoid complications with overloading.

Contrived example:

def print_type(int x):
    print('int')

def print_type(x):
    print('unknown')

translates to

def print_type(x):
    def print_type1(x):
        print('int')

    def print_type2(x):
        print('unknown')

    if isinstance(x, int):
        return print_type1(x)
    elif : # <--- invalid syntax
        return print_type2(x)
    else:
        raise TypeError("Invalid argument types")

(comment added by me)

Add C back end support for simple statically typed classes which contain attributes only. For example, this code should work:

class A:
    int x
a = A()
a.x = 5
print(a.x)

You can use lambdas in code, but the type checker basically ignores them and always gives them the 'any' type.

Code like this is accepted, which is obviously wrong:

int x = lambda y: 'x'

Mypy currently lacks support for function and class decorators.
(Sorry if the example is wrong, still learning mypy syntax)
Ex.
Function decorate(Function f):
f.return foo():
print("hi")
return f()
@decorate
void foo():
print("hi")

A tool to generate module stubs from python files.
From http://www.mypy-lang.org/wiki/StubGeneratorTool

The mypy language overview should mention list and dict literals with explicit types, for example:

names = <str> []  # Equivalent to str[] names = []
counts = <str, int> {}  # Equivalent to dict<str, int> counts = {}

It should not be possible to extend int and float. This code is accepted now, but it should be rejected by the type checker:

class n(int): pass

The code

void f(str na):
    pass

gets translated into

def f( na):
    pass

There should not a space before na.

Note that if na is replaced with n, the extra space is no longer generated.

Invalid Python is generated from this function:

def f(*x, y = 1): pass

The output is like this:

def f(*x =, y 1): pass

If we introduce __init__ with def, the compiler complains if an argument has an explicit type:

class A:
    def __init__(self, int x): pass

t.py: In member "__init__" of class "A":
t.py, line 2: Cannot define return type for "__init__"

This should be valid, since it works for other methods and can be useful with overloading.

More generally, the return type of __init__ should be allowed to be void or any only.

The use of bytes and bytearray in the library stubs is inconsistent. Some modules do not properly support bytes objects (e.g. os) even though they work in Python.

Also some Python functions accept only bytes objects, but others accept bytearrays as well. This needs to be reflected in the library stubs (however, Python documentation is not always explicit about this, which complicates the issue).

The following code is accepted by the parser, even though Python gives a syntax error:

def f(*x, *, y = 1): pass

The code should generate a parse error.

Interfaces such as Iterable and Sized that are defined in the mypy builtins are actually defined in the collections module in Python. Currently references to them are not translated correctly to Python:

class A(Sized): ...

gets translated into

class A(Sized): ...

which is invalid. It should be translated into something like this:

import collections as __collections
class A(__collections.Sized): ...

This affects all references to the interfaces, including these:

• as base type in class or interface definition
• as argument type in overloaded function
• in an isinstance check

The parser parses list<list<int>>() as an expression involving < and >> operators, but it should construct an int[][] instance.

The bug affects other nested generic types as well, not just lists.

Nested list comprehensions are not accepted by the parser. Example:

print([(x, y) for x in range(5) for y in range(3)])

Currently the stubs are only searched automatically from the directory stubs below the directory containing (the translated) mypy.py. It would be useful to also look for stubs in a directory relative to the program being type checked/run.

For example, consider the following hierarchy:

d/foo.py
d/stubs/mylib.py

Now if d/foo.py does import mylib, it should refer to the mylib.py file in the stubs directory under d. stubs is a magic directory name for stubs.

This code

void f(x,  # Foo
       int y):
    pass

gets translated into

def f(x, y):
    pass

Note that the line break, spaces and comment before int y are omitted. The translation should preserve the line break, whitespace and comments in this case and only remove the type int.

The translation works correctly if we remove the type int from the source program.

The following code is not accepted:

str f(str a): pass
t f<t>(t a): pass
f<int>(1) # Error: Invalid type application target type

This code is accepted, even though it should generate an error:

class A: pass
class B: pass
class C(A, B): pass

Add basic support for implementation inheritance in statically typed code to the C back end. The basic support only includes inheritance between non-generic classes.

Python generation preserves explicit type arguments for generic functions/classes. It should erase them.

Support statically typed methods in the C back end.

Code like this should work:

class A:
    void f(self, int x):
        print(x + 1)
a = A()
a.f(4)

Add support for type checking simple chained assignments, for example:

x = y = 1

When the Python back end inserts imports, they are always at the beginning of the generated file. This breaks module docstrings. The imports should be inserted only after the docstring.

Also imports should be inserted after an initial comment such as #!/usr/..., as a #! comment only has any significance as the first line.

Example 1:

"""docstring"""
x = Sized  # imports collections in generated Python code

Generated code:

import collections as __collections
"""docstring"""
x = __collections.Sized  # imports collections in generated Python code

Example 2:

#!/usr/bin/...
x = Sized

(the generated code is similar to example 1)

The following is accepted by the compiler, even though it is not valid in Python:

def f():
    yield 1
    return 1

There should be a compile error if mixing return with an argument and yield within the same function.

Update: only disallow this in Python 2 (see below).

The following code is rejected by the type checker, even though it's valid Python:

import re
print(re.__doc__)   # type check fails; no __doc__ attribute

The docstring attributes of modules, functions and classes should be accessible.

As a workaround, you can cast the module object to any:

import re
print(((any)re).__doc__)   # works

This code is rejected by the type checker, even though return without an argument is valid inside generator in Python:

from typing import Iterator

def f() -> Iterator[int]:
    yield 1
    return # Error

The compiler should accept the above code.

Similar to Python, mypy.py should support running a module as a script with the -m command line argument.

The type inference of the final line in this example fails:

class A(SupportsAbs[float]):
    def __abs__(self) -> float: return 5.5

print(abs(A()))        # Error

The abs type argument should be inferred automatically.

The type checker cannot properly deal with or and and operators (it infers the results to be bool, which is obviously very bad).

Currently there is no documentation for Python standard library modules, other than a list of supported modules in the wiki. Users will have to rely on Python documentation. This is generally fine, but sometimes it's not obvious how some things translate to mypy. Additionally some Python library features are not supported yet in the stubs.

We should generate automatically HTML documentation based on the library stubs. They don't have to duplicate the Python library reference (at least initially; this would be good to have later in the project). There should be at least the following information per module:

• all variables, functions, types and methods defined in the module, including their static types
• link to relevevant information in the Python library reference (version 3.2)
• additional notes about things that are different from Python or that might not be obvious to the reader (these notes could be stored in a separate file and merged automatically when building the HTML files)

Additionally there should be a contents page that lists all supported modules.

For example, the following program should not be accepted:

class A:
    def f(self: Any): pass