Description of issue

grpc.aio.StreamStreamClientInterceptor.intercept_stream_stream is marked as async in grpc-stubs:

https://github.com/shabbyrobe/grpc-stubs/blob/master/grpc-stubs/aio/__init__.pyi#L351

But grpc.aio (as of version 1.54.2) actually fails if you attempt to pass an interceptor with an async implementation.

The same likely applies to the equivalent methods in the other interceptor types.

Minimum Reproducible Example

The following passes strict type checking with mypy, but fails at runtime. Remove async from the intercept_stream_stream function signature to make it work.

import asyncio
from collections.abc import AsyncIterator, Callable
from typing import TYPE_CHECKING, cast, Union, AsyncIterable, Iterable

import grpc.aio
from grpc.aio import ClientCallDetails, StreamStreamCall

import pow_pb2
import pow_pb2_grpc


if TYPE_CHECKING:
    BaseInterceptor = grpc.aio.StreamStreamClientInterceptor[pow_pb2.Message, pow_pb2.Message]
else:
    BaseInterceptor = grpc.aio.StreamStreamClientInterceptor


class MyInterceptor(BaseInterceptor):
    async def intercept_stream_stream(
        self,
        continuation: Callable[
            [ClientCallDetails, pow_pb2.Message],
            StreamStreamCall[pow_pb2.Message, pow_pb2.Message],
        ],
        client_call_details: ClientCallDetails,
        request_iterator: Union[AsyncIterable[pow_pb2.Message], Iterable[pow_pb2.Message]],
    ) -> Union[AsyncIterator[pow_pb2.Message], StreamStreamCall[pow_pb2.Message, pow_pb2.Message]]:
        print(client_call_details.method)
        # https://github.com/shabbyrobe/grpc-stubs/issues/46
        return continuation(client_call_details, request_iterator)  # type: ignore[arg-type]


async def streamer() -> AsyncIterator[pow_pb2.Message]:
    for i in range(10):
        yield pow_pb2.Message(Number=i)


class PowerServicer(pow_pb2_grpc.PowerServicer):
    async def Pow(
        self,
        request_iterator: AsyncIterable[pow_pb2.Message],
        context: grpc.aio.ServicerContext[pow_pb2.Message, pow_pb2.Message],
    ) -> AsyncIterator[pow_pb2.Message]:
        async for i in request_iterator:
            yield pow_pb2.Message(Number=i.Number**2)


async def start_server() -> None:
    server = grpc.aio.server()
    pow_pb2_grpc.add_PowerServicer_to_server(PowerServicer(), server)
    server.add_insecure_port("[::]:50051")
    await server.start()
    await server.wait_for_termination()


async def call_server() -> None:
    channel = grpc.aio.insecure_channel(
        "localhost:50051",
        interceptors=[cast(grpc.aio.ClientInterceptor, MyInterceptor())],
    )
    stub = pow_pb2_grpc.PowerStub(channel)
    if TYPE_CHECKING:
        async_stub = cast(pow_pb2_grpc.PowerAsyncStub, stub)
    else:
        async_stub = stub
    result: pow_pb2.Message
    async for result in async_stub.Pow(streamer()):
        print(result.Number)


async def main() -> None:
    asyncio.create_task(start_server())
    await asyncio.sleep(1)
    await call_server()


if __name__ == "__main__":
    asyncio.run(main())

syntax = "proto3";

service Power {
  rpc Pow(stream Message) returns (stream Message) {};
}

message Message {
  float Number = 1;
}

#!/usr/bin/env bash

set -o errexit -o nounset -o pipefail
python -m venv venv
source ./venv/bin/activate
pip install grpcio==1.54.2 grpcio-tools==1.54.2 mypy==1.3.0 git+https://github.com/shabbyrobe/grpc-stubs.git git+https://github.com/nipunn1313/mypy-protobuf.git

python -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. --mypy_out=. --mypy_grpc_out=. pow.proto
python main.py

Description of issue

Problem:
Status code "OUT_OF_RANGE" is defined in google.rpc but is not present in grpc-stubs.

Solution:
Add + OUT_OF_RANGE = ... to grpc-stubs/__init__.pyi

Minimum Reproducible Example

Mypy fails with no attribute "OUT_OF_RANGE":

@shabbyrobe can you please publish new version on pypi?

I'm unable to use it in our project without #12

I see a new version on pypi: https://pypi.org/project/grpc-stubs/1.53.0.1/#history
But it's not reflected in the commits/branches/tags on this repo.
Is it yours?

Hello 👋. I found this repo and this works really well with mypy. I was wondering if you had any plans of including this in typeshed so that it is automatically included in mypy. I am not sure the exact amount of work needed but it does not sound like a lot. If you do not have the time, I might be able to try to do it as well if you want! Thank you for making this either way!

Hi,
Just curious about a change in the last release (1.24.9)

After this change, I'm finding it hard to set the status of services without a mypy error.

The code straight from the grpc examples https://github.com/grpc/grpc/blob/master/examples/python/xds/server.py:

    # Mark all services as healthy.
    health_pb2_grpc.add_HealthServicer_to_server(health_servicer, server)
    for service in services:
        health_servicer.set(service, health_pb2.HealthCheckResponse.SERVING)

Now causes a mypy error of:

Argument 2 to "set" of "HealthServicer" has incompatible type "ValueType"; expected "ServingStatus"  [arg-type]

I think argument 2 should be a HealthCheckResponse.ServingStatus.ValueType ?
Defined in grpc_health-stubs/v1/health.pyi

This never used to cause a problem with health_pb2 stubs missing, but now they are there I am getting this error.
Or if I am wrong about that arguments type, how do I now set a servicer status without a mypy error?

Description of issue

The object returned from grpc.UnaryStreamMultiCallable and grpc.StreamStreamMultiCallable is a Call, an iterator of response values, and a Future, but the grpc-stubs definition of CallIterator[TResponse] is only a Call and an iterator of response values.

Here's a link to the documentation for this return value: https://grpc.github.io/grpc/python/grpc.html#grpc.UnaryStreamMultiCallable

Returns:

An object that is a Call for the RPC, an iterator of response values, and a Future for the RPC. Drawing response values from the returned Call-iterator may raise RpcError indicating termination of the RPC with non-OK status.

I attached an example main.py that calls a UnaryStreamMultiCallable and uses Future methods done() and exception() to query its status. This works correctly at runtime (the exception is expected).

Exception (expected): <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:1234: Failed to connect to remote host: Connection refused"
	debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:1234: Failed to connect to remote host: Connection refused {created_time:"2023-05-22T19:30:15.558641584-05:00", grpc_status:14}"
>

However, it fails type checking with mypy:

main.py:12: error: "CallIterator[HelloReply]" has no attribute "done"  [attr-defined]
main.py:12: error: "CallIterator[HelloReply]" has no attribute "exception"  [attr-defined]
main.py:13: error: "CallIterator[HelloReply]" has no attribute "exception"  [attr-defined]
Found 3 errors in 1 file (checked 1 source file)

Minimum Reproducible Example

from __future__ import annotations
import grpc
from hellostreamingworld_pb2 import HelloRequest, HelloReply
from hellostreamingworld_pb2_grpc import MultiGreeterStub

if __name__ == "__main__":
    channel = grpc.insecure_channel("localhost:1234")
    stub = MultiGreeterStub(channel)

    stream = stub.sayHello(HelloRequest(name="world", num_greetings="3"))
    stream.initial_metadata()
    if stream.done() and stream.exception() is not None:
        ex = stream.exception()
        print(f"Exception (expected): {ex}")

#!/usr/bin/env bash
set -o errexit -o nounset -o pipefail
python -m venv venv
source ./venv/bin/activate
pip install grpcio grpcio-tools grpc-stubs mypy mypy-protobuf
python -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. --pyi_out=. --mypy_grpc_out=. hellostreamingworld.proto
python main.py
python -m mypy main.py

// Copyright 2015 gRPC authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

syntax = "proto3";

option java_package = "ex.grpc";
option objc_class_prefix = "HSW";

package hellostreamingworld;

// The greeting service definition.
service MultiGreeter {
  // Sends multiple greetings
  rpc sayHello (HelloRequest) returns (stream HelloReply) {}
}

// The request message containing the user's name and how many greetings
// they want.
message HelloRequest {
  string name = 1;
  string num_greetings = 2;
}

// A response message containing a greeting
message HelloReply {
  string message = 1;
}

Requirement already satisfied: grpcio in ./venv/lib/python3.9/site-packages (1.55.0)
Requirement already satisfied: grpcio-tools in ./venv/lib/python3.9/site-packages (1.55.0)
Requirement already satisfied: grpc-stubs in ./venv/lib/python3.9/site-packages (1.53.0.2)
Requirement already satisfied: mypy in ./venv/lib/python3.9/site-packages (1.3.0)
Requirement already satisfied: mypy-protobuf in ./venv/lib/python3.9/site-packages (3.4.0)
Requirement already satisfied: protobuf<5.0dev,>=4.21.6 in ./venv/lib/python3.9/site-packages (from grpcio-tools) (4.23.1)
Requirement already satisfied: setuptools in ./venv/lib/python3.9/site-packages (from grpcio-tools) (58.1.0)
Requirement already satisfied: mypy-extensions>=1.0.0 in ./venv/lib/python3.9/site-packages (from mypy) (1.0.0)
Requirement already satisfied: typing-extensions>=3.10 in ./venv/lib/python3.9/site-packages (from mypy) (4.6.0)
Requirement already satisfied: tomli>=1.1.0 in ./venv/lib/python3.9/site-packages (from mypy) (2.0.1)
Requirement already satisfied: types-protobuf>=3.20.4 in ./venv/lib/python3.9/site-packages (from mypy-protobuf) (4.23.0.1)
WARNING: You are using pip version 22.0.4; however, version 23.1.2 is available.
You should consider upgrading via the '/tmp/unary_stream_call_future/venv/bin/python -m pip install --upgrade pip' command.
Writing mypy to hellostreamingworld_pb2_grpc.pyi
Exception (expected): <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:1234: Failed to connect to remote host: Connection refused"
	debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:1234: Failed to connect to remote host: Connection refused {created_time:"2023-05-22T19:47:20.563451309-05:00", grpc_status:14}"
>
main.py:12: error: "CallIterator[HelloReply]" has no attribute "done"  [attr-defined]
main.py:12: error: "CallIterator[HelloReply]" has no attribute "exception"  [attr-defined]
main.py:13: error: "CallIterator[HelloReply]" has no attribute "exception"  [attr-defined]
Found 3 errors in 1 file (checked 1 source file)

Hey, thanks for making this stub. Has been super useful for me!

I was looking into what type to use for the argument grace of Server.stop() when creating a delegate and to figure it out I took a peek at the gRPC source code for python. The only place that this argument is used seems to be here. It is passed to threading.Event.wait() which according to the docs takes an optional floating point number.

So I believe the correct type for the argument should be typing.Optional[float].

Cheers!

Description of issue

grpc-stubs defines the continuation in interceptors to take a single request value (TRequest) as the second parameter:

https://github.com/shabbyrobe/grpc-stubs/blob/master/grpc-stubs/__init__.pyi#L527
https://github.com/shabbyrobe/grpc-stubs/blob/master/grpc-stubs/__init__.pyi#L537
https://github.com/shabbyrobe/grpc-stubs/blob/master/grpc-stubs/aio/__init__.pyi#L340
https://github.com/shabbyrobe/grpc-stubs/blob/master/grpc-stubs/aio/__init__.pyi#L353

But the examples in the official gRPC repository show that the continuations are expected to take iterators:

https://github.com/grpc/grpc/blob/d63c4709d162c4b8261848a6958e18486752a458/examples/python/interceptors/headers/generic_client_interceptor.py#L50
https://github.com/grpc/grpc/blob/d63c4709d162c4b8261848a6958e18486752a458/examples/python/interceptors/headers/generic_client_interceptor.py#L59

I think the second parameter of the continuation should have the same type as the third parameter of the intercept_stream_* functions, i.e. typing.Iterator[TRequest] for vanilla gRPC and typing.Union[typing.AsyncIterable[TRequest], typing.Iterable[TRequest]] for grpc.aio.

Minimum Reproducible Example

N/A

Description of issue

The type hints for grpc.CallIterator define __iter__ but not __next__. Based on Iterator Types I think this makes grpc.CallIterator an iterable, not an iterator.

The result is that passing a grpc.CallIterator to the next() function works at runtime but causes mypy to emit this error:

main.py:16: error: No overload variant of "next" matches argument type "CallIterator[T]"  [call-overload]
main.py:16: note: Possible overload variants:
main.py:16: note:     def [_T] next(SupportsNext[_T], /) -> _T
main.py:16: note:     def [_T, _VT] next(SupportsNext[_T], _VT, /) -> Union[_T, _VT]

The docs for grpc.UnaryStreamMultiCallable say the returned object is an iterator for response values, which suggests it implements the Iterator protocol, not only the Iterable protocol.

The implementation in the grpc._channel._Rendezvous class supports __next__.

Minimum Reproducible Example

from __future__ import annotations
import grpc
import typing

T = typing.TypeVar("T")

if typing.TYPE_CHECKING:
    # grpc.CallIterator is only defined when type checking                      
    class CallIteratorPlusNext(grpc.CallIterator[T]):
        def __next__(self) -> T:
            raise NotImplementedError()

# This generates mypy errors because grpc.CallIterator does not have a          
# __next__() method.                                                            
def call_next(call_iterator: grpc.CallIterator[T]) -> T:
    return next(call_iterator)

# This does not generate mypy errors because the derived class adds a           
# __next__() method.                                                            
def call_next_plus(call_iterator: CallIteratorPlusNext[T]) -> T:
    return next(call_iterator)

#!/usr/bin/env bash                                                             
set -o errexit -o nounset -o pipefail
python -m venv venv
source ./venv/bin/activate
pip install grpcio grpc-stubs mypy
mypy main.py

Requirement already satisfied: grpcio in ./venv/lib/python3.9/site-packages (1.57.0)
Requirement already satisfied: grpc-stubs in ./venv/lib/python3.9/site-packages (1.53.0.2)
Requirement already satisfied: mypy in ./venv/lib/python3.9/site-packages (1.5.0)
Requirement already satisfied: tomli>=1.1.0 in ./venv/lib/python3.9/site-packages (from mypy) (2.0.1)
Requirement already satisfied: mypy-extensions>=1.0.0 in ./venv/lib/python3.9/site-packages (from mypy) (1.0.0)
Requirement already satisfied: typing-extensions>=4.1.0 in ./venv/lib/python3.9/site-packages (from mypy) (4.7.1)
WARNING: You are using pip version 22.0.4; however, version 23.2.1 is available.
You should consider upgrading via the '/tmp/grpc_itr/venv/bin/python -m pip install --upgrade pip' command.
main.py:16: error: No overload variant of "next" matches argument type "CallIterator[T]"  [call-overload]
main.py:16: note: Possible overload variants:
main.py:16: note:     def [_T] next(SupportsNext[_T], /) -> _T
main.py:16: note:     def [_T, _VT] next(SupportsNext[_T], _VT, /) -> Union[_T, _VT]
Found 1 error in 1 file (checked 1 source file)

Is there any plans to include stubs for grpc.aio? I love this package but it doesn't seem to provide the aio features. I am more than willing to help out if necessary. I also saw a discussion about this in #15, but not sure what's happening here.

Hi! Thanks for this awesome project!

I am TypedDjango team member, we maintain types for, well, django. And we do pretty much the same job.

For example, we test our types like so: tests/. We even created a tool called pytest-mypy-plugins (announcing post) to help us with this task. Maybe it will be also helpful to you as well.

That's how the simplest test looks like:

- case: compose_two_wrong_functions
  main: |
    from returns.functions import compose

    def first(num: int) -> float:
        return float(num)

    def second(num: float) -> str:
        return str(num)

    reveal_type(compose(first, second)(1))  # N: builtins.str*

Ask any questions you have!

P.S. This project was listed in awesome-python-stubs list.

Description of issue

ServerInterceptor.intercept_service returns RpcMethodHandler[TRequest, TResponse] but the continuation returns Optional[RpcMethodHandler[TRequest, TResponse]].

Writing an interceptor that calls return continuation(handler_call_details) like in request_header_validator_interceptor.py causes mypy to emit this error:

error: Incompatible return value type (got "Optional[RpcMethodHandler[grpc.TRequest, grpc.TResponse]]", expected "RpcMethodHandler[grpc.TRequest, grpc.TResponse]")  [return-value]

The docs say that intercept_service may return None.

Minimum Reproducible Example

from __future__ import annotations
import grpc
import typing

class NullServerInterceptor(grpc.ServerInterceptor):
    def intercept_service(
        self,
        continuation: typing.Callable[
            [grpc.HandlerCallDetails],
            typing.Optional[grpc.RpcMethodHandler[grpc.TRequest, grpc.TResponse]]
        ],
        handler_call_details: grpc.HandlerCallDetails,
    ) -> grpc.RpcMethodHandler[grpc.TRequest, grpc.TResponse]:
        return continuation(handler_call_details)

#!/usr/bin/env bash                                                             
set -o errexit -o nounset -o pipefail
python -m venv venv
source ./venv/bin/activate
pip install grpcio grpc-stubs mypy
mypy main.py

Requirement already satisfied: grpcio in ./venv/lib/python3.9/site-packages (1.57.0)
Requirement already satisfied: grpc-stubs in ./venv/lib/python3.9/site-packages (1.53.0.2)
Requirement already satisfied: mypy in ./venv/lib/python3.9/site-packages (1.5.0)
Requirement already satisfied: mypy-extensions>=1.0.0 in ./venv/lib/python3.9/site-packages (from mypy) (1.0.0)
Requirement already satisfied: tomli>=1.1.0 in ./venv/lib/python3.9/site-packages (from mypy) (2.0.1)
Requirement already satisfied: typing-extensions>=4.1.0 in ./venv/lib/python3.9/site-packages (from mypy) (4.7.1)
WARNING: You are using pip version 22.0.4; however, version 23.2.1 is available.
You should consider upgrading via the '/tmp/serverinterceptor/venv/bin/python -m pip install --upgrade pip' command.
main.py:14: error: Incompatible return value type (got "Optional[RpcMethodHandler[grpc.TRequest, grpc.TResponse]]", expected "RpcMethodHandler[grpc.TRequest, grpc.TResponse]")  [return-value]
Found 1 error in 1 file (checked 1 source file)

• https://github.com/grpc/grpc/pull/19299/files
• https://github.com/grpc/grpc/pull/19971/files

Hi,
thanks for this awesome project, it really helps a lot. As the title say, can mypy be an install dependency? In many scenarios, pyi stubs are used for augmenting Pycharm auto-completement instead of type-check jobs, without installing mypy, tens of megabytes of packages will be reduced from the virtual env.

The current distribution package doesn't include the py.typed files in this repo.
This isn't a blocker, as they aren't strictly required in a stub-only package.
However this may be unexpected?

Description of issue

Most of the grpcio package's public classes are abstract base classes. There are over 20 ABCs in grpc/__init__.py alone.

However, the type hints for these classes don't specify a base class of abc.ABC or a metaclass of abc.ABCMeta, so the ABC-specific register class method for registering "virtual subclasses" exists at runtime but not when type checking.

Minimum Reproducible Example

from __future__ import annotations
import grpc
import typing

@grpc.Call.register
class CallProxy:
    def __init__(self, target: grpc.Call) -> None:
        self._target = target

    def __getattr__(self, name: str) -> typing.Any:
        return getattr(self._target, name)

print("main.py: No errors at runtime")

#!/usr/bin/env bash                                                             
set -o errexit -o nounset -o pipefail
python -m venv venv
source ./venv/bin/activate
pip install grpcio grpc-stubs mypy
python main.py
mypy main.py

Requirement already satisfied: grpcio in ./venv/lib/python3.9/site-packages (1.57.0)
Requirement already satisfied: grpc-stubs in ./venv/lib/python3.9/site-packages (1.53.0.2)
Requirement already satisfied: mypy in ./venv/lib/python3.9/site-packages (1.5.0)
Requirement already satisfied: mypy-extensions>=1.0.0 in ./venv/lib/python3.9/site-packages (from mypy) (1.0.0)
Requirement already satisfied: typing-extensions>=4.1.0 in ./venv/lib/python3.9/site-packages (from mypy) (4.7.1)
Requirement already satisfied: tomli>=1.1.0 in ./venv/lib/python3.9/site-packages (from mypy) (2.0.1)
WARNING: You are using pip version 22.0.4; however, version 23.2.1 is available.
You should consider upgrading via the '/tmp/grpc_abc/venv/bin/python -m pip install --upgrade pip' command.
main.py: No errors at runtime
main.py:5: error: "type[Call]" has no attribute "register"  [attr-defined]
Found 1 error in 1 file (checked 1 source file)

Description of issue

If you turn to pep561, which describes how to use partially typed stubs, it says

Type checkers should merge the stub package and runtime package or typeshed directories. This can be thought of as the functional equivalent of copying the stub package into the same directory as the corresponding runtime package or typeshed folder and type checking the combined directory structure. Thus type checkers MUST maintain the normal resolution order of checking *.pyi before *.py files.

This means that the typing file is used to fully type the package, and even if the package is typed, it is not used in type resolution.
Which makes it impossible to use the package when developing with aio.

But everything works fine on the version 1.24.11

Minimum Reproducible Example

python version 3.11
grpc-stubs==1.24.12.1
grpcio-reflection==1.42.0
grpcio-tools==1.42.0
grpcio==1.42.0
protobuf==3.19.6

from grpc.aio import (
    UnaryUnaryClientInterceptor,
    UnaryStreamClientInterceptor,
    StreamUnaryClientInterceptor,
    StreamStreamClientInterceptor,
    AioRpcError,
    ClientCallDetails,
    Channel,
    Server,
    ServicerContext,
)

issue.py:1: error: Module "grpc.aio" has no attribute "UnaryUnaryClientInterceptor"; maybe "ClientInterceptor"?  [attr-defined]
issue.py:1: error: Module "grpc.aio" has no attribute "UnaryStreamClientInterceptor"; maybe "ClientInterceptor"?  [attr-defined]
issue.py:1: error: Module "grpc.aio" has no attribute "StreamUnaryClientInterceptor"; maybe "ClientInterceptor"?  [attr-defined]
issue.py:1: error: Module "grpc.aio" has no attribute "StreamStreamClientInterceptor"  [attr-defined]
issue.py:1: error: Module "grpc.aio" has no attribute "AioRpcError"  [attr-defined]
issue.py:1: error: Module "grpc.aio" has no attribute "ClientCallDetails"  [attr-defined]

Hey, thanks a ton for these stubs :)

I have recently added some client implementation to my server code (it's a gRPC server that queries another gRPC server), meaning that it makes use of the client stub of that other server.

Now mypy started complaining about

error: Call to untyped function "ServiceStub" in typed context  [no-untyped-call]

This error is happening on the line in which I instantiate my client:

channel = grpc.insecure_channel("some-address:8000")
client = ServiceStub(channel)  # This line is triggering the error
response = client.SomeMethod(request)

I had a quick look into this repo to check if I was simply missing something, but I did indeed not find any stubs for the client stubs.

Are they missing? Is there a good way for me to get rid of that error other than ignoring it?

Description of issue

enable_server_reflection currently only allows the sync server, but if you look at the code it allows for the aio.Server too.