haskell-grpc-native / http2-grpc-haskell Goto Github PK

Hi,

I'm trying to use warp-grpc as a control plane server for envoy, which uses nghttp2 for a http2/grpc client.

And I'm getting an error (originating from nghttp2):

invalid frame: Invalid HTTP header field was received on stream 1

Here is response dump from envoy logs:

':status', '200'
'date', 'Sat, 04 Apr 2020 21:39:41 GMT'
'server', 'Warp/3.3.9'
'content-type', 'application/grpc+proto'
'grpc-encoding', 'identity'
'trailer', 'grpc-status'
'trailer', 'grpc-message'

I'm still investigating - but can you suggest what can be wrong with it? Thanks.

commercialhaskell/stack#5210

I'm currently getting a warning that future versions of stack will not support building this dependency without a cabal file

A bug we have with the repository is that I see a "forked from" statement. Plus when I create new MRs, GitHub tries to patch my personal copy.

Does anyone know how to reconfigure this repository on GitHub to appear like if it was original?

This is a port of lucasdicioccio/http2-client-grpc#9 , short enough to be copy-pasted here:

It would be nice to return a way to gracefully kill an ongoing general stream in the Helpers module. We'd like to avoid killing the RPCCall thread and leaving a pending HTTP2 stream (for some RPCs, applications would keep resources open forever).

Operators would be super happy to debug services doing thousands queries per second with some ekg or prometheus counters and latency metrics.

What we should add:

• hooks supporting user-defined IO-actions to count:
  • start of a query
  • end of a query (+duration & gRPC service/method name)
• then it is somewhat trivial to add packages for ekg and for prometheus to provide such IO actions with their own implementations with the following metrics
  • number of processed queries and load
  • number of inflight-queries
  • statistics about durations (average, max)

This is the new home for discussions around lucasdicioccio/http2-client-grpc#16 .

Definitely related to #5 .

Building some packages of this library in GHC 9 leads to compilation issues:

http2-client          > /tmp/stack-49090504ba56a2ab/http2-client-0.10.0.0/src/Network/HTTP2/Client.hs:286:19: error:
http2-client          >     • Couldn't match type: forall a1. StreamStarter a1
http2-client          >                      with: (Http2Stream -> StreamDefinition a)
http2-client          >                            -> ClientIO (Either TooMuchConcurrency a)
http2-client          >       Expected: Http2Client -> StreamStarter a
http2-client          >         Actual: Http2Client -> forall a. StreamStarter a
http2-client          >     • In the expression: _startStream
http2-client          >       In an equation for ‘withHttp2Stream’:
http2-client          >           withHttp2Stream = _startStream
http2-client          >     • Relevant bindings include
http2-client          >         withHttp2Stream :: Http2Client -> StreamStarter a
http2-client          >           (bound at src/Network/HTTP2/Client.hs:286:1)
http2-client          >     |           
http2-client          > 286 | withHttp2Stream = _startStream
http2-client          >     | 

This appears to be because of outdated versions of dependencies that doesn't support GHC 9.

I am looking from some examples on how to use these packages. The README.md states that examples would be provided soon. Can you please point me to a project using these libraries.

Thank you for any help.

Wouldn't it be nice if we had some logo?

This issue is for preparing some communication about the new set of packages + the dedicated organization.

Points to make:

• there is a new organization
• we're focusing on a single code repo
• a new set of packages allow to change serialization
• we will be defining how to name and package stub libraries (i.e., pre-generated from .protos)
• we still need help to contibute library-patches/tooling-to-help-with-codegen and also non-regressions/compatibility-tests/benchmarks/examples/doc

Build with /nix/store/f520j7wfgp9h2mmq6rk36r6vgsb279sf-ghc-8.8.3.
unpacking sources
unpacking source archive /nix/store/qhkylxzwd3g4zq9wwvpps8camycbi6m9-http2-grpc-proto3-wire-0.1.0.0.tar.gz
source root is http2-grpc-proto3-wire-0.1.0.0
setting SOURCE_DATE_EPOCH to timestamp 1572985582 of file http2-grpc-proto3-wire-0.1.0.0/http2-grpc-proto3-wire.cabal
patching sources
compileBuildDriverPhase
setupCompileFlags: -package-db=/private/var/folders/f8/z6c2znxs6xq0ylyhmxqvnjgh0000gp/T/nix-build-http2-grpc-proto3-wire-0.1.0.0.drv-0/setup-package.conf.d -j8 -threaded -rtsopts
[1 of 1] Compiling Main             ( Setup.hs, /private/var/folders/f8/z6c2znxs6xq0ylyhmxqvnjgh0000gp/T/nix-build-http2-grpc-proto3-wire-0.1.0.0.drv-0/Main.o )
Linking Setup ...
configuring
configureFlags: --verbose --prefix=/nix/store/4xm74mw8rfbb3q7x6wi2vxwd8x6kjr04-http2-grpc-proto3-wire-0.1.0.0 --libdir=$prefix/lib/$compiler --libsubdir=$abi/$libname --docdir=/nix/store/qnrdzf9hgpmckfhm4zlghk734ri21s9p-http2-grpc-proto3-wire-0.1.0.0-doc/share/doc/http2-grpc-proto3-wire-0.1.0.0 --with-gcc=clang --package-db=/private/var/folders/f8/z6c2znxs6xq0ylyhmxqvnjgh0000gp/T/nix-build-http2-grpc-proto3-wire-0.1.0.0.drv-0/package.conf.d --ghc-options=-j8 --disable-split-objs --enable-library-profiling --profiling-detail=exported-functions --disable-profiling --enable-shared --disable-coverage --enable-static --disable-executable-dynamic --enable-tests --disable-benchmarks --enable-library-vanilla --disable-library-for-ghci --extra-include-dirs=/nix/store/cfzi980pixrc744yazrrklvbsrr76ql4-libc++-7.1.0/include --extra-lib-dirs=/nix/store/cfzi980pixrc744yazrrklvbsrr76ql4-libc++-7.1.0/lib --extra-include-dirs=/nix/store/8h7d310k65c5fiwb9aq505p535jv8lg4-libc++abi-7.1.0/include --extra-lib-dirs=/nix/store/8h7d310k65c5fiwb9aq505p535jv8lg4-libc++abi-7.1.0/lib --extra-include-dirs=/nix/store/9vxf05fr6jj4kg5j8qpalc6ggg87kfv2-compiler-rt-7.1.0-dev/include --extra-lib-dirs=/nix/store/j7lhhv42lbyxms8dnz8b86yq0i3dwxlb-compiler-rt-7.1.0/lib --extra-lib-dirs=/nix/store/g07i0gc32b46caiimskz846lwbl91wbw-ncurses-6.2/lib --extra-lib-dirs=/nix/store/3jnkm514bs4r4h6lz97jz0z79axnafjc-libffi-3.3/lib --extra-lib-dirs=/nix/store/xn8wzkc12d499xi9rglm9nhfxfyxglpn-gmp-6.2.0/lib --extra-include-dirs=/nix/store/xfz639prvjnrnm6k3wpcir9ahv278nr8-libiconv-osx-10.12.6/include --extra-lib-dirs=/nix/store/xfz639prvjnrnm6k3wpcir9ahv278nr8-libiconv-osx-10.12.6/lib --extra-framework-dirs=/nix/store/ixzl5ryiy599zqqhaqg54w70ccacszya-swift-corefoundation/Library/Frameworks
Using Parsec parser
Configuring http2-grpc-proto3-wire-0.1.0.0...
CallStack (from HasCallStack):
  die', called at libraries/Cabal/Cabal/Distribution/Simple/Configure.hs:1022:20 in Cabal-3.0.1.0:Distribution.Simple.Configure
  configureFinalizedPackage, called at libraries/Cabal/Cabal/Distribution/Simple/Configure.hs:475:12 in Cabal-3.0.1.0:Distribution.Simple.Configure
  configure, called at libraries/Cabal/Cabal/Distribution/Simple.hs:625:20 in Cabal-3.0.1.0:Distribution.Simple
  confHook, called at libraries/Cabal/Cabal/Distribution/Simple/UserHooks.hs:65:5 in Cabal-3.0.1.0:Distribution.Simple.UserHooks
  configureAction, called at libraries/Cabal/Cabal/Distribution/Simple.hs:180:19 in Cabal-3.0.1.0:Distribution.Simple
  defaultMainHelper, called at libraries/Cabal/Cabal/Distribution/Simple.hs:116:27 in Cabal-3.0.1.0:Distribution.Simple
  defaultMain, called at Setup.hs:2:8 in main:Main
Setup: Encountered missing or private dependencies:
proto3-wire >=1 && <1.1

The current available proto3-wire in Nix are https://lazamar.co.uk/nix-versions/?channel=nixpkgs-unstable&package=haskellPackages.proto3-wire

I wonder if it's relatively easy to bump the upper bound in a new quickfix release of this library?

We should provide a few examples (or, even better but harder and in orther repositories) provide real-world APIs for already-existing packages.

Historical repositories to port/take inspiration from:

• https://github.com/lucasdicioccio/http2-client-grpc-example
• https://github.com/lucasdicioccio/warp-grpc-example

The GrpcClientConfig only allows connection by Hostname and PortNumber. It would be nice if it somehow supported Unix sockets too. Additionally, an option to take Network.Socket would be nice too.

This issue is a fork of lucasdicioccio/http2-grpc-types#1

jamesthompson@7cbd56c

Need some time to wrap everything.

The current http2-grpc-proto-lens on Hackage requires proto-lens (>=0.5 && <0.7), which can not work with ghc >= 8.10.4 .

Current source code:

If one tries to add our packages to Stackage, it is welcomed with the following errors:

    bytestring version 0.10.9.0 found
        - http2-client-grpc requires >=0.10.8 && <0.10.9
    http2 version 2.0.3 found
        - http2-client requires >=1.6 && <2
        - http2-client-grpc requires >=1.6 && <1.7
    http2-grpc-types version 0.4.0.0 found
        - warp-grpc requires >=0.5 && <0.6
    lens version 4.18.1 found
        - http2-client-grpc requires >=4.16 && <4.18
    network version 3.1.1.0 found
        - http2-client requires >=2.6 && <3
    proto-lens version 0.6.0.0 found
        - http2-client-grpc requires >=0.5 && <0.6
        - http2-grpc-types requires >=0.5 && <0.6
    tls version 1.5.2 found
        - http2-client-grpc requires >=1.4 && <1.5
    warp version 3.3.3 found
        - warp-grpc requires >=3.2.24 && <3.3

We need to loosen the bounds to allow the programs to compile in newer versions.

This is a trace of the discussion at lucasdicioccio/http2-client-grpc#14 .

Presently if you open up a stream that is a "permanent" streamed reply, there is no way to unsubscribe from the connection. Ordinarily, over http2 this would involve sending a RST_STREAM frame on the with the stream id that is being cancelled. This seems only particularly relevant for streaming responses from the server as both the bidi and client streams have the ability for the client to terminate the stream from a higher level means. Since the current implementation of streamed responses is currently synchronous, we can't include a cancellation action in the return type. However I believe it is possible through a series of masks/unmasks to use asynchronous exceptions to only be delivered inside the scope where the stream id is available, allowing us to use the stream id to create the RST_STREAM frame.

Thoughts @lucasdicioccio?

Typically, gRPC leverages code-generation before compilation. We refer to "stub packages", libraries that are the result of code-generation. We refer to "above-stub packages" libraries that are higher level libraries. For instance, an etcd library generated mechanically from the etcd services definitions is the stub, and a library "above-stub" could help providing higher primitives with typical query/response chains on top of etcd (e.g., to have an async which dies when an etcd lease expires).

We can:

• (a) do nothing and let users do the work of generating stub code from .proto files
• (b) provide some alternative Setup.hs
• (c) pre-generate the stub and upload them to Hackage

I think (a) and (b) are too "low level" for most users that will utimately want above-stub packages and "easy-to-use" APIs (e.g., even hiding away all the gRPC components). Thus, we are sort of forced to provide guidelines on how to avoid name clashes (e.g., the package naming scheme should avoid collisions between proto-lens and proto3-wire serializations). If we spend time providing such guidelines, supporting (c) is very likely extra marginal work.

So I'd say we need to:

• define some naming scheme for stub packages (I can think of service, client, server, serialization, whatmore, what if the services have some naming like etcdv3 and etcdv2?)
• provide some tooling to respect this naming scheme
• provide some repository for sticking the generated stubs in Git
• provide some guidelines for above-stub libraries

I've been checking in on this project for a little over a year, and I'm still unclear on what its ultimate goals are. Would you mind adding a few sentences to the README that indicate what role you want this library to play?

For example, it seems like it could be doing two different things at this point:
(1) An implementation of a subset of gRPC using concepts familiar to the Haskell community, but unfamiliar to the gRPC community, or
(2) An implementation of gRPC using Haskell libraries only (e.g. not wrapping the C implementation, the way Awake Security is doing).

(Or maybe it's something else, like just the library that serializes proto messages over HTTP2.)

In my opinion, clarifying this would make it easier for other developers to help you achieve your goals. As it stands, it's a bit unclear what sorts of bug or code contributions would be most helpful.

Based on the examples I've seen using this library, it looks like the current state is closer to (1). For example, PR #35 has the client directly import newHttp2FrameConnection. Also, in GitHub search I see no code for many of the important high level gRPC concepts, like "channel", "stub", "context", etc. Since HTTP2 is really an implementation detail of gRPC, it seems like this library is a little lower in the stack than other gRPC libraries.

Or, put another way, it seems like the examples mix too many layers of abstraction in the same file (e.g. handlers and HTTP2) which would ordinarily be separated by abstraction. Generally in gRPC I can express the entire server and client without thinking about lower-level concepts like HTTP2 or TCP connections. For example, if you look at the gRPC examples for other languages, you'll typically find that they depend only on gRPC and a few other common libraries.

Hence my question at beginning. Is the goal to do a re-imagining of gRPC in a more Haskell-y way, possibly with the side effect of never achieving full compatibility with the rest of the gRPC ecosystem? Or is it to implement stock gRPC using idiomatic Haskell?

In the first case, (in my opinion) the road is going to be very hard. You'll have to essentially re-design gRPC from scratch, since you'll be running into and solving problems that the existing gRPC spec already has solutions to. In the second case, the road will be moderately hard because the existing solutions were developed in the land of OOP, but arguably easier to parallelize since it will largely be a matter of porting existing concepts to Haskell.

Either way, I think it's a cool project. Thanks!

This feature could be implemented in http2-client as well but for some context: typical usage of gRPC is for microservices within datacenters, where endpoints are replicated. The current design sticks all gRPC calls on a same http2 connection (hence, on a same TCP connection -- and if this connection crashes then the programmers as to restart a client passing all the parameters again). It would be nice to support some sort of connection-managers for balancing all RPCs over multiple http2 sessions (with some programmable control over how to pick a session, how many sessions to open etc.)

Code in https://github.com/haskell-grpc-native/http2-grpc-haskell/blob/master/http2-client-grpc/src/Network/GRPC/Client/Helpers.hs#L94-L108 is overriding the TLS library's default certificate validation when establishing a TLS connection by accepting any certificate, by setting TLS.onServerCertificate = \_ _ _ _ -> return []

Quoting the relevant pieces of documentation here:

onServerCertificate :: OnServerCertificate

Used by the client to validate the server certificate. The default implementation calls validateDefault which validates according to the default hooks and checks provided by Data.X509.Validation. This can be replaced with a custom validation function using different settings.

-> IO [FailedReason] | the return failed reasons (empty list is no failure)

This default behaviour is unacceptable in any real world use case scenario. At the very least, either the function names should indicate an insecure creation of a grpc client; or haddocks and documentation added to warn users about this.

The project README is also lacking any words of warning.

Come up with some way to support Reflexion.

This feature is specified at:

• https://github.com/grpc/grpc/blob/master/doc/server-reflection.md

It would be interesting to create some benchmark tool to compare the number of queries per seconds the servers/clients support. Bonus point if there is some reference in another language.

Such a tool is useful:

• stress the libraries and find edge cases
• help users make decisions about chosing gRPC or not, this library or another one
• pinpoint memory/cpu bottlenecks

Hi devs, I'm new to haskell and not good at how to compose things by looking at the types in general. So i'm sorry if this is a stupid question.

I began with writing a function that creates a client like this (where I assume I can get the host address when I run the warp-server.):

mkClient = GCH.setupGrpcClient $ GCH.grpcClientConfigSimple "HOST" 9000 True

Then, I made a function that makes a call to the server like this:

runClient = do
  client <- mkClient
  GCH.rawUnary (RPC?) client HelloRequest

The main problem is that I don't know how to compose the function with the signature

rawUnary
:: (GRPCInput r i, GRPCOutput r o) 
=> r
-> GrpcClient 
-> i 
-> ClientIO (Either TooMuchConcurrency (RawReply o))

My attempt was trying to figure out what r is, and I eventually find out r must be an instance of IsRPC class. The IsRPC class has a method path which looks like this:

class IsRPC t where
  path :: t -> HeaderValue

I was confused here as how does one create a record that can represent the RPC, and somehow return a HeaderValue? What is a HeaderValue? and how to get it?

I also assume i and o is some generic type that are just the type of the input and output of a grpc call?

Thanks.

Preparing some pic like this one:

It seems that due to the changes in Warp 3.3, warp-grpc is not working correctly. Most of the errors are about trailers:

$ grpcurl -v -plaintext -proto with-persistent.proto -d '{}' localhost:1234 withpersistent.PersistentService.onePerson

Resolved method descriptor:
rpc onePerson ( .google.protobuf.Empty ) returns ( .withpersistent.Person );

Request metadata to send:
(empty)

Response headers received:
content-type: application/grpc+proto
date: Thu, 16 Jan 2020 11:52:38 GMT
server: Warp/3.3.6
trailer: grpc-status
trailer: grpc-message

Response trailers received:
(empty)
Sent 1 request and received 0 responses
ERROR:
  Code: Internal
  Message: server closed the stream without sending trailers

We need to investigate how to solve this, because otherwise warp-grpc cannot work with Stackage nightly.

I am using bidirectional stepped client and sometimes get this error:

InvalidState "stream error"

I tracked it down to this line but it is not clear to me why it is happening.
https://github.com/haskell-grpc-native/http2-grpc-haskell/blob/master/http2-client-grpc/src/Network/GRPC/Client.hs#L229

Could you please point me to the right direction?

Hello,

Thanks for all the great work on this project! I am writing a library that uses this work. The versions on Hackage are out of date and are causing build issues.

After reviewing #62, could new versions be published to Hackage? Specifically, I depend on:

• http2-client-grpc # version in Hackage requires http2 >=1.6 && <2.1
• http2-grpc-proto-lens # depends on an older version of bytestring
• http2-grpc-types # depends on an older version of bytestring
• warp-grpc

Thanks in advance!