huntlabs / grpc-dlang Goto Github PK

View Code? Open in Web Editor NEW

44.0 12.0 6.0 146 KB

Grpc for D programming language, hunt-http library based.

License: Apache License 2.0

D 63.00% CMake 1.26% C++ 35.73%

grpc hunt hunt-grpc dlang rpc

grpc-dlang's People

Contributors

Stargazers

Watchers

Forkers

aravindavk hatf0 carun nocomentsspb mw66 joegana

grpc-dlang's Issues

I think use return value easier to understand.

  import helloworld.helloworld;
  import helloworld.helloworldrpc;
  import grpc;
  import std.stdio;

  auto channel = new Channel("127.0.0.1" , 50051);
  GreeterClient client = new GreeterClient(channel);

  HelloRequest request = new HelloRequest();
  request.name = "test";

  auto reply = client.SayHello(request);
  if(reply !is null)
  {
     writeln(reply.message);
  }

ldc2 build on windows 10 fails

First of all, thank you for creating the grpc binding. I try to build, but get a failure:

$ dub --compiler=ldc2
Fetching hunt-security 0.0.2 (getting selected version)...
Fetching boringssl 0.0.1 (getting selected version)...
Fetching openssl 1.1.6+1.0.1g (getting selected version)...
Fetching hunt 1.0.0-beta.4 (getting selected version)...
Fetching hunt-net 0.0.3 (getting selected version)...
Fetching protobuf 0.3.1 (getting selected version)...
Fetching hunt-http 0.0.4 (getting selected version)...
Performing "debug" build using ldc2 for x86_64.
hunt-grpc ~master: building configuration "hunt-grpc"...
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(106,20): Error: undefined identifier WSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(395,20): Error: undefined identifier WSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(396,20): Error: undefined identifier WSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(523,16): Error: undefined identifier WSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(684,5): Error: undefined identifier LPWSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(686,5): Error: undefined identifier LPWSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(687,5): Error: undefined identifier LPWSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(690,5): Error: undefined identifier LPWSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(692,5): Error: undefined identifier LPWSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(693,5): Error: undefined identifier LPWSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\io\TcpStream.d(138,19): Error: function hunt.io.TcpStream.TcpStream.start does not override any function
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\io\TcpStream.d(191,19): Error: function hunt.io.TcpStream.TcpStream.onRead does not override any function
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\io\TcpStream.d(220,19): Error: function hunt.io.TcpStream.TcpStream.onClose does not override any function
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\io\TcpStream.d(240,19): Error: function hunt.io.TcpStream.TcpStream.onWrite does not override any function
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\io\TcpListener.d(122,19): Error: function hunt.io.TcpListener.TcpListener.start does not override any function ..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\io\TcpListener.d(130,19): Error: function hunt.io.TcpListener.TcpListener.close does not override any function ..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\io\TcpListener.d(139,29): Error: function hunt.io.TcpListener.TcpListener.onRead does not override any function

ldc2 failed with exit code 1.

Please add toplevel README with instructions for use / examples

Please add to top level of project.

Decopoul service class / interface generation from Server/Client implementation

This project is nice, but it does two things at the same time:

Generate classes / interfaces to implement defined service / methods.
Implements transport, channels, server and client, error handling, framing, etc.

I think these two things should be decoupled, so alternative implementations of 2) can be used without changing 1). Using 2) forces ones to use a specific implementation details, i.e. logging, configuration, threading and memory management model, overload protection methods on server, load balancing methods on client, use other hunt dependencies, etc.

Client part not working on Windows 10.

After successfully building the client and the server, it seems the client part is not working very well.

I have this error after the ConnectEx call in IOCP.d (I displayed this line with tracef before the first return in checkErro():

erro=-1, dwLastError=10022

The server part is working, I tried it accessing the ip and port with browser, I can see logs.

Do you know why ? thanks

Compilation error with dmd 2.084.0

Hello, I have this compilation error with latest DMD :

 dub
Performing "debug" build using C:\D\dmd2\windows\bin\dmd.exe for x86.
hunt 1.0.0: target for configuration "library" is up to date.
hunt-security 0.0.6: target for configuration "library" is up to date.
hunt-net 0.0.13: target for configuration "default" is up to date.
hunt-http 0.0.13: target for configuration "default" is up to date.
protobuf 0.3.1: building configuration "protobuf"...
hunt-grpc 0.1.0: building configuration "hunt-grpc"...
..\..\..\AppData\Local\dub\packages\hunt-grpc-0.1.0\hunt-grpc\source\grpc\GrpcClient.d(3,8): Error: module `Promise` is in file 'hunt\util\concurrent\Promise.d' which cannot be read
import path[0] = ..\..\..\AppData\Local\dub\packages\hunt-grpc-0.1.0\hunt-grpc\source
import path[1] = ..\..\..\AppData\Local\dub\packages\hunt-http-0.0.13\hunt-http\source
import path[2] = ..\..\..\AppData\Local\dub\packages\hunt-1.0.0\hunt\source
import path[3] = ..\..\..\AppData\Local\dub\packages\hunt-net-0.0.13\hunt-net\source
import path[4] = ..\..\..\AppData\Local\dub\packages\hunt-security-0.0.6\hunt-security\source
import path[5] = ..\..\..\AppData\Local\dub\packages\protobuf-0.3.1\protobuf\src
import path[6] = C:\D\dmd2\windows\bin\..\..\src\phobos
import path[7] = C:\D\dmd2\windows\bin\..\..\src\druntime\import
C:\D\dmd2\windows\bin\dmd.exe failed with exit code 1.

Seems easy to fix.

EDIT : after fixing that error, there is still a lot of fix to do (hunt.container => hunt.collection, etc...).

Dependencies way out of date

Trying to use this in a project, I get a failure from hunt:

../../../../.dub/packages/hunt-1.1.2/hunt/source/hunt/concurrency/Executors.d(401,12): Error: interface hunt.util.Common.Callable(V) at ../../../../.dub/packages/hunt-1.1.2/hunt/source/hunt/util/Common.d(120,1) conflicts with struct core.thread.context.Callable at /home/jpontaoski/dlang/dmd-2.095.0/linux/bin64/../../src/druntime/import/core/thread/context.d(26,1)

It looks like this is a very old version, as the latest one is 1.6.10; seems that a lot of dependencies are out of date

streaming and bidirectional streaming grpc?

is that supported? if not is there a timeline?

the server does not handle empty request message properly

If we send a request, with empty message: e.g.

https://github.com/huntlabs/grpc-dlang/blob/master/examples/SimpleDemo/source/client.d#L15

comment out this line

  // request.name = "Hunt";

Then the server side will fail this test:

grpc-dlang/source/grpc/GrpcStream.d

Line 141 in 662427b

if(_incomingData.length > 0) {

and be trapped here forever:

grpc-dlang/source/grpc/GrpcStream.d

Line 148 in 662427b

warning("The data is not ready yet.");

and complains:

2022-10-17 02:24:04 | 8291 | warning | onData | The data is not ready yet. | /.dub/packages/grpc-0.5.0-beta.2/grpc/source/grpc/GrpcStream.d:148

and never pass the request to the handler.

On the client side, will see error like this:

2022-10-17 02:43:19 | 11773 | debug | remove | hunt.concurrency.ScheduledThreadPoolExecutor.ScheduledFutureTask!void.ScheduledFutureTask[Cancelled] | /.dub/packages/hunt-extra-1.2.3/hunt-
extra/source/hunt/concurrency/ScheduledThreadPoolExecutor.d:1129

You can also try using Empty message to see this bug on the server side:

google/protobuf/empty.proto
or

message Empty {}

I even tried:

message NumberMsg {
  int64 value = 1;  // return number
}

  // on the client side:
  NumberMsg req = new NumberMsg();
  // but do not assign anything to req.value, the server side show the same bug.

I think we should fix this bug to handle empty message (e.g. empty string) properly.

Thanks.

Server is crashing

Hello,

I'm trying to write a simple grpc server in D, similar to the classic helloworld example.

I'm able to compile and start it just fine, but once I send a client request the server crashes.

The error is:

2020-07-19 18:54:13 (21) [info] onPreface - server received preface: Http2Session@132645144323280{l:127.0.0.1:50051 <-> r:127.0.0.1:33236,sendWindow=65535,recvWindow=65535,streams=0,NOT_CLOSED} - grpc-dlang/source/grpc/GrpcServer.d:64
2020-07-19 18:54:13 (21) [info] onNewStream - server created new stream: 1 - grpc-dlang/source/grpc/GrpcServer.d:69
2020-07-19 18:54:13 (21) [info] onNewStream - server created new stream headers: POST{u=http://127.0.0.1:50051/helloworld.Greeter/SayHello,HTTP/2.0,h=4,cl=-1} - grpc-dlang/source/grpc/GrpcServer.d:70
2020-07-19 18:54:13 (21) [info] onNewStream - server created new stream: 3 - grpc-dlang/source/grpc/GrpcServer.d:69
2020-07-19 18:54:13 (21) [info] onNewStream - server created new stream headers: POST{u=http://127.0.0.1:50051/helloworld.Greeter/SayHello,HTTP/2.0,h=4,cl=-1} - grpc-dlang/source/grpc/GrpcServer.d:70
Segmentation fault

I'm using the latest docker Dlang2 image to build the application and ghz as client.

More details and the step to reproduce the crash are in this ticket: LesnyRumcajs/grpc_bench#30

Thanks!

FYI: dlang-community / grpc-d works better

Just FYI:

There is another grpc-d lib:

https://github.com/dlang-community/grpc-d

and I have tried the server side, it works more reliable than this grpc-dlang library;
also it can talk to both Python and Dart client easily, e.g:

https://github.com/dlang-community/grpc-demo
https://github.com/dlang-community/grpc-d-compiler
https://github.com/dlang-community/grpc-d-interop

(I fixed a few things, so please use the dlang-community fork version of the libs.)

Basically, it's a D wrapper of Google's own libgrpc, not a home-grown library like this one, so we are using the same underlying libs from Google.

D has a relative small community, maybe we should join the effort to maintain one single quality library, instead of re-invent the wheels, unless you are confident you can do better than Google's own libs.

Cheers.

Error on windows


$ dub build
Performing "debug" build using C:\d\dmd2\windows\bin\dmd.exe for x86_64.
grpc 0.3.0-beta.3: building configuration "library"...
source\grpc\GrpcServer.d(24,8): Error: module `core.sys.posix.signal` import `bsd_signal` not found
C:\d\dmd2\windows\bin\dmd.exe failed with exit code 1.

improve doc: README.md

I was following README.md

$ git clone https://github.com/huntlabs/grpc-dlang
$ cd grpc-dlang

# then I was following README.md
$ dub build :protoc-gen-d
Sub package "grpc:protoc-gen-d" doesn't exist.

Finally, I figured out, what it really means is:

$ git clone https://github.com/dcarp/protobuf-d
$ cd protobuf-d

# then following README.md
$ dub build :protoc-gen-d
...

And later in the doc:

from the section "Generating protobuf code"

it should updated with the latest directory change, and just be:

$ cd examples/SimpleDemo/proto/
$ ./generate.sh
$ cd ..
$ ./build.sh

Server running on docker not working.

Hello, server running on windows or ubuntu subsystem is working well.

But when I Dockerize it, its not working anymore :

$ docker-compose up
transaction-service_app_1 is up-to-date
Attaching to transaction-service_app_1
app_1 | Fetching hunt-security 0.1.0 (getting selected version)...
app_1 | Fetching hunt-imf 0.0.7 (getting selected version)...
app_1 | Fetching boringssl 0.0.1 (getting selected version)...
app_1 | Fetching openssl 1.1.6+1.0.1g (getting selected version)...
app_1 | Fetching hunt 1.1.2 (getting selected version)...
app_1 | Fetching hunt-net 0.1.0 (getting selected version)...
app_1 | Fetching hunt-trace 0.1.9 (getting selected version)...
app_1 | Fetching protobuf 0.5.0 (getting selected version)...
app_1 | Fetching grpc 0.2.0 (getting selected version)...
app_1 | Fetching hunt-http 0.1.4 (getting selected version)...
app_1 | Performing "debug" build using dmd for x86_64.
app_1 | hunt 1.1.2: building configuration "library"...
app_1 | hunt-security 0.1.0: building configuration "library"...
app_1 | hunt-net 0.1.0: building configuration "default"...
app_1 | protobuf 0.5.0: building configuration "protobuf"...
app_1 | hunt-imf 0.0.7: building configuration "hunt-imf"...
app_1 | hunt-trace 0.1.9: building configuration "library"...
app_1 | hunt-http 0.1.4: building configuration "default"...
app_1 | grpc 0.2.0: building configuration "grpc"...
app_1 | transaction-service ~master: building configuration "application"...
app_1 | Linking...
app_1 | Running ./transaction-service
app_1 | 2019-06-04 09:31:41 (73) [debug] parseNext - after parseLine =>[pos=16 lim=411 cap=411] - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/decode/HttpParser.d:1287
app_1 | 2019-06-04 09:31:41 (73) [info] onPreface - server received preface: Http2Session@132362403889968{l:172.19.0.2:50051 <-> r:172.19.0.1:47628,sendWindow=65535,recvWindow=65535,streams=0,NOT_CLOSED} - ../../root/.dub/packages/grpc-0.2.0/grpc/source/grpc/GrpcServer.d:46
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - CtxTbl[78620610f300] decoding 255 octets - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:64
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decode 8683400a3a617574686f726974790f3132372e302e302e313a35303035314005... - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:76
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decode 83400a3a617574686f726974790f3132372e302e302e313a353030353140053a... - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:76
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decode 400a3a617574686f726974790f3132372e302e302e313a353030353140053a70... - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:76
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - header, name=:authority, value=127.0.0.1:50051 - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:176
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decoded ':authority: 127.0.0.1:50051(preparsed h=127.0.0.1 p=50051)' by LitName/LitVal/Idx - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:212
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decode 40053a706174681c2f68656c6c6f776f726c642e477265657465722f53617948... - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:76
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - header, name=:path, value=/helloworld.Greeter/SayHello - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:176
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decoded ':path: /helloworld.Greeter/SayHello' by LitName/LitVal/Idx - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:212
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decode 4002746508747261696c657273400c636f6e74656e742d74797065106170706c... - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:76
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - header, name=te, value=trailers - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:176
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decoded 'te: trailers' by LitName/LitVal/Idx - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:212
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decode 400c636f6e74656e742d74797065106170706c69636174696f6e2f6772706340... - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:76
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - header, name=content-type, value=application/grpc - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:176
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decoded 'content-type: application/grpc' by LitName/LitVal/Idx - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:212
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decode 400a757365722d6167656e7439677270632d6373686172702f312e32302e3120... - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:76

The client (started from windows) is waiting for a response indefinitely from server running on docker container. This is the last log I have.

have trouble to use D grpc client to talk to Python grpc server

Hi,

I'm trying the helloworld example, but I'm using a Python server, something like this:

import grpc
import HW_pb2
import HW_pb2_grpc

if __name__ == "__main__":
  handler = HWHandler()
  server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
  HW_pb2_grpc.add_HWServicer_to_server(handler, server)
  server.add_insecure_port('[::]:' + str(PORT))
  server.start()

And then use the D grpc client talk to it, the 1st time I run it: I got:

Thread 11 "" received signal SIGSEGV, Segmentation fault.                
[Switching to Thread 0x7fffcf7fe700 (LWP 2892)]                                                                
0x0000555555aca53c in _D4hunt4http6client18Http1ClientDecoderQu6decodeMFCQBx2io10ByteBufferQmCQCs3net10ConnectionQmZv (warning: (Internal error: pc 0x555
555a69009 in read in psymtab, but not in symtab.)                                  
                                                                                                                                                        
warning: (Internal error: pc 0x555555a68fa0 in read in psymtab, but not in symtab.)
                                                                                   
warning: (Internal error: pc 0x555555a69009 in read in psymtab, but not in symtab.)                                                                      
                                                                                                                                                         
warning: (Internal error: pc 0x555555a69009 in read in psymtab, but not in symtab.)
                                                                                                                                                         
this=0x7fffec8589c0, warning: (Internal error: pc 0x555555a69009 in read in psymtab, but not in symtab.)
                                                                                                                                                         
                                                                                                                                       
buffer=0x7fffeccd3e40, warning: (Internal error: pc 0x555555a69009 in read in psymtab, but not in symtab.)                                               
                                                                                   
session=0x7fffecd36690) at Http1ClientDecoder.d:44                                                                           
44                  throw new IllegalStateException("Client connection is null! The actual type is: "

The 2nd time:

grpc.GrpcException.GrpcTimeoutException@/home//.dub/packages/grpc-0.3.0-beta.4/grpc/source/grpc/GrpcStream.d(225): Timedout after 5 seconds.
----------------                                                                      
??:? [0x555555d64775]
??:? [0x555555d8c9f6]
??:? [0x555555d7056d]
/home//.dub/packages/grpc-0.3.0-beta.4/grpc/source/grpc/GrpcStream.d:225 [0x55555574a6cb]

Since it's a very simple grpc server, before I investigate further, I want to ask: does this D library's grpc client can talk to a Python grpc server?

The Python side grpc code are generated with:

python -m grpc_tools.protoc -I...

with vesion:

grpcio                             1.31.0    
grpcio-tools                       1.31.0    
protobuf                           3.13.0

If you know these 2 can work together, can you provide a Python grpc server example? and with the specific Python package version?

Thanks.

SimpleDemo not working correctly

Python and D examples are not able to properly talk to each other. D example client can talk to the D server, but not Python to D or D to Python. If you run the D server and Python client, the client calls SayHello and never returns but the server receives the message, just no response back. I'm not sure if the server has physically sent a response or not. If you run the Python server and the D client, the server never receives the message and the client never return from the first call to SayHello

Software versions:

xubuntu 20.04
grpc-dlang v0.3.0-beta.5
dmd 2.094.2
dub 1.22.3
python 3.8.5 64-bit
grpcio 1.34.0
grpcio-tools 1.34.0

the server does not send back empty reply message

Hi,

This is related to the previous issue:

#33

If the user set reply message to empty, the server won't send back that message, then the application client side will wait for that empty reply forever.

A request / a reply's body is empty or not is an application logic thing, the grpc library should not check for that and decide if the request need to be handled or reply sent back, the server should always do that.